188 commits

Author	SHA1	Message	Date
Samuel Clay	59b32e6248	Further fixing feed fetching errors by retrying feeds [nearly] immediately when they can be corrected by visiting the feed's site and finding the correct RSS feed.	2010-10-03 22:38:53 -04:00
Samuel Clay	92cff79403	Finally fixed the damn feed error updating issues. Due to the way Django caches ORM models, at the very end of all the feed error checking, the old values would be saved.	2010-10-03 19:05:16 -04:00
Samuel Clay	39d9e13449	Forcing refresh of feed information at every stage of feed fetch pipeline. This really needs a rewrite. Someday.	2010-10-03 18:04:40 -04:00
Samuel Clay	808df31f7f	Forcing new feeds that have issues to save their issues. Some feeds were perpetually unfetched.	2010-10-03 17:22:58 -04:00
Samuel Clay	e849957a8f	Changing default dates for feed models.	2010-10-02 17:16:01 -04:00
Samuel Clay	a75417d917	Fixing errors on existing story search in feed fetching. Turns out it was using the wrong story_date to quickly determine if the story is new or not.	2010-09-28 21:10:10 -04:00
Samuel Clay	7ea19b9927	Stopping duplicate feeds from trying to save.	2010-09-28 05:29:40 -04:00
Samuel Clay	c24a7ffc5f	Handling null titles.	2010-09-28 04:57:24 -04:00
Samuel Clay	7f1638e679	Adding index on feed histories, and cutting down how many errors/successes stored.	2010-09-27 19:12:17 -04:00
Samuel Clay	7dee2103a9	Refactoring timelimit to add feed address re-check to a limited timeout.	2010-09-27 11:44:23 -04:00
Samuel Clay	7e5dcb188a	Adding munin graphs for feed loadtimes: averages, min, and max for the past hour and day. Going to be beautiful.	2010-09-23 10:29:18 -04:00
Samuel Clay	b9bc00a024	Fixing last update dates so they are always accurate, even on feed loads.	2010-09-22 19:59:07 -04:00
Samuel Clay	6e17db8394	Cutoffs for trimming in feeds.	2010-09-22 16:22:52 -04:00
Samuel Clay	bc7accc593	Trimming feeds on every update.	2010-09-22 15:57:55 -04:00
Samuel Clay	e1c031e2fe	Trimming only stories older than the 500th oldest story.	2010-09-22 15:36:20 -04:00
Samuel Clay	e0f84d7aaa	Trimming only stories older than the 500th oldest story.	2010-09-22 15:34:47 -04:00
Samuel Clay	0ac758f230	Trimming more than 500 stories from feeds. One day these can remain, but for now, it's too much to handle.	2010-09-22 14:39:18 -04:00
Samuel Clay	e647edf025	Trimming more than 500 stories from feeds. One day these can remain, but for now, it's too much to handle.	2010-09-22 14:35:26 -04:00
Samuel Clay	ce28fa14a1	Turning off statistics again. It keeps locking the database very inefficiently.	2010-09-22 11:07:19 -04:00
Samuel Clay	b1096503a2	Turning on feed statistics. Don't make me regret this, Mongo.	2010-09-21 19:20:03 -04:00
Samuel Clay	2bd9f72acb	Fetching feedids with folder delete.	2010-09-21 11:16:22 -04:00
Samuel Clay	19e5ebe53b	Increasing verbosity of feed fetching when forced.	2010-09-19 11:36:04 -04:00
Samuel Clay	ca054598b2	Handling broken dates in feeds.	2010-09-17 13:33:11 -04:00
Samuel Clay	b61955c7f3	Fixing etag max_length, whcih was too short for certain websites and their giant-ass etags.	2010-09-17 13:06:28 -04:00
Samuel Clay	9792550dcc	Counting only active subscribers in prep for turning off feed fetching for inactive users.	2010-09-17 12:42:44 -04:00
Samuel Clay	fb3b1fa260	Enforcing constraints on read stories, and fixing operational errors on merge_feeds.	2010-09-10 08:23:04 -07:00
Samuel Clay	29605c7c9e	Story_guids can be unicode IDs.	2010-09-10 01:27:41 -07:00
Samuel Clay	4c268384be	Story_guids can be unicode IDs.	2010-09-10 01:24:12 -07:00
Samuel Clay	3c9714c4b3	Merging feeds, handling null story_guids	2010-09-10 01:19:46 -07:00
Samuel Clay	613ace35bb	Merging feeds, handling null story_guids	2010-09-10 01:18:31 -07:00
Samuel Clay	20a2420542	Merging feeds, handling null story_guids	2010-09-10 01:11:08 -07:00
Samuel Clay	5e780b0438	Merging feeds, handling null story_guids	2010-09-10 01:09:28 -07:00
Samuel Clay	b26dfa87bb	Handling feed fetch unique errors.	2010-09-10 01:01:18 -07:00
Samuel Clay	0d5b964b38	Small code cleanups before relaunching story_guids unique.	2010-09-10 00:51:20 -07:00
Samuel Clay	974d5b4a58	Deleting stories with invalid id's.	2010-09-09 07:00:08 -07:00
Samuel Clay	de3f86f9c9	Half-way to fixing the massive story dupe bug. Removing unqiueness so i can migrate the db.	2010-09-08 18:48:42 -07:00
Samuel Clay	15a27e840f	Bootstrapping mongo to use story_guids as unqiue to feeds.	2010-09-08 18:30:46 -07:00
Samuel Clay	ce55dd19a8	Handling weird exception on bad models in merging feeds.	2010-09-08 17:01:17 -07:00
Samuel Clay	132c561be5	Queue date for feeds for better metrics.	2010-09-08 13:58:05 -07:00
Samuel Clay	2b9ec596f9	Adding indexes to dates on models.	2010-09-08 10:52:04 -07:00
Samuel Clay	f880739fc8	Setting next scheduled date on manual update.	2010-09-07 15:42:22 -07:00
Samuel Clay	b5da6eace1	Refactoring feed fetching logging to get to the bottom of the fetch problems.	2010-09-07 14:41:11 -07:00
Samuel Clay	a69d812629	Removing slow punishment for popular feeds.	2010-09-07 14:02:48 -07:00
Samuel Clay	c3921b859e	Last_update is now last fetch, not last modification. More accurate last updated dates in Reader.	2010-09-07 13:53:37 -07:00
Samuel Clay	5e1fa36a33	Removing Tag relationships.	2010-09-01 18:19:15 -04:00
Samuel Clay	976e6c2c86	Merge branch 'master' of github.com:samuelclay/NewsBlur	2010-09-01 10:38:00 -04:00
Samuel Clay	c755c9f130	Checking for story_content before comparing origin.	2010-09-01 10:37:33 -04:00
Samuel Clay	c42cabcae2	Celery options for performance.	2010-09-01 08:19:58 -04:00
Samuel Clay	10b1455ba3	Merge branch 'master' of github.com:samuelclay/NewsBlur	2010-08-31 16:34:51 -04:00
Samuel Clay	e2f419cb13	Fixing invalid string error on feed error histories.	2010-08-31 16:34:34 -04:00