186 commits

Author	SHA1	Message	Date
Samuel Clay	39d9e13449	Forcing refresh of feed information at every stage of feed fetch pipeline. This really needs a rewrite. Someday.	2010-10-03 18:04:40 -04:00
Samuel Clay	808df31f7f	Forcing new feeds that have issues to save their issues. Some feeds were perpetually unfetched.	2010-10-03 17:22:58 -04:00
Samuel Clay	e849957a8f	Changing default dates for feed models.	2010-10-02 17:16:01 -04:00
Samuel Clay	a75417d917	Fixing errors on existing story search in feed fetching. Turns out it was using the wrong story_date to quickly determine if the story is new or not.	2010-09-28 21:10:10 -04:00
Samuel Clay	7ea19b9927	Stopping duplicate feeds from trying to save.	2010-09-28 05:29:40 -04:00
Samuel Clay	c24a7ffc5f	Handling null titles.	2010-09-28 04:57:24 -04:00
Samuel Clay	7f1638e679	Adding index on feed histories, and cutting down how many errors/successes stored.	2010-09-27 19:12:17 -04:00
Samuel Clay	7dee2103a9	Refactoring timelimit to add feed address re-check to a limited timeout.	2010-09-27 11:44:23 -04:00
Samuel Clay	7e5dcb188a	Adding munin graphs for feed loadtimes: averages, min, and max for the past hour and day. Going to be beautiful.	2010-09-23 10:29:18 -04:00
Samuel Clay	b9bc00a024	Fixing last update dates so they are always accurate, even on feed loads.	2010-09-22 19:59:07 -04:00
Samuel Clay	6e17db8394	Cutoffs for trimming in feeds.	2010-09-22 16:22:52 -04:00
Samuel Clay	bc7accc593	Trimming feeds on every update.	2010-09-22 15:57:55 -04:00
Samuel Clay	e1c031e2fe	Trimming only stories older than the 500th oldest story.	2010-09-22 15:36:20 -04:00
Samuel Clay	e0f84d7aaa	Trimming only stories older than the 500th oldest story.	2010-09-22 15:34:47 -04:00
Samuel Clay	0ac758f230	Trimming more than 500 stories from feeds. One day these can remain, but for now, it's too much to handle.	2010-09-22 14:39:18 -04:00
Samuel Clay	e647edf025	Trimming more than 500 stories from feeds. One day these can remain, but for now, it's too much to handle.	2010-09-22 14:35:26 -04:00
Samuel Clay	ce28fa14a1	Turning off statistics again. It keeps locking the database very inefficiently.	2010-09-22 11:07:19 -04:00
Samuel Clay	b1096503a2	Turning on feed statistics. Don't make me regret this, Mongo.	2010-09-21 19:20:03 -04:00
Samuel Clay	2bd9f72acb	Fetching feedids with folder delete.	2010-09-21 11:16:22 -04:00
Samuel Clay	19e5ebe53b	Increasing verbosity of feed fetching when forced.	2010-09-19 11:36:04 -04:00
Samuel Clay	ca054598b2	Handling broken dates in feeds.	2010-09-17 13:33:11 -04:00
Samuel Clay	b61955c7f3	Fixing etag max_length, whcih was too short for certain websites and their giant-ass etags.	2010-09-17 13:06:28 -04:00
Samuel Clay	9792550dcc	Counting only active subscribers in prep for turning off feed fetching for inactive users.	2010-09-17 12:42:44 -04:00
Samuel Clay	fb3b1fa260	Enforcing constraints on read stories, and fixing operational errors on merge_feeds.	2010-09-10 08:23:04 -07:00
Samuel Clay	29605c7c9e	Story_guids can be unicode IDs.	2010-09-10 01:27:41 -07:00
Samuel Clay	4c268384be	Story_guids can be unicode IDs.	2010-09-10 01:24:12 -07:00
Samuel Clay	3c9714c4b3	Merging feeds, handling null story_guids	2010-09-10 01:19:46 -07:00
Samuel Clay	613ace35bb	Merging feeds, handling null story_guids	2010-09-10 01:18:31 -07:00
Samuel Clay	20a2420542	Merging feeds, handling null story_guids	2010-09-10 01:11:08 -07:00
Samuel Clay	5e780b0438	Merging feeds, handling null story_guids	2010-09-10 01:09:28 -07:00
Samuel Clay	b26dfa87bb	Handling feed fetch unique errors.	2010-09-10 01:01:18 -07:00
Samuel Clay	0d5b964b38	Small code cleanups before relaunching story_guids unique.	2010-09-10 00:51:20 -07:00
Samuel Clay	974d5b4a58	Deleting stories with invalid id's.	2010-09-09 07:00:08 -07:00
Samuel Clay	de3f86f9c9	Half-way to fixing the massive story dupe bug. Removing unqiueness so i can migrate the db.	2010-09-08 18:48:42 -07:00
Samuel Clay	15a27e840f	Bootstrapping mongo to use story_guids as unqiue to feeds.	2010-09-08 18:30:46 -07:00
Samuel Clay	ce55dd19a8	Handling weird exception on bad models in merging feeds.	2010-09-08 17:01:17 -07:00
Samuel Clay	132c561be5	Queue date for feeds for better metrics.	2010-09-08 13:58:05 -07:00
Samuel Clay	2b9ec596f9	Adding indexes to dates on models.	2010-09-08 10:52:04 -07:00
Samuel Clay	f880739fc8	Setting next scheduled date on manual update.	2010-09-07 15:42:22 -07:00
Samuel Clay	b5da6eace1	Refactoring feed fetching logging to get to the bottom of the fetch problems.	2010-09-07 14:41:11 -07:00
Samuel Clay	a69d812629	Removing slow punishment for popular feeds.	2010-09-07 14:02:48 -07:00
Samuel Clay	c3921b859e	Last_update is now last fetch, not last modification. More accurate last updated dates in Reader.	2010-09-07 13:53:37 -07:00
Samuel Clay	5e1fa36a33	Removing Tag relationships.	2010-09-01 18:19:15 -04:00
Samuel Clay	976e6c2c86	Merge branch 'master' of github.com:samuelclay/NewsBlur	2010-09-01 10:38:00 -04:00
Samuel Clay	c755c9f130	Checking for story_content before comparing origin.	2010-09-01 10:37:33 -04:00
Samuel Clay	c42cabcae2	Celery options for performance.	2010-09-01 08:19:58 -04:00
Samuel Clay	10b1455ba3	Merge branch 'master' of github.com:samuelclay/NewsBlur	2010-08-31 16:34:51 -04:00
Samuel Clay	e2f419cb13	Fixing invalid string error on feed error histories.	2010-08-31 16:34:34 -04:00
Samuel Clay	1ac8402807	Adding indexes to feed/page fetch histories.	2010-08-31 08:45:35 -04:00
Samuel Clay	25c0450020	Commenting out map/reduce statistics because mongo keeps crashing.	2010-08-30 13:33:29 -04:00