229 commits

Author	SHA1	Message	Date
Samuel Clay	e4c00a77a8	Feeds get auto-selected in feed chooser by priority. Unfortunately, it prioritizes incorrectly right now by using the minimum cutoff in a non-stable sort.	2010-09-24 18:22:12 -04:00
Samuel Clay	7e5dcb188a	Adding munin graphs for feed loadtimes: averages, min, and max for the past hour and day. Going to be beautiful.	2010-09-23 10:29:18 -04:00
Samuel Clay	b9bc00a024	Fixing last update dates so they are always accurate, even on feed loads.	2010-09-22 19:59:07 -04:00
Samuel Clay	6e17db8394	Cutoffs for trimming in feeds.	2010-09-22 16:22:52 -04:00
Samuel Clay	bc7accc593	Trimming feeds on every update.	2010-09-22 15:57:55 -04:00
Samuel Clay	e1c031e2fe	Trimming only stories older than the 500th oldest story.	2010-09-22 15:36:20 -04:00
Samuel Clay	e0f84d7aaa	Trimming only stories older than the 500th oldest story.	2010-09-22 15:34:47 -04:00
Samuel Clay	0ac758f230	Trimming more than 500 stories from feeds. One day these can remain, but for now, it's too much to handle.	2010-09-22 14:39:18 -04:00
Samuel Clay	e647edf025	Trimming more than 500 stories from feeds. One day these can remain, but for now, it's too much to handle.	2010-09-22 14:35:26 -04:00
Samuel Clay	ce28fa14a1	Turning off statistics again. It keeps locking the database very inefficiently.	2010-09-22 11:07:19 -04:00
Samuel Clay	b1096503a2	Turning on feed statistics. Don't make me regret this, Mongo.	2010-09-21 19:20:03 -04:00
Samuel Clay	2bd9f72acb	Fetching feedids with folder delete.	2010-09-21 11:16:22 -04:00
Samuel Clay	19e5ebe53b	Increasing verbosity of feed fetching when forced.	2010-09-19 11:36:04 -04:00
Samuel Clay	ca054598b2	Handling broken dates in feeds.	2010-09-17 13:33:11 -04:00
Samuel Clay	b61955c7f3	Fixing etag max_length, whcih was too short for certain websites and their giant-ass etags.	2010-09-17 13:06:28 -04:00
Samuel Clay	9792550dcc	Counting only active subscribers in prep for turning off feed fetching for inactive users.	2010-09-17 12:42:44 -04:00
Samuel Clay	fb3b1fa260	Enforcing constraints on read stories, and fixing operational errors on merge_feeds.	2010-09-10 08:23:04 -07:00
Samuel Clay	29605c7c9e	Story_guids can be unicode IDs.	2010-09-10 01:27:41 -07:00
Samuel Clay	4c268384be	Story_guids can be unicode IDs.	2010-09-10 01:24:12 -07:00
Samuel Clay	3c9714c4b3	Merging feeds, handling null story_guids	2010-09-10 01:19:46 -07:00
Samuel Clay	613ace35bb	Merging feeds, handling null story_guids	2010-09-10 01:18:31 -07:00
Samuel Clay	20a2420542	Merging feeds, handling null story_guids	2010-09-10 01:11:08 -07:00
Samuel Clay	5e780b0438	Merging feeds, handling null story_guids	2010-09-10 01:09:28 -07:00
Samuel Clay	b26dfa87bb	Handling feed fetch unique errors.	2010-09-10 01:01:18 -07:00
Samuel Clay	0d5b964b38	Small code cleanups before relaunching story_guids unique.	2010-09-10 00:51:20 -07:00
Samuel Clay	974d5b4a58	Deleting stories with invalid id's.	2010-09-09 07:00:08 -07:00
Samuel Clay	de3f86f9c9	Half-way to fixing the massive story dupe bug. Removing unqiueness so i can migrate the db.	2010-09-08 18:48:42 -07:00
Samuel Clay	15a27e840f	Bootstrapping mongo to use story_guids as unqiue to feeds.	2010-09-08 18:30:46 -07:00
Samuel Clay	ce55dd19a8	Handling weird exception on bad models in merging feeds.	2010-09-08 17:01:17 -07:00
Samuel Clay	132c561be5	Queue date for feeds for better metrics.	2010-09-08 13:58:05 -07:00
Samuel Clay	2b9ec596f9	Adding indexes to dates on models.	2010-09-08 10:52:04 -07:00
Samuel Clay	f880739fc8	Setting next scheduled date on manual update.	2010-09-07 15:42:22 -07:00
Samuel Clay	b5da6eace1	Refactoring feed fetching logging to get to the bottom of the fetch problems.	2010-09-07 14:41:11 -07:00
Samuel Clay	a69d812629	Removing slow punishment for popular feeds.	2010-09-07 14:02:48 -07:00
Samuel Clay	c3921b859e	Last_update is now last fetch, not last modification. More accurate last updated dates in Reader.	2010-09-07 13:53:37 -07:00
Samuel Clay	5e1fa36a33	Removing Tag relationships.	2010-09-01 18:19:15 -04:00
Samuel Clay	976e6c2c86	Merge branch 'master' of github.com:samuelclay/NewsBlur	2010-09-01 10:38:00 -04:00
Samuel Clay	c755c9f130	Checking for story_content before comparing origin.	2010-09-01 10:37:33 -04:00
Samuel Clay	c42cabcae2	Celery options for performance.	2010-09-01 08:19:58 -04:00
Samuel Clay	10b1455ba3	Merge branch 'master' of github.com:samuelclay/NewsBlur	2010-08-31 16:34:51 -04:00
Samuel Clay	e2f419cb13	Fixing invalid string error on feed error histories.	2010-08-31 16:34:34 -04:00
Samuel Clay	1ac8402807	Adding indexes to feed/page fetch histories.	2010-08-31 08:45:35 -04:00
Samuel Clay	25c0450020	Commenting out map/reduce statistics because mongo keeps crashing.	2010-08-30 13:33:29 -04:00
Samuel Clay	ba578804b2	Fixing error sin new mongo models.	2010-08-29 20:11:36 -04:00
Samuel Clay	d85194c165	Moving feed/page fetch histories to mongodb.	2010-08-29 14:32:54 -04:00
Samuel Clay	c9d06f9358	Compressing stories. Also fixing compression of feed_pages bootstrap.	2010-08-29 13:23:50 -04:00
Samuel Clay	e26ee9dbcc	Moving feed_pages from PostgreSQL to Mongo. Adding compression.	2010-08-29 12:35:09 -04:00
Samuel Clay	6881cf94f5	Map reduce for counting stories per month and saving both a per-month breakdown, preserving history of old months, and counting the average stories per month.	2010-08-27 19:09:47 -04:00
Samuel Clay	060f63f8a0	Myriad number of bug fixes to reduce the number of exception e-mails I get. Forbidden JSON requests now send 403s. Adding a URl no longer uses a partial URL saearch on feed_link (which would link usetoday.com -> usatoday.com/sports.xml). Also adding the model for FeedPage for mongo.	2010-08-27 18:35:33 -04:00
Samuel Clay	9ca080138e	Merge branch 'master' of github.com:samuelclay/NewsBlur	2010-08-26 10:04:43 -04:00