Commit graph

1333 commits

Author SHA1 Message Date
Samuel Clay
2917d0b4cf Adding new redis3 host for dupe story read status and feed story status. Now with expiring read stories (that need an expunger, still). 2013-07-01 21:59:09 -07:00
Samuel Clay
e6da8d034d Showing loading bar in all views. Also being more aggressive in finding urls when adding. 2013-06-30 17:00:31 -07:00
Samuel Clay
31592710a9 Fixing fetch history bug. Ugggg, a week of fetch history lost. No big deal. Also dropping unused fetch exception history collection. Sentry handles this quite well. 2013-06-28 20:51:31 -07:00
Samuel Clay
ed7b5f1dac Handling validation for image urls. 2013-06-26 16:26:14 -07:00
Samuel Clay
91cc5e58cf image_url -> image_urls 2013-06-26 11:38:49 -07:00
Samuel Clay
cceda612f6 Better logging for unread counts to see where counts are going moving. 2013-06-21 12:30:06 -07:00
Samuel Clay
aeefbb645b Fixing signle story mode. 2013-06-20 14:45:05 -07:00
Samuel Clay
97633614f8 Fixing a few bugs in image url extraction. 2013-06-20 13:46:11 -07:00
Samuel Clay
bff60f4903 Fixing a few bugs in image url extraction. 2013-06-20 13:44:10 -07:00
Samuel Clay
db7afc8abe Adding image url extraction to the API for all stories. 2013-06-20 13:41:37 -07:00
Samuel Clay
bb0a0b612e Clearing up logic on find_story. 2013-06-19 13:22:11 -07:00
Samuel Clay
7127962f5e Fixing fabfile. 2013-06-18 13:22:31 -07:00
Samuel Clay
9826a6499b Moving all pubsub to a dedicated redis box (called REDIS2). 2013-06-18 12:21:27 -07:00
Samuel Clay
3a7631c638 Boosting story ratio to 500 characters. Maybe even higher? 2013-06-17 14:38:15 -07:00
Samuel Clay
5f6b72a396 Fixing log on drained feeds. 2013-06-12 16:55:54 -07:00
Samuel Clay
f2a4b20d91 Adding some useful logging. 2013-06-12 14:36:02 -07:00
Samuel Clay
85120d3f99 Fixing Twitter sharing by upgrading to API v1.1. 2013-06-12 13:52:43 -07:00
Samuel Clay
3a7e72795f Refactoring story_hash methods lower on the stack into MStory. 2013-06-04 15:34:03 -07:00
Samuel Clay
d0e3958ed7 Forcing calculating for last_story_date for more accurate story deletion. 2013-06-04 11:49:33 -07:00
Samuel Clay
22864da4e4 Rewriting train old stories to work much, much faster without having the db sort feeds. 2013-06-04 11:26:01 -07:00
Samuel Clay
abf91eb778 Trimming 1000 feeds at a time. 2013-06-03 19:34:09 -07:00
Samuel Clay
6933af0f48 Trimming way more expired feeds. 2013-06-03 19:31:30 -07:00
Samuel Clay
09f326f1de Correcting trim stories offset for older feeds. 2013-06-03 18:01:29 -07:00
Samuel Clay
7b1b8f7544 Reducing the cutoff even further for older feeds. 2013-06-03 17:58:27 -07:00
Samuel Clay
36bca15464 Better logging for old story trimmer. 2013-06-03 17:48:11 -07:00
Samuel Clay
94f5b781bd Simplifying old story trimmer. 2013-06-03 17:36:57 -07:00
Samuel Clay
edcbde615d Adding a massive feed trimmer. Cuts feeds, those that have no active subs and haven't a new story for a month, down to 10 stories. 2013-06-03 17:20:36 -07:00
Samuel Clay
6186dd8aa5 One more read_preference switch. 2013-06-03 10:27:05 -07:00
Samuel Clay
bec457545c Fixing read preference on feed fetch history. 2013-06-03 10:20:23 -07:00
Samuel Clay
dc3abdc839 Undoing all idomatic read preferences. 2013-05-31 23:03:58 -07:00
Samuel Clay
3dd09e4c4b Switching read_preference to more idiomatic format. 2013-05-31 23:00:47 -07:00
Samuel Clay
f57d6d6016 Switching read_preference to more idiomatic format. 2013-05-31 22:59:27 -07:00
Samuel Clay
811eb77608 Switching read_preference to more idiomatic format. 2013-05-31 22:57:58 -07:00
Samuel Clay
6a51975293 Fixing fetch history caching. 2013-05-31 17:17:20 -07:00
Samuel Clay
04442aa890 Fixing three random exceptions in feed fetching. 2013-05-31 17:14:17 -07:00
Samuel Clay
084e61d989 Fixing first get_or_create issue with fetch histories. 2013-05-31 15:55:39 -07:00
Samuel Clay
ded52bf397 Returning object on save because mongoengine 0.8.1 no longer automatically returns the object on create. 2013-05-29 19:37:50 -07:00
Samuel Clay
e3f2677ee7 Upgrading mongoengine prep. 2013-05-29 18:00:09 -07:00
Samuel Clay
8de1d2d031 Handling dupe story share when de-duping stories. 2013-05-29 16:26:04 -07:00
Samuel Clay
6fa6a11af7 Battling Explosm and other broken feeds by only chekcing for story diffs of substantive content. 2013-05-29 16:22:31 -07:00
Samuel Clay
5dc1342e1c If a feed has an error, save up to 50 fetches before declaring inactive. Otherwise, only keep 5 good fetches. Thanks to @bdarnell for the suggestion. 2013-05-29 16:05:56 -07:00
Samuel Clay
802a080133 Fixing text view premium callout. Also adding more logging to fix explosm feed. 2013-05-29 15:31:41 -07:00
Samuel Clay
418d949b07 Switching hashes on read stories correctly (wtihout making a slow Redis keys() call). 2013-05-28 10:47:24 -07:00
Samuel Clay
36d042acd1 Adding story timestamp to all stories in the API, to complement localized story_date. 2013-05-26 16:32:48 -07:00
Samuel Clay
f1bfcaa1aa Little bit of future padding for new charts. 2013-05-20 17:29:28 -07:00
Samuel Clay
1e1433de06 Merge branch 'master' into circular
* master:
  Fixing fabfile.
  Fixing bug where dupe feed has no link.
2013-05-15 17:46:49 -07:00
Samuel Clay
f5eac30a5a Fixing bug where dupe feed has no link. 2013-05-15 17:21:10 -07:00
Samuel Clay
fd148e106f Merge branch 'master' into circular
* master:
  Temporarily commenting out story hash switching on read stories because of massive redis locking.
2013-05-15 17:13:11 -07:00
Samuel Clay
88d474d62b Temporarily commenting out story hash switching on read stories because of massive redis locking. 2013-05-15 17:13:06 -07:00
Samuel Clay
be3bc8d16b Merge branch 'master' into muserstory_remove
* master:
  Typo on duplicate feed saving truncation.
  Adding duplicate feed address+link truncation.
  Adding duplicate feed address+link truncation.
2013-05-15 16:06:26 -07:00