Commit graph

628 commits

Author SHA1 Message Date
Samuel Clay
d167788f29 Further pushing out feed updates. 2013-03-20 10:04:25 -07:00
Samuel Clay
e73f3aa66d Increasing feed fetches quite a bit. 2013-03-14 19:26:22 -07:00
Samuel Clay
85c85ab565 Pushing out feed updates until able to handle the giant influx. 2013-03-14 15:52:34 -07:00
Samuel Clay
9c5c210568 Extending random holds for feeds while being crushed. 2013-03-14 00:47:18 -07:00
Samuel Clay
4dba6ce380 Counting average stories if stories were detected in the last month. This should fix feeds that have long delays but shouldn't. 2013-02-25 11:11:47 -08:00
Samuel Clay
fc8b2bd4ec First half of story hash migration. Setting up unique sparse index, then migration to calculate hashes for every story. 2013-02-20 15:42:40 -08:00
Samuel Clay
3bdb4af70a Fixing the timeout on premium setups by doing the scheduling of premium setups in a task, only immediately activating all of a user's feeds. 2013-02-15 09:52:11 -08:00
Samuel Clay
0d75d9ecd8 1 month max on next fetch. 2013-02-13 12:54:14 -08:00
Samuel Clay
6c6ad7ed3d If a feed has no stories in the past month, fetch it 1/12 as often. 2013-02-07 15:30:35 -08:00
Samuel Clay
e2d6b8d6f3 Fixing broken branch id in unicode/str for feed. 2013-01-29 15:58:15 -08:00
Samuel Clay
c18c8b54bc Correcting feeds that get merged but are originally branches. 2013-01-29 14:37:01 -08:00
Samuel Clay
79c6ea09cd Don't collide stories that are too short to be collided. 2013-01-28 16:56:01 -08:00
Samuel Clay
7e52fc37ef Fixing assortment of small bugs. 2013-01-28 15:43:00 -08:00
Samuel Clay
36028cc719 Text view. Needs a few tweaks and a premium only view. 2013-01-08 18:33:30 -08:00
Samuel Clay
0b1b3ee9b7 Undoing some broken changes on reading. 2013-01-08 14:22:53 -08:00
Samuel Clay
4dd1d10d5a Adding avatar photo uploading. 2013-01-08 14:11:59 -08:00
Samuel Clay
cb30fe2ae8 Bumping up duplicate address length for feeds. 2013-01-07 16:35:29 -08:00
Samuel Clay
bc0192c3df Adding broken search for feeds. 2013-01-04 16:34:27 -08:00
Samuel Clay
aefd8de78c Backporting collections.Counter to python2.6 2013-01-03 18:15:00 -08:00
Samuel Clay
40fa172edc Prototype of a dupe checker using real-time update times to figure out which feeds are the same. Last time I walked down this road I got run over. 2013-01-03 13:47:38 -08:00
Samuel Clay
5ed2109bcf Using queue for scheduling feed fetches on load. 2013-01-02 12:27:08 -08:00
Samuel Clay
cba0733a28 Attempting to correctly save duplicate feeds. 2012-12-28 22:03:48 -08:00
Samuel Clay
db63f48bf5 Adding tons of logging (especially user-facing) to feed exception handling. Much much better feed address and link fetch history. 2012-12-28 21:42:52 -08:00
Samuel Clay
70f2da6218 Reducing queue size of push feeds when overloaded. 2012-12-26 02:41:13 -08:00
Samuel Clay
1ca58ed515 Showing maintenance mode message. One day ... put in a live chat. 2012-12-25 15:28:13 -08:00
Samuel Clay
62d210ffd6 Migration to increase max length of feed address. 2012-12-25 14:58:59 -08:00
Samuel Clay
f94939b83e Cleaning up task feeds logging. 2012-12-25 12:08:17 -08:00
Samuel Clay
6311bf41d7 Updating sessions handling on page importer in anticipation of ugprading to latest python requests. 2012-12-24 00:39:52 -08:00
Samuel Clay
63bd36212c Fixing feed fetcher to use updated date if the feed doesn't provide it otherwise. 2012-12-24 00:10:40 -08:00
Samuel Clay
36f99a3b27 Inactive feeds get their stories cut. 2012-12-21 17:14:13 -08:00
Samuel Clay
396bfc5196 Pushing out the quiet feeds. 2012-12-21 16:48:47 -08:00
Samuel Clay
905b2e32f9 Trimming more stories from feeds. 2012-12-21 15:55:25 -08:00
Samuel Clay
3449e22565 Turning off auto-indexing for starred stories until it's ready to go. 2012-12-20 16:20:28 -08:00
Samuel Clay
612a349239 Updating search for saved stories to search fuzzy queries. Connected to UI. 2012-12-20 16:07:22 -08:00
Samuel Clay
93e3a76087 Merge branch 'master' into search
* master: (1171 commits)
  Fixing repeating friend finding check in both intro and friends dialog.
  Saving references to visible feeds when reading by folder.
  Major refactor of ios feed list. No longer relies on fragile locations dictionary.
  No more error messages on deploying new celery/gunicorn. Also adding NUMA cancellation support to mongo db server.
  Fixing ios classifiers in river view.
  Adding beta search (only titles and authors for now) for staff only.
  Updating enterprise distribution ipa.
  Fixing crash when mis-counting feeds. Correcting height of font popover on iphone. Adding horizontal bounce to story page control.
  Doubling resync likelihood.
  Clearing out old read stories for 1% of all feed fetches with new stories.
  Adding unread cutoff to redis sync.
  Syncing redis keys by feed_id for read stories.
  Adding a sync all redis for user stories to be run in a migration.
  Turning off deletion of old stories until the db can be prepped.
  Adding index for read story deletion.
  Deleting old stories.
  Typo in delete read stories log.
  Typo in delete read stories log.
  Typo in delete read stories log.
  Deleting old user stories again. This time going out to 5 times the unread bounds.
  ...

Conflicts:
	local_settings.py.template
	settings.py
2012-12-20 15:07:43 -08:00
Samuel Clay
248d511f90 Adding beta search (only titles and authors for now) for staff only. 2012-12-19 14:21:46 -08:00
Samuel Clay
327b300fe0 Trying out finding by id for shared stories for when the original story is gone. 2012-12-13 17:49:07 -08:00
Samuel Clay
a494e4d865 Fixing typo on dupe feeds. 2012-12-12 16:37:39 -08:00
Samuel Clay
abb09fea54 Only showing statistics for classifiers with more than 1 user. 2012-12-12 16:36:36 -08:00
Samuel Clay
73961886fd Adding missing feeds for user folders. 2012-12-12 16:05:28 -08:00
Samuel Clay
d085035430 Fixing missing dupe stories during merge feeds. 2012-12-12 15:13:17 -08:00
Samuel Clay
2716d0c404 Cleaning up invalid story dates. 2012-12-10 15:48:05 -08:00
Samuel Clay
778353b64d Fixing broken guids by using permalink. 2012-12-10 15:20:58 -08:00
Samuel Clay
1e63a38ce3 Adding logging for dupe feeds. 2012-12-10 14:54:10 -08:00
Samuel Clay
27f5dc18df Trying to fix dupe detection. 2012-12-10 14:40:51 -08:00
Samuel Clay
f5292fa9a1 Cleanup. 2012-11-27 17:12:10 -08:00
Samuel Clay
8c085b0a3c Gutting dynamodb and just keeping good changes. 2012-11-27 16:44:10 -08:00
Samuel Clay
61aac7fc06 Merge branch 'master' into dynamodb
* master: (331 commits)
  Adding blurblog permalink to story on blurblog. Adding story slug to permalink. Fixing story guid hashes.
  Re-enabling message property for facebook.
  Ignoring image for facebook shares.
  Commenting out facebook share message until they approve it.
  Cleaning up Facebook og description.
  Facebook using a website instead of article.
  Facebook being a PITA.
  Facebook wants ISO8601 datetime format.
  Facebook doesn't like going to other URLs.
  Facebook doesn't like going to other URLs.
  Dumb typo in new facebook og crap.
  Using blurblog permalink for new facebook shares.
  Fixing Facebook share to use fancy actions.
  Adding scroll to comments button to share bar. Thanks @afita.
  Turning off microformats for more errors.
  Fixing errors in timeouts to show the correct error. Also fixing microformats parsing issue and allow IPv6 URLs in enclosures to be ignored, fixing a bunch of feeds.
  Cleaning redis stories for 1% of all feed fetches.
  Refreshing feed on fetch.
  Fiddling with logging on dupe feeds.
  Adding bs facebook assets.
  ...

Conflicts:
	apps/rss_feeds/models.py
	settings.py
	utils/feed_fetcher.py
2012-11-27 16:39:39 -08:00
Samuel Clay
931d4bc991 Adding blurblog permalink to story on blurblog. Adding story slug to permalink. Fixing story guid hashes. 2012-11-27 11:59:54 -08:00
Samuel Clay
20439ea85f Fiddling with logging on dupe feeds. 2012-11-26 10:20:41 -08:00