Commit graph

240 commits

Author SHA1 Message Date
Samuel Clay
d90f5fb6a9 Fixing pipeline. 2013-04-08 17:04:23 -07:00
Samuel Clay
d7ccad3b55 Using pipeline for syncing redis user stories. 2013-04-08 17:02:11 -07:00
Samuel Clay
48c5d4d53e Adding logging, feed cleanup to forced fetch feeds, and expirations on user read stories. 2013-04-07 17:19:59 -07:00
Samuel Clay
2a5bd136da Boosting feed fetches. 2013-03-25 13:15:14 -07:00
Samuel Clay
a2c1eb8ae3 Phase 2: switching to using story_hash index instead of compound unique story_feed_id/story_guid index. 2013-02-20 16:08:14 -08:00
Samuel Clay
7ccf61da5d Short-circuiting mark all as read if there's nothing to change. 2013-02-11 08:40:32 -08:00
Samuel Clay
4bb39e6884 Oldest first goes back two weeks on all stories. 2013-01-10 12:51:39 -08:00
Samuel Clay
0b1b3ee9b7 Undoing some broken changes on reading. 2013-01-08 14:22:53 -08:00
Samuel Clay
4dd1d10d5a Adding avatar photo uploading. 2013-01-08 14:11:59 -08:00
Samuel Clay
5db316bab6 Fixing typo in logging. 2013-01-04 16:44:41 -08:00
Samuel Clay
c4381c31af Logging 2013-01-04 12:26:55 -08:00
Samuel Clay
f6c858f5dc Switching to guid-based read story counts from range based counts. Let's see what this screws up. 2013-01-04 12:06:00 -08:00
Samuel Clay
c223fa892c Adding clean stories task. Updating log colors. 2012-12-26 20:00:38 -08:00
Samuel Clay
1cc128b5a2 Doubling resync likelihood. 2012-12-17 18:20:26 -08:00
Samuel Clay
53b5c4e2db Clearing out old read stories for 1% of all feed fetches with new stories. 2012-12-17 17:28:45 -08:00
Samuel Clay
66dca49bb9 Adding unread cutoff to redis sync. 2012-12-17 17:19:28 -08:00
Samuel Clay
89bc4db947 Syncing redis keys by feed_id for read stories. 2012-12-17 17:14:47 -08:00
Samuel Clay
62b0c41494 Adding a sync all redis for user stories to be run in a migration. 2012-12-17 17:01:59 -08:00
Samuel Clay
bb9338c934 Adding index for read story deletion. 2012-12-17 15:57:15 -08:00
Samuel Clay
28d5f44182 Deleting old stories. 2012-12-17 15:48:43 -08:00
Samuel Clay
4cd24de3bd Typo in delete read stories log. 2012-12-17 15:18:57 -08:00
Samuel Clay
137235c6dd Typo in delete read stories log. 2012-12-17 15:17:45 -08:00
Samuel Clay
0a5aa99948 Typo in delete read stories log. 2012-12-17 15:16:40 -08:00
Samuel Clay
033a56dfe9 Deleting old user stories again. This time going out to 5 times the unread bounds. 2012-12-17 15:14:48 -08:00
Samuel Clay
cc1eb7e230 Modifying overlapping redis keys for river and river blurblog. 2012-12-13 16:41:03 -08:00
Samuel Clay
abb09fea54 Only showing statistics for classifiers with more than 1 user. 2012-12-12 16:36:36 -08:00
Samuel Clay
8ce89a66e8 Method to fix everybody's missing feeds. 2012-12-12 16:10:43 -08:00
Samuel Clay
73961886fd Adding missing feeds for user folders. 2012-12-12 16:05:28 -08:00
Samuel Clay
100bc32b7e When merging classifiers, give a default social user id for old classifiers. 2012-12-12 15:41:44 -08:00
Samuel Clay
b670acaabf Fixing missing dupe stories during merge feeds. 2012-12-12 15:08:24 -08:00
Samuel Clay
11d655c79e Fixing multiple feed subscriptions activity. 2012-12-10 18:36:54 -08:00
Samuel Clay
17c4b2e041 Fixing disappearing stories bug when using oldest or unread filter, since as a story gets read it no longer counts against the unread count, resulting in an off-by-one per read story bug. 2012-12-06 17:33:56 -08:00
Samuel Clay
17c1450951 Merge branch 'master' into ios_story
* master:
  Cleanup.
  Gutting dynamodb and just keeping good changes.
  Bumped version number.
  Modified to remove default URL-encoding for GSON.
  Fixed email field not being visible on some smaller screen devices.
  Modified CSS to include max-width for divs, overriding inline styles and reduce max-width.
  Added padding to share button.
  Shifting mgirations for DStory.
  Optional backked by dynamodb.
  First half of DynamoDB trial, converting stories from mongo to dynamodb. Still needs to be updated/inserted on feed update, and then processed with all MStory uses.
2012-11-27 17:34:53 -08:00
Samuel Clay
f5292fa9a1 Cleanup. 2012-11-27 17:12:10 -08:00
Samuel Clay
8c085b0a3c Gutting dynamodb and just keeping good changes. 2012-11-27 16:44:10 -08:00
Samuel Clay
61aac7fc06 Merge branch 'master' into dynamodb
* master: (331 commits)
  Adding blurblog permalink to story on blurblog. Adding story slug to permalink. Fixing story guid hashes.
  Re-enabling message property for facebook.
  Ignoring image for facebook shares.
  Commenting out facebook share message until they approve it.
  Cleaning up Facebook og description.
  Facebook using a website instead of article.
  Facebook being a PITA.
  Facebook wants ISO8601 datetime format.
  Facebook doesn't like going to other URLs.
  Facebook doesn't like going to other URLs.
  Dumb typo in new facebook og crap.
  Using blurblog permalink for new facebook shares.
  Fixing Facebook share to use fancy actions.
  Adding scroll to comments button to share bar. Thanks @afita.
  Turning off microformats for more errors.
  Fixing errors in timeouts to show the correct error. Also fixing microformats parsing issue and allow IPv6 URLs in enclosures to be ignored, fixing a bunch of feeds.
  Cleaning redis stories for 1% of all feed fetches.
  Refreshing feed on fetch.
  Fiddling with logging on dupe feeds.
  Adding bs facebook assets.
  ...

Conflicts:
	apps/rss_feeds/models.py
	settings.py
	utils/feed_fetcher.py
2012-11-27 16:39:39 -08:00
Samuel Clay
1ad7031fbb Fixing tests. 2012-11-27 16:22:03 -08:00
Samuel Clay
76dcecde1e Fixing NotUniqueError on read stories by making sure the corret unique index is used. 2012-10-30 18:10:49 -07:00
Samuel Clay
691598aedd Fixing wonky unread counts. 2012-10-30 08:31:37 -07:00
Samuel Clay
e52b64aec0 Fixing bug in story_date attribute on upgdated feed fetch. 2012-10-29 12:30:34 -07:00
Samuel Clay
2ec10bcf7e Readying unread counting to incorporate the new read preference in mongoengine. 2012-10-29 12:25:28 -07:00
Samuel Clay
3c9114fa05 Merge branch 'master' into dynamodb
* master:
  S3 icons. Untested.
  Removing old feed pages from mongo.
  Allowing proxy of S3 for original pages to be turned off.
  Adding ec2 mdadm setup.
  Adding proxy for S3 pages.
  Migration for s3 flags for pages + icons.
  Migrating to S3 for feed pages.
  Cleaning up footer on welcome page.
  Retina-izing twitter and github logos.
  Adding mongo conf for ec2 (trying out ec2 for some servers).
  Fixing a bunch of small exceptions.
  Small cleanup on welcome page, fixing social subs urls.
  Updating welcome apge to include pricing and a cleaned up footer.
  Fixing a couple of update bugs -- story cascading potentially resulting in invalid ids (probably not), and not unique errors on new page fetches.

Conflicts:
	apps/rss_feeds/models.py
	settings.py
2012-09-19 16:37:47 -07:00
Samuel Clay
ef4a1c866e Fixing a bunch of small exceptions. 2012-09-18 13:08:25 -07:00
Samuel Clay
f4add87dc6 Fixing a couple of update bugs -- story cascading potentially resulting in invalid ids (probably not), and not unique errors on new page fetches. 2012-09-17 17:01:56 -07:00
Samuel Clay
4ec6c9fefd Merge branch 'master' into dynamodb
* master: (23 commits)
  Fixing trainer for stories in social feeds.
  Fixing small bug in story titles header now showing socialsubs correctly.
  Fixing headers for Everything river and river blurblog.
  Massive refactor around feedbar, brought on by @afita, in order to give Folders a menu in the story titles header (and then make the mark as read button work on folders in the header.)
  Updating story position in closed story titles when pane is re-opened. Thanks to @afita for the bug.
  Updating to altest jQuery layout.
  Adding folder story counts to content pane. Thanks to @afita for the suggestion.
  Fixing activities bug when the user is unknown and there is no source/with_user.
  Calculating correct page fetch timing offset (if it was a 304).
  Adding server name and feed code to analytics.
  Turning analytics logging back on.
  Temporarily commenting out analytics to see if they cure the slow down.
  Shortening names on analytics for improved space saving.
  Adding new analytics collections to keep track of page loads and feed fetches. Need to start aggregating before I can tell how useful this data will be.
  Adding new analytics collections to keep track of page loads and feed fetches. Need to start aggregating before I can tell how useful this data will be.
  Adding VentureBeat article to press page.
  Adding order/read_filter to api. Also adding river blurblog.
  Fixing right click on folders.
  Fixing bug where publisher-disabled original pages were not properly disabled.
  Fixing bug where publisher-disabled original pages were not properly disabled.
  ...

Conflicts:
	utils/feed_fetcher.py
2012-09-07 22:26:19 -07:00
Samuel Clay
0456c5757c Fixing activities bug when the user is unknown and there is no source/with_user. 2012-09-07 15:10:07 -07:00
Samuel Clay
c07d7e7578 Adding new analytics collections to keep track of page loads and feed fetches. Need to start aggregating before I can tell how useful this data will be. 2012-09-06 17:24:42 -07:00
Samuel Clay
0a03154473 First half of DynamoDB trial, converting stories from mongo to dynamodb. Still needs to be updated/inserted on feed update, and then processed with all MStory uses. 2012-09-04 12:01:55 -07:00
Samuel Clay
825378d83a Fixing ghost unreads by accounting for the implicit score of 1 in unsorted sets in redis when intersected with a sorted set. Doh! 2012-08-29 18:44:43 -07:00
Samuel Clay
0fbf460bc6 Using a 0 feed_id for shared stories with no feed. 2012-08-25 20:46:48 -07:00