Commit graph

389 commits

Author SHA1 Message Date
Samuel Clay
bb3e0a6567 Putting Google Alerts/News back into rotation. 2013-09-30 12:40:14 -07:00
Samuel Clay
70ea542481 User-dependent days of unread. 14 for free users, 30 for premium. 2013-09-16 16:42:49 -07:00
Samuel Clay
831a771309 Fixing logging message. 2013-09-16 12:11:26 -07:00
Samuel Clay
886f6c0d8b Fixing logging message. 2013-09-16 12:08:51 -07:00
Samuel Clay
5911acaa35 Moving the majority of story fetches to mongo secondaries. 2013-09-16 12:07:15 -07:00
Samuel Clay
143ee770ce Skipping google news feeds again. Need to tone down these fetches. 2013-09-15 23:41:17 -07:00
Samuel Clay
3fc40acd47 Removing double. 2013-09-13 19:10:41 -07:00
Samuel Clay
2fe65adc4d Re-enabling Google News fetches. 2013-09-11 10:50:20 -07:00
Samuel Clay
00a412960a Skipping google alerts. 2013-09-09 13:50:05 -07:00
Samuel Clay
2b2c731bd5 Fixing a couple exceptions and turning off google news rss feeds temporarily. 2013-09-09 13:46:14 -07:00
Samuel Clay
dec22ac776 Moving to simpler read story endpoint. 2013-09-06 12:42:39 -07:00
Samuel Clay
2394e31f89 Moving feed parsing hotspot to specific seq ratio check. 2013-08-06 13:54:06 -07:00
Samuel Clay
ecf11eed98 Upping process feed time limit. 2013-08-06 13:42:53 -07:00
Samuel Clay
683140af2e Moving feed processing to timelimit due to some feeds sucking up all the time. 2013-08-06 13:38:35 -07:00
Samuel Clay
88f2a69a93 Fixing a dozen text and feed fetching bugs. 2013-08-06 13:18:55 -07:00
Samuel Clay
d49e9569a5 Fixing #377. Killing XSS attacks by sanitizing feed titles. 2013-07-29 14:25:43 -07:00
Samuel Clay
ee5c00bce5 Using latest feedparser. 2013-07-10 16:10:19 -07:00
Samuel Clay
cf36df47fa Bumping feed blast limit from 50 to 100 stories at a time. 2013-07-10 15:13:19 -07:00
Samuel Clay
5f7ae25fc4 Normaling user agents between feed/page/icon fetchers. Adding full site permalink. 2013-07-10 14:14:55 -07:00
Samuel Clay
6fda0b6a35 Timing out on PuSH hubs when down. 2013-07-01 18:46:21 -07:00
Samuel Clay
9826a6499b Moving all pubsub to a dedicated redis box (called REDIS2). 2013-06-18 12:21:27 -07:00
Samuel Clay
e3f2677ee7 Upgrading mongoengine prep. 2013-05-29 18:00:09 -07:00
Samuel Clay
b98d129d1b Removing MUserStory entirely. 2013-05-10 16:11:30 -07:00
Samuel Clay
550a33d9ab Using story cache when calculating unreads, for those who need unread counts updating. 2013-05-03 09:43:21 -07:00
Samuel Clay
6011e4f2da Moving from feed/page/push histories to capped fetch history. 2013-04-15 14:30:31 -07:00
Samuel Clay
532d1a67a7 No longer doing overly zealous read story cleanup. 2013-04-08 11:47:39 -07:00
Samuel Clay
36e76759e8 Adding in timing on feed cleanup. 2013-04-07 17:49:34 -07:00
Samuel Clay
48c5d4d53e Adding logging, feed cleanup to forced fetch feeds, and expirations on user read stories. 2013-04-07 17:19:59 -07:00
Samuel Clay
f2281ec7bc Refunding stripe payments. 2013-04-05 17:54:22 -07:00
Samuel Clay
3d190cc3ec Fixing a few small feed collision errors. 2013-04-04 17:18:27 -07:00
Samuel Clay
8a7ea248fd Fixing feed address change by feed link when colliding with existing feed. 2013-04-04 17:09:16 -07:00
Samuel Clay
5898ea9df0 Feed may not longer exist. Toss it. 2013-04-03 21:20:19 -07:00
Samuel Clay
9e2fa3115d Cache busting feeds on force and 1% of the time. 2013-03-24 15:50:57 -07:00
Samuel Clay
d793946e3a If not active subs, turn off real-time. 2013-03-20 16:17:20 -07:00
Samuel Clay
a75554374b Using @dcramer's excellent getsentry.com for all exceptions. Turning off exception emails. 2013-03-20 08:24:11 -07:00
Samuel Clay
e4fc8b6c49 Turning off PuSH if no hub url found. 2013-03-07 10:58:05 -05:00
Samuel Clay
bdd60bf3f7 Fixing push re-sub logging. 2013-03-06 14:47:15 -08:00
Samuel Clay
00f59821be Resubscribing to PuSH feeds that have expired. 2013-03-06 14:45:01 -08:00
Samuel Clay
5170b56689 If last_modified fails, clear it. 2013-01-31 12:21:10 -08:00
Samuel Clay
79c6ea09cd Don't collide stories that are too short to be collided. 2013-01-28 16:56:01 -08:00
Samuel Clay
4e09e4db81 Fixing user agent. 2013-01-22 18:18:07 -08:00
Samuel Clay
bcc24c1253 Fixing PuSH by using the correct hub url. Thanks to @donmelton for finding the bug. 2013-01-22 18:15:28 -08:00
Samuel Clay
cb30fe2ae8 Bumping up duplicate address length for feeds. 2013-01-07 16:35:29 -08:00
Samuel Clay
1cc128b5a2 Doubling resync likelihood. 2012-12-17 18:20:26 -08:00
Samuel Clay
53b5c4e2db Clearing out old read stories for 1% of all feed fetches with new stories. 2012-12-17 17:28:45 -08:00
Samuel Clay
5a5aa0d17b Turning off deletion of old stories until the db can be prepped. 2012-12-17 16:19:28 -08:00
Samuel Clay
28d5f44182 Deleting old stories. 2012-12-17 15:48:43 -08:00
Samuel Clay
033a56dfe9 Deleting old user stories again. This time going out to 5 times the unread bounds. 2012-12-17 15:14:48 -08:00
Samuel Clay
1387d7cfc4 Upgrading celery just in case it's causing priority loss. 2012-12-17 13:44:10 -08:00
Samuel Clay
2716d0c404 Cleaning up invalid story dates. 2012-12-10 15:48:05 -08:00