Commit graph

2213 commits

Author SHA1 Message Date
Samuel Clay
a6837724e3 Increasing task queue size. 2013-04-08 17:11:40 -07:00
Samuel Clay
d90f5fb6a9 Fixing pipeline. 2013-04-08 17:04:23 -07:00
Samuel Clay
d7ccad3b55 Using pipeline for syncing redis user stories. 2013-04-08 17:02:11 -07:00
Samuel Clay
47c2257ef7 Using regular expressions for comment stripping, unless the feed has an error count, in which case switch to lxml. 2013-04-08 16:14:48 -07:00
Samuel Clay
33e605875e Switch to lxml for comment stripping. 2013-04-08 15:38:23 -07:00
Samuel Clay
9565c27784 Moving to 1 feed per task queue (from 12). 2013-04-08 14:39:00 -07:00
Samuel Clay
83c8443ed0 Prioritizing autocomplete matches on title before address. 2013-04-08 13:12:11 -07:00
Samuel Clay
73eb0b6ef4 Handling URLs in autocomplete. 2013-04-08 13:09:43 -07:00
Samuel Clay
c25c4478e6 Adding new autocomplete. Let's hope this doesn't destroy performance in Redis. 2013-04-08 12:54:02 -07:00
Samuel Clay
46f919f919 Using error_feeds to push broken feeds further out in the pipeline. 2013-04-08 10:50:50 -07:00
Samuel Clay
7fc9b494af Adding error_feeds queue to keep track of broken feeds. 2013-04-08 10:04:40 -07:00
Samuel Clay
d63556bc06 Skipping tasking of feeds when queue is too high. Use the random queue. 2013-04-08 09:49:21 -07:00
Samuel Clay
00591ee7ad Pushing out feed updates to cope with influx. 2013-04-08 08:34:12 -07:00
Samuel Clay
c41bf80b4e Removing force from new feeds. 2013-04-07 20:22:46 -07:00
Samuel Clay
5c0b9de511 Increasing fetches for long tail feeds. 2013-04-07 17:31:43 -07:00
Samuel Clay
48c5d4d53e Adding logging, feed cleanup to forced fetch feeds, and expirations on user read stories. 2013-04-07 17:19:59 -07:00
Samuel Clay
1cf8d67db1 Fixing authentication issue on account deletion when using blank passwords. 2013-04-05 19:23:42 -07:00
Samuel Clay
d238dafba7 Bumping up feed fetching for premiums. 2013-04-05 17:59:33 -07:00
Samuel Clay
f2281ec7bc Refunding stripe payments. 2013-04-05 17:54:22 -07:00
Samuel Clay
9c79076aab Better verbose logging for updating feeds. 2013-04-05 12:09:42 -07:00
Samuel Clay
1ba468d27d Checking if feed id has changed, then removing from task queue. 2013-04-04 21:57:58 -07:00
Samuel Clay
d831088a46 Fixed the ridiculous bug with feed queuing: while checking to see if the feed id changed, self.pk changed along with the feed, resulting in the cached value changing! 2013-04-04 20:51:56 -07:00
Samuel Clay
341db243c3 Better logging for queues. 2013-04-04 20:47:41 -07:00
Samuel Clay
49f1401953 Handling 0 images found for shared story. 2013-04-04 17:53:57 -07:00
Samuel Clay
681968033b Fixing broken JPG image sizer. 2013-04-04 17:49:38 -07:00
Samuel Clay
ee647be85b Handling timeouts for feed link change. 2013-04-04 17:27:31 -07:00
Samuel Clay
3d190cc3ec Fixing a few small feed collision errors. 2013-04-04 17:18:27 -07:00
Samuel Clay
e7ad04946d Fixing feed address change by feed link when colliding with existing feed. 2013-04-04 17:13:06 -07:00
Samuel Clay
8a7ea248fd Fixing feed address change by feed link when colliding with existing feed. 2013-04-04 17:09:16 -07:00
Samuel Clay
a36a960566 Bumping numbers on task fetching. 2013-04-03 22:59:25 -07:00
Samuel Clay
95f97a4718 Adding logging. 2013-04-03 22:51:03 -07:00
Samuel Clay
400cad89e6 Tossing deleted feeds from queue. 2013-04-03 21:11:40 -07:00
Samuel Clay
3627221968 Tossing deleted feeds from queue. 2013-04-03 21:10:37 -07:00
Samuel Clay
6b262f8557 Cleaning up counts in munin and work log. 2013-04-03 19:04:30 -07:00
Samuel Clay
4651906f08 What the f? strip_comments now eats CPU on bad stories. But why now and not a few months ago when I introduced it? 2013-04-03 18:20:42 -07:00
Samuel Clay
7a41b6a949 Using proper hostname for task server. Adding verbosity to feed fetching. 2013-04-03 18:11:23 -07:00
Samuel Clay
4461bb9c99 Removing old inactive feeds refresher. 2013-04-03 17:31:37 -07:00
Samuel Clay
6a2a144747 Requeueing old tasked feeds. 2013-04-03 17:28:51 -07:00
Samuel Clay
8f8c88ca47 Adding tasked_feeds set to handle dropped feeds. 2013-04-03 17:22:45 -07:00
Samuel Clay
e0457bf0d0 Fixing missing feed bug. 2013-04-03 16:48:42 -07:00
Samuel Clay
c0efd9ddd3 Cleaning up queue migration. 2013-04-03 15:59:44 -07:00
Samuel Clay
9d7aeb1a51 Merge branch 'master' into queue
# By Samuel Clay (2) and Lance Johnson (1)
# Via Lance Johnson (2) and Samuel Clay (2)
* master:
  Adding feed id to feed selector.
  Giving error on empty mark read endpoint.
  Fix duplicate feed listing bug and fix issus with remembering folder collapse/expande perferences in main feed list view
2013-04-03 15:25:22 -07:00
Samuel Clay
add5187529 Giving error on empty mark read endpoint. 2013-04-02 09:36:43 -07:00
Samuel Clay
842afe7068 Handling decayed feeds with no more active subscribers. They don't get returned to the scheduled_updates queue until there are enough active users. 2013-04-01 15:45:22 -07:00
Samuel Clay
e11e8398d1 Only need next scheduled update. 2013-04-01 15:21:03 -07:00
Samuel Clay
07d1fcb207 Fixing feeds that fall through. 2013-04-01 12:21:00 -07:00
Samuel Clay
ba5919d760 Moving task queue from postgres db backed to redis backed. 2013-03-30 19:05:13 -07:00
Samuel Clay
2cc5ab6202 Boosting feed fetching. 2013-03-30 11:39:49 -07:00
Samuel Clay
03c27b42e8 Removing unused feed hash index (keeping the unique constraint). 2013-03-29 22:41:16 -07:00
Samuel Clay
33bd64f2a0 Adding two indexes and killing one unused one from feeds. 2013-03-29 22:11:36 -07:00