Commit graph

986 commits

Author SHA1 Message Date
Samuel Clay
04442aa890 Fixing three random exceptions in feed fetching. 2013-05-31 17:14:17 -07:00
Samuel Clay
0d6e0e28f3 Instrumenting pymongo to log queries in debug. Including Replica Set Connection. 2013-05-30 10:39:41 -07:00
Samuel Clay
e3f2677ee7 Upgrading mongoengine prep. 2013-05-29 18:00:09 -07:00
Samuel Clay
123de0dcb7 Adding monit script for watching node real-time server. 2013-05-28 18:22:19 -07:00
Samuel Clay
84c16d7464 Adding user queue size to munin. 2013-05-21 14:15:25 -07:00
Samuel Clay
fbb908174a Stripping www from blurblogs. 2013-05-21 06:34:49 -07:00
Samuel Clay
8835458c71 Removing dev references. 2013-05-21 06:05:17 -07:00
Samuel Clay
7afd2ff946 Restructuring fabfile to use dynamic hosts. 2013-05-20 17:29:28 -07:00
Samuel Clay
93fdab24da Auto-naming new app/task servers based on existing servers. 2013-05-20 12:57:45 -07:00
Samuel Clay
88a099a357 Merge branch 'master' into circular
* master:
  Adding dynamic ssh and fab hosting by querying digitalocean on every command.
  Cleaning up time calibration code for new servers.
  Adding automated redis backup.
2013-05-20 12:03:21 -07:00
Samuel Clay
a84ca42310 Adding dynamic ssh and fab hosting by querying digitalocean on every command. 2013-05-20 12:03:14 -07:00
Samuel Clay
d5972c0eef Adding automated redis backup. 2013-05-18 21:48:03 -07:00
Samuel Clay
6c394f18c2 Merge branch 'master' into circular
* master:
  Adding ReadKit to supported clients.
2013-05-16 11:11:52 -07:00
Samuel Clay
0b81e57871 Adding ReadKit to supported clients. 2013-05-16 11:11:46 -07:00
Samuel Clay
f332a90140 Just use regular yaml and link it. 2013-05-15 18:19:17 -07:00
Samuel Clay
ada894441b Just use regular yaml and link it. 2013-05-15 18:19:00 -07:00
Samuel Clay
67f2028aef Merge branch 'master' into circular
* master:
  Moving to vendorized yaml.
  Removing PyYAML as a non-vendorized dependency.
  Vendorizing pyyaml because pyyaml.org is down so often.
2013-05-15 18:08:41 -07:00
Samuel Clay
0b793a3797 Moving to vendorized yaml. 2013-05-15 18:08:37 -07:00
Samuel Clay
6ebaa81ea1 Merge branch 'muserstory_remove' into circular
* muserstory_remove:
  Adding node group to fabfile.
  Using popular instead of homepage user to suggest new users to follow.
  Cleaning up unused detritus. Ready to deploy.
  Final cleanup before deploying.
  Removing MUserStory entirely.
2013-05-14 16:11:44 -07:00
Samuel Clay
a65676bbd2 Forcing dev subdomain on new user queue email. 2013-05-13 18:31:44 -07:00
Samuel Clay
c74fad0feb www -> dev for press. 2013-05-13 18:26:09 -07:00
Samuel Clay
b98d129d1b Removing MUserStory entirely. 2013-05-10 16:11:30 -07:00
Samuel Clay
2bd2589c80 Handling error when user is already refunded. 2013-05-10 12:15:54 -07:00
Samuel Clay
23a7aa84d7 Merge branch 'master' into circular
* master:
  Using story cache when calculating unreads, for those who need unread counts updating.
2013-05-03 13:17:07 -07:00
Samuel Clay
550a33d9ab Using story cache when calculating unreads, for those who need unread counts updating. 2013-05-03 09:43:21 -07:00
Samuel Clay
97041dece7 Merge branch 'master' into circular
* master:
  Fixing story hash migration to run by user.
  Adding story_hash to starred stories, user stories, and shared stories.
  Adding redis backup script.
2013-04-29 18:00:55 -07:00
Samuel Clay
4d014033b6 Adding redis backup script. 2013-04-29 14:06:42 -07:00
Samuel Clay
42b85d485a Merge branch 'schedule' into schedule_dev
* schedule:
  Adding metric for last story date, used to calculate next scheduled update.
2013-04-23 17:04:27 -07:00
Samuel Clay
1584208677 Adding metric for last story date, used to calculate next scheduled update. 2013-04-23 17:04:21 -07:00
Samuel Clay
629fb2fd52 Merge branch 'master' into circular
* master:
  Adding logging to mongo backup.
2013-04-20 15:06:36 -07:00
Samuel Clay
408fb99802 Adding logging to mongo backup. 2013-04-20 15:06:32 -07:00
Samuel Clay
c8907bbab4 Re-designed footer. Adding footer to homepage. 2013-04-18 13:49:30 -07:00
Samuel Clay
976a5f15d1 Merge branch 'master' into circular
* master:
  Upping feed fetches.
  Adding meta refresh to maintenance page.
  Adding work to fabfile. Changing deploy to deploy_code.
2013-04-15 19:26:45 -07:00
Samuel Clay
84bcb54ff2 Adding work to fabfile. Changing deploy to deploy_code. 2013-04-15 18:32:13 -07:00
Samuel Clay
41f9257d7f Merge branch 'master' into circular
* master:
  Refactoring real-time socketio clientside to fix multiple client updates.
  Unread counts only for logged in users.

Conflicts:
	media/js/newsblur/reader/reader.js
2013-04-15 18:00:00 -07:00
Samuel Clay
76cbbd8d8b Refactoring real-time socketio clientside to fix multiple client updates. 2013-04-15 17:59:06 -07:00
Samuel Clay
f6b5262c4f Merge branch 'master' into circular
* master:
  Fixing small bug around missing feeds during update pipeline.
  Fixing race condition between primary adn secondaries when creating feed fetch histories.
  Reversing order for fetch histories.
  Moving from feed/page/push histories to capped fetch history.

Conflicts:
	apps/rss_feeds/views.py
2013-04-15 15:17:29 -07:00
Samuel Clay
6011e4f2da Moving from feed/page/push histories to capped fetch history. 2013-04-15 14:30:31 -07:00
Samuel Clay
cdb936ddd0 Merge branch 'master' into circular
* master:
  Switching feed fetch counts from mongo to redis.
  Versioning postgres backup script.
  Versioning postgres backup script.
2013-04-15 12:09:46 -07:00
Samuel Clay
eb08ef5569 Versioning postgres backup script. 2013-04-14 21:07:49 -07:00
Samuel Clay
4eb49687e0 Versioning postgres backup script. 2013-04-14 21:07:07 -07:00
Samuel Clay
eddcf1eddb Merge branch 'master' into circular
* master:
  Using regular expressions for comment stripping, unless the feed has an error count, in which case switch to lxml.
2013-04-08 16:15:29 -07:00
Samuel Clay
47c2257ef7 Using regular expressions for comment stripping, unless the feed has an error count, in which case switch to lxml. 2013-04-08 16:14:48 -07:00
Samuel Clay
e2a4546eb1 Merge branch 'master' into circular
* master:
  No longer doing overly zealous read story cleanup.
  Adding error_feeds count to munin.
2013-04-08 11:47:45 -07:00
Samuel Clay
532d1a67a7 No longer doing overly zealous read story cleanup. 2013-04-08 11:47:39 -07:00
Samuel Clay
eebba16d89 Adding error_feeds count to munin. 2013-04-08 10:53:11 -07:00
Samuel Clay
ff9bf4fad8 Merge branch 'master' into circular
* master:
  Skipping tasking of feeds when queue is too high. Use the random queue.
  Pushing out feed updates to cope with influx.
  Don't allow stories to come from more than 24 hours into the future.
  Removing force from new feeds.
  Adding in timing on feed cleanup.
  Increasing fetches for long tail feeds.
  Adding logging, feed cleanup to forced fetch feeds, and expirations on user read stories.
  Adding supervisord.conf config to include minfds (ulimit -a) support.

Conflicts:
	utils/feed_fetcher.py
2013-04-08 09:50:08 -07:00
Samuel Clay
639ad60ba1 Don't allow stories to come from more than 24 hours into the future. 2013-04-07 21:07:34 -07:00
Samuel Clay
36e76759e8 Adding in timing on feed cleanup. 2013-04-07 17:49:34 -07:00
Samuel Clay
1175b306cd Adding logging, feed cleanup to forced fetch feeds, and expirations on user read stories. 2013-04-07 17:20:17 -07:00