Commit graph

329 commits

Author SHA1 Message Date
Samuel Clay
cf554bb598 Cleaning redis stories for 1% of all feed fetches. 2012-11-26 10:39:10 -08:00
Samuel Clay
6b9c737366 Refreshing feed on fetch. 2012-11-26 10:27:08 -08:00
Samuel Clay
240f5e4c6e Cleaning up RSS feed header for shared stories feeds. 2012-10-30 15:06:30 -07:00
Samuel Clay
fa692d746c Forcing all unread counts to be counted from primary mongo. 2012-10-30 10:57:12 -07:00
Samuel Clay
2ec10bcf7e Readying unread counting to incorporate the new read preference in mongoengine. 2012-10-29 12:25:28 -07:00
Samuel Clay
b543640554 Fixing raven client on feed fetchers. 2012-10-25 16:23:35 -07:00
Samuel Clay
e572b3d53f Moving to better feed finder (in the case of dupe feeds). 2012-10-25 16:14:25 -07:00
Samuel Clay
7e281e1d7b Adding timeout errors to raven/sentry error reporting. 2012-10-25 16:09:06 -07:00
Samuel Clay
c0b791a01b Fixing an uncaught exception around feeds that change ids. 2012-10-25 15:16:21 -07:00
Samuel Clay
ec57fc927a Fetching more stories. 2012-09-24 13:24:45 -07:00
Samuel Clay
ce52a329c2 Adding 1.5X as many stories to the dupe detector. Hope I don't regret this. 2012-09-21 13:03:38 -07:00
Samuel Clay
b92e564b1f Ignore skip of unread counts if theres no mongodb lag. 2012-09-10 17:50:36 -07:00
Samuel Clay
945b75ab36 Adding server name and feed code to analytics. 2012-09-06 22:31:44 -07:00
Samuel Clay
1511ca7fc3 Turning analytics logging back on. 2012-09-06 21:43:18 -07:00
Samuel Clay
88172988ee Temporarily commenting out analytics to see if they cure the slow down. 2012-09-06 20:56:44 -07:00
Samuel Clay
d7e70cafb0 Shortening names on analytics for improved space saving. 2012-09-06 17:39:18 -07:00
Samuel Clay
c07d7e7578 Adding new analytics collections to keep track of page loads and feed fetches. Need to start aggregating before I can tell how useful this data will be. 2012-09-06 17:24:42 -07:00
Samuel Clay
0fb8c8459d Cleaning up feed calculation in feed fetching. 2012-08-17 00:10:17 -07:00
Samuel Clay
2fa00fd737 Forcing the feed fetcher to not count scores when there's replica lag. God this had better work. 2012-08-16 23:56:21 -07:00
Samuel Clay
88a46dc3b4 Typo 2012-08-16 22:42:13 -07:00
Samuel Clay
92cdfaff66 Removing all removal of read stories, so that they stick around for blurblogs. No good answer for this one. 2012-08-16 22:40:54 -07:00
Samuel Clay
9cbb21c91d Moving Oldest-first date from mark_read_date to 2 weeks ago, everytime. 2012-08-01 21:57:55 -07:00
Samuel Clay
78338bba0d Tons of small fixes: opml uploads, rss feeds for blurblogs, default images on blurblogs. 2012-07-30 21:43:55 -07:00
Samuel Clay
15df6b6b47 Making the icon importer a bit faster and more slave lag resilient by using page fetch data. Also fixing title on blurblogs. 2012-07-30 11:12:58 -07:00
Samuel Clay
72892f0a2b Better watching for story changes. This is both a speed improvement and usability fix, since now stories that haven't changed recently but did change before will no longer be declared as updated every time they are fetched. Props to @textfiles for the push. 2012-07-22 12:25:09 -07:00
Samuel Clay
18428a32a1 Auto-linkifying comments and replies, and stripping html from comments, replies, and story titles, tags, and authors. 2012-07-21 16:38:37 -07:00
Samuel Clay
e40b93882a Fixing issue around feeds thinking they are unfetched. 2012-06-18 15:59:31 -07:00
Samuel Clay
69bf581cd2 Merge branch 'master' into social
* master:
  Adding story permalink to story context-menu. Thanks to @grammargirl for the idea.
  Adding task servers and updating automated deploy.
  Removing dupe unique index in prep for social.
  Adding 3 new task servers.

Conflicts:
	apps/analyzer/models.py
	fabfile.py
	media/js/newsblur/reader/reader.js
	utils/feed_fetcher.py
2012-05-17 13:27:29 -07:00
Samuel Clay
96cf1e2dec Adding task servers and updating automated deploy. 2012-05-17 12:43:13 -07:00
Samuel Clay
d9d7ed279a Merge branch 'master' into social
* master:
  Don't know how I feel about this change, but http://gizmodo.com/author/mat-honan.xml is failing when 'Safari/X' is in the user agent, but only with http opener. Ugh.

Conflicts:
	utils/feed_fetcher.py
2012-05-07 17:17:12 -07:00
Samuel Clay
852202899d Don't know how I feel about this change, but http://gizmodo.com/author/mat-honan.xml is failing when 'Safari/X' is in the user agent, but only with http opener. Ugh. 2012-05-07 17:16:32 -07:00
Samuel Clay
b5c5cdeda3 Merge branch 'master' into social
* master:
  Fixing icon fetching error when the page is corrupted.
  Writing a glorious hack that forces a count of unread scores.

Conflicts:
	apps/reader/models.py
	apps/rss_feeds/management/commands/refresh_feeds.py
	fabfile.py
	utils/feed_fetcher.py
2012-04-24 17:55:20 -07:00
Samuel Clay
c1b5aef983 Fixing icon fetching error when the page is corrupted. 2012-04-24 17:40:34 -07:00
Samuel Clay
60eceda244 Merge branch 'master' into social
* master:
  Trying to fix inaccurate unread counts by forcing the dirty flag and not allowing timeouts.
  Removing unused cache.

Conflicts:
	apps/reader/models.py
	utils/feed_fetcher.py
2012-04-24 16:36:38 -07:00
Samuel Clay
2474c987c5 Trying to fix inaccurate unread counts by forcing the dirty flag and not allowing timeouts. 2012-04-24 16:34:28 -07:00
Samuel Clay
20eeef3dc9 Removing unused cache. 2012-04-24 16:00:14 -07:00
Samuel Clay
4b26da4ad7 Merge branch 'master' into social
* master:
  Forcing a feed refresh now allows it to re-enable PuSH.
2012-04-17 10:45:26 -07:00
Samuel Clay
7e5931134b Forcing a feed refresh now allows it to re-enable PuSH. 2012-04-16 14:44:39 -07:00
Samuel Clay
3fd9e12b8a Merge branch 'master' into social
* master:
  Handling LookupError in requests.
  Adding locals to feed fetch errors.
2012-04-12 15:49:10 -07:00
Samuel Clay
fa558f6157 Adding locals to feed fetch errors. 2012-04-12 15:41:19 -07:00
Samuel Clay
c4731c8211 Merge branch 'master' into social
* master:
  Wordpress PuSH errors aren't really errors, so don't log them.
  Dealing with broken Wordpress PuSH plugin by subscribing to its requested topic url.
  Dealing with broken Wordpress PuSH plugin by subscribing to its requested topic url.
  Fixing Tumblr PuSH by sending the address instead of the link when subscribing.
  Subscribing to a topic or hub sets the urls.
  Launching real-time for premiums. Also fixing a few dupe feed refresh bugs.

Conflicts:
	apps/reader/views.py
	media/js/newsblur/common/assetmodel.js
2012-03-30 17:15:31 -07:00
Samuel Clay
f4c65c23d5 Fixing Tumblr PuSH by sending the address instead of the link when subscribing. 2012-03-30 16:36:16 -07:00
Samuel Clay
8efefc6cdb Merge branch 'master' into social
* master:
  Fixing hub url change subscription.
  Adding failed push feeds to munin.
  Showing correctly real-time status if feed is push.
  Showing correct real-time status if feed is push.
  Emergency fix to not have to reverse url in task fetchers.
  Setting up push feeds for removal.
  Adding munin graph for pushed feed queue and nginx conf.
  Logging real-time.
  Resubscribing to real-time feeds on add/remove feed.
  Adding push feeds queue. Ignoring fat ping, since it might not match URLs.
  Cleanup
  Forgive push subscription errors.
  Adding # of push feeds to munin.
  Correcting a double-encoding bug for story permalinks that was from way back when.
  Fixing the shit out of the feed fetcher's deleted feed handling. Also fixing PuSH support to correctly give parameters to PuSH server.
  Checking for push hub links safetly.
  Adding subscription callbacks for PuSH.
  Stubbing in PuSH support. Need to work it in with Feeds and upgrade rel='hub'.

Conflicts:
	apps/reader/models.py
	media/js/newsblur/reader/reader.js
	settings.py
	utils/feed_fetcher.py
2012-03-28 18:42:35 -07:00
Samuel Clay
d8a343b67b Adding munin graph for pushed feed queue and nginx conf. 2012-03-28 16:43:04 -07:00
Samuel Clay
4348e7f497 Resubscribing to real-time feeds on add/remove feed. 2012-03-28 16:13:17 -07:00
Samuel Clay
a74be1b604 Adding push feeds queue. Ignoring fat ping, since it might not match URLs. 2012-03-28 15:49:21 -07:00
Samuel Clay
494a89f060 Merge branch 'master' into social
* master:
  Adding task04 to fabfile.
  Updating Safari helper app.
  Goodbye StatHat. Hello better logging.

Conflicts:
	utils/feed_fetcher.py
2012-03-28 12:05:51 -07:00
Samuel Clay
9ee6487f58 Correcting a double-encoding bug for story permalinks that was from way back when. 2012-03-27 18:37:04 -07:00
Samuel Clay
fd1e26a5b0 Fixing the shit out of the feed fetcher's deleted feed handling. Also fixing PuSH support to correctly give parameters to PuSH server. 2012-03-27 17:34:39 -07:00
Samuel Clay
a745e654ba Checking for push hub links safetly. 2012-03-27 16:42:45 -07:00