Commit graph

329 commits

Author SHA1 Message Date
Samuel Clay
abc3b20d81 Adding subscription callbacks for PuSH. 2012-03-27 16:26:07 -07:00
Samuel Clay
daef8f3841 Stubbing in PuSH support. Need to work it in with Feeds and upgrade rel='hub'. 2012-03-27 14:58:13 -07:00
Samuel Clay
1788d26374 Goodbye StatHat. Hello better logging. 2012-03-27 11:19:53 -07:00
Samuel Clay
e0d6454a5f Merge branch 'master' into social
* master:
  Fixing missing status code in feed fetcher.
  Fixing missing status code in feed fetcher.
  Adding stathat stats for feed fetching and parsing.
  Small cleanup to parameters of feed update method.
  Refactoring starred stories indexes and adding hints.
  Removing hints.
  Adding hints, revising indexes on user stories, and trying to quicken feed fetching meta work.
  Refactoring refresh feed unread counts. Fixing tests.
  Adding task04 to fabfile.
  Adding django 1.4 compatibility.
  If feed no longer exists after update, find it, then schedule fetch.
  Don't skip unfetched feeds.
  Don't skip unfetched feeds.

Conflicts:
	apps/reader/models.py
	apps/reader/views.py
	fabfile.py
	utils/feed_fetcher.py
2012-03-26 21:06:13 -07:00
Samuel Clay
bdaa2f7e81 Fixing missing status code in feed fetcher. 2012-03-26 17:42:04 -07:00
Samuel Clay
6cc53f42c7 Fixing missing status code in feed fetcher. 2012-03-26 17:24:57 -07:00
Samuel Clay
eec1563549 Adding stathat stats for feed fetching and parsing. 2012-03-26 17:21:49 -07:00
Samuel Clay
5e981635d1 Small cleanup to parameters of feed update method. 2012-03-26 17:04:35 -07:00
Samuel Clay
bf00d6a501 Merge branch 'master' of github.com:samuelclay/NewsBlur
* 'master' of github.com:samuelclay/NewsBlur:
  If feed no longer exists after update, find it, then schedule fetch.
  Don't skip unfetched feeds.
  Don't skip unfetched feeds.
2012-03-26 12:40:24 -07:00
Samuel Clay
1321ade17a Adding hints, revising indexes on user stories, and trying to quicken feed fetching meta work. 2012-03-26 12:40:13 -07:00
Samuel Clay
7313339dbc Refactoring refresh feed unread counts. Fixing tests. 2012-03-26 11:04:05 -07:00
Samuel Clay
0e767e65c3 Don't skip unfetched feeds. 2012-03-24 10:51:04 -07:00
Samuel Clay
3e6329b3e5 Don't skip unfetched feeds. 2012-03-24 10:50:23 -07:00
Samuel Clay
929366221e Merge branch 'master' of github.com:samuelclay/NewsBlur into social
* 'master' of github.com:samuelclay/NewsBlur:
  Fixing bug around missing feed value when skipping fetch (and not updating last_update date).
  Handling zlib/deflate issues, specifically for The Verge.
2012-03-22 13:42:09 -07:00
Samuel Clay
81ccad2942 Fixing bug around missing feed value when skipping fetch (and not updating last_update date). 2012-03-22 10:39:24 -07:00
Samuel Clay
3e83186254 Merge branch 'master' into social
* master:
  Missing last update date.
  Fixing major bug around stale data when updating feeds.
2012-03-21 17:12:20 -07:00
Samuel Clay
d74139e2ed Fixing major bug around stale data when updating feeds. 2012-03-21 16:05:52 -07:00
Samuel Clay
67ea2452a7 Uniqifying profiles. Small cleanup. 2012-03-20 16:46:38 -07:00
Samuel Clay
5b71327531 Merge branch 'master' into social
* master:
  Error handling in orphan detection.
  Collecting orphaned feeds for users. These somehow got lost, and I'm not sure how. Eventually I'd like to run this on every user account.
  Fixing issue around feed titles that disappear. This just confirms that the new feed title is substantive.
  Uncommenting Readability sharing. This was written a loooong time ago, but their API is finally ready.
  Fixing an issue around mal-formed requests aginst WordPress's Bad Behavior plugin, which checks for the 'Accept: *' header.
  Fixing missing folder issue in iOS app. Occurs when user has no feeds in top level, when the Everything psuedo-folder obscures the top folder.

Conflicts:
	apps/reader/models.py
2012-03-20 10:33:15 -07:00
Samuel Clay
3a87a7c418 Fixing issue around feed titles that disappear. This just confirms that the new feed title is substantive. 2012-03-19 20:47:06 -07:00
Samuel Clay
2200aa282a Subscribe to blog during intro. 2012-03-19 15:46:59 -07:00
Samuel Clay
c9f978d8b1 Merge branch 'master' into social
* master:
  Adding tests in prep for Mother Jones duping.
  Fixing rss feed tests.
  Adding press story from Italian site Skimbu.
  Adding Cuk.ch story about NewsBlur.
2012-03-06 16:56:34 -08:00
Samuel Clay
d5926fe577 Fixing rss feed tests. 2012-03-06 16:11:27 -08:00
Samuel Clay
0e43583a6c Merge branch 'master' into social
* master:
  Fixing issue around feeds that are mistakenly declared as not having a page.
  New audio control, with Flash fallback for browsers like Firefox/IE.

Conflicts:
	assets.yml
2012-03-05 13:48:39 -08:00
Samuel Clay
33fa13cf69 Fixing issue around feeds that are mistakenly declared as not having a page. 2012-03-05 13:12:50 -08:00
Samuel Clay
aaa5aa1cd6 Merge branch 'master' into social
* master:
  Changing copy on premium/feedchooser dialog.
  Styling errors on stripe payment form.
  Final stripe.js tweaks before launch.
  Adding Pay by Credit Card as an option to feed chooser dialog. Thus concludes Stripe.js integration. Time to launch!
  Styling the stripe.js background.
  Using correct kwargs for stripe signal.
  Wrapping up stripe.js form. Has validation, styling, and does the right thing for new subscriptions. Needs a link to the form, though.
  Setting up correct customer model for stripe webhook.
  Moving JS assets to bottom of the page for faster loadtimes.
  Fixing exception on missing param in feed address searching.
  Fixing recommendation date serialization bug.
  Fixing bugs around login with blank password using full password. Also fixing bug in signups with no username.
  Stripe.js payments using zebra. Adding zebra as a vendored dependency. Webhooks and views all in. Needs styling, custom username and email fields, and loads of testing.
  Adding error checking on requests in page fetching.
  Using a probability from redis to determine whether or not to skip a fetch.
  Allowing any password to be used on accounts with no password set.
  Adding paypal logo.

Conflicts:
	assets.yml
	media/js/newsblur/reader/reader_feedchooser.js
	settings.py
	templates/base.html
2012-02-29 16:22:47 -08:00
Samuel Clay
936088f09c Allowing any password to be used on accounts with no password set. 2012-02-27 16:36:39 -08:00
Samuel Clay
603ac08faa Merge branch 'master' into social
* master:
  Allow login from signup form.
  Reducing celery prefetch to get more accurate queue sizes.
  Removing RabbitMQ dependency. Moving to redis for task queuing.
  Counting queue size for celery/redis.
  Switching to redis to feed queuing. Also adding a weight for skipping feed fetches under high load.
  Skipping feed resurrection.
  Fixing last update problem that has been choking feed fetchers.
  Fixing last update problem that has been choking feed fetchers.
  Typo in feed pool drain.
  Adding ability to wind down feed queue by faking requests.
  Adding debug message about counting errors.

Conflicts:
	fabfile.py
2012-02-27 13:13:27 -08:00
Samuel Clay
d901ccdae1 Switching to redis to feed queuing. Also adding a weight for skipping feed fetches under high load. 2012-02-26 22:33:06 -08:00
Samuel Clay
ff00e16682 Fixing last update problem that has been choking feed fetchers. 2012-02-24 17:00:55 -08:00
Samuel Clay
bdf84cb64c Typo in feed pool drain. 2012-02-24 16:48:58 -08:00
Samuel Clay
ce536200f6 Adding ability to wind down feed queue by faking requests. 2012-02-24 16:43:08 -08:00
Samuel Clay
408cb7e801 Merge branch 'master' into social
* master:
  Fixing starred stories from not loading past the second page due to a missing flag reset.
  Fixing user-found bug around tooltips continuing to show if the original dom element (and the reference to the tooltip) are destroyed too early, leaving a tooltip floating in the air.
  Geometrically delaying the fetching of bad feeds, while they're still somewhat fresh.
  Fixing error codes on statistics. Also fixing simple error on invalid feed.
  Update media/js/newsblur/reader.js

Conflicts:
	media/js/newsblur/reader/reader.js
2012-02-24 13:01:48 -08:00
Samuel Clay
1ffefaa419 Geometrically delaying the fetching of bad feeds, while they're still somewhat fresh. 2012-02-24 11:47:38 -08:00
Samuel Clay
307e36622b Merge branch 'master' into social
* master:
  Adding asset-less deployment.
  Upgrading feedparser from 5.0.1 to 5.1.
  Catching errors in page links with ill-formed urls.
  Not showing the world favicon until favicons are downloaded.
  Fixing feed exception dialog to actually reload feeds on the page.
  Geometrically decaying feed fetches on known good feeds that have gone bad.
  Fixing issue around empty original pages.

Conflicts:
	media/js/newsblur/reader/reader.js
2012-02-23 15:40:38 -08:00
Samuel Clay
6c7034343b Geometrically decaying feed fetches on known good feeds that have gone bad. 2012-02-23 13:20:10 -08:00
Samuel Clay
b0006d289e Handling turned off redis server (in case redis goes down), and fixing up the /m/ mobile site. 2012-02-21 17:35:51 -08:00
Samuel Clay
b3fffab05a Merge branch 'master' into social
* master:
  Adding build/ back to .gitignore.
  Updating .gitignore.
  Using dsa pem for retrieving rabbit update counts.
  Preserving feed fetch error histories for 2 weeks.
  Adding munin graph for known good feeds.
  Turning down verbosity on feed tasking.
  Adding known good feeds to munin to chart their progress.
  Fixing headers overflow in Chrome.
  Adding new feeds db column 'known_good', for forcing updates of feeds that were once good but are now throwing 500s.
  Adding blogging OT press story.
2012-02-02 17:22:32 -08:00
Samuel Clay
dc986a0a3e Preserving feed fetch error histories for 2 weeks. 2012-02-02 07:46:04 -08:00
Samuel Clay
c9cb7dcbc0 Adding new feeds db column 'known_good', for forcing updates of feeds that were once good but are now throwing 500s. 2012-02-01 17:59:46 -08:00
Samuel Clay
2b9ad97b51 Switching a ton of feed and user fetches to simple ID matches. ".user.pk -> .user_id" 2012-01-26 09:32:24 -08:00
Samuel Clay
619f03f610 Fixing typo in aggressive feed fetching. 2012-01-23 09:12:38 -08:00
Samuel Clay
c37bcdad63 No longer being aggressive about finding a feed's correct address if it returns a 400 or 500 error code, unless it is still the first run or an exception refresh. 2012-01-23 09:08:55 -08:00
Samuel Clay
ccc8878e88 Colorizing feed fetching. 2012-01-04 18:47:40 -08:00
Samuel Clay
b0aa1764b4 Colorizing feed fetching. 2012-01-04 18:30:43 -08:00
Samuel Clay
10ef120e34 Merge branch 'master' into requests
* master:
  Abort old requests and don't cache responses. Trying to fix IE read story bug.
  Force jquery to never cache.
  Checking for exceptions earlier in order to show fixed site.
  Changing user agent to hopefully override overly-sniffy websites.
  Fixing inaccurate substitution on feed urls. Ugh, can't believe this was here this long.
  Adding the setting of a feed's view settings to Site Settings dialog. This should've been in there a loooong time ago.
2012-01-02 16:00:09 -08:00
Samuel Clay
4fd9965949 Changing user agent to hopefully override overly-sniffy websites. 2012-01-02 14:31:50 -08:00
Samuel Clay
6637c05adf Adding a little bit more leeway in page importer by falling back to urllib2 when requests fail. 2011-12-20 22:30:55 -08:00
Samuel Clay
0418538381 Adding slight bit of color to feed fetcher. 2011-12-20 22:02:52 -08:00
Samuel Clay
b5efad9142 Merge branch 'master' into requests
* master: (66 commits)
  Too many stories? Don't animate transitions when switching intelligence levels.
  Fixing typo in iphone app that preventing mark folder as read when the visible stories option was showing.
  Updating the mongo db copy util to also look for updated stories.
  Globals.is_staff on user, not profile. Make the stats referesh every minute instead of every 10 minutes for staff.
  Fixing offsets in river of news.
  Adding ciphering to usernames in log, so I can finally take screenshots of the most colorful logs of all time.
  Adding my very special mongo serialization backup utility to get around the damn unrepairable mongo database. This is taking 14 hours to run.
  Auto-refreshing feedback (1 min for staff, 10 min for everybody).
  Fixing exception around multiple feeds during Google Reader import process. Also switching rate limit to status code 429.
  Adding Nokia MeeGo client to user agents.
  Only show raw feeds in feed autocomplete.
  Handling iphone version for new users on ios app.
  FIXING THE WORST BUG OF MY LIFE -- finally figured out what was causing the story-shows-as-unread bug. Also fixed enclosures on certain types of feeds.
  Fixing menu manage open position to compensate for additional menu items.
  Reducing the amount of work done by feed fetching when there are no new stories.
  Fixing emergency bug around trimming feeds where the cursor is changing. Dammit mongo.
  Simplifying ufw ports in fabfile. Adding env.user.
  Launching the iPhone app on the front-page. Big deal.
  Minor cleanup of river stories view.
  Cleaning up mongoengine imports and settings for default MongoDB.
  ...

Conflicts:
	local_settings.py.template
2011-12-20 21:47:55 -08:00