Commit graph

408 commits

Author SHA1 Message Date
Samuel Clay
c9f978d8b1 Merge branch 'master' into social
* master:
  Adding tests in prep for Mother Jones duping.
  Fixing rss feed tests.
  Adding press story from Italian site Skimbu.
  Adding Cuk.ch story about NewsBlur.
2012-03-06 16:56:34 -08:00
Samuel Clay
d5926fe577 Fixing rss feed tests. 2012-03-06 16:11:27 -08:00
Samuel Clay
0e43583a6c Merge branch 'master' into social
* master:
  Fixing issue around feeds that are mistakenly declared as not having a page.
  New audio control, with Flash fallback for browsers like Firefox/IE.

Conflicts:
	assets.yml
2012-03-05 13:48:39 -08:00
Samuel Clay
33fa13cf69 Fixing issue around feeds that are mistakenly declared as not having a page. 2012-03-05 13:12:50 -08:00
Samuel Clay
aaa5aa1cd6 Merge branch 'master' into social
* master:
  Changing copy on premium/feedchooser dialog.
  Styling errors on stripe payment form.
  Final stripe.js tweaks before launch.
  Adding Pay by Credit Card as an option to feed chooser dialog. Thus concludes Stripe.js integration. Time to launch!
  Styling the stripe.js background.
  Using correct kwargs for stripe signal.
  Wrapping up stripe.js form. Has validation, styling, and does the right thing for new subscriptions. Needs a link to the form, though.
  Setting up correct customer model for stripe webhook.
  Moving JS assets to bottom of the page for faster loadtimes.
  Fixing exception on missing param in feed address searching.
  Fixing recommendation date serialization bug.
  Fixing bugs around login with blank password using full password. Also fixing bug in signups with no username.
  Stripe.js payments using zebra. Adding zebra as a vendored dependency. Webhooks and views all in. Needs styling, custom username and email fields, and loads of testing.
  Adding error checking on requests in page fetching.
  Using a probability from redis to determine whether or not to skip a fetch.
  Allowing any password to be used on accounts with no password set.
  Adding paypal logo.

Conflicts:
	assets.yml
	media/js/newsblur/reader/reader_feedchooser.js
	settings.py
	templates/base.html
2012-02-29 16:22:47 -08:00
Samuel Clay
936088f09c Allowing any password to be used on accounts with no password set. 2012-02-27 16:36:39 -08:00
Samuel Clay
603ac08faa Merge branch 'master' into social
* master:
  Allow login from signup form.
  Reducing celery prefetch to get more accurate queue sizes.
  Removing RabbitMQ dependency. Moving to redis for task queuing.
  Counting queue size for celery/redis.
  Switching to redis to feed queuing. Also adding a weight for skipping feed fetches under high load.
  Skipping feed resurrection.
  Fixing last update problem that has been choking feed fetchers.
  Fixing last update problem that has been choking feed fetchers.
  Typo in feed pool drain.
  Adding ability to wind down feed queue by faking requests.
  Adding debug message about counting errors.

Conflicts:
	fabfile.py
2012-02-27 13:13:27 -08:00
Samuel Clay
d901ccdae1 Switching to redis to feed queuing. Also adding a weight for skipping feed fetches under high load. 2012-02-26 22:33:06 -08:00
Samuel Clay
ff00e16682 Fixing last update problem that has been choking feed fetchers. 2012-02-24 17:00:55 -08:00
Samuel Clay
bdf84cb64c Typo in feed pool drain. 2012-02-24 16:48:58 -08:00
Samuel Clay
ce536200f6 Adding ability to wind down feed queue by faking requests. 2012-02-24 16:43:08 -08:00
Samuel Clay
408cb7e801 Merge branch 'master' into social
* master:
  Fixing starred stories from not loading past the second page due to a missing flag reset.
  Fixing user-found bug around tooltips continuing to show if the original dom element (and the reference to the tooltip) are destroyed too early, leaving a tooltip floating in the air.
  Geometrically delaying the fetching of bad feeds, while they're still somewhat fresh.
  Fixing error codes on statistics. Also fixing simple error on invalid feed.
  Update media/js/newsblur/reader.js

Conflicts:
	media/js/newsblur/reader/reader.js
2012-02-24 13:01:48 -08:00
Samuel Clay
1ffefaa419 Geometrically delaying the fetching of bad feeds, while they're still somewhat fresh. 2012-02-24 11:47:38 -08:00
Samuel Clay
307e36622b Merge branch 'master' into social
* master:
  Adding asset-less deployment.
  Upgrading feedparser from 5.0.1 to 5.1.
  Catching errors in page links with ill-formed urls.
  Not showing the world favicon until favicons are downloaded.
  Fixing feed exception dialog to actually reload feeds on the page.
  Geometrically decaying feed fetches on known good feeds that have gone bad.
  Fixing issue around empty original pages.

Conflicts:
	media/js/newsblur/reader/reader.js
2012-02-23 15:40:38 -08:00
Samuel Clay
6c7034343b Geometrically decaying feed fetches on known good feeds that have gone bad. 2012-02-23 13:20:10 -08:00
Samuel Clay
b0006d289e Handling turned off redis server (in case redis goes down), and fixing up the /m/ mobile site. 2012-02-21 17:35:51 -08:00
Samuel Clay
b3fffab05a Merge branch 'master' into social
* master:
  Adding build/ back to .gitignore.
  Updating .gitignore.
  Using dsa pem for retrieving rabbit update counts.
  Preserving feed fetch error histories for 2 weeks.
  Adding munin graph for known good feeds.
  Turning down verbosity on feed tasking.
  Adding known good feeds to munin to chart their progress.
  Fixing headers overflow in Chrome.
  Adding new feeds db column 'known_good', for forcing updates of feeds that were once good but are now throwing 500s.
  Adding blogging OT press story.
2012-02-02 17:22:32 -08:00
Samuel Clay
dc986a0a3e Preserving feed fetch error histories for 2 weeks. 2012-02-02 07:46:04 -08:00
Samuel Clay
c9cb7dcbc0 Adding new feeds db column 'known_good', for forcing updates of feeds that were once good but are now throwing 500s. 2012-02-01 17:59:46 -08:00
Samuel Clay
2b9ad97b51 Switching a ton of feed and user fetches to simple ID matches. ".user.pk -> .user_id" 2012-01-26 09:32:24 -08:00
Samuel Clay
619f03f610 Fixing typo in aggressive feed fetching. 2012-01-23 09:12:38 -08:00
Samuel Clay
c37bcdad63 No longer being aggressive about finding a feed's correct address if it returns a 400 or 500 error code, unless it is still the first run or an exception refresh. 2012-01-23 09:08:55 -08:00
Samuel Clay
ccc8878e88 Colorizing feed fetching. 2012-01-04 18:47:40 -08:00
Samuel Clay
b0aa1764b4 Colorizing feed fetching. 2012-01-04 18:30:43 -08:00
Samuel Clay
10ef120e34 Merge branch 'master' into requests
* master:
  Abort old requests and don't cache responses. Trying to fix IE read story bug.
  Force jquery to never cache.
  Checking for exceptions earlier in order to show fixed site.
  Changing user agent to hopefully override overly-sniffy websites.
  Fixing inaccurate substitution on feed urls. Ugh, can't believe this was here this long.
  Adding the setting of a feed's view settings to Site Settings dialog. This should've been in there a loooong time ago.
2012-01-02 16:00:09 -08:00
Samuel Clay
4fd9965949 Changing user agent to hopefully override overly-sniffy websites. 2012-01-02 14:31:50 -08:00
Samuel Clay
6637c05adf Adding a little bit more leeway in page importer by falling back to urllib2 when requests fail. 2011-12-20 22:30:55 -08:00
Samuel Clay
0418538381 Adding slight bit of color to feed fetcher. 2011-12-20 22:02:52 -08:00
Samuel Clay
b5efad9142 Merge branch 'master' into requests
* master: (66 commits)
  Too many stories? Don't animate transitions when switching intelligence levels.
  Fixing typo in iphone app that preventing mark folder as read when the visible stories option was showing.
  Updating the mongo db copy util to also look for updated stories.
  Globals.is_staff on user, not profile. Make the stats referesh every minute instead of every 10 minutes for staff.
  Fixing offsets in river of news.
  Adding ciphering to usernames in log, so I can finally take screenshots of the most colorful logs of all time.
  Adding my very special mongo serialization backup utility to get around the damn unrepairable mongo database. This is taking 14 hours to run.
  Auto-refreshing feedback (1 min for staff, 10 min for everybody).
  Fixing exception around multiple feeds during Google Reader import process. Also switching rate limit to status code 429.
  Adding Nokia MeeGo client to user agents.
  Only show raw feeds in feed autocomplete.
  Handling iphone version for new users on ios app.
  FIXING THE WORST BUG OF MY LIFE -- finally figured out what was causing the story-shows-as-unread bug. Also fixed enclosures on certain types of feeds.
  Fixing menu manage open position to compensate for additional menu items.
  Reducing the amount of work done by feed fetching when there are no new stories.
  Fixing emergency bug around trimming feeds where the cursor is changing. Dammit mongo.
  Simplifying ufw ports in fabfile. Adding env.user.
  Launching the iPhone app on the front-page. Big deal.
  Minor cleanup of river stories view.
  Cleaning up mongoengine imports and settings for default MongoDB.
  ...

Conflicts:
	local_settings.py.template
2011-12-20 21:47:55 -08:00
Samuel Clay
d1b0552942 Reducing the amount of work done by feed fetching when there are no new stories. 2011-12-14 14:18:01 -08:00
Samuel Clay
1828d949d8 Killing off the old slave_db pattern. 2011-12-11 11:06:42 -08:00
Samuel Clay
85feadb3f1 Fixing the crap out of the move feed/folder dialog in iOS app. Completely done with this dialog. Yay! 2011-12-04 13:55:57 -08:00
Samuel Clay
f8f2e6ba3a Merge branch 'master' into requests
* master:
  Fixing bogus type error, caused by duplicate feeds.
  Correcting styling on site settings loader icon.
  Removing dupe migration from feed_settings.
  Adding migration for unique hashes on feed addresses + links.
  Counting subscribers on branched feeds correctly, so as to maintain update frequency, even though it's branched.
  Hooking up feed address change nd branching. Also fixing those infernal feed address change bugs for good.
  Finishing up feed_link corrections. Just need feed_address (which is essentially a refactor and abstraction of this feed_link code), and then duplicate UI, which is pretty easy. Yay.
  Refactoring feed rewriting to take advantage of new feed link changer. Seems to work. Still needs some better duplicate locking during fetch.
  Uniqifying address+link hash. Allowing feed_link branches, but it does not yet work. Needs proper dupe detection.
  Beginning new branch feed logic. Needs a huge db migration.
  Swapping titles on feed settings when necessary.
  Swapping rss and website url.
  Stubbing out entire feed settings modal. Needs duplicate feed list. Also needs backend splitting of subscriptions.
  Adding feed setting menu item. Cleaning up feed exception dialog.
2011-11-27 03:19:00 -05:00
Samuel Clay
36105bf851 Merge branch 'master' into feed_settings
* master:
  Revert "Switching to requests from urllib2/httplib. Bring on the page errors."
  Fixing #41: Adding REDIS to local_settings.py.template.
  Revert "Revert "Switching to requests from urllib2/httplib. Bring on the page errors.""
  Revert "Switching to requests from urllib2/httplib. Bring on the page errors."
  Switching to requests from urllib2/httplib. Bring on the page errors.
  Adding a bunch fo keyboard shortcuts.
  Adding elapsed time field to river and feed view.
  Fixing autolinking of text links to no longer indiscriminately replace text, causing double encoding of html entities.
  Fixing the space bar key to not queue, so it's actually useful now!
  Refining media enclosures by adding all types and linking to their source.
  Refining media enclosure detection.
2011-11-27 02:41:40 -05:00
Samuel Clay
b112717d95 Revert "Revert "Switching to requests from urllib2/httplib. Bring on the page errors.""
This reverts commit a827b2c864.
2011-11-27 02:41:12 -05:00
Samuel Clay
a827b2c864 Revert "Switching to requests from urllib2/httplib. Bring on the page errors."
This reverts commit 617dc3bb56.

Conflicts:

	local_settings.py.template
2011-11-27 02:40:44 -05:00
Samuel Clay
d07b03a7a4 Revert "Revert "Switching to requests from urllib2/httplib. Bring on the page errors.""
This reverts commit 7972d0bb5c.
2011-11-26 11:15:01 -05:00
Samuel Clay
7972d0bb5c Revert "Switching to requests from urllib2/httplib. Bring on the page errors."
This reverts commit 617dc3bb56.
2011-11-26 02:16:14 -05:00
Samuel Clay
617dc3bb56 Switching to requests from urllib2/httplib. Bring on the page errors. 2011-11-26 02:07:31 -05:00
Samuel Clay
221b356494 Refining media enclosures by adding all types and linking to their source. 2011-11-25 10:38:39 -05:00
Samuel Clay
3e68aff667 Merge branch 'master' into feed_settings
* master:
  Correcting for feeds that have incorrect media content.
  Adding media enclosures in an audio tag.
  Using story date when there is no guid.
  Using story date when there is no guid.
  Adding verbose flag to feed updates to diagnose errors.
  Fixing issue with stories that have no permalink.

Conflicts:
	apps/rss_feeds/models.py
2011-11-25 00:18:32 -05:00
Samuel Clay
7ea2e01496 Adding verbose flag to feed updates to diagnose errors. 2011-11-24 15:29:26 -05:00
Samuel Clay
fb237e1049 Finishing up feed_link corrections. Just need feed_address (which is essentially a refactor and abstraction of this feed_link code), and then duplicate UI, which is pretty easy. Yay. 2011-11-16 18:41:36 -08:00
Samuel Clay
14ed618c6c File under "fucking finally": Adding client-side error handling and callbacks for feeds and river. 2011-11-15 18:19:09 -08:00
Samuel Clay
3e1d9eb626 Brief cleanup. 2011-11-12 15:36:38 -08:00
Samuel Clay
54e42f6012 Adding bullshit user agent string because some sites are sniffing for browsers in order to serve the correct site. 2011-11-10 10:29:25 -08:00
Samuel Clay
ac3609bf7d Merge branch 'master' into realtime
* master:
  Adding 'E' as a keyboard shortcut for the Everything view.
  Doubling the amount Space pages in a story.
  Auto-linkify urls in stories.
  Reading stories from slave db during feed update.

Conflicts:
	settings.py
2011-11-06 15:28:37 -08:00
Samuel Clay
4691345d91 Preparing realtime for deployment. 2011-11-06 12:28:06 -08:00
Samuel Clay
07fa01ed9a Adding server half of redis real-time unread count updates. That was the easy part. 2011-11-05 17:08:31 -07:00
Samuel Clay
040b755694 Reading stories from slave db during feed update. 2011-11-04 09:45:10 -07:00