Samuel Clay
36105bf851
Merge branch 'master' into feed_settings
...
* master:
Revert "Switching to requests from urllib2/httplib. Bring on the page errors."
Fixing #41 : Adding REDIS to local_settings.py.template.
Revert "Revert "Switching to requests from urllib2/httplib. Bring on the page errors.""
Revert "Switching to requests from urllib2/httplib. Bring on the page errors."
Switching to requests from urllib2/httplib. Bring on the page errors.
Adding a bunch fo keyboard shortcuts.
Adding elapsed time field to river and feed view.
Fixing autolinking of text links to no longer indiscriminately replace text, causing double encoding of html entities.
Fixing the space bar key to not queue, so it's actually useful now!
Refining media enclosures by adding all types and linking to their source.
Refining media enclosure detection.
2011-11-27 02:41:40 -05:00
Samuel Clay
a827b2c864
Revert "Switching to requests from urllib2/httplib. Bring on the page errors."
...
This reverts commit 617dc3bb56
2011-11-27 02:40:44 -05:00
Samuel Clay
d07b03a7a4
Revert "Revert "Switching to requests from urllib2/httplib. Bring on the page errors.""
...
This reverts commit 7972d0bb5c
2011-11-26 11:15:01 -05:00
Samuel Clay
7972d0bb5c
Revert "Switching to requests from urllib2/httplib. Bring on the page errors."
...
This reverts commit 617dc3bb56
2011-11-26 02:16:14 -05:00
Samuel Clay
617dc3bb56
Switching to requests from urllib2/httplib. Bring on the page errors.
2011-11-26 02:07:31 -05:00
Samuel Clay
221b356494
Refining media enclosures by adding all types and linking to their source.
2011-11-25 10:38:39 -05:00
Samuel Clay
3e68aff667
Merge branch 'master' into feed_settings
...
* master:
Correcting for feeds that have incorrect media content.
Adding media enclosures in an audio tag.
Using story date when there is no guid.
Using story date when there is no guid.
Adding verbose flag to feed updates to diagnose errors.
Fixing issue with stories that have no permalink.
Conflicts:
apps/rss_feeds/models.py
2011-11-25 00:18:32 -05:00
Samuel Clay
7ea2e01496
Adding verbose flag to feed updates to diagnose errors.
2011-11-24 15:29:26 -05:00
Samuel Clay
fb237e1049
Finishing up feed_link corrections. Just need feed_address (which is essentially a refactor and abstraction of this feed_link code), and then duplicate UI, which is pretty easy. Yay.
2011-11-16 18:41:36 -08:00
Samuel Clay
14ed618c6c
File under "fucking finally": Adding client-side error handling and callbacks for feeds and river.
2011-11-15 18:19:09 -08:00
Samuel Clay
3e1d9eb626
Brief cleanup.
2011-11-12 15:36:38 -08:00
Samuel Clay
54e42f6012
Adding bullshit user agent string because some sites are sniffing for browsers in order to serve the correct site.
2011-11-10 10:29:25 -08:00
Samuel Clay
ac3609bf7d
Merge branch 'master' into realtime
...
* master:
Adding 'E' as a keyboard shortcut for the Everything view.
Doubling the amount Space pages in a story.
Auto-linkify urls in stories.
Reading stories from slave db during feed update.
Conflicts:
settings.py
2011-11-06 15:28:37 -08:00
Samuel Clay
4691345d91
Preparing realtime for deployment.
2011-11-06 12:28:06 -08:00
Samuel Clay
07fa01ed9a
Adding server half of redis real-time unread count updates. That was the easy part.
2011-11-05 17:08:31 -07:00
Samuel Clay
040b755694
Reading stories from slave db during feed update.
2011-11-04 09:45:10 -07:00
Samuel Clay
acfd9e7614
Adding feed.has_page to weed out feeds with no original page. This should've been in 2 years ago.
2011-09-04 10:59:29 -07:00
Samuel Clay
47556ec303
Locking feed links for testing. This also helps future feed links that need locking. Probsbly should include feed_address_lock's as well.
2011-09-01 09:34:57 -07:00
Samuel Clay
8f5a4c612d
Updating rss_feeds tests for completeness. Updating README to include how to run test suite.
2011-09-01 09:11:29 -07:00
Samuel Clay
4f46b358db
Major optimization on mongo queries. No longer using .rewind(), which is actually re-running the damn query! Wrapping repetitive querysets in list() for performance.
2011-06-09 15:36:24 -04:00
Samuel Clay
06a46993d2
Backup mongo + psql. Removing ridiculous FeedUpdateHistory table.
2011-05-18 14:36:31 -04:00
Samuel Clay
a757b9f88f
Fixing a few feed fetching bugs.
2011-04-04 20:53:12 -04:00
Samuel Clay
bb4e81b278
Fixing a whole lot of feed fetch bugs.
2011-02-15 21:08:40 -05:00
Samuel Clay
b38288d2a5
Forcing refresh feed on master db during feed fetches.
2011-02-10 11:13:42 -05:00
Samuel Clay
4fd6a7cfe0
Fixing a metric-ton of bugs: Google Reader import on badly parsed xml, refreshing toplevel feeds loses their manage menu, retrying exception feeds now de-dupes and correctly updates on the page. Plus a bunch of other bugs.
2011-02-06 15:04:21 -05:00
Samuel Clay
58338b7aac
Fixing invalid feed taglines.
2011-02-05 22:27:47 -05:00
Samuel Clay
33e2ee0960
Length check on feed tagline.
2011-02-05 22:19:28 -05:00
Samuel Clay
ebad038920
Length check on feed tagline.
2011-02-05 22:15:03 -05:00
Samuel Clay
b7b0c769bf
Length check on feed tagline.
2011-02-05 22:11:17 -05:00
Samuel Clay
fc4d39da04
Length check on feed tagline.
2011-02-05 22:09:31 -05:00
Samuel Clay
466c172521
Fixing bug around missing titles in stories.
2011-02-05 21:33:24 -05:00
Samuel Clay
3723160155
A huge number of misc fixes: fixing potential errors on paypal return screen, mailing all feed/page parse errors, and swithcing to mapreduce item_frequencies in mongoengine.
2011-02-05 15:34:43 -05:00
Samuel Clay
b3a53fca0e
Limiting feed entries to 50.
2011-02-04 11:20:29 -05:00
Samuel Clay
22cf5ede6c
Crazy refactor of feedfinder to extract timelimits, so they can be caught by a lower-level api instead of messing up the feed fetcher.
2011-02-02 13:07:12 -05:00
Samuel Clay
c5d380ec3e
Adding timeout to icon fetching, and handling bad .ICOs
2011-01-30 12:53:05 -05:00
Samuel Clay
b37d57613e
Adapting icon_url, which comes from lxml, into unicode for the purposes of storing in the db.
2011-01-30 11:22:57 -05:00
Samuel Clay
75fead0a27
Cleaning up logging, adding several errors and timeout checks on page and icon fetching. Standardizing fetcher headers.
2011-01-29 22:01:09 -05:00
Samuel Clay
b938e95124
Correctly downloading, storing, retrieving, and showing favicons and gradients for favicons. This commit includes numerous fixes for the Feed view, too. Loaded.
2011-01-29 19:16:40 -05:00
Samuel Clay
a182bb1216
Finally saving proper base64 encoded favicons, along with their URLs, autodetecting when not found at /favicon.ico, and now all thats left is serving icons and their colors. Woohoo!
2011-01-27 19:05:50 -05:00
Samuel Clay
48e4670b5c
Fixing errors in feed fetching.
2011-01-26 17:47:10 -05:00
Samuel Clay
a22f650d56
Fixing errors in feed fetching.
2011-01-26 17:45:22 -05:00
Samuel Clay
443fe69156
Moving text fields from Feed to FeedData. Migration includes deletion of old fields on Feed, now unused.
2011-01-17 22:48:38 -05:00
Samuel Clay
7fdd173ac0
Refreshing stale feeds for old users returning.
2011-01-17 14:20:36 -05:00
Samuel Clay
463d2af239
Meaningless version change.
2011-01-13 22:54:54 -05:00
Samuel Clay
4f35c2fade
Making feed.min_to_decay reflect the actual decay time. Using it in statistics, and refreshing it in feed fetching. Also splitting up feed munin.
2010-12-23 13:29:31 -05:00
Samuel Clay
12891aabcf
Let's try using dates again in feed fetching existance checks.
2010-12-16 17:25:51 -05:00
Samuel Clay
6d8c34c752
Are story dates the problem?
2010-12-16 17:16:09 -05:00
Samuel Clay
ae4646e979
Nice middle ground between guids and dates. Now only retrieving as many guids are in the feed, but then comparing them with N stories. Very old stories shouldnt change, and they'll get thwarted by an IntegrityCheck.
2010-12-16 17:10:13 -05:00
Samuel Clay
5b3f199852
OK, a bit too aggressive on the story loading. Sticking to simple queries for now.
2010-12-16 17:00:08 -05:00
Samuel Clay
86af737159
Experimental feed difference change. More expensive, but will detect when things are actually changing.
2010-12-16 16:52:00 -05:00
