Samuel Clay
ac3609bf7d
Merge branch 'master' into realtime
...
* master:
Adding 'E' as a keyboard shortcut for the Everything view.
Doubling the amount Space pages in a story.
Auto-linkify urls in stories.
Reading stories from slave db during feed update.
Conflicts:
settings.py
2011-11-06 15:28:37 -08:00
Samuel Clay
4691345d91
Preparing realtime for deployment.
2011-11-06 12:28:06 -08:00
Samuel Clay
07fa01ed9a
Adding server half of redis real-time unread count updates. That was the easy part.
2011-11-05 17:08:31 -07:00
Samuel Clay
040b755694
Reading stories from slave db during feed update.
2011-11-04 09:45:10 -07:00
Samuel Clay
acfd9e7614
Adding feed.has_page to weed out feeds with no original page. This should've been in 2 years ago.
2011-09-04 10:59:29 -07:00
Samuel Clay
47556ec303
Locking feed links for testing. This also helps future feed links that need locking. Probsbly should include feed_address_lock's as well.
2011-09-01 09:34:57 -07:00
Samuel Clay
8f5a4c612d
Updating rss_feeds tests for completeness. Updating README to include how to run test suite.
2011-09-01 09:11:29 -07:00
Samuel Clay
4f46b358db
Major optimization on mongo queries. No longer using .rewind(), which is actually re-running the damn query! Wrapping repetitive querysets in list() for performance.
2011-06-09 15:36:24 -04:00
Samuel Clay
06a46993d2
Backup mongo + psql. Removing ridiculous FeedUpdateHistory table.
2011-05-18 14:36:31 -04:00
Samuel Clay
a757b9f88f
Fixing a few feed fetching bugs.
2011-04-04 20:53:12 -04:00
Samuel Clay
bb4e81b278
Fixing a whole lot of feed fetch bugs.
2011-02-15 21:08:40 -05:00
Samuel Clay
b38288d2a5
Forcing refresh feed on master db during feed fetches.
2011-02-10 11:13:42 -05:00
Samuel Clay
4fd6a7cfe0
Fixing a metric-ton of bugs: Google Reader import on badly parsed xml, refreshing toplevel feeds loses their manage menu, retrying exception feeds now de-dupes and correctly updates on the page. Plus a bunch of other bugs.
2011-02-06 15:04:21 -05:00
Samuel Clay
58338b7aac
Fixing invalid feed taglines.
2011-02-05 22:27:47 -05:00
Samuel Clay
33e2ee0960
Length check on feed tagline.
2011-02-05 22:19:28 -05:00
Samuel Clay
ebad038920
Length check on feed tagline.
2011-02-05 22:15:03 -05:00
Samuel Clay
b7b0c769bf
Length check on feed tagline.
2011-02-05 22:11:17 -05:00
Samuel Clay
fc4d39da04
Length check on feed tagline.
2011-02-05 22:09:31 -05:00
Samuel Clay
466c172521
Fixing bug around missing titles in stories.
2011-02-05 21:33:24 -05:00
Samuel Clay
3723160155
A huge number of misc fixes: fixing potential errors on paypal return screen, mailing all feed/page parse errors, and swithcing to mapreduce item_frequencies in mongoengine.
2011-02-05 15:34:43 -05:00
Samuel Clay
b3a53fca0e
Limiting feed entries to 50.
2011-02-04 11:20:29 -05:00
Samuel Clay
22cf5ede6c
Crazy refactor of feedfinder to extract timelimits, so they can be caught by a lower-level api instead of messing up the feed fetcher.
2011-02-02 13:07:12 -05:00
Samuel Clay
c5d380ec3e
Adding timeout to icon fetching, and handling bad .ICOs
2011-01-30 12:53:05 -05:00
Samuel Clay
b37d57613e
Adapting icon_url, which comes from lxml, into unicode for the purposes of storing in the db.
2011-01-30 11:22:57 -05:00
Samuel Clay
75fead0a27
Cleaning up logging, adding several errors and timeout checks on page and icon fetching. Standardizing fetcher headers.
2011-01-29 22:01:09 -05:00
Samuel Clay
b938e95124
Correctly downloading, storing, retrieving, and showing favicons and gradients for favicons. This commit includes numerous fixes for the Feed view, too. Loaded.
2011-01-29 19:16:40 -05:00
Samuel Clay
a182bb1216
Finally saving proper base64 encoded favicons, along with their URLs, autodetecting when not found at /favicon.ico, and now all thats left is serving icons and their colors. Woohoo!
2011-01-27 19:05:50 -05:00
Samuel Clay
48e4670b5c
Fixing errors in feed fetching.
2011-01-26 17:47:10 -05:00
Samuel Clay
a22f650d56
Fixing errors in feed fetching.
2011-01-26 17:45:22 -05:00
Samuel Clay
443fe69156
Moving text fields from Feed to FeedData. Migration includes deletion of old fields on Feed, now unused.
2011-01-17 22:48:38 -05:00
Samuel Clay
7fdd173ac0
Refreshing stale feeds for old users returning.
2011-01-17 14:20:36 -05:00
Samuel Clay
463d2af239
Meaningless version change.
2011-01-13 22:54:54 -05:00
Samuel Clay
4f35c2fade
Making feed.min_to_decay reflect the actual decay time. Using it in statistics, and refreshing it in feed fetching. Also splitting up feed munin.
2010-12-23 13:29:31 -05:00
Samuel Clay
12891aabcf
Let's try using dates again in feed fetching existance checks.
2010-12-16 17:25:51 -05:00
Samuel Clay
6d8c34c752
Are story dates the problem?
2010-12-16 17:16:09 -05:00
Samuel Clay
ae4646e979
Nice middle ground between guids and dates. Now only retrieving as many guids are in the feed, but then comparing them with N stories. Very old stories shouldnt change, and they'll get thwarted by an IntegrityCheck.
2010-12-16 17:10:13 -05:00
Samuel Clay
5b3f199852
OK, a bit too aggressive on the story loading. Sticking to simple queries for now.
2010-12-16 17:00:08 -05:00
Samuel Clay
86af737159
Experimental feed difference change. More expensive, but will detect when things are actually changing.
2010-12-16 16:52:00 -05:00
Samuel Clay
144b04feb6
Minor fix to feed fetching -- retry a feed with a bad HTTP status code but an OK feed link which points to a new feed address.
2010-12-10 09:32:06 -05:00
Samuel Clay
37028cc5ed
Forcing unicode string on formatted log exception.
2010-12-07 23:51:58 -05:00
Samuel Clay
c5895a7065
Forcing unicode string on formatted log exception.
2010-12-07 23:45:08 -05:00
Samuel Clay
d25c2dd65f
Only calculate unread scores for active feeds and active users.
2010-11-10 18:22:33 -05:00
Samuel Clay
fa663e910a
Faster feed refreshing (insta-fetching) by only computing the score for the current user and leaving the other users with dirty subscriptions.
2010-11-10 18:04:17 -05:00
Samuel Clay
e7646d7e98
Adding change password to Preferences. Also changing timing for feed updates.
2010-11-08 12:09:55 -05:00
Samuel Clay
8c373c336b
Deleting old read stories.
2010-11-08 11:30:06 -05:00
Samuel Clay
afa9e53a4c
Forcing refresh of feed model at opportune times in the feed fetcher.
2010-11-05 23:36:00 -04:00
Samuel Clay
7ca9b2bee5
Undoing connection close on feed fetcher. If they are being tasked, they don't need to close the connection.
2010-11-05 22:12:35 -04:00
Samuel Clay
b30752a9c1
A huge slew of bug fixes related to feed fetching, missing ids, and missing feeds.
2010-11-05 20:34:17 -04:00
Samuel Clay
80fa9aa6fd
Cutting unread count timeout cutoff by a half.
2010-11-05 15:06:54 -04:00
Samuel Clay
71a32b3120
Refactoring feed unread calculation to allow for timeouts. Now counts in order of last read date for subscriptions.
2010-11-05 12:53:02 -04:00
