Commit graph

114 commits

Author SHA1 Message Date
Samuel Clay
013cf7ab04 I wrote all this code but have no idea why I didn't commit it. Anyway, let's deploy it. (It looks like I'm slowing down feed refreshing, which is bound to not matter in ~10 users or so). 2010-07-05 14:26:35 -04:00
Samuel Clay
9867406ca2 Refactoring feed counts, subscriber counts, story counts, and tightening up refreshes for high-volume, high-subscriber feeds. Less decay on somewhat popular feeds. 2010-07-02 15:49:08 -04:00
Samuel Clay
2133ee81e7 Fixing issues with feature paging. JSON was incorrectly serializing it, the page_counter was being reset when feeds were loaded, and cleaned up feed resetting. 2010-07-02 12:02:10 -04:00
Samuel Clay
aaf68f4c21 Fetching page for high-volume feeds on every fetch instead of only when new stories are in the feed. 2010-07-02 11:46:45 -04:00
Samuel Clay
5b1bb231ef Removing unnecessary cruft above the page importer. 2010-07-01 17:33:58 -04:00
Samuel Clay
4b57a32c0c Removing 2 bigger JOINs that were slowing down feed loading. 2010-07-01 15:16:33 -04:00
Samuel Clay
113a4e058c Adding a Delete This Site manage menu item, complete with cancel button. 2010-07-01 00:29:26 -04:00
Samuel Clay
80bdb79bd9 Better instrumentation through non-zero results on any successfull callback. 2010-06-28 08:06:12 -04:00
Samuel Clay
3d4cf207d3 Adding a feed punishment for slow feeds. Slow feeds get updated less often. 2010-06-24 15:27:25 -04:00
Samuel Clay
cc8beb2379 Fixing caching of feed stories so they load faster. 2010-06-24 15:10:15 -04:00
Samuel Clay
fbe1367739 Protopub -> NewsBlur 2010-06-08 11:19:07 -04:00
Samuel Clay
d39dacb592 Cleanup in prep for move from urllib to eventlets. 2010-05-20 15:13:25 -04:00
Samuel Clay
3ca7fcefca That mass renames continue! Protoread is now Protopub, and that sounds good to me. 2010-05-14 09:32:38 -04:00
Samuel Clay
b1ac2147ba Renaming News Blur to Protoread. 2010-05-13 11:11:01 -04:00
Samuel Clay
0176308858 Stylistic changes, like the name of the site: NewsBlur is now News Blur. 2010-05-11 18:38:32 -04:00
Samuel Clay
ce0495002e Adding load time to feeds to punish slow feeds. 2010-04-29 13:35:46 -04:00
Samuel Clay
8d07355bba Story guid hashing and a few small fixes for feed fetching. 2010-04-29 11:18:49 -04:00
Samuel Clay
2c6433b0de Parallelizing feed fetcher now recognizes other feed fetchers so they do not replicate work. Now multi-threaded and multi-processed aware 2010-04-27 13:44:53 -04:00
Samuel Clay
722a3f7031 Fixing BAD bug around data integrity for adding then deleting feeds. Also added a return button handler to add feeds, and disabled the manage button when necessary. 2010-04-25 18:32:02 -04:00
Samuel Clay
0a664b44a8 Adding feed update history. Gotta get some metrics on the speed of the biggest cpu/network suck. 2010-04-23 21:19:19 -04:00
Samuel Clay
37743431be What was I smoking? Way too exaggerated behavior. 2010-04-23 20:52:04 -04:00
Samuel Clay
fe7d889a12 Toning down the regularity of updating feeds by a half. 2010-04-23 10:49:17 -04:00
Samuel Clay
144bd35be4 Fixing trace errors. 2010-04-19 12:44:49 -04:00
Samuel Clay
4b828865e3 Fixing trace errors. 2010-04-19 12:43:01 -04:00
Samuel Clay
84a1c8c5b1 Fixing trace errors. 2010-04-19 12:42:32 -04:00
Samuel Clay
fc82492cee Fixing trace errors. 2010-04-19 12:38:55 -04:00
Samuel Clay
3621010616 Fixing trace errors. 2010-04-19 12:37:53 -04:00
Samuel Clay
330a09a645 Fixing trace errors. 2010-04-19 12:37:15 -04:00
Samuel Clay
3699ab1bd1 Fixing trace errors. 2010-04-19 12:35:59 -04:00
Samuel Clay
c1c36ad182 Moving story counter out of the feed fetcher. Will run nightly. 2010-04-19 12:18:40 -04:00
Samuel Clay
71b746a48d Feed update cascading. Feeds update relative to how often they have new content. This should save a LOT of processing power. Whew. 2010-04-19 12:09:04 -04:00
Samuel Clay
f169569710 Adding single threaded feed updating (used for tests and debugging) 2010-04-09 16:37:19 -04:00
Samuel Clay
a2458f0973 Add feed and Add folder. Dynamically loads and refreshes, too. 2010-04-06 20:41:00 -04:00
Samuel Clay
73e269ebb1 Styling changes to logging. 2010-04-06 16:56:47 -04:00
Samuel Clay
8f22031ea3 Adding threaded folders and feeds. Also refactored OPML import to handle many edge cases and nested cases. Rewrote feed loading on front-end so everything's faster to load. 2010-02-11 01:28:47 -05:00
samuelclay
27d5c1c421 Not perfect, but a temp fix for feeds that have no dates (like Hacker News). 2010-01-28 13:28:27 -05:00
samuelclay
d9c7ae7771 Auto-compressing all stories and feeds at the db level for better database lookups. 2010-01-27 16:29:11 -05:00
samuelclay
5ed1b06710 Opening the floodgates for proper counting. 2010-01-26 20:39:11 -05:00
samuelclay
6c635eb109 Reducing work of feed_fetcher to check a decent range of existing stories, now including one with explicit GUIDs, to prevent the bug where feeds show really old stories (like Daring Fireball's treatment of the Star stories). 2010-01-26 20:23:41 -05:00
Samuel Clay
1180100e37 - Fixing caching of feeds and users.
- Added loading indicator on new feed.
- Cleaned up numerous bugs around showing/hiding out-of-score stories.
2010-01-24 22:53:46 -05:00
Samuel Clay
4ed7f0e9d1 Success on reducing calculate_scores work. Now, to get it back in to feed_fetching. 2010-01-21 23:56:29 -05:00
Samuel Clay
c6f86dad76 OK, for now, score calculation is too expensive to do on feed fetching. I'll run it manually until I figure out the best way to run it. For one, running 8 threads all calculating scores is *not* the way to go on a single db. 2010-01-21 22:23:42 -05:00
Samuel Clay
b1c72c5c9f Huge update: Hooking up slider to hide/show stories based on their score. Also added new story counts, score calculations, and real-time updating of unread counts as you read. 2010-01-21 13:12:29 -05:00
Samuel Clay
35df77cdf6 Adding classification types for publishers vs. stories. 2010-01-12 01:19:37 +00:00
Samuel Clay
8891a859fe Finishing up tags, and the classifier modal in the Reader. Now shows all sorts of classifiers. 2010-01-04 22:26:53 +00:00
Samuel Clay
44f89023ff Placing upper limits on dates 2009-12-18 20:49:38 +00:00
Samuel Clay
8bf5fc8dca Fixing the out-of-date dupe bug. 2009-12-18 20:47:44 +00:00
Samuel Clay
06e94610dc Adding Phrase Filter to Analyzer. Just need to hook up the phrase filter to the story classifier and then save the results to be used for later processing. 2009-11-15 18:57:53 +00:00
Samuel Clay
0edbb7ed74 I just don't feel comfortable with Django's thread-unsafe ORM. Putting locks back in place around all database .save() calls. It's The Right Thing To Do. 2009-09-16 04:00:37 +00:00
Samuel Clay
19df861456 Forgot to remove tests. 2009-09-16 03:56:58 +00:00