Commit graph

70 commits

Author SHA1 Message Date
Samuel Clay
2e15f84a0d Additional instrumentation on feed fetching, parsing, and importing. 2010-07-06 18:16:41 -04:00
Samuel Clay
08ef889e09 Feed fetch history instrumentation. 2010-07-06 13:21:12 -04:00
Samuel Clay
d3935212a4 Preventing dud feeds to pause the whole feed refresher. 2010-07-05 23:19:07 -04:00
Samuel Clay
c9fd6b5c80 Preventing dud feeds to pause the whole feed refresher. 2010-07-05 23:17:36 -04:00
Samuel Clay
d5cbd1ad29 Tweaks to refreshing algorithm. More subscribers = faster service. Fewer subscribers = way less ervice for less active feeds. Active feeds still get more work 2010-07-05 23:06:37 -04:00
Samuel Clay
1c42a7088d A number of aesthetic changes that make signup and login sharper. 2010-07-05 22:15:09 -04:00
Samuel Clay
013cf7ab04 I wrote all this code but have no idea why I didn't commit it. Anyway, let's deploy it. (It looks like I'm slowing down feed refreshing, which is bound to not matter in ~10 users or so). 2010-07-05 14:26:35 -04:00
Samuel Clay
9867406ca2 Refactoring feed counts, subscriber counts, story counts, and tightening up refreshes for high-volume, high-subscriber feeds. Less decay on somewhat popular feeds. 2010-07-02 15:49:08 -04:00
Samuel Clay
2133ee81e7 Fixing issues with feature paging. JSON was incorrectly serializing it, the page_counter was being reset when feeds were loaded, and cleaned up feed resetting. 2010-07-02 12:02:10 -04:00
Samuel Clay
aaf68f4c21 Fetching page for high-volume feeds on every fetch instead of only when new stories are in the feed. 2010-07-02 11:46:45 -04:00
Samuel Clay
5b1bb231ef Removing unnecessary cruft above the page importer. 2010-07-01 17:33:58 -04:00
Samuel Clay
4b57a32c0c Removing 2 bigger JOINs that were slowing down feed loading. 2010-07-01 15:16:33 -04:00
Samuel Clay
113a4e058c Adding a Delete This Site manage menu item, complete with cancel button. 2010-07-01 00:29:26 -04:00
Samuel Clay
80bdb79bd9 Better instrumentation through non-zero results on any successfull callback. 2010-06-28 08:06:12 -04:00
Samuel Clay
3d4cf207d3 Adding a feed punishment for slow feeds. Slow feeds get updated less often. 2010-06-24 15:27:25 -04:00
Samuel Clay
cc8beb2379 Fixing caching of feed stories so they load faster. 2010-06-24 15:10:15 -04:00
Samuel Clay
fbe1367739 Protopub -> NewsBlur 2010-06-08 11:19:07 -04:00
Samuel Clay
d39dacb592 Cleanup in prep for move from urllib to eventlets. 2010-05-20 15:13:25 -04:00
Samuel Clay
3ca7fcefca That mass renames continue! Protoread is now Protopub, and that sounds good to me. 2010-05-14 09:32:38 -04:00
Samuel Clay
b1ac2147ba Renaming News Blur to Protoread. 2010-05-13 11:11:01 -04:00
Samuel Clay
0176308858 Stylistic changes, like the name of the site: NewsBlur is now News Blur. 2010-05-11 18:38:32 -04:00
Samuel Clay
ce0495002e Adding load time to feeds to punish slow feeds. 2010-04-29 13:35:46 -04:00
Samuel Clay
8d07355bba Story guid hashing and a few small fixes for feed fetching. 2010-04-29 11:18:49 -04:00
Samuel Clay
2c6433b0de Parallelizing feed fetcher now recognizes other feed fetchers so they do not replicate work. Now multi-threaded and multi-processed aware 2010-04-27 13:44:53 -04:00
Samuel Clay
722a3f7031 Fixing BAD bug around data integrity for adding then deleting feeds. Also added a return button handler to add feeds, and disabled the manage button when necessary. 2010-04-25 18:32:02 -04:00
Samuel Clay
0a664b44a8 Adding feed update history. Gotta get some metrics on the speed of the biggest cpu/network suck. 2010-04-23 21:19:19 -04:00
Samuel Clay
37743431be What was I smoking? Way too exaggerated behavior. 2010-04-23 20:52:04 -04:00
Samuel Clay
fe7d889a12 Toning down the regularity of updating feeds by a half. 2010-04-23 10:49:17 -04:00
Samuel Clay
144bd35be4 Fixing trace errors. 2010-04-19 12:44:49 -04:00
Samuel Clay
4b828865e3 Fixing trace errors. 2010-04-19 12:43:01 -04:00
Samuel Clay
84a1c8c5b1 Fixing trace errors. 2010-04-19 12:42:32 -04:00
Samuel Clay
fc82492cee Fixing trace errors. 2010-04-19 12:38:55 -04:00
Samuel Clay
3621010616 Fixing trace errors. 2010-04-19 12:37:53 -04:00
Samuel Clay
330a09a645 Fixing trace errors. 2010-04-19 12:37:15 -04:00
Samuel Clay
3699ab1bd1 Fixing trace errors. 2010-04-19 12:35:59 -04:00
Samuel Clay
c1c36ad182 Moving story counter out of the feed fetcher. Will run nightly. 2010-04-19 12:18:40 -04:00
Samuel Clay
71b746a48d Feed update cascading. Feeds update relative to how often they have new content. This should save a LOT of processing power. Whew. 2010-04-19 12:09:04 -04:00
Samuel Clay
f169569710 Adding single threaded feed updating (used for tests and debugging) 2010-04-09 16:37:19 -04:00
Samuel Clay
a2458f0973 Add feed and Add folder. Dynamically loads and refreshes, too. 2010-04-06 20:41:00 -04:00
Samuel Clay
73e269ebb1 Styling changes to logging. 2010-04-06 16:56:47 -04:00
Samuel Clay
8f22031ea3 Adding threaded folders and feeds. Also refactored OPML import to handle many edge cases and nested cases. Rewrote feed loading on front-end so everything's faster to load. 2010-02-11 01:28:47 -05:00
samuelclay
27d5c1c421 Not perfect, but a temp fix for feeds that have no dates (like Hacker News). 2010-01-28 13:28:27 -05:00
samuelclay
d9c7ae7771 Auto-compressing all stories and feeds at the db level for better database lookups. 2010-01-27 16:29:11 -05:00
samuelclay
5ed1b06710 Opening the floodgates for proper counting. 2010-01-26 20:39:11 -05:00
samuelclay
6c635eb109 Reducing work of feed_fetcher to check a decent range of existing stories, now including one with explicit GUIDs, to prevent the bug where feeds show really old stories (like Daring Fireball's treatment of the Star stories). 2010-01-26 20:23:41 -05:00
Samuel Clay
1180100e37 - Fixing caching of feeds and users.
- Added loading indicator on new feed.
- Cleaned up numerous bugs around showing/hiding out-of-score stories.
2010-01-24 22:53:46 -05:00
Samuel Clay
4ed7f0e9d1 Success on reducing calculate_scores work. Now, to get it back in to feed_fetching. 2010-01-21 23:56:29 -05:00
Samuel Clay
c6f86dad76 OK, for now, score calculation is too expensive to do on feed fetching. I'll run it manually until I figure out the best way to run it. For one, running 8 threads all calculating scores is *not* the way to go on a single db. 2010-01-21 22:23:42 -05:00
Samuel Clay
b1c72c5c9f Huge update: Hooking up slider to hide/show stories based on their score. Also added new story counts, score calculations, and real-time updating of unread counts as you read. 2010-01-21 13:12:29 -05:00
Samuel Clay
35df77cdf6 Adding classification types for publishers vs. stories. 2010-01-12 01:19:37 +00:00