Samuel Clay
|
7cc6681acd
|
Trimming only feeds that were fetched once, to ensure they have stories used to compare duplicate feeds.
|
2010-08-11 16:43:56 -04:00 |
|
Samuel Clay
|
2fe632b4bc
|
Closing the database connection on the parent feed fetcher so it can be re-opened by worker threads.
|
2010-07-30 20:04:02 -04:00 |
|
Samuel Clay
|
3c6fdfdea7
|
Statistics on all feeds, stories, averages. Shown in the new statistics modal (not yet finished).
|
2010-07-25 23:13:27 -04:00 |
|
Samuel Clay
|
31407a998f
|
Handling anonymous users with more grace. They will now be told that they need to signup in order to add a feed/folder/opml. The old way was to show a loading indicator forever. Yeah.
|
2010-07-24 15:54:25 -04:00 |
|
Samuel Clay
|
7b655f4cc4
|
User stories might already exist on original feed if switching from duplicate feed.
|
2010-07-20 23:46:29 -04:00 |
|
Samuel Clay
|
95ba5fcbd8
|
Pass on deleted feeds. Everything will get deleted eventually. The SQL that gets this ID is a bit aggressive.
|
2010-07-20 23:40:52 -04:00 |
|
Samuel Clay
|
dd71b6ab20
|
Uncommented destructive methods/. Actually deleting duped feeds this time.
|
2010-07-20 23:30:31 -04:00 |
|
Samuel Clay
|
da0060fb43
|
Commenting out sql comments. This should have been handled.
|
2010-07-20 23:21:10 -04:00 |
|
Samuel Clay
|
e7b7036cd2
|
Deduping all analyzer models for multiple feeds.
|
2010-07-20 23:18:38 -04:00 |
|
Samuel Clay
|
06fdef9e5e
|
Deduping feeds. Recursively switches all users and read stories and subscriptions to use the original, non-duplicate feed.
|
2010-07-20 22:57:18 -04:00 |
|
Samuel Clay
|
3e379edf3c
|
Really deleting old feeds now. Turned off delete protection.
|
2010-07-05 22:22:15 -04:00 |
|
Samuel Clay
|
e13b41fa4c
|
Deleting old feeds to which nobody is subscribed.
|
2010-07-05 22:17:13 -04:00 |
|
Samuel Clay
|
1c42a7088d
|
A number of aesthetic changes that make signup and login sharper.
|
2010-07-05 22:15:09 -04:00 |
|
Samuel Clay
|
9867406ca2
|
Refactoring feed counts, subscriber counts, story counts, and tightening up refreshes for high-volume, high-subscriber feeds. Less decay on somewhat popular feeds.
|
2010-07-02 15:49:08 -04:00 |
|
Samuel Clay
|
c7488752df
|
Fixing missing page_data errors.
|
2010-06-27 22:40:22 -04:00 |
|
Samuel Clay
|
118f4ec8ec
|
Adding a subscriber bonus. Heavily subscribed feeds get updated more often.
|
2010-06-27 21:44:35 -04:00 |
|
Samuel Clay
|
a4de342d17
|
gUnicorn and restart script.
|
2010-06-27 21:02:55 -04:00 |
|
Samuel Clay
|
1914651425
|
Calculate number of feed subscribers. Will be used to calculate feed update priority.
|
2010-06-27 20:43:17 -04:00 |
|
Samuel Clay
|
d015bcfbfa
|
Fixing feed calculation to allow for calculating all feed scores for a specific user.
|
2010-05-11 21:36:17 -04:00 |
|
Samuel Clay
|
3573ac2e78
|
Adding a mark_read command to have the demo account always at 1 day unread.
|
2010-04-30 18:44:24 -04:00 |
|
Samuel Clay
|
8d07355bba
|
Story guid hashing and a few small fixes for feed fetching.
|
2010-04-29 11:18:49 -04:00 |
|
Samuel Clay
|
2c6433b0de
|
Parallelizing feed fetcher now recognizes other feed fetchers so they do not replicate work. Now multi-threaded and multi-processed aware
|
2010-04-27 13:44:53 -04:00 |
|
Samuel Clay
|
0a664b44a8
|
Adding feed update history. Gotta get some metrics on the speed of the biggest cpu/network suck.
|
2010-04-23 21:19:19 -04:00 |
|
Samuel Clay
|
c1c36ad182
|
Moving story counter out of the feed fetcher. Will run nightly.
|
2010-04-19 12:18:40 -04:00 |
|
Samuel Clay
|
71b746a48d
|
Feed update cascading. Feeds update relative to how often they have new content. This should save a LOT of processing power. Whew.
|
2010-04-19 12:09:04 -04:00 |
|
Samuel Clay
|
f169569710
|
Adding single threaded feed updating (used for tests and debugging)
|
2010-04-09 16:37:19 -04:00 |
|
Samuel Clay
|
dfc8760ea2
|
Adding a dirty feed checker that calculates feed scores for changed feeds.
|
2010-04-08 18:36:48 -04:00 |
|
Samuel Clay
|
d2b12da53a
|
Handling edge cases for tags.
|
2010-04-05 03:20:44 -04:00 |
|
Samuel Clay
|
391aa76dd6
|
Lowering worker threads from 8 to 4 to conserve memory.
|
2010-03-02 17:04:54 +00:00 |
|
Samuel Clay
|
8f22031ea3
|
Adding threaded folders and feeds. Also refactored OPML import to handle many edge cases and nested cases. Rewrote feed loading on front-end so everything's faster to load.
|
2010-02-11 01:28:47 -05:00 |
|
samuelclay
|
d9c7ae7771
|
Auto-compressing all stories and feeds at the db level for better database lookups.
|
2010-01-27 16:29:11 -05:00 |
|
Samuel Clay
|
d748a41d85
|
Trimming > 1000 stories from feeds.
|
2010-01-26 19:59:43 -05:00 |
|
Samuel Clay
|
35df77cdf6
|
Adding classification types for publishers vs. stories.
|
2010-01-12 01:19:37 +00:00 |
|
Samuel Clay
|
7b08770b3a
|
Getting rid of Queues entirely. Replaced with values being distributed directly to Processes. One day this will change. Until then, all feeds fit in memory.
|
2009-09-16 03:54:33 +00:00 |
|
Samuel Clay
|
a7888c2aae
|
Switching from threads to processes. Sure, no big deal. Django is not thread-safe, so I can't use threads. I can, however, use multiprocessing to pass a Queue around between Processes.
|
2009-09-16 02:22:27 +00:00 |
|
Samuel Clay
|
0446ff2ee3
|
Adding locks to feed processing.
|
2009-09-12 20:42:38 +00:00 |
|
Samuel Clay
|
bec2f00979
|
Exiting after it's over.
|
2009-09-10 04:24:51 +00:00 |
|
Samuel Clay
|
0b9b778382
|
Queues not working. Reverting back to 1 feed at a time. Ugh.
|
2009-09-10 04:16:50 +00:00 |
|
Samuel Clay
|
8437ef29f1
|
Handling empty queues and increasing thread count
|
2009-09-10 03:52:43 +00:00 |
|
Samuel Clay
|
67ab0d6c66
|
Let's try changing the load.
|
2009-09-10 03:48:22 +00:00 |
|
Samuel Clay
|
02a4d292e8
|
Moving from threadpool to a Queue. Much simpler.
|
2009-09-10 03:33:05 +00:00 |
|
Samuel Clay
|
0caf49e9a4
|
OK. Turning on threading for feed fetching. Enough logging to catch the bugs. Fixed the segmentation fault on exit.
|
2009-09-10 02:31:55 +00:00 |
|
Samuel Clay
|
9e522ca528
|
Woo - Checking for content and title differences to resolve duplicate entries. Integration tests prove a number of differences are successfully resolved.
|
2009-08-30 00:43:13 +00:00 |
|
Samuel Clay
|
bdd91d714d
|
Refactored the feed fetcher. Added integration tests for dupe matching. Also stubbed in AI score calculations so I can do something with the output soon.
|
2009-08-29 19:34:42 +00:00 |
|
Samuel Clay
|
d24b56fbf4
|
Adding new "Return to Page" button. When the user visits other pages in a feed, security prevents NB from reading the iframe, so we provide a convenient hook to bring the iframe back. Call it SnapBack, except, don't.
|
2009-08-26 02:31:35 +00:00 |
|
Samuel Clay
|
ce9ec58d00
|
Some feeds don't have pages. Who knew.
|
2009-08-23 00:07:37 +00:00 |
|
Samuel Clay
|
9c522416a1
|
Gotta get rid of threading until I can figure out how to thread-safe the Django ORM. It's corrupting my tables!
|
2009-08-23 00:00:55 +00:00 |
|
Samuel Clay
|
fa123e7f3d
|
Updating feeds to crash with noise.
|
2009-08-21 13:14:44 +00:00 |
|
Samuel Clay
|
6cf04497ee
|
Checking for threadpool
|
2009-08-20 02:53:41 +00:00 |
|
Samuel Clay
|
8545e09d9a
|
Using threadpool module and feedjack to import feeds. Largely the same functionality, but now more debugging info, as well as better error handling and threads.
|
2009-08-20 02:43:01 +00:00 |
|