Samuel Clay
|
3699ab1bd1
|
Fixing trace errors.
|
2010-04-19 12:35:59 -04:00 |
|
Samuel Clay
|
c1c36ad182
|
Moving story counter out of the feed fetcher. Will run nightly.
|
2010-04-19 12:18:40 -04:00 |
|
Samuel Clay
|
71b746a48d
|
Feed update cascading. Feeds update relative to how often they have new content. This should save a LOT of processing power. Whew.
|
2010-04-19 12:09:04 -04:00 |
|
Samuel Clay
|
f169569710
|
Adding single threaded feed updating (used for tests and debugging)
|
2010-04-09 16:37:19 -04:00 |
|
Samuel Clay
|
a2458f0973
|
Add feed and Add folder. Dynamically loads and refreshes, too.
|
2010-04-06 20:41:00 -04:00 |
|
Samuel Clay
|
73e269ebb1
|
Styling changes to logging.
|
2010-04-06 16:56:47 -04:00 |
|
Samuel Clay
|
8f22031ea3
|
Adding threaded folders and feeds. Also refactored OPML import to handle many edge cases and nested cases. Rewrote feed loading on front-end so everything's faster to load.
|
2010-02-11 01:28:47 -05:00 |
|
samuelclay
|
27d5c1c421
|
Not perfect, but a temp fix for feeds that have no dates (like Hacker News).
|
2010-01-28 13:28:27 -05:00 |
|
samuelclay
|
d9c7ae7771
|
Auto-compressing all stories and feeds at the db level for better database lookups.
|
2010-01-27 16:29:11 -05:00 |
|
samuelclay
|
5ed1b06710
|
Opening the floodgates for proper counting.
|
2010-01-26 20:39:11 -05:00 |
|
samuelclay
|
6c635eb109
|
Reducing work of feed_fetcher to check a decent range of existing stories, now including one with explicit GUIDs, to prevent the bug where feeds show really old stories (like Daring Fireball's treatment of the Star stories).
|
2010-01-26 20:23:41 -05:00 |
|
Samuel Clay
|
1180100e37
|
- Fixing caching of feeds and users.
- Added loading indicator on new feed.
- Cleaned up numerous bugs around showing/hiding out-of-score stories.
|
2010-01-24 22:53:46 -05:00 |
|
Samuel Clay
|
4ed7f0e9d1
|
Success on reducing calculate_scores work. Now, to get it back in to feed_fetching.
|
2010-01-21 23:56:29 -05:00 |
|
Samuel Clay
|
c6f86dad76
|
OK, for now, score calculation is too expensive to do on feed fetching. I'll run it manually until I figure out the best way to run it. For one, running 8 threads all calculating scores is *not* the way to go on a single db.
|
2010-01-21 22:23:42 -05:00 |
|
Samuel Clay
|
b1c72c5c9f
|
Huge update: Hooking up slider to hide/show stories based on their score. Also added new story counts, score calculations, and real-time updating of unread counts as you read.
|
2010-01-21 13:12:29 -05:00 |
|
Samuel Clay
|
35df77cdf6
|
Adding classification types for publishers vs. stories.
|
2010-01-12 01:19:37 +00:00 |
|
Samuel Clay
|
8891a859fe
|
Finishing up tags, and the classifier modal in the Reader. Now shows all sorts of classifiers.
|
2010-01-04 22:26:53 +00:00 |
|
Samuel Clay
|
44f89023ff
|
Placing upper limits on dates
|
2009-12-18 20:49:38 +00:00 |
|
Samuel Clay
|
8bf5fc8dca
|
Fixing the out-of-date dupe bug.
|
2009-12-18 20:47:44 +00:00 |
|
Samuel Clay
|
06e94610dc
|
Adding Phrase Filter to Analyzer. Just need to hook up the phrase filter to the story classifier and then save the results to be used for later processing.
|
2009-11-15 18:57:53 +00:00 |
|
Samuel Clay
|
0edbb7ed74
|
I just don't feel comfortable with Django's thread-unsafe ORM. Putting locks back in place around all database .save() calls. It's The Right Thing To Do.
|
2009-09-16 04:00:37 +00:00 |
|
Samuel Clay
|
19df861456
|
Forgot to remove tests.
|
2009-09-16 03:56:58 +00:00 |
|
Samuel Clay
|
7b08770b3a
|
Getting rid of Queues entirely. Replaced with values being distributed directly to Processes. One day this will change. Until then, all feeds fit in memory.
|
2009-09-16 03:54:33 +00:00 |
|
Samuel Clay
|
e06b736c5b
|
Split the db connection between multiple Processes. To do this, you must close the db connection at the Process creation, so the process can re-create its own connection.
|
2009-09-16 02:39:45 +00:00 |
|
Samuel Clay
|
4aa0271047
|
Adding semaphore locks on all mysql db requests in threads/processes.
|
2009-09-16 02:34:04 +00:00 |
|
Samuel Clay
|
a7888c2aae
|
Switching from threads to processes. Sure, no big deal. Django is not thread-safe, so I can't use threads. I can, however, use multiprocessing to pass a Queue around between Processes.
|
2009-09-16 02:22:27 +00:00 |
|
Samuel Clay
|
0446ff2ee3
|
Adding locks to feed processing.
|
2009-09-12 20:42:38 +00:00 |
|
Samuel Clay
|
9026bc4401
|
Maybe exiting the thread will work...
|
2009-09-12 20:15:22 +00:00 |
|
Samuel Clay
|
0b9b778382
|
Queues not working. Reverting back to 1 feed at a time. Ugh.
|
2009-09-10 04:16:50 +00:00 |
|
Samuel Clay
|
8437ef29f1
|
Handling empty queues and increasing thread count
|
2009-09-10 03:52:43 +00:00 |
|
Samuel Clay
|
67ab0d6c66
|
Let's try changing the load.
|
2009-09-10 03:48:22 +00:00 |
|
Samuel Clay
|
02a4d292e8
|
Moving from threadpool to a Queue. Much simpler.
|
2009-09-10 03:33:05 +00:00 |
|
Samuel Clay
|
0caf49e9a4
|
OK. Turning on threading for feed fetching. Enough logging to catch the bugs. Fixed the segmentation fault on exit.
|
2009-09-10 02:31:55 +00:00 |
|
Samuel Clay
|
2ac39549ae
|
The "Hacker News" bug is fixed. Weirdo HN site doesn't publish guids or timestamps. Now bubbling priorities on ids. Also added 'j' and 'k' keyboard shortcuts. And fixed the escaping url bug on the Reader.
|
2009-09-08 00:13:49 +00:00 |
|
Samuel Clay
|
105b0e7647
|
Auto-switch to feed view when story not found on page. Auto-switch back if the user does not explicitly switch to feed view. Smart thinking!
|
2009-09-01 12:38:47 +00:00 |
|
Samuel Clay
|
bdd91d714d
|
Refactored the feed fetcher. Added integration tests for dupe matching. Also stubbed in AI score calculations so I can do something with the output soon.
|
2009-08-29 19:34:42 +00:00 |
|