samuelclay
|
d9c7ae7771
|
Auto-compressing all stories and feeds at the db level for better database lookups.
|
2010-01-27 16:29:11 -05:00 |
|
samuelclay
|
5ed1b06710
|
Opening the floodgates for proper counting.
|
2010-01-26 20:39:11 -05:00 |
|
samuelclay
|
6c635eb109
|
Reducing work of feed_fetcher to check a decent range of existing stories, now including one with explicit GUIDs, to prevent the bug where feeds show really old stories (like Daring Fireball's treatment of the Star stories).
|
2010-01-26 20:23:41 -05:00 |
|
Samuel Clay
|
1180100e37
|
- Fixing caching of feeds and users.
- Added loading indicator on new feed.
- Cleaned up numerous bugs around showing/hiding out-of-score stories.
|
2010-01-24 22:53:46 -05:00 |
|
Samuel Clay
|
4ed7f0e9d1
|
Success on reducing calculate_scores work. Now, to get it back in to feed_fetching.
|
2010-01-21 23:56:29 -05:00 |
|
Samuel Clay
|
c6f86dad76
|
OK, for now, score calculation is too expensive to do on feed fetching. I'll run it manually until I figure out the best way to run it. For one, running 8 threads all calculating scores is *not* the way to go on a single db.
|
2010-01-21 22:23:42 -05:00 |
|
Samuel Clay
|
b1c72c5c9f
|
Huge update: Hooking up slider to hide/show stories based on their score. Also added new story counts, score calculations, and real-time updating of unread counts as you read.
|
2010-01-21 13:12:29 -05:00 |
|
Samuel Clay
|
35df77cdf6
|
Adding classification types for publishers vs. stories.
|
2010-01-12 01:19:37 +00:00 |
|
Samuel Clay
|
8891a859fe
|
Finishing up tags, and the classifier modal in the Reader. Now shows all sorts of classifiers.
|
2010-01-04 22:26:53 +00:00 |
|
Samuel Clay
|
44f89023ff
|
Placing upper limits on dates
|
2009-12-18 20:49:38 +00:00 |
|
Samuel Clay
|
8bf5fc8dca
|
Fixing the out-of-date dupe bug.
|
2009-12-18 20:47:44 +00:00 |
|
Samuel Clay
|
06e94610dc
|
Adding Phrase Filter to Analyzer. Just need to hook up the phrase filter to the story classifier and then save the results to be used for later processing.
|
2009-11-15 18:57:53 +00:00 |
|
Samuel Clay
|
0edbb7ed74
|
I just don't feel comfortable with Django's thread-unsafe ORM. Putting locks back in place around all database .save() calls. It's The Right Thing To Do.
|
2009-09-16 04:00:37 +00:00 |
|
Samuel Clay
|
19df861456
|
Forgot to remove tests.
|
2009-09-16 03:56:58 +00:00 |
|
Samuel Clay
|
7b08770b3a
|
Getting rid of Queues entirely. Replaced with values being distributed directly to Processes. One day this will change. Until then, all feeds fit in memory.
|
2009-09-16 03:54:33 +00:00 |
|
Samuel Clay
|
e06b736c5b
|
Split the db connection between multiple Processes. To do this, you must close the db connection at the Process creation, so the process can re-create its own connection.
|
2009-09-16 02:39:45 +00:00 |
|
Samuel Clay
|
4aa0271047
|
Adding semaphore locks on all mysql db requests in threads/processes.
|
2009-09-16 02:34:04 +00:00 |
|
Samuel Clay
|
a7888c2aae
|
Switching from threads to processes. Sure, no big deal. Django is not thread-safe, so I can't use threads. I can, however, use multiprocessing to pass a Queue around between Processes.
|
2009-09-16 02:22:27 +00:00 |
|
Samuel Clay
|
0446ff2ee3
|
Adding locks to feed processing.
|
2009-09-12 20:42:38 +00:00 |
|
Samuel Clay
|
9026bc4401
|
Maybe exiting the thread will work...
|
2009-09-12 20:15:22 +00:00 |
|
Samuel Clay
|
0b9b778382
|
Queues not working. Reverting back to 1 feed at a time. Ugh.
|
2009-09-10 04:16:50 +00:00 |
|
Samuel Clay
|
8437ef29f1
|
Handling empty queues and increasing thread count
|
2009-09-10 03:52:43 +00:00 |
|
Samuel Clay
|
67ab0d6c66
|
Let's try changing the load.
|
2009-09-10 03:48:22 +00:00 |
|
Samuel Clay
|
02a4d292e8
|
Moving from threadpool to a Queue. Much simpler.
|
2009-09-10 03:33:05 +00:00 |
|
Samuel Clay
|
0caf49e9a4
|
OK. Turning on threading for feed fetching. Enough logging to catch the bugs. Fixed the segmentation fault on exit.
|
2009-09-10 02:31:55 +00:00 |
|
Samuel Clay
|
2ac39549ae
|
The "Hacker News" bug is fixed. Weirdo HN site doesn't publish guids or timestamps. Now bubbling priorities on ids. Also added 'j' and 'k' keyboard shortcuts. And fixed the escaping url bug on the Reader.
|
2009-09-08 00:13:49 +00:00 |
|
Samuel Clay
|
105b0e7647
|
Auto-switch to feed view when story not found on page. Auto-switch back if the user does not explicitly switch to feed view. Smart thinking!
|
2009-09-01 12:38:47 +00:00 |
|
Samuel Clay
|
bdd91d714d
|
Refactored the feed fetcher. Added integration tests for dupe matching. Also stubbed in AI score calculations so I can do something with the output soon.
|
2009-08-29 19:34:42 +00:00 |
|