samuelclay
|
5ed1b06710
|
Opening the floodgates for proper counting.
|
2010-01-26 20:39:11 -05:00 |
|
samuelclay
|
6c635eb109
|
Reducing work of feed_fetcher to check a decent range of existing stories, now including one with explicit GUIDs, to prevent the bug where feeds show really old stories (like Daring Fireball's treatment of the Star stories).
|
2010-01-26 20:23:41 -05:00 |
|
Samuel Clay
|
1180100e37
|
- Fixing caching of feeds and users.
- Added loading indicator on new feed.
- Cleaned up numerous bugs around showing/hiding out-of-score stories.
|
2010-01-24 22:53:46 -05:00 |
|
Samuel Clay
|
4ed7f0e9d1
|
Success on reducing calculate_scores work. Now, to get it back in to feed_fetching.
|
2010-01-21 23:56:29 -05:00 |
|
Samuel Clay
|
c6f86dad76
|
OK, for now, score calculation is too expensive to do on feed fetching. I'll run it manually until I figure out the best way to run it. For one, running 8 threads all calculating scores is *not* the way to go on a single db.
|
2010-01-21 22:23:42 -05:00 |
|
Samuel Clay
|
b1c72c5c9f
|
Huge update: Hooking up slider to hide/show stories based on their score. Also added new story counts, score calculations, and real-time updating of unread counts as you read.
|
2010-01-21 13:12:29 -05:00 |
|
Samuel Clay
|
35df77cdf6
|
Adding classification types for publishers vs. stories.
|
2010-01-12 01:19:37 +00:00 |
|
Samuel Clay
|
8891a859fe
|
Finishing up tags, and the classifier modal in the Reader. Now shows all sorts of classifiers.
|
2010-01-04 22:26:53 +00:00 |
|
Samuel Clay
|
44f89023ff
|
Placing upper limits on dates
|
2009-12-18 20:49:38 +00:00 |
|
Samuel Clay
|
8bf5fc8dca
|
Fixing the out-of-date dupe bug.
|
2009-12-18 20:47:44 +00:00 |
|
Samuel Clay
|
06e94610dc
|
Adding Phrase Filter to Analyzer. Just need to hook up the phrase filter to the story classifier and then save the results to be used for later processing.
|
2009-11-15 18:57:53 +00:00 |
|
Samuel Clay
|
0edbb7ed74
|
I just don't feel comfortable with Django's thread-unsafe ORM. Putting locks back in place around all database .save() calls. It's The Right Thing To Do.
|
2009-09-16 04:00:37 +00:00 |
|
Samuel Clay
|
19df861456
|
Forgot to remove tests.
|
2009-09-16 03:56:58 +00:00 |
|
Samuel Clay
|
7b08770b3a
|
Getting rid of Queues entirely. Replaced with values being distributed directly to Processes. One day this will change. Until then, all feeds fit in memory.
|
2009-09-16 03:54:33 +00:00 |
|
Samuel Clay
|
e06b736c5b
|
Split the db connection between multiple Processes. To do this, you must close the db connection at the Process creation, so the process can re-create its own connection.
|
2009-09-16 02:39:45 +00:00 |
|
Samuel Clay
|
4aa0271047
|
Adding semaphore locks on all mysql db requests in threads/processes.
|
2009-09-16 02:34:04 +00:00 |
|
Samuel Clay
|
a7888c2aae
|
Switching from threads to processes. Sure, no big deal. Django is not thread-safe, so I can't use threads. I can, however, use multiprocessing to pass a Queue around between Processes.
|
2009-09-16 02:22:27 +00:00 |
|
Samuel Clay
|
0446ff2ee3
|
Adding locks to feed processing.
|
2009-09-12 20:42:38 +00:00 |
|
Samuel Clay
|
9026bc4401
|
Maybe exiting the thread will work...
|
2009-09-12 20:15:22 +00:00 |
|
Samuel Clay
|
0b9b778382
|
Queues not working. Reverting back to 1 feed at a time. Ugh.
|
2009-09-10 04:16:50 +00:00 |
|
Samuel Clay
|
8437ef29f1
|
Handling empty queues and increasing thread count
|
2009-09-10 03:52:43 +00:00 |
|
Samuel Clay
|
67ab0d6c66
|
Let's try changing the load.
|
2009-09-10 03:48:22 +00:00 |
|
Samuel Clay
|
02a4d292e8
|
Moving from threadpool to a Queue. Much simpler.
|
2009-09-10 03:33:05 +00:00 |
|
Samuel Clay
|
0caf49e9a4
|
OK. Turning on threading for feed fetching. Enough logging to catch the bugs. Fixed the segmentation fault on exit.
|
2009-09-10 02:31:55 +00:00 |
|
Samuel Clay
|
2a0654ea4a
|
updating djangocompress
|
2009-09-08 03:37:17 +00:00 |
|
Samuel Clay
|
f9a0087334
|
Adding django-compress.
|
2009-09-08 03:15:49 +00:00 |
|
Samuel Clay
|
2ac39549ae
|
The "Hacker News" bug is fixed. Weirdo HN site doesn't publish guids or timestamps. Now bubbling priorities on ids. Also added 'j' and 'k' keyboard shortcuts. And fixed the escaping url bug on the Reader.
|
2009-09-08 00:13:49 +00:00 |
|
Samuel Clay
|
105b0e7647
|
Auto-switch to feed view when story not found on page. Auto-switch back if the user does not explicitly switch to feed view. Smart thinking!
|
2009-09-01 12:38:47 +00:00 |
|
Samuel Clay
|
9e522ca528
|
Woo - Checking for content and title differences to resolve duplicate entries. Integration tests prove a number of differences are successfully resolved.
|
2009-08-30 00:43:13 +00:00 |
|
Samuel Clay
|
bdd91d714d
|
Refactored the feed fetcher. Added integration tests for dupe matching. Also stubbed in AI score calculations so I can do something with the output soon.
|
2009-08-29 19:34:42 +00:00 |
|
Samuel Clay
|
8545e09d9a
|
Using threadpool module and feedjack to import feeds. Largely the same functionality, but now more debugging info, as well as better error handling and threads.
|
2009-08-20 02:43:01 +00:00 |
|
Samuel Clay
|
9626b4695a
|
Updating integration tests to include updated feeds. Wahoo! Now to figure out the dupe issues through testing.
|
2009-08-08 16:52:11 +00:00 |
|
Samuel Clay
|
555b1e3bda
|
Removing a few debug messages for cleaner debugging in production.
|
2009-08-07 12:21:39 +00:00 |
|
Samuel Clay
|
bd0099ad81
|
Rewrote ingestion and acquiring of feeds. This needs some serious TLC soon.
|
2009-08-01 04:26:57 +00:00 |
|
Samuel Clay
|
5d185bae74
|
Optimizing and refactoring a ton of queries and calls. Really, I touched almost everything, from the feed ingestion system, to marking stories as read, to merging ReadStories and Opinions.
|
2009-07-28 02:27:27 +00:00 |
|
Samuel Clay
|
ff13e9c4a6
|
Beginnings of minor cache work. So far, only on user accounts.
|
2009-07-25 15:24:27 +00:00 |
|
Samuel Clay
|
fdecb844b6
|
Don't force internal IP on logging.
|
2009-07-25 04:16:48 +00:00 |
|
Samuel Clay
|
5ff436d4f4
|
Don't force internal IP on logging.
|
2009-07-25 04:15:49 +00:00 |
|
Samuel Clay
|
670bdcdb1d
|
Staging server preferences.
|
2009-07-25 02:21:12 +00:00 |
|
Samuel Clay
|
e3d6e00a56
|
Uploading OPML files through AJAX. Also cleaned the hell out of sql logging. fixed up the drop-shadows while at it.
|
2009-07-25 00:16:55 +00:00 |
|
Samuel Clay
|
81296f2783
|
Deleting old reference to django extensions
|
2009-07-22 04:26:50 +00:00 |
|
Samuel Clay
|
be950d917a
|
Django Extensions
|
2009-07-22 04:24:24 +00:00 |
|
Samuel Clay
|
f0de9155d3
|
Finishing up logging properties
|
2009-07-19 17:28:05 +00:00 |
|
Samuel Clay
|
8c4bf1328a
|
Adding the official Django logging util
|
2009-07-19 17:10:54 +00:00 |
|
Samuel Clay
|
48ba40e29b
|
Initial
|
2009-06-16 03:08:55 +00:00 |
|