Samuel Clay
b216ebf48a
Adding a tokenizer to the classifier, as well as using Divmod's Reverend Bayesian classifier.
2009-12-18 18:29:34 +00:00
Samuel Clay
06e94610dc
Adding Phrase Filter to Analyzer. Just need to hook up the phrase filter to the story classifier and then save the results to be used for later processing.
2009-11-15 18:57:53 +00:00
Samuel Clay
bc47a70058
Fixed numerous issues with story finding on frame page. And refactored switcheroo functions.
2009-11-08 03:26:54 +00:00
Samuel Clay
fb6d0692ce
Naive Fisher Bayesian Classifier. Stubbed in some boilerplate for making the analyzer do its job. Still have to connect it to feeds and train on real data, but it's looking good as a starting point.
2009-11-03 03:52:03 +00:00
Samuel Clay
0edbb7ed74
I just don't feel comfortable with Django's thread-unsafe ORM. Putting locks back in place around all database .save() calls. It's The Right Thing To Do.
2009-09-16 04:00:37 +00:00
Samuel Clay
7b08770b3a
Getting rid of Queues entirely. Replaced with values being distributed directly to Processes. One day this will change. Until then, all feeds fit in memory.
2009-09-16 03:54:33 +00:00
Samuel Clay
4aa0271047
Adding semaphore locks on all mysql db requests in threads/processes.
2009-09-16 02:34:04 +00:00
Samuel Clay
a7888c2aae
Switching from threads to processes. Sure, no big deal. Django is not thread-safe, so I can't use threads. I can, however, use multiprocessing to pass a Queue around between Processes.
2009-09-16 02:22:27 +00:00
Samuel Clay
0446ff2ee3
Adding locks to feed processing.
2009-09-12 20:42:38 +00:00
Samuel Clay
bec2f00979
Exiting after it's over.
2009-09-10 04:24:51 +00:00
Samuel Clay
0b9b778382
Queues not working. Reverting back to 1 feed at a time. Ugh.
2009-09-10 04:16:50 +00:00
Samuel Clay
8437ef29f1
Handling empty queues and increasing thread count
2009-09-10 03:52:43 +00:00
Samuel Clay
67ab0d6c66
Let's try changing the load.
2009-09-10 03:48:22 +00:00
Samuel Clay
02a4d292e8
Moving from threadpool to a Queue. Much simpler.
2009-09-10 03:33:05 +00:00
Samuel Clay
0caf49e9a4
OK. Turning on threading for feed fetching. Enough logging to catch the bugs. Fixed the segmentation fault on exit.
2009-09-10 02:31:55 +00:00
Samuel Clay
5185da7b16
Mark All as Read fixed.
2009-09-08 04:37:38 +00:00
Samuel Clay
4f41567628
Stories can now be marked as read. Small bug.
2009-09-08 04:27:27 +00:00
Samuel Clay
2ac39549ae
The "Hacker News" bug is fixed. Weirdo HN site doesn't publish guids or timestamps. Now bubbling priorities on ids. Also added 'j' and 'k' keyboard shortcuts. And fixed the escaping url bug on the Reader.
2009-09-08 00:13:49 +00:00
Samuel Clay
c0ca9277b8
- Added specialized unit tests for crappy RSS input (like Slashdot).
...
- Rewrote the feed view in the reader. It is now a long list and looks a lot better.
- Big update to story title logic. Can now handle many more edge cases. Also uses smarter heading finder.
- Fixed scrollTo bug, so now multiple clicks do not queue awkwardly.
- Moved the story titles pane to the bottom.
2009-09-07 01:56:52 +00:00
Samuel Clay
27e539ba57
What happens if the id/permalink are blank? Same story! No, bad idea.
2009-09-05 02:37:31 +00:00
Samuel Clay
acd464af86
Using story GUID's on top of permalinks.
2009-09-05 02:22:25 +00:00
Samuel Clay
615d1fb530
Whoops - scores should be set to *something*
2009-08-30 00:47:56 +00:00
Samuel Clay
9e522ca528
Woo - Checking for content and title differences to resolve duplicate entries. Integration tests prove a number of differences are successfully resolved.
2009-08-30 00:43:13 +00:00
Samuel Clay
bdd91d714d
Refactored the feed fetcher. Added integration tests for dupe matching. Also stubbed in AI score calculations so I can do something with the output soon.
2009-08-29 19:34:42 +00:00
Samuel Clay
3bd43a8cb4
Updating algorithm to insert the necessary base information into the feed's page.
2009-08-26 03:12:55 +00:00
Samuel Clay
d24b56fbf4
Adding new "Return to Page" button. When the user visits other pages in a feed, security prevents NB from reading the iframe, so we provide a convenient hook to bring the iframe back. Call it SnapBack, except, don't.
2009-08-26 02:31:35 +00:00
Samuel Clay
ce9ec58d00
Some feeds don't have pages. Who knew.
2009-08-23 00:07:37 +00:00
Samuel Clay
9c522416a1
Gotta get rid of threading until I can figure out how to thread-safe the Django ORM. It's corrupting my tables!
2009-08-23 00:00:55 +00:00
Samuel Clay
fa123e7f3d
Updating feeds to crash with noise.
2009-08-21 13:14:44 +00:00
Samuel Clay
6cf04497ee
Checking for threadpool
2009-08-20 02:53:41 +00:00
Samuel Clay
8545e09d9a
Using threadpool module and feedjack to import feeds. Largely the same functionality, but now more debugging info, as well as better error handling and threads.
2009-08-20 02:43:01 +00:00
Samuel Clay
492d1fb229
Better imports through error handling.
2009-08-15 15:10:21 +00:00
Samuel Clay
11a9921c07
2009-08-14 02:32:30 +00:00
Samuel Clay
2728ca5722
Making story pane iframe permanent.
2009-08-14 02:27:49 +00:00
Samuel Clay
001ffc942b
Fixing failure on bad address.
2009-08-14 01:54:22 +00:00
Samuel Clay
379bbcd766
Removing feed refreshing from web urls.
2009-08-14 01:51:19 +00:00
Samuel Clay
88ccfa7e77
Created admin commands to run refresh_feed/s without invoking Apache. Finally!
2009-08-14 01:48:21 +00:00
Samuel Clay
d97c5ff0c3
Catching the proper exception.
2009-08-13 04:05:49 +00:00
Samuel Clay
0f4cfdffb7
Importing feed's original page along with RSS stories. Wowzers.
2009-08-13 03:26:12 +00:00
Samuel Clay
0582b052bf
Tests can now be run from any server/environment.
2009-08-09 19:22:09 +00:00
Samuel Clay
9626b4695a
Updating integration tests to include updated feeds. Wahoo! Now to figure out the dupe issues through testing.
2009-08-08 16:52:11 +00:00
Samuel Clay
555b1e3bda
Removing a few debug messages for cleaner debugging in production.
2009-08-07 12:21:39 +00:00
Samuel Clay
27a315fd07
Invalidating cache when new stories come in.
2009-08-01 15:25:49 +00:00
Samuel Clay
bd0099ad81
Rewrote ingestion and acquiring of feeds. This needs some serious TLC soon.
2009-08-01 04:26:57 +00:00
Samuel Clay
709beb79b7
Cleaning up tests, reload OPML pane, and minor refactorings for errors.
2009-07-28 22:51:16 +00:00
Samuel Clay
5d185bae74
Optimizing and refactoring a ton of queries and calls. Really, I touched almost everything, from the feed ingestion system, to marking stories as read, to merging ReadStories and Opinions.
2009-07-28 02:27:27 +00:00
Samuel Clay
ff13e9c4a6
Beginnings of minor cache work. So far, only on user accounts.
2009-07-25 15:24:27 +00:00
Samuel Clay
0922c7548f
Moving the counting of unread feeds over to the analyzer, not the reader. Duh. This makes loading much faster.
2009-07-25 00:57:39 +00:00
Samuel Clay
e3d6e00a56
Uploading OPML files through AJAX. Also cleaned the hell out of sql logging. fixed up the drop-shadows while at it.
2009-07-25 00:16:55 +00:00
Samuel Clay
d4b25e80d2
It's the start of an actual interface. Menu buttons added (still need to be styled and tweaked to auto-close).
2009-07-21 03:18:29 +00:00
