Samuel Clay
d748a41d85
Trimming > 1000 stories from feeds.
2010-01-26 19:59:43 -05:00
Samuel Clay
b1c72c5c9f
Huge update: Hooking up slider to hide/show stories based on their score. Also added new story counts, score calculations, and real-time updating of unread counts as you read.
2010-01-21 13:12:29 -05:00
Samuel Clay
35df77cdf6
Adding classification types for publishers vs. stories.
2010-01-12 01:19:37 +00:00
Samuel Clay
8891a859fe
Finishing up tags, and the classifier modal in the Reader. Now shows all sorts of classifiers.
2010-01-04 22:26:53 +00:00
Samuel Clay
ef70c288ed
Adding a classifier to all stories. Classifies on:
...
- Story title (highlighted phrase)
- Story author
- Feed
2010-01-04 04:49:59 +00:00
Samuel Clay
838a57baa8
Fixing up sessions so they don't cache. Also adding Authors to admin.
2009-12-18 21:59:14 +00:00
Samuel Clay
8bf5fc8dca
Fixing the out-of-date dupe bug.
2009-12-18 20:47:44 +00:00
Samuel Clay
06e94610dc
Adding Phrase Filter to Analyzer. Just need to hook up the phrase filter to the story classifier and then save the results to be used for later processing.
2009-11-15 18:57:53 +00:00
Samuel Clay
fb6d0692ce
Naive Fisher Bayesian Classifier. Stubbed in some boilerplate for making the analyzer do its job. Still have to connect it to feeds and train on real data, but it's looking good as a starting point.
2009-11-03 03:52:03 +00:00
Samuel Clay
0edbb7ed74
I just don't feel comfortable with Django's thread-unsafe ORM. Putting locks back in place around all database .save() calls. It's The Right Thing To Do.
2009-09-16 04:00:37 +00:00
Samuel Clay
7b08770b3a
Getting rid of Queues entirely. Replaced with values being distributed directly to Processes. One day this will change. Until then, all feeds fit in memory.
2009-09-16 03:54:33 +00:00
Samuel Clay
4aa0271047
Adding semaphore locks on all mysql db requests in threads/processes.
2009-09-16 02:34:04 +00:00
Samuel Clay
a7888c2aae
Switching from threads to processes. Sure, no big deal. Django is not thread-safe, so I can't use threads. I can, however, use multiprocessing to pass a Queue around between Processes.
2009-09-16 02:22:27 +00:00
Samuel Clay
0446ff2ee3
Adding locks to feed processing.
2009-09-12 20:42:38 +00:00
Samuel Clay
bec2f00979
Exiting after it's over.
2009-09-10 04:24:51 +00:00
Samuel Clay
0b9b778382
Queues not working. Reverting back to 1 feed at a time. Ugh.
2009-09-10 04:16:50 +00:00
Samuel Clay
8437ef29f1
Handling empty queues and increasing thread count
2009-09-10 03:52:43 +00:00
Samuel Clay
67ab0d6c66
Let's try changing the load.
2009-09-10 03:48:22 +00:00
Samuel Clay
02a4d292e8
Moving from threadpool to a Queue. Much simpler.
2009-09-10 03:33:05 +00:00
Samuel Clay
0caf49e9a4
OK. Turning on threading for feed fetching. Enough logging to catch the bugs. Fixed the segmentation fault on exit.
2009-09-10 02:31:55 +00:00
Samuel Clay
2ac39549ae
The "Hacker News" bug is fixed. Weirdo HN site doesn't publish guids or timestamps. Now bubbling priorities on ids. Also added 'j' and 'k' keyboard shortcuts. And fixed the escaping url bug on the Reader.
2009-09-08 00:13:49 +00:00
Samuel Clay
c0ca9277b8
- Added specialized unit tests for crappy RSS input (like Slashdot).
...
- Rewrote the feed view in the reader. It is now a long list and looks a lot better.
- Big update to story title logic. Can now handle many more edge cases. Also uses smarter heading finder.
- Fixed scrollTo bug, so now multiple clicks do not queue awkwardly.
- Moved the story titles pane to the bottom.
2009-09-07 01:56:52 +00:00
Samuel Clay
27e539ba57
What happens if the id/permalink are blank? Same story! No, bad idea.
2009-09-05 02:37:31 +00:00
Samuel Clay
acd464af86
Using story GUID's on top of permalinks.
2009-09-05 02:22:25 +00:00
Samuel Clay
9e522ca528
Woo - Checking for content and title differences to resolve duplicate entries. Integration tests prove a number of differences are successfully resolved.
2009-08-30 00:43:13 +00:00
Samuel Clay
bdd91d714d
Refactored the feed fetcher. Added integration tests for dupe matching. Also stubbed in AI score calculations so I can do something with the output soon.
2009-08-29 19:34:42 +00:00
Samuel Clay
3bd43a8cb4
Updating algorithm to insert the necessary base information into the feed's page.
2009-08-26 03:12:55 +00:00
Samuel Clay
d24b56fbf4
Adding new "Return to Page" button. When the user visits other pages in a feed, security prevents NB from reading the iframe, so we provide a convenient hook to bring the iframe back. Call it SnapBack, except, don't.
2009-08-26 02:31:35 +00:00
Samuel Clay
ce9ec58d00
Some feeds don't have pages. Who knew.
2009-08-23 00:07:37 +00:00
Samuel Clay
9c522416a1
Gotta get rid of threading until I can figure out how to thread-safe the Django ORM. It's corrupting my tables!
2009-08-23 00:00:55 +00:00
Samuel Clay
fa123e7f3d
Updating feeds to crash with noise.
2009-08-21 13:14:44 +00:00
Samuel Clay
6cf04497ee
Checking for threadpool
2009-08-20 02:53:41 +00:00
Samuel Clay
8545e09d9a
Using threadpool module and feedjack to import feeds. Largely the same functionality, but now more debugging info, as well as better error handling and threads.
2009-08-20 02:43:01 +00:00
Samuel Clay
492d1fb229
Better imports through error handling.
2009-08-15 15:10:21 +00:00
Samuel Clay
11a9921c07
2009-08-14 02:32:30 +00:00
Samuel Clay
2728ca5722
Making story pane iframe permanent.
2009-08-14 02:27:49 +00:00
Samuel Clay
001ffc942b
Fixing failure on bad address.
2009-08-14 01:54:22 +00:00
Samuel Clay
88ccfa7e77
Created admin commands to run refresh_feed/s without invoking Apache. Finally!
2009-08-14 01:48:21 +00:00
Samuel Clay
d97c5ff0c3
Catching the proper exception.
2009-08-13 04:05:49 +00:00
Samuel Clay
0f4cfdffb7
Importing feed's original page along with RSS stories. Wowzers.
2009-08-13 03:26:12 +00:00
Samuel Clay
0582b052bf
Tests can now be run from any server/environment.
2009-08-09 19:22:09 +00:00
Samuel Clay
9626b4695a
Updating integration tests to include updated feeds. Wahoo! Now to figure out the dupe issues through testing.
2009-08-08 16:52:11 +00:00
Samuel Clay
555b1e3bda
Removing a few debug messages for cleaner debugging in production.
2009-08-07 12:21:39 +00:00
Samuel Clay
bd0099ad81
Rewrote ingestion and acquiring of feeds. This needs some serious TLC soon.
2009-08-01 04:26:57 +00:00
Samuel Clay
709beb79b7
Cleaning up tests, reload OPML pane, and minor refactorings for errors.
2009-07-28 22:51:16 +00:00
Samuel Clay
5d185bae74
Optimizing and refactoring a ton of queries and calls. Really, I touched almost everything, from the feed ingestion system, to marking stories as read, to merging ReadStories and Opinions.
2009-07-28 02:27:27 +00:00
Samuel Clay
d4b25e80d2
It's the start of an actual interface. Menu buttons added (still need to be styled and tweaked to auto-close).
2009-07-21 03:18:29 +00:00
Samuel Clay
8d3bc4ec9b
Reducing logging.
2009-07-20 05:12:11 +00:00
Samuel Clay
0a8449db50
Numerous memory-saving operations. Chilled out some hoggish sql queries.
2009-07-18 23:39:16 +00:00
Samuel Clay
db09737792
Adding integration tests and fixtures for the RSS Feeds app. In hind-sight, it should've been called 'FeedInjestion'
2009-07-08 02:05:45 +00:00
