Commit graph

1452 commits

Author SHA1 Message Date
Samuel Clay
0deb0aff2f Fixing invalid feed tags. 2010-08-24 16:19:09 -04:00
Samuel Clay
538854f8e4 Fixing invalid feed tags. 2010-08-24 16:14:56 -04:00
Samuel Clay
44969c7eee Upgrading feedparser. 2010-08-24 15:35:05 -04:00
Samuel Clay
547d7efd9f Deferring the loading of the iframe + feed to decrease latency before it all loads. 2010-08-23 23:29:43 -04:00
Samuel Clay
d90f1791e1 Refactored a bit here and there. The real changes are coming tonight. 2010-08-23 21:04:51 -04:00
Samuel Clay
fc16a1b8a9 Small refactor on names and amalgamated methods. 2010-08-23 09:55:21 -04:00
Samuel Clay
abe73a1d0e Barring the ability to use multiprocessing for feed fetching, in the meantime we'll have to use separate processes, which means potential for fetchin gthe same feeds. 2010-08-23 08:09:58 -04:00
Samuel Clay
d03bd58a27 Unicode bug on html rewriting. 2010-08-23 08:04:42 -04:00
Samuel Clay
d03b4b8e69 Unicode bug on html rewriting. 2010-08-23 07:58:09 -04:00
Samuel Clay
3cd4ab9774 Fixing issues with bootstrapping read stories. 2010-08-23 01:06:04 -04:00
Samuel Clay
e0282efb80 Rescuing story bootstrapping. Halfway through it crashes, so restarted at a savepoint. 2010-08-22 21:39:45 -04:00
Samuel Clay
496c590016 Rescuing story bootstrapping. Halfway through it crashes, so restarted at a savepoint. 2010-08-22 21:36:12 -04:00
Samuel Clay
def2f5a418 Rescuing story bootstrapping. 2010-08-22 20:05:33 -04:00
Samuel Clay
a7d5ff3cfe Rescuing story bootstrapping. 2010-08-22 20:03:25 -04:00
Samuel Clay
f90029eed1 Whew. Finished entire mongo conversion. Classifiers, Stories, and UserStories. Includes bootstrapping each from Postgres. 2010-08-22 18:34:40 -04:00
Samuel Clay
79d1ffd5db Holy hell, the Mongo transition is nearly complete. All that's left is classifiers. This includes stories, user_stories, and calculating unread counts. 2010-08-21 23:49:36 -04:00
Samuel Clay
b1dfdb1239 Merge branch 'master' of github.com:samuelclay/NewsBlur into mongo 2010-08-21 20:42:51 -04:00
Samuel Clay
ac53f33788 Further Mongo work. Stories are now saved correctly, with tags + authors. Still need to do UserStories. 2010-08-21 20:42:38 -04:00
Samuel Clay
fa3be28b43 Initial stab at MongoDB in feed fetching and feed viewing. Still need to fix authors, tags, counts, aggregations, and any remaining bugs. 2010-08-21 13:57:49 -04:00
Samuel Clay
03fc6aa2ad Single threaded feed fetch on adding feeds. 2010-08-19 20:06:20 -04:00
Samuel Clay
22071da997 Merge branch 'master' of github.com:samuelclay/NewsBlur 2010-08-19 19:54:47 -04:00
Samuel Clay
4288283c95 Small tweaks to feed fetching locks. 2010-08-19 19:54:44 -04:00
Samuel Clay
7e40103a4c Adding a duplicate feed table so future duplicates will be caught and corrected. 2010-08-19 10:43:07 -04:00
Samuel Clay
5e438ac439 Coercing errors into integers. Doh. 2010-08-18 22:06:56 -04:00
Samuel Clay
8b9c28e8f8 Cleaning up exception handling of feeds to show to users during import. 2010-08-18 21:56:30 -04:00
Samuel Clay
68b0ae7254 Adding exception handling UI. Also disabled feeds that are just chock full of exceptions. No need to continually fetch buggy feeds. Still need to figure out a way to let users fix problematic feeds, but letting them know is an improvement. 2010-08-18 20:35:45 -04:00
Samuel Clay
aa99517511 Completely changing how subscriptions are counted. Now happens after page load. Also retooled Google Reader import to load, count, and fetch after page load so the user is not kept waiting. Progress bars for everything. 2010-08-17 23:40:03 -04:00
Samuel Clay
f7574e6d11 Tweaks to fetching times. This will buy me about 3 days to figure out how to scale. 2010-08-16 17:51:15 -04:00
Samuel Clay
f3522cf0ec Adding logging in many other places. 2010-08-16 15:45:35 -04:00
Samuel Clay
55683fc1d5 Skipping feeds with fewer than X stories_per_month. Used to catch the queue up and toss inactive feeds. 2010-08-16 13:21:32 -04:00
Samuel Clay
2e1c3b4db3 Adding stats and training to logging. 2010-08-16 12:55:45 -04:00
Samuel Clay
588207384c Logging tweaks to handle console and file handlers. 2010-08-16 12:52:39 -04:00
Samuel Clay
fcca02717f Silencing logs from the backgorund importer. 2010-08-15 12:09:40 -04:00
Samuel Clay
3df0722dcc Moving from print-based logging to python-based logging. Quite the improvement. Console/Stream output in development, logfile in production. 2010-08-15 12:04:26 -04:00
Samuel Clay
8bde22d1bd Typo in story count history migration. 2010-08-13 11:05:52 -04:00
Samuel Clay
bbc15f4fbd Story count histories. +migrations 2010-08-13 11:03:07 -04:00
Samuel Clay
9ec1cc3914 Gracefully handling long feeds in storiespermonth migration. 2010-08-13 10:56:55 -04:00
Samuel Clay
d4f30d8236 Finishing up statistics graphs. Using Flot instead of Raphael. Added a migration to change how stories are counted. 2010-08-13 10:43:48 -04:00
Samuel Clay
20096efa1f Feed de-duplicator gracefully handles users who have erased too much. 2010-08-11 17:13:39 -04:00
Samuel Clay
7cc6681acd Trimming only feeds that were fetched once, to ensure they have stories used to compare duplicate feeds. 2010-08-11 16:43:56 -04:00
Samuel Clay
aeda525b55 1) Adding new feed story counter. 2) Adding feed fethed initially status. Useful for displaying percentages of fetched feeds for new users. 3) Fade in with loader on feed list. 2010-08-09 20:44:46 -04:00
Samuel Clay
4fe631788e Handling overly long story titles. 2010-08-04 18:55:52 -04:00
Samuel Clay
5cb7d1509a Feeds that don't parse are no longer auto-corrected. This should be handled by an automated scrubber, not the fetcher itself. Ignoring them for now. 2010-08-04 18:30:51 -04:00
Samuel Clay
67efaa91bf Logging statistics, classifiers, and feed trainer. 2010-08-02 23:10:43 -04:00
Samuel Clay
da1bb81cd2 Updated Trainer. Now comes with a full tutorial and resized modals. 2010-08-01 23:47:45 -04:00
Samuel Clay
66eae41e2d Intelligence Trainer. A whopper of a commit. 2010-08-01 19:12:49 -04:00
Samuel Clay
2fe632b4bc Closing the database connection on the parent feed fetcher so it can be re-opened by worker threads. 2010-07-30 20:04:02 -04:00
Samuel Clay
a4c6b093c4 Don't normalize non-string urls. 2010-07-28 18:18:01 -04:00
Samuel Clay
7b7475ac70 Adding feed update stats to stats modal. 2010-07-28 01:14:25 -04:00
Samuel Clay
c692d7d913 Bigrams and Trigrams for collocation search. 2010-07-27 23:29:04 -04:00