Samuel Clay
|
a1bf92ed93
|
Refetching 301s and 302s.
|
2010-10-03 18:19:23 -04:00 |
|
Samuel Clay
|
70e6d71c1e
|
Refetching 301s and 302s.
|
2010-10-03 18:16:32 -04:00 |
|
Samuel Clay
|
39d9e13449
|
Forcing refresh of feed information at every stage of feed fetch pipeline. This really needs a rewrite. Someday.
|
2010-10-03 18:04:40 -04:00 |
|
Samuel Clay
|
2fad7b512c
|
OK, made feed fetching stateless with regardless to the feed information between processor and dispatcher. It now updates.
|
2010-10-03 17:53:35 -04:00 |
|
Samuel Clay
|
f56629363b
|
Passing the modified feed between processor and dispatcher, to ensure they have the correct daata. Maybe make this stateless with an extra call between passes?
|
2010-10-03 17:50:27 -04:00 |
|
Samuel Clay
|
fe152e8d38
|
Forcing new feeds that have issues to save their issues. Some feeds were perpetually unfetched.
|
2010-10-03 17:48:44 -04:00 |
|
Samuel Clay
|
808df31f7f
|
Forcing new feeds that have issues to save their issues. Some feeds were perpetually unfetched.
|
2010-10-03 17:22:58 -04:00 |
|
Samuel Clay
|
00875c8991
|
Setting last_update date even for feeds with 304s, 400s, and 500s. Still updated, just no new stories.
|
2010-10-02 17:06:36 -04:00 |
|
Samuel Clay
|
3dd5dd9d8b
|
Merge branch 'master' of github.com:samuelclay/NewsBlur into chooser
|
2010-09-29 09:49:46 -04:00 |
|
Samuel Clay
|
6260df43e9
|
Only using story_guids when checking for existing copies of stories. The timezone issues make using the date a bit hairy.
|
2010-09-28 20:43:12 -04:00 |
|
Samuel Clay
|
23976bdcfd
|
Merge branch 'master' into chooser
|
2010-09-27 19:18:40 -04:00 |
|
Samuel Clay
|
7dee2103a9
|
Refactoring timelimit to add feed address re-check to a limited timeout.
|
2010-09-27 11:44:23 -04:00 |
|
Samuel Clay
|
e4c00a77a8
|
Feeds get auto-selected in feed chooser by priority. Unfortunately, it prioritizes incorrectly right now by using the minimum cutoff in a non-stable sort.
|
2010-09-24 18:22:12 -04:00 |
|
Samuel Clay
|
bc7accc593
|
Trimming feeds on every update.
|
2010-09-22 15:57:55 -04:00 |
|
Samuel Clay
|
402ef2db2e
|
Deleting old stories just for the specific feed.
|
2010-09-22 11:15:56 -04:00 |
|
Samuel Clay
|
744745b3ee
|
Hiding Delete Folder so I can get these folder actions shipped. Also improving performance of feed fetcher.
|
2010-09-21 18:58:21 -04:00 |
|
Samuel Clay
|
b9cbf6a71e
|
Deleting old read stories and speeding up computing feed scores by caching stories to search through.
|
2010-09-19 11:30:18 -04:00 |
|
Samuel Clay
|
ba3e75ba3f
|
Whoops, sometimes there is no etag.
|
2010-09-17 13:24:23 -04:00 |
|
Samuel Clay
|
b61955c7f3
|
Fixing etag max_length, whcih was too short for certain websites and their giant-ass etags.
|
2010-09-17 13:06:28 -04:00 |
|
Samuel Clay
|
9792550dcc
|
Counting only active subscribers in prep for turning off feed fetching for inactive users.
|
2010-09-17 12:42:44 -04:00 |
|
Samuel Clay
|
0d5b964b38
|
Small code cleanups before relaunching story_guids unique.
|
2010-09-10 00:51:20 -07:00 |
|
Samuel Clay
|
974d5b4a58
|
Deleting stories with invalid id's.
|
2010-09-09 07:00:08 -07:00 |
|
Samuel Clay
|
15a27e840f
|
Bootstrapping mongo to use story_guids as unqiue to feeds.
|
2010-09-08 18:30:46 -07:00 |
|
Samuel Clay
|
f880739fc8
|
Setting next scheduled date on manual update.
|
2010-09-07 15:42:22 -07:00 |
|
Samuel Clay
|
b5da6eace1
|
Refactoring feed fetching logging to get to the bottom of the fetch problems.
|
2010-09-07 14:41:11 -07:00 |
|
Samuel Clay
|
a69d812629
|
Removing slow punishment for popular feeds.
|
2010-09-07 14:02:48 -07:00 |
|
Samuel Clay
|
34a687716f
|
Only closing postgres connection in multithreaded environment.
|
2010-09-01 15:29:35 -04:00 |
|
Samuel Clay
|
bd78133fa6
|
No longer checking for already fetched feeds. There is really no need for this anymore, since it is guaranteed that there is only one handler for each feed, thanks to celery.
|
2010-08-31 23:19:06 -04:00 |
|
Samuel Clay
|
0805c7fc04
|
Adding a timelimit to fetching the feed.
|
2010-08-31 20:17:27 -04:00 |
|
Samuel Clay
|
79ba0e695a
|
Removing weird threading-style time limiting of feed fetching.
|
2010-08-31 08:33:18 -04:00 |
|
Samuel Clay
|
985946e63b
|
Removing timelimit from feed fetcher. This caused massive interrupts.
|
2010-08-31 08:13:20 -04:00 |
|
Samuel Clay
|
48b147751d
|
Adding a timelimit to feed fetching. 20 seconds, that's all you got.
|
2010-08-30 23:55:24 -04:00 |
|
Samuel Clay
|
cb59861098
|
Correcting bug where next_update would be set in the future too quickly.
|
2010-08-30 23:37:39 -04:00 |
|
Samuel Clay
|
468f12e9ff
|
Adding automatic updating of feed counts even on current feeds. Also updating feeds in real-time when refresh is forced.
|
2010-08-30 22:42:44 -04:00 |
|
Samuel Clay
|
cea3bda090
|
Minor bug fixes around error catching in feed exceptions, drag-and-drop on top-level feeds, and styling of feed title buttons.
|
2010-08-30 19:57:27 -04:00 |
|
Samuel Clay
|
25c0450020
|
Commenting out map/reduce statistics because mongo keeps crashing.
|
2010-08-30 13:33:29 -04:00 |
|
Samuel Clay
|
c9d06f9358
|
Compressing stories. Also fixing compression of feed_pages bootstrap.
|
2010-08-29 13:23:50 -04:00 |
|
Samuel Clay
|
060f63f8a0
|
Myriad number of bug fixes to reduce the number of exception e-mails I get. Forbidden JSON requests now send 403s. Adding a URl no longer uses a partial URL saearch on feed_link (which would link usetoday.com -> usatoday.com/sports.xml). Also adding the model for FeedPage for mongo.
|
2010-08-27 18:35:33 -04:00 |
|
Samuel Clay
|
16cea3e174
|
Merging master
|
2010-08-25 19:37:07 -04:00 |
|
Samuel Clay
|
5c48d0fa94
|
Feeds that fetch successfully no longer have an exception.
|
2010-08-25 19:22:53 -04:00 |
|
Samuel Clay
|
98489a0057
|
Also fixing feeds that suffer from non-xml-ity.
|
2010-08-25 19:13:28 -04:00 |
|
Samuel Clay
|
49d8b6d505
|
Huge fix for exception feeds. Now tries to fetch the correct URL from the feed_link or the feed_address (if the rss URL is actually HTML), and uses that to figure out the right feed. Also merges feeds if a duplicate is found during this process. Really nifty. Let's hope nobody dies.
|
2010-08-25 19:10:55 -04:00 |
|
Samuel Clay
|
fc16a1b8a9
|
Small refactor on names and amalgamated methods.
|
2010-08-23 09:55:21 -04:00 |
|
Samuel Clay
|
d03b4b8e69
|
Unicode bug on html rewriting.
|
2010-08-23 07:58:09 -04:00 |
|
Samuel Clay
|
79d1ffd5db
|
Holy hell, the Mongo transition is nearly complete. All that's left is classifiers. This includes stories, user_stories, and calculating unread counts.
|
2010-08-21 23:49:36 -04:00 |
|
Samuel Clay
|
b1dfdb1239
|
Merge branch 'master' of github.com:samuelclay/NewsBlur into mongo
|
2010-08-21 20:42:51 -04:00 |
|
Samuel Clay
|
ac53f33788
|
Further Mongo work. Stories are now saved correctly, with tags + authors. Still need to do UserStories.
|
2010-08-21 20:42:38 -04:00 |
|
Samuel Clay
|
fa3be28b43
|
Initial stab at MongoDB in feed fetching and feed viewing. Still need to fix authors, tags, counts, aggregations, and any remaining bugs.
|
2010-08-21 13:57:49 -04:00 |
|
Samuel Clay
|
a3025a2df7
|
Back to square 1 on optimizing the DB queries.
|
2010-08-19 23:18:11 -04:00 |
|
Samuel Clay
|
fc18a1f19c
|
Backwards. Just use the GUIDs of stories. This may lead to dupe-stories when the GUID changes. Ugh.
|
2010-08-19 22:45:44 -04:00 |
|