Daniel Aleksandersen
19a24d8b7e
Support HTTP 308 from RFC 7238
2017-08-16 09:17:23 +02:00
Samuel Clay
d1e9571093
Future dated stories get forced to current date. This fixed multiple issues: makring as read on oldest first only marks stories older than current date as read. Also fixes issues with publishers gaming the dates.
2017-08-04 13:31:03 -07:00
Samuel Clay
3bcfba0a9b
Fixing YouTube Playlists.
2017-07-27 17:14:15 -07:00
Samuel Clay
9caeb9822c
Twitter fetcher should include all images.
2017-06-12 17:21:09 -07:00
Samuel Clay
ad145d2b21
Loop through twitter usersubs until working api is found.
2017-05-24 21:08:06 -07:00
Samuel Clay
717ab6248f
Loop through twitter usersubs until working api is found.
2017-05-24 21:07:22 -07:00
Samuel Clay
8d14dcf008
Using JSON feed external url if available.
2017-05-23 23:12:10 -07:00
Samuel Clay
c1834703d9
Adding support for JSON Feeds.
2017-05-22 16:46:56 -07:00
Samuel Clay
3b36b4eec5
Better looking twitter dates to match stories. Too bad about timezones.
2017-05-07 16:22:07 -07:00
Samuel Clay
9af256a387
Fixing twitter dates, adding Retweeting author to twitter stories, and adding quoted tweets into retweets. Twitter support is so much better now.
2017-05-07 15:40:44 -07:00
Samuel Clay
3c1ecbb0fd
Forgot to handle missing twitter api.
2017-05-06 19:50:22 -07:00
Samuel Clay
6bc9a55bfc
Rewriting twitter fetching. Now fetching truncated text, embedding quoted tweets, and fixing URLs so that they show the display url (and not t.co) and link to the expanded url.
2017-05-06 19:38:36 -07:00
Samuel Clay
85355a5c9c
Tweets now show full, untruncated text. They also expand all URLs.
2017-05-06 17:52:31 -07:00
Samuel Clay
44de405195
Only fetch as many guidas as there are stories in the feed. No need to get everything assuming the feed has the latest N stories.
2017-05-01 12:06:56 -07:00
Samuel Clay
08c897e5a1
This is it, the big kahuna of fixes. This corrects for messed up guids that are causing lots of read stories to become unread.
2017-05-01 11:39:24 -07:00
Samuel Clay
b753cf6819
This won't fix it but in the event of missing guids on feed fetch, check for stories within the mark read period.
2017-05-01 10:51:04 -07:00
Samuel Clay
c9326a6f02
Perhaps this is the way to find the missing story hashes. Shouldn't cause an issue, but logic for dates may be backwards.
2017-05-01 09:27:31 -07:00
Samuel Clay
ffeeb170e0
Finally have a test case for the Google Blog duping.
2017-04-30 18:47:10 -07:00
Samuel Clay
de59c9471c
Don't add images in media enclosures if the URL is found in the story already.
2017-04-21 11:42:38 -07:00
Samuel Clay
9d9a5accbd
Better handling of tweepy errors.
2017-04-13 11:43:39 -07:00
Samuel Clay
d84e2af636
New experimental data collector for debugging feeds over time.
2017-04-12 19:13:33 -07:00
Samuel Clay
557438008b
Fixing a couple Twitter and YouTube fetch issues.
2017-04-12 11:19:42 -07:00
Samuel Clay
461c1c4b65
Changing feed log format to include id at the beginning. Also normalizing all feed titles in logs for better searchability.
2017-03-31 19:52:24 -07:00
Samuel Clay
937a67f67c
Only using active twitter accounts.
2017-03-24 13:04:38 -07:00
Samuel Clay
26f9456c2e
Monkey patching SSL for new python, since hostnames don't match with S3.
2017-03-23 14:07:59 -07:00
Samuel Clay
4f8a1c43b3
Fixing attribution in twitter RTs.
2017-03-15 19:40:43 -07:00
Samuel Clay
a6fdf45d33
Adding support for native RTs in Twitter.
2017-03-15 19:26:39 -07:00
Samuel Clay
937a1fa999
Faking user agent when rss bans NewsBlur.
2017-02-17 12:19:55 -08:00
Samuel Clay
255b30f3c1
Better recording of non-200/300 status codes on feed fetch.
2017-02-17 12:05:27 -08:00
Samuel Clay
2954ef7318
Adding new debug to feed fetcher.
2017-02-17 11:45:43 -08:00
Samuel Clay
1b35039535
Redis 3.2.
2017-01-11 20:13:04 -08:00
Samuel Clay
2aa82a4eef
Updating redis
2017-01-11 19:12:00 -08:00
Samuel Clay
c5d5ea5001
Attempting to fix dashboard stories by no longer relying on full river reloads when a new story comes in but to fetch that story if it is newer (or older) than last visible dashboard story.
2016-12-13 16:29:42 -08:00
Samuel Clay
046f97558b
Checking for memory errors while feed fetching.
2016-12-11 18:59:02 -08:00
Samuel Clay
d1e2e3e1d9
Handling MemoryError while feed fetching.
2016-12-11 18:29:00 -08:00
Samuel Clay
b5e0870fa8
Merge pull request #958 from da2x/patch-1
...
HTTP feed fetcher improvements
2016-11-25 13:02:44 -06:00
Samuel Clay
0f7c17eb9f
Fixing twitter error message.
2016-11-21 17:23:53 -08:00
Samuel Clay
6698967021
Handling twitter issues.
2016-11-21 13:15:50 -08:00
Samuel Clay
d9676d64e7
Fixing tweepy error handling.
2016-11-20 13:43:49 -08:00
Samuel Clay
56b3dfa659
More aggregation changes.
2016-11-20 13:11:56 -08:00
Samuel Clay
5dc079efc2
Fixing task times for updated pymongo aggregation>
2016-11-20 13:09:30 -08:00
Samuel Clay
e9d5b39d90
Fixing aggregation on latest pymongo for munin graphs.
2016-11-20 11:14:17 -08:00
Samuel Clay
55dd39d983
Virtualenv for munin.
2016-11-20 11:05:20 -08:00
Samuel Clay
b3aba908d7
Handling twitter fetch issues.
2016-11-18 10:54:46 -08:00
Samuel Clay
055fcbc0ea
Fixing notifications dialog bug.
2016-11-17 11:37:10 -08:00
Samuel Clay
fc6379c0ed
Merge branch 'master' into notifications
...
* master:
Virtualenv redis bgsave monitor.
2016-11-16 18:29:27 -08:00
Samuel Clay
9ffa63ec2b
Max os 3 stories sent per notification group. Also limiting notifications to staff for live test.
2016-11-16 18:29:13 -08:00
Samuel Clay
8f6bfa1c55
Notifications now support hidden stories and focus stories. Web also all done. Just need to build iOS push and iOS notification chooser.
2016-11-16 17:49:43 -08:00
Samuel Clay
0284793b3a
Adding in push notifications for web. Needs filtering.
2016-11-15 20:45:59 -08:00
Samuel Clay
c0299f4143
Merge branch 'master' into notifications
...
* master:
Switching to indexed story hashes instead of story feeds for starred stories.
When adding a feed to a folder, check against the lower case folder name to correct for dupes.
Newsletter params to fix the Skimm
Undoing mistaken date correct.
2016-11-14 16:23:56 -08:00
