1742 commits

Author	SHA1	Message	Date
Samuel Clay	7b9f857c84	Correcting error log.	2018-03-26 19:16:33 -07:00
Samuel Clay	4219c56f43	Handling more Facebook errors.	2018-03-26 19:15:40 -07:00
Samuel Clay	e4b43ff592	Using vendorzed facebook lib.	2018-03-26 19:13:30 -07:00
Samuel Clay	68ea7b2fcd	Handling session error.	2018-03-26 19:12:33 -07:00
Samuel Clay	471d5434c6	Facebook pages feed gets the correct favicon.	2018-03-26 19:05:22 -07:00
Samuel Clay	6a5a2820ba	Better handling of attachments and subattachments in facebook posts (image galleries).	2018-03-26 18:54:50 -07:00
Samuel Clay	da4741a3fa	Adding videos to facebook posts. Seems to work pretty well.	2018-03-26 17:31:12 -07:00
Samuel Clay	fdce59bc2c	Stubbing in new facebook pages fetcher.	2018-03-20 17:51:38 -07:00
Samuel Clay	364ff9dc85	Fixing YouTube to not verify SSL certs.	2018-03-09 09:44:05 -08:00
Samuel Clay	f2fc30f0c0	Updating munin graph for app loads.	2018-02-26 16:09:59 -08:00
Samuel Clay	f9d9bce5ec	Saving page load stats for munin graphs.	2018-02-26 16:00:44 -08:00
Samuel Clay	95366bcf99	Stringifying JSON feed ids (in case they're integers).	2018-02-13 00:02:25 -08:00
Samuel Clay	895869b334	Better sanitization of tweets.	2018-01-18 16:12:59 -08:00
Samuel Clay	e69965ce19	Handling RSS feed for tweets when there are no tweets.	2018-01-18 16:11:39 -08:00
thinkpoop	17df09b65b	Add "reply-to-self" tags to tweets ... Add "reply-to-self" tags to tweets where the author and "in reply to" user have the same id	2017-12-26 16:35:16 -06:00
Daniel Aleksandersen	19a24d8b7e	Support HTTP 308 from RFC 7238	2017-08-16 09:17:23 +02:00
Samuel Clay	d1e9571093	Future dated stories get forced to current date. This fixed multiple issues: makring as read on oldest first only marks stories older than current date as read. Also fixes issues with publishers gaming the dates.	2017-08-04 13:31:03 -07:00
Samuel Clay	3bcfba0a9b	Fixing YouTube Playlists.	2017-07-27 17:14:15 -07:00
Samuel Clay	9caeb9822c	Twitter fetcher should include all images.	2017-06-12 17:21:09 -07:00
Samuel Clay	ad145d2b21	Loop through twitter usersubs until working api is found.	2017-05-24 21:08:06 -07:00
Samuel Clay	717ab6248f	Loop through twitter usersubs until working api is found.	2017-05-24 21:07:22 -07:00
Samuel Clay	8d14dcf008	Using JSON feed external url if available.	2017-05-23 23:12:10 -07:00
Samuel Clay	c1834703d9	Adding support for JSON Feeds.	2017-05-22 16:46:56 -07:00
Samuel Clay	3b36b4eec5	Better looking twitter dates to match stories. Too bad about timezones.	2017-05-07 16:22:07 -07:00
Samuel Clay	9af256a387	Fixing twitter dates, adding Retweeting author to twitter stories, and adding quoted tweets into retweets. Twitter support is so much better now.	2017-05-07 15:40:44 -07:00
Samuel Clay	3c1ecbb0fd	Forgot to handle missing twitter api.	2017-05-06 19:50:22 -07:00
Samuel Clay	6bc9a55bfc	Rewriting twitter fetching. Now fetching truncated text, embedding quoted tweets, and fixing URLs so that they show the display url (and not t.co) and link to the expanded url.	2017-05-06 19:38:36 -07:00
Samuel Clay	85355a5c9c	Tweets now show full, untruncated text. They also expand all URLs.	2017-05-06 17:52:31 -07:00
Samuel Clay	44de405195	Only fetch as many guidas as there are stories in the feed. No need to get everything assuming the feed has the latest N stories.	2017-05-01 12:06:56 -07:00
Samuel Clay	08c897e5a1	This is it, the big kahuna of fixes. This corrects for messed up guids that are causing lots of read stories to become unread.	2017-05-01 11:39:24 -07:00
Samuel Clay	b753cf6819	This won't fix it but in the event of missing guids on feed fetch, check for stories within the mark read period.	2017-05-01 10:51:04 -07:00
Samuel Clay	c9326a6f02	Perhaps this is the way to find the missing story hashes. Shouldn't cause an issue, but logic for dates may be backwards.	2017-05-01 09:27:31 -07:00
Samuel Clay	ffeeb170e0	Finally have a test case for the Google Blog duping.	2017-04-30 18:47:10 -07:00
Samuel Clay	de59c9471c	Don't add images in media enclosures if the URL is found in the story already.	2017-04-21 11:42:38 -07:00
Samuel Clay	9d9a5accbd	Better handling of tweepy errors.	2017-04-13 11:43:39 -07:00
Samuel Clay	d84e2af636	New experimental data collector for debugging feeds over time.	2017-04-12 19:13:33 -07:00
Samuel Clay	557438008b	Fixing a couple Twitter and YouTube fetch issues.	2017-04-12 11:19:42 -07:00
Samuel Clay	461c1c4b65	Changing feed log format to include id at the beginning. Also normalizing all feed titles in logs for better searchability.	2017-03-31 19:52:24 -07:00
Samuel Clay	937a67f67c	Only using active twitter accounts.	2017-03-24 13:04:38 -07:00
Samuel Clay	26f9456c2e	Monkey patching SSL for new python, since hostnames don't match with S3.	2017-03-23 14:07:59 -07:00
Samuel Clay	4f8a1c43b3	Fixing attribution in twitter RTs.	2017-03-15 19:40:43 -07:00
Samuel Clay	a6fdf45d33	Adding support for native RTs in Twitter.	2017-03-15 19:26:39 -07:00
Samuel Clay	937a1fa999	Faking user agent when rss bans NewsBlur.	2017-02-17 12:19:55 -08:00
Samuel Clay	255b30f3c1	Better recording of non-200/300 status codes on feed fetch.	2017-02-17 12:05:27 -08:00
Samuel Clay	2954ef7318	Adding new debug to feed fetcher.	2017-02-17 11:45:43 -08:00
Samuel Clay	1b35039535	Redis 3.2.	2017-01-11 20:13:04 -08:00
Samuel Clay	2aa82a4eef	Updating redis	2017-01-11 19:12:00 -08:00
Samuel Clay	c5d5ea5001	Attempting to fix dashboard stories by no longer relying on full river reloads when a new story comes in but to fetch that story if it is newer (or older) than last visible dashboard story.	2016-12-13 16:29:42 -08:00
Samuel Clay	046f97558b	Checking for memory errors while feed fetching.	2016-12-11 18:59:02 -08:00
Samuel Clay	d1e2e3e1d9	Handling MemoryError while feed fetching.	2016-12-11 18:29:00 -08:00