Commit graph

423 commits

Author SHA1 Message Date
Slavik Svyrydiuk
753d5fc018 use django.utils.encoding.smart_unicode instead of utils.feed_functions.utf8encode 2015-12-02 06:46:18 +01:00
Samuel Clay
101e8477e0 Handling feeds that go missing. 2015-11-28 13:54:21 -08:00
Samuel Clay
f8775f8d54 Handling more errors when fetching feeds. 2015-11-28 13:48:20 -08:00
Samuel Clay
c72d0d4c06 Turning off annotations on Youtube videos by default. 2015-10-23 15:58:55 -07:00
Samuel Clay
3d92dfd0ad Handling YouTube client errors gracefully. 2015-08-24 14:51:01 -07:00
Samuel Clay
a774cddcb3 Fixing broken sub counts when reids sub count expires. 2015-08-05 13:59:26 -07:00
Samuel Clay
9cd8124419 Combing through all updates to feeds table when fetching feed. 2015-07-22 13:53:20 -07:00
Samuel Clay
da26ecdb46 1/2 Cleaning up feed fetcher's updates to feeds table. Still need to find 6 dupes. 2015-07-22 13:18:54 -07:00
Samuel Clay
45a6243bfd Handling youtube feed brokenness. 2015-07-07 13:40:22 -07:00
Samuel Clay
e446103392 Handling broken youtube feeds. 2015-07-07 13:37:07 -07:00
Samuel Clay
e6fef18976 Fixing redirect history saving. 2015-06-01 18:06:39 -07:00
Samuel Clay
2817181f33 Forcing all 301 redirects to occur 20 times before committing, saving feeds from disappearing. 2015-06-01 17:14:04 -07:00
Samuel Clay
29ca355ef3 Fixing list_id 2015-05-12 08:41:45 -07:00
Samuel Clay
fb4ac3757b Fixing YouTube playlist urls. 2015-05-11 17:49:40 -07:00
Samuel Clay
d22d81859a Adding duration and channel info to youtube videos. 2015-05-08 15:34:35 -07:00
Samuel Clay
23fd3cccc1 Adding channel_id youtube.com video feeds. 2015-05-08 14:50:44 -07:00
Samuel Clay
93cf7e5baf Allowing fullscreen videos for youtube. 2015-05-08 14:17:48 -07:00
Samuel Clay
5eb46a278e YouTube API doesn't always close the iframe tag. Better close it. 2015-04-29 18:07:00 -07:00
Samuel Clay
d7daf21694 Adding another youtube url style: https://www.youtube.com/feeds/videos.xml?user=SciShow 2015-04-29 17:58:44 -07:00
Samuel Clay
bb100fa0d8 Typo in youtube fetch error handler. 2015-04-29 17:50:05 -07:00
Samuel Clay
d7f6fa3928 Fixing two small bugs with youtube fetch. 2015-04-29 17:49:04 -07:00
Samuel Clay
dabd81608d Fixing three youtube issues: broken usernames/urls (non-uploads), 404 pages, and over-expanding content due to fixvid. 2015-04-29 17:16:15 -07:00
Samuel Clay
bdd12cf2ee Allowing both youtube/feeds/base and youtube/feeds/api. 2015-04-29 16:59:05 -07:00
Samuel Clay
367282c934 Fixing expanding youtube iframe. 2015-04-29 16:55:51 -07:00
Samuel Clay
f78db7e17e Using YouTube's dumb permalink. 2015-04-29 16:43:48 -07:00
Samuel Clay
83c7517b35 Changing permalink for youtube videos to original rss permalink. 2015-04-29 16:40:12 -07:00
Samuel Clay
1656762299 Trying new YouTube v3 API feed fetcher. 2015-04-29 16:30:44 -07:00
Samuel Clay
0da661c71b Fixing A&L Daily bug where stories all have the same GUID and permalink. Using permalink or timestamp, whichever is unique across the feed. 2015-02-19 10:39:10 -08:00
Samuel Clay
870f21f51f Boosting time limit on feed fetches. 2014-11-18 13:37:55 -08:00
Samuel Clay
0aaf661da1 Removing feedparser trunk. 2014-11-18 11:08:02 -08:00
Samuel Clay
c20ef4e5d0 Adding updates_off run-time setting for handling large spikes of updates. 2014-05-29 17:53:16 -07:00
Samuel Clay
0fee7af7c6 Moving to stricter story diff check. Using story hashes instead of grab bag by date when checking existing stories. 2014-04-17 12:10:04 -07:00
Samuel Clay
a4efcde236 Allowing feeds to fail fetch. 2014-03-29 17:14:10 -07:00
Samuel Clay
534e16fb84 Handling odd compressed stream error. 2014-03-13 16:53:20 -07:00
Samuel Clay
bb3e0a6567 Putting Google Alerts/News back into rotation. 2013-09-30 12:40:14 -07:00
Samuel Clay
70ea542481 User-dependent days of unread. 14 for free users, 30 for premium. 2013-09-16 16:42:49 -07:00
Samuel Clay
831a771309 Fixing logging message. 2013-09-16 12:11:26 -07:00
Samuel Clay
886f6c0d8b Fixing logging message. 2013-09-16 12:08:51 -07:00
Samuel Clay
5911acaa35 Moving the majority of story fetches to mongo secondaries. 2013-09-16 12:07:15 -07:00
Samuel Clay
143ee770ce Skipping google news feeds again. Need to tone down these fetches. 2013-09-15 23:41:17 -07:00
Samuel Clay
3fc40acd47 Removing double. 2013-09-13 19:10:41 -07:00
Samuel Clay
2fe65adc4d Re-enabling Google News fetches. 2013-09-11 10:50:20 -07:00
Samuel Clay
00a412960a Skipping google alerts. 2013-09-09 13:50:05 -07:00
Samuel Clay
2b2c731bd5 Fixing a couple exceptions and turning off google news rss feeds temporarily. 2013-09-09 13:46:14 -07:00
Samuel Clay
dec22ac776 Moving to simpler read story endpoint. 2013-09-06 12:42:39 -07:00
Samuel Clay
2394e31f89 Moving feed parsing hotspot to specific seq ratio check. 2013-08-06 13:54:06 -07:00
Samuel Clay
ecf11eed98 Upping process feed time limit. 2013-08-06 13:42:53 -07:00
Samuel Clay
683140af2e Moving feed processing to timelimit due to some feeds sucking up all the time. 2013-08-06 13:38:35 -07:00
Samuel Clay
88f2a69a93 Fixing a dozen text and feed fetching bugs. 2013-08-06 13:18:55 -07:00
Samuel Clay
d49e9569a5 Fixing #377. Killing XSS attacks by sanitizing feed titles. 2013-07-29 14:25:43 -07:00