Commit graph

1095 commits

Author SHA1 Message Date
Samuel Clay
2275a4ba6c Sanity check for years. 2015-09-02 15:26:55 -07:00
Samuel Clay
3d92dfd0ad Handling YouTube client errors gracefully. 2015-08-24 14:51:01 -07:00
Samuel Clay
b6e1e03bd5 Upgrading feedparser, cleaing up logging on feed fetch. 2015-08-24 14:27:01 -07:00
Samuel Clay
508864b4cc Adding roles to zgrep util. 2015-08-21 13:12:12 -07:00
Samuel Clay
0ea8da6ce6 Adding log tail to redis bgsave monitor. 2015-08-18 09:00:40 -07:00
Samuel Clay
7e679cdf7a a+x on redis bgsave monitor. 2015-08-18 08:52:37 -07:00
Samuel Clay
80cd1e5a3b Adding redis bgsave monitor. 2015-08-18 08:51:49 -07:00
Samuel Clay
a774cddcb3 Fixing broken sub counts when reids sub count expires. 2015-08-05 13:59:26 -07:00
Samuel Clay
6a20478a7f Renaming redis feed pool to redis feed update pool in prep for redis feed sub pool. 2015-07-27 18:35:25 -07:00
Samuel Clay
9cd8124419 Combing through all updates to feeds table when fetching feed. 2015-07-22 13:53:20 -07:00
Samuel Clay
da26ecdb46 1/2 Cleaning up feed fetcher's updates to feeds table. Still need to find 6 dupes. 2015-07-22 13:18:54 -07:00
Samuel Clay
34d008d600 Adding color to postgres slowlogs. Making login a bit less heavy on the db. And fixing feed count munin. 2015-07-21 15:53:37 -07:00
Samuel Clay
e65cd64e8a Speeding up (and caching) feed/sub counts in munin. 2015-07-21 15:29:21 -07:00
Samuel Clay
66ca1d7628 Changing resoltuion of feed counts munin to 12 hours to spare the database. 2015-07-21 15:15:51 -07:00
Samuel Clay
70c6dc78b2 PostgreSQL 9.2 -> 9.4. 2015-07-20 18:55:41 -07:00
Samuel Clay
8ef0454982 Backtracking on mongoengine and pymongo upgrades. 2015-07-20 17:31:45 -07:00
Samuel Clay
1a578193a4 Updating to latest pymongo and mongoengine. 2015-07-20 16:44:50 -07:00
Samuel Clay
45a6243bfd Handling youtube feed brokenness. 2015-07-07 13:40:22 -07:00
Samuel Clay
e446103392 Handling broken youtube feeds. 2015-07-07 13:37:07 -07:00
Samuel Clay
ec066f54e9 Zgrep shouldnt use dev. 2015-07-07 08:42:15 -07:00
Samuel Clay
e6fef18976 Fixing redirect history saving. 2015-06-01 18:06:39 -07:00
Samuel Clay
baf0abae85 Fixing tlnb to allow single hostnames. 2015-06-01 17:50:06 -07:00
Samuel Clay
2817181f33 Forcing all 301 redirects to occur 20 times before committing, saving feeds from disappearing. 2015-06-01 17:14:04 -07:00
Samuel Clay
a266d9f585 Adding DF missing users. 2015-05-26 17:58:45 -07:00
Samuel Clay
bd4e3c322e Fixing delete feeds recreator. 2015-05-26 17:57:26 -07:00
Samuel Clay
29ca355ef3 Fixing list_id 2015-05-12 08:41:45 -07:00
Samuel Clay
fb4ac3757b Fixing YouTube playlist urls. 2015-05-11 17:49:40 -07:00
Samuel Clay
d22d81859a Adding duration and channel info to youtube videos. 2015-05-08 15:34:35 -07:00
Samuel Clay
23fd3cccc1 Adding channel_id youtube.com video feeds. 2015-05-08 14:50:44 -07:00
Samuel Clay
93cf7e5baf Allowing fullscreen videos for youtube. 2015-05-08 14:17:48 -07:00
Samuel Clay
5eb46a278e YouTube API doesn't always close the iframe tag. Better close it. 2015-04-29 18:07:00 -07:00
Samuel Clay
d7daf21694 Adding another youtube url style: https://www.youtube.com/feeds/videos.xml?user=SciShow 2015-04-29 17:58:44 -07:00
Samuel Clay
bb100fa0d8 Typo in youtube fetch error handler. 2015-04-29 17:50:05 -07:00
Samuel Clay
d7f6fa3928 Fixing two small bugs with youtube fetch. 2015-04-29 17:49:04 -07:00
Samuel Clay
dabd81608d Fixing three youtube issues: broken usernames/urls (non-uploads), 404 pages, and over-expanding content due to fixvid. 2015-04-29 17:16:15 -07:00
Samuel Clay
bdd12cf2ee Allowing both youtube/feeds/base and youtube/feeds/api. 2015-04-29 16:59:05 -07:00
Samuel Clay
367282c934 Fixing expanding youtube iframe. 2015-04-29 16:55:51 -07:00
Samuel Clay
f78db7e17e Using YouTube's dumb permalink. 2015-04-29 16:43:48 -07:00
Samuel Clay
83c7517b35 Changing permalink for youtube videos to original rss permalink. 2015-04-29 16:40:12 -07:00
Samuel Clay
1656762299 Trying new YouTube v3 API feed fetcher. 2015-04-29 16:30:44 -07:00
Samuel Clay
9467755349 Reimporting stripe history. 2015-04-16 19:29:11 -07:00
Samuel Clay
487bc4afb0 Fixing image size calculator for new Twitter Cards implementation. 2015-04-07 10:47:08 -07:00
Samuel Clay
275551e396 Adding user agents to logging. Reeder, Fever, Superfeedr, Old Reader, IE11. 2015-03-11 10:55:17 -07:00
Samuel Clay
19fba9cbd8 Adding modern android app user agent in logging. 2015-03-11 10:47:57 -07:00
Samuel Clay
d6e6e6410c Fixing backups. 2015-03-10 12:02:23 -07:00
Samuel Clay
17412b6277 Adding redis sessions backups. Also adding crontabs for backups directly to fabfile and db server creation. 2015-03-09 14:50:07 -07:00
Samuel Clay
159db6090e Using folders for backups. 2015-03-09 14:27:33 -07:00
Samuel Clay
0da661c71b Fixing A&L Daily bug where stories all have the same GUID and permalink. Using permalink or timestamp, whichever is unique across the feed. 2015-02-19 10:39:10 -08:00
Samuel Clay
bdea41f131 Fixing pythonpath for disk monitor. 2015-02-11 17:37:41 -08:00
Samuel Clay
afee20edca Adding disk usage monitor to all database servers. 2015-02-11 17:30:15 -08:00