Commit graph

1753 commits

Author SHA1 Message Date
Samuel Clay
a5229ce5cb Backfilling Youtube videos for premium archive subscribers. 2024-11-04 21:32:15 -08:00
Samuel Clay
f4d559187b Handling no content when finding feeds. 2024-11-01 10:16:11 -07:00
Samuel Clay
5c1adfd403 Don't guess RSS feed urls on openrss/feedburner domains. 2024-10-31 10:52:44 -07:00
Samuel Clay
cdaec2487f Adding youtube feeds to PuSH. 2024-09-10 06:41:57 -07:00
Samuel Clay
be9bbbd349 Handling missing youtube duration. 2024-07-11 11:01:09 -04:00
Samuel Clay
16cbf30e60 Fixing strip underscore 2024-07-11 10:59:42 -04:00
Samuel Clay
84d0e4545e Don't use the underscore cache buster anymore. 2024-07-03 12:10:19 -04:00
Samuel Clay
546f338b8c Strip underscores from feed addresses, special case jwz 2024-06-30 12:13:29 -04:00
Samuel Clay
7f045ba535 Thresholding bounced newsletters email. 2024-06-03 09:25:21 -04:00
Samuel Clay
7165aa1bf6 Black formatting and isort 2024-04-24 09:50:42 -04:00
Samuel Clay
d1dafe7606 Black formatting. 2024-04-24 09:43:56 -04:00
Samuel Clay
f552d6f2c7 Adding support for hyphens in youtube usernames. 2024-04-01 08:33:45 -04:00
Samuel Clay
65e51ac6ec Handling broken youtube channel 2024-03-04 09:24:23 -05:00
Samuel Clay
0181a2e4b0 Youtube username/title 2024-03-04 09:21:00 -05:00
Samuel Clay
7d08632e9c Handling youtube usernames that are actually handles. 2024-03-03 13:09:59 -05:00
Samuel Clay
a5222d9675 Updating youtube fetcher to use channels/playlists/users for everything, no longer relying on RSS/xml url. 2024-03-03 12:59:12 -05:00
Samuel Clay
ec44039cc9 Updating certs. 2024-03-02 08:37:20 -05:00
Samuel Clay
fe708196ee Tailing logs needs a bit more width for new task servers. 2024-02-18 08:23:41 -05:00
Samuel Clay
7bd7d4ba9d Tailing logs needs a bit more width for new task servers. 2024-02-18 08:23:21 -05:00
Samuel Clay
0e71bfd27a Rewriting tlnb command to handle naming servers. 2024-02-10 15:01:28 -05:00
Samuel Clay
f9ad93e336 Adding hetzner servers. 2024-01-07 10:00:39 -05:00
Samuel Clay
ea22256c59 Increasing apns payload size close to the max of 4kb. 2023-05-10 07:26:22 -04:00
Samuel Clay
8adc2035ea Also handling time out in feedparser. 2023-05-09 09:35:56 -04:00
Samuel Clay
792ba9d204 Removing fetch user agent on timeout as well as 400 errors. 2023-05-09 09:31:09 -04:00
Samuel Clay
3de497d57e Bumping log colors. 2023-04-30 18:12:11 -04:00
Samuel Clay
9c49e0d5fe New twitter auth error. 2023-04-04 09:36:40 -04:00
Samuel Clay
6095bd709d Fixing relative image urls to be absolute urls. Looks like a BeautifulSoup4 upgrade didn't catch the new attrs syntax. Also fixing bookmarklet loading wrong JS/CSS in development. 2022-11-24 12:53:40 -05:00
Samuel Clay
855a96d633 Fixing NYPost feed by ensuring enclosures with href attrbiutes are added to stories even without content. Thanks @kleinmatic! 2022-11-23 12:21:46 -05:00
Samuel Clay
f93eb2520c Adding file size and archive count to rss feed fetch status board. 2022-09-11 22:13:54 -04:00
Samuel Clay
57cf284f13 Adding slow log aggregator to uncover which users and paths are slowest. 2022-09-11 21:37:02 -04:00
Samuel Clay
e83a8e5a9b Revert "Bumping min to fetch by 1.5X to root out 12 minute broken feed. If this succeeds, narrow down the min_to_decay and then find the feed."
This reverts commit fd16bb0b73.
2022-09-11 08:45:29 -04:00
Samuel Clay
fd16bb0b73 Bumping min to fetch by 1.5X to root out 12 minute broken feed. If this succeeds, narrow down the min_to_decay and then find the feed. 2022-09-11 04:54:35 -04:00
Samuel Clay
cdf5a36b81 Don't delete redis keys because they take time to rebuild and subs can be counted incorrectly during that time. 2022-08-10 17:45:25 -04:00
Samuel Clay
fa77ff6a27 Also scrub twitter authors for xml invalid chars. 2022-07-02 13:11:25 -04:00
Samuel Clay
b26817ba77 Refactoring xml serialization. 2022-07-02 13:09:37 -04:00
Samuel Clay
80a1d57944 Remove unserializable control characters from tweets. 2022-07-02 12:58:30 -04:00
Samuel Clay
63635f42e1 Replacing @username's in tweets with links. 2022-07-01 17:00:22 -04:00
Samuel Clay
fa78ec335a Handling missing feed 2022-07-01 13:37:55 -04:00
Samuel Clay
3cf7f1e7c8 Need a feed to process. 2022-07-01 13:32:33 -04:00
Samuel Clay
1d1859db7d Handling missing feed on exception retry. 2022-07-01 13:29:49 -04:00
Samuel Clay
d4833a0d56 Updating redis query log to include response time as well. Huge improvement in redis monitoring. 2022-06-24 12:30:29 -04:00
Samuel Clay
4fe6110e96 Moving redis introspection to send_command, not just pack_command 2022-06-23 17:57:31 -04:00
Samuel Clay
81f51f1723 Handling soft time limits while fetching archives. 2022-06-22 17:40:16 -04:00
Samuel Clay
1665dc73a1 Don't skip pages on paged archives. 2022-06-22 14:49:31 -04:00
Samuel Clay
6c86bb479a Using correct url for rfc5005. 2022-06-22 12:11:52 -04:00
Samuel Clay
295bd65add Adding Twitter search. 2022-06-21 16:07:12 -04:00
Samuel Clay
6d5c1f29d5 Merge branch 'master' into pro
* master:
  Substack now uses a <text> element, so we need to add it to the sanitizer.
2022-06-07 09:36:18 -07:00
Samuel Clay
a269abc96d Substack now uses a <text> element, so we need to add it to the sanitizer. 2022-06-07 09:36:13 -07:00
Samuel Clay
9899816196 Fixing starting page for archives. 2022-05-24 11:40:49 -04:00
Samuel Clay
000c9b0954 Adding starting page for archive to help debug. 2022-05-24 11:37:54 -04:00