Samuel Clay
5a60bbd703
Tailing logs needs a bit more width for new task servers.
2024-03-29 10:31:41 -07:00
Samuel Clay
fcc105cf0d
Tailing logs needs a bit more width for new task servers.
2024-03-29 10:31:41 -07:00
Samuel Clay
6b4c3b5a28
Rewriting tlnb command to handle naming servers.
2024-03-29 10:31:41 -07:00
Samuel Clay
f230e6c079
Adding hetzner servers.
2024-03-29 10:31:40 -07:00
Samuel Clay
ea22256c59
Increasing apns payload size close to the max of 4kb.
2023-05-10 07:26:22 -04:00
Samuel Clay
8adc2035ea
Also handling time out in feedparser.
2023-05-09 09:35:56 -04:00
Samuel Clay
792ba9d204
Removing fetch user agent on timeout as well as 400 errors.
2023-05-09 09:31:09 -04:00
Samuel Clay
3de497d57e
Bumping log colors.
2023-04-30 18:12:11 -04:00
Samuel Clay
9c49e0d5fe
New twitter auth error.
2023-04-04 09:36:40 -04:00
Samuel Clay
6095bd709d
Fixing relative image urls to be absolute urls. Looks like a BeautifulSoup4 upgrade didn't catch the new attrs syntax. Also fixing bookmarklet loading wrong JS/CSS in development.
2022-11-24 12:53:40 -05:00
Samuel Clay
855a96d633
Fixing NYPost feed by ensuring enclosures with href attrbiutes are added to stories even without content. Thanks @kleinmatic!
2022-11-23 12:21:46 -05:00
Samuel Clay
f93eb2520c
Adding file size and archive count to rss feed fetch status board.
2022-09-11 22:13:54 -04:00
Samuel Clay
57cf284f13
Adding slow log aggregator to uncover which users and paths are slowest.
2022-09-11 21:37:02 -04:00
Samuel Clay
e83a8e5a9b
Revert "Bumping min to fetch by 1.5X to root out 12 minute broken feed. If this succeeds, narrow down the min_to_decay and then find the feed."
...
This reverts commit fd16bb0b73 .
2022-09-11 08:45:29 -04:00
Samuel Clay
fd16bb0b73
Bumping min to fetch by 1.5X to root out 12 minute broken feed. If this succeeds, narrow down the min_to_decay and then find the feed.
2022-09-11 04:54:35 -04:00
Samuel Clay
cdf5a36b81
Don't delete redis keys because they take time to rebuild and subs can be counted incorrectly during that time.
2022-08-10 17:45:25 -04:00
Samuel Clay
fa77ff6a27
Also scrub twitter authors for xml invalid chars.
2022-07-02 13:11:25 -04:00
Samuel Clay
b26817ba77
Refactoring xml serialization.
2022-07-02 13:09:37 -04:00
Samuel Clay
80a1d57944
Remove unserializable control characters from tweets.
2022-07-02 12:58:30 -04:00
Samuel Clay
63635f42e1
Replacing @username's in tweets with links.
2022-07-01 17:00:22 -04:00
Samuel Clay
fa78ec335a
Handling missing feed
2022-07-01 13:37:55 -04:00
Samuel Clay
3cf7f1e7c8
Need a feed to process.
2022-07-01 13:32:33 -04:00
Samuel Clay
1d1859db7d
Handling missing feed on exception retry.
2022-07-01 13:29:49 -04:00
Samuel Clay
d4833a0d56
Updating redis query log to include response time as well. Huge improvement in redis monitoring.
2022-06-24 12:30:29 -04:00
Samuel Clay
4fe6110e96
Moving redis introspection to send_command, not just pack_command
2022-06-23 17:57:31 -04:00
Samuel Clay
81f51f1723
Handling soft time limits while fetching archives.
2022-06-22 17:40:16 -04:00
Samuel Clay
1665dc73a1
Don't skip pages on paged archives.
2022-06-22 14:49:31 -04:00
Samuel Clay
6c86bb479a
Using correct url for rfc5005.
2022-06-22 12:11:52 -04:00
Samuel Clay
295bd65add
Adding Twitter search.
2022-06-21 16:07:12 -04:00
Samuel Clay
6d5c1f29d5
Merge branch 'master' into pro
...
* master:
Substack now uses a <text> element, so we need to add it to the sanitizer.
2022-06-07 09:36:18 -07:00
Samuel Clay
a269abc96d
Substack now uses a <text> element, so we need to add it to the sanitizer.
2022-06-07 09:36:13 -07:00
Samuel Clay
9899816196
Fixing starting page for archives.
2022-05-24 11:40:49 -04:00
Samuel Clay
000c9b0954
Adding starting page for archive to help debug.
2022-05-24 11:37:54 -04:00
Samuel Clay
e6cf58a1d8
Boosting archive to 150 pages back.
2022-05-24 11:36:16 -04:00
Samuel Clay
b0a50f1834
Turning up the full archive.
2022-05-24 11:32:58 -04:00
Samuel Clay
8a5748fa37
Adding # of stories per dashboard module. Also fixing long-standing double load bug on initial page load. Fixed issues around phantom read stories popping in and out of existance on page load.
2022-05-16 17:21:46 -04:00
Samuel Clay
3a106b51bb
Merge branch 'master' into pro
...
* master:
Don't change file permissions, let's see what happens.
Bumping postgres server size, fixing monitor disk usage cronjob.
2022-05-16 10:34:40 -04:00
Samuel Clay
b2cdff8a85
Bumping postgres server size, fixing monitor disk usage cronjob.
2022-05-13 16:59:57 -04:00
Samuel Clay
bfdfa10bfd
Adding file system size (bytes) for feeds, decompressing stories to figure out their true size. Needs to be integrated into the UI.
2022-05-11 15:00:23 -04:00
Samuel Clay
9bb9700cc5
Merge branch 'master' into pro
...
* master:
Handling getMore mongo log.
Naming pg_basebackup
2022-05-11 13:12:06 -04:00
Samuel Clay
6b1d8cef78
Handling getMore mongo log.
2022-05-11 13:09:59 -04:00
Samuel Clay
5ff6a340f1
Merge branch 'master' into pro
...
* master:
Upgrading to latest pymongo v3 to work with mongodb v4.0. Required rewrite of mongodb introspection, now using built-in pymongo monitoring. Should work on both web and celery.
Removing unused mongo mms utils and configs.
Updating pymongo to mongodb 4.0 compatible. Any further and we lose mongo wire protocol inspection.
Full arm64 setup for imageproxy.
Rearranging TF vars
Upgrading to MongoDB 4.0, and upping mongo servers.
2022-05-10 20:59:03 -04:00
Samuel Clay
31de0f7a97
Upgrading to latest pymongo v3 to work with mongodb v4.0. Required rewrite of mongodb introspection, now using built-in pymongo monitoring. Should work on both web and celery.
2022-05-10 20:58:47 -04:00
Samuel Clay
7870eee728
Merge branch 'master' into pro
...
* master:
Moving pg basebackup to a backup role
Adding a secondary postgres that starts from a pg_basebackup. Still needs testing.
Adding timestamps to backup logs
No TTY/input needed on backups.
Paths are in different environments.
Renaming postgresql backups
Adding postgres restore command and updating backup script.
2022-05-02 11:42:52 -04:00
Samuel Clay
1dc0e98448
Renaming postgresql backups
2022-04-29 16:53:47 -04:00
Samuel Clay
464fecd18e
Adding postgres restore command and updating backup script.
2022-04-29 16:45:17 -04:00
Samuel Clay
2d2ed0a30d
Adding postgres restore command and updating backup script.
2022-04-29 16:44:46 -04:00
Samuel Clay
de1fb57406
Finishing RFC 5005 archive fetcher (for #1109 ). Also fixing mongo commands and adding checks for archive subscribers.
2022-04-29 11:31:32 -04:00
Samuel Clay
4f69d15fcb
RFC5005 fetching should be working, but the test site is changing urls to a weird http://x.com:443 , so maybe we need to normalize the url?
2022-04-20 22:38:51 -04:00
Samuel Clay
104d50ff46
Use new url_normalize library, preserving old just in case anything changes.
2022-04-19 15:28:18 -04:00