Commit graph

1742 commits

Author SHA1 Message Date
Samuel Clay
abfacef9c9 Splitting redis query logging into three dbs: user, story, and session. 2021-08-03 16:05:48 -04:00
Samuel Clay
607da1a524 Executable backups for mongo 2021-07-30 16:07:35 -04:00
Samuel Clay
07e3292ed6 Changing datetime format for mongo backups 2021-07-29 18:22:58 -04:00
Samuel Clay
535497a1b0 Backing up mongo on db-mongo-secondary1 2021-07-29 18:21:00 -04:00
Samuel Clay
5aaf4d5c38 Rewriting mongo backup, adding in rotation. 2021-07-29 18:17:02 -04:00
Samuel Clay
d7c8f5f79c Fixing author name in json feeds. 2021-07-29 17:27:53 -04:00
Samuel Clay
6286b798a7 Fixing JSON feeds. 2021-07-29 17:25:09 -04:00
Samuel Clay
9be6db2027 Fixing move folder to folder, need to strip river: prefix. 2021-07-29 16:24:44 -04:00
Samuel Clay
b57b70a9df Better exception handling 2021-07-23 19:06:00 -04:00
Samuel Clay
ea094cc4d6 Remove unused s3 bucket code. 2021-07-23 19:05:05 -04:00
Samuel Clay
ad9b805d0a Updating backup to use boto3 2021-07-23 19:04:30 -04:00
Samuel Clay
37451fbafd Mongo backup exec line. 2021-07-23 19:01:06 -04:00
Samuel Clay
286809ace7 Backup permissions 2021-07-16 12:50:13 -04:00
Samuel Clay
047c86f8c8 Don't strip iframes from RSS. 2021-07-16 11:20:59 -04:00
Samuel Clay
45846d0d59 Shortening post parameters when debugging. 2021-07-15 18:59:30 -04:00
Samuel Clay
4170505f1e Widening significant figures 2021-07-15 17:07:21 -04:00
Samuel Clay
eb47c17905 Higher thresholds 2021-07-15 17:05:58 -04:00
Samuel Clay
bfce5b8dc9 Adding colors to db logging. 2021-07-15 17:03:51 -04:00
Samuel Clay
c055656869 Adding DEBUG_QUERIES_SUMMARY_ONLY to assist db introspection without the verbosity. 2021-07-15 16:50:54 -04:00
Samuel Clay
acd41d8d48 Fixing mongo auth on monitor. 2021-07-14 22:09:47 -04:00
Samuel Clay
4ab47b7b42 Found more instances of mongodb changing timestamps 2021-07-14 22:00:25 -04:00
Samuel Clay
dce7ece0aa Newsletters bouncing looks good. 2021-07-14 11:11:44 -04:00
Samuel Clay
154cfc2dca Fixing string interpolation 2021-07-12 23:21:59 -04:00
Samuel Clay
4458994e1c Using admin auth source for mongodb. 2021-07-12 22:01:07 -04:00
Samuel Clay
4dd3bafbfb Adding authSource to mongodb auth 2021-07-12 21:53:16 -04:00
Samuel Clay
514258df76 Merge branch 'master' into mongoauth
* master:
  8 app-refresh servers
  Adding volume for metrics server.
  Adding imageproxy2 before switching image proxies.
  Ignoring symlink requirements.txt.
2021-07-12 21:32:49 -04:00
Samuel Clay
f437f810e4 Adding volume for metrics server. 2021-07-12 11:46:35 -04:00
Samuel Clay
0754c0c049 Adding authentication for mongo. Hopefully this covers everything needed. 2021-07-12 11:45:25 -04:00
Samuel Clay
f888e8a204 Only disconnecting twitter on expired token. 2021-07-09 12:46:07 -04:00
Samuel Clay
55adec1a74 Fixing zgrep to search python3 logs 2021-07-09 12:31:52 -04:00
Samuel Clay
f2e64ca36e Fixing sanity checkers. None of them were running. Turns out they needed to be shell scripts if in the /etc/cron.hourly dir (and crontab format if in the /etc/cron.d dir). 2021-07-08 15:39:29 -04:00
Samuel Clay
2fea152441 No need to decode twice. 2021-07-05 12:57:21 -04:00
Samuel Clay
8d11cd68f1 Better normalization of feeds with feed:// in the protocol. 2021-07-05 12:54:23 -04:00
Samuel Clay
87b25c6bf6 Adding more granularity to mongo queries. Fixing double query bug. 2021-07-02 16:12:14 -04:00
Samuel Clay
14e3d6dd25 Fixing broken requirements.txt, downgrading pymongo to 3.6, since monitoring was changed and we're not ready for it yet. 2021-07-02 12:42:02 -04:00
Jonathan Math
18e43fd7f1 rm utils/test_settings.py. It is never used 2021-06-29 08:39:37 -06:00
Jonathan Math
0e81d2f346 no need to pass filename into backup_psql.py in ansible postgres backup. removing it 2021-06-27 14:40:00 -06:00
Jonathan Math
a78dec3ed9 fix syntax for backup_redis and duse socket for getting hostname instead of passing in as an argument 2021-06-27 14:36:24 -06:00
Jonathan Math
4846457bc4 fix syntax for mongo backups 2021-06-27 14:32:37 -06:00
Jonathan Math
8ede5a3987 change get_postgres_credentials.py to get_credentials.py so it can also be used to get the s3 bucket name. Fix postgres backups 2021-06-27 14:11:14 -06:00
Jonathan Math
4b257bce00 remove syntax error from redis bgsave sanity checker 2021-06-27 13:07:32 -06:00
Jonathan Math
fb1cdc9689 remove unused subprocess imports from sanity checkers 2021-06-25 11:41:29 -06:00
Samuel Clay
ae95b25910 Cleaning up monitors, switching to cron file, adding hostname. 2021-06-24 11:50:38 -04:00
Samuel Clay
f56acc1360 Enabling feed fetch monitor. 2021-06-24 11:24:59 -04:00
Samuel Clay
1d5173e7bc Fixing monitor task fetches 2021-06-24 11:19:45 -04:00
Samuel Clay
96ccc7bdd9 Turning off work queue checker from firing all the time. 2021-06-24 11:12:52 -04:00
Jonathan Math
871947a849 fix disk usage sanity checker 2021-06-22 09:48:16 -06:00
Jonathan Math
4cea718d40 add redis backup and remove redundant redis backup scripts. Use one script and pass in hostname argument instead 2021-06-19 12:52:04 -06:00
Jonathan Math
c2bef5565b add backup to ansible for postgres 2021-06-19 12:35:28 -06:00
Jonathan Math
97bc306567 add python paths to backup_mongo.py and backup_psql.py to make them executable 2021-06-19 12:24:33 -06:00