Commit graph

595 commits

Author SHA1 Message Date
Samuel Clay
df050acb8e Don't disregard authentication error 2021-07-21 21:27:56 -04:00
Samuel Clay
1ac16508f8 Changing how mongo is registered on consul. Now allows members before they are part of the replica set. 2021-07-21 21:10:02 -04:00
Samuel Clay
4f37953df4 Mongo backups should use a dual-docker solution: 1) Use docker to backup mongo from the mongo container, 2) use newsblur_web container to upload to S3. 2021-07-16 12:25:25 -04:00
Samuel Clay
82e46fe04b Mounting DO volume on mongo machines. 2021-07-16 11:51:52 -04:00
Samuel Clay
5eea1c6b76 Mongo optime format namespaced under 'ts'. 2021-07-14 21:45:12 -04:00
Samuel Clay
a78e21de90 Capturing mongo internal IP for use with db monitor. 2021-07-14 21:37:49 -04:00
Samuel Clay
57bb169629 Turning off auth, need to move to template 2021-07-14 20:09:14 -04:00
Samuel Clay
bf5bf29bfc Turning auth back on 2021-07-14 20:06:42 -04:00
Samuel Clay
d454ab40d4 Removing auth 2021-07-14 20:03:58 -04:00
Samuel Clay
4ddb345e32 Retrying mongodb keyfile auth. 2021-07-14 19:59:00 -04:00
Samuel Clay
925c13bd6e Sanity checker logs needs permissions for writing. 2021-07-14 17:42:13 -04:00
Samuel Clay
be7fe3cf1e
Merge pull request #1517 from samuelclay/fix_sanity_checker_cron
fix sanity checkers for task_worker and move to hourly
2021-07-14 17:37:20 -04:00
Samuel Clay
dce7ece0aa Newsletters bouncing looks good. 2021-07-14 11:11:44 -04:00
Jonathan Math
ea489636c9 fix sanity checkers for task_worker and move to hourly 2021-07-13 15:16:58 -05:00
Samuel Clay
95a447cb1c Turning off keyfile 2021-07-12 22:29:44 -04:00
Samuel Clay
4458994e1c Using admin auth source for mongodb. 2021-07-12 22:01:07 -04:00
Samuel Clay
4dd3bafbfb Adding authSource to mongodb auth 2021-07-12 21:53:16 -04:00
Samuel Clay
514258df76 Merge branch 'master' into mongoauth
* master:
  8 app-refresh servers
  Adding volume for metrics server.
  Adding imageproxy2 before switching image proxies.
  Ignoring symlink requirements.txt.
2021-07-12 21:32:49 -04:00
Samuel Clay
1b2fc53f37 Adding imageproxy2 before switching image proxies. 2021-07-12 11:46:22 -04:00
Samuel Clay
0754c0c049 Adding authentication for mongo. Hopefully this covers everything needed. 2021-07-12 11:45:25 -04:00
Samuel Clay
e11d4db4ea Fixing prometheus to use consul internal dns. 2021-07-10 16:11:04 -04:00
Samuel Clay
5f58d9ccee Better rescue of broken task servers. 2021-07-09 17:39:34 -04:00
Samuel Clay
f2e64ca36e Fixing sanity checkers. None of them were running. Turns out they needed to be shell scripts if in the /etc/cron.hourly dir (and crontab format if in the /etc/cron.d dir). 2021-07-08 15:39:29 -04:00
Samuel Clay
4c5a5907c5 Adding redirects for dashboard refreshes. Also attempting to cache global river for dashboard. 2021-07-02 09:15:22 -04:00
Samuel Clay
dd9de64b29 Updating dashboard3 -> master. 2021-07-01 17:11:11 -04:00
Samuel Clay
f652941687 Adding autoheal to celery label. 2021-06-30 20:41:17 -04:00
Samuel Clay
e160a86d2a Switching celery to local log driver for docker due to logs filling up disk space. 2021-06-30 00:22:09 -04:00
Samuel Clay
17e6f7f424 Adding logrotate for mongo. 2021-06-29 17:51:14 -04:00
Jonathan Math
0e81d2f346 no need to pass filename into backup_psql.py in ansible postgres backup. removing it 2021-06-27 14:40:00 -06:00
Jonathan Math
a78dec3ed9 fix syntax for backup_redis and duse socket for getting hostname instead of passing in as an argument 2021-06-27 14:36:24 -06:00
Jonathan Math
4846457bc4 fix syntax for mongo backups 2021-06-27 14:32:37 -06:00
Jonathan Math
8ede5a3987 change get_postgres_credentials.py to get_credentials.py so it can also be used to get the s3 bucket name. Fix postgres backups 2021-06-27 14:11:14 -06:00
Jonathan Math
baee7a798b add multiline command syntax >- to cronjobs for sanity checkers 2021-06-27 13:24:07 -06:00
Jonathan Math
5cbe285888 run command to get redis log path in cronjob for redis bgsave. This way if the container id changes for any reason, the change will be picked up when the cronjob starts instead of being evaluated only when the cronjob is deployed with ansible 2021-06-25 11:49:38 -06:00
Jonathan Math
dfe72b3e04 add -it to disk usage commands for cronjobs. capitalize OUTPUT in disk usage commands 2021-06-25 11:46:09 -06:00
Jonathan Math
972f13b394 edit ansible files so that all disk usage sanity checkers run hourly 2021-06-25 11:42:21 -06:00
Samuel Clay
cea069d842 Fixing redis sanity checker. 2021-06-24 13:21:25 -04:00
Samuel Clay
57c2713029 Fixing redis bgsave and disk checker monitors. 2021-06-24 11:54:56 -04:00
Samuel Clay
ae95b25910 Cleaning up monitors, switching to cron file, adding hostname. 2021-06-24 11:50:38 -04:00
Samuel Clay
1d5173e7bc Fixing monitor task fetches 2021-06-24 11:19:45 -04:00
Samuel Clay
b533d824f1 Turning off Docker iptables firewall exclusion. See https://stackoverflow.com/questions/30383845/what-is-the-best-practice-of-docker-ufw-under-ubuntu 2021-06-23 23:16:10 -04:00
Samuel Clay
7d8ba31ee1 Adding blog role and server. 2021-06-22 12:50:08 -04:00
Jonathan Math
15f40588f3 fix celery_task sanity checkers 2021-06-22 09:06:10 -06:00
Jonathan Math
309155faeb fix docker commands for backups 2021-06-19 13:01:38 -06:00
Jonathan Math
4cea718d40 add redis backup and remove redundant redis backup scripts. Use one script and pass in hostname argument instead 2021-06-19 12:52:04 -06:00
Jonathan Math
c2bef5565b add backup to ansible for postgres 2021-06-19 12:35:28 -06:00
Jonathan Math
a71069994c add mongo backup cronjob for ansible and integrate backup_mongo.py with docker setup 2021-06-19 12:02:29 -06:00
Jonathan Math
8d6a9732b3 use argv instead of env vars for redis log path 2021-06-19 10:03:02 -06:00
Jonathan Math
bf140f1aea fix disk usage sanity checker to check host machine and pass the data into the container to evaluate the disk usage and send an email 2021-06-18 14:57:29 -06:00
Jonathan Math
7394517a40 try to make redis bgsave monitor work in docker 2021-06-16 11:58:57 -06:00