Commit graph

703 commits

Author SHA1 Message Date
Samuel Clay
55a200a5b3 Graceful deploys for gunicorn. Turns out a HUP signal to the docker container isn't the same thing as a HUP signal to the actual process, which can be done outside of the container. 2021-03-26 18:44:56 -04:00
Samuel Clay
02138db5ca Switching from elasticsearch and dsl to pyelasticsearch. Feed autocomplete still doesn't work yet. 2021-03-24 21:09:26 -04:00
Samuel Clay
3e61c5e907 Feed autocomplete is trnaslated to elasticsearch 7, but doesn't work quite yet. 2021-03-23 15:30:44 -04:00
Samuel Clay
fb322a6573 Querying works but returns wrong results. 2021-03-19 12:22:28 -04:00
Samuel Clay
e1b2583950 Swapping out pyes for elasticsearch-py. Half-way done, got indices and mappings in, now need to work on querying and storing. 2021-03-18 16:34:28 -04:00
Samuel Clay
d52a8d362c Reducing number of workers on docker-compose. Also removing extraneous task servers and consolidating to task-celery for feeds and task-work for cron and work 2021-03-17 20:13:58 -04:00
Samuel Clay
15cfd7a592 Using the gunicorn config so workers can be toned down. 2021-03-17 17:16:49 -04:00
Samuel Clay
5d2849a9fd Downgrading elasticsearch on docker-compose to handle aging pyes library. 2021-03-17 16:51:17 -04:00
Samuel Clay
a4af967ccb Sentry SDK 1.0.0 2021-03-09 16:51:27 -05:00
Samuel Clay
4ad908e233 Fixing updates for new premium users. Also attempting to fix redis server data persistance issues. 2021-03-08 19:47:24 -05:00
Samuel Clay
0c19aec2a3 Updating to apns2 library for Apple Push Notifications over Http/2. 2021-03-04 18:26:40 -05:00
Jonathan Math
8c736ac489 add performance test make commands, dockerfile for locust, and rough draft of performance test (locust.py) 2021-02-24 10:21:47 -05:00
Samuel Clay
32eb8539c1 Attempting to get old inventory working with new, but its not yet resolving for hosts. Not sure why. Also reducing node docker image down. 2021-02-19 19:58:11 -05:00
Jonathan Math
fca7f97b47 alphabetize requirements and also change celery version to celery>=4,<5 2021-02-16 13:11:58 -05:00
Samuel Clay
e1b0c0f8e8 Adding dnsmasq and disabling systemd-resolved. Consul clients and servrs are now being setup, although there ar ehealth check issues still unresolved. 2021-02-11 19:39:46 -05:00
Samuel Clay
f9b7bbcd6e Merge branch 'master' into dashboard3
* master:
  Adding a preference to notification title only.
  Adding in preference to only send full title in iOS notifications.
  Forgot primary state
  Adding replset oplog timing to db monitor.
2021-02-01 15:01:58 -05:00
Samuel Clay
79b0643992 Adding replset oplog timing to db monitor. 2021-02-01 12:39:28 -05:00
Samuel Clay
b865fcb8f5 Merge branch 'master' into dashboard3
* master:
  Removing ssl from nginx, adding mongo.conf back in to all servers.
2021-02-01 02:27:32 -05:00
Samuel Clay
d88e86d4cb Removing ssl from nginx, adding mongo.conf back in to all servers. 2021-02-01 02:26:56 -05:00
Samuel Clay
dccb87bd26 Adding files for common setup on app. Finishing up dynamic inventory. 2021-01-28 19:06:30 -05:00
Samuel Clay
c7f4cbb13d Fixing a substantial number of encoding errors. Also fixing OPML uploads. 2021-01-24 21:05:22 -05:00
Samuel Clay
d3449db1ec Merge branch 'docker_django3.1' into dashboard3
* docker_django3.1:
  add swarm_instructions.sh
  update celery task name to cron_queue in settings.py.orig
  change redis port to 6579 in docker local settings
  update celery task name to cron_queue in settings.py
  add image building and deployment functionality to Makefile
  update docker files to include code in the image so we can use with swarm (without configuring shared volumes yet)
  change beat_tasks to cron_queue in config/supervision_celeryd_beat.conf
  change pymysql version in __init__ to the new django compatible version
  change names for swarm service in swarm haproxy.conf file
  remove ssl from nginx
  fix dependency issues
  change nginx check time in haproxy
  add .dockerignore
  update newsblur dir to newsblur_web in code
  adjust resource limits on containers in stack-compose and correct monitor command
  clean up and fix stack-compose.yml file
  correct docker-compose images
2021-01-23 19:31:06 -05:00
Samuel Clay
3ad5295a33 Merge branch 'master' into docker_django3.1
* master:
  Upgrading to mongo 3.4.
  Use email if username doesnt work.
  Removing old types field. Needed to upgrade to mongo 3.4.
  Checking if a feed has any notifications on it, set a max of 30 min fetch (yes this is a pretty good way of getting your feeds to fetch faster, but youll have to deal with the notifications).
  Fix task fetch monitor.
  Attempting to see iOS premium status over email.
  Handling work queue missing error.
  Limiting work queue checks.
  Executable monitor work queue.
  Adding monitor for work queue.
  Pointing to the new ssl cert.
  More descriptive failure email
  No default value
  Spitting out error on failure.
  Handling missing values
  Needs redis
  Only warn about falling feed fetches when actually falling.
  Revert "Removing unused types key from index."
  Don't need full health checks in debug.
2021-01-23 19:29:11 -05:00
Samuel Clay
33508f7361 Merge branch 'master' into dashboard3
* master:
  Executable monitor work queue.
  Adding monitor for work queue.
  Pointing to the new ssl cert.
  More descriptive failure email
  No default value
  Spitting out error on failure.
  Handling missing values
  Needs redis
  Only warn about falling feed fetches when actually falling.
  Revert "Removing unused types key from index."
2021-01-20 14:21:48 -05:00
Samuel Clay
c3732d6d1c Fixing celery issues. 2021-01-19 18:00:10 -05:00
Jonathan Math
ac5a9e8da1 change beat_tasks to cron_queue in config/supervision_celeryd_beat.conf 2021-01-19 09:47:30 -05:00
Jonathan Math
be24ec241e fix dependency issues 2021-01-19 09:41:47 -05:00
Samuel Clay
7d218a652a Pointing to the new ssl cert. 2021-01-18 20:16:25 -05:00
Samuel Clay
37d24cdb6b Merge branch 'master' into dashboard3
* master:
  Don't need full health checks in debug.
2021-01-12 14:50:32 -05:00
Samuel Clay
b3944cab07 Don't need full health checks in debug. 2021-01-12 14:50:27 -05:00
Samuel Clay
1baa7b6e22 Merge branch 'docker_django3.1' into dashboard3
* docker_django3.1:
  Adding imageproxy check using a real image.
  Adding httpchk
  Imageproxy behind haproxy.
2021-01-12 14:31:11 -05:00
Samuel Clay
10af9a8c3b Merge branch 'master' into docker_django3.1
* master:
  Adding imageproxy check using a real image.
  Adding httpchk
  Imageproxy behind haproxy.
2021-01-12 14:31:05 -05:00
Samuel Clay
28788b43b0 Adding imageproxy check using a real image. 2021-01-12 14:30:51 -05:00
Samuel Clay
e688e3dd5d Adding httpchk 2021-01-12 13:59:01 -05:00
Samuel Clay
173beef3d7 Imageproxy behind haproxy. 2021-01-11 18:46:24 -05:00
Samuel Clay
e5287c80f9 Merge branch 'master' into docker_django3.1
* master:
  Moving imageproxy and push to behind HAProxy for single point configuration.
  Installing certbot.
2021-01-11 17:50:35 -05:00
Samuel Clay
0e6470fbf1 Merge branch 'master' into dashboard3
* master:
  Moving imageproxy and push to behind HAProxy for single point configuration.
2021-01-11 17:50:24 -05:00
Samuel Clay
74a2fe0d5d Moving imageproxy and push to behind HAProxy for single point configuration. 2021-01-11 17:49:52 -05:00
Samuel Clay
fff445253f Updating HAProxy 2021-01-11 10:57:29 -05:00
Samuel Clay
f5b144f22f Merge branch 'docker_django3.1' into dashboard3
* docker_django3.1:
  Ignore send_message_with_response if not available (for some reason).
  Updating to latest pymongo and mongoengine for py2.
2021-01-06 16:39:23 -05:00
Samuel Clay
30828e8291 Merge branch 'master' into docker_django3.1
* master:
  Ignore send_message_with_response if not available (for some reason).
  Updating to latest pymongo and mongoengine for py2.
2021-01-06 16:39:00 -05:00
Samuel Clay
aff5d1d0a9 Updating to latest pymongo and mongoengine for py2. 2021-01-06 15:52:46 -05:00
Samuel Clay
87b94d3ad1 Gunicorn should forward stderr to logs. 2021-01-06 15:47:25 -05:00
Samuel Clay
46cb290985 Updating supervisor, which won't actually be used but is for testing. 2021-01-06 13:37:39 -05:00
Samuel Clay
6dbd9fedd3 Updating requirements. 2021-01-05 19:55:16 -05:00
Samuel Clay
82fecb23a6 Merge branch 'docker_django3.1' into dashboard3
* docker_django3.1: (117 commits)
  Adding docker to node scripts.
  Removing node_modules from repo
  Whoops
  Better logging for broken paypal ipns.
  Adding RSS Tracker for Windows to Goodies.
  Refactoring socketio so real-time works around username restrictions.
  Socket.IO v3 on the server too.
  Switching to Socket.IO v3.
  Haproxy splitting for socketio 3
  Socket.io 3
  Handling missing feed.
  Only nb.local.com uses ports for websockets.
  Adding GrazeTEN
  Full node install on setup.
  Updating toprc
  correct newsblur to newsblur_web in tasks
  Changing mongodb server.
  Fixing toprc
  Attempting to collect queries for 1% of requests by using DEBUG.
  Fixing mongo install.
  ...
2021-01-05 19:36:49 -05:00
Samuel Clay
6e07dbaf0f Merge branch 'master' into docker_django3.1
* master: (22 commits)
  Whoops
  Better logging for broken paypal ipns.
  Adding RSS Tracker for Windows to Goodies.
  Refactoring socketio so real-time works around username restrictions.
  Socket.IO v3 on the server too.
  Switching to Socket.IO v3.
  Haproxy splitting for socketio 3
  Socket.io 3
  Handling missing feed.
  Only nb.local.com uses ports for websockets.
  Adding GrazeTEN
  Full node install on setup.
  Updating toprc
  Changing mongodb server.
  Fixing toprc
  Attempting to collect queries for 1% of requests by using DEBUG.
  Fixing mongo install.
  Recalculating premium expiration to take into account forced expirations after gifts.
  Fixing never expire -> 1 year
  Adding 1 year expire to admin.
  ...
2021-01-05 19:12:19 -05:00
Samuel Clay
5892a8a9c0 Switching to Socket.IO v3. 2021-01-04 17:55:47 -05:00
Samuel Clay
75f0288c62 Haproxy splitting for socketio 3 2021-01-04 17:53:06 -05:00
Samuel Clay
8bb1be8de6 Only nb.local.com uses ports for websockets. 2021-01-04 16:47:24 -05:00