Commit graph

117 commits

Author SHA1 Message Date
Samuel Clay
9432e9a5b8 Merge branch 'master' into pipeline
* master:
  Pruning docker, updating sentry release webhook only on local.
  Delete .github directory
  Adding Sentry release webhook to deployment of web, task, monitor, and node.
  Add Sentry action workflow for actions
  Moving from hosted Sentry to self-hosted sentry.
  Flask transactions are way too often.
  Cleanup
  Seems redundant with newsblur_web/docker_local_settings.py
  Using new repo password for postgres.
  Moving around postgres config
  Moving directories.
  Adding postgresql backup restoration. Working on config.
  Using user log for db times.
  Last 5 minutes is actually last minute.
  Adding 5min loadtime to prometheus graphs.
  Switching back to nbanalytics with user auth.
  Mongo auth
2022-01-03 18:04:40 -05:00
Samuel Clay
7ebce0bcdd Switching back to nbanalytics with user auth. 2021-12-16 20:50:23 -05:00
Samuel Clay
8be0192702 Cleaning up pipeline 2021-12-16 20:06:15 -05:00
Samuel Clay
cf948c725a Renaming css files and fixing debug assets pipeline collector by forcing it to munge the static url with the media, non-cached, name. nginx should do the caching, not the url. 2021-12-16 17:30:05 -05:00
Samuel Clay
3d59e59636 Turning on DEBUG_ASSETS so developers dont need to compress their own assets. 2021-12-16 16:07:45 -05:00
Samuel Clay
9ced4ccadd Merge branch 'master' into pipeline
* master: (84 commits)
  Deploying new analytics server.
  Create mongodb database user. Only runs on analytics, but should work fine on main mongo.
  Adding second mongo analytics db, cleaning up mongo analytics auth.
  Adding RSS autodiscovery to blog. Thanks https://forum.newsblur.com/t/irony-newsblur-cant-auto-extract-rss-feed-from-blog-newsblur-com/8849.
  Revert "Merge branch 'state_timeline'"
  Fixing syntax error
  Solving docker networking container issues by turning iptables back on but using ufw after.rules that allow ufw to retain blocking ability. Works on Redis, now to move to mongo.
  fix flask_metrics ansible role
  add state timeline dashboard to grafana
  move state timeline to one single flask that gets astate for all services
  Mongo analytics monitor not reayd yet
  Turning consul monitor back on.
  Turning on analytics
  Need to strip csv values.
  Binding redis to public interface.
  Restore postgres backup. Needs work.
  Prepping postgres backup
  Prepping postgres transition, writing redis db maintenance.
  Prepping redis for moving back to primary.
  Updating redis backup to include percentage progress and access to the correct data dump.
  ...
2021-12-16 15:41:57 -05:00
Samuel Clay
ea6103bc8f Adding second mongo analytics db, cleaning up mongo analytics auth. 2021-12-16 13:48:20 -05:00
Samuel Clay
01974e2706 Thanks to @jeff-tyrrill for helping find and secure these vulnerabilities having to do with the Story view and auth cookie. 2021-12-10 17:24:04 -05:00
Samuel Clay
678662de8f Bumping celery concurrency 2021-12-06 11:23:34 -05:00
Samuel Clay
b71f383122 Fixing remaining broken js files. Collect static command now works as expected. 2021-11-26 16:44:11 -05:00
Samuel Clay
fd31c534e8 Cleaning up pipeline dirs. 2021-11-22 13:17:12 -05:00
Samuel Clay
45b3cfdecf Switching from yuglify to the original closure compiler. Everything's working well, although deployment needs to be rearranged a bit. 2021-11-17 13:41:22 -05:00
Jonathan Math
f26ae11707 add collectstatic make command and newsblur_deploy dockerfile 2021-11-16 12:41:09 -05:00
Samuel Clay
c4bbd3a7a8 New file system finder, finds static assets. 2021-11-16 11:27:09 -05:00
Samuel Clay
b6db14dacd Switching from Jammit to django-pipeline. Doesn't work yet, needs compressor hooked up and assets checked. 2021-11-14 13:42:06 -05:00
Samuel Clay
92413c8956 REDIS_USER needs to be earlier. 2021-09-21 16:50:07 -04:00
Jonathan Math
7fa72f5be3 fix flask metrics python files 2021-09-21 12:13:46 -05:00
Samuel Clay
f03be46425 Removing boto2, upgrading to boto3. Needs testing. 2021-08-04 20:04:53 -04:00
Samuel Clay
3804fbba5d Correcting mongodb on local installs since there is no auth. 2021-08-04 16:26:41 -04:00
Samuel Clay
96b29bb037 Upping data upload limit to 5mb. 2021-08-04 09:37:01 -04:00
Samuel Clay
6107dcaf66 Default username and password for analytics. 2021-08-03 21:23:31 -04:00
Samuel Clay
53f3ba5b49 Removing unused mongodb connect code 2021-08-03 21:13:23 -04:00
Samuel Clay
ff0a98aba1 Using nbanalytics db as name 2021-08-03 21:12:36 -04:00
Samuel Clay
e310848247 Typo 2021-08-03 21:11:36 -04:00
Samuel Clay
c518acade6 Mongoengine connect is so persnickity. 2021-08-03 21:11:03 -04:00
Samuel Clay
48c5967094 Adding db name 2021-08-03 21:08:01 -04:00
Samuel Clay
fc1c9abf26 No port in mongo analytics 2021-08-03 21:04:29 -04:00
Samuel Clay
f82f2149a7 New mongodb connect string for analytics. 2021-08-03 21:02:20 -04:00
Samuel Clay
6286b798a7 Fixing JSON feeds. 2021-07-29 17:25:09 -04:00
Samuel Clay
c055656869 Adding DEBUG_QUERIES_SUMMARY_ONLY to assist db introspection without the verbosity. 2021-07-15 16:50:54 -04:00
Samuel Clay
ad61923c5e Typo in server_name 2021-07-09 17:29:24 -04:00
Samuel Clay
6d2c436835 Sentry requires a server_name param. 2021-07-09 17:21:56 -04:00
Samuel Clay
5886f1cc1b Another attempt at setting SERVER_NAME in sentry. 2021-07-09 17:11:55 -04:00
Samuel Clay
d9e2b0db30 Adding SERVER_NAME to sentry to get through docker. 2021-07-09 17:09:28 -04:00
Samuel Clay
b74fc79266 Load local_settings on top of docker-local_settings. 2021-07-06 10:37:36 -04:00
Samuel Clay
fc0b26437d Turning off elasticsearch logging. 2021-06-09 13:09:53 -04:00
Samuel Clay
1012779bf3 Logging elasticsearch, fixing autocomplete on feeds styling. Still need to fix feed search. 2021-06-09 12:52:22 -04:00
Samuel Clay
1ab551f73d Turning off spam deleter due to captcha. 2021-05-13 14:33:56 -04:00
Samuel Clay
aac3fba63d Cleaner logs. 2021-05-12 19:00:07 -04:00
Samuel Clay
428647f70c Lowering readability logs. 2021-05-12 18:54:04 -04:00
Samuel Clay
0235a907da Let's try reducing celery worker counts and seeing if that boosts work performed. 2021-04-20 19:24:50 -04:00
Samuel Clay
230716465e Adding new optional app/task env settings. 2021-04-20 18:02:12 -04:00
Samuel Clay
c00630ecf2 Typo 2021-04-11 15:24:24 -04:00
Samuel Clay
1af8183550 Adding celery integration to sentry. 2021-04-11 12:25:17 -04:00
Samuel Clay
b561ce756b Fixing issue with dashboard river folder not being changeable or selectable. 2021-04-07 15:32:17 -04:00
Samuel Clay
fc5751da60 Adding unicode decode error handling for mongo. Let's hope I don't regret this. Test with Metafilter feed: 1168 2021-03-26 10:43:34 -04:00
Samuel Clay
e1b2583950 Swapping out pyes for elasticsearch-py. Half-way done, got indices and mappings in, now need to work on querying and storing. 2021-03-18 16:34:28 -04:00
Samuel Clay
05afdce691 Fixing search indexing. Also removing superfluous search_indexer_tasker queue. 2021-03-03 17:04:05 -05:00
Samuel Clay
01248e5a1b Fixing task servers. Also fixing sentry max size limit. 2021-02-25 18:50:20 -05:00
Samuel Clay
d3449db1ec Merge branch 'docker_django3.1' into dashboard3
* docker_django3.1:
  add swarm_instructions.sh
  update celery task name to cron_queue in settings.py.orig
  change redis port to 6579 in docker local settings
  update celery task name to cron_queue in settings.py
  add image building and deployment functionality to Makefile
  update docker files to include code in the image so we can use with swarm (without configuring shared volumes yet)
  change beat_tasks to cron_queue in config/supervision_celeryd_beat.conf
  change pymysql version in __init__ to the new django compatible version
  change names for swarm service in swarm haproxy.conf file
  remove ssl from nginx
  fix dependency issues
  change nginx check time in haproxy
  add .dockerignore
  update newsblur dir to newsblur_web in code
  adjust resource limits on containers in stack-compose and correct monitor command
  clean up and fix stack-compose.yml file
  correct docker-compose images
2021-01-23 19:31:06 -05:00