Commit graph

753 commits

Author SHA1 Message Date
Samuel Clay
a5222d9675 Updating youtube fetcher to use channels/playlists/users for everything, no longer relying on RSS/xml url. 2024-03-03 12:59:12 -05:00
Samuel Clay
4d6f0b5000 Upping app gunicorn workers. 2024-02-18 08:12:28 -05:00
Samuel Clay
1453c3ee27 Updating postgres for standby. 2024-01-27 23:11:36 -05:00
Samuel Clay
1065c964fd Fixing elasticsearch to allow consul to assume its OK. 2022-12-26 19:17:14 -05:00
Samuel Clay
9e21d03255 Merging SDIFFSTORE and ZINTERSTORE into a single ZDIFFSTORE, thanks to redis 6.2.0. Requires new docker image. 2022-07-19 21:02:42 -04:00
Samuel Clay
8f1655d7d1 Revert "Merging SDIFFSTORE and ZINTERSTORE into a single ZDIFFSTORE, thanks to redis 6.2.0. Requires new docker image."
This reverts commit b3ec9fd748.
2022-07-18 22:45:22 -04:00
Samuel Clay
b3ec9fd748 Merging SDIFFSTORE and ZINTERSTORE into a single ZDIFFSTORE, thanks to redis 6.2.0. Requires new docker image. 2022-07-18 22:34:07 -04:00
Samuel Clay
ed7fc4a4e8 Updating signed image to use https for image proxy. 2022-05-26 12:17:45 -04:00
Samuel Clay
5dba6549ef Removing unused mongo mms utils and configs. 2022-05-10 20:53:56 -04:00
Samuel Clay
30898df3e5 Updating pymongo to mongodb 4.0 compatible. Any further and we lose mongo wire protocol inspection. 2022-05-10 11:05:26 -04:00
Samuel Clay
57b190e241 Adding url normalize library to update aging urlnorm.py from Mark Pilgrim. 2022-04-19 15:09:26 -04:00
Samuel Clay
283bba5cd3 Rearranging urls for paypal. Also updating to latest gunicorn. 2022-04-11 11:01:02 -04:00
Samuel Clay
893619596b Adding django prometheus metrics. 2022-03-31 15:34:33 -04:00
Samuel Clay
96c7416d1e Gunicorn workers at 3 or 4, depending on having at least 3.1+ GB of memory. 2022-02-28 14:41:29 -05:00
Samuel Clay
8cc4956f73 Adding paypal dependencies for new premium archive and premium pro. 2022-02-16 13:53:45 -05:00
Samuel Clay
b0acb97ac1 Archiving munin. 2022-02-01 14:59:05 -05:00
Samuel Clay
9ced4ccadd Merge branch 'master' into pipeline
* master: (84 commits)
  Deploying new analytics server.
  Create mongodb database user. Only runs on analytics, but should work fine on main mongo.
  Adding second mongo analytics db, cleaning up mongo analytics auth.
  Adding RSS autodiscovery to blog. Thanks https://forum.newsblur.com/t/irony-newsblur-cant-auto-extract-rss-feed-from-blog-newsblur-com/8849.
  Revert "Merge branch 'state_timeline'"
  Fixing syntax error
  Solving docker networking container issues by turning iptables back on but using ufw after.rules that allow ufw to retain blocking ability. Works on Redis, now to move to mongo.
  fix flask_metrics ansible role
  add state timeline dashboard to grafana
  move state timeline to one single flask that gets astate for all services
  Mongo analytics monitor not reayd yet
  Turning consul monitor back on.
  Turning on analytics
  Need to strip csv values.
  Binding redis to public interface.
  Restore postgres backup. Needs work.
  Prepping postgres backup
  Prepping postgres transition, writing redis db maintenance.
  Prepping redis for moving back to primary.
  Updating redis backup to include percentage progress and access to the correct data dump.
  ...
2021-12-16 15:41:57 -05:00
Samuel Clay
966f0998fc Removing unused traces of older redis servers. 2021-12-06 13:56:09 -05:00
Samuel Clay
c4bbd3a7a8 New file system finder, finds static assets. 2021-11-16 11:27:09 -05:00
Samuel Clay
b6db14dacd Switching from Jammit to django-pipeline. Doesn't work yet, needs compressor hooked up and assets checked. 2021-11-14 13:42:06 -05:00
Samuel Clay
5734225d6f Updating Redis to v6, adding redis servers. 2021-10-20 16:46:19 -04:00
Samuel Clay
1fdebf5c76 Removing haproxy=1 from health checks. Haproxy tests the full mongo db, so it's the default case, no need for exceptional query string like consul. 2021-10-19 11:11:11 -04:00
Jonathan Math
dd13af5bf1 add querystring logic to mongo db health check to accommodate consul vs haproxy checking 2021-10-11 11:11:42 -05:00
Samuel Clay
3e23b6394a Revert "Revert "Merge branch 'add_grafana_monitors'""
This reverts commit a7f5025a16.
2021-09-16 11:57:57 -05:00
Samuel Clay
3f00139541 Upgrading sentry on both web and node. 2021-09-10 15:58:22 -04:00
Samuel Clay
a7f5025a16 Revert "Merge branch 'add_grafana_monitors'"
This reverts commit f0c2176908, reversing
changes made to 72ef2e2f00.
2021-09-07 11:36:11 -04:00
Jonathan Math
25efd6d973 merge master 2021-09-05 22:42:32 -05:00
Samuel Clay
bf2a54e755 Fixing elasticsearch sysctl and config. 2021-08-13 11:44:18 -04:00
Jonathan Math
a415ba457c rm nb.local.com and replace with localhost 2021-08-10 10:06:16 -05:00
Samuel Clay
f03be46425 Removing boto2, upgrading to boto3. Needs testing. 2021-08-04 20:04:53 -04:00
Samuel Clay
6555f8ddcc Vendorizing readability due to v0.8.1.1 not being released yet. 2021-08-03 18:11:17 -04:00
Samuel Clay
8348da9c59 Updating to latest readability-lxml. 2021-08-03 16:05:28 -04:00
Samuel Clay
f91b5164dc Upgrading readability and removing vendorized package. May regret losing noscript support but Postlight's Mercury Reader probably handles those edge cases. 2021-08-02 17:02:03 -04:00
Samuel Clay
5aaf4d5c38 Rewriting mongo backup, adding in rotation. 2021-07-29 18:17:02 -04:00
Samuel Clay
05fc33b453 Updating dependencies. 2021-07-29 16:24:56 -04:00
Samuel Clay
4ab47b7b42 Found more instances of mongodb changing timestamps 2021-07-14 22:00:25 -04:00
Samuel Clay
14e3d6dd25 Fixing broken requirements.txt, downgrading pymongo to 3.6, since monitoring was changed and we're not ready for it yet. 2021-07-02 12:42:02 -04:00
Jonathan Math
fa92ac24fd update requirements for new image. Use boto3 instead of s3 and use nosetests 2021-06-30 21:33:35 -06:00
Jonathan Math
09dccbab87 update requirements.txt to match what is in the newsblur/newsblur_python3 image 2021-06-27 20:26:11 -06:00
Samuel Clay
2f602a8d85 Removing consul health check for web, lowering workers. 2021-05-12 17:07:37 -04:00
Samuel Clay
32cc9b8f2b Merge branch 'master' into dashboard3
* master:
  Adding in node_page to haproxy to reroute dashboard3 to the correct server. This is corrected in dashboard3 with consul.
2021-05-05 12:06:46 -04:00
Samuel Clay
20b3d24d09 Adding in node_page to haproxy to reroute dashboard3 to the correct server. This is corrected in dashboard3 with consul. 2021-05-05 12:06:41 -04:00
Samuel Clay
9466f336e4 Switching from pyelasticsearch to elasticsearch. 2021-04-27 15:00:52 -04:00
Jonathan Math
e068c9ff4e change nb.local.com to localhost 2021-04-20 08:04:08 -05:00
Samuel Clay
f281b7cd5e De-vendorizing pynliner. 2021-04-19 14:07:26 -04:00
Samuel Clay
c7d7764318 Mongo logs shouldnt be over 256MB 2021-04-11 12:09:15 -04:00
Samuel Clay
ec443f6117 Docker compose needs elasticsearch.yml config. 2021-04-07 15:32:17 -04:00
Jonathan Math
28e876df6d increase gunicorn workers for docker build 2021-04-07 10:36:16 -05:00
Jonathan Math
bb52814648 update gunicorn so the reload feature can be used 2021-04-03 12:28:50 -05:00
Samuel Clay
68170652d1 Wrong syntax 2021-03-29 18:01:13 -04:00