Commit graph

1742 commits

Author SHA1 Message Date
Samuel Clay
71eae2c640 Better handling of Twitter being down (and returning a 503). 2022-01-21 06:52:10 -05:00
Samuel Clay
8164767a59 Merge branch 'master' into pro
* master:
  Cleaning up django collection pipeline.
  Upgrading jQuery and jQuery UI, fixing closure compiler language output to work in older browser.
  Fixing scope of webmanifest.
  Changing webmanifest scope.
  Adding Web App Manifest for PWA.
2022-01-20 16:56:51 -05:00
Samuel Clay
0011249cad Cleaning up django collection pipeline. 2022-01-20 16:32:37 -05:00
Samuel Clay
ced2e7d6a6 Upgrading jQuery and jQuery UI, fixing closure compiler language output to work in older browser. 2022-01-20 13:19:35 -05:00
Samuel Clay
0baaa951b9 Adding preference for changing days of unreads. Need to kick off a re-counter on preference change. 2022-01-18 12:40:49 -05:00
Samuel Clay
2792e7305a Syncing redis on premium upgrade to sync new unread date possibilities for archive users. 2022-01-13 16:56:30 -05:00
Samuel Clay
3ef72e6f83 Fixing feed fetch status screen to better show how subs are being fetched. 2022-01-12 11:58:28 -05:00
Samuel Clay
88d000a2f9 Turning off logging for debug_assets=True. 2022-01-11 11:02:40 -05:00
Samuel Clay
a8d73f3685 Fixing static files so that at least newsblur is packaged correctly. Admin static is still not able to be processed. 2022-01-06 12:42:13 -05:00
Samuel Clay
ddeaf55be0 Cleaning up pipeline and adding admin static assets, but now certain admin static assets are being post-processed multiple times but with different cache manifests, resulting in a file not found error. This needs work. 2022-01-06 11:01:35 -05:00
Samuel Clay
76ffe7bb8a Fixing bookmarklet, moving endlne fleuron, adding load-line, archviing Jammit. 2022-01-05 11:18:31 -05:00
Samuel Clay
b8eccdbed4 Merge branch 'master' into pipeline
* master:
  Fixing names for state timeline, also fixing node server bootstrap.
  Moving up haproxy state timeline
  Bumping haproxy state timeline scrapes.
  Only need haproxy for haproxy metrics
  Refactored state timeline to work as a flask metrics server on haproxy. Needs ansible configs to deploy on www server.
  Forgot to save merge conflict.
  fix service state timeline key error and change scrape interval
  Revert "Revert "Merge branch 'state_timeline'""
  Revert "Merge branch 'state_timeline'"
  Fixing syntax error
  Solving docker networking container issues by turning iptables back on but using ufw after.rules that allow ufw to retain blocking ability. Works on Redis, now to move to mongo.
  Mongo analytics monitor not reayd yet
  Turning consul monitor back on.
  Turning on analytics
  Need to strip csv values.
  Binding redis to public interface.
  add grafana dashboard backup functionality
2022-01-04 14:44:57 -05:00
Samuel Clay
0db15560da Merge branch 'master' into state_timeline
* master: (29 commits)
  Pruning docker, updating sentry release webhook only on local.
  Delete .github directory
  Adding Sentry release webhook to deployment of web, task, monitor, and node.
  Add Sentry action workflow for actions
  Moving from hosted Sentry to self-hosted sentry.
  Flask transactions are way too often.
  Cleanup
  Seems redundant with newsblur_web/docker_local_settings.py
  Using new repo password for postgres.
  Moving around postgres config
  Moving directories.
  Adding postgresql backup restoration. Working on config.
  Using user log for db times.
  Last 5 minutes is actually last minute.
  Adding 5min loadtime to prometheus graphs.
  Switching back to nbanalytics with user auth.
  Mongo auth
  Deploying new analytics server.
  Create mongodb database user. Only runs on analytics, but should work fine on main mongo.
  Adding second mongo analytics db, cleaning up mongo analytics auth.
  ...
2022-01-04 11:18:48 -05:00
Samuel Clay
9432e9a5b8 Merge branch 'master' into pipeline
* master:
  Pruning docker, updating sentry release webhook only on local.
  Delete .github directory
  Adding Sentry release webhook to deployment of web, task, monitor, and node.
  Add Sentry action workflow for actions
  Moving from hosted Sentry to self-hosted sentry.
  Flask transactions are way too often.
  Cleanup
  Seems redundant with newsblur_web/docker_local_settings.py
  Using new repo password for postgres.
  Moving around postgres config
  Moving directories.
  Adding postgresql backup restoration. Working on config.
  Using user log for db times.
  Last 5 minutes is actually last minute.
  Adding 5min loadtime to prometheus graphs.
  Switching back to nbanalytics with user auth.
  Mongo auth
2022-01-03 18:04:40 -05:00
Samuel Clay
647b1100fd Using user log for db times. 2021-12-17 13:55:35 -05:00
Samuel Clay
8be0192702 Cleaning up pipeline 2021-12-16 20:06:15 -05:00
Samuel Clay
cf948c725a Renaming css files and fixing debug assets pipeline collector by forcing it to munge the static url with the media, non-cached, name. nginx should do the caching, not the url. 2021-12-16 17:30:05 -05:00
Samuel Clay
9ced4ccadd Merge branch 'master' into pipeline
* master: (84 commits)
  Deploying new analytics server.
  Create mongodb database user. Only runs on analytics, but should work fine on main mongo.
  Adding second mongo analytics db, cleaning up mongo analytics auth.
  Adding RSS autodiscovery to blog. Thanks https://forum.newsblur.com/t/irony-newsblur-cant-auto-extract-rss-feed-from-blog-newsblur-com/8849.
  Revert "Merge branch 'state_timeline'"
  Fixing syntax error
  Solving docker networking container issues by turning iptables back on but using ufw after.rules that allow ufw to retain blocking ability. Works on Redis, now to move to mongo.
  fix flask_metrics ansible role
  add state timeline dashboard to grafana
  move state timeline to one single flask that gets astate for all services
  Mongo analytics monitor not reayd yet
  Turning consul monitor back on.
  Turning on analytics
  Need to strip csv values.
  Binding redis to public interface.
  Restore postgres backup. Needs work.
  Prepping postgres backup
  Prepping postgres transition, writing redis db maintenance.
  Prepping redis for moving back to primary.
  Updating redis backup to include percentage progress and access to the correct data dump.
  ...
2021-12-16 15:41:57 -05:00
Jonathan Math
6189dd4c01 add grafana dashboard backup functionality 2021-12-15 13:40:23 -05:00
Samuel Clay
777e69ae82 Prepping postgres backup 2021-12-13 17:09:37 -05:00
Samuel Clay
8bceaada80 Updating redis backup to include percentage progress and access to the correct data dump. 2021-12-13 14:26:20 -05:00
Samuel Clay
be3237cdc3 Updating to boto3 syntax. 2021-12-13 12:45:47 -05:00
Samuel Clay
d677373e3b No more settings.get. 2021-12-13 12:42:30 -05:00
Samuel Clay
1dc79686bc Skipping requests for RSS, letting feedparser do the fetching. Requests si still used for json feeds. 2021-12-08 20:44:16 -05:00
Samuel Clay
35bc9ad07f Rewriting readme from #1545. 2021-12-08 14:55:26 -05:00
Samuel Clay
404c97a600 Revert "Rewriting readme from #1545."
This reverts commit ee06578679.
2021-12-08 14:47:44 -05:00
Samuel Clay
7081c95217
Merge pull request #1541 from louis-pre/fix_youtube_duration
fix YouTube video duration calculation
2021-12-08 14:37:00 -05:00
Samuel Clay
ee06578679 Rewriting readme from #1545. 2021-12-08 12:41:19 -05:00
Samuel Clay
779348c495 Removing unused db redis query counters. 2021-12-06 10:49:37 -05:00
Samuel Clay
45b3cfdecf Switching from yuglify to the original closure compiler. Everything's working well, although deployment needs to be rearranged a bit. 2021-11-17 13:41:22 -05:00
Samuel Clay
c4bbd3a7a8 New file system finder, finds static assets. 2021-11-16 11:27:09 -05:00
Samuel Clay
e6367605cd Merge branch 'master' into pipeline
* master:
  Removing dependencies from ssh helper.
  Specifying docker build platform (--platform linux/amd64) so M1 Macs don't build arm64/v8.
  Haproxy shoudl reload on config update, adding celery task group, and allowing task servers to update inventory.
2021-11-14 20:11:10 -05:00
Samuel Clay
87c3b111a8 Removing dependencies from ssh helper. 2021-11-14 18:43:17 -05:00
Samuel Clay
b6db14dacd Switching from Jammit to django-pipeline. Doesn't work yet, needs compressor hooked up and assets checked. 2021-11-14 13:42:06 -05:00
Samuel Clay
714f31a2b0 Work queue size is allowed to grow a llittle. 2021-10-06 11:36:17 -04:00
Samuel Clay
7a44ab24a7 Typo 2021-10-06 10:46:29 -04:00
Samuel Clay
7ce5ca63f0 Ignoring feed fetch falling emails when not actually falling. 2021-10-06 10:07:21 -04:00
Samuel Clay
153675b43d Adding feeds as users to feed fetching in sentry. 2021-09-12 13:27:24 -04:00
Samuel Clay
72ef2e2f00 Found another old s3 call. Used in avatar uploads. 2021-09-06 17:20:07 -04:00
Louis Pré
4bef983db3 fix YouTube video duration calculation 2021-08-26 21:38:33 -07:00
Samuel Clay
1469992f2d S3 content-type to contenttype. 2021-08-26 15:50:11 -04:00
Samuel Clay
9bd863f74f Removing usused imports 2021-08-09 11:20:18 -04:00
Samuel Clay
f03be46425 Removing boto2, upgrading to boto3. Needs testing. 2021-08-04 20:04:53 -04:00
Samuel Clay
3804fbba5d Correcting mongodb on local installs since there is no auth. 2021-08-04 16:26:41 -04:00
Samuel Clay
3d63d95671 Forcing reconnect of analytics. 2021-08-03 21:17:37 -04:00
Samuel Clay
fe3a8bca3e Don't reconnect analytics server. 2021-08-03 21:16:16 -04:00
Samuel Clay
436580377b Fixing redis logging color 2021-08-03 17:21:49 -04:00
Samuel Clay
2a45cd0df7 Adding query logging for redis pubsub. 2021-08-03 17:19:45 -04:00
Samuel Clay
da381c9d78 Reducing reids hostname logging size. 2021-08-03 16:51:11 -04:00
Samuel Clay
078ce11034 Adding db-mongo-analytics specific conf. 2021-08-03 16:40:28 -04:00