Commit graph

133 commits

Author SHA1 Message Date
Samuel Clay
5a5457ded0 Verbose logging on feed decay algorithm 2025-09-16 13:39:34 -07:00
Samuel Clay
e632f13c68 Fixing zgrep, updating certbot procedure for auto renewal, openrss script for counting. 2025-08-13 09:47:51 -04:00
Samuel Clay
e2ef0666ba make nb-fast restarts all services without rebuilding 2025-05-12 22:15:43 -07:00
Samuel Clay
9cd40a71e8 Switching to key-based APNS cert. 2025-04-25 18:18:22 -07:00
Samuel Clay
5774aa389a No need to clean up mongo. 2025-02-25 17:40:21 -08:00
Samuel Clay
8f55e8ee9a Mongo user permissions 2025-02-25 17:37:40 -08:00
Samuel Clay
86a0e10374 Handling ubuntu + darwin 2025-02-25 17:31:37 -08:00
Samuel Clay
f35b99c295 Discover blog post draft. 2025-01-27 22:28:08 -08:00
Samuel Clay
153cb876b5 Moving discover indexing to its own queue. 2024-12-14 13:39:21 -08:00
Samuel Clay
6d58c79b02 Merge branch 'master' into discover
* master:
  Deprecating old blog.
  Adding macOS app to Goodies.
  Adding hblog as possible server in nginx.
  Adding docker rule for hblog:80
  New macOS blog post.
  Finishing macOS blog post. Need to upgrade minima theme.
  New screenshots of the macOS app
2024-10-26 09:44:18 -07:00
Samuel Clay
553c3db0a6 New macOS blog post. 2024-10-24 07:13:37 -07:00
Samuel Clay
1bd3de276f Showing 5 distinct related feeds in title bar. 2024-10-13 13:48:08 -07:00
Samuel Clay
838a16fdcd Styling third party shares. 2024-07-17 10:13:13 -04:00
Samuel Clay
573476e003 Merge branch 'master' into discover
* master: (84 commits)
  Cleanup postgres
  Adding support for hyphens in youtube usernames.
  New APNS updating cert instructions.
  Fiddling with metrics server.
  Handling broken youtube channel
  Youtube username/title
  Handling youtube usernames that are actually handles.
  Handling @handle youtube.com feeds when adding a feed.
  Users who are too far into paging now get a 404
  Updating youtube fetcher to use channels/playlists/users for everything, no longer relying on RSS/xml url.
  Updating certs.
  Handling too many feeds tasked.
  Bumping concurrency for feed fetchers.
  Icon importer also uses new consul domain.
  User not user agent
  Adding individual IP addresses from hetzner.
  Moving hetzner migration doc to archive
  Formatting
  Adding dwww
  Moving node consul
  ...
2024-04-24 09:25:45 -04:00
Samuel Clay
006bdc7964 Logging failed page fetch timeout. 2024-02-25 15:41:35 -05:00
Samuel Clay
f9ad93e336 Adding hetzner servers. 2024-01-07 10:00:39 -05:00
Samuel Clay
95a0acbc30 Adding similiar feeds many-to-many field. Using KNN classifier for similar feeds using user-feed-rating csv. Now to productionalize. 2023-12-27 08:24:49 -05:00
Samuel Clay
3acd6d1f82 It's a wash if this does anything. 2023-11-10 17:13:46 -05:00
Samuel Clay
2feaf087e7 Running scikit-surprise 2023-10-11 09:06:44 -04:00
Samuel Clay
423434f9b0 Adding surprise scipy package, adding new CollaborativelyFilteredRecommendation model and management command. 2023-10-09 20:06:13 -04:00
Samuel Clay
cd2a32bee3 Reducing dbs. 2023-06-08 07:51:39 -04:00
Samuel Clay
1065c964fd Fixing elasticsearch to allow consul to assume its OK. 2022-12-26 19:17:14 -05:00
Samuel Clay
e06147aa71 Adding make collectstatic to make nb because even though it's slower, it fixes the #1 issue people have when deploying on their own machines. Fixes #1717. 2022-07-30 13:02:47 -04:00
Samuel Clay
619ac8c4b4 Adding make app=rss_feeds datamigration 2022-05-17 09:30:27 -04:00
Samuel Clay
3a106b51bb Merge branch 'master' into pro
* master:
  Don't change file permissions, let's see what happens.
  Bumping postgres server size, fixing monitor disk usage cronjob.
2022-05-16 10:34:40 -04:00
Samuel Clay
b2cdff8a85 Bumping postgres server size, fixing monitor disk usage cronjob. 2022-05-13 16:59:57 -04:00
Samuel Clay
de1fb57406 Finishing RFC 5005 archive fetcher (for #1109). Also fixing mongo commands and adding checks for archive subscribers. 2022-04-29 11:31:32 -04:00
Samuel Clay
98d30cbc75 Mongo dump and restore. Needs crossover between but for now can be handled manually by ensuring the data directory is available between dump and restore.. 2022-04-20 22:37:54 -04:00
Samuel Clay
b90501d955 Adding a paypal archive return page. Also redrawing when premium expire/renewal status changes on payment fetch in account dialog. 2022-04-06 15:56:13 -04:00
Samuel Clay
65fac88590 Merge branch 'master' into pro
* master:
  Renaming www in terraform until next www server
  Removing audio handler
  Removing unused audio handler shim that turned audio into flash for older browsers.
  Adding global shared stories feeds so they appear on the dashboard even when not subscribed.
  Adding metrics to a single server, let's see how these shake out.
  Don't show prometheus metrics for migrations.
  Adding django prometheus metrics.
  Beginning upgrade to Mongo 4.0 (eventually to 5.0 to get time-series dbs)
2022-04-01 16:47:03 -04:00
Samuel Clay
893619596b Adding django prometheus metrics. 2022-03-31 15:34:33 -04:00
Samuel Clay
e1ad3bcad1 Merge branch 'master' into pro
* master:
  www -> haproxy group
  Rearranging mongo servers
2022-03-15 17:19:11 -04:00
Samuel Clay
8c56c6f98e www -> haproxy group 2022-03-15 17:19:01 -04:00
Samuel Clay
f33cbcf312 Merge branch 'master' into pro
* master: (72 commits)
  Missing import
  Correctly decoding shared stories in RSS feed.
  Adding scroll to feed and story options popover (Style menu).
  Magazine view blog post.
  Refactoring story content preview with newlines preserved.
  Boosting db profiling from 1% to 5%.
  Ignore data uris for widget, they are never the big images
  Wait to strip data uri header until it's time.
  Moving encoding
  Don't double encode base64 images for widgets.
  Preserving paragraphs in magazine and grid views.
  Fixing bad instrumentation on mongo and redis logs.
  Wrong url for db profiler.
  Let's try showing only single feeds in the DB profiler to see if it's better at showing abnormalities with less noise.
  Fixing strict bug where  wasn't defined.
  Showing grid image previews. Not sure how this ever worked before.
  Grid has image if found already.
  Showing grid image on load if youtube
  Only monitor river_stories for db times
  Not hitting a db shouldnt be colored any differently than a small value.
  ...
2022-03-09 11:41:40 -05:00
Samuel Clay
13533f84b7 Removing broken CSS. 2022-02-24 10:41:20 -05:00
Samuel Clay
01ad0e8656 Paypal refunds are now working. Need to add partial balances when upgrading tiers on paypal. Eventually need to think about refunds for switching providers. 2022-02-10 13:40:07 -05:00
Samuel Clay
a1517b6266 Adding paypal sub id and paypalrestsdk library. Paypal has what it needs, now to hook up the API which is still unauthorized. 2022-02-08 12:49:40 -05:00
Samuel Clay
02cb4cf2b4 Cleaning up docker compose 2022-01-26 13:35:06 -05:00
Samuel Clay
18aeafffa2 Cleaning up docker compose 2022-01-26 13:33:59 -05:00
Samuel Clay
97fc9a4f29 Adding makefile command for stripe testing commands 2022-01-25 15:47:01 -05:00
Samuel Clay
71b9b2f3d8 Paypal/stripe return needs user profile for JS. 2022-01-25 10:26:45 -05:00
Samuel Clay
2554984717 Merge branch 'pipeline' into pro
* pipeline:
  Installing Java in deploy image to get closure compiler working. Also updating closure binary path.
  Ensuring dependencies met for deployment on local.
  Updating to docker buildx for multi-arch builds (amd64 + arm64)
2022-01-19 13:01:38 -05:00
Samuel Clay
0f27ba81e0 Updating to docker buildx for multi-arch builds (amd64 + arm64) 2022-01-18 17:25:07 -05:00
Samuel Clay
2792e7305a Syncing redis on premium upgrade to sync new unread date possibilities for archive users. 2022-01-13 16:56:30 -05:00
Samuel Clay
f615fccdad Coutning archive and pro premiums correctly. 2022-01-11 15:56:19 -05:00
Samuel Clay
04c62b11c7 Cleaning up static assets. Needs to run on first make nb or DEBUG_ASSETS needs to always be set to True. 2022-01-04 15:54:37 -05:00
Samuel Clay
b8eccdbed4 Merge branch 'master' into pipeline
* master:
  Fixing names for state timeline, also fixing node server bootstrap.
  Moving up haproxy state timeline
  Bumping haproxy state timeline scrapes.
  Only need haproxy for haproxy metrics
  Refactored state timeline to work as a flask metrics server on haproxy. Needs ansible configs to deploy on www server.
  Forgot to save merge conflict.
  fix service state timeline key error and change scrape interval
  Revert "Revert "Merge branch 'state_timeline'""
  Revert "Merge branch 'state_timeline'"
  Fixing syntax error
  Solving docker networking container issues by turning iptables back on but using ufw after.rules that allow ufw to retain blocking ability. Works on Redis, now to move to mongo.
  Mongo analytics monitor not reayd yet
  Turning consul monitor back on.
  Turning on analytics
  Need to strip csv values.
  Binding redis to public interface.
  add grafana dashboard backup functionality
2022-01-04 14:44:57 -05:00
Samuel Clay
9ced4ccadd Merge branch 'master' into pipeline
* master: (84 commits)
  Deploying new analytics server.
  Create mongodb database user. Only runs on analytics, but should work fine on main mongo.
  Adding second mongo analytics db, cleaning up mongo analytics auth.
  Adding RSS autodiscovery to blog. Thanks https://forum.newsblur.com/t/irony-newsblur-cant-auto-extract-rss-feed-from-blog-newsblur-com/8849.
  Revert "Merge branch 'state_timeline'"
  Fixing syntax error
  Solving docker networking container issues by turning iptables back on but using ufw after.rules that allow ufw to retain blocking ability. Works on Redis, now to move to mongo.
  fix flask_metrics ansible role
  add state timeline dashboard to grafana
  move state timeline to one single flask that gets astate for all services
  Mongo analytics monitor not reayd yet
  Turning consul monitor back on.
  Turning on analytics
  Need to strip csv values.
  Binding redis to public interface.
  Restore postgres backup. Needs work.
  Prepping postgres backup
  Prepping postgres transition, writing redis db maintenance.
  Prepping redis for moving back to primary.
  Updating redis backup to include percentage progress and access to the correct data dump.
  ...
2021-12-16 15:41:57 -05:00
Jonathan Math
6189dd4c01 add grafana dashboard backup functionality 2021-12-15 13:40:23 -05:00
Samuel Clay
ba6b4e5d1c Prepping postgres transition, writing redis db maintenance. 2021-12-13 16:09:27 -05:00