Commit graph

775 commits

Author SHA1 Message Date
Samuel Clay
619dec1688 gunicorn can still run on port 8000, docker will translate the port. 2025-02-02 12:34:23 -08:00
Samuel Clay
8053c40b7b Fixing docker commands, adding health check. 2025-02-02 12:30:43 -08:00
Samuel Clay
c0c7a2f2de Blue green deploy, doesn't work yet. 2025-01-21 14:30:31 -08:00
Samuel Clay
be7380a0a4 Small fix to requirements (wrong pyflakes) 2024-12-24 18:05:28 -05:00
Samuel Clay
58754abaa7 Fixing touch discover date typo 2024-12-21 12:50:50 -05:00
Samuel Clay
ba87788afb isort 2024-12-09 21:33:57 -08:00
Samuel Clay
ecc3609244 Default to 'all' if no section is highlighted in Related Stories. 2024-12-07 11:29:07 -08:00
Samuel Clay
2931e5d578 Moving to OpenAI Embeddings API. 2024-10-10 21:56:49 -07:00
Samuel Clay
796111e13c Adding Copy URL/Text/Print to default sharing services. 2024-08-04 19:09:43 -07:00
Samuel Clay
92f881e54b If hostname has staging in it, only 2 workers 2024-07-10 12:28:02 -04:00
Samuel Clay
711a6fbbfc Removing unused scikit-learn code. 2024-07-06 08:01:28 -04:00
Samuel Clay
0f579cbc03 Using sentence transformers and the MiniLM model to create embeddings for feeds. 2024-06-30 09:13:25 -04:00
Samuel Clay
27b1069302 Fixing docker image build 2024-06-29 14:37:51 -04:00
Samuel Clay
c63067754d Merge branch 'black-formatting' into discover
* black-formatting:
  Black formatting.
2024-04-24 09:49:17 -04:00
Samuel Clay
d1dafe7606 Black formatting. 2024-04-24 09:43:56 -04:00
Samuel Clay
573476e003 Merge branch 'master' into discover
* master: (84 commits)
  Cleanup postgres
  Adding support for hyphens in youtube usernames.
  New APNS updating cert instructions.
  Fiddling with metrics server.
  Handling broken youtube channel
  Youtube username/title
  Handling youtube usernames that are actually handles.
  Handling @handle youtube.com feeds when adding a feed.
  Users who are too far into paging now get a 404
  Updating youtube fetcher to use channels/playlists/users for everything, no longer relying on RSS/xml url.
  Updating certs.
  Handling too many feeds tasked.
  Bumping concurrency for feed fetchers.
  Icon importer also uses new consul domain.
  User not user agent
  Adding individual IP addresses from hetzner.
  Moving hetzner migration doc to archive
  Formatting
  Adding dwww
  Moving node consul
  ...
2024-04-24 09:25:45 -04:00
Samuel Clay
a5222d9675 Updating youtube fetcher to use channels/playlists/users for everything, no longer relying on RSS/xml url. 2024-03-03 12:59:12 -05:00
Samuel Clay
4d6f0b5000 Upping app gunicorn workers. 2024-02-18 08:12:28 -05:00
Samuel Clay
1453c3ee27 Updating postgres for standby. 2024-01-27 23:11:36 -05:00
Samuel Clay
95a0acbc30 Adding similiar feeds many-to-many field. Using KNN classifier for similar feeds using user-feed-rating csv. Now to productionalize. 2023-12-27 08:24:49 -05:00
Samuel Clay
c360682cc3 Using tfidf and lightfm for recs, only recommeneds the same feeds every time. 2023-12-26 11:01:53 -05:00
Samuel Clay
6acac46143 More scikit-learn experiments. 2023-10-11 12:54:16 -04:00
Samuel Clay
2feaf087e7 Running scikit-surprise 2023-10-11 09:06:44 -04:00
Samuel Clay
423434f9b0 Adding surprise scipy package, adding new CollaborativelyFilteredRecommendation model and management command. 2023-10-09 20:06:13 -04:00
Samuel Clay
7afed3e897 Attempting to add openai model. 2023-08-20 07:43:21 -04:00
Samuel Clay
1065c964fd Fixing elasticsearch to allow consul to assume its OK. 2022-12-26 19:17:14 -05:00
Samuel Clay
9e21d03255 Merging SDIFFSTORE and ZINTERSTORE into a single ZDIFFSTORE, thanks to redis 6.2.0. Requires new docker image. 2022-07-19 21:02:42 -04:00
Samuel Clay
8f1655d7d1 Revert "Merging SDIFFSTORE and ZINTERSTORE into a single ZDIFFSTORE, thanks to redis 6.2.0. Requires new docker image."
This reverts commit b3ec9fd748.
2022-07-18 22:45:22 -04:00
Samuel Clay
b3ec9fd748 Merging SDIFFSTORE and ZINTERSTORE into a single ZDIFFSTORE, thanks to redis 6.2.0. Requires new docker image. 2022-07-18 22:34:07 -04:00
Samuel Clay
ed7fc4a4e8 Updating signed image to use https for image proxy. 2022-05-26 12:17:45 -04:00
Samuel Clay
5dba6549ef Removing unused mongo mms utils and configs. 2022-05-10 20:53:56 -04:00
Samuel Clay
30898df3e5 Updating pymongo to mongodb 4.0 compatible. Any further and we lose mongo wire protocol inspection. 2022-05-10 11:05:26 -04:00
Samuel Clay
57b190e241 Adding url normalize library to update aging urlnorm.py from Mark Pilgrim. 2022-04-19 15:09:26 -04:00
Samuel Clay
283bba5cd3 Rearranging urls for paypal. Also updating to latest gunicorn. 2022-04-11 11:01:02 -04:00
Samuel Clay
893619596b Adding django prometheus metrics. 2022-03-31 15:34:33 -04:00
Samuel Clay
96c7416d1e Gunicorn workers at 3 or 4, depending on having at least 3.1+ GB of memory. 2022-02-28 14:41:29 -05:00
Samuel Clay
8cc4956f73 Adding paypal dependencies for new premium archive and premium pro. 2022-02-16 13:53:45 -05:00
Samuel Clay
b0acb97ac1 Archiving munin. 2022-02-01 14:59:05 -05:00
Samuel Clay
9ced4ccadd Merge branch 'master' into pipeline
* master: (84 commits)
  Deploying new analytics server.
  Create mongodb database user. Only runs on analytics, but should work fine on main mongo.
  Adding second mongo analytics db, cleaning up mongo analytics auth.
  Adding RSS autodiscovery to blog. Thanks https://forum.newsblur.com/t/irony-newsblur-cant-auto-extract-rss-feed-from-blog-newsblur-com/8849.
  Revert "Merge branch 'state_timeline'"
  Fixing syntax error
  Solving docker networking container issues by turning iptables back on but using ufw after.rules that allow ufw to retain blocking ability. Works on Redis, now to move to mongo.
  fix flask_metrics ansible role
  add state timeline dashboard to grafana
  move state timeline to one single flask that gets astate for all services
  Mongo analytics monitor not reayd yet
  Turning consul monitor back on.
  Turning on analytics
  Need to strip csv values.
  Binding redis to public interface.
  Restore postgres backup. Needs work.
  Prepping postgres backup
  Prepping postgres transition, writing redis db maintenance.
  Prepping redis for moving back to primary.
  Updating redis backup to include percentage progress and access to the correct data dump.
  ...
2021-12-16 15:41:57 -05:00
Samuel Clay
966f0998fc Removing unused traces of older redis servers. 2021-12-06 13:56:09 -05:00
Samuel Clay
c4bbd3a7a8 New file system finder, finds static assets. 2021-11-16 11:27:09 -05:00
Samuel Clay
b6db14dacd Switching from Jammit to django-pipeline. Doesn't work yet, needs compressor hooked up and assets checked. 2021-11-14 13:42:06 -05:00
Samuel Clay
5734225d6f Updating Redis to v6, adding redis servers. 2021-10-20 16:46:19 -04:00
Samuel Clay
1fdebf5c76 Removing haproxy=1 from health checks. Haproxy tests the full mongo db, so it's the default case, no need for exceptional query string like consul. 2021-10-19 11:11:11 -04:00
Jonathan Math
dd13af5bf1 add querystring logic to mongo db health check to accommodate consul vs haproxy checking 2021-10-11 11:11:42 -05:00
Samuel Clay
3e23b6394a Revert "Revert "Merge branch 'add_grafana_monitors'""
This reverts commit a7f5025a16.
2021-09-16 11:57:57 -05:00
Samuel Clay
3f00139541 Upgrading sentry on both web and node. 2021-09-10 15:58:22 -04:00
Samuel Clay
a7f5025a16 Revert "Merge branch 'add_grafana_monitors'"
This reverts commit f0c2176908, reversing
changes made to 72ef2e2f00.
2021-09-07 11:36:11 -04:00
Jonathan Math
25efd6d973 merge master 2021-09-05 22:42:32 -05:00
Samuel Clay
bf2a54e755 Fixing elasticsearch sysctl and config. 2021-08-13 11:44:18 -04:00