Commit graph

1270 commits

Author SHA1 Message Date
Samuel Clay
221bf86664 Color 2022-01-11 10:48:07 -05:00
Samuel Clay
9857ddbcb5 Typo 2022-01-10 17:09:55 -05:00
Samuel Clay
db2a87df06 Counting pro subscribers. 2022-01-10 17:00:27 -05:00
Samuel Clay
6db2ed94ce Bump twitter feeds up to 2 hours to fix rate limits. 2021-08-24 17:25:31 -04:00
Samuel Clay
0cd123b60f Ignore feedburner images. 2021-07-08 13:55:56 -04:00
Samuel Clay
c6ce8cc36a Everything is smart bytes when it comes to original pages. 2021-05-12 21:19:09 -04:00
Samuel Clay
c7fac210d5 -1 minute is -1 days 23:59:00, so bumping days. 2021-05-05 14:56:08 -04:00
Samuel Clay
34c9ee712c Fixing search on docker. Also setting up docker installs to always have active subscribers so feeds are never stale. 2021-04-27 20:49:10 -04:00
Samuel Clay
ddc8563080 Boosting single subscriber feeds that publish once a month from 12 hours to 6 hours. 2021-04-20 18:46:16 -04:00
Samuel Clay
a4b7927b92 Nice number of feed fetchers 2021-04-16 17:25:57 -04:00
Samuel Clay
fd2aa291d1 Typo 2021-04-13 16:36:00 -04:00
Samuel Clay
5a682ff2b9 Docker compose should be v2 for mem_limits, not v3 for swarm. 2021-04-13 16:35:06 -04:00
Samuel Clay
4db47a1771 Using redis exclusively for well read score. 2021-04-13 16:24:49 -04:00
Samuel Clay
ba46b2f85a Don't fetch broken feeds immediately as they can get stuck in a loop. 2021-04-07 15:32:17 -04:00
Samuel Clay
1deae3d810 Fixing broken pages. 2021-04-02 14:46:52 -04:00
Samuel Clay
1bd5ac2219 Fixing #1434. Original pages weren't working until now due to encoding issues. 2021-04-01 18:22:50 -04:00
Samuel Clay
5822af1645 Py3 urlparse 2021-03-25 18:47:15 -04:00
Samuel Clay
9f7f00efaa Combining absolute with relative urls. 2021-03-25 18:46:21 -04:00
Samuel Clay
712e44619b Fixing feed autocomplete 2021-03-25 15:38:15 -04:00
Samuel Clay
9d1a61fd79 Handling broken latest_story_content. 2021-03-25 14:54:49 -04:00
Samuel Clay
2fe3153770 Adding MStory str and removing vendorized paypal. 2021-03-25 14:35:14 -04:00
Samuel Clay
1f9eaf4489 5 seconds for existing story check. 2021-03-25 12:23:19 -04:00
Samuel Clay
02138db5ca Switching from elasticsearch and dsl to pyelasticsearch. Feed autocomplete still doesn't work yet. 2021-03-24 21:09:26 -04:00
Samuel Clay
3e61c5e907 Feed autocomplete is trnaslated to elasticsearch 7, but doesn't work quite yet. 2021-03-23 15:30:44 -04:00
Samuel Clay
2906e92499 Merging all four node servers into a single one on port 8008. Updating HAProxy on both consul and docker-compose. 2021-03-16 19:34:11 -04:00
Samuel Clay
dbec2708da Only show active feeds in dashboard river. Also link dashboard river title and icon. 2021-03-04 10:24:17 -05:00
Samuel Clay
8e088ba350 Switching to py3 html parser. 2021-03-02 09:46:59 -05:00
Samuel Clay
88d051abbf Adding first image in original text when the story image no longer appears in the text. This should show more photos when reading by text. 2021-02-26 12:10:30 -05:00
Samuel Clay
c36373de61 cutoff needs to be an integer so mongo can handle the slice. 2021-02-25 19:54:00 -05:00
Samuel Clay
07e7218d4d Fixing py3 issue with total being a float. 2021-02-25 19:34:21 -05:00
Samuel Clay
c5f4734de4 Finally getting sentry to work in feed fetching. 2021-02-25 19:32:48 -05:00
Samuel Clay
1b7addab44 Undoing type change 2021-02-25 19:23:43 -05:00
Samuel Clay
1d16ec88b1 Wrapping feed fetcher in sentry exception capturing. 2021-02-25 19:13:21 -05:00
Jonathan Math
bc66ad7659 handle empty list for drain_task_feeds 2021-02-23 10:08:40 -05:00
Samuel Clay
c7f4cbb13d Fixing a substantial number of encoding errors. Also fixing OPML uploads. 2021-01-24 21:05:22 -05:00
Samuel Clay
8cf070a72d Merge branch 'master' into dashboard3
* master:
  Removing old types field. Needed to upgrade to mongo 3.4.
  Checking if a feed has any notifications on it, set a max of 30 min fetch (yes this is a pretty good way of getting your feeds to fetch faster, but youll have to deal with the notifications).
  Fix task fetch monitor.
  Attempting to see iOS premium status over email.
  Handling work queue missing error.
2021-01-23 17:55:01 -05:00
Samuel Clay
6badcaf294 Removing old types field. Needed to upgrade to mongo 3.4. 2021-01-23 16:48:13 -05:00
Samuel Clay
33183967b3 Checking if a feed has any notifications on it, set a max of 30 min fetch (yes this is a pretty good way of getting your feeds to fetch faster, but youll have to deal with the notifications). 2021-01-23 14:20:37 -05:00
Samuel Clay
ab2d9804ef Adjusting to newsblur_web. Also adding feed chooser to dashboard river. Needs hookup. 2021-01-12 15:52:10 -05:00
Samuel Clay
a1f4d67d27 Changing how tasks are called. 2021-01-06 14:42:24 -05:00
Samuel Clay
82fecb23a6 Merge branch 'docker_django3.1' into dashboard3
* docker_django3.1: (117 commits)
  Adding docker to node scripts.
  Removing node_modules from repo
  Whoops
  Better logging for broken paypal ipns.
  Adding RSS Tracker for Windows to Goodies.
  Refactoring socketio so real-time works around username restrictions.
  Socket.IO v3 on the server too.
  Switching to Socket.IO v3.
  Haproxy splitting for socketio 3
  Socket.io 3
  Handling missing feed.
  Only nb.local.com uses ports for websockets.
  Adding GrazeTEN
  Full node install on setup.
  Updating toprc
  correct newsblur to newsblur_web in tasks
  Changing mongodb server.
  Fixing toprc
  Attempting to collect queries for 1% of requests by using DEBUG.
  Fixing mongo install.
  ...
2021-01-05 19:36:49 -05:00
Samuel Clay
f39a0405cf NullBooleanField deprecated 2020-12-09 17:20:08 -05:00
Samuel Clay
05756155b1 Merge branch 'django3.0' into docker_django3.0
* django3.0: (184 commits)
  Removing log override
  Moving logging over to the newsblur log.
  Fixing search indexer background task for new celery.
  Attempting to add gunicorn errors to console/log.
  Better handling of missing subs.
  Handling missing user sub on feed delete.
  Correct encoding for strings on systems that don't have utf-8 as default encoding.
  Writing in the real urllib3 dependency for requests.
  Upgrading requests due to urllib3 incompatibility.
  Login required should use the next parameter.
  Upgrading django oauth toolkit for django 1.11.
  Handling newsletters with multiple recipients.
  Extracting image urls sometimes fails.
  Handling ajax errors in json views.
  Adding timeouts to most outbound requests.
  Sentry SDK 0.19.4.
  Removing imperfect proxy warning for every story.
  Found four more GET/POST crosses.
  Feed unread count may need a POST.
  Namespacing settings.
  ...
2020-12-08 13:04:39 -05:00
Samuel Clay
8371c635f7 Merge branch 'master' into django2.0
* master: (27 commits)
  Removing log override
  Moving logging over to the newsblur log.
  Fixing search indexer background task for new celery.
  Attempting to add gunicorn errors to console/log.
  Better handling of missing subs.
  Handling missing user sub on feed delete.
  Correct encoding for strings on systems that don't have utf-8 as default encoding.
  Writing in the real urllib3 dependency for requests.
  Upgrading requests due to urllib3 incompatibility.
  Login required should use the next parameter.
  Upgrading django oauth toolkit for django 1.11.
  Handling newsletters with multiple recipients.
  Extracting image urls sometimes fails.
  Handling ajax errors in json views.
  Adding timeouts to most outbound requests.
  Sentry SDK 0.19.4.
  Removing imperfect proxy warning for every story.
  Found four more GET/POST crosses.
  Feed unread count may need a POST.
  Namespacing settings.
  ...
2020-12-08 09:09:25 -05:00
Samuel Clay
8bf74cbcef Handling newsletters with multiple recipients. 2020-12-06 18:23:31 -05:00
Samuel Clay
18db90df9a Extracting image urls sometimes fails. 2020-12-06 12:30:24 -05:00
Samuel Clay
77465354a3 Merge branch 'django1.11'
* django1.11: (140 commits)
  Massaging roles.
  Adding stats logging
  Separating app from web role.
  Upgrading work servers.
  request.raw_post_data -> request.body (django 1.6)
  Upgrading pgbouncer to 1.15.0.
  Fixing supervisor path issues
  Upgrading setuptools
  Fixing flask
  Removing dependency, fixing encoding issue for pages.
  Fixing DB Monitor.
  Updating User Agent for all fetchers.
  Ignoring VSCode.
  Fixing Statistics by fixing how timezones are handled.
  Fixing API page.
  Fixing both Stripe form and Login form.
  Handling missing user during auth.
  Going back on AllowAllUsers backend, as it doesn't work with django 1.5 authentication sessions.
  Going back and forth on this but without AllowAllUsersModelBackend, then is_active=False won't allow authentication.
  De-vendorizing django-paypal
  ...
2020-12-05 14:37:44 -05:00
Samuel Clay
47f1b59621 get_feed_by_url should do the same as get_feed_from_url. 2020-12-05 12:51:30 -05:00
Samuel Clay
b89e7dc429 Merge branch 'django1.11' into django2.0
* django1.11: (152 commits)
  request.raw_post_data -> request.body (django 1.6)
  Upgrading pgbouncer to 1.15.0.
  Finishing off Postgresql 13 upgrade.
  Upgrading to Postgresql 13.
  Ubuntu 20.04
  Fixing supervisor path issues
  Upgrading setuptools
  Fixing flask
  Handling over capacity for twitter.
  Max length for image_urls.
  Properly filtering newsletter feeds.
  Fixing issue with text importer on feed-less urls.
  Removing dependency, fixing encoding issue for pages.
  Fixing DB Monitor.
  Updating User Agent for all fetchers.
  Ignoring VSCode.
  Fixing DB Monitor.
  Updating User Agent for all fetchers.
  Ignoring VSCode.
  Fixing Statistics by fixing how timezones are handled.
  ...
2020-12-03 14:04:26 -05:00
Samuel Clay
f2010483b7 Merge branch 'master' into django1.11
* master:
  Max length for image_urls.
  Properly filtering newsletter feeds.
  Fixing issue with text importer on feed-less urls.
2020-11-30 18:55:57 -05:00