Commit graph

1878 commits

Author SHA1 Message Date
Samuel Clay
1f9eaf4489 5 seconds for existing story check. 2021-03-25 12:23:19 -04:00
Samuel Clay
02138db5ca Switching from elasticsearch and dsl to pyelasticsearch. Feed autocomplete still doesn't work yet. 2021-03-24 21:09:26 -04:00
Samuel Clay
3e61c5e907 Feed autocomplete is trnaslated to elasticsearch 7, but doesn't work quite yet. 2021-03-23 15:30:44 -04:00
Samuel Clay
e1b2583950 Swapping out pyes for elasticsearch-py. Half-way done, got indices and mappings in, now need to work on querying and storing. 2021-03-18 16:34:28 -04:00
Samuel Clay
33c4c5ed4f Removing tracing handler for sentry. 2021-03-17 10:42:58 -04:00
Samuel Clay
2906e92499 Merging all four node servers into a single one on port 8008. Updating HAProxy on both consul and docker-compose. 2021-03-16 19:34:11 -04:00
Samuel Clay
dbec2708da Only show active feeds in dashboard river. Also link dashboard river title and icon. 2021-03-04 10:24:17 -05:00
Samuel Clay
8e088ba350 Switching to py3 html parser. 2021-03-02 09:46:59 -05:00
Samuel Clay
23abeb0b0a Handling feedparser backup failing. 2021-02-28 20:39:57 -05:00
Samuel Clay
4cb24cf7f2 Since NewsBlur proxies all http images over https, the url can change, so acknowledge urls that are https on the original text but http on the feed 2021-02-26 14:06:42 -05:00
Samuel Clay
88d051abbf Adding first image in original text when the story image no longer appears in the text. This should show more photos when reading by text. 2021-02-26 12:10:30 -05:00
Samuel Clay
c36373de61 cutoff needs to be an integer so mongo can handle the slice. 2021-02-25 19:54:00 -05:00
Samuel Clay
ba6a8c81e0 Switching from raven to sentry sdk. 2021-02-25 19:52:36 -05:00
Samuel Clay
07e7218d4d Fixing py3 issue with total being a float. 2021-02-25 19:34:21 -05:00
Samuel Clay
c5f4734de4 Finally getting sentry to work in feed fetching. 2021-02-25 19:32:48 -05:00
Samuel Clay
1b7addab44 Undoing type change 2021-02-25 19:23:43 -05:00
Samuel Clay
1d16ec88b1 Wrapping feed fetcher in sentry exception capturing. 2021-02-25 19:13:21 -05:00
Jonathan Math
bc66ad7659 handle empty list for drain_task_feeds 2021-02-23 10:08:40 -05:00
Samuel Clay
c7f4cbb13d Fixing a substantial number of encoding errors. Also fixing OPML uploads. 2021-01-24 21:05:22 -05:00
Samuel Clay
8cf070a72d Merge branch 'master' into dashboard3
* master:
  Removing old types field. Needed to upgrade to mongo 3.4.
  Checking if a feed has any notifications on it, set a max of 30 min fetch (yes this is a pretty good way of getting your feeds to fetch faster, but youll have to deal with the notifications).
  Fix task fetch monitor.
  Attempting to see iOS premium status over email.
  Handling work queue missing error.
2021-01-23 17:55:01 -05:00
Samuel Clay
6badcaf294 Removing old types field. Needed to upgrade to mongo 3.4. 2021-01-23 16:48:13 -05:00
Samuel Clay
33183967b3 Checking if a feed has any notifications on it, set a max of 30 min fetch (yes this is a pretty good way of getting your feeds to fetch faster, but youll have to deal with the notifications). 2021-01-23 14:20:37 -05:00
Samuel Clay
ab2d9804ef Adjusting to newsblur_web. Also adding feed chooser to dashboard river. Needs hookup. 2021-01-12 15:52:10 -05:00
Samuel Clay
f5b144f22f Merge branch 'docker_django3.1' into dashboard3
* docker_django3.1:
  Ignore send_message_with_response if not available (for some reason).
  Updating to latest pymongo and mongoengine for py2.
2021-01-06 16:39:23 -05:00
Samuel Clay
30828e8291 Merge branch 'master' into docker_django3.1
* master:
  Ignore send_message_with_response if not available (for some reason).
  Updating to latest pymongo and mongoengine for py2.
2021-01-06 16:39:00 -05:00
Samuel Clay
a1f4d67d27 Changing how tasks are called. 2021-01-06 14:42:24 -05:00
Samuel Clay
82fecb23a6 Merge branch 'docker_django3.1' into dashboard3
* docker_django3.1: (117 commits)
  Adding docker to node scripts.
  Removing node_modules from repo
  Whoops
  Better logging for broken paypal ipns.
  Adding RSS Tracker for Windows to Goodies.
  Refactoring socketio so real-time works around username restrictions.
  Socket.IO v3 on the server too.
  Switching to Socket.IO v3.
  Haproxy splitting for socketio 3
  Socket.io 3
  Handling missing feed.
  Only nb.local.com uses ports for websockets.
  Adding GrazeTEN
  Full node install on setup.
  Updating toprc
  correct newsblur to newsblur_web in tasks
  Changing mongodb server.
  Fixing toprc
  Attempting to collect queries for 1% of requests by using DEBUG.
  Fixing mongo install.
  ...
2021-01-05 19:36:49 -05:00
Jonathan Math
a2c7acf3ff correct newsblur to newsblur_web in tasks 2021-01-04 13:33:56 -05:00
Jonathan Math
79f249878b remove use of ORIGINAL_PAGE_SERVER 2021-01-03 11:09:25 -05:00
Samuel Clay
f39a0405cf NullBooleanField deprecated 2020-12-09 17:20:08 -05:00
Samuel Clay
92d8caf2b4 Merge branch 'django3.1' into docker_django3.1
* django3.1:
  Auto-enabling newsblur3 virtualenv.
  Fixing 2to3 string encoding.
  Rolling back pymongo.
  Upgrading all dependencies. This probably won't work (looking at you timezones).
  Updating feedparser.
  Django 3.1.
2020-12-08 13:06:56 -05:00
Samuel Clay
05756155b1 Merge branch 'django3.0' into docker_django3.0
* django3.0: (184 commits)
  Removing log override
  Moving logging over to the newsblur log.
  Fixing search indexer background task for new celery.
  Attempting to add gunicorn errors to console/log.
  Better handling of missing subs.
  Handling missing user sub on feed delete.
  Correct encoding for strings on systems that don't have utf-8 as default encoding.
  Writing in the real urllib3 dependency for requests.
  Upgrading requests due to urllib3 incompatibility.
  Login required should use the next parameter.
  Upgrading django oauth toolkit for django 1.11.
  Handling newsletters with multiple recipients.
  Extracting image urls sometimes fails.
  Handling ajax errors in json views.
  Adding timeouts to most outbound requests.
  Sentry SDK 0.19.4.
  Removing imperfect proxy warning for every story.
  Found four more GET/POST crosses.
  Feed unread count may need a POST.
  Namespacing settings.
  ...
2020-12-08 13:04:39 -05:00
Samuel Clay
aea4fcbe5b Merge branch 'django3.0' into django3.1
* django3.0: (27 commits)
  Removing log override
  Moving logging over to the newsblur log.
  Fixing search indexer background task for new celery.
  Attempting to add gunicorn errors to console/log.
  Better handling of missing subs.
  Handling missing user sub on feed delete.
  Correct encoding for strings on systems that don't have utf-8 as default encoding.
  Writing in the real urllib3 dependency for requests.
  Upgrading requests due to urllib3 incompatibility.
  Login required should use the next parameter.
  Upgrading django oauth toolkit for django 1.11.
  Handling newsletters with multiple recipients.
  Extracting image urls sometimes fails.
  Handling ajax errors in json views.
  Adding timeouts to most outbound requests.
  Sentry SDK 0.19.4.
  Removing imperfect proxy warning for every story.
  Found four more GET/POST crosses.
  Feed unread count may need a POST.
  Namespacing settings.
  ...
2020-12-08 09:10:51 -05:00
Samuel Clay
8371c635f7 Merge branch 'master' into django2.0
* master: (27 commits)
  Removing log override
  Moving logging over to the newsblur log.
  Fixing search indexer background task for new celery.
  Attempting to add gunicorn errors to console/log.
  Better handling of missing subs.
  Handling missing user sub on feed delete.
  Correct encoding for strings on systems that don't have utf-8 as default encoding.
  Writing in the real urllib3 dependency for requests.
  Upgrading requests due to urllib3 incompatibility.
  Login required should use the next parameter.
  Upgrading django oauth toolkit for django 1.11.
  Handling newsletters with multiple recipients.
  Extracting image urls sometimes fails.
  Handling ajax errors in json views.
  Adding timeouts to most outbound requests.
  Sentry SDK 0.19.4.
  Removing imperfect proxy warning for every story.
  Found four more GET/POST crosses.
  Feed unread count may need a POST.
  Namespacing settings.
  ...
2020-12-08 09:09:25 -05:00
Samuel Clay
8bf74cbcef Handling newsletters with multiple recipients. 2020-12-06 18:23:31 -05:00
Samuel Clay
18db90df9a Extracting image urls sometimes fails. 2020-12-06 12:30:24 -05:00
Samuel Clay
1a5d440582 Adding timeouts to most outbound requests. 2020-12-06 11:37:01 -05:00
Samuel Clay
5085cd8323 Found four more GET/POST crosses. 2020-12-06 10:26:09 -05:00
Samuel Clay
77465354a3 Merge branch 'django1.11'
* django1.11: (140 commits)
  Massaging roles.
  Adding stats logging
  Separating app from web role.
  Upgrading work servers.
  request.raw_post_data -> request.body (django 1.6)
  Upgrading pgbouncer to 1.15.0.
  Fixing supervisor path issues
  Upgrading setuptools
  Fixing flask
  Removing dependency, fixing encoding issue for pages.
  Fixing DB Monitor.
  Updating User Agent for all fetchers.
  Ignoring VSCode.
  Fixing Statistics by fixing how timezones are handled.
  Fixing API page.
  Fixing both Stripe form and Login form.
  Handling missing user during auth.
  Going back on AllowAllUsers backend, as it doesn't work with django 1.5 authentication sessions.
  Going back and forth on this but without AllowAllUsersModelBackend, then is_active=False won't allow authentication.
  De-vendorizing django-paypal
  ...
2020-12-05 14:37:44 -05:00
Samuel Clay
47f1b59621 get_feed_by_url should do the same as get_feed_from_url. 2020-12-05 12:51:30 -05:00
Samuel Clay
4bb9d47144 Upgrading all dependencies. This probably won't work (looking at you timezones). 2020-12-03 16:50:54 -05:00
Samuel Clay
f041bbceae 2to3 2020-12-03 14:16:47 -05:00
Samuel Clay
b89e7dc429 Merge branch 'django1.11' into django2.0
* django1.11: (152 commits)
  request.raw_post_data -> request.body (django 1.6)
  Upgrading pgbouncer to 1.15.0.
  Finishing off Postgresql 13 upgrade.
  Upgrading to Postgresql 13.
  Ubuntu 20.04
  Fixing supervisor path issues
  Upgrading setuptools
  Fixing flask
  Handling over capacity for twitter.
  Max length for image_urls.
  Properly filtering newsletter feeds.
  Fixing issue with text importer on feed-less urls.
  Removing dependency, fixing encoding issue for pages.
  Fixing DB Monitor.
  Updating User Agent for all fetchers.
  Ignoring VSCode.
  Fixing DB Monitor.
  Updating User Agent for all fetchers.
  Ignoring VSCode.
  Fixing Statistics by fixing how timezones are handled.
  ...
2020-12-03 14:04:26 -05:00
Samuel Clay
f2010483b7 Merge branch 'master' into django1.11
* master:
  Max length for image_urls.
  Properly filtering newsletter feeds.
  Fixing issue with text importer on feed-less urls.
2020-11-30 18:55:57 -05:00
Samuel Clay
0f48e88681 Max length for image_urls. 2020-11-30 18:53:52 -05:00
Samuel Clay
4b8589a259 Fixing issue with text importer on feed-less urls. 2020-11-30 18:27:49 -05:00
Samuel Clay
e75da8771c Removing dependency, fixing encoding issue for pages. 2020-11-30 18:17:26 -05:00
Samuel Clay
21f40c253f Updating User Agent for all fetchers. 2020-11-30 18:03:15 -05:00
Samuel Clay
fa43eed1b8 Updating User Agent for all fetchers. 2020-11-30 15:48:59 -05:00
Samuel Clay
ce52802ebd Fixing Statistics by fixing how timezones are handled. 2020-11-23 18:58:00 -05:00