Commit graph

1257 commits

Author SHA1 Message Date
Samuel Clay
ba46b2f85a Don't fetch broken feeds immediately as they can get stuck in a loop. 2021-04-07 15:32:17 -04:00
Samuel Clay
1deae3d810 Fixing broken pages. 2021-04-02 14:46:52 -04:00
Samuel Clay
1bd5ac2219 Fixing #1434. Original pages weren't working until now due to encoding issues. 2021-04-01 18:22:50 -04:00
Samuel Clay
5822af1645 Py3 urlparse 2021-03-25 18:47:15 -04:00
Samuel Clay
9f7f00efaa Combining absolute with relative urls. 2021-03-25 18:46:21 -04:00
Samuel Clay
712e44619b Fixing feed autocomplete 2021-03-25 15:38:15 -04:00
Samuel Clay
9d1a61fd79 Handling broken latest_story_content. 2021-03-25 14:54:49 -04:00
Samuel Clay
2fe3153770 Adding MStory str and removing vendorized paypal. 2021-03-25 14:35:14 -04:00
Samuel Clay
1f9eaf4489 5 seconds for existing story check. 2021-03-25 12:23:19 -04:00
Samuel Clay
02138db5ca Switching from elasticsearch and dsl to pyelasticsearch. Feed autocomplete still doesn't work yet. 2021-03-24 21:09:26 -04:00
Samuel Clay
3e61c5e907 Feed autocomplete is trnaslated to elasticsearch 7, but doesn't work quite yet. 2021-03-23 15:30:44 -04:00
Samuel Clay
2906e92499 Merging all four node servers into a single one on port 8008. Updating HAProxy on both consul and docker-compose. 2021-03-16 19:34:11 -04:00
Samuel Clay
dbec2708da Only show active feeds in dashboard river. Also link dashboard river title and icon. 2021-03-04 10:24:17 -05:00
Samuel Clay
8e088ba350 Switching to py3 html parser. 2021-03-02 09:46:59 -05:00
Samuel Clay
88d051abbf Adding first image in original text when the story image no longer appears in the text. This should show more photos when reading by text. 2021-02-26 12:10:30 -05:00
Samuel Clay
c36373de61 cutoff needs to be an integer so mongo can handle the slice. 2021-02-25 19:54:00 -05:00
Samuel Clay
07e7218d4d Fixing py3 issue with total being a float. 2021-02-25 19:34:21 -05:00
Samuel Clay
c5f4734de4 Finally getting sentry to work in feed fetching. 2021-02-25 19:32:48 -05:00
Samuel Clay
1b7addab44 Undoing type change 2021-02-25 19:23:43 -05:00
Samuel Clay
1d16ec88b1 Wrapping feed fetcher in sentry exception capturing. 2021-02-25 19:13:21 -05:00
Jonathan Math
bc66ad7659 handle empty list for drain_task_feeds 2021-02-23 10:08:40 -05:00
Samuel Clay
c7f4cbb13d Fixing a substantial number of encoding errors. Also fixing OPML uploads. 2021-01-24 21:05:22 -05:00
Samuel Clay
8cf070a72d Merge branch 'master' into dashboard3
* master:
  Removing old types field. Needed to upgrade to mongo 3.4.
  Checking if a feed has any notifications on it, set a max of 30 min fetch (yes this is a pretty good way of getting your feeds to fetch faster, but youll have to deal with the notifications).
  Fix task fetch monitor.
  Attempting to see iOS premium status over email.
  Handling work queue missing error.
2021-01-23 17:55:01 -05:00
Samuel Clay
6badcaf294 Removing old types field. Needed to upgrade to mongo 3.4. 2021-01-23 16:48:13 -05:00
Samuel Clay
33183967b3 Checking if a feed has any notifications on it, set a max of 30 min fetch (yes this is a pretty good way of getting your feeds to fetch faster, but youll have to deal with the notifications). 2021-01-23 14:20:37 -05:00
Samuel Clay
ab2d9804ef Adjusting to newsblur_web. Also adding feed chooser to dashboard river. Needs hookup. 2021-01-12 15:52:10 -05:00
Samuel Clay
a1f4d67d27 Changing how tasks are called. 2021-01-06 14:42:24 -05:00
Samuel Clay
82fecb23a6 Merge branch 'docker_django3.1' into dashboard3
* docker_django3.1: (117 commits)
  Adding docker to node scripts.
  Removing node_modules from repo
  Whoops
  Better logging for broken paypal ipns.
  Adding RSS Tracker for Windows to Goodies.
  Refactoring socketio so real-time works around username restrictions.
  Socket.IO v3 on the server too.
  Switching to Socket.IO v3.
  Haproxy splitting for socketio 3
  Socket.io 3
  Handling missing feed.
  Only nb.local.com uses ports for websockets.
  Adding GrazeTEN
  Full node install on setup.
  Updating toprc
  correct newsblur to newsblur_web in tasks
  Changing mongodb server.
  Fixing toprc
  Attempting to collect queries for 1% of requests by using DEBUG.
  Fixing mongo install.
  ...
2021-01-05 19:36:49 -05:00
Samuel Clay
f39a0405cf NullBooleanField deprecated 2020-12-09 17:20:08 -05:00
Samuel Clay
05756155b1 Merge branch 'django3.0' into docker_django3.0
* django3.0: (184 commits)
  Removing log override
  Moving logging over to the newsblur log.
  Fixing search indexer background task for new celery.
  Attempting to add gunicorn errors to console/log.
  Better handling of missing subs.
  Handling missing user sub on feed delete.
  Correct encoding for strings on systems that don't have utf-8 as default encoding.
  Writing in the real urllib3 dependency for requests.
  Upgrading requests due to urllib3 incompatibility.
  Login required should use the next parameter.
  Upgrading django oauth toolkit for django 1.11.
  Handling newsletters with multiple recipients.
  Extracting image urls sometimes fails.
  Handling ajax errors in json views.
  Adding timeouts to most outbound requests.
  Sentry SDK 0.19.4.
  Removing imperfect proxy warning for every story.
  Found four more GET/POST crosses.
  Feed unread count may need a POST.
  Namespacing settings.
  ...
2020-12-08 13:04:39 -05:00
Samuel Clay
8371c635f7 Merge branch 'master' into django2.0
* master: (27 commits)
  Removing log override
  Moving logging over to the newsblur log.
  Fixing search indexer background task for new celery.
  Attempting to add gunicorn errors to console/log.
  Better handling of missing subs.
  Handling missing user sub on feed delete.
  Correct encoding for strings on systems that don't have utf-8 as default encoding.
  Writing in the real urllib3 dependency for requests.
  Upgrading requests due to urllib3 incompatibility.
  Login required should use the next parameter.
  Upgrading django oauth toolkit for django 1.11.
  Handling newsletters with multiple recipients.
  Extracting image urls sometimes fails.
  Handling ajax errors in json views.
  Adding timeouts to most outbound requests.
  Sentry SDK 0.19.4.
  Removing imperfect proxy warning for every story.
  Found four more GET/POST crosses.
  Feed unread count may need a POST.
  Namespacing settings.
  ...
2020-12-08 09:09:25 -05:00
Samuel Clay
8bf74cbcef Handling newsletters with multiple recipients. 2020-12-06 18:23:31 -05:00
Samuel Clay
18db90df9a Extracting image urls sometimes fails. 2020-12-06 12:30:24 -05:00
Samuel Clay
77465354a3 Merge branch 'django1.11'
* django1.11: (140 commits)
  Massaging roles.
  Adding stats logging
  Separating app from web role.
  Upgrading work servers.
  request.raw_post_data -> request.body (django 1.6)
  Upgrading pgbouncer to 1.15.0.
  Fixing supervisor path issues
  Upgrading setuptools
  Fixing flask
  Removing dependency, fixing encoding issue for pages.
  Fixing DB Monitor.
  Updating User Agent for all fetchers.
  Ignoring VSCode.
  Fixing Statistics by fixing how timezones are handled.
  Fixing API page.
  Fixing both Stripe form and Login form.
  Handling missing user during auth.
  Going back on AllowAllUsers backend, as it doesn't work with django 1.5 authentication sessions.
  Going back and forth on this but without AllowAllUsersModelBackend, then is_active=False won't allow authentication.
  De-vendorizing django-paypal
  ...
2020-12-05 14:37:44 -05:00
Samuel Clay
47f1b59621 get_feed_by_url should do the same as get_feed_from_url. 2020-12-05 12:51:30 -05:00
Samuel Clay
b89e7dc429 Merge branch 'django1.11' into django2.0
* django1.11: (152 commits)
  request.raw_post_data -> request.body (django 1.6)
  Upgrading pgbouncer to 1.15.0.
  Finishing off Postgresql 13 upgrade.
  Upgrading to Postgresql 13.
  Ubuntu 20.04
  Fixing supervisor path issues
  Upgrading setuptools
  Fixing flask
  Handling over capacity for twitter.
  Max length for image_urls.
  Properly filtering newsletter feeds.
  Fixing issue with text importer on feed-less urls.
  Removing dependency, fixing encoding issue for pages.
  Fixing DB Monitor.
  Updating User Agent for all fetchers.
  Ignoring VSCode.
  Fixing DB Monitor.
  Updating User Agent for all fetchers.
  Ignoring VSCode.
  Fixing Statistics by fixing how timezones are handled.
  ...
2020-12-03 14:04:26 -05:00
Samuel Clay
f2010483b7 Merge branch 'master' into django1.11
* master:
  Max length for image_urls.
  Properly filtering newsletter feeds.
  Fixing issue with text importer on feed-less urls.
2020-11-30 18:55:57 -05:00
Samuel Clay
0f48e88681 Max length for image_urls. 2020-11-30 18:53:52 -05:00
Samuel Clay
21f40c253f Updating User Agent for all fetchers. 2020-11-30 18:03:15 -05:00
Samuel Clay
fa43eed1b8 Updating User Agent for all fetchers. 2020-11-30 15:48:59 -05:00
Samuel Clay
ea57237d48 Monkey patching get_user to remove the session authentication introduced in 1.7 and made required in 1.10. After a year we can remove this monkey patching as all users would have logged in by now, recycling their session. 2020-11-09 12:51:02 -05:00
Samuel Clay
0b92bca89d Merge branch 'django1.10' into django1.11
* django1.10: (66 commits)
  Styling new features for dark mode.
  Adding style option for grid height. XS/S/M/L/XL
  Adding applinks to .well-known/apple app json. For #1361.
  #1377 (mark as read for titles only)
  iOS: project upgrade check
  https for all links.
  Saving saved story slugs.
  Slugifying saved story tags that have no slugs.
  Adding ssl to db machines.
  Fixing subscription cancelation error messge.
  Android v10.1b7.
  #1364 In App Review
  Showing addresses in email story share.
  #1369 Notification alerts. Skip old stories
  Android v10.1b6.
  iOS v10.1.1.
  Fixing backgrounds in newsletters.
  Fixing newsltter check which inaccurately added newsletter div to all stories.
  Styling side option buttons in iOS to be cleaner.
  iOS v10.1.
  ...
2020-11-09 11:38:35 -05:00
Samuel Clay
a09250523b Merge branch 'django1.8' into django1.9
* django1.8: (66 commits)
  Styling new features for dark mode.
  Adding style option for grid height. XS/S/M/L/XL
  Adding applinks to .well-known/apple app json. For #1361.
  #1377 (mark as read for titles only)
  iOS: project upgrade check
  https for all links.
  Saving saved story slugs.
  Slugifying saved story tags that have no slugs.
  Adding ssl to db machines.
  Fixing subscription cancelation error messge.
  Android v10.1b7.
  #1364 In App Review
  Showing addresses in email story share.
  #1369 Notification alerts. Skip old stories
  Android v10.1b6.
  iOS v10.1.1.
  Fixing backgrounds in newsletters.
  Fixing newsltter check which inaccurately added newsletter div to all stories.
  Styling side option buttons in iOS to be cleaner.
  iOS v10.1.
  ...
2020-11-09 11:38:21 -05:00
Jonathan Math
6bd4ecb569 change smart_text to smart_str to get rid of deprecation warning 2020-11-06 14:46:40 +07:00
Samuel Clay
78c290e98d Saving saved story slugs. 2020-10-22 20:14:11 -04:00
Samuel Clay
b01aa852f7 Slugifying saved story tags that have no slugs. 2020-10-22 20:12:56 -04:00
Jonathan Math
89e51a2937 change celery Task import and task initialization as per the new version of celery 2020-10-05 00:45:20 +07:00
Samuel Clay
185ea01823 /reader/starred_story_hashes?include_timestamp=1 now uses the modified date of the starred story in case highlighting changes. Thanks to Renaud Preat of Actu. 2020-08-21 11:48:57 -04:00
Samuel Clay
c508c58224 Merge branch 'django1.11' into django2.0
* django1.11: (102 commits)
  Beginning fix of rss_feeds unit tests.
  Fixing unit test for profile app and signup.
  Remove highlights count when it reaches zero.
  Android v10.1b1.
  Stubbing in profile tests.
  Adding nginx.local.conf
  Adding original text and original story to API docs.
  #1282 Adding feed to root folder
  #1319 In app and external browser options
  Adding a smarter wakeup for real-time to handle cases where a laptop is re-opened but real-time is not immediately reestablished.
  #1348 (scroll indicators theme)
  #1344 (search loses focus)
  #1335 Auto theme option for OS level dark mode
  Fixing signup flow.
  #1347 Show pager with stories after using the intel trainer and refreshing
  #1272 Load HTML in comments
  New icon for Infrequent Site Stories.
  Allowing selection in private notes.
  Autoresizing private notes field.
  For #1035: Adding private notes to saved stories.
  ...
2020-08-10 17:53:48 -04:00
Samuel Clay
ac1471017d Merge branch 'django1.10' into django1.11
* django1.10: (102 commits)
  Beginning fix of rss_feeds unit tests.
  Fixing unit test for profile app and signup.
  Remove highlights count when it reaches zero.
  Android v10.1b1.
  Stubbing in profile tests.
  Adding nginx.local.conf
  Adding original text and original story to API docs.
  #1282 Adding feed to root folder
  #1319 In app and external browser options
  Adding a smarter wakeup for real-time to handle cases where a laptop is re-opened but real-time is not immediately reestablished.
  #1348 (scroll indicators theme)
  #1344 (search loses focus)
  #1335 Auto theme option for OS level dark mode
  Fixing signup flow.
  #1347 Show pager with stories after using the intel trainer and refreshing
  #1272 Load HTML in comments
  New icon for Infrequent Site Stories.
  Allowing selection in private notes.
  Autoresizing private notes field.
  For #1035: Adding private notes to saved stories.
  ...
2020-08-10 17:52:24 -04:00