Commit graph

121 commits

Author SHA1 Message Date
Samuel Clay
936ca40d70 Sorting long feeds in date descending order so we cut off the oldest stories, not the newest. 2025-01-07 05:52:05 -08:00
Samuel Clay
7165aa1bf6 Black formatting and isort 2024-04-24 09:50:42 -04:00
Samuel Clay
d1dafe7606 Black formatting. 2024-04-24 09:43:56 -04:00
Samuel Clay
855a96d633 Fixing NYPost feed by ensuring enclosures with href attrbiutes are added to stories even without content. Thanks @kleinmatic! 2022-11-23 12:21:46 -05:00
Samuel Clay
d0571e2ab0 Wait to strip data uri header until it's time. 2022-03-08 13:11:08 -05:00
Samuel Clay
ca728af375 Don't double encode base64 images for widgets. 2022-03-08 12:48:57 -05:00
Samuel Clay
bae6cf7348 Fixing feedback date format. 2021-03-09 12:42:02 -05:00
Samuel Clay
eac7e71b81 Refactoring image size fetcher to be much simpler, albeit slower. 2021-03-03 13:50:22 -05:00
Samuel Clay
94e27a1f7b No need to decode 2021-03-02 18:33:00 -05:00
Samuel Clay
5adbafb046 bytes need bytes 2021-03-02 18:16:57 -05:00
Samuel Clay
cd62b41acf Convert bytes to str 2021-03-02 10:25:56 -05:00
Samuel Clay
8e088ba350 Switching to py3 html parser. 2021-03-02 09:46:59 -05:00
Samuel Clay
dffcd484d9 Handling SerialisationError 2021-02-28 20:20:41 -05:00
Samuel Clay
1fe20b91a9 Fixing py3 encoding issue when stripping comments using lxml. 2021-02-25 19:47:13 -05:00
Samuel Clay
e8fb94af23 Fixing new stories coming into dashboard being filtered out incorrectly. 2021-02-03 20:06:08 -05:00
Samuel Clay
aea4fcbe5b Merge branch 'django3.0' into django3.1
* django3.0: (27 commits)
  Removing log override
  Moving logging over to the newsblur log.
  Fixing search indexer background task for new celery.
  Attempting to add gunicorn errors to console/log.
  Better handling of missing subs.
  Handling missing user sub on feed delete.
  Correct encoding for strings on systems that don't have utf-8 as default encoding.
  Writing in the real urllib3 dependency for requests.
  Upgrading requests due to urllib3 incompatibility.
  Login required should use the next parameter.
  Upgrading django oauth toolkit for django 1.11.
  Handling newsletters with multiple recipients.
  Extracting image urls sometimes fails.
  Handling ajax errors in json views.
  Adding timeouts to most outbound requests.
  Sentry SDK 0.19.4.
  Removing imperfect proxy warning for every story.
  Found four more GET/POST crosses.
  Feed unread count may need a POST.
  Namespacing settings.
  ...
2020-12-08 09:10:51 -05:00
Samuel Clay
8371c635f7 Merge branch 'master' into django2.0
* master: (27 commits)
  Removing log override
  Moving logging over to the newsblur log.
  Fixing search indexer background task for new celery.
  Attempting to add gunicorn errors to console/log.
  Better handling of missing subs.
  Handling missing user sub on feed delete.
  Correct encoding for strings on systems that don't have utf-8 as default encoding.
  Writing in the real urllib3 dependency for requests.
  Upgrading requests due to urllib3 incompatibility.
  Login required should use the next parameter.
  Upgrading django oauth toolkit for django 1.11.
  Handling newsletters with multiple recipients.
  Extracting image urls sometimes fails.
  Handling ajax errors in json views.
  Adding timeouts to most outbound requests.
  Sentry SDK 0.19.4.
  Removing imperfect proxy warning for every story.
  Found four more GET/POST crosses.
  Feed unread count may need a POST.
  Namespacing settings.
  ...
2020-12-08 09:09:25 -05:00
Samuel Clay
30f16161ad Correct encoding for strings on systems that don't have utf-8 as default encoding. 2020-12-07 12:05:56 -05:00
Samuel Clay
4bb9d47144 Upgrading all dependencies. This probably won't work (looking at you timezones). 2020-12-03 16:50:54 -05:00
Samuel Clay
b89e7dc429 Merge branch 'django1.11' into django2.0
* django1.11: (152 commits)
  request.raw_post_data -> request.body (django 1.6)
  Upgrading pgbouncer to 1.15.0.
  Finishing off Postgresql 13 upgrade.
  Upgrading to Postgresql 13.
  Ubuntu 20.04
  Fixing supervisor path issues
  Upgrading setuptools
  Fixing flask
  Handling over capacity for twitter.
  Max length for image_urls.
  Properly filtering newsletter feeds.
  Fixing issue with text importer on feed-less urls.
  Removing dependency, fixing encoding issue for pages.
  Fixing DB Monitor.
  Updating User Agent for all fetchers.
  Ignoring VSCode.
  Fixing DB Monitor.
  Updating User Agent for all fetchers.
  Ignoring VSCode.
  Fixing Statistics by fixing how timezones are handled.
  ...
2020-12-03 14:04:26 -05:00
Samuel Clay
a860ffd2d1 Checking the 'credit' field if author not found. Works for TechRepublic, which I suspect is a one off. 2020-11-23 16:22:43 -05:00
Samuel Clay
1351aee6a8 Vendorizing new feedparser. 2020-07-01 18:38:37 -04:00
Samuel Clay
9155216e46 So many bytes<>str encoding issues. Also mongoengine wants to create new indexes all the time. 2020-06-30 14:49:12 -04:00
Jonathan Math
3a0a2c28b8 2to3 utils/*.py 2020-06-19 02:27:48 -04:00
Samuel Clay
bda71167ac Adding smart crop to image proxy thumbnails. 2019-12-25 19:08:22 -05:00
Samuel Clay
5bf7c0af18 Adding secret signature to proxied image urls and thumbnails. Ready to launch! 2019-12-25 18:36:52 -05:00
Samuel Clay
a631710d90 Don't hexify url 2019-12-25 18:18:40 -05:00
Samuel Clay
0d9f787d89 Handling a mix of image proxy options. 2019-12-25 18:15:50 -05:00
Samuel Clay
7baa187566 Handling a mix of image proxy options. 2019-12-25 18:14:55 -05:00
Samuel Clay
ed81559e4e Adding imageproxy and switching from camo. Built for #1162. 2019-12-25 18:13:36 -05:00
Samuel Clay
9ca17dd3b3 Undoing commit that changed story dates. 2019-12-25 15:59:38 -05:00
Samuel Clay
dc1567b4f4 Silencing warning. 2019-12-21 09:49:23 -05:00
Samuel Clay
74e8fd4606 Deploying camo https proxy for images. 2019-08-11 14:44:15 -07:00
Samuel Clay
e5c4726932 Creating sgined urls for camo. 2019-01-17 15:05:32 -05:00
Samuel Clay
90de811b9c Fixing truncate chars to do a proper unicode truncate. 2018-08-10 15:17:43 -04:00
Samuel Clay
33d521b6f3 Handling APNS unicode encode errors. 2018-08-09 10:25:39 -04:00
Samuel Clay
8603c8e24d Fixing APNS notifications. 2018-08-09 10:24:45 -04:00
Samuel Clay
0a49e1b073 Attempting to corral APNS push notification max sizes due to chinese unicode characters being 3 bytes. 2018-08-09 10:01:59 -04:00
Samuel Clay
54a0da95bd On stories with unmarked dates, add a random number of seconds (under a minute) to give stories their own timestamp. Makes feed timelines more deterministic once the stories are injested. 2018-07-26 10:45:09 -04:00
Samuel Clay
f54151cae4 Less restriction on blank titles. 2018-07-12 14:21:25 -04:00
Samuel Clay
ab939f72fe Adding a 'story_title_blank' field to stories to signify that the story title was originally ommitted (even though NewsBlur added one in). Thanks to @jbrayton for the suggestion. 2018-07-12 14:19:14 -04:00
Samuel Clay
e11c26d05b Adding video enclosures. 2018-07-08 16:24:33 -04:00
Samuel Clay
8e63c14267 Switching away from Get Satisfaction and to Discourse. forum.newsblur.com is the new support forum. 2018-04-17 16:50:34 -07:00
Samuel Clay
d1e9571093 Future dated stories get forced to current date. This fixed multiple issues: makring as read on oldest first only marks stories older than current date as read. Also fixes issues with publishers gaming the dates. 2017-08-04 13:31:03 -07:00
Samuel Clay
de59c9471c Don't add images in media enclosures if the URL is found in the story already. 2017-04-21 11:42:38 -07:00
Samuel Clay
9f64fcb572 Undoing mistaken date correct. 2016-11-14 08:18:31 -08:00
Samuel Clay
7dee3ae771 Upgrading to mongo 3.0. Still needs fixes for mongoengine. 2016-11-10 10:11:46 -08:00
Samuel Clay
43b90a6ab3 Handling missing rel attr. 2016-08-15 14:58:45 -07:00
Samuel Clay
2de794f6a6 Accepting media:content attachments using a 'medium' attribute instead of a 'type' attribute. Useful for images in deviantart. 2016-08-12 20:43:52 -07:00
Samuel Clay
3ef5071a61 Allowing unicode encode errors for text/plain. 2016-05-11 13:37:35 -07:00