Commit graph

1933 commits

Author SHA1 Message Date
Samuel Clay
ab939f72fe Adding a 'story_title_blank' field to stories to signify that the story title was originally ommitted (even though NewsBlur added one in). Thanks to @jbrayton for the suggestion. 2018-07-12 14:19:14 -04:00
Samuel Clay
9223000139 Adding timeout to feed finder on initial feed add. 2018-06-28 13:38:56 -04:00
Samuel Clay
471d5434c6 Facebook pages feed gets the correct favicon. 2018-03-26 19:05:22 -07:00
Samuel Clay
da4741a3fa Adding videos to facebook posts. Seems to work pretty well. 2018-03-26 17:31:12 -07:00
Samuel Clay
3d31ed2507 Handling connection error when finding feed. 2018-01-18 16:15:17 -08:00
Samuel Clay
94114595a6 Sanity check on extracting image urls in Text view. 2018-01-18 08:06:32 -08:00
Samuel Clay
8421f667d7 Fixing broken image handling from Mercury Reader that was causing image urls with a srcset to be concat'd together. This one's for @yesthatjwz. 2018-01-17 16:51:06 -08:00
Samuel Clay
b99ccd7045 YouTube feeds now have an auto extracted title. Thank you Bruno! 2017-12-18 21:48:26 -08:00
Samuel Clay
ef96f59c2c Only count months that have stories for the average count. 2017-12-15 17:12:14 -08:00
Samuel Clay
f2ab8145c5 Adding options to control infrequency of infrequent site stories feed. 2017-11-05 14:01:25 -08:00
Samuel Clay
6a27023e12 Merge branch 'master' into infrequent
* master:
  Handling no original doc returning in text importer.
  Assets for icons.
  No longer finding the largest image in a story if the text view already successfully found one. Also using Mercury's builtin image finder.
  Fixing warnings.
  Removing ESPN from original pages.
2017-11-03 14:46:02 -07:00
Samuel Clay
27688c7593 Handling no original doc returning in text importer. 2017-11-03 13:48:44 -07:00
Samuel Clay
2d05dc9222 No longer finding the largest image in a story if the text view already successfully found one. Also using Mercury's builtin image finder. 2017-11-03 13:47:17 -07:00
Samuel Clay
8adaaec03a Removing ESPN from original pages. 2017-11-03 13:47:17 -07:00
Samuel Clay
b7574a1ff7 No longer finding the largest image in a story if the text view already successfully found one. Also using Mercury's builtin image finder. 2017-11-02 22:09:37 -07:00
Samuel Clay
0d6d4de92e Removing ESPN from original pages. 2017-11-02 16:39:36 -07:00
Samuel Clay
00430b302b Merge branch 'master' into infrequent
* master:
  Adding mark read button for ipad when story titles on bottom.
  Fixing image sizes on both 10" and 12" iPad Pro.
  Bottom size margin.
  Safe margins on story page view.
  Adding notifier back to detail pages.
  Got it, notifier now animates smoothly. Now need to turn it back on for other views.
  Animation progress bar, but there's a jump that needs fixing.
  Handling mercury errors.
  Aiutolayout for notifier. Needs animations fixed.
  Fixing auto-scroll to last read position in stories.
2017-11-02 11:53:12 -07:00
Samuel Clay
f242a49d24 Handling mercury errors. 2017-10-30 11:47:18 -07:00
Samuel Clay
368f0e15d7 Merge branch 'master' into infrequent
* master:
  Handling issue when story has no original content.
  Switching to Mercury text parser, which is an upgraded Readability. Using old readability as backup.
  Baseline autolayout changes for story page control for iPhone X, but a lot needs to happen to get sizing correct for story traversal buttons and story web views.
  Fixing feed detail view for iPhone X.
  Updating twitter avatar photo on homepage.
2017-10-24 15:48:54 -07:00
Samuel Clay
2827b896b5 Handling issue when story has no original content. 2017-10-24 15:33:27 -07:00
Samuel Clay
ec7e032c28 Switching to Mercury text parser, which is an upgraded Readability. Using old readability as backup. 2017-10-24 15:28:36 -07:00
Samuel Clay
f543e408e9 Attempting new Infrequent Site Stories river. 2017-10-16 14:22:37 -07:00
Samuel Clay
c476d89e1f Removing breaking text importer UTF-8 encoding. 2017-10-15 17:15:56 -07:00
Samuel Clay
ef51152bcd Updating readability class names to look for. 2017-09-29 10:50:13 -07:00
Diego Elio Pettenò
4ff8ed7798 favicon fetcher: provide an Accept header on requests.
This makes the fetcher work with the default ModSecurity ruleset, and would also avoid providing unsupported favicon formats (e.g. webp), by explictly preferring PNG and
x-icon formats.
2017-08-16 18:21:14 +00:00
Samuel Clay
920e4be4bd Handling case where every story is the same time. 2017-06-28 17:19:54 -07:00
Samuel Clay
c1834703d9 Adding support for JSON Feeds. 2017-05-22 16:46:56 -07:00
Samuel Clay
6bc9a55bfc Rewriting twitter fetching. Now fetching truncated text, embedding quoted tweets, and fixing URLs so that they show the display url (and not t.co) and link to the expanded url. 2017-05-06 19:38:36 -07:00
Samuel Clay
262d67abf9 Whoops, zrevrange is high to low, which means newest to oldest. 2017-05-01 12:08:56 -07:00
Samuel Clay
44de405195 Only fetch as many guidas as there are stories in the feed. No need to get everything assuming the feed has the latest N stories. 2017-05-01 12:06:56 -07:00
Samuel Clay
08c897e5a1 This is it, the big kahuna of fixes. This corrects for messed up guids that are causing lots of read stories to become unread. 2017-05-01 11:39:24 -07:00
Samuel Clay
b753cf6819 This won't fix it but in the event of missing guids on feed fetch, check for stories within the mark read period. 2017-05-01 10:51:04 -07:00
Samuel Clay
c9326a6f02 Perhaps this is the way to find the missing story hashes. Shouldn't cause an issue, but logic for dates may be backwards. 2017-05-01 09:27:31 -07:00
Samuel Clay
ffeeb170e0 Finally have a test case for the Google Blog duping. 2017-04-30 18:47:10 -07:00
Samuel Clay
d84e2af636 New experimental data collector for debugging feeds over time. 2017-04-12 19:13:33 -07:00
Samuel Clay
78568f0914 Handling broken URL for icons. 2017-03-31 20:30:43 -07:00
Samuel Clay
461c1c4b65 Changing feed log format to include id at the beginning. Also normalizing all feed titles in logs for better searchability. 2017-03-31 19:52:24 -07:00
Samuel Clay
440d2c5841 Fixing broken S3 names. 2017-03-28 21:17:11 -07:00
Samuel Clay
9f15958c0e Only connect to S3 when needed. 2017-03-28 10:43:31 -07:00
Samuel Clay
930abb9adb Merge branch 'master' into saved_searches
* master: (22 commits)
  Turn original text caching back on.
  Extracting images from original text's noscript.
  Fetcing the original text now extracts the image url for others.
  Monkey patching SSL for new python, since hostnames don't match with S3.
  Converting videos in email notifications to images.
  Upping quota to 100 shared stories a day.
  Bumping premium shares to 50 per day.
  Android v5.1.0
  Only 20 stories may be shared per day for premiums, 3 for free users. Also hits IFTTT sharing.
  Improving messaging on emails that have OPML backups. Thanks to @frenetic for bringing this up. Closes #1003
  Fixing up postgresql backup.
  Hiding cookie lost message.
  Parallel pgbouncer kill.
  Fixing attribution in twitter RTs.
  Downgrading to elasticsearch 2.4.4, since pyes isn't ready for ES 5.
  Adding support for native RTs in Twitter.
  Automatically disbaling transparent huge pages (THP) on mongo and redis. Also upgrading to elasticsearch 5.2.2, although its untested.
  Add two buttons to get the app
  Fixing broken getsatisfaction community feedback.
  Goodbye Turn Touch campaign.
  ...
2017-03-23 16:54:37 -07:00
Samuel Clay
fdfcc8e798 Turn original text caching back on. 2017-03-23 16:29:15 -07:00
Samuel Clay
82cdae1e4d Extracting images from original text's noscript. 2017-03-23 16:28:47 -07:00
Samuel Clay
2c195cde2a Fetcing the original text now extracts the image url for others. 2017-03-23 16:06:06 -07:00
Samuel Clay
4f8a1c43b3 Fixing attribution in twitter RTs. 2017-03-15 19:40:43 -07:00
Samuel Clay
e1bd42612f Adding unique index. 2017-03-07 12:28:21 -08:00
Samuel Clay
2c0bf76e20 Saved search for feeds now works. Need to fix scroll to selected feed (and keyboard shortcuts). Also need to hook up other search types. 2017-03-06 19:55:18 -08:00
Samuel Clay
3435bac504 Showing saved searches. Titles need work. And they don't show the feed yet. 2017-03-03 18:12:27 -05:00
Samuel Clay
00695b04af Merge branch 'master' into saved_searches
* master: (771 commits)
  Fixing horrible session loss bug. Turns out the django-redis-sessions library changed timeout behavior, from silently retrying to silently failing. Adding necessary SESSION_* vars.
  Better handling of booleans on /reader/feeds.
  Fixing redis bootstrap
  Add Firefox to story browser list
  Dashboard should not show old stories.
  Faking user agent when rss bans NewsBlur.
  Better recording of non-200/300 status codes on feed fetch.
  Adding new debug to feed fetcher.
  Embedding kickstarter module for TT on dashboard.
  notifications alpha 2
  Updating TT launch email.
  Linking image on TT email.
  No sudo on virtualenv.
  Adding missing feeds for users as part of cleanup.
  Fixing spacing issues in Turn Touch email.
  Updating Turn Touch email.
  Upgrading Readability and forcing images to remain. THis should add a bunch of images back to the Text view.
  Andorid v5.0.
  Fixing bug that would clear river cache while reading the river, caused by the dashboard river reloading when it wasn't visible.
  Revert "Moving from Whitney to Whitney ScreenSmart."
  ...
2017-02-24 17:05:51 -08:00
Samuel Clay
0b63c42be0 Correct feedfinder. 2017-02-24 16:46:58 -08:00
Samuel Clay
46396cd488 Only branch feed if feed is OK on page and feed. 2017-02-24 16:46:58 -08:00