Commit graph

1200 commits

Author SHA1 Message Date
Samuel Clay
937a1fa999 Faking user agent when rss bans NewsBlur. 2017-02-17 12:19:55 -08:00
Samuel Clay
255b30f3c1 Better recording of non-200/300 status codes on feed fetch. 2017-02-17 12:05:27 -08:00
Samuel Clay
2954ef7318 Adding new debug to feed fetcher. 2017-02-17 11:45:43 -08:00
Samuel Clay
1b35039535 Redis 3.2. 2017-01-11 20:13:04 -08:00
Samuel Clay
2aa82a4eef Updating redis 2017-01-11 19:12:00 -08:00
Samuel Clay
c5d5ea5001 Attempting to fix dashboard stories by no longer relying on full river reloads when a new story comes in but to fetch that story if it is newer (or older) than last visible dashboard story. 2016-12-13 16:29:42 -08:00
Samuel Clay
046f97558b Checking for memory errors while feed fetching. 2016-12-11 18:59:02 -08:00
Samuel Clay
d1e2e3e1d9 Handling MemoryError while feed fetching. 2016-12-11 18:29:00 -08:00
Samuel Clay
b5e0870fa8 Merge pull request #958 from da2x/patch-1
HTTP feed fetcher improvements
2016-11-25 13:02:44 -06:00
Samuel Clay
0f7c17eb9f Fixing twitter error message. 2016-11-21 17:23:53 -08:00
Samuel Clay
6698967021 Handling twitter issues. 2016-11-21 13:15:50 -08:00
Samuel Clay
d9676d64e7 Fixing tweepy error handling. 2016-11-20 13:43:49 -08:00
Samuel Clay
56b3dfa659 More aggregation changes. 2016-11-20 13:11:56 -08:00
Samuel Clay
5dc079efc2 Fixing task times for updated pymongo aggregation> 2016-11-20 13:09:30 -08:00
Samuel Clay
e9d5b39d90 Fixing aggregation on latest pymongo for munin graphs. 2016-11-20 11:14:17 -08:00
Samuel Clay
55dd39d983 Virtualenv for munin. 2016-11-20 11:05:20 -08:00
Samuel Clay
b3aba908d7 Handling twitter fetch issues. 2016-11-18 10:54:46 -08:00
Samuel Clay
055fcbc0ea Fixing notifications dialog bug. 2016-11-17 11:37:10 -08:00
Samuel Clay
fc6379c0ed Merge branch 'master' into notifications
* master:
  Virtualenv redis bgsave monitor.
2016-11-16 18:29:27 -08:00
Samuel Clay
9ffa63ec2b Max os 3 stories sent per notification group. Also limiting notifications to staff for live test. 2016-11-16 18:29:13 -08:00
Samuel Clay
8f6bfa1c55 Notifications now support hidden stories and focus stories. Web also all done. Just need to build iOS push and iOS notification chooser. 2016-11-16 17:49:43 -08:00
Samuel Clay
0284793b3a Adding in push notifications for web. Needs filtering. 2016-11-15 20:45:59 -08:00
Samuel Clay
c0299f4143 Merge branch 'master' into notifications
* master:
  Switching to indexed story hashes instead of story feeds for starred stories.
  When adding a feed to a folder, check against the lower case folder name to correct for dupes.
  Newsletter params to fix the Skimm
  Undoing mistaken date correct.
2016-11-14 16:23:56 -08:00
Samuel Clay
8b2812afc9 Virtualenv redis bgsave monitor. 2016-11-14 12:59:06 -08:00
Samuel Clay
bb089f4aeb When adding a feed to a folder, check against the lower case folder name to correct for dupes. 2016-11-14 12:31:41 -08:00
Samuel Clay
637f4d6955 Stubbing in notifications dialog. 2016-11-14 11:12:13 -08:00
Samuel Clay
9f64fcb572 Undoing mistaken date correct. 2016-11-14 08:18:31 -08:00
Samuel Clay
982d1c9589 Handling twitter user not found error. 2016-11-11 15:12:13 -08:00
Samuel Clay
7dee3ae771 Upgrading to mongo 3.0. Still needs fixes for mongoengine. 2016-11-10 10:11:46 -08:00
Samuel Clay
fde0c6caef Higher random weight for quick fetch. 2016-11-09 09:00:30 -08:00
Samuel Clay
1454da0b43 Fixing tlnb 2016-11-02 22:07:10 -07:00
Daniel Aleksandersen
2f7bdd1672 HTTP feed fetcher improvements
* Send an Accept header asking for feed formats (instead of the current */*)

* Don’t send A-IM without an If-None-Match or If-Modified-Since header.

* Fixed Train-Case in one HTTP header.
2016-10-28 01:25:08 +02:00
Samuel Clay
a616ad50a5 Adding support for tweets that start with '@' to be categorized as a reply. 2016-09-22 15:54:39 -07:00
Julien Rottenberg
bf2e2258b8 Simplify onboarding
missed some from "Upgrading tweepy, hoping it fixes auth issues w/ twitter."
2016-09-15 20:24:34 -07:00
Samuel Clay
96afee3732 Attempting to fix zgrep 2016-08-15 16:27:17 -07:00
Samuel Clay
43b90a6ab3 Handling missing rel attr. 2016-08-15 14:58:45 -07:00
Samuel Clay
2de794f6a6 Accepting media:content attachments using a 'medium' attribute instead of a 'type' attribute. Useful for images in deviantart. 2016-08-12 20:43:52 -07:00
Samuel Clay
840bb43fc2 Miscellaneous old utils. 2016-08-04 11:00:33 -07:00
Samuel Clay
7ecf37b8c3 Temporarily locked = suspended on twitter. 2016-06-28 17:11:22 -07:00
Samuel Clay
a7ceebcb0f Adding exception handling for twitter feeds. 2016-06-28 16:09:14 -07:00
Samuel Clay
76b48a5525 Fixing twitter status permalink. 2016-06-14 21:36:30 -07:00
Samuel Clay
0a86563321 Checking for links in tweet, adding link tag. 2016-05-26 15:52:41 -07:00
Samuel Clay
453f9d67eb Handling insta-fetching twitter feeds. 2016-05-26 15:36:20 -07:00
Samuel Clay
bab64e0447 Disconnecting twitter connection if user has broken token. 2016-05-26 15:33:02 -07:00
Samuel Clay
c7420dc888 Retrying twitter fetch with any access token that works. 2016-05-26 15:21:12 -07:00
Samuel Clay
0b33889881 Adding retweet tag, changing likes and retweets to liked and retweeted. 2016-05-26 15:12:22 -07:00
Samuel Clay
2895fc398d Adding photo category to tweets. 2016-05-26 15:09:24 -07:00
Samuel Clay
aa397194a1 Allowing twitter URL to not have https/http in it. 2016-05-26 15:06:52 -07:00
Samuel Clay
5db1138cdf Tweets now have enclosed images. 2016-05-26 15:05:04 -07:00
Samuel Clay
7ad8dd89db Better match for twitter.com/username 2016-05-26 14:41:44 -07:00