Commit graph

1505 commits

Author SHA1 Message Date
Samuel Clay
461c1c4b65 Changing feed log format to include id at the beginning. Also normalizing all feed titles in logs for better searchability. 2017-03-31 19:52:24 -07:00
Samuel Clay
937a67f67c Only using active twitter accounts. 2017-03-24 13:04:38 -07:00
Samuel Clay
26f9456c2e Monkey patching SSL for new python, since hostnames don't match with S3. 2017-03-23 14:07:59 -07:00
Samuel Clay
4f8a1c43b3 Fixing attribution in twitter RTs. 2017-03-15 19:40:43 -07:00
Samuel Clay
a6fdf45d33 Adding support for native RTs in Twitter. 2017-03-15 19:26:39 -07:00
Samuel Clay
937a1fa999 Faking user agent when rss bans NewsBlur. 2017-02-17 12:19:55 -08:00
Samuel Clay
255b30f3c1 Better recording of non-200/300 status codes on feed fetch. 2017-02-17 12:05:27 -08:00
Samuel Clay
2954ef7318 Adding new debug to feed fetcher. 2017-02-17 11:45:43 -08:00
Samuel Clay
1b35039535 Redis 3.2. 2017-01-11 20:13:04 -08:00
Samuel Clay
2aa82a4eef Updating redis 2017-01-11 19:12:00 -08:00
Samuel Clay
c5d5ea5001 Attempting to fix dashboard stories by no longer relying on full river reloads when a new story comes in but to fetch that story if it is newer (or older) than last visible dashboard story. 2016-12-13 16:29:42 -08:00
Samuel Clay
046f97558b Checking for memory errors while feed fetching. 2016-12-11 18:59:02 -08:00
Samuel Clay
d1e2e3e1d9 Handling MemoryError while feed fetching. 2016-12-11 18:29:00 -08:00
Samuel Clay
b5e0870fa8 Merge pull request #958 from da2x/patch-1
HTTP feed fetcher improvements
2016-11-25 13:02:44 -06:00
Samuel Clay
0f7c17eb9f Fixing twitter error message. 2016-11-21 17:23:53 -08:00
Samuel Clay
6698967021 Handling twitter issues. 2016-11-21 13:15:50 -08:00
Samuel Clay
d9676d64e7 Fixing tweepy error handling. 2016-11-20 13:43:49 -08:00
Samuel Clay
56b3dfa659 More aggregation changes. 2016-11-20 13:11:56 -08:00
Samuel Clay
5dc079efc2 Fixing task times for updated pymongo aggregation> 2016-11-20 13:09:30 -08:00
Samuel Clay
e9d5b39d90 Fixing aggregation on latest pymongo for munin graphs. 2016-11-20 11:14:17 -08:00
Samuel Clay
55dd39d983 Virtualenv for munin. 2016-11-20 11:05:20 -08:00
Samuel Clay
b3aba908d7 Handling twitter fetch issues. 2016-11-18 10:54:46 -08:00
Samuel Clay
055fcbc0ea Fixing notifications dialog bug. 2016-11-17 11:37:10 -08:00
Samuel Clay
fc6379c0ed Merge branch 'master' into notifications
* master:
  Virtualenv redis bgsave monitor.
2016-11-16 18:29:27 -08:00
Samuel Clay
9ffa63ec2b Max os 3 stories sent per notification group. Also limiting notifications to staff for live test. 2016-11-16 18:29:13 -08:00
Samuel Clay
8f6bfa1c55 Notifications now support hidden stories and focus stories. Web also all done. Just need to build iOS push and iOS notification chooser. 2016-11-16 17:49:43 -08:00
Samuel Clay
0284793b3a Adding in push notifications for web. Needs filtering. 2016-11-15 20:45:59 -08:00
Samuel Clay
c0299f4143 Merge branch 'master' into notifications
* master:
  Switching to indexed story hashes instead of story feeds for starred stories.
  When adding a feed to a folder, check against the lower case folder name to correct for dupes.
  Newsletter params to fix the Skimm
  Undoing mistaken date correct.
2016-11-14 16:23:56 -08:00
Samuel Clay
8b2812afc9 Virtualenv redis bgsave monitor. 2016-11-14 12:59:06 -08:00
Samuel Clay
bb089f4aeb When adding a feed to a folder, check against the lower case folder name to correct for dupes. 2016-11-14 12:31:41 -08:00
Samuel Clay
637f4d6955 Stubbing in notifications dialog. 2016-11-14 11:12:13 -08:00
Samuel Clay
9f64fcb572 Undoing mistaken date correct. 2016-11-14 08:18:31 -08:00
Samuel Clay
982d1c9589 Handling twitter user not found error. 2016-11-11 15:12:13 -08:00
Samuel Clay
7dee3ae771 Upgrading to mongo 3.0. Still needs fixes for mongoengine. 2016-11-10 10:11:46 -08:00
Samuel Clay
fde0c6caef Higher random weight for quick fetch. 2016-11-09 09:00:30 -08:00
Samuel Clay
1454da0b43 Fixing tlnb 2016-11-02 22:07:10 -07:00
Daniel Aleksandersen
2f7bdd1672 HTTP feed fetcher improvements
* Send an Accept header asking for feed formats (instead of the current */*)

* Don’t send A-IM without an If-None-Match or If-Modified-Since header.

* Fixed Train-Case in one HTTP header.
2016-10-28 01:25:08 +02:00
Samuel Clay
a616ad50a5 Adding support for tweets that start with '@' to be categorized as a reply. 2016-09-22 15:54:39 -07:00
Julien Rottenberg
bf2e2258b8 Simplify onboarding
missed some from "Upgrading tweepy, hoping it fixes auth issues w/ twitter."
2016-09-15 20:24:34 -07:00
Samuel Clay
96afee3732 Attempting to fix zgrep 2016-08-15 16:27:17 -07:00
Samuel Clay
43b90a6ab3 Handling missing rel attr. 2016-08-15 14:58:45 -07:00
Samuel Clay
2de794f6a6 Accepting media:content attachments using a 'medium' attribute instead of a 'type' attribute. Useful for images in deviantart. 2016-08-12 20:43:52 -07:00
Samuel Clay
840bb43fc2 Miscellaneous old utils. 2016-08-04 11:00:33 -07:00
Samuel Clay
7ecf37b8c3 Temporarily locked = suspended on twitter. 2016-06-28 17:11:22 -07:00
Samuel Clay
a7ceebcb0f Adding exception handling for twitter feeds. 2016-06-28 16:09:14 -07:00
Samuel Clay
76b48a5525 Fixing twitter status permalink. 2016-06-14 21:36:30 -07:00
Samuel Clay
0a86563321 Checking for links in tweet, adding link tag. 2016-05-26 15:52:41 -07:00
Samuel Clay
453f9d67eb Handling insta-fetching twitter feeds. 2016-05-26 15:36:20 -07:00
Samuel Clay
bab64e0447 Disconnecting twitter connection if user has broken token. 2016-05-26 15:33:02 -07:00
Samuel Clay
c7420dc888 Retrying twitter fetch with any access token that works. 2016-05-26 15:21:12 -07:00