Samuel Clay
95e7b9c84e
Add purge_tag and purge_author to Feed.
2015-12-26 12:59:44 -08:00
Samuel Clay
ad39df5d6d
Adding purge stories to fix db corruption bug.
2015-12-26 12:53:32 -08:00
Samuel Clay
10e265d0c4
Counting stories for 1% of all fetches, regardless of new stories. Should push back feeds with no new stories for a while.
2015-12-18 12:57:11 -08:00
Slavik Svyrydiuk
dc8829c991
smart_unicode instead of unicode
2015-12-02 08:08:38 +01:00
Slavik Svyrydiuk
e9504d2e42
smart_unicode instead of unicode
2015-12-02 07:46:25 +01:00
Samuel Clay
53e4998146
Merge pull request #835 from sv0/text_importer
...
Text importer
2015-11-30 16:03:50 -08:00
Samuel Clay
2540f9a7a9
No need for www in youtube urls.
2015-11-30 11:58:26 -08:00
Slavik Svyrydiuk
c1bcdad067
convert story_content to string before compression.
...
zlib.compress fails with unicode text that contains non-ASCII symbols.
2015-11-27 12:26:58 +01:00
Samuel Clay
2c6c3cc9cf
Fixing missing years in stats.
2015-09-02 14:58:58 -07:00
Samuel Clay
b6e1e03bd5
Upgrading feedparser, cleaing up logging on feed fetch.
2015-08-24 14:27:01 -07:00
Samuel Clay
e82b1974a1
Logging sub counts before expired recount.
2015-08-05 14:00:58 -07:00
Samuel Clay
a774cddcb3
Fixing broken sub counts when reids sub count expires.
2015-08-05 13:59:26 -07:00
Samuel Clay
bebccafb3c
Easing subscriber expirations for now.
2015-08-05 13:16:15 -07:00
Samuel Clay
793a560958
Adding story deletion logging.
2015-08-05 13:13:01 -07:00
Samuel Clay
c227a68be3
Adding story deletion logging.
2015-08-05 13:12:15 -07:00
Samuel Clay
f02128afdf
Adding story deletion logging.
2015-08-05 13:11:53 -07:00
Samuel Clay
fdd76f6c9f
Logging expired sub counts.
2015-07-30 13:55:45 -07:00
Samuel Clay
e285c95145
Only clearing recount scores if they exists.
2015-07-30 13:52:56 -07:00
Samuel Clay
40a1692a09
Further refining feed search. Upgrading to elasticsearch 1.7.1.
2015-07-30 11:51:21 -07:00
Samuel Clay
62b607ebfc
Correcting discrepency of active subs when reading cache or having just recounted.
2015-07-29 17:01:16 -07:00
Samuel Clay
1e56d4f3a1
Adjust feed schedule duration. Looks like the bug has to do with exception feeds not being set correctly. Better fix that next.
2015-07-29 16:34:48 -07:00
Samuel Clay
e602b52a5d
Typo.
2015-07-29 12:08:48 -07:00
Samuel Clay
89db8a5b9f
Easing off of forcing updates when not counted in redis.
2015-07-29 12:07:22 -07:00
Samuel Clay
9b89a07fe1
Adding cleanup sub counts for users on cleanup. Also enabling redis sub store for all feeds.
2015-07-29 11:44:39 -07:00
Samuel Clay
98ac28d8f8
Adding random factor in to convert feeds over to redis cache.
2015-07-28 20:58:56 -07:00
Samuel Clay
f399604369
Handling only case where a recount isn't actually necessary. Still need to add in automatic feed conversion to redis subscriber cache.
2015-07-28 20:48:51 -07:00
Samuel Clay
a81864d8cd
Fixing counts when taking into account muted feeds.
2015-07-28 20:46:30 -07:00
Samuel Clay
ce45ed8d6c
Cleaning up pipeline.
2015-07-28 19:19:58 -07:00
Samuel Clay
cf43747b20
Adding user sub counts to redis, using new redis cache to count subscribers only when feed is entirely cached. Still need automatic conversion from postgres to redis sub counts.
2015-07-28 18:46:37 -07:00
Samuel Clay
6a20478a7f
Renaming redis feed pool to redis feed update pool in prep for redis feed sub pool.
2015-07-27 18:35:25 -07:00
Samuel Clay
97b0910e67
Breaking up tasking of normal feeds and broken feeds so broken feeds are a less drain on postgresql.
2015-07-23 16:29:47 -07:00
Samuel Clay
487b79674c
Simplifying common call to update next scheduled update.
2015-07-22 15:08:15 -07:00
Samuel Clay
9cd8124419
Combing through all updates to feeds table when fetching feed.
2015-07-22 13:53:20 -07:00
Samuel Clay
da26ecdb46
1/2 Cleaning up feed fetcher's updates to feeds table. Still need to find 6 dupes.
2015-07-22 13:18:54 -07:00
Samuel Clay
48949f22a7
Deleting old shared stories by free users no longer using the service.
2015-07-09 18:03:30 -07:00
Samuel Clay
2817181f33
Forcing all 301 redirects to occur 20 times before committing, saving feeds from disappearing.
2015-06-01 17:14:04 -07:00
Samuel Clay
23fd3cccc1
Adding channel_id youtube.com video feeds.
2015-05-08 14:50:44 -07:00
Samuel Clay
d7f6fa3928
Fixing two small bugs with youtube fetch.
2015-04-29 17:49:04 -07:00
Samuel Clay
df21449e74
Adding support for entering html page for youtube, like: https://www.youtube.com/user/CookingChannel
2015-04-29 17:26:21 -07:00
Samuel Clay
d8a7a9c772
Correcting premium-preview next fetch dates in statistics.
2015-04-01 16:21:08 -07:00
Samuel Clay
46b78f70a9
Updating push logging message when skipping.
2015-02-23 13:31:42 -08:00
Samuel Clay
e351b6602b
Chopping off milliseconds in push delta.
2015-02-19 14:21:08 -08:00
Samuel Clay
ed32cb5aed
Chopping off milliseconds in push delta.
2015-02-19 14:20:08 -08:00
Samuel Clay
5fae9b6362
Logging last push date delta.
2015-02-19 13:41:09 -08:00
Samuel Clay
0da661c71b
Fixing A&L Daily bug where stories all have the same GUID and permalink. Using permalink or timestamp, whichever is unique across the feed.
2015-02-19 10:39:10 -08:00
Samuel Clay
86a73d7c91
Deleting unstarred story as part of starred story cleanup.
2015-02-11 15:55:18 -08:00
Samuel Clay
dc1b722cf3
Removing ttl from redis munin.
2015-02-11 15:11:56 -08:00
Samuel Clay
3a5f8be8d6
Adding proxy for Story view to handle https loading http sites. Needs testing, but done.
2014-11-06 09:46:33 -08:00
Samuel Clay
9e6f19d594
Merge branch 'master' into organizer
...
* master:
Adding API explainer for ps/nt/ng counts.
Adding .tm_properties to ignore.
Adding private spam catcher.
Twitter photo resync happens on every trim.
Don't mangle stories with code when updating, just use new story. Thanks Bruno!
Fixing landscape iphone 5s image width.
https domain for profile photos.
2014-10-21 14:26:37 -07:00
Samuel Clay
021fe32d9f
Don't mangle stories with code when updating, just use new story. Thanks Bruno!
2014-10-15 17:31:00 -07:00
