Commit graph

717 commits

Author SHA1 Message Date
Samuel Clay
f5eac30a5a Fixing bug where dupe feed has no link. 2013-05-15 17:21:10 -07:00
Samuel Clay
88d474d62b Temporarily commenting out story hash switching on read stories because of massive redis locking. 2013-05-15 17:13:06 -07:00
Samuel Clay
be3bc8d16b Merge branch 'master' into muserstory_remove
* master:
  Typo on duplicate feed saving truncation.
  Adding duplicate feed address+link truncation.
  Adding duplicate feed address+link truncation.
2013-05-15 16:06:26 -07:00
Samuel Clay
44b4559d0b Typo on duplicate feed saving truncation. 2013-05-15 15:43:07 -07:00
Samuel Clay
3a0cded0d9 Adding duplicate feed address+link truncation. 2013-05-15 15:41:39 -07:00
Samuel Clay
f6f053b285 Adding duplicate feed address+link truncation. 2013-05-15 15:40:38 -07:00
Samuel Clay
7068119208 Removing story blurblog permalink, making it called explicitly. 2013-05-14 16:36:03 -07:00
Samuel Clay
9a5a4f9562 Merge branch 'master' into muserstory_remove
# By Samuel Clay (1) and ojiikun (1)
* master:
  Correcting for feedburner stupidity.
  Auto-sync the feed/folder list if it has been more than 10 minutes since the last refresh.
2013-05-13 18:56:15 -07:00
Samuel Clay
f7f9565756 Correcting for feedburner stupidity. 2013-05-13 15:41:58 -07:00
Samuel Clay
b98d129d1b Removing MUserStory entirely. 2013-05-10 16:11:30 -07:00
Samuel Clay
777d5a5e9c Removing Redis autocomplete. Upgrading elasticsearch. 2013-05-08 12:23:10 -07:00
Samuel Clay
365e0c2813 Using haystack and elasticsearch for a third autocomplete feature. 2013-05-08 12:23:09 -07:00
Samuel Clay
d91cc66178 Preventing infinite loop on merged dupe feeds. 2013-05-07 16:23:30 -07:00
Samuel Clay
5ceb0c45bf Migrating comment feeds to non-comment feeds. 2013-05-07 15:37:36 -07:00
Samuel Clay
477d928d70 Disallowing branched feeds from feed search. 2013-05-07 11:37:01 -07:00
Samuel Clay
e27b7f5688 Removing sparse constraint on MStory's story_hash index. Goodbye old stories that didn't make the cut. 2013-05-04 19:03:02 -07:00
Samuel Clay
009010f879 Removing unused redis story objectid db. 2013-05-02 12:27:37 -07:00
Samuel Clay
49245ce9e3 Another bug found by @kyleniemeyer's story. a - b was supposed to be b - a. 2013-05-01 14:33:12 -07:00
Samuel Clay
38d01df4ab Fixing Global shared stories by checking for missing story hashes properly. 2013-05-01 14:33:12 -07:00
Samuel Clay
ed1e71e407 Using index in msharedstory search. 2013-05-01 14:33:12 -07:00
Samuel Clay
c01d8e7ba9 Using new story_hashes instead of story object ids. 2013-05-01 14:33:12 -07:00
Samuel Clay
8d516fc493 Adding story_hash to starred stories, user stories, and shared stories. 2013-04-29 16:07:08 -07:00
Samuel Clay
fbe5f4422e If a feed has no stories, don't choke on latest story. 2013-04-23 21:22:49 -07:00
Samuel Clay
1584208677 Adding metric for last story date, used to calculate next scheduled update. 2013-04-23 17:04:21 -07:00
Samuel Clay
9f8009632a Correcting for premiums being counted twice. 2013-04-23 16:03:45 -07:00
Samuel Clay
0b6a5bd7ce Use average stories per month in calc. 2013-04-23 15:58:02 -07:00
Samuel Clay
630029cfcb Complete rewrite of feed update schedule. 2013-04-23 15:44:31 -07:00
Samuel Clay
2b991fd031 Forcing earlier fetch if stats calc is less than scheduled fetch. 2013-04-23 13:46:07 -07:00
Samuel Clay
e24a30bbed Fixing small bug for feeds that have no feed link. 2013-04-19 11:09:31 -07:00
Samuel Clay
7468a8e305 Upping feed fetching. 2013-04-18 17:19:27 -07:00
Samuel Clay
5db1cebecf Removing random interval in feed fetches. 2013-04-18 16:56:54 -07:00
Samuel Clay
2d1a81694a No longer random intervals for dead feeds. 2013-04-18 16:47:55 -07:00
Samuel Clay
409b829d24 Fixing autocomplete errors when feed address is invalid. 2013-04-17 09:04:45 -07:00
Samuel Clay
cafab8c86b Boosting the hell out of feed fetches. 2013-04-16 11:49:37 -07:00
Samuel Clay
45e137b22c Upping feed fetches. 2013-04-15 19:26:41 -07:00
Samuel Clay
2ccd7b6cfe Fixing race condition between primary adn secondaries when creating feed fetch histories. 2013-04-15 15:09:04 -07:00
Samuel Clay
abbc05660f Reversing order for fetch histories. 2013-04-15 14:38:43 -07:00
Samuel Clay
6011e4f2da Moving from feed/page/push histories to capped fetch history. 2013-04-15 14:30:31 -07:00
Samuel Clay
1b482a78c6 Beginning big redis migration from story objectid to story guid hash. 2013-04-13 22:31:05 -07:00
Samuel Clay
13616665c1 Dumb, dumb bug causing division by zero errors because the autocomplete boost was 0. 2013-04-13 19:44:43 -07:00
Samuel Clay
3eaad16adf Upping feed fetches. 2013-04-12 17:41:18 -07:00
Samuel Clay
bd2ad514ea Further pushing up feed fetches. Let the good times roll. 2013-04-11 16:57:10 -07:00
Samuel Clay
4ee659a1a4 Upping feed fetching (4->3, still need 3->2 and 2->1) before I crank it. 2013-04-11 13:41:44 -07:00
Samuel Clay
ccb6d38fa3 Adding support for automatically converting google alerts. 2013-04-10 19:14:19 -07:00
Samuel Clay
20c089175b Rejiggering autocomplete to only include > 10 subs. 2013-04-10 11:23:03 -07:00
Samuel Clay
47c2257ef7 Using regular expressions for comment stripping, unless the feed has an error count, in which case switch to lxml. 2013-04-08 16:14:48 -07:00
Samuel Clay
33e605875e Switch to lxml for comment stripping. 2013-04-08 15:38:23 -07:00
Samuel Clay
9565c27784 Moving to 1 feed per task queue (from 12). 2013-04-08 14:39:00 -07:00
Samuel Clay
83c8443ed0 Prioritizing autocomplete matches on title before address. 2013-04-08 13:12:11 -07:00
Samuel Clay
c25c4478e6 Adding new autocomplete. Let's hope this doesn't destroy performance in Redis. 2013-04-08 12:54:02 -07:00