Samuel Clay
855a96d633
Fixing NYPost feed by ensuring enclosures with href attrbiutes are added to stories even without content. Thanks @kleinmatic!
2022-11-23 12:21:46 -05:00
Samuel Clay
44f9f0d1b0
Fixing typo on blog.
2022-11-23 09:19:05 -05:00
Samuel Clay
1e19fdf6d4
Looking for nb.premium.archive.99 in android subscriptions for #1696 .
2022-10-25 09:48:58 -04:00
Samuel Clay
3e33a96b73
Handling missing user in slow log (from SF).
2022-09-12 08:58:05 -07:00
Samuel Clay
002a327308
Logging for Michael_Novakhov account.
2022-09-12 08:00:57 -07:00
Samuel Clay
ad26559354
Moving redis and disabling Michael_Novakhov account.
2022-09-12 07:50:12 -07:00
Samuel Clay
8363ba3ff8
RSS status can include all users.
2022-09-11 21:01:16 -07:00
Samuel Clay
9868001f22
Adjusting datetimes on slow log.
2022-09-11 23:48:34 -04:00
Samuel Clay
f93eb2520c
Adding file size and archive count to rss feed fetch status board.
2022-09-11 22:13:54 -04:00
Samuel Clay
57cf284f13
Adding slow log aggregator to uncover which users and paths are slowest.
2022-09-11 21:37:02 -04:00
Samuel Clay
e83a8e5a9b
Revert "Bumping min to fetch by 1.5X to root out 12 minute broken feed. If this succeeds, narrow down the min_to_decay and then find the feed."
...
This reverts commit fd16bb0b73
2022-09-11 08:45:29 -04:00
Samuel Clay
fd16bb0b73
Bumping min to fetch by 1.5X to root out 12 minute broken feed. If this succeeds, narrow down the min_to_decay and then find the feed.
2022-09-11 04:54:35 -04:00
Samuel Clay
92c117bb9f
Spit out feeds being fetched to spot an errant one.
2022-09-10 10:11:22 -04:00
Samuel Clay
d049dd0da3
Fixing date hack for folder rss feeds.
2022-08-19 09:02:46 -04:00
Samuel Clay
dc35fe1a3e
Fixing date hack for RSS feeds for folders to not turn off until a year after premium archive has launched.
2022-08-19 08:59:44 -04:00
Samuel Clay
cdf5a36b81
Don't delete redis keys because they take time to rebuild and subs can be counted incorrectly during that time.
2022-08-10 17:45:25 -04:00
Samuel Clay
2915a4b2ec
Fixing issue of Unread only not seeing every story. Due to the removal of UserSubscription.get_stories in 30127d7c72, we were no longer caching story hashes for single feeds.
2022-07-28 23:49:48 -04:00
Samuel Clay
2eb80a31ea
User count should include beyond yearly.
2022-07-28 23:14:42 -04:00
Samuel Clay
a22aaf349d
Deferring execution of expensive statistics.
2022-07-28 23:08:49 -04:00
Samuel Clay
ee3b9e61a5
Caching newsblur_users.py statistics.
2022-07-28 22:47:35 -04:00
Samuel Clay
78547af24b
Adding cutoff from trimming stories for memory profiling.
2022-07-27 08:19:46 -04:00
Samuel Clay
92f48a0821
Switching timestamp to better calculator that doesn't spit out -1's. Matches c469608 and fixes #1671 .
2022-07-26 09:54:41 -04:00
Samuel Clay
9e21d03255
Merging SDIFFSTORE and ZINTERSTORE into a single ZDIFFSTORE, thanks to redis 6.2.0. Requires new docker image.
2022-07-19 21:02:42 -04:00
Samuel Clay
381953ee78
Bumping highlights max from 1024 characters to 16384 characters.
2022-07-19 10:00:16 -04:00
Samuel Clay
8f1655d7d1
Revert "Merging SDIFFSTORE and ZINTERSTORE into a single ZDIFFSTORE, thanks to redis 6.2.0. Requires new docker image."
...
This reverts commit b3ec9fd748
2022-07-18 22:45:22 -04:00
Samuel Clay
b3ec9fd748
Merging SDIFFSTORE and ZINTERSTORE into a single ZDIFFSTORE, thanks to redis 6.2.0. Requires new docker image.
2022-07-18 22:34:07 -04:00
Samuel Clay
c94bb6640d
Improving redis performance by reading the config.
2022-07-18 21:10:42 -04:00
Samuel Clay
6508bc222b
Don't remove unread stories for U: unread stories list because users are actively paging through.
2022-07-13 06:41:10 -07:00
Samuel Clay
b3f2c90c8c
Trying 1 hour for story unread list.
2022-07-11 21:40:59 -04:00
Samuel Clay
c469608fc6
Fixing starred story timestamp issue. Thanks for uncovering this @sictiru.
2022-07-11 17:40:29 -04:00
Samuel Clay
642848822a
Remvoing all stories from unread stories when assembling unreads. No longer storing all stories, just using what's already in the feed.
2022-07-08 07:36:34 -04:00
Samuel Clay
5b737c7dd6
Splitting unread and read story lists so they don't clobber each other.
2022-07-08 07:25:10 -04:00
Samuel Clay
13ac3535d0
Refreshing unread counts when changing days_of_unread.
2022-07-07 21:44:57 -04:00
Samuel Clay
b8884995f4
Merge branch 'master' into unread_stories
...
* master:
Expiration unread stories key after 24 hours.
2022-07-07 16:34:00 -04:00
Samuel Clay
a37c8593f7
Expiration unread stories key after 24 hours.
2022-07-07 16:23:18 -04:00
Samuel Clay
6377a993b8
24 hour expiration on unread stories to drive home the point of it being a driven list.
2022-07-07 16:14:10 -04:00
Samuel Clay
501db5895b
Don't delete user unread stories keys in prep for using them as part of the river.
2022-07-07 15:42:57 -04:00
Samuel Clay
e4f2528b08
I don't know how manual unreads older than the unread cutoff were ever counted before removing this date check.
2022-07-07 13:44:19 -04:00
Samuel Clay
3a90f39d6e
Fixing issue with manual unreads in oldest first.
2022-07-07 13:22:39 -04:00
Samuel Clay
58be9e8415
If no subs found during unread filter, check all because we probably want it and it may have manual unreads.
2022-07-07 13:14:26 -04:00
Samuel Clay
5bb5a4ec5b
Removing read stories from unread story cache. Also removing extra +1 second to compensate for zinterstore's sum aggregation.
2022-07-07 12:52:45 -04:00
Samuel Clay
e7c0bd0979
Removing duplicate calls to UserSub.
2022-07-07 11:51:01 -04:00
Samuel Clay
bcc3dfd04c
Adding expiration to new unread stories lists.
2022-07-07 11:45:18 -04:00
Samuel Clay
937a5e66af
Checking existence of unread_story:user:feed, assembling on the fly. This breaks down when a feed has no unread stories, but we may need to figure out a zero cardinality list solution.
2022-07-07 11:13:16 -04:00
Samuel Clay
aca79771a9
New unread stories collector, now relies on already existing unread story per-feed per-user lists instead of creating them on the fly every time a river is assembled.
...
Massive performance improvement at the risk of stale unreads. Luckily this won't cause read stories to show as unread, but it may cause an issue of stale unread story lists not showing the latest unread.
The next commit will focus on additional existance checks for those unread story lists, so that at a small performance hit, no data migration is needed to enforce a recount for all feeds.
2022-07-07 10:22:05 -04:00
Samuel Clay
9314832cf1
Even bigger chunks?
2022-07-06 14:34:52 -04:00
Samuel Clay
8a6330951b
Moving to huge redis chunks.
2022-07-06 14:02:56 -04:00
Samuel Clay
c318955283
Reducing redis-story chunk size down to 10 to see if it alleviates pressure.
2022-07-05 16:46:11 -04:00
Samuel Clay
eed3de0f82
Use latest unread_cutoff to fetch stories for inverting.
2022-07-05 12:14:04 -04:00
Samuel Clay
c296948ba8
Fixing issue where marking a story as unread would turn all of the stories read if the mark read date was older than unread cutoff for the user. No need to invert stories if mark read date user-specific unread cutoff is older than story.
2022-07-05 12:10:03 -04:00
