Samuel Clay
7165aa1bf6
Black formatting and isort
2024-04-24 09:50:42 -04:00
Samuel Clay
d1dafe7606
Black formatting.
2024-04-24 09:43:56 -04:00
Samuel Clay
2915a4b2ec
Fixing issue of Unread only not seeing every story. Due to the removal of UserSubscription.get_stories in 30127d7c72, we were no longer caching story hashes for single feeds.
2022-07-28 23:49:48 -04:00
Samuel Clay
9e21d03255
Merging SDIFFSTORE and ZINTERSTORE into a single ZDIFFSTORE, thanks to redis 6.2.0. Requires new docker image.
2022-07-19 21:02:42 -04:00
Samuel Clay
8f1655d7d1
Revert "Merging SDIFFSTORE and ZINTERSTORE into a single ZDIFFSTORE, thanks to redis 6.2.0. Requires new docker image."
...
This reverts commit b3ec9fd748
2022-07-18 22:45:22 -04:00
Samuel Clay
b3ec9fd748
Merging SDIFFSTORE and ZINTERSTORE into a single ZDIFFSTORE, thanks to redis 6.2.0. Requires new docker image.
2022-07-18 22:34:07 -04:00
Samuel Clay
6508bc222b
Don't remove unread stories for U: unread stories list because users are actively paging through.
2022-07-13 06:41:10 -07:00
Samuel Clay
b3f2c90c8c
Trying 1 hour for story unread list.
2022-07-11 21:40:59 -04:00
Samuel Clay
642848822a
Remvoing all stories from unread stories when assembling unreads. No longer storing all stories, just using what's already in the feed.
2022-07-08 07:36:34 -04:00
Samuel Clay
5b737c7dd6
Splitting unread and read story lists so they don't clobber each other.
2022-07-08 07:25:10 -04:00
Samuel Clay
b8884995f4
Merge branch 'master' into unread_stories
...
* master:
Expiration unread stories key after 24 hours.
2022-07-07 16:34:00 -04:00
Samuel Clay
a37c8593f7
Expiration unread stories key after 24 hours.
2022-07-07 16:23:18 -04:00
Samuel Clay
6377a993b8
24 hour expiration on unread stories to drive home the point of it being a driven list.
2022-07-07 16:14:10 -04:00
Samuel Clay
501db5895b
Don't delete user unread stories keys in prep for using them as part of the river.
2022-07-07 15:42:57 -04:00
Samuel Clay
e4f2528b08
I don't know how manual unreads older than the unread cutoff were ever counted before removing this date check.
2022-07-07 13:44:19 -04:00
Samuel Clay
3a90f39d6e
Fixing issue with manual unreads in oldest first.
2022-07-07 13:22:39 -04:00
Samuel Clay
58be9e8415
If no subs found during unread filter, check all because we probably want it and it may have manual unreads.
2022-07-07 13:14:26 -04:00
Samuel Clay
5bb5a4ec5b
Removing read stories from unread story cache. Also removing extra +1 second to compensate for zinterstore's sum aggregation.
2022-07-07 12:52:45 -04:00
Samuel Clay
e7c0bd0979
Removing duplicate calls to UserSub.
2022-07-07 11:51:01 -04:00
Samuel Clay
bcc3dfd04c
Adding expiration to new unread stories lists.
2022-07-07 11:45:18 -04:00
Samuel Clay
937a5e66af
Checking existence of unread_story:user:feed, assembling on the fly. This breaks down when a feed has no unread stories, but we may need to figure out a zero cardinality list solution.
2022-07-07 11:13:16 -04:00
Samuel Clay
aca79771a9
New unread stories collector, now relies on already existing unread story per-feed per-user lists instead of creating them on the fly every time a river is assembled.
...
Massive performance improvement at the risk of stale unreads. Luckily this won't cause read stories to show as unread, but it may cause an issue of stale unread story lists not showing the latest unread.
The next commit will focus on additional existance checks for those unread story lists, so that at a small performance hit, no data migration is needed to enforce a recount for all feeds.
2022-07-07 10:22:05 -04:00
Samuel Clay
9314832cf1
Even bigger chunks?
2022-07-06 14:34:52 -04:00
Samuel Clay
8a6330951b
Moving to huge redis chunks.
2022-07-06 14:02:56 -04:00
Samuel Clay
c318955283
Reducing redis-story chunk size down to 10 to see if it alleviates pressure.
2022-07-05 16:46:11 -04:00
Samuel Clay
eed3de0f82
Use latest unread_cutoff to fetch stories for inverting.
2022-07-05 12:14:04 -04:00
Samuel Clay
c296948ba8
Fixing issue where marking a story as unread would turn all of the stories read if the mark read date was older than unread cutoff for the user. No need to invert stories if mark read date user-specific unread cutoff is older than story.
2022-07-05 12:10:03 -04:00
Samuel Clay
e32e581562
Cutting feed chunks in half
2022-07-05 10:25:17 -04:00
Samuel Clay
b1c8cf64cd
Changing unread cutoff date to use unread cutoff, not days of story hashes.
2022-07-02 11:36:58 -04:00
Samuel Clay
0b77ec3e86
Refactoring story_hashes to be more efficient with the zunionstore's.
2022-06-27 16:16:02 -04:00
Samuel Clay
4081b7b278
Upping river chunk size to decrease latency.
2022-06-27 12:33:36 -04:00
Samuel Clay
7991c9da26
Wrong feed ids when removing temporary unread sets.
2022-06-27 12:32:16 -04:00
Samuel Clay
6e71f847a2
Clearing out unread stories after being used.
2022-06-27 12:28:19 -04:00
Samuel Clay
108d88b785
Attempting to reduce redis db size by deleting unread stories list when no longer needed.
2022-06-27 12:24:49 -04:00
Samuel Clay
4b0a599a03
Removing stories outside a mark read date cutoff. Also slightly optimizing a check for old unread stories by checking existance instead of zrevrange -1 -1.
2022-06-24 18:12:00 -04:00
Samuel Clay
71cd36f215
Removing unecessary pulling of story hashes from pipeline when we can just index into the stored set.
2022-06-24 14:29:58 -04:00
Samuel Clay
f459944b88
Accidentally running full aggregation on chunks and not end of pipeline.
2022-06-23 17:50:47 -04:00
Samuel Clay
c38fcc59cf
Refactoring all stories redis calls, now with 99% less traffic.
2022-06-23 17:32:55 -04:00
Samuel Clay
23260a4b72
Removing dump and restore by querying on zinterstore of river hashes.
2022-06-23 17:13:06 -04:00
Samuel Clay
30127d7c72
Refactoring out get_stories and its duplicate implementation and using story_hashes.
2022-06-23 15:50:56 -04:00
Samuel Clay
6792762e27
Refactoring story_hashes grouP-by_feed in prep for removing .get_stories().
2022-06-23 15:35:20 -04:00
Samuel Clay
8aff53c2ba
Re-enabling archive redis calls.
2022-06-23 14:38:13 -04:00
Samuel Clay
81f51f1723
Handling soft time limits while fetching archives.
2022-06-22 17:40:16 -04:00
Samuel Clay
c20eed028c
Attempting to impose longer time limits for archive fetch.
2022-06-22 15:44:39 -04:00
Samuel Clay
9759bda53a
Overriding time limit for archive feed fetch.
2022-06-22 15:06:30 -04:00
Samuel Clay
04544f9c9f
Using correct unread cutoff.
2022-06-21 11:11:38 -04:00
Samuel Clay
82bba3f6a7
Temporarily disabling extra redis calls for archive users.
2022-05-25 18:21:42 -04:00
Samuel Clay
d65ece02d8
Left in a query that should've been higher.
2022-05-25 17:46:46 -04:00
Samuel Clay
1116569160
Adding graphic to archive thank you email.
2022-05-05 16:03:14 -04:00
Samuel Clay
03cab31143
Only use oldest unread story in unread only read filter for archive subscribers. Otherwise it will trim stories older than the oldest unread in All Stories.
2022-04-18 16:23:52 -04:00
