Samuel Clay
d6f9ef24f8
Preprocess feed content to fix Verge specific encoding issues.
2025-04-05 14:10:34 -07:00
Samuel Clay
16e22348a7
Adjusting hours per fetch for forbidden feeds based on number of subscribers.
2025-04-02 09:26:11 -07:00
Samuel Clay
5d85cfeae2
Adding /reader/save_dashboard_rivers for #1899
2025-03-23 13:10:22 -07:00
Samuel Clay
e47a291ea7
553 feeds are also fetched through ninja
2025-03-23 12:42:10 -07:00
Samuel Clay
70e1928558
Bumping forbidden feed fetch rate.
2025-03-20 14:27:03 -07:00
Samuel Clay
302074814d
Handling connection error in search.
2025-03-18 10:39:28 -07:00
Samuel Clay
90c34120bd
Forbidden feeds get a min of 6 hours.
2025-03-04 22:40:36 -08:00
Samuel Clay
6556d7c789
Sending 403'd sites over to scrapeninja.
2025-03-04 22:25:04 -08:00
Samuel Clay
077228004c
Upgrading to elasticsearch 8 on docker-compose local installations. If you are upgrading to this release, make sure you delete your old search index otherwise you'll get errors in ES about an old index. Delete by running docker exec -it newsblur_web ./manage.py reindex_stories --reindex and docker exec -it newsblur_web ./manage.py reindex_feeds
2025-02-21 16:30:55 -08:00
Samuel Clay
20e49ec586
Rate-limiting original text to 30 times a minute, which should be more than enough but enough to stop android clients from hammering the servers.
2025-02-05 22:46:51 -08:00
Samuel Clay
8053c40b7b
Fixing docker commands, adding health check.
2025-02-02 12:30:43 -08:00
Samuel Clay
c4e33ff541
Missing import.
2025-01-27 15:06:45 -08:00
Samuel Clay
bacc3b9e4d
User searches in prometheus.
2025-01-27 15:02:53 -08:00
Samuel Clay
0476f7b83c
Fixing issue where dupe classifiers cause new classifiers to not be saved.
2025-01-27 15:02:53 -08:00
Samuel Clay
54c60c1a23
Ratelimit starred stories
2025-01-21 14:28:52 -08:00
Samuel Clay
74237b6c48
Run discover in the same thread and not in the background.
2025-01-14 09:58:33 -08:00
Samuel Clay
c0cbb8cc11
5 min fetches should be between 5-10 minutes.
2025-01-14 09:57:43 -08:00
Samuel Clay
96540f02a2
Defaulting to /month price for stripe premium pro.
2025-01-07 18:01:10 -08:00
Samuel Clay
f6677f512a
New Premium Pro subscription.
2025-01-07 17:50:52 -08:00
Samuel Clay
8278d9fe1d
Don't count similar feeds on the regular.
2024-12-25 17:51:49 -05:00
Samuel Clay
9967692d53
Logging
2024-12-25 17:33:58 -05:00
Samuel Clay
eb5ea92390
Switching to elasticsearch v8 for discover.
2024-12-24 19:53:37 -05:00
Samuel Clay
5f02400567
Re-normalizing dense vectors from 1584 dimensions to 256 dimensions, applying bbq_hnsw index to save 96% memory usage on vector embeddings.
2024-12-24 19:40:48 -05:00
Samuel Clay
335bf1922d
Logging old discover date vs unindexed.
2024-12-21 12:52:41 -05:00
Samuel Clay
58754abaa7
Fixing touch discover date typo
2024-12-21 12:50:50 -05:00
Samuel Clay
6cfd38f6e3
Re-scheduling discover indexing if indexing didn't finish in the last day and user is checking related again.
2024-12-20 11:35:29 -05:00
Samuel Clay
78945886f0
Bumping time limit for discover indexing.
2024-12-19 14:16:18 -05:00
Samuel Clay
4427eb27d0
Indexing discover stories as they come in a) in a task, and b) only for archive subscribers.
2024-12-19 12:48:27 -05:00
Samuel Clay
f837804fbf
Indexing discover stories when using Related Stories sideoption. Also reindexes discover on upgrade.
2024-12-14 23:18:34 -08:00
Samuel Clay
9a2a52c190
Real-time updates on discover being indexed. Need to sort out exactly when to start indexing, becuase on MUserSearch creation won't cover people who only use Related Stories. That should touch the MUserSearch for discover.
2024-12-14 23:04:57 -08:00
Samuel Clay
58e66f7f23
Only index discover stories for premium archive users.
2024-12-14 14:06:09 -08:00
Samuel Clay
153cb876b5
Moving discover indexing to its own queue.
2024-12-14 13:39:21 -08:00
Samuel Clay
ba87788afb
isort
2024-12-09 21:33:57 -08:00
Samuel Clay
f8b667edda
Adding preferences to hide story sideoptions buttons.
2024-12-01 23:38:01 -05:00
Samuel Clay
5b9f6d402f
No need to reindex stories once indexed.
2024-12-01 12:32:18 -05:00
Samuel Clay
e67399fe46
Refactoring discover stories so that it now loads the correct story when clicked on. Still work to be done on layout showing the feed, the double embedding padding offset, and moving to below the sideoptions when the sideoptions are below the story. Also maybe paging?
2024-11-27 23:20:34 -05:00
Samuel Clay
4cfb64b6bd
Adding index debug to all search models.
2024-11-24 13:57:48 -08:00
Samuel Clay
a45170f0dc
Adding debug_index
2024-11-24 13:20:37 -08:00
Samuel Clay
74f0f25711
Cleaning up discover stories index, adding discover_indexed field, beginning to split discover/search indexing. Needs further splitting at the task/queue level.
2024-11-21 08:38:14 -08:00
Samuel Clay
a0cf47abaf
Separating index discover from index stories. Need to separate out index discover and index story when initially indexing so as to return search results to the user faster. Also need to index stories when adding a feed for premium users.
2024-11-21 07:17:57 -08:00
Samuel Clay
d3280ff95b
Splitting DiscoverStory from SearchStory so the indexes can be separate.
2024-11-20 08:37:31 -08:00
Samuel Clay
6dc247b693
Storing last used discover stories folder for all stories in that folder/feed.
2024-11-19 22:10:05 -08:00
Samuel Clay
3e961c60e6
Merge branch 'master' into discover
...
* master: (36 commits)
Static deploys should force gunicorn reload.
New X/Twitter icons.
New Android screenshots.
New screenshots for homepage, updating all twitter links to x.com. Still need Android screenshot from S22.
Android 13.3.2 230
#1886 Support for themed icons
#1898 (Marking story as read/unread right after loading will mark different story)
Bumped build number
#1907 (Resizable middle column)
Fixed #1906 (story list leaves a space after all stories read)
#1903 (Saved searches don't work)
#1902 (Adjust position of unread indicator on story title select)
#1247 (Mac Catalyst edition)
Prep
Add nullability annotations to ReadingAction
#1247 (Mac Catalyst edition)
Fixed offline images not appearing
Android 13.3.1 229
#1891 Handle Android 9 and below for OPML file export
Add nullability annotation to help with Kotlin conversion
...
2024-11-17 20:24:17 -08:00
Samuel Clay
0e312ae7fd
New screenshots for homepage, updating all twitter links to x.com. Still need Android screenshot from S22.
2024-11-13 23:18:16 -08:00
Samuel Clay
19522cc7f1
Not perfect but stories are now rendered. Need to calculate the height and disable scroll.
2024-11-06 22:11:17 -08:00
Samuel Clay
d04b4b038a
Adding API endpoints for discover stories. Need to finish the UI for stories. Then prob add premium limits.
2024-11-06 21:29:07 -08:00
Samuel Clay
a2d3785c7c
Merge branch 'master' into discover
...
* master:
Backfilling Youtube videos for premium archive subscribers.
2024-11-04 21:58:20 -08:00
Samuel Clay
a5229ce5cb
Backfilling Youtube videos for premium archive subscribers.
2024-11-04 21:32:15 -08:00
Samuel Clay
9e7b08b536
Indexing stories with content vectors.
2024-11-03 21:40:27 -08:00
Samuel Clay
9a605c1918
Fixing paging on folders, no longer unnecessarily loading river on discover scroll.
2024-10-27 12:24:37 -07:00
