Commit graph

149 commits

Author SHA1 Message Date
Samuel Clay
e1b2583950 Swapping out pyes for elasticsearch-py. Half-way done, got indices and mappings in, now need to work on querying and storing. 2021-03-18 16:34:28 -04:00
Samuel Clay
dbec2708da Only show active feeds in dashboard river. Also link dashboard river title and icon. 2021-03-04 10:24:17 -05:00
Samuel Clay
ab2d9804ef Adjusting to newsblur_web. Also adding feed chooser to dashboard river. Needs hookup. 2021-01-12 15:52:10 -05:00
Jonathan Math
89e51a2937 change celery Task import and task initialization as per the new version of celery 2020-10-05 00:45:20 +07:00
jmath1
34e017721b 2to3 apps/rss_feeds 2020-06-15 03:00:09 -04:00
jmath1
6021afaec3 2to3 apps/rss_feeds 2020-06-15 02:54:37 -04:00
jmath1
ebd0da2d63 change management commands to use argparse instead of optparse 2020-06-08 07:55:17 -04:00
jmath1
63dbaa82b3 change requires_model_validation to requires_system_checks in shellplus management command 2020-06-07 03:00:35 -04:00
Samuel Clay
c8439da97e Removing unused merge_feeds command, this is better used in rss_feeds/models.py 2020-05-21 16:54:26 -04:00
Samuel Clay
3ebf31334e Minting xlsx spreadsheet from queries. 2016-10-06 17:20:30 -07:00
Samuel Clay
4a334a50b0 Rudimentary sorting of global keywords based on subscribers. 2016-10-06 15:24:53 -07:00
Samuel Clay
97b0910e67 Breaking up tasking of normal feeds and broken feeds so broken feeds are a less drain on postgresql. 2015-07-23 16:29:47 -07:00
Samuel Clay
da26ecdb46 1/2 Cleaning up feed fetcher's updates to feeds table. Still need to find 6 dupes. 2015-07-22 13:18:54 -07:00
Samuel Clay
c20ef4e5d0 Adding updates_off run-time setting for handling large spikes of updates. 2014-05-29 17:53:16 -07:00
Samuel Clay
3f50a1b311 Adding stories to search index. 2014-04-11 17:25:13 -07:00
Samuel Clay
073098750c Swapping out haystack for pyes. 2014-04-11 15:40:58 -07:00
Samuel Clay
1584208677 Adding metric for last story date, used to calculate next scheduled update. 2013-04-23 17:04:21 -07:00
Samuel Clay
bd9f992262 Fixing credit card images. Doh\! 2013-04-19 09:51:30 -07:00
Samuel Clay
c41bf80b4e Removing force from new feeds. 2013-04-07 20:22:46 -07:00
Samuel Clay
edc3a80667 Adding backup mongo task to be run by cron instead. 2012-12-10 11:37:35 -08:00
Samuel Clay
c19713eabf Adding offsets to unread count fixing. 2012-10-30 11:41:01 -07:00
Samuel Clay
769a1e084e Manually counting can now use offsets in case of broken feeds. 2012-10-30 09:11:39 -07:00
Samuel Clay
e572b3d53f Moving to better feed finder (in the case of dupe feeds). 2012-10-25 16:14:25 -07:00
Samuel Clay
f0dc76b71e Merging separate task feeds tasks. 2012-08-09 21:18:45 -07:00
Samuel Clay
34ad54e1f5 Fixing feed update times for inactive feeds to never go inadctive but to instead go into geometrically longer feed fetches. 2012-08-09 13:26:52 -07:00
Samuel Clay
e40b93882a Fixing issue around feeds thinking they are unfetched. 2012-06-18 15:59:31 -07:00
Samuel Clay
b5c5cdeda3 Merge branch 'master' into social
* master:
  Fixing icon fetching error when the page is corrupted.
  Writing a glorious hack that forces a count of unread scores.

Conflicts:
	apps/reader/models.py
	apps/rss_feeds/management/commands/refresh_feeds.py
	fabfile.py
	utils/feed_fetcher.py
2012-04-24 17:55:20 -07:00
Samuel Clay
c1b5aef983 Fixing icon fetching error when the page is corrupted. 2012-04-24 17:40:34 -07:00
Samuel Clay
d209aa5e25 Writing a glorious hack that forces a count of unread scores. 2012-04-24 17:02:55 -07:00
Samuel Clay
2200aa282a Subscribe to blog during intro. 2012-03-19 15:46:59 -07:00
Samuel Clay
d901ccdae1 Switching to redis to feed queuing. Also adding a weight for skipping feed fetches under high load. 2012-02-26 22:33:06 -08:00
Samuel Clay
cbc8df2c1c Skipping feed resurrection. 2012-02-24 17:07:42 -08:00
Samuel Clay
ce536200f6 Adding ability to wind down feed queue by faking requests. 2012-02-24 16:43:08 -08:00
Samuel Clay
c9cb7dcbc0 Adding new feeds db column 'known_good', for forcing updates of feeds that were once good but are now throwing 500s. 2012-02-01 17:59:46 -08:00
Samuel Clay
b5efad9142 Merge branch 'master' into requests
* master: (66 commits)
  Too many stories? Don't animate transitions when switching intelligence levels.
  Fixing typo in iphone app that preventing mark folder as read when the visible stories option was showing.
  Updating the mongo db copy util to also look for updated stories.
  Globals.is_staff on user, not profile. Make the stats referesh every minute instead of every 10 minutes for staff.
  Fixing offsets in river of news.
  Adding ciphering to usernames in log, so I can finally take screenshots of the most colorful logs of all time.
  Adding my very special mongo serialization backup utility to get around the damn unrepairable mongo database. This is taking 14 hours to run.
  Auto-refreshing feedback (1 min for staff, 10 min for everybody).
  Fixing exception around multiple feeds during Google Reader import process. Also switching rate limit to status code 429.
  Adding Nokia MeeGo client to user agents.
  Only show raw feeds in feed autocomplete.
  Handling iphone version for new users on ios app.
  FIXING THE WORST BUG OF MY LIFE -- finally figured out what was causing the story-shows-as-unread bug. Also fixed enclosures on certain types of feeds.
  Fixing menu manage open position to compensate for additional menu items.
  Reducing the amount of work done by feed fetching when there are no new stories.
  Fixing emergency bug around trimming feeds where the cursor is changing. Dammit mongo.
  Simplifying ufw ports in fabfile. Adding env.user.
  Launching the iPhone app on the front-page. Big deal.
  Minor cleanup of river stories view.
  Cleaning up mongoengine imports and settings for default MongoDB.
  ...

Conflicts:
	local_settings.py.template
2011-12-20 21:47:55 -08:00
Samuel Clay
f3d2f8e5b5 Moving from slave_db to ReplicaSetConnection in pymongo 2.1. 2011-12-11 10:54:07 -08:00
Samuel Clay
b112717d95 Revert "Revert "Switching to requests from urllib2/httplib. Bring on the page errors.""
This reverts commit a827b2c864.
2011-11-27 02:41:12 -05:00
Samuel Clay
a827b2c864 Revert "Switching to requests from urllib2/httplib. Bring on the page errors."
This reverts commit 617dc3bb56.

Conflicts:

	local_settings.py.template
2011-11-27 02:40:44 -05:00
Samuel Clay
d07b03a7a4 Revert "Revert "Switching to requests from urllib2/httplib. Bring on the page errors.""
This reverts commit 7972d0bb5c.
2011-11-26 11:15:01 -05:00
Samuel Clay
7972d0bb5c Revert "Switching to requests from urllib2/httplib. Bring on the page errors."
This reverts commit 617dc3bb56.
2011-11-26 02:16:14 -05:00
Samuel Clay
617dc3bb56 Switching to requests from urllib2/httplib. Bring on the page errors. 2011-11-26 02:07:31 -05:00
Samuel Clay
ac3609bf7d Merge branch 'master' into realtime
* master:
  Adding 'E' as a keyboard shortcut for the Everything view.
  Doubling the amount Space pages in a story.
  Auto-linkify urls in stories.
  Reading stories from slave db during feed update.

Conflicts:
	settings.py
2011-11-06 15:28:37 -08:00
Samuel Clay
07fa01ed9a Adding server half of redis real-time unread count updates. That was the easy part. 2011-11-05 17:08:31 -07:00
Samuel Clay
040b755694 Reading stories from slave db during feed update. 2011-11-04 09:45:10 -07:00
Samuel Clay
c2b1d7eb35 Fixing the infernal FeedNotFound error whenever a dupe feed is created in the middle of another transaction. Yarr! 2011-05-19 19:04:10 -04:00
Samuel Clay
8e2936fc1b Utils -> vendor 2011-04-11 21:57:45 -04:00
Samuel Clay
1254d350d6 Only tasking feeds that are active. Clears the exception pile. 2011-04-07 12:36:38 -04:00
Samuel Clay
bdc8e7bdea Separating nginx configuration into a setup and a configure, since one half is much more common than the other. 2011-03-31 18:51:23 -04:00
Samuel Clay
09021ea08c Trying to fix stale feeds by checking for a last update of a day. 2011-02-22 09:37:09 -05:00
Samuel Clay
3723160155 A huge number of misc fixes: fixing potential errors on paypal return screen, mailing all feed/page parse errors, and swithcing to mapreduce item_frequencies in mongoengine. 2011-02-05 15:34:43 -05:00