Commit graph

97 commits

Author SHA1 Message Date
Samuel Clay
31de0f7a97 Upgrading to latest pymongo v3 to work with mongodb v4.0. Required rewrite of mongodb introspection, now using built-in pymongo monitoring. Should work on both web and celery. 2022-05-10 20:58:47 -04:00
Samuel Clay
2097c5276e Boosting db profiling from 1% to 5%. 2022-03-08 13:32:49 -05:00
Samuel Clay
a4ae23c34f Wrong url for db profiler. 2022-03-07 15:36:23 -05:00
Samuel Clay
7a6eaf883b Let's try showing only single feeds in the DB profiler to see if it's better at showing abnormalities with less noise. 2022-03-07 15:33:11 -05:00
Samuel Clay
bddd0ca215 Only monitor river_stories for db times 2022-03-04 12:59:29 -05:00
Samuel Clay
d2097c3331 Black formatting 2022-03-02 10:58:15 -05:00
Samuel Clay
e502dd2294 Missing redis db insight. 2021-12-08 16:44:36 -05:00
Samuel Clay
b2fde6f8f7 Turning refresh stale feeds back on. 2021-12-07 10:45:06 -05:00
Samuel Clay
779348c495 Removing unused db redis query counters. 2021-12-06 10:49:37 -05:00
Samuel Clay
2a45cd0df7 Adding query logging for redis pubsub. 2021-08-03 17:19:45 -04:00
Samuel Clay
3cc5fbe184 Log missing queries 2021-08-03 17:16:24 -04:00
Samuel Clay
d767310a31 Not sure where redis log is coming from, but it should still be recorded. 2021-08-03 16:46:34 -04:00
Samuel Clay
abfacef9c9 Splitting redis query logging into three dbs: user, story, and session. 2021-08-03 16:05:48 -04:00
Samuel Clay
c055656869 Adding DEBUG_QUERIES_SUMMARY_ONLY to assist db introspection without the verbosity. 2021-07-15 16:50:54 -04:00
Samuel Clay
87b25c6bf6 Adding more granularity to mongo queries. Fixing double query bug. 2021-07-02 16:12:14 -04:00
Samuel Clay
04f4f6b903 Only make the sql template if showing. 2021-04-12 15:57:58 -04:00
Samuel Clay
2d8df4af28 Don't spit out queries unless requested. 2021-04-12 15:52:38 -04:00
Samuel Clay
c86a79c414 Don't turn off DEBUG if it wasn't off before. 2021-01-11 20:39:38 -05:00
Samuel Clay
a1f4d67d27 Changing how tasks are called. 2021-01-06 14:42:24 -05:00
Samuel Clay
6e07dbaf0f Merge branch 'master' into docker_django3.1
* master: (22 commits)
  Whoops
  Better logging for broken paypal ipns.
  Adding RSS Tracker for Windows to Goodies.
  Refactoring socketio so real-time works around username restrictions.
  Socket.IO v3 on the server too.
  Switching to Socket.IO v3.
  Haproxy splitting for socketio 3
  Socket.io 3
  Handling missing feed.
  Only nb.local.com uses ports for websockets.
  Adding GrazeTEN
  Full node install on setup.
  Updating toprc
  Changing mongodb server.
  Fixing toprc
  Attempting to collect queries for 1% of requests by using DEBUG.
  Fixing mongo install.
  Recalculating premium expiration to take into account forced expirations after gifts.
  Fixing never expire -> 1 year
  Adding 1 year expire to admin.
  ...
2021-01-05 19:12:19 -05:00
Samuel Clay
a5cac63a74 Attempting to collect queries for 1% of requests by using DEBUG. 2021-01-04 10:22:04 -05:00
Samuel Clay
1c62266127 mimetype -> content_type 2020-11-13 11:10:03 -05:00
Jonathan Math
5877fd979e missed some mimetype edits in django upgrade 2020-10-24 19:25:26 +07:00
Jonathan Math
89e51a2937 change celery Task import and task initialization as per the new version of celery 2020-10-05 00:45:20 +07:00
Samuel Clay
1351aee6a8 Vendorizing new feedparser. 2020-07-01 18:38:37 -04:00
Samuel Clay
eeb44e3400 Fixing a bunch of feed fetch py3 errors. Fetching now works! 2020-06-30 20:50:30 -04:00
Samuel Clay
1fc063ff80 Fixing middleware for new format. 2020-06-30 19:46:32 -04:00
Samuel Clay
0d6cb69548 Merge branch 'django1.11' into django2.0
* django1.11: (73 commits)
  Switching to new celery 4 standalone binary.
  Fixing various mongo data calls.
  Upgrading to latest celery 4 (holy moly), which required some big changes to project layout. Still needs supervisor scripts updated.
  Removing unused log on cookies.
  I believe this Context wrapping is still preserved. See this django ticket: https://code.djangoproject.com/ticket/28125. Reverting this fixes the error, so I'm assuming this is that type of render.
  Have to revert 3f122d5e03 because this broke existing sessions (logged me out) because the model has changed and the serialized model stored in redis no longer matches. Whew, this took a while to figure out.
  Upgrading redis cache.
  Adding cookies to path inspector.
  Removing dupe db log.
  Fixing missing DB logs (redis and mongo) due to this change in django 1.8: "connections.queries is now a read-only attribute."
  Removing migrations that set a default date of 2020-05-08. Not sure why this was committed. I thought we resolved the issue with default datetimes?
  Fixing CallableBool.
  Missing import
  Fixing runtime errors on django 1.10
  Fixing OAuth connect.
  Fixing various django1.9 issues, mainly around templates.
  BASE_DIR
  Not every story is from a feed.
  Styling background colors for newsletters.
  Styling more newsletter elements.
  ...
2020-06-30 12:34:59 -04:00
Samuel Clay
553483ce52 I believe this Context wrapping is still preserved. See this django ticket: https://code.djangoproject.com/ticket/28125. Reverting this fixes the error, so I'm assuming this is that type of render. 2020-06-28 14:52:09 -04:00
Samuel Clay
36b0823786 Merge branch 'django1.10' into django1.11
* django1.10: (131 commits)
  Have to revert 3f122d5e03 because this broke existing sessions (logged me out) because the model has changed and the serialized model stored in redis no longer matches. Whew, this took a while to figure out.
  Upgrading redis cache.
  Adding cookies to path inspector.
  Removing dupe db log.
  Fixing missing DB logs (redis and mongo) due to this change in django 1.8: "connections.queries is now a read-only attribute."
  Removing migrations that set a default date of 2020-05-08. Not sure why this was committed. I thought we resolved the issue with default datetimes?
  Fixing CallableBool.
  Missing import
  Fixing runtime errors on django 1.10
  Fixing OAuth connect.
  Fixing various django1.9 issues, mainly around templates.
  BASE_DIR
  Not every story is from a feed.
  Styling background colors for newsletters.
  Styling more newsletter elements.
  Using the correct attribute.
  Typo in template
  Attempting to style newsletters in dark mode.
  Loading dark theme if selected on load.
  Adding scrollbar colors.
  ...
2020-06-28 14:00:09 -04:00
Samuel Clay
56b17fe72f Merge branch 'django1.9' into django1.10
* django1.9:
  Removing dupe db log.
  Fixing missing DB logs (redis and mongo) due to this change in django 1.8: "connections.queries is now a read-only attribute."
  Removing migrations that set a default date of 2020-05-08. Not sure why this was committed. I thought we resolved the issue with default datetimes?
2020-06-26 16:27:50 -04:00
Samuel Clay
2956cb5861 Merge branch 'master' into django1.8
* master:
  Removing dupe db log.
2020-06-26 16:27:16 -04:00
Samuel Clay
d0f15ab141 Removing dupe db log. 2020-06-26 16:26:59 -04:00
Samuel Clay
c475fd2edc Fixing missing DB logs (redis and mongo) due to this change in django 1.8: "connections.queries is now a read-only attribute." 2020-06-26 15:56:06 -04:00
Jonathan Math
8dfa0053a0 call process_request() in profile middleware 2020-06-20 00:06:01 -04:00
Jonathan Math
044370e2f4 use updated middleware syntax 2020-06-17 03:24:16 -04:00
Jonathan Math
7112b19f18 upgrade middleware classes and run 2to3 on apps/profile/ 2020-06-17 00:40:16 -04:00
jmath1
88f19ffaf6 render receives a dict instead of a context object in apprs.profiel.middleware.py 2020-06-12 03:19:53 -04:00
jmath1
7d95dbb5cf change User.is_authenticated() and User.is_anonymous() to attributes 2020-06-11 15:13:12 -04:00
Samuel Clay
182c8f8d18 Profiling fewer app servers. 2018-02-26 18:35:02 -08:00
Samuel Clay
6a8dd09ecc Only profiling 10%. 2018-02-26 15:16:20 -08:00
Samuel Clay
5ae1e3f341 Profiling individual feeds. 2018-02-26 15:06:06 -08:00
Samuel Clay
55eb59c0cd Adding db profiler to 1% of app calls. 2018-02-26 15:02:26 -08:00
Samuel Clay
d904c6ad1a Removing banned user. 2017-05-30 15:14:22 -07:00
Samuel Clay
811520ef0c Banning users. 2017-05-26 18:29:10 -07:00
Samuel Clay
4a12c220fe Risky business: now capturing db times (sql/mongo/redis) for 1% of all tasks. Also updating munin to recognize new db profiling times. 2015-12-16 17:31:41 -08:00
Samuel Clay
9bf765ba07 DB checks OK in middleware. 2015-08-05 20:05:54 -07:00
Samuel Clay
cf43747b20 Adding user sub counts to redis, using new redis cache to count subscribers only when feed is entirely cached. Still need automatic conversion from postgres to redis sub counts. 2015-07-28 18:46:37 -07:00
Samuel Clay
0b398f4194 Capping IP at 15 characters. 2015-07-27 18:21:40 -07:00
Samuel Clay
553688c629 Fixing HTTP_X_FORWARDED_FOR header. 2015-07-21 10:35:02 -07:00