Commit graph

57 commits

Author SHA1 Message Date
Samuel Clay
da26ecdb46 1/2 Cleaning up feed fetcher's updates to feeds table. Still need to find 6 dupes. 2015-07-22 13:18:54 -07:00
Samuel Clay
82fbccd1c2 Handling invalid urls for icons. 2015-07-07 13:35:00 -07:00
Samuel Clay
94ce13183b Handling requests issue with too many redirects. 2015-07-07 13:30:46 -07:00
Samuel Clay
5b99ab01ec Adding error handling for new icon feed link fetcher. 2015-05-19 16:31:25 -07:00
Samuel Clay
e1f31b9277 Attempting to fetch Tumblr feed icons. 2015-05-18 17:51:23 -07:00
Samuel Clay
b746dda72d Fixing massive icon problems by removing support for Windows ICO files. 2014-05-16 12:33:06 -07:00
Samuel Clay
403a4ced90 Converting from PIL to Pillow. 2014-05-16 12:18:59 -07:00
Samuel Clay
07c5f95c36 Fixing a couple small bugs. 2013-10-07 16:16:38 -07:00
Samuel Clay
d3fa017cd9 Fixing broken icon bug. 2013-08-21 12:40:04 -07:00
Samuel Clay
5f7ae25fc4 Normaling user agents between feed/page/icon fetchers. Adding full site permalink. 2013-07-10 14:14:55 -07:00
Samuel Clay
8d9da758fd Better handling of busted favicons. 2013-07-05 17:43:14 -07:00
Samuel Clay
2408156024 Reverting most PIL cleanup. 2013-06-21 13:15:04 -07:00
Samuel Clay
bb95ea8984 Cleaning up PIL. 2013-06-21 13:11:54 -07:00
Samuel Clay
cc33a3e663 Fixing icon fetch for images > 500KB. 2013-06-04 16:30:21 -07:00
Samuel Clay
4de16b77f8 Adding expires header to icons. 2013-03-22 18:29:54 -07:00
Samuel Clay
279041249e Fixing missing scipy import missing form PILlow. 2013-03-22 15:33:26 -07:00
Samuel Clay
acb1efd3ea Last imports for pillow. 2013-03-22 15:27:10 -07:00
Samuel Clay
4a7516cb83 Fixing a bunch of feed fetch errors. 2012-10-01 19:31:33 -07:00
Samuel Clay
1252819c8c Fixing issue when opening feed in Story view but second story gets selected instead of first. 2012-09-21 11:36:27 -07:00
Samuel Clay
838cabbd42 Typo in s3 icon. 2012-09-20 12:44:41 -07:00
Samuel Clay
a8975a3bc4 Forcing icon save if the icon is not yet in s3. 2012-09-20 12:43:34 -07:00
Samuel Clay
1ace793648 Completing feed icon migration to S3. 2012-09-20 12:29:52 -07:00
Samuel Clay
88c86e273a S3 icons. Untested. 2012-09-19 16:33:23 -07:00
Samuel Clay
00ba259c66 Migrating to S3 for feed pages. 2012-09-18 17:09:07 -07:00
Samuel Clay
2e42cd3e8f Encoding utf-8 strings into bytecodes so lxml doesn't choke. 2012-08-29 23:16:23 -07:00
Samuel Clay
15df6b6b47 Making the icon importer a bit faster and more slave lag resilient by using page fetch data. Also fixing title on blurblogs. 2012-07-30 11:12:58 -07:00
Samuel Clay
febd0ae0a3 Lots of little bugs: avatar images, icon importer, page importer, adding analytics tracking to intro. 2012-07-12 12:57:35 -07:00
Samuel Clay
cd589951c9 Adding etags (304's) to feed icons. 2012-03-29 16:03:06 -07:00
Samuel Clay
53efae20ff Fixing a few bugs introduced with feed_id conversions. 2012-01-27 15:29:14 -08:00
Samuel Clay
2b9ad97b51 Switching a ton of feed and user fetches to simple ID matches. ".user.pk -> .user_id" 2012-01-26 09:32:24 -08:00
Samuel Clay
6a68181974 Fixing image processing issue and updating to latest feedparser. 2011-08-27 13:27:21 -07:00
Samuel Clay
130196e6e0 Adding attribution on favicon .ico handling. 2011-07-13 17:45:43 -07:00
Samuel Clay
bb18f3e284 Icon importer now saves to both mongo and psql for icons. 2011-04-26 19:43:19 -04:00
Samuel Clay
232af01b98 Commenting the icon importer for the Braindump Algorithms talk. 2011-04-13 20:05:34 -04:00
Samuel Clay
74c1227739 Fixing icon download errors. 2011-02-02 13:32:52 -05:00
Samuel Clay
1ed82654aa Fixing icon download errors. 2011-02-02 13:24:58 -05:00
Samuel Clay
7424dc9715 Adding a bit for finding the favicon. Also handling all errors on fetching the HTML of the site for finding the icon. 2011-01-30 20:42:11 -05:00
Samuel Clay
70bb450b8d Handling a weird typeerror on feed parsaing. 2011-01-30 20:16:34 -05:00
Samuel Clay
4b4c5b76aa Screw it. Too many different exceptions come from fetching URLs. If anything goes wrong, not much we can do. 2011-01-30 14:58:14 -05:00
Samuel Clay
5927f857b2 Handling non-numeric ports in icon fetching. 2011-01-30 14:26:18 -05:00
Samuel Clay
470af9fa19 Handling non-numeric ports in icon fetching. 2011-01-30 14:23:26 -05:00
Samuel Clay
f3d922f6b1 Fixing up statistics modal.Also handling weird URL cases in icon fetching. 2011-01-30 13:33:09 -05:00
Samuel Clay
40b6ac4c7c Handling parser errors from lxml in icon fetching. 2011-01-30 13:25:45 -05:00
Samuel Clay
3f4018249f Handling parser errors from lxml in icon fetching. 2011-01-30 13:24:17 -05:00
Samuel Clay
eefb189e52 Handling parser errors from lxml in icon fetching. 2011-01-30 13:23:54 -05:00
Samuel Clay
7941e2f8b0 Handling parser errors from lxml in icon fetching. 2011-01-30 13:23:20 -05:00
Samuel Clay
0175a8b26f Forcing useful feed_links. 2011-01-30 13:07:55 -05:00
Samuel Clay
bb0b8502af Handling timeouts from inside the icon_importer. 2011-01-30 13:06:31 -05:00
Samuel Clay
c5d380ec3e Adding timeout to icon fetching, and handling bad .ICOs 2011-01-30 12:53:05 -05:00
Samuel Clay
d4bed1399c Don't bother fetching an icon if we don't even have the feed_link. 2011-01-30 12:44:34 -05:00