<?xml version="1.0" encoding="utf-8"?><feedxmlns="http://www.w3.org/2005/Atom"><generatoruri="https://jekyllrb.com/"version="4.3.4">Jekyll</generator><linkhref="https://blog.newsblur.com/feed.xml"rel="self"type="application/atom+xml"/><linkhref="https://blog.newsblur.com/"rel="alternate"type="text/html"/><updated>2025-02-03T01:02:37-05:00</updated><id>https://blog.newsblur.com/feed.xml</id><titletype="html">The NewsBlur Blog</title><subtitle>NewsBlur is a personal news reader that brings people together to talk about the world.
</subtitle><entry><titletype="html">Discover related stories and sites</title><linkhref="https://blog.newsblur.com/2025/01/28/discover-related-stories-and-sites/"rel="alternate"type="text/html"title="Discover related stories and sites"/><published>2025-01-28T00:00:00-05:00</published><updated>2025-01-28T00:00:00-05:00</updated><id>https://blog.newsblur.com/2025/01/28/discover-related-stories-and-sites</id><contenttype="html"xml:base="https://blog.newsblur.com/2025/01/28/discover-related-stories-and-sites/"><![CDATA[<p>I want to introduce you to the new Discover Stories and Discover Sites features. Sometimes you’re reading a story and want to know everything there is to know about that topic. You want other stories, but depending on the topic, you might want them from the same site, from similar sites, or from all of your subscriptions. That’s the new Discover Stories feature, and it’s only for NewsBlur Premium Archive subscribers. The Premium Archive subscription is meant for this use case of being able to peer deeply into your story archive and not just what’s been published in the last month.</p>
<p>Second I’m introducing Discover Sites, which is available at the top of every feed and folder to everybody, both free and premium users. Having tried all of the competing discover sites features, I built the popover dialog that has all the features I wanted. It’s an infinite scroll of related sites, showing the most recent five stories, formatted exactly as your story titles are personally styled. You can read stories from unsubscribed feeds and easily subscribe to them while scrolling through the discover stories dialog.</p>
<p>Here’s a set of features I’ve been wanting to build since the very first days of NewsBlur in 2009. I built prototypes of this feature using a few of the modern text tools at the time: nltk (the natural language toolkit), support vector machines, and LDA (Latent Dirichlet Allocation) to group stories by topic. It didn’t work, or it was too slow, and even then not accurate enough. I read the tea leaves and could tell a better tool would come out eventually that was basically a drop-in classifier and topic grouper. Out came word embeddings (word2vec initially, then <ahref="https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2">sentence transformers</a>). And now those transformers are available basically for free.</p>
<p>As you can see, this isn’t your normal related stories feature. It shows all of the related stories, segmented by the folders that a site is a part of. This folder control allows you to filter down to an individual site and up to every feed you subscribe to when finding related stories.</p>
<p>And it’s important to note that none of the data presented in the Discover Stories or Discover Sites dialog is trained on your personal data, like feeds that other people subscribe to in relation to any particular site. All of the data is extracted and grouped by the content of the RSS feed’s title, description, and the titles of the first few stories.</p>
<p>Above we see that Discover Sites is right on the money. An infinite scroll of related sites, showing story previews, and multiple interaction points that let you choose between trying out a site by reading one of the stories, adding it directly to a folder, or checking the statistics of the site. The stats dialog is great in this case because it gives you a feel for what other people like and dislike about the site.</p>
<p>I’m super proud of this release; it took years to build and a decade to plan. And while the Discover Stories feature is technically only available to Premium Archive subscribers, you can see related stories if another Premium Archive subscriber is subscribed to that site. I don’t think hiding those stories from free and premium users is worthwhile.</p>
<p>Please post your feedback on the <ahref="https://forum.newsblur.com">NewsBlur forum</a>, ideally as an “idea,” but you know I love responding to all feedback. For every person who writes up their thoughts on the forum, there are ten people who are thinking the same thing, so it’s worthwhile to hear from you, knowing the multiplier it represents.</p>]]></content><author><name></name></author><categoryterm="web"/><summarytype="html"><![CDATA[I want to introduce you to the new Discover Stories and Discover Sites features. Sometimes you’re reading a story and want to know everything there is to know about that topic. You want other stories, but depending on the topic, you might want them from the same site, from similar sites, or from all of your subscriptions. That’s the new Discover Stories feature, and it’s only for NewsBlur Premium Archive subscribers. The Premium Archive subscription is meant for this use case of being able to peer deeply into your story archive and not just what’s been published in the last month.]]></summary></entry><entry><titletype="html">NewsBlur’s native macOS App offers news notifications directly on your desktop</title><linkhref="https://blog.newsblur.com/2024/10/22/newsblur-macos-app/"rel="alternate"type="text/html"title="NewsBlur’s native macOS App offers news notifications directly on your desktop"/><published>2024-10-22T00:00:00-04:00</published><updated>2024-10-22T00:00:00-04:00</updated><id>https://blog.newsblur.com/2024/10/22/newsblur-macos-app</id><contenttype="html"xml:base="https://blog.newsblur.com/2024/10/22/newsblur-macos-app/"><![CDATA[<p>If you’re like me and like to have NewsBlur sitting open all day, then you’ll love the new NewsBlur macOS app. It’s a first-class app that supports all of NewsBlur’s features, from intelligence training to sharing/blurblogs.</p>
<p>It’s configurable and supports ay=utomatic hiding and showing of the feed list so you can focus on the stories you want to read. Use your mouse to swipe left and right on both stories and to swap which pane is visible.</p>
<p>In the Grid view, you can swipe right with your mouse to temporarily show the feed list, giving you a compact view of your news stories without having to give up screen real estate.</p>
<p>It’s important to be able to train, because you can set notifications to be sent from either your Unread list or your Focus list, ensuring you only see the notifications from sites you want to see. And clicking on those native macOS notifications takes you directly to the story in the new macOS app.</p>
<p>Coming up soon are the discover feeds feature, where you can see related feeds based purely on semantic similarity (and not based on mined usage data), as well as real-time updates to the macOS app similar to the dashboard on the web.</p>]]></content><author><name></name></author><categoryterm="macos"/><summarytype="html"><![CDATA[If you’re like me and like to have NewsBlur sitting open all day, then you’ll love the new NewsBlur macOS app. It’s a first-class app that supports all of NewsBlur’s features, from intelligence training to sharing/blurblogs.]]></summary></entry><entry><titletype="html">Introducing the Grid view on iOS</title><linkhref="https://blog.newsblur.com/2023/12/06/ios-grid-view/"rel="alternate"type="text/html"title="Introducing the Grid view on iOS"/><published>2023-12-06T00:00:00-05:00</published><updated>2023-12-06T00:00:00-05:00</updated><id>https://blog.newsblur.com/2023/12/06/ios-grid-view</id><contenttype="html"xml:base="https://blog.newsblur.com/2023/12/06/ios-grid-view/"><![CDATA[<p>The Grid view is now on iOS. Read stories with large thumbnails in a magazine-like format, where you can see a customizable number of story previews at once. Works beautifully for both iPhone and iPad.
<p>Just like on the web, you can customize how many stories you see and how large each story is, giving you the freedom to read stories with large thumbnails or small image previews.</p>
<p>If you have any other ideas you’d like to see on iPad and iPhone, feel free to post an idea on the <ahref="https://forum.newsblur.com">NewsBlur Forum</a>.</p>
<p>This is a huge release and has been a year in the making. Coming up soon: a new Mac app and intelligent feed discovery.</p>]]></content><author><name></name></author><categoryterm="ios"/><summarytype="html"><![CDATA[The Grid view is now on iOS. Read stories with large thumbnails in a magazine-like format, where you can see a customizable number of story previews at once. Works beautifully for both iPhone and iPad.]]></summary></entry><entry><titletype="html">2022 redesign: new dashboard layout, refreshed stories and story titles, and entirely redrawn icons</title><linkhref="https://blog.newsblur.com/2022/07/01/dashboard-redesign-2022/"rel="alternate"type="text/html"title="2022 redesign: new dashboard layout, refreshed stories and story titles, and entirely redrawn icons"/><published>2022-07-01T00:00:00-04:00</published><updated>2022-07-01T00:00:00-04:00</updated><id>https://blog.newsblur.com/2022/07/01/dashboard-redesign-2022</id><contenttype="html"xml:base="https://blog.newsblur.com/2022/07/01/dashboard-redesign-2022/"><![CDATA[<p>The launch of the new <ahref="/2022/07/01/premium-archive-subscription/">Premium Archive subscription tier</a> also includes the 2022 redesign. You’ll see a third dashboard layout which stretches out your dashboard rivers across the width of the screen.</p>
<p>The latest redesign style has more accomodations for spacing and padding around each story title element. The result is a cleaner story title with easier to read headlines. The author has been moved and restyled to be next to the story date. Favicons and unread status indicators have been swapped, and font sizes, colors, and weights have been adjusted.</p>
<p>If you find the interface to be too airy, there is a setting in the main Manage menu allowing you to switch between Comfortable and Compact. The compact interface is denser than before, giving power users a highly detailed view.</p>
<p>Transitions have also been added to help you feel the difference. And there are new animations during many of the transitions that accompany changing settings.</p>
<p>And lastly, this redesign comes with a suite of all new icons. The goal with this icon redesign is to bring a consistent weight to each icon as well as vectorize them with SVG so they look good at all resolutions.</p>
<p>A notable icon change is the unread indicator, which now has different size icons for both unread stories and focus stories, giving focus stories more depth.</p>
<p>Here’s a screenshot that’s only possible with the new premium archive, complete with backfilled blog post from the year 2000, ready to be marked as unread.</p>
<p>I tried to find every icon, so if you spot a dialog or menu that you’d like to see given some more love, reach out on the support forum.</p>]]></content><author><name></name></author><categoryterm="web"/><summarytype="html"><![CDATA[The launch of the new Premium Archive subscription tier also includes the 2022 redesign. You’ll see a third dashboard layout which stretches out your dashboard rivers across the width of the screen.]]></summary></entry><entry><titletype="html">NewsBlur Premium Archive subscription keeps all of your stories searchable, shareable, and unread forever</title><linkhref="https://blog.newsblur.com/2022/07/01/premium-archive-subscription/"rel="alternate"type="text/html"title="NewsBlur Premium Archive subscription keeps all of your stories searchable, shareable, and unread forever"/><published>2022-07-01T00:00:00-04:00</published><updated>2022-07-01T00:00:00-04:00</updated><id>https://blog.newsblur.com/2022/07/01/premium-archive-subscription</id><contenttype="html"xml:base="https://blog.newsblur.com/2022/07/01/premium-archive-subscription/"><![CDATA[<p>For $99/year every story from every site you subscribe to will stay in NewsBlur’s archive. This new premium tier also allows you to mark any story as unread as well as choose when stories are automatically marked as read. You can now have full control of your story archive, letting you search, share, and read stories forever without having to worry about them being deleted.</p>
<li><imgsrc="/assets/icons8/icons8-bursts-100.png"style="width: 16px;margin: 0 6px 0 0;display: inline-block;"/> Everything in the premium subscription, of course</li>
<li><imgsrc="/assets/icons8/icons8-relax-with-book-100.png"style="width: 16px;margin: 0 6px 0 0;display: inline-block;"/> Choose when stories are automatically marked as read</li>
<li><imgsrc="/assets/icons8/icons8-filing-cabinet-100.png"style="width: 16px;margin: 0 6px 0 0;display: inline-block;"/> Every story from every site is archived and searchable forever</li>
<li><imgsrc="/assets/icons8/icons8-quadcopter-100.png"style="width: 16px;margin: 0 6px 0 0;display: inline-block;"/> Feeds that support paging are back-filled in for a complete archive</li>
<li><imgsrc="/assets/icons8/icons8-rss-100.png"style="width: 16px;margin: 0 6px 0 0;display: inline-block;"/> Export trained stories from folders as RSS feeds</li>
<p>NewsBlur uses two techniques to retrieve older stories that are no longer in the RSS feed. The first strategy is to append `?page=2` and `?paged=2` to the RSS feed and seeing if we're about to blindly iterate through the blog's archive. For WordPress and a few other CMSs, this works great and gives us a full archive. </p>
<p>A second technique is to use <ahref="https://datatracker.ietf.org/doc/html/rfc5005">RFC 5005</a>, which supports links embedded inside the RSS feed to denote next and previous pages of an archive.</p>
<p>NewsBlur attempts all of these techniques on every single feed you’ve subscribed to, and when it’s done backfilling stories, you’ll receive an email showing you how big your archive grew during this backfill process.</p>
<p>The launch of the new Premium Archive subscription tier also contains the <ahref="/2022/07/01/dashboard-redesign-2022/">2022 redesign</a>, which includes a new dashboard layout, a refreshed design for story titles and feed title, and all new icons.</p>
<p>Here’s a screenshot that’s only possible with the new premium archive, complete with backfilled blog post from the year 2000, ready to be marked as unread.</p>
<p>How’s that for an archive?</p>]]></content><author><name></name></author><categoryterm="web"/><categoryterm="ios"/><categoryterm="android"/><summarytype="html"><![CDATA[For $99/year every story from every site you subscribe to will stay in NewsBlur’s archive. This new premium tier also allows you to mark any story as unread as well as choose when stories are automatically marked as read. You can now have full control of your story archive, letting you search, share, and read stories forever without having to worry about them being deleted.]]></summary></entry><entry><titletype="html">New gesture-based layout for the NewsBlur iPad App</title><linkhref="https://blog.newsblur.com/2022/03/28/redesigned-ios-layout/"rel="alternate"type="text/html"title="New gesture-based layout for the NewsBlur iPad App"/><published>2022-03-28T00:00:00-04:00</published><updated>2022-03-28T00:00:00-04:00</updated><id>https://blog.newsblur.com/2022/03/28/redesigned-ios-layout</id><contenttype="html"xml:base="https://blog.newsblur.com/2022/03/28/redesigned-ios-layout/"><![CDATA[<p>We have a big update for you on iOS, complete with a redesigned layout engine. You’ll see this mostly on iPad, where you can now interactively swipe between panes, customize how many panes you see, and even customize where the story titles are on the screen relative to the story content.</p>
<p>A whole bunch of new controls and customizations have been added to the settings menu in the story titles menu, which is where you can find these new options for 2-column/3-column/full screen view panes.</p>
<p>You can also expect to find feature parity with Android and the web when it comes to the new image preview and content preview options for story titles.</p>
<p>You can now save stories and subscribe to feeds from other apps using the NewsBlur Share extension. This includes your saved story tags and associated counts.</p>
<p>If you have any ideas for how you would like to see the iPad and iPhone app improved, please post ideas to the <ahref="https://forum.newsblur.com">NewsBlur Forum</a>. I’m all ears and would love to prioritize improvements or changes that create a better mobile reading experience.</p>]]></content><author><name></name></author><categoryterm="ios"/><summarytype="html"><![CDATA[We have a big update for you on iOS, complete with a redesigned layout engine. You’ll see this mostly on iPad, where you can now interactively swipe between panes, customize how many panes you see, and even customize where the story titles are on the screen relative to the story content.]]></summary></entry><entry><titletype="html">Magazine view offers a new perspective</title><linkhref="https://blog.newsblur.com/2022/03/10/magazine-view/"rel="alternate"type="text/html"title="Magazine view offers a new perspective"/><published>2022-03-10T00:00:00-05:00</published><updated>2022-03-10T00:00:00-05:00</updated><id>https://blog.newsblur.com/2022/03/10/magazine-view</id><contenttype="html"xml:base="https://blog.newsblur.com/2022/03/10/magazine-view/"><![CDATA[<p>Here’s a nice feature that brings a new perspective to your stories. It’s called the Magazine view and features larger images, longer story content previews, and improved legibility of text.</p>
<p>The Magazine view is also customizable. By default, fonts are a bit larger in the Magazine view. You can still change font sizes as well as customize the position and size of the image preview. There is also a control for how long the story content previews are, all found inside the Style popover.</p>
<p>The Grid view also features improvements to the story content preview. New lines are now preserved in both Magazine and Grid views, so you can capture a bit of longer form stories without opening them up.</p>
<p>Also included are some backend changes to how YouTube videos thumbnail are found, so you should see even more image previews in your feeds.</p>]]></content><author><name></name></author><categoryterm="web"/><summarytype="html"><![CDATA[Here’s a nice feature that brings a new perspective to your stories. It’s called the Magazine view and features larger images, longer story content previews, and improved legibility of text.]]></summary></entry><entry><titletype="html">Redesigning NewsBlur on the web, iOS, and Android</title><linkhref="https://blog.newsblur.com/2021/07/01/refreshing-newsblur-design/"rel="alternate"type="text/html"title="Redesigning NewsBlur on the web, iOS, and Android"/><published>2021-07-01T00:00:00-04:00</published><updated>2021-07-01T00:00:00-04:00</updated><id>https://blog.newsblur.com/2021/07/01/refreshing-newsblur-design</id><contenttype="html"xml:base="https://blog.newsblur.com/2021/07/01/refreshing-newsblur-design/"><![CDATA[<p>This past year we’ve focused on maintenance and improving quality behind the scenes. It just so happens that the urge to clean is so strong that this work extended to the front-end. After months of work, today we’re launching a redesigned NewsBlur for all three platforms: on the web, on iOS, and on Android. There’s a lot that’s new. And what better day to launch a redesign than on <ahref="/2013/03/17/three-months-to-scale-newsblur/">the ninth anniversary of the sunset of Google Reader</a>.</p>
<p>Those multiple rivers come in handy when you want to follow different interests at a glance. You can of course change which individual feeds or folders is loaded, letting you focus on saved searches, infrequent stories, a single feed, or everything you’re subscribed to.</p>
<p>Below you can see the design in action. Notice how easy it is to change where the image preview is located as well as adjust the number of lines of story text to show.</p>
<p>The reading experience itself has also seen improvement. Full bleed images have been ported over from iOS to both Android and the web. This means that images will now run edge-to-edge. And the controls at the top and bottom of the web app have been restyled to be easier to understand at a quick glance.</p>
<p>The redesign has also come to both of the official Android and iOS apps. Right now both are in beta testing, but you can join the <ahref="https://testflight.apple.com/join/hYk9WU3f">iOS TestFlight</a> or the <ahref="https://play.google.com/store/apps/details?id=com.newsblur&hl=en_US&gl=US">Android beta</a>.</p>
<p>This whole redesign weighs in at a whopping 1,316 commits, which <ahref="https://github.com/samuelclay/NewsBlur/compare/python2...master">you can view on GitHub</a>. Much of the work that took place here involves upgrading from Python 2 to Python 3 and containerizing everything with Docker. In a few weeks, we’ll post a technical writeup of what those backend changes are and how you can now run a local version of NewsBlur on your own computer with a single line of code. For those that want to run their own private instance of NewsBlur, that line of code is <codeclass="language-plaintext highlighter-rouge">make nb</code> and <ahref="https://github.com/samuelclay/NewsBlur">instructions are found on the repo</a>.</p>
<p>If you’ve enjoyed using NewsBlur and are a fan of this grand redesign, please take a moment to share on social media that you read your news with the help of NewsBlur.</p>]]></content><author><name></name></author><categoryterm="backend"/><categoryterm="web"/><categoryterm="ios"/><categoryterm="android"/><summarytype="html"><![CDATA[This past year we’ve focused on maintenance and improving quality behind the scenes. It just so happens that the urge to clean is so strong that this work extended to the front-end. After months of work, today we’re launching a redesigned NewsBlur for all three platforms: on the web, on iOS, and on Android. There’s a lot that’s new. And what better day to launch a redesign than on the ninth anniversary of the sunset of Google Reader.]]></summary></entry><entry><titletype="html">How a Docker footgun led to a vandal deleting NewsBlur’s MongoDB database</title><linkhref="https://blog.newsblur.com/2021/06/28/story-of-a-hacking/"rel="alternate"type="text/html"title="How a Docker footgun led to a vandal deleting NewsBlur’s MongoDB database"/><published>2021-06-28T00:00:00-04:00</published><updated>2021-06-28T00:00:00-04:00</updated><id>https://blog.newsblur.com/2021/06/28/story-of-a-hacking</id><contenttype="html"xml:base="https://blog.newsblur.com/2021/06/28/story-of-a-hacking/"><![CDATA[<p><em>tl;dr: A vandal deleted NewsBlur’s MongoDB database during a migration. No data was stolen or lost.</em></p>
<p>I’m in the process of moving everything on NewsBlur over to Docker containers in prep for a <ahref="https://beta.newsblur.com">big redesign launching next week</a>. It’s been a great year of maintenance and I’ve enjoyed the fruits of Ansible + Docker for NewsBlur’s 5 database servers (PostgreSQL, MongoDB, Redis, Elasticsearch, and soon ML models). The day was wrapping up and I settled into <ahref="https://en.wikipedia.org/wiki/Human_Compatible">a new book on how to tame the machines once they’re smarter than us</a> when I received a strange NewsBlur error on my phone.</p>
<divclass="language-plaintext highlighter-rouge"><divclass="highlight"><preclass="highlight"><code>"query killed during yield: renamed collection 'newsblur.feed_icons' to 'newsblur.system.drop.1624498448i220t-1.feed_icons'"
</code></pre></div></div>
<p>There is honestly no set of words in that error message that I ever want to see again. What is <codeclass="language-plaintext highlighter-rouge">drop</code> doing in that error message? Better go find out.</p>
<p>Logging into the MongoDB machine to check out what state the DB is in and I come across the following…</p>
<spanclass="dl">"</span><spanclass="s2">content</span><spanclass="dl">"</span><spanclass="p">:</span><spanclass="dl">"</span><spanclass="s2">All your data is a backed up. You must pay 0.03 BTC to XXXXXXFTHISGUYXXXXXXX 48 hours for recover it. After 48 hours expiration we will leaked and exposed all your data. In case of refusal to pay, we will contact the General Data Protection Regulation, GDPR and notify them that you store user data in an open form and is not safe. Under the rules of the law, you face a heavy fine or arrest and your base dump will be dropped from our server! You can buy bitcoin here, does not take much time to buy https://localbitcoins.com or https://buy.moonpay.io/ After paying write to me in the mail with your DB IP: FTHISGUY@recoverme.one and you will receive a link to download your database dump.</span><spanclass="dl">"</span>
<p>Three and a half hours before this happened, I switched the MongoDB cluster over to the new servers. When I did that, I shut down the original primary in order to delete it in a few days when all was well. And thank goodness I did that as it came in handy a few hours later. Knowing this, I realized that the hacker could not have taken all that data in so little time.</p>
<p>I can definitively write that no data was leaked during the hack. I know this because of two different sets of logs showing that the automated attacker only issued deletion commands and did not transfer any data off of the MongoDB server.</p>
<p>You can imagine the stress I experienced in the forty minutes between 9:35p, when the hack began, and 10:15p, when the fresh backup snapshot was identified and put into gear. Let’s breakdown each moment:</p>
<li><strong>6:10p</strong>: The new db-mongo1 server was put into rotation as the MongoDB primary server. This machine was the first of the new, soon-to-be private cloud.</li>
<li><strong>9:35p</strong>: Three hours later an automated hacking attempt opened a connection to the db-mongo1 server and immediately dropped the database. Downtime ensued.</li>
<li><strong>10:15p</strong>: Before the former primary server could be placed into rotation, a snapshot of the server was made to ensure the backup would not delete itself upon reconnection. This cost a few hours of downtime, but saved nearly 18 hours of a day’s data by not forcing me to go into the daily backup archive.</li>
<li><strong>3:00a</strong>: Snapshot completes, replication from original primary server to new db-mongo1 begins. What you see in the next hour and a half is what the transfer of the DB looks like in terms of bandwidth.</li>
<li><strong>4:30a</strong>: Replication, which is inbound from the old primary server, completes, and now replication begins outbound on the new secondaries. NewsBlur is now back up.</li>
<p>The most important bit of information the above chart shows us is what a full database transfer looks like in terms of bandwidth. From 6p to 9:30p, the amount of data was the expected amount from a working primary server with multiple secondaries syncing to it. At 3a, you’ll see an enormous amount of data transfered.</p>
<p>This tells us that the hacker was an automated digital vandal rather than a concerted hacking attempt. And if we were to pay the ransom, it wouldn’t do anything because the vandals don’t have the data and have nothing to release.</p>
<p>We can also reason that the vandal was not able to access any files that were on the server outside of MongoDB due to using a recent version of MongoDB in a Docker container. Unless the attacker had access to a 0-day to both MongoDB and Docker, it is highly unlikely they were able to break out of the MongoDB server connection.</p>
<p>Turns out the ufw firewall I enabled and diligently kept on a strict allowlist with only my internal servers didn’t work on a new server because of Docker. When I containerized MongoDB, Docker helpfully inserted an allow rule into iptables, opening up MongoDB to the world. So while my firewall was “active”, doing a <codeclass="language-plaintext highlighter-rouge">sudo iptables -L | grep 27017</code> showed that MongoDB was open the world. This has been <ahref="https://github.com/moby/moby/issues/4737">a Docker footgun since 2014</a>.</p>
<p>To be honest, I’m a bit surprised it took over 3 hours from when I flipped the switch to when a hacker/vandal dropped NewsBlur’s MongoDB collections and pretended to ransom about 250GB of data. This is the work of an automated hack and one that I was prepared for. NewsBlur was back online a few hours later once the backups were restored and the Docker-made hole was patched.</p>
<p>It would make for a much more dramatic read if I was hit through a vulnerability in Docker instead of a footgun. By having Docker silently override the firewall, Docker has made it easier for developers who want to open up ports on their containers at the expense of security. Better would be for Docker to issue a warning when it detects that the most popular firewall on Linux is active and filtering traffic to a port that Docker is about to open.</p>
<p>The second reason we know that no data was taken comes from looking through the MongoDB access logs. With these rich and verbose logging sources we can invoke a pretty neat command to find everybody who is not one of the 100 known NewsBlur machines that has accessed MongoDB.</p>
2021-06-24T01:34:08.018+0000 I COMMAND [conn63456637] dropDatabase config - starting
2021-06-24T01:34:08.018+0000 I COMMAND [conn63456637] dropDatabase config - dropping 1 collections
2021-06-24T01:34:08.018+0000 I COMMAND [conn63456637] dropDatabase config - dropping collection: config.transactions
2021-06-24T01:34:08.020+0000 I STORAGE [conn63456637] dropCollection: config.transactions (no UUID) - renaming to drop-pending collection: config.system.drop.1624498448i1t-1.transactions with drop optime { ts: Timestamp(1624498448, 1), t: -1 }
2021-06-24T01:34:08.029+0000 I REPL [replication-14545] Completing collection drop for config.system.drop.1624498448i1t-1.transactions with drop optime { ts: Timestamp(1624498448, 1), t: -1 } (notification optime: { ts: Timestamp(1624498448, 1), t: -1 })
2021-06-24T01:34:08.030+0000 I STORAGE [replication-14545] Finishing collection drop for config.system.drop.1624498448i1t-1.transactions (no UUID).
2021-06-24T01:34:08.030+0000 I COMMAND [conn63456637] dropDatabase config - successfully dropped 1 collections (most recent drop optime: { ts: Timestamp(1624498448, 1), t: -1 }) after 7ms. dropping database
2021-06-24T01:34:08.032+0000 I REPL [replication-14546] Completing collection drop for config.system.drop.1624498448i1t-1.transactions with drop optime { ts: Timestamp(1624498448, 1), t: -1 } (notification optime: { ts: Timestamp(1624498448, 5), t: -1 })
2021-06-24T01:34:08.041+0000 I COMMAND [conn63456637] dropDatabase config - finished
2021-06-24T01:34:08.398+0000 I COMMAND [conn63456637] dropDatabase newsblur - starting
2021-06-24T01:34:08.398+0000 I COMMAND [conn63456637] dropDatabase newsblur - dropping 37 collections
<p>The above is a lot, but the important bit of information to take from it is that by using a subtractive filter, capturing everything that doesn’t match a known IP, I was able to find the two connections that were made a few seconds apart. Both connections from these unknown IPs occured only moments before the database-wide deletion. By following the connection ID, it became easy to see the hacker come into the server only to delete it seconds later.</p>
<p>Interestingly, when I visited the IP address of the <ahref="http://185.220.101.6/">two</a><ahref="http://171.25.193.78/">connections</a> above, I found a Tor exit router:</p>
<p>This means that it is virtually impossible to track down who is responsible due to the anonymity-preserving quality of Tor exit routers. <ahref="https://blog.cloudflare.com/the-trouble-with-tor/">Tor exit nodes have poor reputations</a> due to the havoc they wreak. Site owners are split on whether to block Tor entirely, but some see the value of allowing anonymous traffic to hit their servers. In NewsBlur’s case, because NewsBlur is a home of free speech, allowing users in countries with censored news outlets to bypass restrictions and get access to the world at large, the continuing risk of supporting anonymous Internet traffic is worth the cost.</p>
<p>Of course, being in support of free speech and providing enhanced ways to access speech comes at a cost. So for NewsBlur to continue serving traffic to all of its worldwide readers, several changes have to be made.</p>
<p>The first change is the one that, ironically, we were in the process of moving to. A VPC, a virtual private cloud, keeps critical servers only accessible from others servers in a private network. But in moving to a private network, I need to migrate all of the data off of the publicly accessible machines. And this was the first step in that process.</p>
<p>The second change is to use database user authentication on all of the databases. We had been relying on the firewall to provide protection against threats, but when the firewall silently failed, we were left exposed. Now who’s to say that this would have been caught if the firewall failed but authentication was in place. I suspect the password needs to be long enough to not be brute-forced, because eventually, knowing that an open but password protected DB is there, it could very possibly end up on a list.</p>
<p>Lastly, a change needs to be made as to which database users have permission to drop the database. Most database users only need read and write privileges. The ideal would be a localhost-only user being allowed to perform potentially destructive actions. If a rogue database user starts deleting stories, it would get noticed a whole lot faster than a database being dropped all at once.</p>
<p>But each of these is only one piece of a defense strategy. <ahref="https://news.ycombinator.com/item?id=27613217">As this well-attended Hacker News thread from the day of the hack made clear</a>, a proper defense strategy can never rely on only one well-setup layer. And for NewsBlur that layer was a allowlist-only firewall that worked perfectly up until it didn’t.</p>
<p>As usual the real heros are backups. Regular, well-tested backups are a necessary component to any web service. And with that, I’ll prepare to <ahref="https://beta.newsblur.com">launch the big NewsBlur redesign later this week</a>.</p>]]></content><author><name></name></author><categoryterm="backend"/><summarytype="html"><![CDATA[tl;dr: A vandal deleted NewsBlur’s MongoDB database during a migration. No data was stolen or lost.]]></summary></entry><entry><titletype="html">Android app update: premium subscriptions, saved searches, in-app browser, auto-dark mode</title><linkhref="https://blog.newsblur.com/2020/11/03/android-app-update-premium-subscriptions-saved/"rel="alternate"type="text/html"title="Android app update: premium subscriptions, saved searches, in-app browser, auto-dark mode"/><published>2020-11-03T07:41:03-05:00</published><updated>2020-11-03T07:41:03-05:00</updated><id>https://blog.newsblur.com/2020/11/03/android-app-update-premium-subscriptions-saved</id><contenttype="html"xml:base="https://blog.newsblur.com/2020/11/03/android-app-update-premium-subscriptions-saved/"><![CDATA[<p>For a point release this one sure is big. The Android app has been upgraded to include a bunch of features found on the web.</p>
<p>For one, premium subscriptions can now be purchased in the Android app itself. Reading by folder, saved story tags, searching and saved searches are all premium features that you can unlock directly in the app.</p>
<li>Premium subscriptions are now available on Android! Read by folder, saved story tags, searching, and more is exclusive to premium subscribers.</li>
<li>Saved searches</li>
<li>In-app browser, so you don’t need to leave NewsBlur</li>
<li>Auto-theme option for dark mode so it can turn on automatically at night</li>
<li>You can now delete and rename folders and add a folder while adding a feed</li>
<li>Fixed issues around the intelligence trainer, HTML in comments, some images not loading</li>
<p>If you would like to request a new feature on Android, please submit an idea on the <ahref="https://forum.newsblur.com">NewsBlur Forum</a>. We’re prioritizing the next big release and would love to hear your input.</p>]]></content><author><name></name></author><categoryterm="android"/><summarytype="html"><![CDATA[For a point release this one sure is big. The Android app has been upgraded to include a bunch of features found on the web.]]></summary></entry></feed>
