NewsBlur-viq/README.md

NewsBlur

• A visual feed reader with intelligence.
• Live at www.newsblur.com.
• Created by Samuel Clay.
• Twitter: @samuelclay and @newsblur.

Features

1. Shows the original site (you have to see it to believe it)
2. Hides stories you don't want to read based on tags, keywords, authors, etc.
3. Highlights stories you want to read, based on the same criteria.

Technologies

Server-side

• Django: Web framework written in Python, used to serve all pages.
• Celery & RabbitMQ: Asynchronous queueing server, used to fetch and parse RSS feeds.
• MongoDB, Pymongo, & Mongoengine: Non-relational database, used to store stories, read stories, feed/page fetch histories, and proxied sites.
• PostgreSQL: Relational database, used to store feeds, subscriptions, and user accounts.

Client-side and design

• jQuery: Cross-browser compliant JavaScript code. IE works without effort.
• Underscore.js: Functional programming for JavaScript. Indispensible.
• Miscellaneous jQuery Plugins: Everything from resizable layouts, to progress bars, sortables, date handling, colors, corners, JSON, animations. See the complete list.

Installation Instructions

Prerequisites

RDBMS (MySQL, PostgreSQL)

You will want to have your database setup before you begin installation. Fabric can install both PostgreSQL and MongoDB for you, but only on Ubuntu. Mac OS X users will want to have MySQL or PostgreSQL already installed. You can download MySQL or download PostgreSQL. Additionally, if running as a development machine on Mac OS X, I would recommend using MySQL with Sequel Pro as a GUI.

If you are installing MySQL, you will also need the MySQLDB python library:

sudo easy_install mysql-python

Fabric

Both Mac OS X and Linux require Fabric to be installed. Many common tasks, such as installing dependencies, deploying servers, migrations, and configurations are in fabfile.py.

sudo easy_install fabric

On recent installations of Mac OS X using XCode 4, you may run into issues around the ppc architecture. To fix this, simply run:

sudo ln -s /Developer/Platforms/iPhoneOS.platform/Developer/usr/libexec/gcc/darwin/ppc \
/Developer/usr/libexec/gcc/darwin
sudo ln -s /Developer/Platforms/iPhoneOS.platform/Developer/usr/libexec/gcc/darwin/ppc \
/usr/libexec/gcc/darwin

Sym-linking the ppc architecture comes from this StackOverflow answer on "assembler for architecture ppc not installed on Mac OS".

MongoDB

On top of MySQL/PostgreSQL, NewsBlur uses MongoDB to store non-relational data. You will want to download MongoDB. If you are on Ubuntu, the setup_mongo Fabric command will automatically do this for you, but Mac OS X needs to have it installed manually.

Configure paths

In fabfile.py there are two paths that need to be configured.

• env.paths.NEWSBLUR is the relative path to the NewsBlur repository.
• env.paths.VENDOR is the relative path to where all downloaded code should go.

In local_settings.py there are a few paths that need to be configured. Configure these after the installation below.

Installing on Mac OS X

1. Using Mac OS X as a development environment, you can run all three servers (app, db, task) on the same system. You should have Fabric installed to run the fabfile.py. You should also have MySQL/PostgreSQL and MongoDB already installed.

   fab -R local setup_python
   fab -R local setup_mongoengine
   fab -R local setup_forked_mongoengine
   fab -R local setup_repo_local_settings
   

   If any of the packages fail to install (lxml, for instance), look through fabfile.py and check if there is a function that can be used to circumvent broken easy_install processes. For example, lxml may need libxslt and libxml2 to be installed. This is automated with the following Fabric command:

   fab -R local setup_libxml_code
   

2. Configure MySQL/PostgreSQL by adding in a newsblur user and a newsblur database. Then load up the database with empty NewsBlur tables and bootstrap the database:

   ./manage.py syncdb --all
   ./manage.py migrate --fake
   ./manage.py loaddata config/fixtures/bootstrap.json
   

   If you don't create a user during syncdb, the bootstrap.json file will create a newsblur user with no password.

3. Run the development server. At this point, all dependencies should be installed and no additional configuration is needed. If you find that something is not working at this point, please email the resulting output to Samuel Clay at samuel@ofbrooklyn.com.

   ./manage.py runserver
   

Installing on Linux / Ubuntu

If you are on Ubuntu, you can simple use Fabric to install NewsBlur and its many components. NewsBlur is designed to run on three separate servers: an app server, a db server, and assorted task servers. To install everything on a single machine, read through fabfile.py and setup all three servers without repeating the setup_common steps.

Finishing Installation

You must perform a few tasks to tie all of the various systems together.

1. First, copy local_settings.py and fill in your OAuth keys, S3 keys, database names (if not newsblur), task server/broker address (RabbitMQ), and paths:

   cp local_settings.py.template local_settings.py

2. Create the newsblur database in both MongoDB and MySQL/PostgreSQL

   MySQL/PostgreSQL

   ./manage.py syncdb
   

   MongoDB

App server

fab -R local setup_app

Database server

fab -R local setup_db

Task server

fab -R local setup_task

Roadmap

Winter 2011

• River of News
• Starred stories

Summer 2011

• iPhone app
• Implicit sorting in River of News

Fall 2011

• Social features

Author

• Samuel Clay samuel@ofbrooklyn.com
• @samuelclay on Twitter

License

NewsBlur is licensed under the MIT License. (See LICENSE)