| .vscode | ||
| ansible | ||
| api | ||
| apps | ||
| archive | ||
| blog | ||
| clients | ||
| config | ||
| docker | ||
| documents | ||
| flask_monitor | ||
| logs | ||
| media | ||
| newsblur_web | ||
| node | ||
| perf | ||
| templates | ||
| terraform | ||
| utils | ||
| vendor | ||
| .dockerignore | ||
| .gitignore | ||
| .venv | ||
| ansible.cfg | ||
| ansible.old.cfg | ||
| docker-compose.metrics.yml | ||
| docker-compose.yml | ||
| LICENSE.md | ||
| Maintenance.md | ||
| Makefile | ||
| manage.py | ||
| pytest.ini | ||
| README.md | ||
| requirements.txt | ||
| UPGRADING.md | ||
NewsBlur
- NewsBlur is a personal news reader bringing people together to talk about the world. A new sound of an old instrument.
- www.newsblur.com.
- Created by Samuel Clay.
- Twitter: @samuelclay and @newsblur.
Features
- Shows the original site (you have to see it to believe it).
- Hides stories you don't want to read based on tags, keywords, authors, etc.
- Highlights stories you want to read, based on the same criteria.
Technologies
Server-side
- Python 3.7+: The language of choice.
- Django: Web framework written in Python, used to serve all pages.
- Celery & RabbitMQ: Asynchronous queueing server, used to fetch and parse RSS feeds.
- MongoDB, Pymongo, & Mongoengine: Non-relational database, used to store stories, read stories, feed/page fetch histories, and proxied sites.
- PostgreSQL: Relational database, used to store feeds, subscriptions, and user accounts.
- Redis: Programmer's database, used to assemble stories for the river, store story ids, manage feed fetching schedules, and the minuscule bit of caching that NewsBlur uses.
- Elasticsearch: Search database, use for searching stories. Optional.
Client-side and design
- jQuery: Cross-browser compliant JavaScript code. IE works without effort.
- Underscore.js: Functional programming for JavaScript. Indispensable.
- Backbone.js: Framework for the web app. Also indispensable.
- Miscellaneous jQuery Plugins: Everything from resizable layouts, to progress bars, sortables, date handling, colors, corners, JSON, animations. See the complete list.
Prerequisites
* Docker
* Docker-Compose
Installation Instructions
-
Run
make nbto build Newsblur containers. This will set up all necessary databases, celery tasks, node applications, flask database monitor, NGINX, and a Haproxy load balancer. -
Navigate to:
https://localhostNote: You will be warned that you are using a self signed certificate. In order to get around this warning you must type "thisisunsafe" as per https://dblazeski.medium.com/chrome-bypass-net-err-cert-invalid-for-development-daefae43eb12
Making docker-compose work with your database
To make docker-compose work with your database, upgrade your local database to the docker-compose version and then volumize the database data path by changing the ./docker/volumes/ part of the volume directive in the service to point to your local database's data directory.
To make docker-compose work with an older database version, change the image version for the database service in the docker-compose file.
Contribution Instructions
-
Making Changes:
- To apply changes to the Python or JavaScript code, use the
makecommand. - To apply changes to the docker-compose.yml file, use the
make rebuildcommand. - To apply changes to the docker/haproxy/haproxy.conf file, node packages, or any new database migrations you will need to use the
make nbcommand.
- To apply changes to the Python or JavaScript code, use the
-
Adding Python packages: Currently, the docker-compose.yml file uses the newsblur/newsblur_python3 image. It is built using the Dockerfile found in
docker/newsblur_base_image.Dockerfile. Because of how the docker image is set up, you will need to create your own image and direct your docker-compose.yml file to use it. Please follow the following steps to do so.- Add your new site-packages to config/requirements.txt.
- Add the following lines of code to your docker-compose.yml file to replace anywhere where it says
image: newsblur/newsblur_python3
build: context: . dockerfile: docker/newsblur_base_image.Dockerfile- Run the
make nbcommand to rebuild your docker-compose containers
-
Debugging Python
- To debug your code, drop
import pdb; pdb.set_trace()into the Python code where you would like to start debugging and runmakeand thenmake debug.
- To debug your code, drop
-
Using Django shell within Docker
- Make sure your docker containers are up and run
make shellto open the Django shell within the newsblur_web container.
- Make sure your docker containers are up and run
Running unit and integration tests
NewsBlur comes complete with a test suite that tests the functionality of the rss_feeds, reader, and feed importer. To run the test suite:
`make test`
Running a performance test
Performance tests use the locust performance testing tool. To run performance tests via CLI, use
make perf-cli users=1 rate=1 host=https://localhost. Feel free to change the users, rate, and host
variables in the command to meet you needs.
You can also run locust performance tests using a UI by running make perf-ui and then navigating to
http://127.0.0.1:8089. This allows you to chart and export your performance data.
To run locust using docker, just run make perf-docker and navigate to http://127.0.0.1:8089
Author
- Created by Samuel Clay.
- Email address: samuel@newsblur.com
- @samuelclay on Twitter.
License
NewsBlur is licensed under the MIT License. (See LICENSE)