Shards database, collecting shard repositories, releases and dependencies.
- Crystal 0.30.1
- PostgresSQL database (for data storage). Should probably work with all versions => 10. Tested with PostgreSQL 11.5
PostgreSQL databases needs to be created manually. Connection configuration is read from environment variables:
- development database:
DATABASE_URL
- testing database:
TEST_DATABASE_URL
(only for running tests) - sentry:
SENTRY_DSN
(optional, for sending error reports to sentry)
Install database schema by running make db test_db
.
Run shards install
to install dependencies.
Run shards build worker
to build the worker executable.
bin/worker import_catalog
: Reads repositories mentioned incatalog/*.yml
files and synchronizes them with the database.bin/worker sync_repos [hours [ratio]]
: Synchronizes data from git repositories to database for all repositories not synced in the lasthours
hours (default:24
) or never synced at all.ratio
specifies the amount of outdated repositories being run in this job (default:2.0 / hours
). This enables an even distribution of batch sizes. Recommended to be run every hour.bin/worker updated_metrics
: Updates shard metrics in the database. Should be run once per day.
A web application for browsing the database is available at https://github.com/shardbox/shardbox-web
Run make test
to execute the spec suite.
- Fork it (https://github.com/shardbox/shardbox-web/fork)
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create a new Pull Request
- Johannes Müller - creator and maintainer