This project was a working proof of concept for a Python verion of Atlas Library.
The future of Atlas Libary will most likely use concepts from this repo but with a different frontend.
Until that project happens this repo will be archived.

This is a Python version of the Atlas BI Library DotNet webapp. It is currently under development.

In our love of open source we felt the need for a version of the Atlas BI Library webapp that used completely open source tools. This version of the app will effectively mirror the DotNet version in functionality and appearance, but will be installed on Ubuntu server.

Take a look at the github project to see a list of planned features.

See the project documentation

• Suggest a new feature or idea in our discussion board!
• Try out the daily build demo. Please create an issue for any bugs you find!
• Contribute to the code!

This version of the app is built using python + django.

• Precommit pre-commit install
• Poetry
• Pyenv pyenv local 3.6.2 3.7.0 3.8.0 3.9.0
• NodeJS
• Developing on windows is difficult, but can be done. Git sdk64 works well for installing and running these tools.

There are a few settings files to run the app. The required settings have already been set in the existing files. Org specific settings can be added in *_cust.py files. They will be ignored in commits.

If you need to override any of the default config, add your overrides to the *_cust.py files.

The names should be:

• settings_cust.py
• dev_cust.py
• prod_cust.py
• test_cust.py

As an example, if you want to use an existing Atlas sql server database, you can add a database config like this:

# stop multiple queries for db version
from sql_server.pyodbc.base import DatabaseWrapper
DatabaseWrapper.sql_server_version = 2017

DATABSES = "default": {
    "ENGINE": "sql_server.pyodbc",
    "NAME": "atlas",
    "HOST": "server_name",
    "USER": "datagov",
    "PASSWORD": "12345",
    "OPTIONS": {
        "driver": "ODBC Driver 17 for SQL Server",
        "extra_params": "MARS_Connection=Yes",
    },
    "schemas": ["app", "dbo"],
},
# note, sql server will only allow connections if app is the default schema for the user.

Redis, Solr and a database should be up. See solr/readme.md for a guide to starting up a demo solr instance for development.

In terminal 1, start webapp:

cd atlas && poetry run python manage.py runserver

In terminal 2, start celery (for ETL's):

DJANGO_SETTINGS_MODULE='atlas.settings.dev' poetry run celery -A atlas worker -l DEBUG

In terminal 3, start celery beat (for scheduled ETL's):

DJANGO_SETTINGS_MODULE='atlas.settings.dev' poetry run celery -A atlas beat -l DEBUG --scheduler django_celery_beat.schedulers:DatabaseScheduler

In terminal 4, start static file watcher

npm run watch

npm run build

# or live
npm run watch

Testing uses a local postgres server and redis server. The server names are postgres and redis to allow them to run as a service in the ci/cd pipelines. The best thing is to add a mapping in your local host file of 127.0.0.1 postgres and 127.0.0.1 redis.

1. Start postgres in a docker container. (You can do the same with redis, or, as in our case, install with homebrew.)

   docker run --name postgresql-container -p 5432:5432 -e POSTGRES_HOST_AUTH_METHOD=trust -d postgres

2. Start solr

See /solr/readme.rst for a guide.

3. Run code tests directly

   poetry run python manage.py test --no-input --pattern="test_views.py" --settings atlas.settings.test
   
   # or with tox
   # run with py36, 37, 38 or 39.
   tox -e clean,py39,cov

4. Run browser tests

   BROWSERSTACK_USERNAME=<browserstack username> \
   BROWSERSTACK_ACCESS_KEY=<browserstack accesskey> \
   BROWSERSTACK_BUILD_NAME="local" \
   BROWSERSTACK_PROJECT_NAME="Atlas-Py" \
   poetry run python manage.py test --no-input --pattern="test_browser.py" --settings atlas.settings.test_browser
   
   # or with tox
   tox -e clean,browsertest,cov -r

Integrates with db-first sqlserver, or managed postres db.

Using python-memcached

to create cache:

python manage.py createcachetable

create a new tag

git tag x.x.x
git push origin --tags

This will trigger a workflow to build a release.

Edit release to trigger build of .deb installer.