Provides core components of the Oasis platform, specifically:
- DJango application that provides the Oasis REST API
- Celery worker for running a model
A simple environment for the testing the API and worker can be created using Docker compose.
First make sure that you have Docker running. Then build the images for the API server and model execution worker:
docker build -f Dockerfile.oasis_api_server.base -t coreoasis/oasis_api_base .
docker build -f Dockerfile.oasis_api_server.mysql -t coreoasis/oasis_api_server .
docker build -f Dockerfile.model_execution_worker -t coreoasis/model_execution_worker .
The docker images for the server are structured to all inherit from oasis_api_server:base
.
Then each child image will be setup for a specific database backend (for example
oasis_api_server:mysql
uses mysql as the database backend).
Start the docker network:
docker-compose up -d
Check that the server is running:
curl localhost:8000/healthcheck
(For the Rabbit management application browse to https://localhost:15672, for Flower, the Celery management application, browse to https://localhost:5555.)
To run a test analysis using the worker image and example model data, run the following command:
docker exec oasisplatform_runner_1 sh run_api_test_analysis.sh
The API server provides a REST interface which is described on the home page of the api server at https://localhost:8000/. This documentation also provides an interactable interface to the api or you can use any command line client such as curl.
We use the GitFlow model described here .
The main idea is that the central repo holds two main branches with an infinite lifetime:
- master: main branch where the source code of HEAD always reflects a production-ready state
- develop: main branch where the source code of HEAD always reflects a state with the latest delivered development changes for the next release. This is where our automatic builds are built from.
When the source code in the develop branch reaches a stable point and is ready to be released, all of the changes should be merged back into master. Feature branchs should be used for new features and fixes, then a Pull Request isues to merge into develop.
Dependencies are controlled by pip-tools
. To install the development dependencies, first install pip-tools
using:
pip install pip-tools
and run:
pip-sync
To add new dependencies to the development requirements add the package name to requirements.in
or to add a new dependency to the installed package add the package name to requirements-package.in
.
Version specifiers can be supplied to the packages but these should be kept as loose as possible so that all packages can be easily updated and there will be fewer conflict when installing.
After adding packages to either *.in
file, the following command should be ran ensuring the development dependencies are kept up to date:
pip-compile && pip-sync
The demo project also needs the PiWind model. This is available here.
You should also set OASIS_MODEL_DATA_DIR
to the root directory of the pi wind repo.
Once the dependencies have been installed you will need to create the database. For development we use mysql for simplicity. To create the database run:
python manage.py migrate
This should also be ran whenever there are new database changes. If you change the database fields you will need to generate new migrations by running:
python manage.py makemigrations
And adding all generated files to git.
Once the database has been migrated the pi wind model needs to be added to the database, either in the admin interface or the api setting:
{
supplier_id: "OasisIM",
model_id: "PiWind",
version_id: "1",
}
The server can be run directly in python by ./manage.py runserver
, note that if the server has debug=false
set (the default value)
then the command ./manage.py collectstatic
must be executed first
To test the code style run::
flake8
To test against all supported python versions run::
tox
To test against your currently installed version of python run::
py.test
To run the full test suite run::
./runtests.sh
Due to different settings required to enable keycloak they need to be executed with a different django configuration:
OASIS_API_AUTH_TYPE=keycloak pytest src/server/oasisapi/auth/tests/test_oidc.py
The Oasis CI system builds and deploys the following Docker images to DockerHub:
Note that the Dockerfiles cannot be used directly as there are version stubs that get substitued at build time by the CI system.
By default the server uses the standard model backend, this started users with username and password in
the database. This can be configured by setting OASIS_API_SERVER_AUTH_BACKEND
to a comma separated
list of python paths. Information about django authentication backends can be found here
To create an admin user call python manage.py createsuperuser
and follow the prompts, this user
can be used as a normal user on in the api but it also gains access to the admin interface at
https://localhost:8000/admin/. From here you can edit entries and create new users.
For authenticating with the api the HTTP-AUTHORIZATION
header needs to be set. The token to use can
be obtained by posting your username and password to /refresh_token/
. This gives you both a refresh
token and access token. The access token should be used for most requests however this will expire after
a while. When it expires a new key can be retrieved by posting to /access_token/
using the refresh
token in the authorization header.
The authorization header takes the following form Bearer <token>
.
The general workflow is as follows
- Create a portfolio (post to
/portfolios/
). - Add a locations file to the portfolio (post to
/portfolios/<id>/locations_file/
) - Create the model object for your model (post to
/models/
). - Create an analysis (post to
/portfolios/<id>/create_analysis
). This will generate the input files for the analysis. - Add analysis settings file to the analysis (post to
/analyses/<pk>/analysis_settings/
). - Run the analysis (post to
/analyses/<pk>/run/
) - Get the outputs (get
/analuses/<pk>/output_file/
)
- Issues
- Releases
- General Oasis documentation
- Oasis API documentation
- Oasis Platform module documentation
The code in this project is licensed under BSD 3-clause license.