This is meant for personal use for my own projects.
There are 4 different pipeline templates, one for testing, one for building docs, one for releasing the library to PyPI, and one for approving pull requests by certain authors automatically.
All templates require the project to use poetry. The testing pipeline requires tox and coverage, as well as Coverage results are sent to coveralls. The docs building pipeline requires mkdocs.
Here is a minimal pyproject.toml
setup to get started:
# ...
[tool.poetry.group.test.dependencies]
pytest = "..." # use latest version
coverage = "..." # use latest version
tox = "..." # use latest version
tox-gh-actions = "..." # use latest version
# 'distutils' is not included in virtual environments from Python 3.12 onwards,
# so you might need to explcitly include it via setuptools.
setuptools = {version = "...", python = ">=3.12"} # use latest version
# This is only needed for the docs CI
[tool.poetry.group.docs.dependencies]
mkdocs = "..." # use latest version
[tool.coverage.run]
relative_files = true
[tool.coverage.report]
omit = [
"tests/*",
".tox/*",
]
[tool.tox]
legacy_tox_ini = """
[tox]
envlist = py{39, 310, 311, 312}
isolated_build = true
[gh-actions]
python =
3.9: py39
3.10: py310
3.11: py311
3.12: py312
[testenv]
allowlist_externals =
poetry
setenv =
PYTHONPATH = {toxinidir}
commands =
poetry install
poetry run coverage run -m pytest
"""
[build-system]
requires = ["poetry-core>=1.8.1"]
build-backend = "poetry.core.masonry.api"
This pipeline uses a job stategy matrix to run tests in a number of python environments and operating systems in parallel. All dependencies are cached for each os and environment resulting from the strategy to ensure the CI runs fast when dependencies are not updated.
To set up the pipeline, add a yml
file to ./.github/workflows/
with the following job configuration.
name: Tests
on:
push:
branches:
- main
paths:
- "**.py"
- "pyproject.toml"
- "poetry.lock"
pull_request:
workflow_dispatch:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
This job can take a number of inputs via the with-keyword.
Configure the pyhton versions the test will be run with.
The tox environments used are configured with the
[gh-actions]
setting in pyproject.toml
.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
python-version: '["3.9", "3.10", "3.11"]'
Configure the operating systems the tests will be run with.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
os: '["ubuntu-latest", "macos-latest", "windows-latest"]'
Configure the poetry version used in the pipeline.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
poetry-version: "1.7.1"
GitHub job stategy matrix exclusion pattern, in JSON form. Using yaml flow style, a list of dicts can be conveted into multiple exclusions if necessary.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
exclude: '[{"os": "none", "python-version": "none"}]' # this ignores nothing
Should submodules be checked out with the repository?
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
submodules: false
How many commit to fetch from the branch history? If set to 0, every commit is fetched.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
fetch-depth: 1
Should coverage results be submitted to coveralls? Requires coveralls setup to already exist.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
coveralls: true
This pipeline can be used to build and push the docs used for
mkdocs from the docs/
directory into GitHub pages.
Note that mkdocs also requires a separate configuration file where the pages are set up. Here's a minimal example configuration.
site_name: {{ Site name here }} nav: - Home: index.md # ./docs/index.md
To set up the pipeline, add a yml
file to ./.github/workflows/
with the following job configuration.
name: Docs
on:
push:
branches:
- main
paths:
- "docs/**"
- "mkdocs.yml"
workflow_dispatch:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
This job can take a number of inputs via the with-keyword.
Configure the poetry version used in the pipeline.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
poetry-version: "1.7.1"
Configure the python version used in the pipeline.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
python-version: "3.11"
Configure the operating system used in the pipeline.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
os: "ubuntu-latest"
Should submodules be checked out with the repository?
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
submodules: false
How many commit to fetch from the branch history? If set to 0, every commit is fetched.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
fetch-depth: 1
This pipeline can be used to build and release the library to PyPI with poetry using a PyPI token stored in the repository's actions secrets.
Note that the poetry version configuration needs to be updated and match the tag created for the release or this job will fail (can include v-prefix, e.g.,
v0.0.1
).
To set up the pipeline, add a yml
file to ./.github/workflows/
with the following job configuration. The pypi-token
input is required.
name: Release
on:
release:
types:
- released
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
secrets:
pypi-token: ${{ secrets.PYPI_API_TOKEN }}
Replace
PYPI_API_TOKEN
with the secret name of your choice
This job can take a number of inputs via the with-keyword.
Configure the poetry version used in the pipeline.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
poetry-version: "1.7.1"
Configure the python version used in the pipeline.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
python-version: "3.11"
Configure the operating system used in the pipeline.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
os: "ubuntu-latest"
Should submodules be checked out with the repository?
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
submodules: false
How many commit to fetch from the branch history? If set to 0, every commit is fetched.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
fetch-depth: 1
This pipeline can be used to automatically approve pull request by some users. By default, it is set to approve pull requests by the dependabot and pre-commit.ci bots.
To set up the pipeline, add a yml
file to ./.github/workflows/
with the following job configuration.
name: Auto approve PRs
on:
pull_request_target:
jobs:
approve:
permissions:
pull-requests: write
contents: write
uses: MrThearMan/CI/.github/workflows/[email protected]
This job can take a number of inputs via the with-keyword.
Configure the users whose pull requests can be automatically approved.
Default configuration:
jobs:
approve:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
users: '["dependabot[bot]", "pre-commit-ci[bot]"]'
Should submodules be checked out with the repository?
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
submodules: false
How many commit to fetch from the branch history? If set to 0, every commit is fetched.
Default configuration:
jobs:
test:
uses: MrThearMan/CI/.github/workflows/[email protected]
with:
fetch-depth: 1
Some of the pipeline templates have hooks that can be defined
to run additional setup and postprocessing steps if necessary.
To enable them, simply add a yaml
file to the appropriate directory
in your project and write a composite action. Here is a template for one.
runs:
using: composite
steps:
- name: "Setup"
shell: bash
run: ...
Here is a little trick you can do to pass the inputs from the testing job to the composite action when needed
inputs: python-version: default: ${{ matrix.python-version }}
For the testing pipeline, the hooks are:
Pre-test
: Add the file.github/actions/pre-test/action.yml
Post-test
: Add the file.github/actions/post-test/action.yml
For the release pipeline, the hooks are:
Pre-release
: Add the file.github/actions/pre-release/action.yml
Post-release
: Add the file.github/actions/post-release/action.yml
Installs poetry to the current python version, e.g., if used after
actions/setup-python
, poetry is installed with that python version.
jobs:
<foo>:
steps:
- ...
- uses: MrThearMan/CI/.github/actions/[email protected]
with:
os: "ubuntu-latest"
poetry-version: "1.7.1"
However, actions/setup-python
poetry caching cannot be used if poetry is not installed.
In this case, a custom cache must be created:
jobs:
<foo>:
steps:
- ...
- name: "Load cached poetry environment"
uses: actions/cache@v4
with:
path: .venv
key: <unique-key-per-env>
Not tested yet.
Can be used to check if certain filetypes were changed in a pull request.
jobs:
<foo>:
steps:
- uses: MrThearMan/CI/.github/actions/[email protected]
id: changed
with:
filetypes: "py|yaml"
- if: ${{ changed.changed-filetypes }}