This is meant for personal use for my own projects.

There are 4 different pipeline templates, one for testing, one for building docs, one for releasing the library to PyPI, and one for approving pull requests by certain authors automatically.

All templates require the project to use poetry. The testing pipeline requires tox and coverage, as well as Coverage results are sent to coveralls. The docs building pipeline requires mkdocs.

Here is a minimal pyproject.toml setup to get started:

# ...

[tool.poetry.group.test.dependencies]
pytest = "..."  # use latest version
coverage = "..."  # use latest version
tox = "..."  # use latest version
tox-gh-actions = "..."  # use latest version
# 'distutils' is not included in virtual environments from Python 3.12 onwards,
# so you might need to explcitly include it via setuptools.
setuptools = {version = "...", python = ">=3.12"}  # use latest version

# This is only needed for the docs CI
[tool.poetry.group.docs.dependencies]
mkdocs = "..."  # use latest version

[tool.coverage.run]
relative_files = true

[tool.coverage.report]
omit = [
    "tests/*",
    ".tox/*",
]

[tool.tox]
legacy_tox_ini = """
[tox]
envlist = py{39, 310, 311, 312}
isolated_build = true

[gh-actions]
python =
    3.9: py39
    3.10: py310
    3.11: py311
    3.12: py312

[testenv]
allowlist_externals =
    poetry
setenv =
    PYTHONPATH = {toxinidir}
commands =
    poetry install
    poetry run coverage run -m pytest
"""

[build-system]
requires = ["poetry-core>=1.8.1"]
build-backend = "poetry.core.masonry.api"

This pipeline uses a job stategy matrix to run tests in a number of python environments and operating systems in parallel. All dependencies are cached for each os and environment resulting from the strategy to ensure the CI runs fast when dependencies are not updated.

To set up the pipeline, add a yml file to ./.github/workflows/ with the following job configuration.

name: Tests

on:
  push:
    branches:
      - main
    paths:
      - "**.py"
      - "pyproject.toml"
      - "poetry.lock"
  pull_request:
  workflow_dispatch:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]

This job can take a number of inputs via the with-keyword.

Configure the pyhton versions the test will be run with. The tox environments used are configured with the [gh-actions] setting in pyproject.toml.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      python-version: '["3.9", "3.10", "3.11"]'

Configure the operating systems the tests will be run with.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      os: '["ubuntu-latest", "macos-latest", "windows-latest"]'

Configure the poetry version used in the pipeline.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      poetry-version: "1.7.1"

GitHub job stategy matrix exclusion pattern, in JSON form. Using yaml flow style, a list of dicts can be conveted into multiple exclusions if necessary.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      exclude: '[{"os": "none", "python-version": "none"}]'  # this ignores nothing

Should submodules be checked out with the repository?

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      submodules: false

How many commit to fetch from the branch history? If set to 0, every commit is fetched.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      fetch-depth: 1

Should coverage results be submitted to coveralls? Requires coveralls setup to already exist.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      coveralls: true

This pipeline can be used to build and push the docs used for mkdocs from the docs/ directory into GitHub pages.

Note that mkdocs also requires a separate configuration file where the pages are set up. Here's a minimal example configuration.

site_name: {{ Site name here }}

nav:
  - Home: index.md  # ./docs/index.md

To set up the pipeline, add a yml file to ./.github/workflows/ with the following job configuration.

name: Docs

on:
  push:
    branches:
      - main
    paths:
      - "docs/**"
      - "mkdocs.yml"
  workflow_dispatch:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]

This job can take a number of inputs via the with-keyword.

Configure the poetry version used in the pipeline.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      poetry-version: "1.7.1"

Configure the python version used in the pipeline.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      python-version: "3.11"

Configure the operating system used in the pipeline.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      os: "ubuntu-latest"

Should submodules be checked out with the repository?

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      submodules: false

How many commit to fetch from the branch history? If set to 0, every commit is fetched.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      fetch-depth: 1

This pipeline can be used to build and release the library to PyPI with poetry using a PyPI token stored in the repository's actions secrets.

Note that the poetry version configuration needs to be updated and match the tag created for the release or this job will fail (can include v-prefix, e.g., v0.0.1).

To set up the pipeline, add a yml file to ./.github/workflows/ with the following job configuration. The pypi-token input is required.

name: Release

on:
  release:
    types:
      - released

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    secrets:
      pypi-token: ${{ secrets.PYPI_API_TOKEN }}

Replace PYPI_API_TOKEN with the secret name of your choice

This job can take a number of inputs via the with-keyword.

Configure the poetry version used in the pipeline.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      poetry-version: "1.7.1"

Configure the python version used in the pipeline.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      python-version: "3.11"

Configure the operating system used in the pipeline.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      os: "ubuntu-latest"

Should submodules be checked out with the repository?

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      submodules: false

How many commit to fetch from the branch history? If set to 0, every commit is fetched.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      fetch-depth: 1

This pipeline can be used to automatically approve pull request by some users. By default, it is set to approve pull requests by the dependabot and pre-commit.ci bots.

To set up the pipeline, add a yml file to ./.github/workflows/ with the following job configuration.

name: Auto approve PRs

on: 
  pull_request_target:

jobs:
  approve:
    permissions:
      pull-requests: write
      contents: write
    uses: MrThearMan/CI/.github/workflows/[email protected]

This job can take a number of inputs via the with-keyword.

Configure the users whose pull requests can be automatically approved.

Default configuration:

jobs:
  approve:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      users: '["dependabot[bot]", "pre-commit-ci[bot]"]'

Should submodules be checked out with the repository?

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      submodules: false

How many commit to fetch from the branch history? If set to 0, every commit is fetched.

Default configuration:

jobs:
  test:
    uses: MrThearMan/CI/.github/workflows/[email protected]
    with:
      fetch-depth: 1

Some of the pipeline templates have hooks that can be defined to run additional setup and postprocessing steps if necessary. To enable them, simply add a yaml file to the appropriate directory in your project and write a composite action. Here is a template for one.

runs:
  using: composite
  
  steps:
    - name: "Setup"
      shell: bash
      run: ...

Here is a little trick you can do to pass the inputs from the testing job to the composite action when needed

inputs:
  python-version:
    default: ${{ matrix.python-version }}

For the testing pipeline, the hooks are:

• Pre-test: Add the file .github/actions/pre-test/action.yml
• Post-test: Add the file .github/actions/post-test/action.yml

For the release pipeline, the hooks are:

• Pre-release: Add the file .github/actions/pre-release/action.yml
• Post-release: Add the file .github/actions/post-release/action.yml

Installs poetry to the current python version, e.g., if used after actions/setup-python, poetry is installed with that python version.

jobs:
  <foo>:
    steps:
      - ...
      - uses: MrThearMan/CI/.github/actions/[email protected]
        with:
          os: "ubuntu-latest"
          poetry-version: "1.7.1"

However, actions/setup-python poetry caching cannot be used if poetry is not installed. In this case, a custom cache must be created:

jobs:
  <foo>:
    steps:
      - ...
      - name: "Load cached poetry environment"
        uses: actions/cache@v4
        with:
          path: .venv
          key: <unique-key-per-env>

Not tested yet.

Can be used to check if certain filetypes were changed in a pull request.

jobs:
  <foo>:
    steps:
      - uses: MrThearMan/CI/.github/actions/[email protected]
        id: changed
        with:
          filetypes: "py|yaml"
      - if: ${{ changed.changed-filetypes }}