Skip to content

Commit

Permalink
Update docs for Hatch
Browse files Browse the repository at this point in the history
  • Loading branch information
br3ndonland committed Jan 6, 2023
1 parent e5fb5d6 commit 3e7e049
Show file tree
Hide file tree
Showing 4 changed files with 83 additions and 80 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ This project aims to:
- [x] **Replace the aging [python-dotenv](https://github.com/theskumar/python-dotenv) project** with a similar, but more intuitive API, and modern syntax and tooling.
- [x] **Implement asynchronous file I/O**. Reading and writing files can be done asynchronously with packages like [AnyIO](https://github.com/agronholm/anyio).
- [x] **Implement asynchronous object storage integration**. Dotenv files are commonly kept in cloud object storage, but environment variable management packages typically don't integrate with object storage clients. Additional logic is therefore required to download _.env_ files from object storage prior to loading environment variables. This project aims to integrate with S3-compatible object storage, with a focus on downloading and uploading file objects.
- [ ] **Read settings from TOML**. [It's all about _pyproject.toml_ now](https://snarky.ca/what-the-heck-is-pyproject-toml/). [Poetry](https://python-poetry.org/) has pushed [PEP 517](https://www.python.org/dev/peps/pep-0517/) build tooling and [PEP 518](https://www.python.org/dev/peps/pep-0518/) build requirements forward, and [even `setuptools` has come around](https://setuptools.readthedocs.io/en/latest/build_meta.html). [PEP 621](https://www.python.org/dev/peps/pep-0621/) defined how to store package metadata in _pyproject.toml_, and [PEP 631](https://www.python.org/dev/peps/pep-0631/) defined how to specify dependencies in _pyproject.toml_. Why don’t we use the metadata from our _pyproject.toml_ files in our Python applications?
- [ ] **Read settings from TOML**. [It's all about _pyproject.toml_ now](https://snarky.ca/what-the-heck-is-pyproject-toml/). The Python community has pushed [PEP 517](https://www.python.org/dev/peps/pep-0517/) build tooling and [PEP 518](https://www.python.org/dev/peps/pep-0518/) build requirements forward, and [even `setuptools` has come around](https://setuptools.readthedocs.io/en/latest/build_meta.html). [PEP 621](https://www.python.org/dev/peps/pep-0621/) defined how to store package metadata in _pyproject.toml_, and [PEP 631](https://www.python.org/dev/peps/pep-0631/) defined how to specify dependencies in _pyproject.toml_. Why don’t we use the metadata from our _pyproject.toml_ files in our Python applications?
- [ ] **Unify settings management for FastAPI**. [Uvicorn](https://www.uvicorn.org/), [Starlette](https://www.starlette.io/config/), and _[pydantic](https://pydantic-docs.helpmanual.io/usage/settings/)_ each have their own ways of loading environment variables and configuring application settings. This means that, when [configuring a FastAPI application](https://fastapi.tiangolo.com/advanced/settings/), there are at least three different settings management tools available, each with their own pros and cons. It would be helpful to address the limitations of each of these options, potentially providing a similar, improved API for each one.

The source code is 100% type-annotated and unit-tested.
Expand Down
2 changes: 1 addition & 1 deletion docs/cloud-object-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Dotenv files are commonly kept in [cloud object storage](https://en.wikipedia.or

### Set up a virtual environment

To get started, let's set up a virtual environment and install fastenv from the command line. If you've been through the [environment variable docs](environment.md#getting-started), the only change here is installing the optional extras with `poetry add fastenv -E httpx` or `python -m pip install fastenv[httpx]`.
To get started, let's set up a virtual environment and install fastenv from the command line. If you've been through the [environment variable docs](environment.md#getting-started), the only change here is installing the optional extras like `python -m pip install fastenv[httpx]`.

!!!example "Setting up a virtual environment"

Expand Down
157 changes: 80 additions & 77 deletions docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,55 @@
- Commit your changes with a [properly-formatted Git commit message](https://chris.beams.io/posts/git-commit/).
- Create a [pull request (PR)](https://docs.github.com/en/github/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests) to incorporate your changes into the upstream project you forked.

## Python

### Hatch

This project uses [Hatch](https://hatch.pypa.io/latest/) for dependency management and packaging.

#### Highlights

- **Automatic virtual environment management**: [Hatch automatically manages the application environment](https://hatch.pypa.io/latest/environment/).
- **Dependency resolution**: Hatch will automatically resolve any dependency version conflicts using the [`pip` dependency resolver](https://pip.pypa.io/en/stable/topics/dependency-resolution/).
- **Dependency separation**: [Hatch supports separate lists of optional dependencies in the _pyproject.toml_](https://hatch.pypa.io/latest/config/dependency/). Production installs can skip optional dependencies for speed.
- **Builds**: [Hatch has features for easily building the project into a Python package](https://hatch.pypa.io/latest/build/) and [publishing the package to PyPI](https://hatch.pypa.io/latest/publish/).

#### Installation

[Hatch can be installed with Homebrew or `pipx`](https://hatch.pypa.io/latest/install/).

**Install project with all dependencies: `hatch env create`**.

#### Key commands

```sh
# Basic usage: https://hatch.pypa.io/latest/cli/reference/
hatch env create # create virtual environment and install dependencies
hatch version # list or update version of this package
hatch shell # activate the virtual environment, like source venv/bin/activate
hatch run COMMAND # run a command within the virtual environment
hatch env find # show path to virtual environment
hatch env show # show info about available virtual environments
export HATCH_ENV_TYPE_VIRTUAL_PATH=.venv # install virtualenvs into .venv
```

### Testing with pytest

- Tests are in the _tests/_ directory.
- Run tests by [invoking `pytest` from the command-line](https://docs.pytest.org/en/latest/how-to/usage.html) within the virtual environment in the root directory of the repo.
- [pytest](https://docs.pytest.org/en/latest/) features used include:
- [capturing `stdout` with `capfd`](https://docs.pytest.org/en/latest/how-to/capture-stdout-stderr.html)
- [fixtures](https://docs.pytest.org/en/latest/how-to/fixtures.html)
- [monkeypatch](https://docs.pytest.org/en/latest/how-to/monkeypatch.html)
- [parametrize](https://docs.pytest.org/en/latest/how-to/parametrize.html)
- [temporary directories and files (`tmp_path` and `tmp_dir`)](https://docs.pytest.org/en/latest/how-to/tmpdir.html)
- [pytest plugins](https://docs.pytest.org/en/latest/how-to/plugins.html) include:
- [pytest-mock](https://github.com/pytest-dev/pytest-mock)
- [pytest configuration](https://docs.pytest.org/en/latest/reference/customize.html) is in _[pyproject.toml](https://github.com/br3ndonland/fastenv/blob/develop/pyproject.toml)_.
- Test coverage reports are generated by [coverage.py](https://github.com/nedbat/coveragepy). To generate test coverage reports, first run tests with `coverage run`, then generate a report with `coverage report`. To see interactive HTML coverage reports, run `coverage html` instead of `coverage report`.

Integration tests will be skipped if cloud credentials are not present. Running integration tests locally will take some additional setup.

## Code quality

### Code style
Expand All @@ -30,7 +79,7 @@
2. Dependencies
3. Project
- Within each of those groups, `import` statements occur first, then `from` statements, in alphabetical order.
- You can run isort from the command line with `poetry run isort .`.
- You can run isort from the command line with `hatch run isort .`.
- Configuration for isort is stored in _[pyproject.toml](https://github.com/br3ndonland/fastenv/blob/HEAD/pyproject.toml)_.
- Other web code (JSON, Markdown, YAML) is formatted with [Prettier](https://prettier.io/).

Expand All @@ -43,6 +92,29 @@
- [Mypy](https://mypy.readthedocs.io/en/stable/) is used for type-checking. [Mypy configuration](https://mypy.readthedocs.io/en/stable/config_file.html) is included in _pyproject.toml_.
- Mypy strict mode is enabled. Strict includes `--no-explicit-reexport` (`implicit_reexport = false`), which means that objects imported into a module will not be re-exported for import into other modules. Imports can be made into explicit exports with the syntax `from module import x as x` (i.e., changing from `import logging` to `import logging as logging`), or by including imports in `__all__`. This explicit import syntax can be confusing. Another option is to apply mypy overrides to any modules that need to leverage implicit exports.

### Pre-commit

[Pre-commit](https://pre-commit.com/) runs [Git hooks](https://www.git-scm.com/book/en/v2/Customizing-Git-Git-Hooks). Configuration is stored in _[.pre-commit-config.yaml](https://github.com/br3ndonland/fastenv/blob/HEAD/.pre-commit-config.yaml)_. It can run locally before each commit (hence "pre-commit"), or on different Git events like `pre-push`. Pre-commit is installed in the Python virtual environment. To use:

```sh
~
cd path/to/fastenv

path/to/fastenv
❯ hatch env create

path/to/fastenv
❯ hatch shell

# install hooks that run before each commit
path/to/fastenv
.venv ❯ pre-commit install

# and/or install hooks that run before each push
path/to/fastenv
.venv ❯ pre-commit install --hook-type pre-push
```

### Spell check

Spell check is performed with [CSpell](https://cspell.org/).
Expand Down Expand Up @@ -80,79 +152,6 @@ CSpell is not currently used with pre-commit in this project because behavior of
- [CSpell matches files with globs](https://cspell.org/docs/globs/), but [pre-commit uses Python regexes](https://pre-commit.com/#regular-expressions). Two separate file patterns have to be specified (a regex for pre-commit and a glob for CSpell), which is awkward and error-prone.
- When running with pre-commit, CSpell seems to output each error multiple times.

### Pre-commit

[Pre-commit](https://pre-commit.com/) runs [Git hooks](https://www.git-scm.com/book/en/v2/Customizing-Git-Git-Hooks). Configuration is stored in _[.pre-commit-config.yaml](https://github.com/br3ndonland/fastenv/blob/HEAD/.pre-commit-config.yaml)_. It can run locally before each commit (hence "pre-commit"), or on different Git events like `pre-push`. Pre-commit is installed in the Poetry environment. To use:

```sh
# after running `poetry install`
path/to/fastenv
❯ poetry shell

# install hooks that run before each commit
path/to/fastenv
.venv ❯ pre-commit install

# and/or install hooks that run before each push
path/to/fastenv
.venv ❯ pre-commit install --hook-type pre-push
```

## Python

### Poetry

This project uses [Poetry](https://python-poetry.org/) for dependency management.

**Install project with all dependencies: `poetry install -E all`**.

#### Highlights

- **Automatic virtual environment management**: Poetry automatically manages the `virtualenv` for the application.
- **Automatic dependency management**: rather than having to run `pip freeze > requirements.txt`, Poetry automatically manages the dependency file (called _pyproject.toml_), and enables SemVer-level control over dependencies like [npm](https://semver.npmjs.com/). Poetry also manages a lockfile (called _poetry.lock_), which is similar to _package-lock.json_ for npm. Poetry uses this lockfile to automatically track specific versions and hashes for every dependency.
- **Dependency resolution**: Poetry will automatically resolve any dependency version conflicts. pip did not have dependency resolution [until the end of 2020](https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020).
- **Dependency separation**: Poetry can maintain separate lists of dependencies for development and production in the _pyproject.toml_. Production installs can skip development dependencies for speed.
- **Builds**: Poetry has features for easily building the project into a Python package.

#### Installation

The recommended installation method is through the [Poetry custom installer](https://python-poetry.org/docs/#installation), which vendorizes dependencies into an isolated environment, and allows you to update Poetry with `poetry self update`.

You can also install Poetry however you prefer to install your user Python packages (`pipx install poetry`, `pip install --user poetry`, etc). Use the standard update methods with these tools (`pipx upgrade poetry`, `pip install --user --upgrade poetry`, etc).

#### Key commands

```sh
# Basic usage: https://python-poetry.org/docs/basic-usage/
poetry install # create virtual environment and install dependencies
poetry show --tree # list installed packages
poetry add PACKAGE@VERSION # add package production dependencies
poetry add PACKAGE@VERSION --dev # add package to development dependencies
poetry update # update dependencies (not available with standard tools)
poetry version # list or update version of this package
poetry shell # activate the virtual environment, like source venv/bin/activate
poetry run COMMAND # run a command within the virtual environment
poetry env info # https://python-poetry.org/docs/managing-environments/
poetry config virtualenvs.in-project true # install virtualenvs into .venv
poetry export -f requirements.txt > requirements.txt --dev # export deps
```

### Testing with pytest

- Tests are in the _tests/_ directory.
- Run tests by [invoking `pytest` from the command-line](https://docs.pytest.org/en/latest/how-to/usage.html) within the Poetry environment in the root directory of the repo.
- [pytest](https://docs.pytest.org/en/latest/) features used include:
- [fixtures](https://docs.pytest.org/en/latest/how-to/fixtures.html)
- [monkeypatch](https://docs.pytest.org/en/latest/how-to/monkeypatch.html)
- [parametrize](https://docs.pytest.org/en/latest/how-to/parametrize.html)
- [temporary directories and files (`tmp_path`)](https://docs.pytest.org/en/latest/how-to/tmp_path.html)
- [pytest plugins](https://docs.pytest.org/en/latest/how-to/plugins.html) include:
- [pytest-mock](https://github.com/pytest-dev/pytest-mock)
- [pytest configuration](https://docs.pytest.org/en/latest/reference/customize.html) is in _[pyproject.toml](https://github.com/br3ndonland/fastenv/blob/HEAD/pyproject.toml)_.
- Test coverage reports are generated by [coverage.py](https://github.com/nedbat/coveragepy). To generate test coverage reports, first run tests with `coverage run`, then generate a report with `coverage report`. To see interactive HTML coverage reports, run `coverage html` instead of `coverage report`.

Integration tests will be skipped if cloud credentials are not present. Running integration tests locally will take some additional setup.

#### Make buckets on each supported cloud platform

- [AWS S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html)
Expand Down Expand Up @@ -319,11 +318,14 @@ See the [Backblaze B2 S3-compatible API docs](https://www.backblaze.com/b2/docs/
- Do not allow force pushes
- Require status checks to pass before merging (commits must have previously been pushed to `develop` and passed all checks)
- **To create a release:**
- Bump the version number in `pyproject.toml` with `poetry version` and commit the changes to `develop`.
- Bump the version number in `fastenv.__version__` with `hatch version` and commit the changes to `develop`.
- Follow [SemVer](https://semver.org/) guidelines when choosing a version number. Note that [PEP 440](https://peps.python.org/pep-0440/) Python version specifiers and SemVer version specifiers differ, particularly with regard to specifying prereleases. Use syntax compatible with both.
- The PEP 440 default (like `1.0.0a0`) is different from SemVer. Hatch and PyPI will use this syntax by default.
- An alternative form of the Python prerelease syntax permitted in PEP 440 (like `1.0.0-alpha.0`) is compatible with SemVer, and this form should be used when tagging releases. As Hatch uses PEP 440 syntax by default, prerelease versions need to be written directly into `fastenv.__version__`.
- Examples of acceptable tag names: `1.0.0`, `1.0.0-alpha.0`, `1.0.0-beta.1`
- Push to `develop` and verify all CI checks pass.
- Fast-forward merge to `main`, push, and verify all CI checks pass.
- Create an [annotated and signed Git tag](https://www.git-scm.com/book/en/v2/Git-Basics-Tagging)
- Follow [SemVer](https://semver.org/) guidelines when choosing a version number.
- Create an [annotated and signed Git tag](https://www.git-scm.com/book/en/v2/Git-Basics-Tagging).
- List PRs and commits in the tag message:
```sh
git log --pretty=format:"- %s (%h)" \
Expand All @@ -332,3 +334,4 @@ See the [Backblaze B2 S3-compatible API docs](https://www.backblaze.com/b2/docs/
- Omit the leading `v` (use `1.0.0` instead of `v1.0.0`)
- Example: `git tag -a -s 1.0.0`
- Push the tag. GitHub Actions will build and publish the Python package and update the changelog.
- Squash and merge the changelog PR, removing any [`Co-authored-by` trailers](https://docs.github.com/en/pull-requests/committing-changes-to-your-project/creating-and-editing-commits/creating-a-commit-with-multiple-authors) before merging.
Loading

0 comments on commit 3e7e049

Please sign in to comment.