Enable fastenv to manage environment variables and .env files

.github/workflows/ci.yml

@@ -44,14 +44,14 @@ jobs:
  python install-poetry.py -y
  echo "$POETRY_HOME/bin" >> $GITHUB_PATH
  - name: Install dependencies
- run: poetry install --no-interaction
+ run: poetry install --no-interaction -E all
  - name: Run pre-commit hooks
  run: poetry run pre-commit run --all-files
  - name: Run unit tests
  run: poetry run pytest --cov-report=xml
  - name: Upload test coverage report to Codecov
  uses: codecov/codecov-action@v1
- if: env.CODECOV_UPLOAD == 'true'
+ if: env.CODECOV_UPLOAD == 'true' && matrix.python-version == 3.9
  with:
  fail_ci_if_error: true
  flags: unit

README.md

@@ -9,16 +9,14 @@ _Unified environment variable and settings management for FastAPI and beyond_
 [![ci](https://github.com/br3ndonland/fastenv/workflows/ci/badge.svg)](https://github.com/br3ndonland/fastenv/actions/workflows/ci.yml)
 [![codecov](https://codecov.io/gh/br3ndonland/fastenv/branch/main/graph/badge.svg?token=WDNHES5OYR)](https://codecov.io/gh/br3ndonland/fastenv)
 
-🚧 UNDER CONSTRUCTION - NOT FOR PRODUCTION 🚧
-
 ## Description
 
 [Environment variables](https://en.wikipedia.org/wiki/Environment_variable) are key-value pairs provided to the operating system with syntax like `VARIABLE_NAME=value`. Collections of environment variables are stored in files commonly named _.env_ and called "dotenv" files. The Python standard library `os` module provides tools for reading environment variables, such as `os.getenv("VARIABLE_NAME")`, but only handles strings, and doesn't include tools for file I/O. Additional logic is therefore needed to load environment variables from files before they can be read by Python, and to convert variables from strings to other Python types.
 
 This project aims to:
 
-- [ ] **Replace the aging [python-dotenv](https://github.com/theskumar/python-dotenv) project** with a similar, but more intuitive API, and modern syntax and tooling.
-- [ ] **Implement asynchronous file I/O**. Reading and writing files can be done asynchronously with packages like [AnyIO](https://github.com/agronholm/anyio).
+- [x] **Replace the aging [python-dotenv](https://github.com/theskumar/python-dotenv) project** with a similar, but more intuitive API, and modern syntax and tooling.
+- [x] **Implement asynchronous file I/O**. Reading and writing files can be done asynchronously with packages like [AnyIO](https://github.com/agronholm/anyio).
 - [ ] **Implement asynchronous object storage integration**. Dotenv files are commonly kept in object storage like AWS S3, but environment variable management packages typically don't integrate with object storage clients. Additional logic is therefore required to download _.env_ files from object storage prior to loading the variables. This project aims to integrate with S3-compatible object storage so that, in addition to accepting local file paths, like `load_dotenv("/path/to/my/.env")`, fastenv would also accept object storage URIs, like `load_dotenv("s3:https://mybucket/.env")`. fastenv would then download the object and load its environment variables. Downloading file objects can be done asynchronously with packages like [aioaws](https://github.com/samuelcolvin/aioaws).
 - [ ] **Read settings from TOML**. [It's all about _pyproject.toml_ now](https://snarky.ca/what-the-heck-is-pyproject-toml/). [Poetry](https://python-poetry.org/) has pushed [PEP 517](https://www.python.org/dev/peps/pep-0517/) build tooling and [PEP 518](https://www.python.org/dev/peps/pep-0518/) build requirements forward, and [even `setuptools` has come around](https://setuptools.readthedocs.io/en/latest/build_meta.html). Why don’t we use the metadata from our _pyproject.toml_ files in our Python APIs?
 - [ ] **Unify settings management for FastAPI**. [Uvicorn](https://www.uvicorn.org/), [Starlette](https://www.starlette.io/config/), and _[pydantic](https://pydantic-docs.helpmanual.io/usage/settings/)_ each have their own ways of loading environment variables and configuring application settings. This means that, when [configuring a FastAPI application](https://fastapi.tiangolo.com/advanced/settings/), there are at least three different settings management tools available, each with their own pros and cons. It would be helpful to address the limitations of each of these options, potentially providing a similar, improved API for each one.

docs/comparisons.md

@@ -65,13 +65,102 @@ _pydantic_ offers a [`BaseSettings` model](https://pydantic-docs.helpmanual.io/u
 
 ## python-dotenv
 
-- [python-dotenv](https://github.com/theskumar/python-dotenv) is a package for loading _.env_ files and setting environment variables. It was [started](https://github.com/theskumar/python-dotenv/commit/5fc02b7303e8854243970e12564f2433da7a1f7f) by Django co-creator Jacob Kaplan-Moss in 2013, and was originally called django-dotenv. It is used by [Uvicorn](https://www.uvicorn.org/) and _[pydantic](https://pydantic-docs.helpmanual.io/usage/settings/)_, and suggested in the [FastAPI docs](https://fastapi.tiangolo.com/advanced/settings/).
-- Its primary data structure, `dotenv.main.DotEnv`, inherits from `object`. As a result, it requires its own mapping mathods (`dict`, `get_key`, `set_key`, `unset_key`) that could be easily eliminated by inheriting from a mapping data structure such as `collections.abc.MutableMapping`.
-- Other methods have confusing, counter-intuitive APIs. For example, the `load_dotenv()` function is supposed to "Parse a _.env_ file and then load all the variables found as environment variables," according to its docstring. However, the function always returns `True`, even if no _.env_ file is found or no environment variables are set, because of `DotEnv.set_as_environment_variables()`. Furthermore, this confusing behavior is not documented, because, as the maintainer [commented](https://github.com/theskumar/python-dotenv/issues/164#issuecomment-494750043), "The return value of `load_dotenv` is undocumented as I was planning to do something useful with it, but could not settle down to one."
-- Loads files with the synchronous `open()` built-in function. Asyncio support is not provided.
+[python-dotenv](https://github.com/theskumar/python-dotenv) is a package for loading _.env_ files and setting environment variables. It was [started](https://github.com/theskumar/python-dotenv/commit/5fc02b7303e8854243970e12564f2433da7a1f7f) by Django co-creator Jacob Kaplan-Moss in 2013, and was originally called django-dotenv. It is used by [Uvicorn](https://www.uvicorn.org/) and _[pydantic](https://pydantic-docs.helpmanual.io/usage/settings/)_, and suggested in the [FastAPI docs](https://fastapi.tiangolo.com/advanced/settings/).
+
+### Environment variables
+
+- Its primary data structure, `dotenv.main.DotEnv`, inherits from `object`. As a result, it requires its own mapping methods (such as `dict()`) that could be obviated by inheriting from a mapping data structure such as `collections.abc.MutableMapping`.
+- Other methods have confusing, counter-intuitive APIs. For example, the `load_dotenv()` function is supposed to "Parse a .env file and then load all the variables found as environment variables," according to its docstring. However, the function always returns `True`, even if no _.env_ file is found or no environment variables are set, because of `DotEnv.set_as_environment_variables()`. Furthermore, this confusing behavior is not documented, because, as the maintainer [commented](https://github.com/theskumar/python-dotenv/issues/164#issuecomment-494750043), "The return value of `load_dotenv` is undocumented as I was planning to do something useful with it, but could not settle down to one."
+
+### File I/O
+
+- Loads files with the synchronous `open()` built-in function. Async support is not provided.
 - Does not integrate with object storage like AWS S3.
-- Maintainers have not been receptive to improvements. See [theskumar/python-dotenv#263](https://github.com/theskumar/python-dotenv/pull/263) for context.
-- Continues supporting Python 2 after its [end-of-life](https://www.python.org/doc/sunset-python-2/), so it has to use [Python 2 type comments](https://mypy.readthedocs.io/en/stable/python2.html) and other legacy cruft.
+
+### Project maintenance
+
+- Continued supporting Python 2 after its [end-of-life](https://www.python.org/doc/sunset-python-2/) (until 0.19.0), so it had to use [Python 2 type comments](https://mypy.readthedocs.io/en/stable/python2.html) and other legacy cruft.
+- Maintainers have not been receptive to improvements (see [theskumar/python-dotenv#263](https://github.com/theskumar/python-dotenv/pull/263) for context).
+
+### Comparing fastenv and python-dotenv
+
+#### `DotEnv`
+
+- Both fastenv and python-dotenv provide a `DotEnv` class for managing environment variables
+- `fastenv.DotEnv` inherits from `collections.abc.MutableMapping`, `dotenv.main.DotEnv` inherits from `object`
+- fastenv includes `DotEnv` in its `__all__`, python-dotenv does not (it must be directly imported from `dotenv.main`)
+
+#### `find_dotenv`
+
+- fastenv: `await fastenv.find_dotenv()` (async)
+- python-dotenv: `dotenv.find_dotenv()` (sync)
+- Both fastenv and python-dotenv look for `".env"` by default
+- Both python-dotenv and fastenv return `os.PathLike` objects
+- fastenv raises `FileNotFoundError` exceptions by default if files are not found, python-dotenv does not
+
+#### `load_dotenv`
+
+- fastenv: `await fastenv.load_dotenv()` (async)
+- python-dotenv: `dotenv.load_dotenv()` (sync)
+- `fastenv.load_dotenv` logs the number of environment variables loaded, `dotenv.load_dotenv` does not
+- `fastenv.load_dotenv` returns a `DotEnv` model, `dotenv.load_dotenv` returns `True` (even if no _.env_ file was found and no environment variables were loaded)
+
+#### `find_dotenv` with `load_dotenv`
+
+Users who would like to ensure their _.env_ files are found, and log the result, should be aware that `dotenv.load_dotenv`:
+
+- Only calls `find_dotenv` if a file path is not provided, and does not pass an argument through to `find_dotenv` to raise exceptions if the file is not found
+- Requires a call to `DotEnv.set_as_environment_variables` to actually set environment variables
+- Does not provide logging
+- Does not provide exception handling (its `verbose` argument does not necessarily raise an exception)
+- Does not return the `DotEnv` instance created by `load_dotenv`, but always returns `True`, even if no _.env_ file is found or no environment variables are set
+
+Something like the following is therefore needed instead of using `dotenv.load_dotenv`:
+
+!!!example "Finding and loading a _.env_ file with python-dotenv"
+
+ ```py
+ import logging
+
+ from dotenv import find_dotenv
+ from dotenv.main import DotEnv
+
+
+ def find_and_load_my_dotenv(env_file: str = ".env") -> DotEnv:
+ try:
+ logger = logging.getLogger()
+ source = find_dotenv(filename=env_file, raise_error_if_not_found=True)
+ dotenv = DotEnv(source, verbose=True)
+ dotenv.set_as_environment_variables()
+ logger.info(
+ f"Python-dotenv loaded {len(dotenv.dict())} variables from {env_file}"
+ )
+ return dotenv
+ except Exception as e:
+ logger.error(f"Error loading {env_file}: {e.__class__.__qualname__} {e}")
+ raise
+ ```
+
+The above effect can be accomplished with fastenv in a single call, `await fastenv.load_dotenv(find_source=True)`. This call to `fastenv.load_dotenv`:
+
+- Finds the _.env_ file (`find_source=True`) with its `find_dotenv` method and the file name provided (`".env"` by default), logging and raising a `FileNotFoundError` if not found
+- Sets environment variables automatically
+- Logs successes and errors automatically
+- Raises exceptions by default
+- Returns a `DotEnv` instance
+
+#### `dotenv_values`
+
+- fastenv: `await fastenv.dotenv_values()` (async)
+- python-dotenv: `dotenv.dotenv_values()` (sync)
+- `fastenv.dotenv_values` offers a `find_dotenv` argument to find files before loading and returning values, `dotenv.dotenv_values` does not
+- `fastenv.dotenv_values` offers a `raise_exceptions` argument to determine whether or not exceptions will be raised, `dotenv.dotenv_values` does not (its `verbose` argument does not necessarily raise an exception)
+- `fastenv.dotenv_values` logs successes and errors automatically, `dotenv.dotenv_values` does not
+
+#### Writing to _.env_ files
+
+- fastenv: `await fastenv.dump_dotenv()` (async, and writes an entire `DotEnv` model to a file)
+- python-dotenv: `dotenv.get_key()`, `dotenv.set_key()`, `dotenv.unset_key()` (sync, and can only write single variables to a file)
 
 ## Starlette