Capsula, a Latin word meaning box, is a Python package designed to help researchers and developers easily capture their command/function execution context for reproducibility. See the documentation for more information.
With Capsula, you can capture:
- CPU information with
CpuContext - Python version with
PlatformContext - Current working directory with
CwdContext - Git repository information (commit hash, branch, etc.) with
GitRepositoryContext - Output of shell commands (e.g.,
poetry check --lock) withCommandContext - Files (e.g., output files,
pyproject.toml,requirements.txt) withFileContext - Arguments of Python functions with
FunctionContext - Environment variables with
EnvVarContext - Uncaught exceptions with
UncaughtExceptionWatcher - Execution time with
TimeWatcher
The captured contexts are dumped into JSON files for future reference and reproduction.
For project-wide settings, prepare a capsula.toml file in the root directory of your project. An example of the capsula.toml file is as follows:
[pre-run]
contexts = [
{ type = "CwdContext" },
{ type = "CpuContext" },
{ type = "PlatformContext" },
{ type = "GitRepositoryContext", name = "capsula", path = ".", path_relative_to_project_root = true },
{ type = "CommandContext", command = "poetry check --lock", cwd = ".", cwd_relative_to_project_root = true },
{ type = "FileContext", path = "pyproject.toml", copy = true, path_relative_to_project_root = true },
{ type = "FileContext", path = "poetry.lock", copy = true, path_relative_to_project_root = true },
{ type = "CommandContext", command = "pip freeze --exclude-editable > requirements.txt", cwd = ".", cwd_relative_to_project_root = true },
{ type = "FileContext", path = "requirements.txt", move = true, path_relative_to_project_root = true },
{ type = "EnvVarContext", name = "HOME" },
]
reporters = [{ type = "JsonDumpReporter" }]
[in-run]
watchers = [{ type = "UncaughtExceptionWatcher" }, { type = "TimeWatcher" }]
reporters = [{ type = "JsonDumpReporter" }]
[post-run]
reporters = [{ type = "JsonDumpReporter" }]Then, all you need to do is decorate your Python function with the @capsula.run() decorator. You can also use the @capsula.context() decorator to add a context specific to the function.
The following is an example of a Python script that estimates the value of π using the Monte Carlo method:
import random
import capsula
@capsula.run()
@capsula.context(capsula.FunctionContext.builder(), mode="pre")
@capsula.context(capsula.FileContext.builder("pi.txt", move=True), mode="post")
def calculate_pi(n_samples: int = 1_000, seed: int = 42) -> None:
random.seed(seed)
xs = (random.random() for _ in range(n_samples))
ys = (random.random() for _ in range(n_samples))
inside = sum(x * x + y * y <= 1.0 for x, y in zip(xs, ys))
# You can record values to the capsule using the `record` method.
capsula.record("inside", inside)
pi_estimate = (4.0 * inside) / n_samples
print(f"Pi estimate: {pi_estimate}")
capsula.record("pi_estimate", pi_estimate)
print(f"Run name: {capsula.current_run_name()}")
with open("pi.txt", "w") as output_file:
output_file.write(f"Pi estimate: {pi_estimate}.")
if __name__ == "__main__":
calculate_pi(n_samples=1_000)After running the script, a directory (calculate_pi_20240711_190108_6I2M in this example) will be created under the <project-root>/vault directory, and you will find the output files in the directory:
$ tree vault/calculate_pi_20240711_190108_6I2M
vault/calculate_pi_20240711_190108_6I2M
├── in-run-report.json # Generated by the `JsonDumpReporter` in `capsula.toml` (`in-run` section)
├── pi.txt # Moved by the `FileContext` specified with the decorator in the script
├── poetry.lock # Copied by the `FileContext` specified in `capsula.toml` (`pre-run` section)
├── post-run-report.json # Generated by the `JsonDumpReporter` in `capsula.toml` (`post-run` section)
├── pre-run-report.json # Generated by the `JsonDumpReporter` in `capsula.toml` (`pre-run` section)
├── pyproject.toml # Copied by the `FileContext` specified in `capsula.toml` (`pre-run` section)
└── requirements.txt # Moved by the `FileContext` specified in `capsula.toml` (`pre-run` section)The contents of the JSON files are as follows:
Example of output pre-run-report.json:
{
"cwd": "/home/nomura/ghq/github.com/shunichironomura/capsula",
"cpu": {
"python_version": "3.8.19.final.0 (64 bit)",
"cpuinfo_version": [
9,
0,
0
],
"cpuinfo_version_string": "9.0.0",
"arch": "X86_64",
"bits": 64,
"count": 12,
"arch_string_raw": "x86_64",
"vendor_id_raw": "GenuineIntel",
"brand_raw": "Intel(R) Core(TM) i5-10400 CPU @ 2.90GHz",
"hz_advertised_friendly": "2.9000 GHz",
"hz_actual_friendly": "2.9040 GHz",
"hz_advertised": [
2900000000,
0
],
"hz_actual": [
2904008000,
0
],
"stepping": 5,
"model": 165,
"family": 6,
"flags": [
"3dnowprefetch",
"abm",
"adx",
"aes",
"apic",
"arch_capabilities",
"arch_perfmon",
"avx",
"avx2",
"bmi1",
"bmi2",
"clflush",
"clflushopt",
"cmov",
"constant_tsc",
"cpuid",
"cx16",
"cx8",
"de",
"ept",
"ept_ad",
"erms",
"f16c",
"flush_l1d",
"fma",
"fpu",
"fsgsbase",
"fxsr",
"ht",
"hypervisor",
"ibpb",
"ibrs",
"ibrs_enhanced",
"invpcid",
"invpcid_single",
"lahf_lm",
"lm",
"mca",
"mce",
"md_clear",
"mmx",
"movbe",
"msr",
"mtrr",
"nopl",
"nx",
"osxsave",
"pae",
"pat",
"pcid",
"pclmulqdq",
"pdcm",
"pdpe1gb",
"pge",
"pni",
"popcnt",
"pse",
"pse36",
"rdrand",
"rdrnd",
"rdseed",
"rdtscp",
"rep_good",
"sep",
"smap",
"smep",
"ss",
"ssbd",
"sse",
"sse2",
"sse4_1",
"sse4_2",
"ssse3",
"stibp",
"syscall",
"tpr_shadow",
"tsc",
"vme",
"vmx",
"vnmi",
"vpid",
"x2apic",
"xgetbv1",
"xsave",
"xsavec",
"xsaveopt",
"xsaves",
"xtopology"
],
"l3_cache_size": 12582912,
"l2_cache_size": "1.5 MiB",
"l1_data_cache_size": 196608,
"l1_instruction_cache_size": 196608,
"l2_cache_line_size": 256,
"l2_cache_associativity": 6
},
"platform": {
"machine": "x86_64",
"node": "SHUN-DESKTOP",
"platform": "Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.34",
"release": "5.15.153.1-microsoft-standard-WSL2",
"version": "#1 SMP Fri Mar 29 23:14:13 UTC 2024",
"system": "Linux",
"processor": "x86_64",
"python": {
"executable_architecture": {
"bits": "64bit",
"linkage": "ELF"
},
"build_no": "default",
"build_date": "Jul 7 2024 07:23:53",
"compiler": "GCC 11.4.0",
"branch": "",
"implementation": "CPython",
"version": "3.8.19"
}
},
"git": {
"capsula": {
"working_dir": "/home/nomura/ghq/github.com/shunichironomura/capsula",
"sha": "a308c82bf9c8670de62d155e83ebf78f816f7851",
"remotes": {
"origin": "ssh:https://[email protected]/shunichironomura/capsula.git"
},
"branch": "main",
"is_dirty": false,
"diff_file": null
}
},
"command": {
"poetry check --lock": {
"command": "poetry check --lock",
"cwd": "/home/nomura/ghq/github.com/shunichironomura/capsula",
"returncode": 0,
"stdout": "All set!\n",
"stderr": ""
},
"pip freeze --exclude-editable > requirements.txt": {
"command": "pip freeze --exclude-editable > requirements.txt",
"cwd": "/home/nomura/ghq/github.com/shunichironomura/capsula",
"returncode": 0,
"stdout": "",
"stderr": ""
}
},
"file": {
"/home/nomura/ghq/github.com/shunichironomura/capsula/pyproject.toml": {
"copied_to": [
"/home/nomura/ghq/github.com/shunichironomura/capsula/vault/calculate_pi_20240711_190108_6I2M/pyproject.toml"
],
"moved_to": null,
"hash": {
"algorithm": "sha256",
"digest": "bfd58ba4947798d61ae3679cf3b06def700aefcdc33a2e4935164f480f16191c"
}
},
"/home/nomura/ghq/github.com/shunichironomura/capsula/poetry.lock": {
"copied_to": [
"/home/nomura/ghq/github.com/shunichironomura/capsula/vault/calculate_pi_20240711_190108_6I2M/poetry.lock"
],
"moved_to": null,
"hash": {
"algorithm": "sha256",
"digest": "210e74a7cd2db48b95dee0def67d5bbba33a86ab85859bea6c17ca74b48a2448"
}
},
"/home/nomura/ghq/github.com/shunichironomura/capsula/requirements.txt": {
"copied_to": [],
"moved_to": "/home/nomura/ghq/github.com/shunichironomura/capsula/vault/calculate_pi_20240711_190108_6I2M",
"hash": {
"algorithm": "sha256",
"digest": "7d8d12ce44cae648c0f7cc7a636e226c857510276cd3a221de1ffa4d7125c5b0"
}
}
},
"env": {
"HOME": "/home/nomura"
},
"function": {
"calculate_pi": {
"file_path": "examples/simple_decorator.py",
"first_line_no": 6,
"bound_args": {
"n_samples": 1000,
"seed": 42
}
}
}
}Example of output in-run-report.json:
{
"inside": 782,
"pi_estimate": 3.128,
"time": {
"execution_time": "0:00:00.000612"
},
"exception": {
"exception": {
"exc_type": null,
"exc_value": null,
"traceback": null
}
}
}Example of output post-run-report.json:
{
"file": {
"pi.txt": {
"copied_to": [],
"moved_to": "/home/nomura/ghq/github.com/shunichironomura/capsula/vault/calculate_pi_20240711_190108_6I2M",
"hash": {
"algorithm": "sha256",
"digest": "a64c761cb6b6f9ef1bc1f6afa6ba44d796c5c51d14df0bdc9d3ab9ced7982a74"
}
}
}
}You can install Capsula via pip:
pip install capsulaOr via conda:
conda install conda-forge::capsulaThis project is licensed under the terms of the MIT.
Additionally, this project includes code derived from the Python programming language, which is licensed under the Python Software Foundation License Version 2 (PSF-2.0). For details, see the LICENSE file.