Evaluate Julia packages.
PkgEval.jl is a package to test one or more Julia versions against the Julia package ecosystem, and is used by Nanosoldier.jl for keeping track of package compatibility of upcoming Julia versions.
Note that for now, PkgEval.jl is Linux-only, and even requires a sufficiently recent kernel (at least 5.11, or a distribution like Ubuntu that has back-ported support for unprivileged overlayfs mounts in user namespaces).
PkgEval is not a registered package, so you'll need to install it from Git:
git clone https://github.com/JuliaCI/PkgEval.jl.git
cd PkgEval.jl
julia --project -e 'import Pkg; Pkg.instantiate()'
To quickly test a package, a script has been provided under the bin/
folder:
$ julia --project bin/test_package.jl --name=Example
Package evaluation of Example started at 2022-11-27T09:30:27.777
...
Testing completed after 1.04s
This script can also be used to test specific versions of a package by setting any of the
--version
, --rev
, or --url
arguments. To test a version of a package you only have
locally, e.g., a development version, use the --path
argument instead:
$ julia --project bin/test_package.jl --name Example --path=~/.julia/dev/Example
By default, this will use the latest nightly
version of Julia, which is what PkgEval
uses. To use another version, use the --julia
argument, e.g., --julia=1.8
.
To use PkgEval programmatically, there are three main interfaces do deal with:
Configuration
objects to determine how to execute tests (which Julia version, build flags, any environment variables, ...)Package
objects to select packages to test- the
evaluate
function to evaluate every package on each provieded configuration, returning the results in a DataFrame
julia> using PkgEval
julia> config = Configuration(; julia="1.7");
julia> package = Package(; name="Example");
julia> evaluate([config], [package])
1×9 DataFrame
Row │ julia_spec julia_version compiled name version ⋯
│ String VersionNumber Bool String VersionN…? ⋯
─────┼───────────────────────────────────────────────────────────
1 │ 1.7 1.7.0 false Example 0.5.3 ⋯
4 columns omitted
Test logs are part of this dataframe in the log
column. For example, in this case:
Resolving package versions...
Installed Example ─ v0.5.3
...
Testing Example tests passed
If you want to debug why your package fails, it's probably easiest to use an interactive shell:
julia> using PkgEval
julia> config = Configuration()
PkgEval configuration(
...
)
julia> PkgEval.sandboxed_julia(config)
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.9.0-DEV.1163 (2022-08-21)
_/ |\__'_|_|_|\__'_| | Commit 696f7d3dfe1 (1 day old master)
|__/ |
julia> # we're in the PkgEval sandbox here
Now you can install, load and test your package. This will, by default, use a nightly build of Julia. If you want PkgEval.jl to compile Julia, e.g. to test a specific version, create a Configuration instance as such:
julia> config = Configuration(julia="master",
buildflags=["JULIA_CPU_TARGET=native", "JULIA_PRECOMPILE=0"])
# NOTE: buildflags are specified to speed-up the build
PkgEval uses cgroups for restricting the resources each package can use. By default however,
non-root users can control the memory
and pids
cgroup controllers. To enable PkgEval
to control more resources, run the following commands:
$ sudo mkdir -p /etc/systemd/system/[email protected]
$ cat <<EOF | sudo tee /etc/systemd/system/[email protected]/delegate.conf
[Service]
Delegate=cpu cpuset io memory pids
EOF
$ sudo systemctl daemon-reload
In addition, some container runtimes (i.e. runc
) want full control over the current
cgroup, which can be done by launching Julia as a scoped service:
systemd-run --user --scope -p Delegate=yes julia ...