Docker image with dev mode misses volumes

[credbbl]

Describe the bug
Starting openbao in dev mode from out of the Docker image fails with:

Error initializing Dev mode: open /home/openbao/.vault-token.tmp: read-only file system

To Reproduce
Steps to reproduce the behavior:

1. Run podman run --rm --read-only -e SKIP_SETCAP=true -ti quay.io/openbao/openbao:2.0.0-alpha20240329
2. See error

Expected behavior
Docker images that write data are supposed to define volumes in those places, so the environment knows about.

[cipherboy]

@credbbl I think it is the interaction with --read-only: note that this makes the entire filesystem read-only. If you want to use this mode, I'd suggest passing -dev-no-store-token:

$ podman run --rm --read-only -e SKIP_SETCAP=true -ti quay.io/openbao/openbao:2.0.0-alpha20240329 bao server -dev -dev-no-store-token

==> OpenBao server configuration:

Administrative Namespace:
Api Address: http:https://127.0.0.1:8200
Cgo: disabled
Cluster Address: https://127.0.0.1:8201
Environment Variables: BAO_DISABLE_FILE_PERMISSIONS_CHECK, HOME, HOSTNAME, NAME, PATH, PWD, SHLVL, SKIP_SETCAP, TERM, VERSION, container
Go Version: go1.22.1
Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
Log Level:
Mlock: supported: true, enabled: false
Recovery Mode: false
Storage: inmem
Version: OpenBao v2.0.0-alpha20240329, built 2024-03-29T21:37:50Z
Version Sha: f20d468

==> OpenBao server started! Log data will stream in below:

2024-06-13T17:02:36.446Z [INFO] proxy environment: http_proxy="" https_proxy="" no_proxy=""
2024-06-13T17:02:36.447Z [WARN] no api_addr value specified in config or in BAO_API_ADDR; falling back to detection if possible, but this value should be manually set
2024-06-13T17:02:36.448Z [INFO] core: Initializing version history cache for core
2024-06-13T17:02:36.449Z [INFO] core: security barrier not initialized
2024-06-13T17:02:36.449Z [INFO] core: security barrier initialized: stored=1 shares=1 threshold=1
2024-06-13T17:02:36.450Z [INFO] core: post-unseal setup starting
2024-06-13T17:02:36.460Z [INFO] core: loaded wrapping token key
2024-06-13T17:02:36.460Z [INFO] core: successfully setup plugin catalog: plugin-directory=""
2024-06-13T17:02:36.460Z [INFO] core: no mounts; adding default mount table
2024-06-13T17:02:36.465Z [INFO] core: successfully mounted: type=cubbyhole version="v2.0.0+builtin.bao" path=cubbyhole/ namespace="ID: root. Path: "
2024-06-13T17:02:36.465Z [INFO] core: successfully mounted: type=system version="v2.0.0+builtin.bao" path=sys/ namespace="ID: root. Path: "
2024-06-13T17:02:36.466Z [INFO] core: successfully mounted: type=identity version="v2.0.0+builtin.bao" path=identity/ namespace="ID: root. Path: "
2024-06-13T17:02:36.468Z [INFO] core: successfully mounted: type=token version="v2.0.0+builtin.bao" path=token/ namespace="ID: root. Path: "
2024-06-13T17:02:36.469Z [INFO] rollback: Starting the rollback manager with 256 workers
2024-06-13T17:02:36.469Z [INFO] rollback: starting rollback manager
2024-06-13T17:02:36.469Z [INFO] core: restoring leases
2024-06-13T17:02:36.470Z [INFO] expiration: lease restore complete
2024-06-13T17:02:36.471Z [INFO] identity: entities restored
2024-06-13T17:02:36.471Z [INFO] identity: groups restored
2024-06-13T17:02:36.471Z [INFO] core: Recorded vault version: vault version=2.0.0 upgrade time="2024-06-13 17:02:36.471619931 +0000 UTC" build date=2024-03-29T21:37:50Z
2024-06-13T17:02:36.472Z [INFO] core: post-unseal setup complete
2024-06-13T17:02:36.473Z [INFO] core: root token generated
2024-06-13T17:02:36.473Z [INFO] core: pre-seal teardown starting
2024-06-13T17:02:36.473Z [INFO] rollback: stopping rollback manager
2024-06-13T17:02:36.473Z [INFO] core: pre-seal teardown complete
2024-06-13T17:02:36.473Z [INFO] core.cluster-listener.tcp: starting listener: listener_address=127.0.0.1:8201
2024-06-13T17:02:36.474Z [INFO] core.cluster-listener: serving cluster requests: cluster_listen_address=127.0.0.1:8201
2024-06-13T17:02:36.474Z [INFO] core: post-unseal setup starting
2024-06-13T17:02:36.474Z [INFO] core: loaded wrapping token key
2024-06-13T17:02:36.474Z [INFO] core: successfully setup plugin catalog: plugin-directory=""
2024-06-13T17:02:36.475Z [INFO] core: successfully mounted: type=system version="v2.0.0+builtin.bao" path=sys/ namespace="ID: root. Path: "
2024-06-13T17:02:36.476Z [INFO] core: successfully mounted: type=identity version="v2.0.0+builtin.bao" path=identity/ namespace="ID: root. Path: "
2024-06-13T17:02:36.476Z [INFO] core: successfully mounted: type=cubbyhole version="v2.0.0+builtin.bao" path=cubbyhole/ namespace="ID: root. Path: "
2024-06-13T17:02:36.477Z [INFO] core: successfully mounted: type=token version="v2.0.0+builtin.bao" path=token/ namespace="ID: root. Path: "
2024-06-13T17:02:36.478Z [INFO] rollback: Starting the rollback manager with 256 workers
2024-06-13T17:02:36.478Z [INFO] rollback: starting rollback manager
2024-06-13T17:02:36.478Z [INFO] core: restoring leases
2024-06-13T17:02:36.478Z [INFO] identity: entities restored
2024-06-13T17:02:36.478Z [INFO] identity: groups restored
2024-06-13T17:02:36.479Z [INFO] expiration: lease restore complete
2024-06-13T17:02:36.479Z [INFO] core: post-unseal setup complete
2024-06-13T17:02:36.479Z [INFO] core: vault is unsealed
2024-06-13T17:02:36.484Z [INFO] core: successful mount: namespace="" path=secret/ type=kv version=""
2024-06-13T17:02:36.494Z [INFO] secrets.kv.kv_90b8301d: collecting keys to upgrade
2024-06-13T17:02:36.494Z [INFO] secrets.kv.kv_90b8301d: done collecting keys: num_keys=1
2024-06-13T17:02:36.494Z [INFO] secrets.kv.kv_90b8301d: upgrading keys finished
WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.

You may need to set the following environment variables:

$ export BAO_ADDR='http:https://127.0.0.1:8200'

The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.

Unseal Key: mudTzQ04zPl8rNweCeCr4ZiwkL8vV8rcPej5AwpgmUk=
Root Token: s.Us5VfUdmAxlluDpkrX1lKqt9

Development mode should NOT be used in production installations!

(or with other arguments to make it more similar to the existing invocation). HTH!

[credbbl]

A container is supposed to be immutable. If you just write everywhere, this is not longer true. Setup a proper home, if openbao expects to be able to write there?

[cipherboy]

@credbbl The -dev-no-store-token option should give you this. :-)

This write does not occur in prod mode, only dev mode.

[credbbl]

So you mean, this is to be fixed in https://github.com/openbao/openbao/blob/main/Dockerfile

CMD ["server", "-dev", "-dev-no-store-token"]

[cipherboy]

@credbbl Yes. If you want to open this as a PR, we can merge it. However, please add a changelog entry as this is breaking behavior if you're using the dev container interactively.