An interactive shell written in Go, to run the Eth2 network stack, attach to testnets, debug clients, and extract data for tooling.
The shell is built on top of mvdan/sh
, which aims to be POSIX compatible, and has some Bash features.
Control flow, multi-line inputs, and command history are all supported.
The core idea of the shell is to start and maintain p2p processes, and make the resulting information available though logs, to auto-fill the environment with anything you would want.
Also note that actors and running commands can be shared and persisted between sessions.
To just install it in one command:
GO111MODULE=on go get github.com/protolambda/rumor
To run it from source (instead of go get
to install)
git clone [email protected]:protolambda/rumor.git
cd rumor
go get ./...
- Eth2:
protolambda/zrnt
: For Eth2 consensus codeprotolambda/ztyp
: For SSZ tree structuresethereum/go-ethereum
Geth, for ENR and Discv5 packages
- Libp2p
ipfs/go-datastore
: New experimental peerstore codelibp2p/go-libp2p
: Gossipsub, mutli-select, transports, multiplexers, identification, RPC base, and more.
mvdan/sh
: For shell parsing and interpretingchzyer/readline
: For history/shell codeprotolambda/ask
: For dynamically-evaluated command structuregorilla/websocket
: For websocket server/clientgliderlabs/ssh
: For SSH serversirupsen/logrus
: For logging infrastructure
There are a few sub-commands to choose the mode of operation:
attach Attach to a running rumor server
bare Rumor as a bare JSON-formatted input/output process, suitable for use as subprocess.
file Run from a file
help Help about any command
serve Rumor as a server to attach to
shell Rumor as a human-readable shell <- recommended
tool Run rumor toolchain
Serve and attach supported types (build any infra you want):
http
(optional auth key,X-Api-Key
header)ws
(http upgraded to websocket)script
(http requests, not supported forattach
)
tcp
(raw socket)ipc
(Unix socket)ssh
(Direct SSH into rumor shell, no need forattach
)
See https://notes.ethereum.org/@protolambda/rumor-tutorial
For more examples, see the /scripts
directory. With include
you can try snippets from the convenience of the interactive shell!
# Have a look at all functionality
help
# Start the libp2p host
host start
# You probably want to know everything that is happening, run this in the background:
host notify all
# Start listening (optionally specify ip or tcp/udp ports to listen at)
# Note that the peer-manager is active by default.
host listen
# See the host up and running
host view
# Broswe your peers
peer list
# Connect to peer
peer connect <enr/enode/multi-addr>
# Start discv5 and connect to the Lighthouse testnet
# Give one or more bootnode addresses (ENR or enode format).
dv5 run enr:-Iu4QLNTiVhgyDyvCBnewNcn9Wb7fjPoKYD2NPe-jDZ3_TqaGFK8CcWr7ai7w9X8Im_ZjQYyeoBP_luLLBB4wy39gQ4JgmlkgnY0gmlwhCOhiGqJc2VjcDI1NmsxoQMrmBYg_yR_ZKZKoLiChvlpNqdwXwodXmgw_TRow7RVwYN0Y3CCIyiDdWRwgiMo
# Get your local Discv5 node info. Other discv5 nodes can bootstrap from this.
dv5 self
# Start querying for random nodes in the background
dv5 random
# Get nearby nodes
dv5 lookup
# Log gossip blocks of prysmatic testnet to file
gossip start
gossip join /eth2/beacon_block/ssz_snappy
# listening on the topic!
gossip list
# Check your peers on the topic
gossip list-peers /eth2/beacon_block/ssz_snappy
# Subscribe to the topic
gossip log /eth2/beacon_block/ssz_snappy
# Exit task
cancel
# Leave channel
gossip leave /eth2/beacon_block/ssz_snappy
# Ask a connected node for a Status
rpc status req raw 16Uiu2HAmQ9WByeSsnxLb2tBW3MkGYMfg1BQowkwyVdUD9WwMdnrc
And there are commands to make debugging various things easy as well:
enr view --kv enr:-Iu4QLNTiVhgyDyvCBnewNcn9Wb7fjPoKYD2NPe-jDZ3_TqaGFK8CcWr7ai7w9X8Im_ZjQYyeoBP_luLLBB4wy39gQ4JgmlkgnY0gmlwhCOhiGqJc2VjcDI1NmsxoQMrmBYg_yR_ZKZKoLiChvlpNqdwXwodXmgw_TRow7RVwYN0Y3CCIyiDdWRwgiMo
By prefixing a command with {actor-name}:
, you can run multiple p2p hosts, by name {actor-name}
in the same process.
alice: host start
alice: host listen --tcp=9000
bob: host start
bob: host listen --tcp=9001
# Get ENR of alice
alice: host view
# Connect bob to alice
bob: peer connect <ENR from alice>
To change the default actor, use the me
command:
protolambda: me
The environment var ACTOR
is reserved and always points to the current default actor name
(not to what the command may say, which is evaluated later).
Writing some things may become repetitive.
To not repeat yourself, while still getting the same results as typing it yourself, you can use include
to run rumor files.
host.rumor
host start
host notify all
# maybe start some other tasks
Then, in a shell, or in another rumor file:
# The include will use the new default actor
alice: me
include host.rumor
alice: host view
The loglevel individual commands can be changed with the lvl_{name}
proxy commands.
name
can be any of trace, debug, info, warn, error, fatal, panic
E.g. lvl_warn peer connect
to ignore most logs, except warnings and higher-severity errors.
By prefixing a command with _
, like _{my-call-id}
, you can see which logs are a result of the given call {my-call-id}
,
even if interleaved with other logs (because of async results).
The call-ID should be placed before the actor name.
If no call-ID is specified, the call uses an incrementing counter for each new call.
I.e. an auto-generated call ID, with just the session id and a number, the command is prefixed with e.g. s42_123
.
Using such bare strings as custom call-IDs is unsafe, as these call-IDs will overlap with the auto-generated IDs.
_my_call alice: host start
.... log output with call_id="my_call"
# also valid, but not recommend:
alice: _my_call host start
The call-ID is useful to correlate results, and for tooling that interacts with Rumor to get results.
Also, to reference log-output in environment variables, the call IDs are very useful to select an exact variable.
Each log-entry will automatically be stored in the Rumor environment, to retrieve by its call ID.
Example:
_my_call alice: host listen
bob: peer connect _my_call_addr
And, the log entry variables of the last call in the session can be referenced with just _
(and still separated with _
from the log-entry name):
Example:
alice: host listen
bob: peer connect __addr
Both the call ID and the log entry name may contain multiple underscores. References to call IDs always start with a _
.
After running many calls, the environment may grow too big.
To flush it, run clear_log_data
to clear every entry, except those prefixed with a call ID of an open call.
If you need the environment variable data in pure JSON format, instead of bash variable types, then you can simply grab
it:
# logs new private key to variable
_example host start # (call will named "example" here)
# Grab the data
grab priv > priv.txt
grab example_priv > priv.txt
# Or, with the regular environment variable prefixes
grab __priv > priv.txt
grab _example_priv > priv.txt
To close a long-running call (which may run in the background), you can:
- If it is the latest call, just run
cancel
- Cancel it directly by its call ID:
_my_call_ cancel
Optionally, provide a duration, like 3s
, to set a timeout on the cancellation.
_my_call_ cancel 3s
Some long-running calls can be stepped through: by running next
you enable the call to progress.
Optionally, provide a duration as timeout, to not indefinitely wait for the next step of the call,
or to stop steps that take too long to complete.
_my_call_ next 3s
An actor can be killed by running a kill
command on them, after that a fresh libp2p host can be started.
alice: kill
Enter exit
, or send an interrupt signal.
All remaining open calls will be canceled.
To see what Rumor is running, use calls
These are reserved names, used for shell functionality:
"false", "exit", "set", "shift", "unset",
"echo", "printf", "break", "continue", "pwd", "cd",
"wait", "builtin", "trap", "type", "source", ".", "command",
"dirs", "pushd", "popd", "umask", "alias", "unalias",
"fg", "bg", "getopts", "eval", "test", "[", "exec",
"return", "read", "shopt"
SSH with socket linking to connect via TCP/IPC is extra work, if all you need is a human-readable shell.
Instead, Rumor can serve its shell
mode as SSH server, to make remote-access easy.
# One-time only: generate a key for the server, so it has the same identity after restarting (avoid SSH trust warnings)
ssh-keygen -f rumor_ssh_server -t rsa -b 4096 -P ""
SSH users have two options for authentication:
- Password based:
# Serve rumor on SSH (change user details)
rumor serve --ssh=0.0.0.0:5000 --ssh-key=rumor_ssh_server --ssh-users=myuser:1234 --ssh-users=other:abcd
# Login directly into rumor shell (change host details)
ssh [email protected] -p 5000
- Key based:
# Serve rumor on SSH (change user details)
rumor serve --ssh=0.0.0.0:5000 --ssh-key=rumor_ssh_server --ssh-authorized-keys ~/.ssh/authorized_keys
# Login directly into rumor shell (change host details)
ssh -i some-key-that-has-been-authorized [email protected] -p 5000
To run rumor scripts with HTTP POST requests, send scripts to the /scripts
endpoint, customize with serve --post-path=/mypath
option.
The HTTP post requests type can run a rumor script and return the full logs once the script completes.
POST
is recommended as method, but not enforced.
With HTTP status codes from exit codes:
0 = 200
1 = 500
2 = 400
And headers:
X-Api-Key
like WSX-Log-Format
:json
orterminal
X-Log-Level
:trace
,debug
,info
,warn
,error
MIT, see LICENSE
file.