Implement rctl limits for the VMs

[yaroslav-gwit]

The problem

Sometimes you want to limit the amount of CPU in %, or a storage read/write capabilities for a particular VM because it slows down the whole system. rctl can be used to do just that. vm_supervisor will need to patched to apply the limits reliably.

In particular, I'll implement these to start with:

pcpu=0  # %CPU, in percents of a single CPU core
readbps=0  # filesystem reads, in bytes per second
writebps=0  # filesystem writes, in bytes per second
readiops=0  # filesystem reads, in operations per second
writeiops=0  # filesystem writes, in operations per second

The limits can be applied to both a live/running VM, and a VM that has just started. So there will be a CLI flag to apply the limit, but also a config flag to apply limits at the VM start-up.

Some info for the future me

Get bhyve VM/process current usage:

rctl -hu process:59351

List current rules:

rctl -l process:59351

Apply a new rule:

rctl -a process:59351:pcpu:deny=50
rctl -a process:59351:readbps:throttle=512000
rctl -a process:59351:writebps:throttle=512000

Remove a rule:

rctl -r process:59351

Visit rctl man page for more info: https://man.freebsd.org/cgi/man.cgi?query=rctl&sektion=8

[yaroslav-gwit]

Closing this issue due to a lack of time.

Might come back to it later at some point, given there is an outside interest.
As it stands right now - I don't have a use-case for it myself.