The SwitchML controller will program the switch at runtime using the Barefoot Runtime Interface (BRI). The controller accepts connections from end-hosts through gRPC to set up a job (which is a sequence of allreduce operations involving the same set of workers). It also provides a CLI interface that can be used to configure the switch and read counters values at runtime.
The controller requires python 3.8 and the following python packages:
grpcio>=1.34.0 pyyaml asyncio ipaddress ansicolorsIt also requires the gRPC code autogenerated from the .proto file, which is done by running make in the controller folder. If you installed GRPC on your own then you should pass GRPC_HOME=path_to_grpc_installation to the make command.
Additionally, the following two modules are required:
bfrt_grpc.bfruntime_pb2 bfrt_grpc.clientThese modules are autogenerated when P4 Studio is compiled. The controller expects that the SDE_INSTALL environment variable points to the SDE install directory. It will search for those modules in the following folder:
$SDE_INSTALL/lib/python*/site-packages/tofino/bfrt_grpc/To enable switch ports and configure the switch to forward regular traffic, the controller reads the ports.yml file that describes the machines connected to the switch ports. Each front panel port is identified with port number and lane. The parameters per port are:
- speed (one of 10G, 25G, 40G, 50G, 100G, the default is 100G)
- fec (one of none, fc, rs, the default is "none")
- autoneg (one of default, enable, disable, the default is "default")
- mac (mac address of the NIC connected to the port)
This is an example:
ports:
1/0 : {speed: "100G", fec: "none", autoneg: "disable", mac: "00:11:22:33:44:55"}
2/0 : {speed: "100G", fec: "none", autoneg: "disable", mac: "00:11:22:33:44:66"}The controller is started with:
python switchml.pyThe optional arguments are the following:
| Argument | Description | Default |
|---|---|---|
| --program PROGRAM | P4 program name | SwitchML |
| --bfrt-ip ADDRESS | Name/address of the BFRuntime server | 127.0.0.1 |
| --bfrt-port PORT | Port of the BFRuntime server | 50052 |
| --switch-mac SWITCH_MAC | MAC address of the switch | 00:11:22:33:44:55 |
| --switch-ip SWITCH_IP | IP address of switch | 10.0.0.254 |
| --ports FILE | YAML file describing machines connected to ports | ports.yaml |
| --enable-folded-pipe | Enable the folded pipeline (requires a 4 pipes switch) | disabled |
| --log-level LEVEL | Logging level: ERROR, WARNING, INFO, DEBUG | INFO |
The BFRuntime server is the switch reference drivers application. The switch MAC and IP are the addresses of your choosing that will be used by the switch when acting as a SwitchML endpoint.