This application emulates various peripherals on CHERI platform:
- Altera Modular Scatter-Gather DMA engine (mSGDMA)
- ARM Generic ECAM PCI-controller
- Intel E1000 ethernet device
- AHCI controller and memory-backed Serial ATA device
CHERI CPU used by this application is synthesized on Altera FPGA and prototyped on Altera DE4 board.
This is bare-metal software, i.e. it runs on a dedicated CPU core of CHERI processor.
See the device-model branch of CheriBSD repository: https://github.com/CTSRD-CHERI/cheribsd
Note: cheribuild is required to build this project.
$ git clone -b device-model https://github.com/CTSRD-CHERI/cheribsd
$ make -j4 TARGET=mips TARGET_ARCH=mips64 buildworld
$ make -j4 TARGET=mips TARGET_ARCH=mips64 KERNCONF=BERI_DE4_USBROOT buildkernel
$ make -j4 TARGET=mips TARGET_ARCH=mips64 -DNO_ROOT DESTDIR=${HOME}/world-mips64 installworld
$ make -j4 TARGET=mips TARGET_ARCH=mips64 -DNO_ROOT DESTDIR=${HOME}/world-mips64 distribution
$ cd ${HOME}/world-mips64
$ echo '# Altera JTAG UART' >> etc/ttys
$ echo 'ttyj0 "/usr/libexec/getty std.115200" xterm on secure' >> etc/ttys
$ echo 'ttyj1 "/usr/libexec/getty std.115200" xterm on secure' >> etc/ttys
$ echo 'hostname="mips64"' >> etc/rc.conf
$ echo 'sendmail_enable="NONE"' >> etc/rc.conf
$ echo 'cron_enable="NO"' >> etc/rc.conf
$ echo "/dev/da0 / ufs ro 1 1" > etc/fstab
$ echo "./etc/fstab type=file uname=root gname=wheel mode=0644" >> METALOG
$ echo "./etc/rc.conf type=file uname=root gname=wheel mode=0644" >> METALOG
$ makefs -B big -D -f 30000 -o version=2 -s 1200m mips64.img METALOG
$ sudo pkg install llvm-cheri
$ git clone --recursive https://github.com/CTSRD-CHERI/device-model
$ cd device-model
$ gmake
$ cd ctsrd/cherilibs/trunk/tools/debug
$ ./berictl -j loadsof -z DE4_BERI_EPW.sof.bz2
$ ./berictl -j loadbin -z kernel.bz2 0x100000
$ ./berictl -j boot && ./berictl -j console
$ ifconfig atse0 10.10.0.2/24
$ /etc/rc.d/var start
$ nc -l 1234 > /var/tmp/dm.bin
$ nc 10.10.0.2 1234 < dm.bin
This enables peripheral window (register interface).
$ bm -rRl /var/tmp/dm.bin
$ nios2-terminal -i 2
$ devctl disable atse1
$ devctl disable msgdma2
$ devctl disable msgdma3
$ sleep 1
$ devctl enable msgdma2
$ devctl enable msgdma3
$ devctl enable atse1
$ ifconfig atse1 10.4.0.2/24
$ devctl disable pci0
$ devctl enable pci0
Note: both mSGMDA and e1000 PCI-e device use the same Altera FIFO, which means emulating both of them same time leads to undefined behaviour.
pcie@7fb10000 {
compatible = "pci-host-ecam-generic";
device_type = "pci";
#interrupt-cells = <1>;
#address-cells = <3>;
#size-cells = <2>;
reg = <0x7fb10000 0xe0000>;
reg-names = "PCI ECAM";
//attributes pci_addr cpu_addr size
ranges =
<0x02000000 0 0x7fb20000 0x7fb20000 0 0x00020000 //e1000 bar 0
0x01000000 0 0x00000010 0x00000010 0 0x00010000 //e1000 bar 2
0x43000000 0 0x7fb40000 0x7fb40000 0 0x00010000 //e1000 bar 1
0x02000000 0 0x7fb50000 0x7fb50000 0 0x00010000>; //ahci-hd bar 0
// PCI_DEVICE(3) INT#(1) CONTROLLER(PHANDLE) CONTROLLER_DATA(1)
interrupt-map = < 0x000 0 0 1 &beripic0 0x13 //e1000
0x800 0 0 1 &beripic0 0x14 >; //ahci
interrupt-map-mask = <0x800 0 0 7>;
};
...
device pci
device em
device ahci
...
| Start | End | Size | Description |
|---|---|---|---|
| 0x00000000 | 0x40000000 | 1 GB | Entire physical memory of SoC |
| 0x10000000 | 0x20000000 | 256 MB | Reserved by FreeBSD for device-model |
| 0x10000000 | 0x10800000 | 8 MB | device-model static data |
| 0x10800000 | 0x11000000 | 8 MB | device-model malloc() |
| 0x11000000 | 0x13000000 | 32 MB | free space |
| 0x13000000 | 0x20000000 | 208 MB | AHCI SATA device memory block |
IOMMU module translates addresses for various device-model peripherals and manages BERI TLB.
Only Altera FIFO-based models are currently evaluated.
To enable IOMMU module build device-model project with DM_IOMMU=1 environment variable.
On the FreeBSD side, add the following IOMMU devices to your DTS file:
va_space0: iommu-va@7fb60000 {
compatible = "beri,iommu-va-space";
reg = <0x0 0x20000000>;
};
va_space1: iommu-va@7fb60100 {
compatible = "beri,iommu-va-space";
reg = <0x20000000 0x20000000>;
};
iommu0: iommu@7fb60000 {
compatible = "beri,iommu";
reg = <0x7fb60000 0x100>;
va-region = <&va_space0>;
status = "okay";
};
iommu1: iommu@7fb60100 {
compatible = "beri,iommu";
reg = <0x7fb60100 0x100>;
va-region = <&va_space1>;
status = "okay";
};
Add the "xdma,iommu" property to the nodes of the models, for example
dm_msgdma0: msgdma@7fb04080 {
xdma,iommu = <&iommu0>;
};
dm_msgdma1: msgdma@7fb04000 {
xdma,iommu = <&iommu1>;
};
