Skip to content

PCIe to OpenCAPI-Compatible SlimSAS 8x Adapter PCB

License

Notifications You must be signed in to change notification settings

mwrnd/OpenCAPI-to-PCIe

Repository files navigation

Work-in-Progress - Third Revision gerbers completed. Second Revision is being tested. PCIe x8 using the OpenCAPI connector works but requires a high quality cable and uses a PCIe Lane to Transceiver Channel ordering that Vivado complains about.

PCIe to OpenCAPI-Compatible SlimSAS8x

The Open Coherant Accelerator Processor Interface (OpenCAPI) was a standard that had FPGA-based Advanced Accelerated Cable (AAC) Add-In cards such as ADM-PCIE-9H3, ADM-PCIE-9H7, ADM-PCIE-9V3, ADM-PCIE-9V5, BittWare XUP-VV4, BittWare XUP-VVH, and Nvidia Innova-2 Flex SmartNIC.

The OpenCAPI interface is based on PCI-Express and uses a SlimSAS 8X (SFF-8654) Connector. This adapter enables connecting an OpenCAPI FPGA board to a host using PCIe over a SlimSAS cable.

OpenCAPI-to-PCIe Board

Testing

The innova2_xdma_opencapi project is designed to test the OpenCAPI-to-PCIe Adapter using an Innova2 Flex SmartNIC.

The Innova2 SmartNIC's XCKU15P FPGA does not have its Configuration Block in the same column as the OpenCAPI GTY transceivers so it is impossible to configure the FPGA within the PCIe Specification's 100ms time limit. Motherboard boot must be delayed to allow the FPGA to configure itself before PCIe devices are enumerated by the host system. This can be accomplished by toggling the POWER button, then pressing and holding the RESET button for a second before releasing it. Or, connect a capacitor across the reset pins of an ATX motherboard's Front Panel Header:

Delay Boot Using Capacitor across Front Panel Header Reset Pins

Using a 3M 8ES8-1DF21-0.75 cable:

System with 3M 8ES8-1DF21-0.75 Cable

PCIe Link Status is usually excellent:

lspci Link Status

Using an SFPCables.com SFF-8654 to SFF-8654 8i cable:

System with SFPCables SFF-8654 8i 85Ohm Cable

PCIe Link Status is downgraded:

lspci Link Status

I am working on a third revision of the OpenCAPI-to-PCIe adapter to improve signal integrity.

Additional OpenCAPI Signals

Additional useful signals from the OpenCAPI connector are routed to a 6x1 0.1" Header. The pinout matches a TC74 I2C Temperature Sensor. Note 3.3V is from the PCIe connector. PRE is a Presence Detect pin which is connected to GND via a 50-Ohm resistor on the OpenCAPI AAC Add-In card. RST is connected to PCIe/OpenCAPI RESET.

TC74A0-3.3VAT in OpenCAPI-to-PCIe Adapter

The innova2_xdma_opencapi project features the ability to test a TC74Ax-3.3VAT in an OpenCAPI-to-PCIe Adapter.

TC74A0-3.3VAT Testing in a System

PCB Layout

4-Layer PCB. Inner 2 layers are GND planes. Differential pairs are matched to a length of 61mm +/- 1mm both inter-pair and intra-pair (N-to-P).

OpenCAPI to PCIe x8 PCB Layout

Schematic

OpenCAPI to PCIe x8 Schematic

Design Notes

Refer to the ADM-PCIE-9V5 User Manual (Pg15-19of38) for the OpenCAPI pinout. Useful High Speed Design presentation.

Only a single component is required for the adapter, a U10A474200T/U10A474240T SlimSAS 8x Right-Angle SMD Connector. A SlimSAS 8x Cable such as the 3M 8ES8-1DF21(Datasheet) is required to use the adapter with an OpenCAPI FPGA Board.

Resistor R1 is shorted to connect nPRSNT1 to nPRSNT2_x8. The trace can be scratched off and nPRSNT1 can be connected to nPRSNT2_x1 or nPRSNT2_x4 using 28AWG to change the PCIe width.

Second Revision

The Second Revision of this project worked successfully with the First Release of the innova2_xdma_opencapi project but required a non-standard GTY Channel to PCIe Lane mapping.

Vivado Critical Warning about Lane Ordering

The Second Revision used the OpenCAPI pinout from the ADM-PCIE-9V5 User Manual (Pg15-19of38):

OpenCAPI Pinout from

Second Revision PCB Layout

4-Layer PCB. Inner 2 layers are GND planes. Differential pairs are matched to a length of 65mm +/- 1mm both inter-pair and intra-pair (N-to-P).

OpenCAPI to PCIe x8 PCB Layout Second Revision

Second Revision Schematic

OpenCAPI to PCIe x8 Schematic Second Revision

PCB Stackup

I am using values from JLCPCB.

4-Layer Stackup

Trace Impedance Control

OpenCAPI uses 85ohm impedance cables. I played with the values until I got the loosest differential pair coupling that is manufacturable with larger tolerances.

85ohm Differential Impedance in DigiKey Calculator