[Shengzhou Qiang]

Hi FABRIC team,

I’ve read the BlueField notebook, fablib‘s Component.configure(), and the DPU forum threads, so the basics are clear: the whole card is passed through (rshim function included), and bfb-install / ssh ubuntu@192.168.100.2 are driven from inside the slice VM. Four things I couldn’t find answered anywhere.

1. Operating mode. What is INTERNAL_CPU_OFFLOAD_ENGINE set to on the deployed BF-3s — DPU (embedded-function) mode or NIC mode? Nothing published states this, and it decides whether ARM-side datapath work is possible at all.
2. DPU↔NVMe PCIe topology. Every DPU host also advertises NVME_P4510 (capacity 4). On a Dell R7525, do the BF-3 functions and those NVMe drives sit under the same PCIe root / same IOD quadrant, and is peer-to-peer DMA between two passed-through devices permitted (ACS / IOMMU posture)? An lspci -tvnn from a DPU worker would settle it. We need HCA↔SSD P2P for NVMe-oF target offload; on CloudLab’s d7525 (same model) the NIC and NVMe land on different roots, so I expect the answer is no — I’d rather know now than design around it.
3. Firmware-config path. With no user-initiated cold power cycle, is there any supported route to apply an mlxconfig change on a DPU (a scheduled maintenance window, a host with no other slivers)? I know PCI_SWITCH_EMULATION_ENABLE currently conflicts with passthrough — I’m asking about the general mechanism, not SNAP.
4. CPU accounting. Are slice vCPUs pinned/dedicated (is steal guaranteed zero), and are the BF-3’s 16 ARM cores exclusive to the slice holding the card? We measure cycles-per-operation, so any sharing invalidates the numbers.

For planning: which sites currently have an allocatable ConnectX-7 (SEAT / KANS / TOKY, by my reading of /api/resources and post #9962), and is there a way to reserve one rather than race for it?

Thanks,

[Author]

Posts