[Ivan Torres]

Hello Komal,

I was able to diagnose the problem myself, I no longer have the previous slice so I wanted to let you know so there is no confusion on what happened.

Thank you

[Ivan Torres]

Slice ID: 26676675-b9c4-4826-a1b9-05550877af0e

Thank you Komal

[Ivan Torres]

BlueField-3 DPL runtime failure: “Failed creating flex parser graph”

Hello FABRIC team,

I am using DOCA Pipeline Language on a reserved BlueField-3 DPU to compare packet payloads against a synthetic reference. My end goal is to parse the required payload bytes, without re-invoking the packet, to perform the comparisons and forward the packet at line-rate.

As of now, the program compiles successfully, but loading it through P4Runtime fails during flex-parser graph creation. At runtime it reports:
Error setting program: Failed creating flex parser graph.
The corresponding runtime log from DPL containers say the following:

NV_HWS[cmd_flex_node_create:2024]: Failed to allocate flex node (syndrome: 0x784f91)
[hws_objects.cc:4635][flex_parser_graph_create] Failed to bind graph (1)
[dpl_rt_core.cc:97][program_load] Failed creating flex parser graph 4294967274
[dpl_rt_core.cc:1826][dpl_rt_core_device_program_load] Failed to load program: Failed loading program: 4294967274

Keep in mind, packets being used are small… The payload size is 32 bytes to stay within custom parsing budget. So far, I have tested the amount of stateful data that can be compared, and found that only a single 4-byte payload word can be compared without the runtime client complaining about creating the flex graph. As far as I am aware, the BlueField-3 has a maximum amount of user metadata at 32 bytes at any single point in the program. The rest of space should be held by firmware details… Which is beyond me though I thought it important to mention.

I believe the attempted layout is within the sampling budgets, but need help confirming whether that is the case. Here is my environment details:

Component	Configuration
Hardware	BlueField-3 B3240 P-Series, 900-9D3B6-00CN-A_Ax
Firmware, running and stored	32.45.1020
Development image	dpl_dev:1.1.10-doca3.0.0-host
Compiler	nvp4c-doca 1.2.106
Uplink driver	mlx5_core, version 25.04-0.6.1
eSwitch mode	switchdev

The relevant firmware settings should be active:

FLEX_PARSER_PROFILE_ENABLE = 4
PROG_PARSE_GRAPH = True(1)
LAG_RESOURCE_ALLOCATION = PRE_ALLOCATION(1)
INTERNAL_CPU_MODEL = EMBEDDED_CPU(1)
INTERNAL_CPU_ESWITCH_MANAGER = ECPF(0)

Multiport eSwitch is also confirmed enabled:

pci/0000:03:00.0:
name esw_multiport type driver-specific
values:
cmode runtime value true

Could you help me determine whether the environment that I have is suitable for custom parsing and comparison of a 32 byte payload? I am unsure of how to tackle the graph-layout limitation.

Thank you

• This reply was modified 6 days, 5 hours ago by Ivan Torres.
• This reply was modified 6 days, 5 hours ago by Ivan Torres.
• This reply was modified 6 days, 5 hours ago by Ivan Torres.

[Ivan Torres]

Thank you so much!

It is functioning as expected

[Ivan Torres]

Hello Mert,

Thank you for the response. I was able to figure out the previous issue and have since been looking into experimenting with the 400 Gbps BlueField-3 DPU.

I am aiming to set up a topology across three sites on the terabit core. The DPU is located at SALT, while LOSA and STAR each reserve two host VMs, adding up to four worker nodes that communicate through the DPU. Each site is connected to a separate physical port on the DPU using its own L2STS network.

When attempting to provision this topology, ‘net1’ expressed the following error:

failed lease update- all units failed priming: Exception during create for unit: 38255f9e-59b7-49fe-ad6f-a2119c4fa192 Playbook has failed tasks: NSO validate_commit returned JSON-RPC error: type: trans.validation_failed, code: -32000, message: Validation failed, data: errors: [reason: Python cb_create error. FourHundredGigE0/0/0/24 not in /ncs:devices/devicesalt-data-sw/config/cisco-ios-xr:evpn/interface]#all units failed priming: Exception during create for unit: 38255f9e-59b7-49fe-ad6f-a2119c4fa192 Playbook has failed tasks: NSO validate_commit returned JSON-RPC error: type: trans.validation_failed, code: -32000, message: Validation failed, data: errors: [reason: Python cb_create error. FourHundredGigE0/0/0/24 not in /ncs:devices/devicesalt-data-sw/config/cisco-ios-xr:evpn/interface]

From what I understand, this is an exception dealing with a physical port on the DPU? Could you please help me determine what needs to be corrected so this topology can be provisioned?

Thank you!

• This reply was modified 1 month, 3 weeks ago by Ivan Torres.

[Ivan Torres]

BlueField-3 host becomes unreachable after DOCA installation and reboot

Hi all,

I am experiencing an issue with a FABRIC node that has a BlueField-3 DPU attached.

The node is initially reachable normally and I am able to SSH into the host using its management IPv6 address. Other nodes in the same slice are also reachable without issue.

Where I begin to experience issues is on installing the NVIDIA DOCA software for the BlueField-3 and rebooting the host to apply changes on the host. From that moment on the node becomes unreachable over SSH.

The FABRIC slice still reports the node as Active, and the management IPv6 address appears unchanged. However, attempting to reconnect results in:

channel 0: open failed: connect failed: No route to host
stdio forwarding failed
Connection closed by UNKNOWN port 65535

I have waited 30 minutes to an hour after reboot, but the host does not become reachable again.

The other reserved nodes remain accessible with the same bastion, SSH configuration, and keys. As far as I am aware of, this behavior appears specific to the node with the attached BlueField-3 and occurs after the NVIDIA/DOCA software installation followed by a host reboot.

I noticed this forum discussing DPU BlueField-3 upgrades, so I wanted to report this in case the behavior is related.

Thank you.

[Author]

Posts