Issue with SmartNIC Configuration on nodes

[Hemil Panchiwala]

Hi,

I’m encountering a problem while setting up nodes with SmartNICs on the Fabric testbed. While I can obtain the MAC addresses for the NICs, other crucial fields such as Physical Device, Device, and IP Address are showing up as None. This is causing issues when I try to execute scripts on the nodes.

Setup Details

I’m using the following code to set up the nodes and SmartNICs:

send_params = {"name": "sender", "site": "SEAT", "host": None, "image": "default_ubuntu_20"} 
send_nic_params = {"model": "NIC_ConnectX_5", "name": "SmartNIC"}
slice = fablib.new_slice(name=slice_name)
send_node = slice.add_node(**send_params)
send_iface = send_node.add_component(**send_nic_params).get_interfaces()[0]

I am attaching the details of the interface printed once the setup is done.

Error

When I try to execute the script (setup.sh as shown below in code) on the node, I am getting an “Authentication failed” error:

send = slice.get_node(send_params["name"])
send_iface = send.get_interface(network_name=network_name)
send.execute(f"echo '{setup_script.format(interface_name=send_iface.get_device_name())}' > ~/setup.sh && chmod +x ~/setup.sh")

This error seems to occur in the last line where send_iface.get_device_name() is returning None.

Questions

I am not sure why are the Physical Device, Device, and IP Address fields showing up as None for the SmartNIC. Also, how can I resolve the “Authentication failed” error? Any insights or suggestions would be greatly appreciated. Thank you for your help!

Slice details:

Name: dpdk_with_timestamps_l2bridge

UUID: 9265b19b-1407-4582-8ebd-697afd4ae65e

Fablib version: 1.7.3

Site tried: STAR and SEAT

[Komal Thareja]

Hi Hemil,

Could you please run the jupyter-examples-*/configure_and_validate/configure_and_validate.ipynb notebook?
This shall resolve any SSH key issues by renewing the expired bastion keys if any.

Try your setup.sh script after that and let us know if you still see this error.
In addition, could you please try to SSH to the VMs using the command shown in SSH Command coloumn.

Regarding auto configuring the IP addresses, please specify the subnet when creating a network and set mode to auto for the interfaces at slice creation. Please refer to one of the following examples for more details.

https://github.com/fabric-testbed/jupyter-examples/blob/main/fabric_examples/fablib_api/create_l2network_basic/create_l2network_basic_auto.ipynb

https://github.com/fabric-testbed/jupyter-examples/blob/main/fabric_examples/fablib_api/create_l2network_wide_area/create_l2network_wide_area_auto.ipynb

https://github.com/fabric-testbed/jupyter-examples/blob/main/fabric_examples/fablib_api/create_l3network_fabnet_ipv4/create_l3network_fabnet_ipv4_auto.ipynb

Please let us know if you still run into errors or questions.

Thanks,
Komal

[Hemil Panchiwala]

Thanks for the quick response. I ran the jupyter-examples-*/configure_and_validate/configure_and_validate.ipynb, but I still see the same error. When I try to ssh to VM from the terminal in jupyter-notebook, it gives permission denied (seems to be the error with key verification).

fabric@spring:fabric_config-96%$ ssh -i /home/fabric/work/fabric_config/slice_key -F /home/fabric/work/fabric_config/ssh_config ubuntu@2001:400:a100:3080:f816:3eff:fe28:f4bf
Warning: Permanently added 'bastion.fabric-testbed.net' (ED25519) to the list of known hosts.
hpanchiwala_0000176766@bastion.fabric-testbed.net: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
kex_exchange_identification: Connection closed by remote host
Connection closed by UNKNOWN port 65535

I also tried creating a new slice by specifying the subnet as well as mode to auto for the interfaces, but doesn’t seem to work (attaching the output once setup is done). Slice ID: 944f4af0-283e-4032-b215-d6abe48cf86a

slice_name = "dpdk_with_timestamps_l2bridge"
network_name='l2'

send_params = {"name": "sender", "site": "GATECH", "host": None, "image": "default_ubuntu_20"}
send_nic_params = {"model": "NIC_ConnectX_5", "name": "SmartNIC"}
recv_params = {"name": "receiver", "site": "GATECH", "host": None, "image": "default_ubuntu_20"}
recv_nic_params = {"model": "NIC_ConnectX_5", "name": "SmartNIC"}

try:
slice = fablib.new_slice(name=slice_name)
send_node = slice.add_node(**send_params)
send_iface = send_node.add_component(**send_nic_params).get_interfaces()[0]
send_iface.set_mode('auto')
recv_node = slice.add_node(**recv_params)
recv_iface = recv_node.add_component(**recv_nic_params).get_interfaces()[0]
recv_iface.set_mode('auto')
net1 = slice.add_l2network(name=network_name, interfaces=[send_iface, recv_iface], type='L2Bridge', subnet=IPv4Network("192.168.1.0/24"))
slice.submit()
except Exception as e:
print(e)

[Komal Thareja]

Worked with Hemil over a zoom meeting and was able to resolve the issue by renaming the bastion key in fabric_rc and re-executing the configure_and_validate.ipynb

Thanks,
Komal

[Author]

Posts