get_pci_addr() usefulness

[yoursunny]

In FABlib v1.3.0, Component.get_pci_addr() function returns the PCI address of a component such as a network interface.
However, it appears that the returned PCI address refers to the PCI address on the host node.
Since the experimenter has no access to the host node, I think it’s more useful to return the PCI address within the KVM guest instead.
This would allow the experimenter to directly use this information in kernel bypass networking software such as Data Plane Development Kit, which refers to Ethernet adapters using its PCI address.

My script to demonstrate this problem:

import json
import time
from fabrictestbed_extensions.fablib.fablib import FablibManager as fablib_manager

fablib = fablib_manager()

slice = fablib.new_slice(name=f'demo-{int(time.time())}')
nodeA = slice.add_node(name='A', site='WASH', image='default_ubuntu_22')
intfA = nodeA.add_component(model='NIC_Basic', name='nic').get_interfaces()[0]
nodeB = slice.add_node(name='B', site='WASH', image='default_ubuntu_22')
intfB = nodeB.add_component(model='NIC_Basic', name='nic').get_interfaces()[0]
slice.add_l2network(name='l2', interfaces=[intfA, intfB], type='L2Bridge')
slice.submit()

slice = fablib.get_slice(name=slice.get_name())
nodeA = slice.get_node(name='A')
intfA = slice.get_interface(name='A-nic-p1')
nodeB = slice.get_node(name='B')
intfB = slice.get_interface(name='B-nic-p1')

slice.wait_ssh(progress=True)
print(nodeA.get_ssh_command())
print(nodeB.get_ssh_command())

def extract_pci_vlan(iplink, ifname):
link = [link for link in iplink if link['ifname']==ifname]
if len(link) != 1:
return None, None
link = link[0]
try:
if link['linkinfo']['info_kind'] == 'vlan' and link['linkinfo']['info_data']['protocol'] == '802.1Q':
pci, vlan = extract_pci_vlan(iplink, link['link'])
return pci, link['linkinfo']['info_data']['id']
except KeyError:
pass
return link['parentdev'], None

for intf in [intfA, intfB]:
print(f'Interface {intf.get_name()}')
print(f' FABlib PCI address {intf.get_component().get_pci_addr()} VLAN {intf.get_vlan()}')
stdout, stderr = intf.get_node().execute(f'ip -j -d link')
if stderr:
print(f' ip link error: {stderr}')
pci, vlan = extract_pci_vlan(json.loads(stdout), intf.get_os_interface())
print(f' ip link PCI address {pci} VLAN {vlan}')

slice.delete()

The output is:

Interface A-nic-p1
  FABlib PCI address 0000:a1:02.1 VLAN None
  ip link PCI address 0000:00:07.0 VLAN None
Interface B-nic-p1
  FABlib PCI address 0000:a1:0b.4 VLAN None
  ip link PCI address 0000:00:07.0 VLAN None

If I change the slice definition to use ConnectX-5 and L2PTP:

slice = fablib.new_slice(name=f'demo-{int(time.time())}')
nodeA = slice.add_node(name='A', site='DALL', image='default_ubuntu_22')
intfA = nodeA.add_component(model='NIC_ConnectX_5', name='nic').get_interfaces()[0]
nodeB = slice.add_node(name='B', site='TACC', image='default_ubuntu_22')
intfB = nodeB.add_component(model='NIC_ConnectX_5', name='nic').get_interfaces()[0]
slice.add_l2network(name='l2', interfaces=[intfA, intfB], type='L2PTP')
slice.submit()

The output is:

Interface A-nic-p1
  FABlib PCI address ['0000:41:00.0', '0000:41:00.1'] VLAN 100
  ip link PCI address 0000:00:07.0 VLAN 100
Interface B-nic-p1
  FABlib PCI address ['0000:a1:00.0', '0000:a1:00.1'] VLAN 100
  ip link PCI address 0000:00:07.0 VLAN 100

This output reveals another problem.
In the FABlib API documentation, get_pci_addr function is described to return a string.
However, in this output it’s returning a List[str] , not a string.

[Ilya Baldin]

Thanks for pointing this out – we have discussed augmenting this behavior. We will see if we can get this feature into the next release.

[Author]

Posts