Warning: L2STS service does not support hairpins

[Brandon Rice]

Hi all,

I came across the following warning when running slice.submit() following I think this commit to fablib:

WARNING: Current implementation of L2STS service does not support hairpins (connections withing the same physical port), if your VMs are assigned to the same worker node, communications between them over this service will not be possible! We recommend not using L2STS with shared ports unless you know what you are doing.

The code of interest (I think) is, (copied and edited from “create_l2network_basic.ipynb”):

client_iface = client_node.add_component(model='NIC_Basic', name=client_name+"_nic").get_interfaces()[0]

server_node = slice.add_node(name=server_name, site=server_site)

server_iface = server_node.add_component(model='NIC_Basic', name=server_name+"_nic").get_interfaces()[0]

network = slice.add_l2network(name=network_name, interfaces=[client_iface, server_iface])

Also, it seems that with this warning, node.upload_file() now gives a error:

Fail: cannot unpack non-iterable SFTPAttributes object

although the file seems to upload to the node correctly.

 

What is different or is now considered bad practice and how should we now use slice.add_l2network() to connect multiple computers in a network?

• This topic was modified 2 years, 7 months ago by Brandon Rice.

[Paul Ruth]

Re: the unpacking error with file upload:  That is because file upload returns single string that describes the file attributes. It’s very common to assume that it returns a tuple of stdout, stderr like the execute method does… but upload is not a command so there is no stdout,stderr.

The error you are seeing is when you do this:

stdout,stderr = node.upload(...)

It will try to unpack the single string to the tuple stdout,stderr but it fails. Of course, this happens upon processing the return value, so the upload has already happened. Instead do something like:

file_attributes = node.upload(...)

Re: the warning about hairpins: This is a low-level limitation of using the wide-area (aka Site-to-site) networks with Basic NICs (which are implemented as SR-IOV virtual functions).  The problem is that if you have a S2S network with multiple Basic NICs that end up on the same physical host, the physical switch can’t connect the two Basic NICs because they are using the same port on the switch (i.e. it can’t create a hairpin on the port).

This warning is printed in a low level of the library and FABLib cannot suppress it (I’ve asked for it to be removed and I think it will be soon).   There are several cases where you can be sure there will be no problems.

• It will be fine if you only have on Basic NIC per site. Often this is because you only have two Basic NICs, one on either end of a WAN link connecting two routers or switches (this is your case).
• Alternatively, you can explicitly choose different host for each node with the node.set_host().    This will work, but can be tricky because there are a limited number hosts per site and you need to ensure that the host you pick has any other components types you might have added you your node.
• Also, you could just use dedicated NICs, but I wouldn’t recommend that if you can avoid it.   There are not that many and you will likely have issues finding sites with enough available for your experiment.

Ultimately,  FABlib should be able to help automatically picking hosts or at least testing the request to see if it should will work. Right now, that warning appears any time you have a Basic NIC on a STS network… even if you know its fine.

Its worth mentioning here that this is not an issue with networks that are completely in one site (aka L2Bridges).  These are implemented a slightly different way which allows for hairpins.

Basically, the suggestion is that you should try not to put a lot of nodes on one S2S link unless you really need to. Most real word scenarios don’t really work that way anyway.

I have a notebook I wrote with you mind that shows how to deploy a set of routers at a few sites… I’ll set you know when it is ready.

• This reply was modified 2 years, 7 months ago by Paul Ruth.
• This reply was modified 2 years, 7 months ago by Paul Ruth.

[Author]

Posts