Slice Configuration Failure

[Pilar Fernandez Gayol]

Hello,

When I try to create a new slice using I get an error that does not allow me to continue with the configuration of the slice.

The error that I get is posted below:

---------------------------------------------------------------------------
ChannelException                          Traceback (most recent call last)
Cell In[7], line 26
     24     continue
     25 if verbose: print(f'{hostname}:sudo ifconfig {ifaces[hostname]} del {ips[hostname]} && sudo ip -6 route del {subnet2} && sudo ip -6 route del {subnet1}')
---> 26 stdout, stderr = host.execute(f'sudo ifconfig {ifaces[hostname]} del {ips[hostname]} && sudo ip -6 route del {subnet2} && sudo ip -6 route del {subnet1}', quiet=True)
     27 if verbose: print(f'{hostname}:sudo ifconfig {ifaces[hostname]} add {ips[hostname]} up && sudo ip -6 route add {subnet2} dev {ifaces[hostname]} && sudo ip -6 route add {subnet1} dev {ifaces[hostname]}')
     28 stdout, stderr = host.execute(f'sudo ifconfig {ifaces[hostname]} add {ips[hostname]} up && sudo ip -6 route add {subnet2} dev {ifaces[hostname]} && sudo ip -6 route add {subnet1} dev {ifaces[hostname]}', quiet=True)

File /opt/conda/lib/python3.11/site-packages/fabrictestbed_extensions/fablib/node.py:1541, in Node.execute(self, command, retry, retry_interval, username, private_key_file, private_key_passphrase, quiet, read_timeout, timeout, output_file)
   1536 logging.warning(
   1537     f"Exception in node.execute() (attempt #{attempt} of {retry}): {e}"
   1538 )
   1540 if attempt + 1 == retry:
-> 1541     raise e
   1543 # Fail, try again
   1544 if self.get_fablib_manager().get_log_level() == logging.DEBUG:

File /opt/conda/lib/python3.11/site-packages/fabrictestbed_extensions/fablib/node.py:1401, in Node.execute(self, command, retry, retry_interval, username, private_key_file, private_key_passphrase, quiet, read_timeout, timeout, output_file)
   1394 bastion.connect(
   1395     self.get_fablib_manager().get_bastion_public_addr(),
   1396     username=bastion_username,
   1397     key_filename=bastion_key_file,
   1398 )
   1400 bastion_transport = bastion.get_transport()
-> 1401 bastion_channel = bastion_transport.open_channel(
   1402     "direct-tcpip", dest_addr, src_addr
   1403 )
   1405 client = paramiko.SSHClient()
   1406 # client.load_system_host_keys()
   1407 # client.set_missing_host_key_policy(paramiko.MissingHostKeyPolicy())

File /opt/conda/lib/python3.11/site-packages/paramiko/transport.py:1101, in Transport.open_channel(self, kind, dest_addr, src_addr, window_size, max_packet_size, timeout)
   1099 if e is None:
   1100     e = SSHException("Unable to open channel.")
-> 1101 raise e

ChannelException: ChannelException(2, 'Connect failed')

 

Thank you,

Pilar

 

[Hussam Nasir]

This is just a guess, but in your commands to delete IPs, you are deleting the IP/interface that you SSH into thus killing your SSH connection to the VM.

[Pilar Fernandez Gayol]

Hello Nasir,

Thanks for your reply. This is the code that is giving me an error. My UUID is 8109bb6a-7124-4a3c-8d81-b657a9ae09f9, and the slice ID is 2f76400a-be9a-4897-b613-46ddf91057c6. I am deleting the existing configuration so that I can add a new one. I will try without that step and check if it works that way.

The thing is, I have been using this code for over a month, and it had never failed until now, so I do not know if someone can get an idea of what’s happening and how to fix it by looking at both the code and the error.

In addition, sometimes it gives me an error on line 26 and other times on line 28 or 36 instead on line 24.

The code which I am refering to is:

ips = {}
ips[“h11”] = h11_ipv6
ips[“h12”] = h12_ipv6
ips[“h13”] = h13_ipv6
ips[“h21”] = h21_ipv6
ips[“h22”] = h22_ipv6
ifaces = {}
ifaces[“h11”] = h11_iface_name
ifaces[“h12”] = h12_iface_name
ifaces[“h13”] = h13_iface_name
ifaces[“h21”] = h21_iface_name
ifaces[“h22”] = h22_iface_name
nhop_macs = {}
nhop_macs[“h11”] = s1_macs[1]
nhop_macs[“h12”] = s1_macs[2]
nhop_macs[“h13”] = s1_macs[3]
nhop_macs[“h21”] = s2_macs[1]
nhop_macs[“h22”] = s2_macs[2]
switches = [s1, s2]
hosts = [h11, h12, h13, h21, h22, h23]
for host in hosts:
hostname = host.get_name()
if hostname == “h23”:
continue
if verbose: print(f'{hostname}:sudo ifconfig {ifaces[hostname]} del {ips[hostname]} && sudo ip -6 route del {subnet2} && sudo ip -6 route del {subnet1}’)
stdout, stderr = host.execute(f’sudo ifconfig {ifaces[hostname]} del {ips[hostname]} && sudo ip -6 route del {subnet2} && sudo ip -6 route del {subnet1}’, quiet=True)
if verbose: print(f'{hostname}:sudo ifconfig {ifaces[hostname]} add {ips[hostname]} up && sudo ip -6 route add {subnet2} dev {ifaces[hostname]} && sudo ip -6 route add {subnet1} dev {ifaces[hostname]}’)
stdout, stderr = host.execute(f’sudo ifconfig {ifaces[hostname]} add {ips[hostname]} up && sudo ip -6 route add {subnet2} dev {ifaces[hostname]} && sudo ip -6 route add {subnet1} dev {ifaces[hostname]}’, quiet=True)
for dest in hosts:
destname = dest.get_name()
if dest == host or destname == “h23”:
continue
if verbose: print(f'{hostname}:sudo ip -6 neigh del {ips[destname]} dev {ifaces[hostname]}’)
stdout, stderr = host.execute(f’sudo ip -6 neigh del {ips[destname]} dev {ifaces[hostname]}’, quiet=True)
if verbose: print(f'{hostname}:sudo ip -6 neigh add {ips[destname]} lladdr {nhop_macs[hostname]} dev {ifaces[hostname]}’)
stdout, stderr = host.execute(f’sudo ip -6 neigh add {ips[destname]} lladdr {nhop_macs[hostname]} dev {ifaces[hostname]}’, quiet=True)

 

And the new error that I get is:

 

---------------------------------------------------------------------------
ChannelException                          Traceback (most recent call last)
Cell In[3], line 28
     26 stdout, stderr = host.execute(f'sudo ifconfig {ifaces[hostname]} del {ips[hostname]} && sudo ip -6 route del {subnet2} && sudo ip -6 route del {subnet1}', quiet=True)
     27 if verbose: print(f'{hostname}:sudo ifconfig {ifaces[hostname]} add {ips[hostname]} up && sudo ip -6 route add {subnet2} dev {ifaces[hostname]} && sudo ip -6 route add {subnet1} dev {ifaces[hostname]}')
---> 28 stdout, stderr = host.execute(f'sudo ifconfig {ifaces[hostname]} add {ips[hostname]} up && sudo ip -6 route add {subnet2} dev {ifaces[hostname]} && sudo ip -6 route add {subnet1} dev {ifaces[hostname]}', quiet=True)
     29 for dest in hosts:
     30     destname = dest.get_name()

File /opt/conda/lib/python3.11/site-packages/fabrictestbed_extensions/fablib/node.py:1541, in Node.execute(self, command, retry, retry_interval, username, private_key_file, private_key_passphrase, quiet, read_timeout, timeout, output_file)
   1536 logging.warning(
   1537     f"Exception in node.execute() (attempt #{attempt} of {retry}): {e}"
   1538 )
   1540 if attempt + 1 == retry:
-> 1541     raise e
   1543 # Fail, try again
   1544 if self.get_fablib_manager().get_log_level() == logging.DEBUG:

File /opt/conda/lib/python3.11/site-packages/fabrictestbed_extensions/fablib/node.py:1401, in Node.execute(self, command, retry, retry_interval, username, private_key_file, private_key_passphrase, quiet, read_timeout, timeout, output_file)
   1394 bastion.connect(
   1395     self.get_fablib_manager().get_bastion_public_addr(),
   1396     username=bastion_username,
   1397     key_filename=bastion_key_file,
   1398 )
   1400 bastion_transport = bastion.get_transport()
-> 1401 bastion_channel = bastion_transport.open_channel(
   1402     "direct-tcpip", dest_addr, src_addr
   1403 )
   1405 client = paramiko.SSHClient()
   1406 # client.load_system_host_keys()
   1407 # client.set_missing_host_key_policy(paramiko.MissingHostKeyPolicy())

File /opt/conda/lib/python3.11/site-packages/paramiko/transport.py:1101, in Transport.open_channel(self, kind, dest_addr, src_addr, window_size, max_packet_size, timeout)
   1099 if e is None:
   1100     e = SSHException("Unable to open channel.")
-> 1101 raise e

ChannelException: ChannelException(2, 'Connect failed')

Thank you,

Pilar Fernandez

[Komal Thareja]

Hi Pilar,

Could you please check if your bastion keys are expired via on the Portal ->Experiments -> Manage SSH Keys -> Bastion Keys? If so, Please re-run the notebook jupyter-examples-rel1.6.1/configure_and_validate.ipynb

This should renew your bastion keys. Please try creating your slice again after this.

Thanks,

Komal

[Author]

Posts