Fabric slice creation stuck after creating 8 out of 10 nodes.

[Sadat Bin Faruque]

I am getting a issue while creating slice using 10 sites. After completion of 8 sites it remain stuck for 45-50 minutes and then show this screenshot(attached). It doesnt able to assign ips and interfaces on those 10 nodes. But previously it was not happened.  Below is my code snippet for creating slices.

 

from ipaddress import ip_address, IPv4Address, IPv6Address, IPv4Network, IPv6Network
import ipaddress
from fabrictestbed_extensions.fablib.fablib import FablibManager as fablib_manager
import json
import traceback
import datetime
import random
fablib = fablib_manager()

 

# Number of nodes in the cluster.
num_nodes=10

# Give a cluster name
slice_name=’cassandra_mul_sites_2_4′

# Site name, pick one site from the above list of resources.
sites = [‘CLEM’,’LOSA’, ‘WASH’,’STAR’,’UTAH’,’CLEM’,’LOSA’, ‘WASH’,’STAR’,’UTAH’]

# LEGACY VARIABLES
master_gpu=False

# Operation system, Linux distribution e.g. default_ubuntu_18, default_ubuntu_20, etc.
image=’default_ubuntu_20′

 

node_names=[]
nic_names=[]
iface_names=[]
network_names=[]

for i in range(1,num_nodes+1):
node_names.append(“node{0}”.format(i))
nic_names.append(“nic{0}”.format(i))
iface_names.append(“iface{0}”.format(i))
network_names.append(“net{0}”.format(i))

print(node_names)
print(nic_names)
print(iface_names)

try:
slice = fablib.new_slice(name=slice_name)

# Use dictionaries to store nodes, interfaces, and networks
nodes = {}
interfaces = {}
networks = {}

# Create networks
for i in range(1, num_nodes + 1):
networks = slice.add_l3network(name=network_names[i-1], type=’IPv4′)

# Create nodes and configure them
for i in range(1, num_nodes + 1):
node_name = node_names[i-1] # Ensure correct indexing
nodes = slice.add_node(name=node_name, site=sites[i-1],cores=2,
ram=4,
disk=40,
image=image)

# Add NIC component
interfaces = nodes.add_component(model=’NIC_Basic’, name=f’nic{i}’).get_interfaces()[0]
interfaces.set_mode(‘auto’)

# Attach NIC to the network
networks.add_interface(interfaces)

# Add route for the node
nodes.add_route(subnet=fablib.FABNETV4_SUBNET, next_hop=networks.get_gateway())

# Submit the slice request
slice.submit()

except Exception as e:
print(f’Exception: {e}’)

[Komal Thareja]

Hi Sadat,

Could you please provide following information?

• Slice ID
• Verify and report the status of the Slice from Portal via Experiments -> My Slices
• Verify and report if any errors observed in /tmp/fablib/fablib.log from JH container.
• Verify Bastion SSH Keys are not expired
  • Check via portal Experiments -> Manage SSH Keys -> Bastion Key
  • In JH Container – Run jupyter-examples-rel1.8*/configure_and_validate.ipynb – this shall renew your expired keys
  • Please try your slice again if your Bastion Keys are expired.

Thanks,

Komal

• This reply was modified 3 weeks ago by Komal Thareja.

[Sadat Bin Faruque]

Thank you so much. It works!

[Author]

Posts