Unable to create a slice : redeem predecessor reservation

[Khawar Shehzad]

Hello,

Im trying to create a slice but keep running into this error
“redeem predecessor reservation# 3d2e4608-d116-4268-b72e-018cf47b314f is in a terminal state, failing the reservation# 77fbd8ce-e477-45cf-a987-194cac15f5bc#”

Moreover, It is also throwing me error for insufficient resources despite the fact that I’m trying to provision the slice in a location which has the resources available. Im using the slice builder to create the slice and using the following code snippet to get the details of available resources.

error : “”” Insufficient resources: No candidates nodes found to serve res: #3d2e4608-d116-4268-b72e-018cf47b314f slice: [4gpus(1ba91f16-f086-4146-833d-021394feb44f) Owner:khawar.shehzad@missouri.edu] state:[Nascent,Ticketing] # “”””

code snippet :
“””
try:
resources = fablib.list_sites()
except Exception as e:
print(f”Exception: {e}”)
“””

At the moment I’m provisioning a 2 node cluster at UTAH with following config for each node:
cpu : 20 cores
ram : 128gb
gpu : 2 rtx6000

Available resources using the above provided code snippet for UTAH are:
cpu : 404 cores available
ram : 1578 available (i assume its in GBs)
gpu : 6 rtx6000 available

I’m running into the same errors on other sites too.

Could you please have a look. Thanks.

Best,
Khawar

[Komal Thareja]

Hello Khawar,

Your slice is requesting 2 VMs. This is unsupported configuration. On UTAH, we have two hosts each with 3 GPUs but none of them have a dedicated CX-6. So your slice configuration is seemed unsupported.

Also, I checked, all 6 RTX-6000 GPUs are in use. Please note that the resource usage displayed on the portal may be outdated by 30 minutes.

• n1 – with RTX-6000 GPU and dedicated NIC CX-6
• n2 – with two RTX-600 GPU

We do have ongoing work for users to identify such invalid slice configurations using fablib API. This should be available soon with the upcoming Release 1.7. We also plan to provide host level resource usage details to user in 1.7 that may help with this too. Hope this helps!

Thanks,

Komal

[Komal Thareja]

To clarify, requesting two VMs is acceptable. However, requesting VMs with GPUs and SmartNICs in the mentioned slice is invalid because none of the hosts have SmartNICs and GPUs available on the same host.

[Author]

Posts