Home › Forums › FABRIC General Questions and Discussion › TACC always failing with insufficient resources:Disk#
- This topic has 3 replies, 2 voices, and was last updated 5 months, 3 weeks ago by Komal Thareja.
-
AuthorPosts
-
June 5, 2024 at 2:02 pm #7056
I am trying to obtain VMs on TACC site with minimal resources; 2 cores, 8GB RAM and 10GB hard disk. However, the slice submission always fails with:
Slice Exception: Slice Name: Traffic Listening Demo TACC, Slice ID: 30f9fb42-37be-420f-899e-082a41bfb735: Slice Exception: Slice Name: Traffic Listening Demo TACC, Slice ID: 30f9fb42-37be-420f-899e-082a41bfb735: Node: TACC_node4, Site: TACC, State: Closed, Insufficient resources : [disk]#
Even if I submit the slice with a 1GB hard disk request, it fails with the above error. Can someone explain what this error means and how to fix it ?
June 5, 2024 at 2:33 pm #7058Hi Nishanth,
The error
Insufficient resources : [disk]
implies that there is not enough disk available on the host on which your VM is being requested. Looking at your slice, following VM requesting a ConnectX5 is being rejected as it maps totacc-w4
There is not enough disk available ontacc-w4
to accomodate your VM hence the failure.
Reservation ID: 478b2a91-5a02-4cf0-9bcd-de04c3b873ea Slice ID: 30f9fb42-37be-420f-899e-082a41bfb735
Resource Type: VM Notices: Reservation 478b2a91-5a02-4cf0-9bcd-de04c3b873ea (Slice Traffic Listening Demo TACC(30f9fb42-37be-420f-899e-082a41bfb735) Graph Id:a58b7bc7-55d6-42e9-b457-5a8a32ebebc9 Owner:nshyamkumar@iit.edu) is in state (Closed,None_) (Last ticket update: Insufficient resources : ['disk'])
Start: 2024-06-05 17:55:24 +0000 End: 2024-06-06 17:55:23 +0000 Requested End: 2024-06-06 17:55:23 +0000
Units: 1 State: Closed Pending State: None_
Predecessors
Sliver: {'node_id': '9a579143-79b2-44fb-bacb-e6a5db4da3bf', 'capacities': '{ core: 2 , ram: 8 G, disk: 1 G}', 'capacity_hints': '{ instance_type: fabric.c2.m8.d10}', 'image_ref': 'default_ubuntu_20', 'image_type': 'qcow2', 'name': 'TACC_node4', 'reservation_info': '{"reservation_id": "478b2a91-5a02-4cf0-9bcd-de04c3b873ea", "reservation_state": "Closed"}', 'site': 'TACC', 'type': 'VM', 'user_data': '{"fablib_data": {"instantiated": "False", "run_update_commands": "False", "post_boot_commands": [], "post_update_commands": []}}'}
Component: {'node_id': '670d117f-19ac-477b-bff7-36ac4e90107a', 'details': 'Mellanox ConnectX-5 Dual Port 10/25GbE', 'model': 'ConnectX-5', 'name': 'TACC_node4-pmnic_2', 'type': 'SmartNIC', 'user_data': '{}'}
NS: {'node_id': 'adeede90-a808-45a6-8e1e-8c8de7a4ee6e', 'layer': 'L2', 'name': 'TACC_node4-TACC_node4-pmnic_2-l2ovs', 'site': 'TACC', 'type': 'OVS'}
IFS: {'node_id': '2f9a52b4-3108-48f2-b0f9-e0ccd7716cdc', 'capacities': '{ bw: 25 Gbps, unit: 1 }', 'labels': '{ local_name: p1}', 'name': 'TACC_node4-pmnic_2-p1', 'type': 'DedicatedPort', 'user_data': '{"fablib_data": {"mode": "config"}}'}
IFS: {'node_id': 'b6c42c3e-a570-4ed1-b633-607e90777f34', 'capacities': '{ bw: 25 Gbps, unit: 1 }', 'labels': '{ local_name: p2}', 'name': 'TACC_node4-pmnic_2-p2', 'type': 'DedicatedPort', 'user_data': '{"fablib_data": {"mode": "config"}}'}
Thanks,
Komal
June 5, 2024 at 2:51 pm #7059Thanks, so it does indeed stand for disk space.
When I look at the graphical stats on the Fabric Portal, it mentions that TACC has 103263/107463 GB free (it may not be the latest info, but I don’t think it varies by much). How can I ask Fabric to assign my VM on an underlying server where there is enough hard disk space ?
June 5, 2024 at 2:59 pm #7060Portal currently display overall site TACC disk usage which is combined disk space on all the hosts at TACC. Control Framework determines possible candidate nodes for your VM on the basis of the resources requested.
Your slice is requesting for a VM with CX5, which in case of TACC are only available on
tacc-w4
and hence it is trying to allocate it ontacc-w4
but fails due to no disk space. I would recommend using a different site than TACC.Also, we are working on improving the Resource Usage display to show per Host level information.
Thanks,
Komal
-
AuthorPosts
- You must be logged in to reply to this topic.