1. Komal Thareja

Komal Thareja

Forum Replies Created

Viewing 15 posts - 391 through 405 (of 442 total)
  • Author
    Posts
  • in reply to: Cannot create slice, error:redeem predecessor reservation #3937
    Komal Thareja
    Participant

      The closed/dead slices do not have any resources associated with them. They get cleared periodically. Slices Dead/Closing older than 30 days are removed automatically.

      I was just trying to suggest that when your slice closes, it could be helpful to see a failure reason on the portal to modify the request accordingly as in this case ucsd-w9.fabric-testbed.net cannot accomodate the VMs requested. So altering your slice request would be helpful.

      Thanks,
      Komal

      in reply to: Cannot create slice, error:redeem predecessor reservation #3934
      Komal Thareja
      Participant

        Hi Manas,

        Could you please look for Closed/Dead slices on the portal to check the failure reason?

        I looked some of the VMs requested by your slice were in Failed state with “Insufficient resources error”. Since VMs could not be provisioned, network is closed as the VMs which are pre-requisite for it are in failed state.

        Reservation ID: ebb64ccb-476f-4373-8876-17af819df8c6 Slice ID: 9f998c48-6f4f-4af7-be33-337e94f7afb7
        Resource Type: VM Notices: Reservation ebb64ccb-476f-4373-8876-17af819df8c6 (Slice cluster_gatk(9f998c48-6f4f-4af7-be33-337e94f7afb7) Graph Id:116fe360-7f45-4384-9a15-2aec138c4519 Owner:mjdbz4@health.missouri.edu) is in state (Closed,None_)  (Last ticket update: Insufficient Resources: ucsd-w9.fabric-testbed.net cannot serve the requested sliver)

        Thanks,
        Komal

        Komal Thareja
        Participant

          Maintenance is complete! Testbed is ready for use again!

          Komal Thareja
          Participant

            We are still working on updating software. Apologies for the delay, we will notify once the maintenance is completed and testbed is available for use!

            Thanks,
            Komal

            Komal Thareja
            Participant

              We have observed sometimes stale cookies in the browser result in this error. Generally opening in an incognito window clear this issue. We are looking at ways to clear stale cookies to avoid this to improve user experience.

              We will keep you posted once a fix is deployed. Appreciate your feedback in helping us improve the testbed.

              Thanks,
              Komal

              Komal Thareja
              Participant

                Hi Gregory,

                Could you please try to download the tokens by opening the portal in an incognito window?

                Thanks,
                Komal

                Komal Thareja
                Participant

                  Maintenance is complete.

                  Komal Thareja
                  Participant

                    Maintenance is complete. Testbed is back online and ready to use.

                    Komal Thareja
                    Participant

                      Hi Nagmat,

                      We are currently under maintenance and will send out an update once the issue is resolved.

                      Thanks,
                      Komal

                      Maintenance on FABRIC-Testbed – 02/09/2022 (11:00am-1:00pm EST)

                      Komal Thareja
                      Participant

                        Hello Nagmat,

                        Could you please try creating your slice again?

                        The error you are observing is because VM creation failed before other resources could be requested to be provisioned. In this scenario, orchestrator triggers a Close for the rest of the resources and results: TicketReviewPolicy: Closing reservation due to failure in slice.

                        VM creation failed on UCSD because of PCI attach/detach due to a leaked VM. I have resolved that. Could you please try your slice again?

                        Thanks,
                        Komal

                        in reply to: Unable to allocate resources after the updates/maintenance. #3752
                        Komal Thareja
                        Participant

                          @Manas – Is it possible for you to span your slice across multiple sites instead of single site and use other network services FabNet or L2STS or L2PTP?

                          In the meanwhile, I will discuss this within our team and share our approach forward.

                          in reply to: Unable to allocate resources after the updates/maintenance. #3747
                          Komal Thareja
                          Participant

                            @Manas – It looks like you are requesting: Cores=20, RAM=128GB, Disk=2000GB. Even though we may have an overall storage available to account for such a request, some of your VMs are causing the Worker to be exhausted. Specifically Disk requested cannot be served.

                            I was able to create this slice with Disk usage set to 500GB instead on SALT but your storage volume doesn’t exist there. Could you please try this slice with lower disk?

                            in reply to: Unable to allocate resources after the updates/maintenance. #3735
                            Komal Thareja
                            Participant
                              in reply to: Unable to allocate resources after the updates/maintenance. #3733
                              Komal Thareja
                              Participant

                                Hello,

                                I was debugging this and noticed that VMs requested by your slice are being mapped to capacities indicated below which seem to exhaust the worker where your request lands. The capacity mapping from requested to allocated seems strange.
                                I am unable to reproduce this. Could you please share your notebook?

                                Also, could you please share the output of command: pip3 list | grep fabric from your environment?
                                We expect the following versions to be present.

                                
                                fabric                        3.0.0
                                fabric-credmgr-client         1.3.2
                                fabric-fim                    1.4.2
                                fabric-fss-utils              1.4.0
                                fabric-orchestrator-client    1.4.3
                                fabrictestbed                 1.4.3
                                fabrictestbed-extensions      1.4.0
                                

                                Capacity Allocations for your slice: cluster_gatk(d8bfce5c-c721-4d41-a7fa-e8d658a40a43)

                                
                                'capacities': '{ core: 2 , ram: 8 G, disk: 10 G}',                      ===> Requested by User
                                'capacity_allocations': '{ core: 16 , ram: 128 G, disk: 500 G}',        ===> Allocated by Orchestrator
                                'capacity_hints': '{ instance_type: fabric.c16.m128.d500}'
                                

                                Thanks,
                                Komal

                                • This reply was modified 2 years, 4 months ago by Komal Thareja.
                                in reply to: Importing the plugins #3686
                                Komal Thareja
                                Participant

                                  Could you please verify from the Portal, that project for which storage volumes were created has the Tag: Component.Storage available? Also, Please make sure you are using the same project ID in JH.

                                  If your project does not have the tag: Component.Storage, Please raise a request to enable that for your project.

                                Viewing 15 posts - 391 through 405 (of 442 total)