1. observing problems with IPv4Ext and IPv6Ext

observing problems with IPv4Ext and IPv6Ext

Home Forums FABRIC General Questions and Discussion observing problems with IPv4Ext and IPv6Ext

Viewing 9 posts - 1 through 9 (of 9 total)
  • Author
    Posts
  • #5005
    Gregory Daues
    Participant

      I am seeing problems with IPv4Ext and IPv6Ext;   I am following the examples

      https://github.com/fabric-testbed/jupyter-examples/blob/main/fabric_examples/fablib_api/create_l3network_fabnet_ipv6ext_manual/create_l3network_fabnet_ipv6ext_manual.ipynb
      https://github.com/fabric-testbed/jupyter-examples/blob/main/fabric_examples/fablib_api/create_l3network_fabnet_ipv4ext_manual/create_l3network_fabnet_ipv4ext_manual.ipynb

      The initial Slice creation is successful, the Slice reaches a StableOK state.
      I then try to

      # Enable Public IPv4 make_ip_publicly_routable
      network1.make_ip_publicly_routable(ipv4=[str(network1_available_ips[0])])
      slice.submit()

      that is, the re-submit phase, and the error reported is

      Waiting for slice . Slice state: ModifyOK
      Waiting for ssh in slice . ssh successful
      Running post boot config … Running post boot config threads …
      Saving fablib data… Exception: Failed to submit slice: Status.FAILURE, (500)
      Reason: INTERNAL SERVER ERROR
      HTTP response headers:
      HTTPHeaderDict({‘Server’: ‘nginx/1.21.6’, ‘Date’: ‘Mon, 14 Aug 2023 16:01:24 GMT’,
      ‘Content-Type’: ‘text/html; charset=utf-8’, ‘Content-Length’: ‘274’, ‘Connection’: ‘keep-alive’,
      ‘Access-Control-Allow-Credentials’: ‘true’, ‘Access-Control-Allow-Headers’:
      ‘DNT, User-Agent, X-Requested-With, If-Modified-Since, Cache-Control, Content-Type, Range, Authorization’,
      ‘Access-Control-Allow-Methods’: ‘GET, POST, PUT, PATCH, DELETE, OPTIONS’,
      ‘Access-Control-Allow-Origin’: ‘*’,
      ‘Access-Control-Expose-Headers’: ‘Content-Length, Content-Range, X-Error’,
      ‘X-Error’: ‘Unable to modify Slice# ea1653aa-e881-49a7-b917-6b4f6729493a
      that is not yet stable, try again later’})
      HTTP response body: b'{\n “errors”: [\n {\n
      “details”: “Unable to modify Slice# ea1653aa-e881-49a7-b917-6b4f6729493a
      that is not yet stable, try again later”,\n
      “message”: “Internal Server Error”\n }\n ],\n
      “size”: 1,\n “status”: 500,\n “type”: “error”\n}’

      Finally I see the Slice to rest in ModifyOK state.

      So there appears to be some issue of order of operations, etc

      Should IPv4Ext IPv6Ext be working at this time?

       

       

      #5011
      Komal Thareja
      Participant

        Hi Gregory,

        Which container are you using? I tried the Ipv4Ext notebook on the “Beyond Bleeding Edge” IPv6Ext from 1.5.3 container and did not see this issue. Also, if you run into this issue, please do not delete the slice. It will help to get the information for the slice for debugging.

        Also, please share the output of the following command from your container:
        pip list | grep fabric
        cat ~/work/fabric_config/requirements.txt

        Thanks,
        Komal

        #5019
        Gregory Daues
        Participant

          I had “the latest of each”

          ~> pip3 list | grep fabric
          fabric-credmgr-client 1.5.2
          fabric_fim 1.5.5
          fabric_fss_utils 1.5.1
          fabric-orchestrator-client 1.5.5
          fabrictestbed 1.5.6
          fabrictestbed-extensions 1.5.3

          but there was a warning about inconsistency (with “fabrictestbed-extensions 1.5.3 requires fabrictestbed==1.5.4, but you have fabrictestbed 1.5.6”) and so I’ll backtrack and try again.

           

          #5024
          Gregory Daues
          Participant

            With this set

            > pip3 list | grep fabric
            fabric-credmgr-client 1.5.1
            fabric_fim 1.5.4
            fabric_fss_utils 1.5.1
            fabric-orchestrator-client 1.5.3
            fabrictestbed 1.5.4
            fabrictestbed-extensions 1.5.3

            I see the issue to occur.   The Slice id  a5019147-99ad-4619-bb26-d468d9cfd82e   is still running.

            I had not used containers so I will look into that.

            #5025
            Komal Thareja
            Participant

              Thank you for sharing your slice details! Are you running this as a script?
              The difference in behavior could be because of the wait handling in Jupyter Notebook v/s a script.
              ModifyOK state indicates that the modify was successful.
              You can invoke: slice.update() to move the slice into StableOK state.

              Connectivity should work as well. Also, could you please share the script you are running so I can emulate this and address the difference in the behavior of the wait to avoid invoking explicit update.

              Thanks,
              Komal

              #5026
              Gregory Daues
              Participant

                The second script that runs against the existing Slice in StableOK is (just cutting lines from the example)

                import json
                import os
                import time
                import traceback
                
                from fabrictestbed_extensions.fablib.fablib import FablibManager as fablib_manager
                try:
                fablib = fablib_manager()
                fablib.show_config();
                except Exception as e:
                print(f"Exception: {e}")
                
                slice_name = 'MySliceAug14D'
                network1_name='net1'
                try:
                slice = fablib.get_slice(name=slice_name)
                network1 = slice.get_network(name=network1_name)
                network1_available_ips = network1.get_available_ips()
                # Enable Public IPv6 make_ip_publicly_routable
                network1.make_ip_publicly_routable(ipv4=[str(network1_available_ips[0])])
                slice.submit()
                
                except Exception as e:
                print(f"Exception: {e}")
                traceback.print_exc()
                #5027
                Gregory Daues
                Participant

                  I do not see that adding

                  slice.update()

                  within this script will affect the issue. Though  the slice.update() will change ModifyOk to StableOK,  the following resubmit seems to take actions to take things back to the current state  (“Unable to modify Slice# ea1653aa-e881-49a7-b917-6b4f6729493a that is not yet stable, try again later”).

                   

                  #5033
                  Gregory Daues
                  Participant

                    ok I originally misunderstood the slice.update() comments ;
                    Assuming that the “make_ip_publicly_routable”  was actually successful,  I’ve executed the slice.update()
                    and proceeded to add routes for the node/network,   and I will test that external access thru IPv4Ext is working,
                    As such,  the exception that occurs above is mostly a distraction then.

                    #5043
                    Komal Thareja
                    Participant

                      Thanks Gregory for reporting this. It was indeed a bug when running in script mode. The fix is now available on the main branch for https://github.com/fabric-testbed/fabrictestbed-extensions. The fix is also available in the “Beyond Bleeding Edge Container”

                      Once the new version for fabrictestbed-extensions==1.5.4 is pushed to pypi it would be available there as well.

                      Thanks,
                      Komal

                    Viewing 9 posts - 1 through 9 (of 9 total)
                    • You must be logged in to reply to this topic.