1. Authentication failed after hosts being in Active state

Authentication failed after hosts being in Active state

Home Forums FABRIC General Questions and Discussion Authentication failed after hosts being in Active state

Viewing 6 posts - 1 through 6 (of 6 total)
  • Author
    Posts
  • #2930
    Arash SARABI
    Participant

      Hello, I am experiencing an authentication failed error. I have not changed my notebook, and it was working fine a short while ago. Could you please check?

       

       ID                                    Name    Site    Host                          Cores    RAM    Disk  Image              Management IP                           State    Error
      ------------------------------------  ------  ------  --------------------------  -------  -----  ------  -----------------  --------------------------------------  -------  -------
      0f4f2cbb-a9ce-41ab-9999-6145b70e7ccd  client  STAR    star-w5.fabric-testbed.net        4     16     500  default_ubuntu_20  2001:400:a100:3030:f816:3eff:fe39:35f   Active
      2120e30c-6539-4e79-a210-844f72fd5596  server  STAR    star-w6.fabric-testbed.net        4     16     500  default_ubuntu_20  2001:400:a100:3030:f816:3eff:fec1:52ed  Active
      b3d6bdb0-8519-4e72-8c70-a07b815abe81  router  STAR    star-w6.fabric-testbed.net        2      8      10  default_ubuntu_20  2001:400:a100:3030:f816:3eff:fe45:3908  Active
      
      Time to stable 182 seconds
      Running post_boot_config ... 
      
      ---------------------------------------------------------------------------
      AuthenticationException                   Traceback (most recent call last)
      /tmp/ipykernel_53/869729168.py in <module>
           22 
           23 
      ---> 24 slice.submit()
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/slice.py in submit(self, wait, wait_timeout, wait_interval, progress, wait_jupyter)
         1222 
         1223         if progress and wait_jupyter == 'text' and self.fablib_manager.is_jupyter_notebook():
      -> 1224             self.wait_jupyter(timeout=wait_timeout, interval=wait_interval)
         1225             return self.slice_id
         1226 
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/slice.py in wait_jupyter(self, timeout, interval)
         1174 
         1175         print("Running post_boot_config ... ", end="")
      -> 1176         self.post_boot_config()
         1177         print(f"Time to post boot config {time.time() - start:.0f} seconds")
         1178 
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/slice.py in post_boot_config(self)
         1112 
         1113         for node_thread in node_threads:
      -> 1114             node_thread.result()
         1115 
         1116         for interface in self.get_interfaces():
      
      /opt/conda/lib/python3.9/concurrent/futures/_base.py in result(self, timeout)
          443                     raise CancelledError()
          444                 elif self._state == FINISHED:
      --> 445                     return self.__get_result()
          446                 else:
          447                     raise TimeoutError()
      
      /opt/conda/lib/python3.9/concurrent/futures/_base.py in __get_result(self)
          388         if self._exception:
          389             try:
      --> 390                 raise self._exception
          391             finally:
          392                 # Break a reference cycle with the exception in self._exception
      
      /opt/conda/lib/python3.9/concurrent/futures/thread.py in run(self)
           50 
           51         try:
      ---> 52             result = self.fn(*self.args, **self.kwargs)
           53         except BaseException as exc:
           54             self.future.set_exception(exc)
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py in network_manager_stop(self)
         1206         except Exception as e:
         1207             logging.warning(f"Failed to stop network manager: {e}")
      -> 1208             raise e
         1209 
         1210     def network_manager_start(self):
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py in network_manager_stop(self)
         1190             #             logging.info(f"No conn for device. conn: '{conn}'")
         1191 
      -> 1192             stdout, stderr = self.execute(f"sudo systemctl stop NetworkManager")
         1193             logging.info(f"Stopped NetworkManager with 'sudo systemctl stop "
         1194                          f"NetworkManager': stdout: {stdout}\nstderr: {stderr}")
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py in execute(self, command, retry, retry_interval, username, private_key_file, private_key_passphrase)
          724 
          725                 if attempt+1 == retry:
      --> 726                     raise e
          727 
          728                 #Fail, try again
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py in execute(self, command, retry, retry_interval, username, private_key_file, private_key_passphrase)
          682                 bastion=paramiko.SSHClient()
          683                 bastion.set_missing_host_key_policy(paramiko.AutoAddPolicy())
      --> 684                 bastion.connect(self.get_fablib_manager().get_bastion_public_addr(), username=bastion_username,
          685                                 key_filename=bastion_key_file)
          686 
      
      /opt/conda/lib/python3.9/site-packages/paramiko/client.py in connect(self, hostname, port, username, password, pkey, key_filename, timeout, allow_agent, look_for_keys, compress, sock, gss_auth, gss_kex, gss_deleg_creds, gss_host, banner_timeout, auth_timeout, gss_trust_dns, passphrase, disabled_algorithms)
          433             key_filenames = key_filename
          434 
      --> 435         self._auth(
          436             username,
          437             password,
      
      /opt/conda/lib/python3.9/site-packages/paramiko/client.py in _auth(self, username, password, pkey, key_filenames, allow_agent, look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host, passphrase)
          764         # if we got an auth-failed exception earlier, re-raise it
          765         if saved_exception is not None:
      --> 766             raise saved_exception
          767         raise SSHException("No authentication methods available")
          768 
      
      /opt/conda/lib/python3.9/site-packages/paramiko/client.py in _auth(self, username, password, pkey, key_filenames, allow_agent, look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host, passphrase)
          740                     # in ['password']
          741                     allowed_types = set(
      --> 742                         self._transport.auth_publickey(username, key)
          743                     )
          744                     two_factor = allowed_types & two_factor_types
      
      /opt/conda/lib/python3.9/site-packages/paramiko/transport.py in auth_publickey(self, username, key, event)
         1633             # caller wants to wait for event themselves
         1634             return []
      -> 1635         return self.auth_handler.wait_for_response(my_event)
         1636 
         1637     def auth_interactive(self, username, handler, submethods=""):
      
      /opt/conda/lib/python3.9/site-packages/paramiko/auth_handler.py in wait_for_response(self, event)
          257             if issubclass(e.__class__, PartialAuthentication):
          258                 return e.allowed_types
      --> 259             raise e
          260         return []
          261 
      
      AuthenticationException: Authentication failed. 

       

      #2932
      Arash SARABI
      Participant

        I’m assuming this has something to do with the bastion key, which has went missing from my portal.

        • This reply was modified 2 years, 3 months ago by Arash SARABI.
        #2934
        Arash SARABI
        Participant

          How can I get the slice ID?
          I know I can get the ID by using print(f"{slice}") but is there anyway to pass this automatically?

          slice = fablib.get_slice(SLICENAME,"something to get the ID")

          #2935
          Paul Ruth
          Keymaster

            Yeah, I think your bastion key expired.  They expire every 6 months.  Try creating another one and retry.

            Paul

            #2936
            Arash SARABI
            Participant

              Hi Paul,
              Thanks,
              Do you know how to get the slice ID?
              slice = fablib.get_slice(SLICENAME,"something to get the ID")

              #2937
              Paul Ruth
              Keymaster

                If you have the slice you can call:

                slice_id = slice.get_slice_id()

                In your example, you could do something like:

                slice_id = fablib.get_slice(SLICENAME).get_slice_id()

              Viewing 6 posts - 1 through 6 (of 6 total)
              • You must be logged in to reply to this topic.