1. Authentication failed after hosts being in Active state

Authentication failed after hosts being in Active state

Home › Forums › FABRIC General Questions and Discussion › Authentication failed after hosts being in Active state

Viewing 6 posts - 1 through 6 (of 6 total)
  • Author
    Posts
  • #2930
    Arash
    Participant

      Hello, I am experiencing an authentication failed error. I have not changed my notebook, and it was working fine a short while ago. Could you please check?

       

       ID                                    Name    Site    Host                          Cores    RAM    Disk  Image              Management IP                           State    Error
      ------------------------------------  ------  ------  --------------------------  -------  -----  ------  -----------------  --------------------------------------  -------  -------
      0f4f2cbb-a9ce-41ab-9999-6145b70e7ccd  client  STAR    star-w5.fabric-testbed.net        4     16     500  default_ubuntu_20  2001:400:a100:3030:f816:3eff:fe39:35f   Active
      2120e30c-6539-4e79-a210-844f72fd5596  server  STAR    star-w6.fabric-testbed.net        4     16     500  default_ubuntu_20  2001:400:a100:3030:f816:3eff:fec1:52ed  Active
      b3d6bdb0-8519-4e72-8c70-a07b815abe81  router  STAR    star-w6.fabric-testbed.net        2      8      10  default_ubuntu_20  2001:400:a100:3030:f816:3eff:fe45:3908  Active
      
      Time to stable 182 seconds
      Running post_boot_config ... 
      
      ---------------------------------------------------------------------------
      AuthenticationException                   Traceback (most recent call last)
      /tmp/ipykernel_53/869729168.py in <module>
           22 
           23 
      ---> 24 slice.submit()
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/slice.py in submit(self, wait, wait_timeout, wait_interval, progress, wait_jupyter)
         1222 
         1223         if progress and wait_jupyter == 'text' and self.fablib_manager.is_jupyter_notebook():
      -> 1224             self.wait_jupyter(timeout=wait_timeout, interval=wait_interval)
         1225             return self.slice_id
         1226 
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/slice.py in wait_jupyter(self, timeout, interval)
         1174 
         1175         print("Running post_boot_config ... ", end="")
      -> 1176         self.post_boot_config()
         1177         print(f"Time to post boot config {time.time() - start:.0f} seconds")
         1178 
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/slice.py in post_boot_config(self)
         1112 
         1113         for node_thread in node_threads:
      -> 1114             node_thread.result()
         1115 
         1116         for interface in self.get_interfaces():
      
      /opt/conda/lib/python3.9/concurrent/futures/_base.py in result(self, timeout)
          443                     raise CancelledError()
          444                 elif self._state == FINISHED:
      --> 445                     return self.__get_result()
          446                 else:
          447                     raise TimeoutError()
      
      /opt/conda/lib/python3.9/concurrent/futures/_base.py in __get_result(self)
          388         if self._exception:
          389             try:
      --> 390                 raise self._exception
          391             finally:
          392                 # Break a reference cycle with the exception in self._exception
      
      /opt/conda/lib/python3.9/concurrent/futures/thread.py in run(self)
           50 
           51         try:
      ---> 52             result = self.fn(*self.args, **self.kwargs)
           53         except BaseException as exc:
           54             self.future.set_exception(exc)
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py in network_manager_stop(self)
         1206         except Exception as e:
         1207             logging.warning(f"Failed to stop network manager: {e}")
      -> 1208             raise e
         1209 
         1210     def network_manager_start(self):
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py in network_manager_stop(self)
         1190             #             logging.info(f"No conn for device. conn: '{conn}'")
         1191 
      -> 1192             stdout, stderr = self.execute(f"sudo systemctl stop NetworkManager")
         1193             logging.info(f"Stopped NetworkManager with 'sudo systemctl stop "
         1194                          f"NetworkManager': stdout: {stdout}\nstderr: {stderr}")
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py in execute(self, command, retry, retry_interval, username, private_key_file, private_key_passphrase)
          724 
          725                 if attempt+1 == retry:
      --> 726                     raise e
          727 
          728                 #Fail, try again
      
      /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py in execute(self, command, retry, retry_interval, username, private_key_file, private_key_passphrase)
          682                 bastion=paramiko.SSHClient()
          683                 bastion.set_missing_host_key_policy(paramiko.AutoAddPolicy())
      --> 684                 bastion.connect(self.get_fablib_manager().get_bastion_public_addr(), username=bastion_username,
          685                                 key_filename=bastion_key_file)
          686 
      
      /opt/conda/lib/python3.9/site-packages/paramiko/client.py in connect(self, hostname, port, username, password, pkey, key_filename, timeout, allow_agent, look_for_keys, compress, sock, gss_auth, gss_kex, gss_deleg_creds, gss_host, banner_timeout, auth_timeout, gss_trust_dns, passphrase, disabled_algorithms)
          433             key_filenames = key_filename
          434 
      --> 435         self._auth(
          436             username,
          437             password,
      
      /opt/conda/lib/python3.9/site-packages/paramiko/client.py in _auth(self, username, password, pkey, key_filenames, allow_agent, look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host, passphrase)
          764         # if we got an auth-failed exception earlier, re-raise it
          765         if saved_exception is not None:
      --> 766             raise saved_exception
          767         raise SSHException("No authentication methods available")
          768 
      
      /opt/conda/lib/python3.9/site-packages/paramiko/client.py in _auth(self, username, password, pkey, key_filenames, allow_agent, look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host, passphrase)
          740                     # in ['password']
          741                     allowed_types = set(
      --> 742                         self._transport.auth_publickey(username, key)
          743                     )
          744                     two_factor = allowed_types & two_factor_types
      
      /opt/conda/lib/python3.9/site-packages/paramiko/transport.py in auth_publickey(self, username, key, event)
         1633             # caller wants to wait for event themselves
         1634             return []
      -> 1635         return self.auth_handler.wait_for_response(my_event)
         1636 
         1637     def auth_interactive(self, username, handler, submethods=""):
      
      /opt/conda/lib/python3.9/site-packages/paramiko/auth_handler.py in wait_for_response(self, event)
          257             if issubclass(e.__class__, PartialAuthentication):
          258                 return e.allowed_types
      --> 259             raise e
          260         return []
          261 
      
      AuthenticationException: Authentication failed. 

       

      #2932
      Arash
      Participant

        I’m assuming this has something to do with the bastion key, which has went missing from my portal.

        • This reply was modified 4 years, 1 month ago by Arash.
        #2934
        Arash
        Participant

          How can I get the slice ID?
          I know I can get the ID by using print(f"{slice}") but is there anyway to pass this automatically?

          slice = fablib.get_slice(SLICENAME,"something to get the ID")

          #2935
          Paul Ruth
          Keymaster

            Yeah, I think your bastion key expired.  They expire every 6 months.  Try creating another one and retry.

            Paul

            #2936
            Arash
            Participant

              Hi Paul,
              Thanks,
              Do you know how to get the slice ID?
              slice = fablib.get_slice(SLICENAME,"something to get the ID")

              #2937
              Paul Ruth
              Keymaster

                If you have the slice you can call:

                slice_id = slice.get_slice_id()

                In your example, you could do something like:

                slice_id = fablib.get_slice(SLICENAME).get_slice_id()

              Viewing 6 posts - 1 through 6 (of 6 total)
              • You must be logged in to reply to this topic.