Home › Forums › FABRIC General Questions and Discussion › Authentication failed after hosts being in Active state
- This topic has 5 replies, 2 voices, and was last updated 2 years, 3 months ago by Paul Ruth.
-
AuthorPosts
-
September 8, 2022 at 3:31 pm #2930
Hello, I am experiencing an authentication failed error. I have not changed my notebook, and it was working fine a short while ago. Could you please check?
ID Name Site Host Cores RAM Disk Image Management IP State Error ------------------------------------ ------ ------ -------------------------- ------- ----- ------ ----------------- -------------------------------------- ------- ------- 0f4f2cbb-a9ce-41ab-9999-6145b70e7ccd client STAR star-w5.fabric-testbed.net 4 16 500 default_ubuntu_20 2001:400:a100:3030:f816:3eff:fe39:35f Active 2120e30c-6539-4e79-a210-844f72fd5596 server STAR star-w6.fabric-testbed.net 4 16 500 default_ubuntu_20 2001:400:a100:3030:f816:3eff:fec1:52ed Active b3d6bdb0-8519-4e72-8c70-a07b815abe81 router STAR star-w6.fabric-testbed.net 2 8 10 default_ubuntu_20 2001:400:a100:3030:f816:3eff:fe45:3908 Active Time to stable 182 seconds Running post_boot_config ... --------------------------------------------------------------------------- AuthenticationException Traceback (most recent call last) /tmp/ipykernel_53/869729168.py in <module> 22 23 ---> 24 slice.submit() /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/slice.py in submit(self, wait, wait_timeout, wait_interval, progress, wait_jupyter) 1222 1223 if progress and wait_jupyter == 'text' and self.fablib_manager.is_jupyter_notebook(): -> 1224 self.wait_jupyter(timeout=wait_timeout, interval=wait_interval) 1225 return self.slice_id 1226 /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/slice.py in wait_jupyter(self, timeout, interval) 1174 1175 print("Running post_boot_config ... ", end="") -> 1176 self.post_boot_config() 1177 print(f"Time to post boot config {time.time() - start:.0f} seconds") 1178 /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/slice.py in post_boot_config(self) 1112 1113 for node_thread in node_threads: -> 1114 node_thread.result() 1115 1116 for interface in self.get_interfaces(): /opt/conda/lib/python3.9/concurrent/futures/_base.py in result(self, timeout) 443 raise CancelledError() 444 elif self._state == FINISHED: --> 445 return self.__get_result() 446 else: 447 raise TimeoutError() /opt/conda/lib/python3.9/concurrent/futures/_base.py in __get_result(self) 388 if self._exception: 389 try: --> 390 raise self._exception 391 finally: 392 # Break a reference cycle with the exception in self._exception /opt/conda/lib/python3.9/concurrent/futures/thread.py in run(self) 50 51 try: ---> 52 result = self.fn(*self.args, **self.kwargs) 53 except BaseException as exc: 54 self.future.set_exception(exc) /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py in network_manager_stop(self) 1206 except Exception as e: 1207 logging.warning(f"Failed to stop network manager: {e}") -> 1208 raise e 1209 1210 def network_manager_start(self): /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py in network_manager_stop(self) 1190 # logging.info(f"No conn for device. conn: '{conn}'") 1191 -> 1192 stdout, stderr = self.execute(f"sudo systemctl stop NetworkManager") 1193 logging.info(f"Stopped NetworkManager with 'sudo systemctl stop " 1194 f"NetworkManager': stdout: {stdout}\nstderr: {stderr}") /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py in execute(self, command, retry, retry_interval, username, private_key_file, private_key_passphrase) 724 725 if attempt+1 == retry: --> 726 raise e 727 728 #Fail, try again /opt/conda/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py in execute(self, command, retry, retry_interval, username, private_key_file, private_key_passphrase) 682 bastion=paramiko.SSHClient() 683 bastion.set_missing_host_key_policy(paramiko.AutoAddPolicy()) --> 684 bastion.connect(self.get_fablib_manager().get_bastion_public_addr(), username=bastion_username, 685 key_filename=bastion_key_file) 686 /opt/conda/lib/python3.9/site-packages/paramiko/client.py in connect(self, hostname, port, username, password, pkey, key_filename, timeout, allow_agent, look_for_keys, compress, sock, gss_auth, gss_kex, gss_deleg_creds, gss_host, banner_timeout, auth_timeout, gss_trust_dns, passphrase, disabled_algorithms) 433 key_filenames = key_filename 434 --> 435 self._auth( 436 username, 437 password, /opt/conda/lib/python3.9/site-packages/paramiko/client.py in _auth(self, username, password, pkey, key_filenames, allow_agent, look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host, passphrase) 764 # if we got an auth-failed exception earlier, re-raise it 765 if saved_exception is not None: --> 766 raise saved_exception 767 raise SSHException("No authentication methods available") 768 /opt/conda/lib/python3.9/site-packages/paramiko/client.py in _auth(self, username, password, pkey, key_filenames, allow_agent, look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host, passphrase) 740 # in ['password'] 741 allowed_types = set( --> 742 self._transport.auth_publickey(username, key) 743 ) 744 two_factor = allowed_types & two_factor_types /opt/conda/lib/python3.9/site-packages/paramiko/transport.py in auth_publickey(self, username, key, event) 1633 # caller wants to wait for event themselves 1634 return [] -> 1635 return self.auth_handler.wait_for_response(my_event) 1636 1637 def auth_interactive(self, username, handler, submethods=""): /opt/conda/lib/python3.9/site-packages/paramiko/auth_handler.py in wait_for_response(self, event) 257 if issubclass(e.__class__, PartialAuthentication): 258 return e.allowed_types --> 259 raise e 260 return [] 261 AuthenticationException: Authentication failed.
September 8, 2022 at 3:35 pm #2932I’m assuming this has something to do with the bastion key, which has went missing from my portal.
- This reply was modified 2 years, 3 months ago by Arash SARABI.
September 8, 2022 at 3:57 pm #2934How can I get the slice ID?
I know I can get the ID by usingprint(f"{slice}")
but is there anyway to pass this automatically?slice = fablib.get_slice(SLICENAME,"something to get the ID")
September 8, 2022 at 4:35 pm #2935Yeah, I think your bastion key expired. They expire every 6 months. Try creating another one and retry.
Paul
September 8, 2022 at 4:44 pm #2936Hi Paul,
Thanks,
Do you know how to get the slice ID?
slice = fablib.get_slice(SLICENAME,"something to get the ID")
September 8, 2022 at 5:16 pm #2937If you have the slice you can call:
slice_id = slice.get_slice_id()
In your example, you could do something like:
slice_id = fablib.get_slice(SLICENAME).get_slice_id()
-
AuthorPosts
- You must be logged in to reply to this topic.