Home › Forums › FABRIC General Questions and Discussion › SSH Key authenticating error
- This topic has 10 replies, 7 voices, and was last updated 3 months, 3 weeks ago by Mert Cevik.
-
AuthorPosts
-
July 23, 2024 at 11:59 am #7284
Hi, I am new to FABRIC.
I have generated a public private key pair in the FABRIC portal. Then I upload it to the jupyter hub, under /home/fabric/work/fabric_config
I have set the private key to the correct permission.
fabric@spring:work-11%$ ls -l fabric_config/fabric_bastion_key
-rw——- 1 fabric users 492 Jul 23 14:48 fabric_config/fabric_bastion_keyand I have the correct ssh_config file under the same folder:
UserKnownHostsFile /dev/null
StrictHostKeyChecking no
ServerAliveInterval 120Host bastion.fabric-testbed.net
User yao_peter_0000204996
ForwardAgent yes
Hostname %h
IdentityFile /home/fabric/work/fabric_config/fabric_bastion_key
IdentitiesOnly yesHost * !bastion.fabric-testbed.net
ProxyJump yao_peter_0000204996@bastion.fabric-testbed.net:22I can see the right key created in the portal, and the finger print matches.
However, the authentication still fails.
fabric@spring:work-11%$ ssh -i fabric_config/fabric_bastion_key -C2T -D 14000 -M -N yao_peter_0000204996@bastion.fabric-testbed.net
yao_peter_0000204996@bastion.fabric-testbed.net: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).Do you have ideas about what might be wrong? Many thanks in advance!
July 24, 2024 at 10:00 am #7308Hi,
Could you please try running this notebook?
jupyter-examples-rel1.6.1/configure_and_validate.ipynb
This will validate your configuration and update it if needed.
Please try running the
Hello Fabirc Notebook
after that. This notebook creates a VM and also displays the SSH command to use to login into the VM. Please check if the SSH command works and let us know if the issue still persists.NOTE: Bastion host is only used as a jump box, we do not allow login to Bastion Nodes.
Thanks,
Komal
- This reply was modified 4 months ago by Komal Thareja.
July 24, 2024 at 10:18 am #7310Re:
NOTE: Bastion host is only used as a jump box, we do not allow login to Bastion Nodes.
the knowledge base says we can test the bastion host login using
ssh -i ~/.ssh/fabric_bastion -C2T -D 14000 -M -N username_0123456789@bastion.fabric-testbed.net
is this no longer current guidance?
July 24, 2024 at 10:22 am #7311Sorry for the confusion, Users can still validate their keys against the bastion host using the command above as indicated on the learn article.
Thanks,
KomalJuly 24, 2024 at 3:27 pm #7316I an trying to test the coneection from my own laptop to the bastion.
Here is what I tried:
- Generate a key pair from the fabric protal
- download the public private key and fix the permission on the private key to 600
- I did the bastion host login test using that key
The screenshot attached showed that the key permission is correct and the mD5 sum of the key matches what is shown on the portal and that I am using the correct bastion user name. Yet I get a permission denied error.
I am also sharing the verbose ssh output below:
(base) root@peter:~/.ssh# ssh -i ./fabric_bastion_key -C2T -D 14000 -M -N yao_peter_0000204996@bastion.fabric-testbed.net -v OpenSSH_8.9p1 Ubuntu-3ubuntu0.10, OpenSSL 3.0.2 15 Mar 2022 debug1: Reading configuration data /root/.ssh/config debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files debug1: /etc/ssh/ssh_config line 21: Applying options for * debug1: Connecting to bastion.fabric-testbed.net [128.163.180.149] port 22. debug1: Connection established. debug1: identity file ./fabric_bastion_key type 2 debug1: identity file ./fabric_bastion_key-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.10 debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0 debug1: compat_banner: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000 debug1: Authenticating to bastion.fabric-testbed.net:22 as 'yao_peter_0000204996' debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: algorithm: curve25519-sha256@libssh.org debug1: kex: host key algorithm: ssh-ed25519 debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug1: SSH2_MSG_KEX_ECDH_REPLY received debug1: Server host key: ssh-ed25519 SHA256:8ZrrCmYlgVJciScTwz3sDTsV3MV0kvcwtTgSOGYGwzg debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory debug1: Host 'bastion.fabric-testbed.net' is known and matches the ED25519 host key. debug1: Found key in /root/.ssh/known_hosts:37 debug1: ssh_packet_send2_wrapped: resetting send seqnr 3 debug1: rekey out after 134217728 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: ssh_packet_read_poll2: resetting read seqnr 3 debug1: SSH2_MSG_NEWKEYS received debug1: rekey in after 134217728 blocks debug1: Will attempt key: ./fabric_bastion_key ECDSA SHA256:+GCNKCQJR/hXBrJx79dAZh7AmmTeGGPVZtM7IHT8N5Y explicit debug1: SSH2_MSG_EXT_INFO received debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521> debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug1: No credentials were supplied, or the credentials were unavailable or inaccessible No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_0) debug1: No credentials were supplied, or the credentials were unavailable or inaccessible No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_0) debug1: Next authentication method: publickey debug1: Offering public key: ./fabric_bastion_key ECDSA SHA256:+GCNKCQJR/hXBrJx79dAZh7AmmTeGGPVZtM7IHT8N5Y explicit debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: No more authentication methods to try. yao_peter_0000204996@bastion.fabric-testbed.net: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
Please give me some advice, many thanks.
July 24, 2024 at 5:06 pm #7318I’m seeing the same problem on my end. I’ve created a key pair for bastion in the portal, and made sure that the permissions are 600 for the private key.
From my understanding there is load balancing happening with the bastion servers when using bastion.fabric-testbed.net. Is it possible for us to test on different bastion servers, or explicitly use a specific bastion server?
July 24, 2024 at 5:15 pm #7319I am similarly experiencing this issue. I created a fresh bastion key, ran configure_and_validate which confirmed the key was valid, and my verbose printout is similar to the above. Within Jupyter this is causing slice creation post_boot configuration to fail due to authentication errors.
July 24, 2024 at 6:12 pm #7320We are working on this and we will provide an update.
July 29, 2024 at 12:59 pm #7328Hi,
I am also facing similar issue. I tried to refresh key on WebUI and run validate (which outputs all is OK) and ssh again – getting similar problem as above to test bastion connection:
ssh -i /home/fabric/work/fabric_config/fabric-bastion-key -C2T -D 14000 -M -N jbalcas_0000188368@bastion.fabric-testbed.net -vvv
…
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic
debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug1: No credentials were supplied, or the credentials were unavailable or inaccessible
No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_1000)debug1: No credentials were supplied, or the credentials were unavailable or inaccessible
No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_1000)debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/fabric/.ssh/id_rsa RSA SHA256:NOtxkwnls/LzNSWNu5Y3oLsTsNVo97FwGGmFxVS9td4 agent
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Offering public key: /home/fabric/work/fabric_config/fabric-bastion-key ECDSA SHA256:3QIPXPoP5Shg2M+0PRksuF+Yvi1z1hWht9vIdL46fWI explicit
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
jbalcas_0000188368@bastion.fabric-testbed.net: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).July 29, 2024 at 3:24 pm #7330We are still working on a resolution for this issue. Bastion key for the user id ‘jbalcas_0000188368″ could not be ingested into the bastion hosts yet (along with others posted on this thread). Once the issue is resolved, all bastion keys will be pushed in to the bastion hosts. Updates will be posted.
July 29, 2024 at 6:32 pm #7332Problem with the SSH bastion keys is resolved. All of the new keys are added to the bastion hosts. Please, try your logins and let us know it you have any problems.
-
AuthorPosts
- You must be logged in to reply to this topic.