bastion ssh login

[GIORGIO SCORZELLI]

Hi Fabric Team,

I was able to configure and create my first slice.

First minor thing: following instructions of the Jupyter Notebook configure.ipynb , I got this error message zip command not found. I solved the issue by running the following:

conda install zip

Also, I created this simplified ssh config file:

Host fabric-bastion
  Hostname bastion.fabric-testbed.net
  User scrgiorgio_0039551217
  ForwardAgent yes
  IdentityFile ~/.nsdf/vault/fabric/id_nsdf_fabric_bastion
  IdentitiesOnly yes
  StrictHostKeyChecking no
  ServerAliveInterval 120

Host fabric-slice
  Hostname 2620:103:a006:12:f816:3eff:fe70:52dc
  User rocky
  ForwardAgent yes
  ProxyJump fabric-bastion
  IdentityFile ~/.nsdf/vault/fabric/id_nsdf_fabric_slice
  StrictHostKeyChecking no

 

The ssh fabric-slice  works perfectly from a Fabric Jupyter Lab Terminal.

But it stuck if run from the “outside”. Debugging by ssh -vvvvv fabric-slice, I see the problem is that (*) from inside Fabric, the bastion node solves to 152.54.15.121; (*) from the outside, it solves to 128.163.180.149.

This is the log when it stuck:

ssh -vvvvvvv fabric-slice
OpenSSH_8.2p1 Ubuntu-4ubuntu0.4, OpenSSL 1.1.1f 31 Mar 2020
debug1: Reading configuration data /home/scrgiorgio/.ssh/config
debug1: /home/scrgiorgio/.ssh/config line 347: Applying options for fabric-slice
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug2: resolve_canonicalize: hostname 2620:103:a006:12:f816:3eff:fe70:52dc is address
debug1: Setting implicit ProxyCommand from ProxyJump: ssh -vvv -W '[%h]:%p' fabric-bastion
debug1: Executing proxy command: exec ssh -vvv -W '[2620:103:a006:12:f816:3eff:fe70:52dc]:22' fabric-bastion
debug1: identity file /home/scrgiorgio/.nsdf/vault/fabric/id_nsdf_fabric_slice type 2
debug1: identity file /home/scrgiorgio/.nsdf/vault/fabric/id_nsdf_fabric_slice-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.4
OpenSSH_8.2p1 Ubuntu-4ubuntu0.4, OpenSSL 1.1.1f 31 Mar 2020
debug1: Reading configuration data /home/scrgiorgio/.ssh/config
debug1: /home/scrgiorgio/.ssh/config line 338: Applying options for fabric-bastion
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug2: resolving "bastion.fabric-testbed.net" port 22
debug2: ssh_connect_direct
debug1: Connecting to bastion.fabric-testbed.net [128.163.180.149] port 22.
debug1: Connection established.
debug1: identity file /home/scrgiorgio/.nsdf/vault/fabric/id_nsdf_fabric_bastion type 2
debug1: identity file /home/scrgiorgio/.nsdf/vault/fabric/id_nsdf_fabric_bastion-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0
debug1: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to bastion.fabric-testbed.net:22 as 'scrgiorgio_0039551217'
debug3: hostkeys_foreach: reading file "/home/scrgiorgio/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/scrgiorgio/.ssh/known_hosts:38
debug3: load_hostkeys: loaded 1 keys from bastion.fabric-testbed.net
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
Connection closed by 128.163.180.149 port 22
kex_exchange_identification: Connection closed by remote host

As you see, it’s not getting any reply to the SSH2_MSG_KEXINIT message.

Setting explicitly the bastion hostname to the working IP4 address 152.54.15.121fixes the problem, and I can always connect.

Thanks. G.

 

 

[Ilya Baldin]

Hello,

A couple of things:

1. Probably not the best strategy to create a fabric ssh config that includes your sliver address – this address will change from slice to slice and you don’t want to be editing the config file every time, but this may be a personal preference.

2. Jupyter Hub is ‘outside’ of FABRIC just like your laptop – it is hosted in Google Cloud

3. FABRIC has multiple bastion hosts (currently two with the two addresses you noted) which round-robin resolve to the name bastion.fabric-testbed.net. One of them apparently is giving you trouble – not clear why – I can see your key is properly installed there. We will look into it, thank you for reporting.

 

[Mert Cevik]

There may be multiple issues that we will be working on.

In order to eliminate the possibility of an issue with the “outside” IP address that you’re using, can you please send your IP address?

[GIORGIO SCORZELLI]

Hi Ilya and Mert,

I created a new sliver (2001:18e8:fff0:3:f816:…) and now specifying 128.163.180.149, or 128.163.180.149, or bastion.fabric-testbed.net; they all work. Probably my IP address (curl ifconfig.me) probably changed since I do not have a static IP address.

I will let you know if it happens again with full logs and new public IP.

Thanks for the help.

PS I totally agree about Ilya’s point (1).

 

[Ilya Baldin]

Giorgio,

We have been observing some instability in the bastion hosts – thank you for bringing it to our attention (you happen to have run into it into one of them but it appears both are affected). We think we know the issue and will update their configuration to avoid problems of this kind.

[Author]

Posts