Forum Replies Created
-
Posts
-
There is a feature in fablib that allows for cpu/core pinining. That is as close as you can get to core exclusivity i guess.
Which SITE was this VM on ? Also, could you please share the notebook you used when you encountered the issue? you can email the notebook to help@fabric-testbed.net.
You are most likely running into the IPv6 vs IPv4 issue. The site you are trying to reach linux.mellanox.com is an IPv4-only site. FIU is an IPv4 site, which is why it works seamlessly there. Whereas DALL and SEAT are IPv6-only sites. Since you are using ip forwarding rule,s as you mentioned, you may have to put those rules in the ip6tables .
The FABRIC VMs use the FABRIC DNS server which have the NAT64 capability on these racks. So i believe the issue may possibly lie with your forwarding rules.
I see the issue on your VM. I believe you are using a FABNETv4 EXT and FABnetv6 EXT. During its configuration, you may have accidentally added the NIC to be used in the default route. This caused the system to have two default routes going out two different interfaces.
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc fq_codel state UP group default qlen 1000
link/ether fa:16:3e:07:66:5e brd ff:ff:ff:ff:ff:ff
inet 10.30.6.168/23 metric 100 brd 10.30.7.255 scope global dynamic enp3s0
valid_lft 58217sec preferred_lft 58217sec
inet6 2001:400:a100:3030:f816:3eff:fe07:665e/64 scope global dynamic mngtmpaddr noprefixroute
valid_lft 86383sec preferred_lft 14383sec
inet6 fe80::f816:3eff:fe07:665e/64 scope link
valid_lft forever preferred_lft forever
3: enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
link/ether 2a:77:c8:60:d3:bf brd ff:ff:ff:ff:ff:ff
inet 10.129.130.253/24 scope global enp7s0
valid_lft forever preferred_lft forever
inet 23.134.235.195/28 scope global enp7s0
valid_lft forever preferred_lft forever
inet6 2602:fcfb:101::3/28 scope global
valid_lft forever preferred_lft forever
inet6 2602:fcfb:101:0:2877:c8ff:fe60:d3bf/64 scope global dynamic mngtmpaddr
valid_lft 2591807sec preferred_lft 604607sec
inet6 fe80::2877:c8ff:fe60:d3bf/64 scope link
valid_lft forever preferred_lft forever
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether ca:03:f6:60:d5:34 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft foreverroot@SenderSTAR:~# ip -6 route show
::1 dev lo proto kernel metric 256 pref medium
2001:400:a100:3030::/64 dev enp3s0 proto ra metric 100 expires 86371sec pref medium
2001:400:a300::/48 via 2602:fcfb:101::1 dev enp7s0 metric 1024 pref medium
2602:fcfb:101::/64 dev enp7s0 proto kernel metric 256 expires 2591914sec pref medium
2602:fcf0::/28 dev enp7s0 proto kernel metric 256 pref medium
fe80::a9fe:a9fe via fe80::f816:3eff:fe79:edec dev enp3s0 proto ra metric 100 expires 271sec pref medium
fe80::/64 dev enp3s0 proto kernel metric 256 pref medium
fe80::/64 dev enp7s0 proto kernel metric 256 pref medium
default via fe80::f816:3eff:fe79:edec dev enp3s0 proto ra metric 100 expires 271sec mtu 9000 pref medium
default via fe80::c28b:2aff:fe82:6d02 dev enp7s0 proto ra metric 1024 expires 1714sec hoplimit 64 pref mediumAs soon as i disable the enp7s0 NIC using ip link set dev enp7s0 down , the VM started working via ssh.
It is possible that there is a routing issue when FABNETv6 is used at STAR with STAR Bastion. I will ask for this use case to be investigated.
Good news is that i have narrowed the issue down to your VM at STAR. Is the issue when using VMs at other sites too ?
Can you post the full ssh command? The issue may be the connection between STAR and the destination rack
