Nodes in the same slice using FABNetv6 cannot reach each other

[Yifan Cai]

Hello,

 

I am new to FABRIC, and I am setting up a slice with multiple sites, all using FABNetv6. I checked I was using the data plane address (starting with 2602:fcfb:). However, the ping command says “From 2001:400:2004:12::3 icmp_seq=1 Destination unreachable: Administratively prohibited” if two servers are from different sites. I have a customized program listening to port 12345, and that cannot be reached even when the client and the server are from the same site. I am wondering what might be wrong. Attached is my topology.

 

Thank you!

[Komal Thareja]

Hi Yifan,

Could you please share your slice id?

Best,

Komal

[Yifan Cai]

Komal Thareja wrote:

Hi Yifan,

Could you please share your slice id?

Best,

Komal

Hi Komal,

14fac478-d7a7-4d75-ad80-1452ca7fbe0c is my slice ID. Thank you!

Yifan

[Yifan Cai]

A potentially relevant note. Is port mirroring necessary for my use case, where I need to listen on a customized port? I currently do not have this in my slice.

 

Thank you!

[Komal Thareja]

Hi Yifan,

I’m not sure how the VMs were originally provisioned—whether auto configuration or manual setup was used, or which JupyterHub container was involved.

I checked your MASS VMs and noticed that IPv6 addresses were not assigned to the data plane interfaces and the required routes were missing. I manually configured both VMs by assigning IPv6 addresses and adding the appropriate routes:

mass-0:

sudo ip -6 addr add 2602:fcfb:7:1::2/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:7:1::1 dev enp7s0

mass-1:

sudo ip -6 addr add 2602:fcfb:7:1::3/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:7:1::1 dev enp7s0

After applying these changes, connectivity between the MASS VMs is working as expected (verified via ping).

I also attempted to access the UTAH and ATLA VMs, but I wasn’t able to SSH using the NOVA keys, so I couldn’t validate their configuration.

Could you please run the following commands on the remaining VMs to configure the data plane interfaces?

UTAH VMs

ut-0:

sudo ip -6 addr add 2602:fcfb:8:d1::2/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:8:d1::1 dev enp7s0

ut-1:

sudo ip -6 addr add 2602:fcfb:8:d1::3/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:8:d1::1 dev enp7s0

ATLA VMs

atl-0:

sudo ip -6 addr add 2602:fcfb:15:1::2/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:15:1::1 dev enp7s0

atl-1:

sudo ip -6 addr add 2602:fcfb:15:1::3/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:15:1::1 dev enp7s0

GATECH VMs

gatech-0:

sudo ip -6 addr add 2602:fcfb:11:2::3/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:11:2::1 dev enp7s0

gatech-1:

sudo ip -6 addr add 2602:fcfb:11:2::2/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:11:2::1 dev enp7s0

WASH VMs

wash-0:

sudo ip -6 addr add 2602:fcfb:a:1::3/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:a:1::1 dev enp7s0

wash-1:

sudo ip -6 addr add 2602:fcfb:a:1::2/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:a:1::1 dev enp7s0

LOSA VMs

la-0 (uses enp6s0):

sudo ip -6 addr add 2602:fcfb:12:c::3/64 dev enp6s0
sudo ip link set enp6s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:12:c::1 dev enp6s0

la-1 (uses enp6s0):

sudo ip -6 addr add 2602:fcfb:12:c::2/64 dev enp6s0
sudo ip link set enp6s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:12:c::1 dev enp6s0

Note: The LOSA VMs use enp6s0 instead of enp7s0 for the data plane interface.

Please let me know if you need any help with this.

Best,
Komal

1 user thanked author for this post.

Yifan Cai

[Yifan Cai]

Hi Komal,

Thank you so much for your detailed instructions! It works now!

I used the graphical portal to set up the machines. If I understand correctly, for future experiments, setting up the interface(enp6/7s0), IP, and the routes are the things I need to do. Is that correct?

Thank you again!

Yifan

[Komal Thareja]

Hi Yifan,

When creating a slice through the Portal, the network configuration needs to be set up manually. However, if you create the slice via the JupyterHub interface (Portal → JupyterHub), the network configuration is handled automatically. You can follow the steps outlined here: https://learn.fabric-testbed.net/knowledge-base/creating-your-first-experiment-in-jupyter-hub/

Best,
Komal

1 user thanked author for this post.

Yifan Cai

[Yifan Cai]

Thank you so much!

[Author]

Posts