Nodes in the same slice using FABNetv6 cannot reach each other

This topic has 7 replies, 2 voices, and was last updated 1 week, 3 days ago by Yifan Cai.

Viewing 8 posts - 1 through 8 (of 8 total)

Author

Posts
April 9, 2026 at 10:44 pm #9674
Yifan Cai
Participant
Hello,

I am new to FABRIC, and I am setting up a slice with multiple sites, all using FABNetv6. I checked I was using the data plane address (starting with 2602:fcfb:). However, the ping command says “From 2001:400:2004:12::3 icmp_seq=1 Destination unreachable: Administratively prohibited” if two servers are from different sites. I have a customized program listening to port 12345, and that cannot be reached even when the client and the server are from the same site. I am wondering what might be wrong. Attached is my topology.

Thank you!
April 9, 2026 at 10:50 pm #9676
Komal Thareja
Participant
Hi Yifan,

Could you please share your slice id?

Best,

Komal
April 9, 2026 at 10:55 pm #9677
Yifan Cai
Participant
Komal Thareja wrote:

Hi Yifan,

Could you please share your slice id?

Best,

Komal

Hi Komal,

14fac478-d7a7-4d75-ad80-1452ca7fbe0c is my slice ID. Thank you!

Yifan
April 9, 2026 at 11:20 pm #9678
Yifan Cai
Participant
A potentially relevant note. Is port mirroring necessary for my use case, where I need to listen on a customized port? I currently do not have this in my slice.

Thank you!
April 9, 2026 at 11:25 pm #9679
Komal Thareja
Participant
Hi Yifan,

I’m not sure how the VMs were originally provisioned—whether auto configuration or manual setup was used, or which JupyterHub container was involved.

I checked your MASS VMs and noticed that IPv6 addresses were not assigned to the data plane interfaces and the required routes were missing. I manually configured both VMs by assigning IPv6 addresses and adding the appropriate routes:

mass-0:
```
sudo ip -6 addr add 2602:fcfb:7:1::2/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:7:1::1 dev enp7s0
```
mass-1:
```
sudo ip -6 addr add 2602:fcfb:7:1::3/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:7:1::1 dev enp7s0
```
After applying these changes, connectivity between the MASS VMs is working as expected (verified via ping).

I also attempted to access the UTAH and ATLA VMs, but I wasn’t able to SSH using the NOVA keys, so I couldn’t validate their configuration.

Could you please run the following commands on the remaining VMs to configure the data plane interfaces?

UTAH VMs

ut-0:
```
sudo ip -6 addr add 2602:fcfb:8:d1::2/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:8:d1::1 dev enp7s0
```
ut-1:
```
sudo ip -6 addr add 2602:fcfb:8:d1::3/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:8:d1::1 dev enp7s0
```
ATLA VMs

atl-0:
```
sudo ip -6 addr add 2602:fcfb:15:1::2/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:15:1::1 dev enp7s0
```
atl-1:
```
sudo ip -6 addr add 2602:fcfb:15:1::3/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:15:1::1 dev enp7s0
```
GATECH VMs

gatech-0:
```
sudo ip -6 addr add 2602:fcfb:11:2::3/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:11:2::1 dev enp7s0
```
gatech-1:
```
sudo ip -6 addr add 2602:fcfb:11:2::2/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:11:2::1 dev enp7s0
```
WASH VMs

wash-0:
```
sudo ip -6 addr add 2602:fcfb:a:1::3/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:a:1::1 dev enp7s0
```
wash-1:
```
sudo ip -6 addr add 2602:fcfb:a:1::2/64 dev enp7s0
sudo ip link set enp7s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:a:1::1 dev enp7s0
```
LOSA VMs

la-0 (uses enp6s0):
```
sudo ip -6 addr add 2602:fcfb:12:c::3/64 dev enp6s0
sudo ip link set enp6s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:12:c::1 dev enp6s0
```
la-1 (uses enp6s0):
```
sudo ip -6 addr add 2602:fcfb:12:c::2/64 dev enp6s0
sudo ip link set enp6s0 up
sudo ip -6 route add 2602:fcfb:00::/40 via 2602:fcfb:12:c::1 dev enp6s0
```
Note: The LOSA VMs use enp6s0 instead of enp7s0 for the data plane interface.

Please let me know if you need any help with this.

Best,
Komal

1 user thanked author for this post.

Yifan Cai
April 9, 2026 at 11:33 pm #9680
Yifan Cai
Participant
Hi Komal,

Thank you so much for your detailed instructions! It works now!

I used the graphical portal to set up the machines. If I understand correctly, for future experiments, setting up the interface(enp6/7s0), IP, and the routes are the things I need to do. Is that correct?

Thank you again!

Yifan
April 10, 2026 at 6:41 am #9681
Komal Thareja
Participant
Hi Yifan,

When creating a slice through the Portal, the network configuration needs to be set up manually. However, if you create the slice via the JupyterHub interface (Portal → JupyterHub), the network configuration is handled automatically. You can follow the steps outlined here: https://learn.fabric-testbed.net/knowledge-base/creating-your-first-experiment-in-jupyter-hub/

Best,
Komal

1 user thanked author for this post.

Yifan Cai
April 10, 2026 at 10:00 am #9682
Yifan Cai
Participant
Thank you so much!
Author

Posts

Viewing 8 posts - 1 through 8 (of 8 total)

You must be logged in to reply to this topic.

UTAH VMs

ATLA VMs

GATECH VMs

WASH VMs

LOSA VMs

1 user thanked author for this post.

1 user thanked author for this post.