[Komal Thareja]

Hello Yuanhao,

Thank you for the detailed description of your experiment and topology — that’s very helpful.

For your described setup with multiple independent, point-to-point internal links between your own nodes across sites, using L2STS is perfectly valid and appropriate. L2STS provides private Layer-2 connectivity between nodes within your slice and is generally recommended when you want internal point-to-point links without stitching to external networks.

L2PTP, on the other hand, is typically used when:

• You are connecting Dedicated NICs,
• You need guaranteed QoS (bandwidth reservation), or
• You want to explicitly control the path (e.g., define intermediate hop sites).

Given that, your current L2STS-based slice (l25gclplus-yuanhao5) is indeed a good configuration for your 5G experiment.

Regarding the L2PTP error, you’re correct — the message indicates that the interface must be tagged with a VLAN ID. There appears to be a bug in the portal’s Slice Builder that currently prevents specifying VLAN tags for L2PTP connections. Thank you for helping us identify that and we will work to address it.

In the meantime, you can continue using L2STS, which should meet your needs for internal connectivity.

For additional context and examples, please refer to our Network Services documentation:
https://learn.fabric-testbed.net/knowledge-base/network-services-in-fabric/

Also, I’d recommend checking out the example notebooks available under the JupyterHub tab on the FABRIC Portal — they provide working examples of various network configurations, including multi-site and point-to-point topologies.

Best regards,
Komal Thareja

[Komal Thareja]

It prints the interfaces after the slice is completely up.

Could you please share a snippet of code or screenshot where you are observing this?

Best,

Komal

[Komal Thareja]

Hi Sourya,

Could you please delete your slice and try creating it again?

Best,

Komal

[Komal Thareja]

This was resolved offline.

Issue here was downloading .c code from JH container to desktop results in txt files.

Recommended solution: Create a tar file of the entire directory and download the directory.

Best,

Komal

[Komal Thareja]

Maintenance is complete and network model has been updated!

[Komal Thareja]

Hi Nirmala,

I tried an example as below and it seems to be working:

1st parameter is the local path where file should be saved and second parameter being the remote location of the file to be downloaded.

node.download_file('hello.c','/home/rocky/hello.c')

Could you please share which version of fablib are you using?

Best,

Komal

[Komal Thareja]

Hi Rasman,

I forgot to mention that the steps for installing iperf3 should be run as the root user. On your VM, I did the following:

sudo su -

curl -L https://github.com/esnet/iperf/releases/download/3.18/iperf-3.18.tar.gz > iperf-3.18.tar.gz
tar -zxvf iperf-3.18.tar.gz
cd iperf-3.18

sudo apt update
sudo apt install build-essential

sudo ./configure; make; make install
sudo ldconfig

I also applied the following host tuning (node_tools/host_tune.sh) on both VMs:

#!/bin/bash

# Linux host tuning from https://fasterdata.es.net/host-tuning/linux/
cat >> /etc/sysctl.conf <<EOL
# allow testing with buffers up to 128MB
net.core.rmem_max = 536870912
net.core.wmem_max = 536870912
# increase Linux autotuning TCP buffer limit to 64MB
net.ipv4.tcp_rmem = 4096 87380 536870912
net.ipv4.tcp_wmem = 4096 65536 536870912
# recommended default congestion control is htcp or bbr
net.ipv4.tcp_congestion_control = bbr
# recommended for hosts with jumbo frames enabled
net.ipv4.tcp_mtu_probing = 1
# recommended to enable 'fair queueing'
net.core.default_qdisc = fq
#net.core.default_qdisc = fq_codel
EOL

sysctl --system

# Turn on jumbo frames
for dev in basename -a /sys/class/net/*; do
    ip link set dev $dev mtu 9000
done

With these changes, I’m now seeing bandwidth close to 10G (see snapshot below).

According to fablib.list_links(), links from GATECH are capped at 8G. I’d suggest trying a different site instead of GATECH.

Regarding the slice getting stuck at Submit: your keys may have expired. Please try running the notebook jupyter-examples-rel1.9.0/configure_and_validate/configure_an_validate.ipynb. This should automatically renew your keys if needed.

If it still hangs at submit, please check /tmp/fablib/fablib.log for errors and share here.

Best,

Komal

[Komal Thareja]

Hi Rasman,

By default, the standard iperf3 version does not perform well with multiple streams. ESnet provides a patched version that resolves this issue and delivers significantly better performance. This fixed iperf3 is already packaged inside the container.

If you would like to run it directly on the host, you can install it with the following steps:

curl -L https://github.com/esnet/iperf/releases/download/3.18/iperf-3.18.tar.gz > iperf-3.18.tar.gz
tar -zxvf iperf-3.18.tar.gz
cd iperf-3.18
sudo apt update
sudo apt install build-essential
sudo ./configure
make
sudo make install

Additionally, please make sure that the script node_tools/host_tune.sh (included with the notebook) has been executed on the relevant nodes.

If you continue to see lower bandwidth, kindly share your slice ID so I can take a closer look.

Thanks,
Komal

[Komal Thareja]

Thank you for reporting this Maureen! We have identified the issue and working to identify a solution.

We will keep you posted about the resolution. Apologies for the inconvenience.

Best,

Komal

[Komal Thareja]

Hi Tanay,

We’re in the process of deploying them and are targeting DPU availability at KNIT11 around October 13–14.

Best.

Komal

[Komal Thareja]

Hi Nishanth,

Could you please try again? It should work now, freebsd fix introduced this check, i have disabled it now.

Best,

Komal

[Komal Thareja]

Hi,

Fix for this issue has been deployed on production. Please try creating a slice and let us know if you run into any issues.

Best,

Komal

[Komal Thareja]

Hi Tejas,

You can use FABRIC’s Layer 3 FabNetv4 or FabNetv6 Network Service to establish connectivity between slices.

Any VM connected to FabNetv* in one slice can communicate with a VM connected to FabNetv* in another slice, provided the routes are configured correctly. You just need to add the following routes:

ip route add 10.128.0.0/10 via <fabnetv4_gateway>
ip -6 route add 2602:FCFB:00::/40 via <fabnetv6_gateway>

You may also find this example artifact helpful, as it demonstrates inter-slice connectivity using FabNetv4.

Best,

Komal

[Komal Thareja]

Thank you for sharing this, Nishant and YoursSunny.

I was able to reproduce the issue. On IPv4 sites, user SSH keys are not being injected, and on IPv6, SSH connections are failing completely. We’ll work on addressing this and will let you know once the fix has been deployed. Apologies for the inconvenience in the meantime.

Best,

Komal

[Komal Thareja]

Hi Dagim,

Thank you for sharing this observation. Could you update the instantiation of the fablib object in the first cell to the following and then try running the notebook again?

fablib = fablib_manager(project_id=project_id, validate_config=False)

Thanks,
Komal

 

[Author]

Posts