[Komal Thareja]

MASS is working as well. We checked your FABRIC nodes, Fabnet services seems to be connected properly and we can ping the gateway. FABRIC VMs in your slice can ping each other too.

Not sure how your Chameleon Server is setup.

You should see routes and interface setup something similar to below on your Chameleon Node:

cc@kthare10-fabric-stitch-server-1:~$ ip route list
default via 10.130.163.2 dev eno1np0 proto dhcp src 10.130.163.10 metric 100
10.128.0.0/10 via 10.130.163.1 dev eno1np0 proto dhcp src 10.130.163.10 metric 100
10.130.163.0/24 dev eno1np0 proto kernel scope link src 10.130.163.10
169.254.169.254 via 10.130.163.3 dev eno1np0 proto dhcp src 10.130.163.10 metric 100
cc@kthare10-fabric-stitch-server-1:~$
cc@kthare10-fabric-stitch-server-1:~$
cc@kthare10-fabric-stitch-server-1:~$
cc@kthare10-fabric-stitch-server-1:~$ ifconfig
eno1np0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.130.163.10 netmask 255.255.255.0 broadcast 10.130.163.255
inet6 fe80::be97:e1ff:fec4:8e0 prefixlen 64 scopeid 0x20<link>
ether bc:97:e1:c4:08:e0 txqueuelen 1000 (Ethernet)
RX packets 4937 bytes 1058216 (1.0 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 4804 bytes 410390 (410.3 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

P.S: I did execute the cell indicated as “(Optionally) Add a Router and Attach it to the Subnet”.

 

• This reply was modified 12 months ago by Komal Thareja.
• This reply was modified 12 months ago by Komal Thareja.
• This reply was modified 12 months ago by Komal Thareja.

[Komal Thareja]

Hello Violet,

Thank you for informing us about this. The OpenVswitch notebook has been uploaded to the Jupyter Examples and is now accessible on GitHub here.

Please be aware that, due to limitations imposed by NVIDIA/Mellanox on how bridging is managed, the VM running the OVS switch must utilize dedicated NICs instead of BasicNICs. This example assumes that condition. We have identified an alternative solution currently undergoing testing. We will keep you updated once a solution is deployed to make this work with BasicNICs.

Appreciate your understanding.

Best regards,

Komal

[Komal Thareja]

Hi Sanjana,

I am able to reproduce this issue on MASS. But I was able to get this to work on other sites like SEAT, PSC. Could you please use a different site like SEAT or PSC while we investigate this issue. I will keep you updated with the findings for MASS.

Thank you for sharing your observations and helping us make the testbed better.

Thanks,

Komal

[Komal Thareja]

@Khawar – Could you please try accessing JH again and let me know how it goes?

Thanks,

Komal

[Komal Thareja]

Hello Nagmat,

I suspect your bastion keys are expired and hence the failure.

Could you please run the notebook: jupyter-examples-rel1.6.1/configure_and_validate.ipynb ?

This should update your bastion keys if expired. Please try SSH access again after that from console as well as the notebook. Please restart the notebook kernel after the keys are updated.

Thanks,

Komal

[Komal Thareja]

Just realized you also had problem with network reachability between Chameleon and Fabric nodes. Could you please share your slice ID for FABRIC?

Also, please check the interface and routes are setup correctly on Chameleon node.

[Komal Thareja]

@Sanjana – Chameleon team would be better equipped to help you regarding the failure observed while creating Node on Chameleon. Jupyter Notebook referred in your post uses chameleon python API.

FABRIC portal doesn’t provide support to provision resources on Chameleon. You would have to use Chameleon Portal to use their Graphical Interface.

Thanks,

Komal

[Komal Thareja]

Maintenance is completed and testbed is ready to use. Happy Slicing on FABRIC!

[Komal Thareja]

Unfortunately, we ran into an issue with Network AM. Provisioning for network services is impacted. The testbed would remain in Maintenance until this issue is resolved.

We will share updates here once the issue is resolved. Apologies for the inconvenience.

[Komal Thareja]

Hello Ze,

You can connect your VMs with various Layer 3 or Layer 2 network service available on FABRIC. Please follow along the examples Networking examplees from start_here.ipynb as shown below.

Local Ethernet (Layer 2) could be a good start. More details about the various network services can be found here.

• Networking
  • FABNet IPv4 (Layer 3): Connect to FABRIC’s IPv4 internet (manual, auto, full auto)
  • FABNet IPv4 Ext (Layer 3): Connect to FABRIC’s IPv4 internet with external access (manual)
  • FABNet IPv6 (Layer 3): Connect to FABRIC’s IPv6 internet (manual, auto, full auto)
  • FABNet IPv6 Ext (Layer 3): Connect to FABRIC’s IPv6 internet with external access (manual)
  • Local Ethernet (Layer 2): Create a private local Ethernet network on a FABRIC site (manual, auto, user defined)
  • Wide Area Link (Layer 2): Create a private WAN Ethernet link between FABRIC sites (manual, auto, user defined)
  • Facility Ports: Connect a FABRIC experiment to an external facility such as Chameleon.
  • Port Mirroring: Listen in on traffic in FABRIC dataplane using PortMirror service.

1 user thanked author for this post.

Ze Yang

[Komal Thareja]

Hello Ze,

If VM is created via JH, in the list of Nodes/VM, there is a column which displays the SSH command that can be used to SSH to VM. Please use the SSH command to VMs.

Also, following commands may be useful to install mpi4py.

sudo apt update -y
sudo apt-get -y install python3-pip
sudo apt install mpich
sudo apt install libopenmpi-dev
sudo pip3 install mpi4py

 

NOTE: Please refer to https://learn.fabric-testbed.net/knowledge-base/generating-ssh-configuration-and-ssh-keys/#using-ssh-to-access-your-vms for more guidance on SSH to VMs.

 

Thanks,

Komal

1 user thanked author for this post.

Ze Yang

[Komal Thareja]

Issue has been resolved now. You slice provisioning should work now.

[Komal Thareja]

FABRIC only allows SSH and few ICMP messages over the management interface. Hosting services on management network is not recommended. Instead, we recommend using data plane network for your service.

FABRIC serves as a secure sandbox, allowing students and researchers to experiment with potentially disruptive and vulnerable software architectures in a protected environment. When connecting external devices, such as laptops or servers, to nodes within a slice, it is crucial to employ secure methods like SSH tunnels. A Jupyter notebook example illustrates how to create SSH tunnels through the FABRIC bastion host. Alternatively, users can utilize personal VPNs like Tailscale for secure connections.

Exposing ports to the entire Internet is restricted, reserved only for exceptional cases where alternative solutions are not viable. Moreover, users undertaking such capabilities are responsible for deploying, maintaining, and ensuring the security of experiments, akin to a production data center. IPv4Ext and IPv6Ext services facilitate these capabilities.

For newcomers, getting acquainted with SSH tunnels is recommended due to their simplicity and security. If users have additional questions or require further guidance, they are encouraged to reach out.

Thanks,

Komal

[Komal Thareja]

Hi Robin,

There are libvirt issues on the worker your VMs are being created. We are looking at the issue and will keep you informed. However, could you please try using a different site than AMST in the meanwhile to unblock you?

Apologies for the inconvenience!

Thanks,

Komal

[Komal Thareja]

No, you don’t need to run “add/remove SSH keys” notebook. By default, when you create a slice, keys specified in your fabric_rc are already added to the VMs.

[Author]

Posts