What topics do you want FABRIC tutorials about?

[Paul Ruth]

The FABRIC team is planning to develop a set of tutorials that will be shared through a webinar series, youtube videos, and will be presented at the upcoming FABRIC workshop in Chicago.

Although we have a list of several planned topics, we would like to hear from the FABRIC community (i.e. you) about what topics you think should be included.  These topics can be either topics you would like to learn about or topics you struggled to understand and think a tutorial will help future users.

Please reply to reply in this thread with any ideas and plan to attend the workshop if you can!

[Shivam Patel]

quick and easy way to install/configure/run DPDK pipelines.  In particular the P4 SWX pipeline.  Pretty much, its a nice way to get a “hardware” target up and running.  Utilize CPU and NICs even if the NICs aren’t directly programmable.  And I believe it supports P4’s v1model, PSA, and PNA architectures.  The problem is, its a big hassle of chasing dependencies, doing virtualization/kernel module games, and etc and takes way to much time/delicacy to get right.  The t4p4s guys apparently have a universal translator/compiler that takes in a load of architectures and produces code that can be ran on DPDK, so this would allow folks to do everything in v1model on run on hardware without having to rewrite the programs for different architectures/targets.  I recall hearing that you guys are going to bring in some compile to/run on FPGA based systems.  But this would be a nice complement.  and potentially a less resource/target constrained way assuming anyone can get a good/reproducible install of DPDK and t4p4s going since its all on CPU, vs the potential limitations/problems that might arise in non specialists trying to use FPGAs.

Food for thought.

[Paul Ruth]

@Shivam: These are great topics.  Are these topics you have figured out, or ones that you are still trying to figure out?  We do have an early prototype of a BMv2 P4 software switch. It will probably need some tweaking to fit your needs but a start is there.  Were you able to get that to work?

• This reply was modified 2 years, 4 months ago by Paul Ruth.

1 user thanked author for this post.

Nagmat Nazarov

[Shivam Patel]

@Paul No  I wasn’t able to get them to work unfortunately.  I tried to set up a DPDK P4 software switch locally, both manually installing from source and from the t4p4s guys system(essentially installs the translator, and installs DPDK, and the translator take in v1model, PSA, PNA, eBpf, etc code and converts it all to an architecture that can be compiled with p4c-dpdk to .spec and then run on DPDK’s pipeline) but it wasnt fruitful.  It took up too much time, about 4 days was spent converting my desired program from v1model to PSA(bmv2 doesnt offer a full PSA implementation unfortunately, no checksum and packet length) and about 10 days – 2 weeks trying to get the surrounding stuff installed locally to run it, but gave up due to it being a time sink.  My hope was if i was able to get it working locally then just script it up and try to run it on fabric but it was no good in the end.

My intention was to do it inside a vm locally and then be able to export it and run it on fabric but with better CPU/NIC capacities.  The main difficulty is with having to do huge page allocations, activate certain bios features, load  kernel modules, and expose/bind nic interfaces from host into the guest in a particular way.  in particular binding nics and interfaces was never successful for me.  And then the issue is, its not guaranteed that all those configurations if made to work locally can be pushed onto a fabric slice and a different virtual environment. So who knows maybe one day.  Because it definitely is worth setting it up once and having it as a generic template.

But definitely If you guys have a bmv2 software switch that can run v1model code, then I’m highly interested in using it.

• This reply was modified 2 years, 4 months ago by Shivam Patel.

[Manas Das]

Hi Paul,

Jupyter notebooks are good for prototyping, but it will be great if we can use the terminal to run scripts. So if there is a tutorial from the very beginning and a detailed one, on how to install the fablab libraries with all requirements and dependencies already taken care of. Or which dependencies need to be installed and from where. It can be for both Rocky and the latest ubuntu images. We really need to write python scripts and integrate it with hadoop. I think many using the fablab will find this tutorial helpful since most of the project will not fit the Jupyter notebook style analysis and experimentation. If such a tutorial is already there please point me towards it. But please remember, if you people are making such a tutorial please make it a detailed one, from scratch.

Thank you in advance.

[Brandon Rice]

Hi Manas,

I might be missing exactly what you meant, but are you wanting to reserve a slice without using a Jupyter Notebook, or are you more interested in running scripts and programs directly on a node itself using a terminal?

 

The former is relatively straightforward, and a written tutorial to install FABRIC and FABlib locally can be found here. NOTE: these libraries only help reserve nodes, they are not required to interface or connect with nodes.

The latter is something I have been working a lot on trying to make easier. Currently, you have two options to run programs on nodes: FABlib’s node.execute() and locally SSHing into the node and then running the command directly. Would a tutorial on how to SSH into a node and run a script, all from a local terminal be helpful? Both cases have the issue that the node does not have a display, so no graphical information can be displayed by default.

If you need graphical output, say to view a MatPlotLib plot, you can setup X11 forwarding on the node to stream the graphical output from the node to your local computer. We will make a video tutorial on how to do this soon!

• This reply was modified 2 years, 4 months ago by Brandon Rice.

[Paul Ruth]

@Shivam The P4 BMv2 software switch example is probably already in your JupyterHub.  If not, you can pull the updated examples from here:  https://github.com/fabric-testbed/jupyter-examples

The specific examples is here: https://github.com/fabric-testbed/jupyter-examples/tree/master/fabric_examples/complex_recipes/P4_bmv2

It is actually designed to support the P4Lang tutorials that you can find here: https://github.com/p4lang/tutorials

You should know that this is really a prototype example.  I can help you get it running what you need but it will probably need some tweaking.  Since you have expressed interest (and other user are probably interested too), I will revisit this example and make it easier to adapt.

Give it a shot and let me know about your progress. I can help modify the example to your needs.

Also, thanks for your feedback. We will look into the hardware DPDK support but that may take some time.

 

[Paul Ruth]

@Manas Does the terminal window in JupyterHub fit your needs?  In JupyterHub, try  File->New->Terminal.  You should be able to ssh to your VMs that way.  You will need to setup your ssh config file correctly and use ‘ssh -F /path/to/the/ssh_config …’

Also, this doc shows how to install FABlib on your local machine. There are some issues with some Python libraries on Windows that we haven’t yet figured out but installing this on a Mac or Linux machine works great.  The main issues that we have heard about relate to conflicts between Anaconda and PyPI.  https://learn.fabric-testbed.net/knowledge-base/install-the-python-api/

Thanks  for the suggestions. I think you are right, we need a video about tips/tricks for JupyterHub and another that shows a simple install of FABlib on a laptop.

[Manas Das]

@ Brandon

I will try to execute the steps given in the link, I hope everything works. you asked “Would a tutorial on how to SSH into a node and run a script, all from a local terminal be helpful? ” of course. I am still figuring out many things, read your recent post, yes been there in that situation with ssh. Still I have a problem.

@ Paul

Yes, you are correct we need such a tutorial.

Just a general question, suppose I have a slice of two nodes, and both are connected via an interface. I can ssh from my laptop both the nodes individually with their IP address. But I am having a problem ssh to another node while I am inside a node. So, suppose I am inside node1, I cannot ssh into node2. Please tell us how to do it. I have been trying it for quite some time now.

Do we need a key to access the node2 from node1. Which key?. How to put that key inside of the node (if needed). Also few times when I do sudo <some_command> it work fine, but sometimes it ask for root password. Which of course I do not have.

Please let us know the solution to this problem.

Thank you in advance..

[Paul Ruth]

@ Manaus  Creating connections between nodes with ssh is a common requirement for experiments. I think this is a question that other user might have as well.

I’m hoping other users add more tutorial requests to this forum topic.  Can you re-ask you question about ssh’ing in a new forum topic so we can keep this one clean and other users can easily find the ssh discussion?  I will answer it there.

thanks.

[Author]

Posts