Basic Facts
FABRIC has three ways of interacting with it:
- Portal – manage projects, SSH keys, API tokens. Create or visualize slice topologies
- Jupyter Notebooks in FABRIC’s own Jupyter Hub – using FABRIC APIs to run experiments and not just create topologies – all in a browser
- Use FABRIC APIs from experimenter laptop or desktop – FABRIC API libraries are available for download from PyPi.
We strongly recommend starting your FABRIC experiments with Jupyter Notebooks
The types of resources FABRIC offers are
- Virtual machines of different sizes (up to 64 cores, 384G RAM) with directly attached PCI devices – GPUs, Network Cards, NVMe drives, FPGAs.
- Virtual machines can be placed at the sites of your choice and interconnected by a rich set of on-demand Layer 2 and Layer 3 network services
- Support for P4 switches is in development
- Virtual machines can have large storage volumes attached to them
- VMs or slivers are arranged in slices representing individual experiment topologies. Slices can be created, modified and deleted.
- Slices have a finite lifetime, but can typically be extended before they expire
- FABRIC Slices can connect to other testbeds or facilities using Facility Ports
A few things to remember:
Everything (infrastructure and software) continues to be in active development
Infrastructure is updated as it becomes available. This is generally not disruptive to experiments.
Software updates are on a 3-4 month cadence between releases. Backward compatibility is not always guaranteed. We do our best to warn about upcoming changes.
Getting a FABRIC Account
FABRIC uses different credential types, however everything is tied to your institutional identity – all elements of the infrastructure – the Portal, Jupyter Hub, this site (Knowledge Base), the use of APIs – everything requires logging in via CI Logon using your university credentials or one of the commercial identity providers (Google, GitHub).
Note that using Google, ORCID, GitHub or Microsoft as your identity provider requires extra verification steps by our staff. Whenever possible please try to use your university identity provider.
To get an account, before you can do anything you must enroll via the portal. The enrollment has multiple steps. After you complete the enrollment you can login to the Portal and be added to projects.
If using Google, GitHub, ORCID or Microsoft please know that it will be more difficult for us to grant Project Lead status to you.
Some universities allow you to use multiple different emails with the same password to login to their systems. Please always use a single email with FABRIC. Once you sign up using user@university.edu, continue using that address for all other activities on FABRIC.
Using and managing Projects
A Project is a group of experimenters with the same permissions. Projects are created by users with Project Lead role – a role typically reserved for faculty and senior research staff. You can become a Project Lead by request. Project Lead who created a project may assign additional Project Owners to help manage Project Members. Each project has a set of permissions to specific FABRIC resources.
A good example of a Project Lead is a professor, who may create a project and delegate to a TA or an RA to be a Project Owner to help manage other students/members of the project.
You can only create slices after you’ve been added to one or more projects. Every experiment you create on FABRIC is done on behalf of some project you are part of.
Interacting with FABRIC
Experimenters interact with FABRIC using REST-based APIs which are invoked using FABlib library or the Portal. Portal only allows you to manage slice topologies while FABlib also lets you configure experiments. This is a crucial difference as in addition to a topology an experiment has configuration states, data and may include resources from other testbeds. The Portal can only create slices on FABRIC. FABlib is developing integrations with other testbeds’ APIs to support cross-testbed experiments.
To authenticate APIs FABRIC uses API tokens instead of certificates (like e.g. GENI). Tokens can expire, they can also be renewed. Note that Jupyter Notebooks automatically manage tokens for you.
There is an equivalent of GENI RSpec for topology descriptions. It serializes to GraphML and JSON and allows you to save and recall your experiment topologies, however as we mentioned above, the topology is not the whole experiment.
We provide a rich set of Jupyter notebooks that showcase the different capabilities of the testbed and simple experiments. Login to the Jupyter Hub from the Portal to see them.
The best way to create an experiment artifact is to embed it inside a Jupyter Notebook which comprises the
– topology definition
– experiment configuration
– experiment workflow
and can also include annotations and graphical results. It can be shared and reproduced by others.
Logging into VMs
Unlike many testbeds, FABRIC relies on bastion hosts to restrict access to VMs you create in your slices – you cannot login to your VMs directly. Because of this, two SSH keypairs are needed to login to a VM – a bastion keypair and a slice/sliver keypair. FABRIC allows managing bastion and slice/sliver keys via the Portal and APIs. When added in the Portal, bastion keys are automatically propagated to the bastion hosts.
FABRIC Portal only keeps the public SSH keys. Storing private SSH keys is left up to the experimenter.
Bastion keys expire after 6 months and need to be regenerated. Slice/sliver keys registered with the Portal expire after 2 years. You can find your currently active keys in the Portal.
You cannot login to the bastion hosts. You can only hop over them into your VMs. Simply using -J option will not work. A special SSH client configuration file needs to be created to use the bastion hosts.
When using Jupyter notebooks we rely on Paramiko Python library to perform logins and most of the complexity described above is hidden from view.
Management vs Dataplane
FABRIC has a concept of a management plane and a dataplane. Management plane is used for
- Logging into VMs via bastion hosts
- Downloading necessary software into VMs from the Internet
Management plane is rate-limited to about 1Gbps and shared among many experimenters. It has traffic filtering rules to prevent hosting Internet-facing services and should not be used as part of experiment data flows.
Most FABRIC sites’ management networks are on IPv6 network, i.e. they provide IPv6 management addresses to experimenter VMs. Bastion hosts have access to both IPv4 and IPv6 network so they make using IPv4 and IPv6 sites transparent to experimenters. Generally you will not have any problems accessing IPv6 sites from the IPv4 network, however in some cases fetching software from the IPv4 network into your IPv6 VMs may require additional configuration steps.
Dataplane on the other hand is intended for running experiments – connecting VMs with one another into desired topologies and connecting to external resources – HPC facilities, other testbeds, public cloud providers and the Internet. It provides high-bandwidth (100Gbps on many links) and many types of on-demand network services.
Getting Help
Visit this article for more information. This article discusses how to search for information.