1. Home
  2. Frequently Asked Questions

Frequently Asked Questions

As we talk with potential FABRIC collaborators, we are getting a lot of wonderful questions. We’ve created a list of questions that we hear often to help you navigate your own FABRIC curiosities.

How is FABRIC different from GENI?
While FABRIC takes advantage of experience and approaches to building widely distributed testbeds collected from GENI, there are differences between FABRIC and GENI that drastically change and expand the types of experiments that can be performed:

  • FABRIC has a programmable core network infrastructure.
  • FABRIC provides guaranteed quality of service by utilizing its own dedicated optical infrastructure or relying on dedicated L2 capacity wherever possible to create QoS-guaranteed connections.
  • FABRIC provides access to a variety of programmable PCI devices Network cards, GPUs, FPGAs.
  • FABRIC supports highly accurate timestamping of events using GPS-based clocks.
  • FABRIC interconnects a large number of existing scientific, computational and experimental facilities.
  • FABRIC experimenter network topologies can peer with production networks on-demand.
  • FABRIC supports a broader range of data transfer speeds (10Gbs, 25Gbps, 100Gbps, 1000Gbps) along with the ability to compute on data at these line speeds within the network.
  • What types of resources does FABRIC provide and how are they allocated and accessed?
    FABRIC allows experimenters to instantiate compute instances in the form of bare-metal servers, VMs and containers with significant compute and storage capabilities as well as directly attached PCI devices: multi-port SmartNICs, GPUs and FPGAs. These compute instances are interconnected with on-demand high-bandwidth links up to 25-100Gbps in the Core and higher bit rates in the SuperCore portion of the topology. These compute instances can also have storage in the form of dedicated NVMe drives and slower, but larger dynamically partitionable block storage. Several sites provide on-demand access to 100Gbps P4-capable switches, which can similarly be linked into experimenter topologies. Compute instances are accessible by experimenters via SSH, other more scalable experiment control tools may be contributed by the community later. FABRIC offers different network services: IPv4 and IPv6 networks matching FABRIC footprint, on-demand Layer 2 topologies and peering between slice topologies and public IPv6 network. Public cloud peering at 100Gbps in two locations will be offered through a partnership with Internet2.
    How do experimenters interact with FABRIC?
    A FABRIC Portal provides basic browser-based access to FABRIC resources. Programmatic access is provided with help of FABRIC client libraries that can be operated as a CLI or embedded into experimenter code. Initially Python bindings are provided which later may be extended to other languages. FABRIC provides a hosted Jupyter Hub environment for experimenters, which comes with Jupter notebooks automatically preloaded with FABRIC libraries and CLI tools. The portal and Jupyter Hub use federated identity through CI Logon and support any institutional identity provider that is part of InCommon R&S (Research and Scholarship) federation. This includes the majority of US educational institutions.
    How will FABRIC resources be allocated to experimenters?
    Initial FABRIC experiments during the construction phase (2019-2023) will be done through individual agreements between specific researchers identified as early experimenters, and the FABRIC team. When FABRIC enters production it will have an AUP and an allocation policy that balances the needs of various experimenter groups (long-running and resource-intensive) and educational uses. Mechanisms will be put in place to enforce those allocation policies across the FABRIC footprint and will provide flexibility for FABRIC expansion site owners and Facility Partners to allow them to enforce site-specific allocation policies.
    Does FABRIC support Expansion Sites or BYOE?
    Yes, FABRIC supports expansion sites (e.g. universities) that purchase their own FABRIC node (‘hank’) based on standard configurations, FABRIC team will also help facilitate connecting expansion sites to FABRIC topology either via Internet2 or with dedicated Layer2 or optical connectivity. We expect standard FABRIC Site BOMs to be available in the fall of 2020. Stay in touch by completing the Get Involved form.

    FABRIC nodes also support a Bring Your Own Equipment (BYOE) model of contributions. Contributions of hardware to FABRIC can take a number of forms:

  • PCI devices to be included in the available servers
  • Network switching equipment added to individual sites
  • Discrete servers added to individual sites

  • BYOE contributions are subject to a number of constraints:

  • Available rack space at individual sites
  • Available power (core sites are limited to only 4kW total power budget)
  • Available PCI slots in servers
  • Available ports on FABRIC node dataplane and management switches

  • Contributed hardware will be linked into the FABRIC management plane and individual resources become provisionable via FABRIC CF. More details on BYOE support can be found in the FABRIC Node Design document.

    What measurement capabilities does FABRIC have, and what kinds of measurement/experiment data will be made available to experimenters and where can the data be stored?
    The FABRIC Measurement Framework is designed to be adaptable/programmable, scalable, extensible, and shareable. It’s goal is to support an automated instrumentation of experiments, enabling experimenters to collect, store, and publish detailed measurement data from users and the system/experiment. Examples include CPU, disk and network utilization, packet error counts and other interface statistics, packet capture traces with precise timestamps from inside the experiment topologies. FABRIC allows users to collect measurements and error statistics from the underlying optical/DWDM infrastructure and electrical power consumption data. The framework is realized using a measurement bus infrastructure based on pub/sub technology with efficient filtering, searching, and processing of measurement data within the testbed using a combination of testbed resources (e.g. GPUs and FPGAs for real-time training and inference on the data) and off-line processing outside the testbed using experimenter’s institutional or public cloud resources. The framework allows for fine-grained precise measurements leveraging a highly-accurate PTP timing signal from a GPS receiver located at every FABRIC site. This enables precise timestamping of packets using NIC cards (a.k.a., PacketGPS) with the accuracy of 10s of milliseconds. FABRIC also allows for the collection of high-speed packet traces using RAM or high-speed NVMe drives at each site, which can be attached to individual experimenter compute instances. Larger data sets can be stored within experiment-attached block storage. The facility has between ½ and 1PB of file storage space to help ease moving measurement and other experiment-related data in- and out- of experiments in programmatic fashion. FABRIC collects logging and metrics from the underlying infrastructure to be made available to operators and experimenters. Additional details about the types of available data and data sharing policies for the facility will be devised in consultations with and shared with the community.
    Is there an interest in international partners participating – both from the hosting of internationally located nodes and international research teams?
    Yes, we definitely are interested. Using separate funding from IRNC FAB project we are deploying FABRIC nodes to CERN, Amsterdam, Bristol and Tokyo. We are interested in collaborating with international partners to deploy exchange points that will allow them to interconnect their facilities to FABRIC. We will share our software and hardware designs, so that experimental facilities like FABRIC can emerge in other countries and be federated with us to create a world-wide ‘testbed-of-testbeds’. We will be present at many international meetings where details of federation and connectivity are discussed so we can include them in our plans.
    If you are interested in collaborating internationally with us, please complete the Get Involved form at our website.
    How does FABRIC fit in with other shared experimental infrastructures (NSF Clouds, PRP/NRP, PAWR testbeds)?
    FABRIC is not meant to be an isolated testbed facility, rather it aims to interconnect many other experimental and production facilities to create a rich substrate for experimentation, which allows experimenters to pick and choose portions of the infrastructure for their experiment. We are interested in ideas for early experiments that combine the capabilities of these facilities together, please reach out to us at info@fabric-testbed.net. FABRIC team works with organizations running these other facilities in order to enable joint experiments through network connectivity, joint tooling, identity federation and other mechanisms that reduce the friction of creating cross-facility/cross-testbed experiments.
    Will FABRIC always be experimental and no production?
    FABRIC is a scientific instrument for supporting research in cyber-infrastructure. This implies that its primary use is for experiments in a variety of areas related to internet architectures, cyber-security, distributed protocols and applications. Many of these experiments may need to be long-running and carry production-like traffic in order to truly understand the tradeoffs in the system that is being experimented on. However, by virtue of being experiments, they can’t be expected to provide production-grade reliability and robustness.

    In order to support the widest possible variety of experiments spanning from blue-sky future Internet ideas to support for novel production science workflows, FABRIC will provide on-demand connectivity with production networks and facilities, to allow experimenters mix and match production and experimental resources in combinations that make sense for their experiments.

    What specific types of PCI peripherals will FABRIC provide access to?
    Current FABRIC designs envision supporting Mellanox ConnectX family of dual-port network cards, NVIDIA GPUs (T4s, RTX6000s, A30s) and Xilinx Alveo family FPGAs. FABRIC expects to provide limited support for GPU and FPGA application development workflows, instead partnering with other testbeds that offer similar devices so they can be used for developing the application codes that are then deployed in FABRIC.

    What is FABRIC?

    Below are two resources—a presentation and a webinar—that may provide further information to assist with answering your FABRIC questions.