[yoursunny]

Powder Testbed has USRP devices.

You can make a facility port on FABRIC and communicate with Powder nodes via Ethernet.

[yoursunny]

Where’s the proxy?

A switchport mirror is a network instrumentation technique. It doesn’t involve any proxy software.

[yoursunny]

One possible idea:

1. Create a FABNetv4Ext or FABNetv6Ext network.
2. Insert IP routes so that the traffic to the Internet goes through FABNetv4Ext/FABNetv6Ext interface, instead of the management interface.
3. Setup switchport mirroring.
4. Capture the traffic on the mirror port.

[yoursunny]

I wonder how you determined “interfaces for L2 connections on some of my nodes gets deleted”?

[yoursunny]

Mert Cevik wrote:

On the IPv4 sites, number of VMs that can be provisioned is limited with the available IPv4 addresses in the subnet.

Can “management IP address” show up as a resource on Fabric Portal – Resources page?
This would allow the experimenter to avoid this limitation.

[yoursunny]

Oversubscription support – EDC and EDUKY sites have been enabled to support CPU over subscription.

I remember the CPU core capacity of STAR site was 384.
It’s now 768.
Did this site receive new hardware or is it oversubscription?

[yoursunny]

The entire FABRIC is red this week: https://portal.fabric-testbed.net/resources/all

See notice here: https://learn.fabric-testbed.net/forums/topic/multi-day-fabric-maintenance-january-1-5-2024/

Look at this part:

On other sites, slices will continue running, and slivers will be accessible during the maintenance, however, we will place the testbed in maintenance mode between Jan 1-5, therefore it will not be possible to perform slice operations (create, extend, delete).

[yoursunny]

Will the FABNetv4Ext peering point (located in WASH) be affected?

[yoursunny]

channel 0: open failed: connect failed: No route to host

This error typically means the VM is turned off / deleted, or is otherwise unreachable via management netif.

The fastest solution is simply deleting the experiment slice and creating a new one.

[yoursunny]

Komal Thareja wrote:

Upper limit for a VM connected with only one component would map to a single Numa Node.

What happens if there are multiple components that are on distinct NUMA sockets?
Is it possible to specify how much RAM to pin to each NUMA socket?

Komal Thareja wrote:

Max limit on memory for a numa node is 64G so exceeding that limit would not work.

If we pin a CPU core or certain amount of RAM onto a NUMA socket, does it prevent other VMs from using the same CPU core or RAM capacity?

[yoursunny]

Is there a way where I can convert my rspec into a FABRIC native object?

This would make a nice hackathon project. It should be feasible to make a converter that covers the topology, links, IP assignments, and startup scripts.

It’s even better if FABRIC platform can directly accept Request RSpec and return Manifest RSpec, so that existing tooling for Emulab can still work.

[yoursunny]

Management IP cannot communicate with each other, to prevent abuse.

You should create FABNetv4 or FABNetv6 network service in each slice. They can all communicate with each other, regardless of whether the slides are on the same or different sites.

[yoursunny]

NetworkService in the affected slice:

    <node id="73" labels=":GraphNode:NetworkService">
      <data key="d0">:GraphNode:NetworkService</data>
      <data key="d1">["bbf6a0a7-8981-4613-b797-0960e7e8ea9d", "node+amst-data-sw:ip+192.168.42.3-ipv4-ns"]</data>
      <data key="d21">L3</data>
      <data key="d8">{"fablib_data": {"instantiated": "False", "mode": "manual"}}</data>
      <data key="d22">{"ipv4": "10.145.7.1", "ipv4_subnet": "10.145.7.0/24"}</data>
      <data key="d3">{"error_message": "", "reservation_id": "4c9b702b-1346-4fe5-b61e-f5cb7790e75f", "reservation_state": "Active"}</data>
      <data key="d15">NetworkService</data>
      <data key="d10">false</data>
      <data key="d11">AMST</data>
      <data key="d13">FABNetv4</data>
      <data key="d14">8</data>
      <data key="d9">net4</data>
      <data key="d16">be2e2e72-5bdd-4301-98aa-bb9e3fe23a56</data>
      <data key="d17">98452967-6246-4517-a030-7d76d7044d05</data>
    </node>

NetworkService in a “normal” slice:

    <node id="18" labels=":GraphNode:NetworkService">
      <data key="d0">:GraphNode:NetworkService</data>
      <data key="d4">{"fablib_data": {"instantiated": "True", "mode": "manual", "subnet": {"subnet": "10.138.131.0/24", "allocated_ips": ["10.138.131.1"], "gateway": "10.138.131.1"}}}</data>
      <data key="d22">{"ipv4": "10.138.131.1", "ipv4_subnet": "10.138.131.0/24"}</data>
      <data key="d2">["bbf6a0a7-8981-4613-b797-0960e7e8ea9d", "node+atla-data-sw:ip+192.168.33.3-ipv4-ns"]</data>
      <data key="d3">{"error_message": "", "reservation_id": "17d077ce-b66e-48e0-aafb-a48033a02ff1", "reservation_state": "Active"}</data>
      <data key="d10">LAN</data>
      <data key="d11">false</data>
      <data key="d12">ATLA</data>
      <data key="d21">L3</data>
      <data key="d13">FABNetv4</data>
      <data key="d15">NetworkService</data>
      <data key="d16">c0950a90-1312-4ac4-a7e9-a87aa52d8fda</data>
      <data key="d17">553597ac-26ea-4729-97cd-f9f1b6a23ea9</data>
      <data key="d14">5</data>
    </node>

I think instantiated: False is the problem. network.get_gateway() would not return the IP in this case.

[yoursunny]

GraphML file here: https://cdn1.frocdn.ch/oq8kXrhxQRTtAR4.graphml

I can definitely see the IP addresses in the GraphML file, but it’s not showing up in the list_node_and_networks.ipynb notebook or the net.get_gateway() function call.

I’m using the “default 08/22/2023” JupyterHub environment, with these package versions:

fabric@fall:work-10%$ pip list | grep fabric
fabric 3.2.2
fabric-credmgr-client 1.5.2
fabric_fim 1.5.5
fabric_fss_utils 1.5.1
fabric-orchestrator-client 1.5.5
fabrictestbed 1.5.6
fabrictestbed-extensions 1.5.4
fabrictestbed-mflib 1.0.3

• This reply was modified 2 years, 8 months ago by yoursunny.

[yoursunny]

FABNetv4Ext establishment is working, but I’m see connectivity issues to many destinations.

ubuntu@v4gateway:~$ mtr -4bwz -c4 --tcp -P 6363 hobo.cs.arizona.edu
Start: 2023-09-20T16:06:38+0000
HOST: v4gateway                                                                     Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. AS398900 23.134.233.81                                                          0.0%     4    0.5   0.5   0.5   0.5   0.0
  2. AS???    10.133.0.141                                                           0.0%     4   13.2  13.2  13.1  13.2   0.0
  3. AS11537  hundredge-0-0-0-28.1000.core1.wash.net.internet2.edu (198.71.45.162)   0.0%     4   15.4  15.1  14.5  15.7   0.5
  4. AS???    ???                                                                   100.0     4    0.0   0.0   0.0   0.0   0.0

ubuntu@v4gateway:~$ mtr -4bwz -c4 --tcp -P 5201 ash.speedtest.clouvider.net
Start: 2023-09-20T16:07:49+0000
HOST: v4gateway              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. AS398900 23.134.233.81   0.0%     4    0.5   0.5   0.5   0.5   0.0
  2. AS???    10.133.0.141    0.0%     4   13.4  13.2  13.1  13.4   0.1
  3. AS???    ???            100.0     4    0.0   0.0   0.0   0.0   0.0

Maybe some routing adjustment is needed too?

[Author]

Posts