[Fraida Fund]

They’re back up. thanks!

[Fraida Fund]

Thanks for keeping me informed!

[Fraida Fund]

The project did not have any members before.

I realized that the CSV file had an “Email” header at the top. It appears that the first existing FABRIC user with “email” in their email address was matched to this line. You can reproduce with a blank file with just the text

Email

in it. Similarly, if I use this CSV file:

nyu

it tries to add the first FABRIC user with “nyu” in their email address.

It seems to be matching on partial string instead of the entire string, so I guess if there were two users “xx@email.org” and “axx@email.org”, and I upload a CSV with “xx@email.org”, it might match to “axx@email.org” instead of “xx@email.org”.

[Fraida Fund]

Perhaps the instructions at https://learn.fabric-testbed.net/knowledge-base/obtaining-and-using-fabric-api-tokens/#using-tokens-within-the-jupyter-hub can be updated. Currently, it says to generate a new token and upload to JH when you get a “Refresh Token: (invalid grant)” error. But at least for this instance of that error it doesn’t work (and, my students say that solution also has not worked for them when they encounter this error) – whatever is not initialized properly fails even with a new token. It only works if the JH is stopped and restarted from the Hub Control Panel.

[Fraida Fund]

Following up on this to share more info –

In Step 4 above, the first JH server I start after the timeout does have a new refresh token. the contents of .tokens.json show the token is created when I start the JH server:

{
    "refresh_token": "XXX",
    "created_at": "2023-08-30 15:13:26"
}

but when I try to use fablib I get that token error, and no ID token.

After stopping the JH server from the Hub Control Panel and starting it again, then it gets another new refresh token – .tokens.json has –

{
    "refresh_token": "XXX",
    "created_at": "2023-08-30 15:16:47"
}

and this one works. When attempting to use fablib, I get an ID token and no error.

Not clear why the first refresh token does not work, even though it is new.

[Fraida Fund]

Thanks, the part that I consider a “bug” is that when I log in again and start a new server in Step 4, it does not get a new “good” token.  Is that behavior expected?

[Fraida Fund]

Thanks, I appreciate the update!

[Fraida Fund]

(related question – is that jupyter_startup.py anywhere in the fabric-testbed Github? I thought it should be https://github.com/fabric-testbed/jupyternb-setup but that hasn’t been updated in a while, and neither branch matches what’s currently on the “default” server.)

[Fraida Fund]

Thanks for following up! I look forward to trying this feature.

[Fraida Fund]

Hi! I wanted to follow up on this, since this functionality is used in educational materials, I am working to transition those materials ahead of the imminent retirement of InstaGENI, and I need to consider what platform to transition them to.

Is this issue expected to be fix-able? If yes, is there a rough timeline? (Is it likely to be fixed before InstaGENI is retired?)

[Fraida Fund]

Thanks. Did I get this right –

• A dedicated  ConnectX-6/5 has its full bandwidth within a site (even in a hypothetical situation where the site has high utilization)
• A dedicated ConnectX-6/5s is currently best effort between sites, but eventually we’ll be able to reserve bandwidth on these links between sites.
• Basic NICs have (and only ever will have) best effort, with a 780 Mbps minimum in the hypothetical where the site has high utilization.

[Fraida Fund]

Thanks! Could you clarify this point –

Basic NICs: The existing Basic NICs are implemented as SR-IOV virtual functions on a 100Gbps ConnectX-6.  The only limitation is that the bandwidth is shared with the other Basic NICs on that port.

This means that 100 Gbps is divided by all of the Basic NICs on that port, and the port may be shared by Basic NICs across my slice but also other users’ slices? Hypothetically, if all 128 SR-IOV VFs on the port are used, then the bandwidth could max out at ~780 Mbps? (And I don’t have any visibility into how many SR-IOV VFs are on the port.)

[Fraida Fund]

As an experimenter, I would prefer if images were not updated so that I could develop experiments against a hosted image, and I wouldn’t have to keep updating experiments to reflect the latest software versions.

(Keeping the images stable gives me two choices: I could update to latest software versions, or I could choose not to. Updating the images means I have no choice.)

This is especially a concern for e.g. education experiments, where we may also record video materials to go along with each experiment, and it’s very time intensive to prepare. I have a strong interest in those experiments staying stable.

Maybe there could be one hosted image for each OS that is updated (e.g. “default_ubuntu_latest” is keep updated and there is an announcement so we know when it is updated), but also keep some stable images (e.g. “default_ubuntu_20” is not updated except for security updates).

[Fraida Fund]

Yes, let’s discuss further. I can think of a bunch of scenarios where we would want the interfaces to be in “promiscuous mode” and in some of them, it will not be practical to use dedicated interfaces (we need too many interfaces in “promiscuous mode”).

[Fraida Fund]

Well, one observation is that Paul said so elsewhere in the forum.

L2Bridge: These bridges are like a local network switch/bridge that connects any number of local nodes within a single site.  These local bridges are directly connected to the nodes so your bandwidth will be limited by the maximum bandwidth of the NICs that you are using (i.e. ConnectX_6 NICs will provide 100Gbps). This bridge is not programable and only performs simple MAC learning. The key use of these bridges are that they can only connect to nodes within a single FABRIC site.

Another observation: suppose I create 4 VMs with a basic NIC on each, and connect each basic NIC to an L2Bridge-type network. I capture traffic on each of the NICs with tcpdump. A frame sent by host 1 with host 2’s address as the destination MAC only appears at the NIC on host 2, and not on host 3 or host 4.

Another observation: suppose I create a Linux switch connecting multiple hosts, using a L2Bridge between my Linux switch and each of the connected hosts. (as in e.g. this example.) Non-broadcast frames sent from the hosts don’t make it to the Linux switch interfaces, so the bridge does not work.

What I am trying to do: I am trying to connect multiple basic NIC interfaces with a network link, so that any frame sent by any NIC on the link appears at every other NIC on the link. Like the way an Ethernet segment behaves, or Ethernet segments connected by a hub, or Ethernet segments connected by a switch with MAC learning disabled.

[Author]

Posts