Forum Replies Created
-
AuthorPosts
-
August 29, 2024 at 12:25 pm in reply to: Unable to (consistently) reach FABNetv4Ext addresses from outside FABRIC #7480
Hi Sourya,
You will need permission for FabNetv*Ext services to enable public connections to your VMs. This request can be made by your Project Lead. For more details, please check here!
Thanks,
KomalHi Prateek,
This looks like a bug, we have a race condition which is preventing the updates to the Slice Graph Model. I will work on addressing this. For now as a workaround, you can determine the IP Addresses via slice commander using show commands. Refer https://learn.fabric-testbed.net/knowledge-base/using-slicecommander-with-fabric/ for slice-commander usage.
Alternatively, you can get the sliver information also via
for s in slice.get_slivers():
print(s._sliver) # This is a json object will provide the needed information
Thanks,
Komal
Hi Prateek,
Could you please share your slice ID?
Thanks,
Komal
Hi Kriti,
This looks like a version mismatch for the
fabrictestbed-extensions
. Are you running into this any of the JH containers? If so, please ensure you have no entries in/home/fabric/work/fabric_config/requirements.txt
and then restart your JH container via File -> Hub Start Control Panel -> Stop My Server -> Start My Server.If you are running into this on you local environment, please consider updating to the latest fablib via
pip install fabrictestbed-extensions==1.7.3
Thanks,
Komal
Thank you, @yoursunny, for the excellent suggestion!
@Jestus,
In addition to @yoursunny’s point, the Project Lead for your project will need to request permissions to enable FabNetv*Ext services. You can find more details here.
Additionally, we recommend avoiding the use of the management interface for data transfer and instead utilizing the FABNetv4Ext/FABNetv6Ext services. Please feel free to reach out if you have any questions or comments.
Thanks,
KomalJust closing the loop here as the issue was resolved over a short zoom call.
The error message “The token is not yet valid (iat)” indicates that the token’s “issued at” (
iat
) timestamp is set in the future compared to the current time. This can happen if there is a clock discrepancy between the system where the token was generated and the system validating the token.Time on the server was behind current time, synching the clock with NTP server resolved the error.
Thanks,
Komal
Hi Yu,
I created a token and successfully validated it using the Credential Manager, but I am unable to reproduce the issue you’re experiencing. This error can sometimes occur due to stale cookies in the browser. Could you please try opening https://cm.fabric-testbed.net/ in an incognito window, then generate and validate the token? Let me know if the error persists.
Thanks,
Komal
-
This reply was modified 9 months, 3 weeks ago by
Komal Thareja.
Also, could you please post queries in this channel instead https://learn.fabric-testbed.net/forums/forum/fabric-general-questions-and-discussion/
Thanks,
Komal
Hi Yuanjun,
You can check the GPU types from the portal. For example information about STAR can be found here: https://portal.fabric-testbed.net/sites/STAR
Thanks,
Komal
Hi Prateek,
I recommend setting up your own FABRIC environment on your laptop or machine to run your experiments. This approach will allow you to capture more data and reduce reliance on Jupyter Hub. Consider configuring a local Python environment for the FABRIC API as described here, and run the notebooks locally.
Thanks,
Komal
Sharing the SSH and SCP commands again, due to an error in the above:
ssh -i /path/to/vm/private/key -J bastion.fabric-testbed.net [username]@[VM_IP]
scp -i /path/to/vm/private/key -o "ProxyJump bastion.fabric-testbed.net" [username]@[VM_IP]:file.txt .
scp -i /path/to/vm/private/key -o "ProxyJump bastion.fabric-testbed.net" ubuntu@[2001:400:a100:3030:f816:3eff:fe93:a1a0]:file.txt .
-
This reply was modified 9 months, 4 weeks ago by
Komal Thareja.
Hi Prateek,
The
/home/fabric/work
directory in the JupyterHub environment serves as persistent storage for code, notebooks, scripts, and other materials related to configuring and running experiments, including the addition of extra Python modules. However, it is not designed to handle large datasets.**Transfer via the Bastion Host**
All virtual machines (VMs) are connected to a management network that you use for SSH access. You may have noticed that accessing your VMs requires jumping through a bastion host for enhanced security.
The simplest method for transferring data to and from a VM is by usingscp
(or a similar tool). However, you must still route through the bastion host. To do this, you will need to configure your external machine (such as your laptop) to use the bastion host for standard SSH connections.
You can find SSH configuration details here: [Logging into FABRIC VMs](https://learn.fabric-testbed.net/knowledge-base/logging-into-fabric-vms/).
The key takeaway from that document is to create (or modify) your~/.ssh/config
file to include the bastion host configuration. A basic setup might look like this:
#### Bastion
UserKnownHostsFile /dev/null
StrictHostKeyChecking no
ServerAliveInterval 120
Host bastion-?.fabric-testbed.net
User
ForwardAgent yes
Hostname %h
IdentityFile /path/to/bastion/private/key
IdentitiesOnly yes
Host bastion-*.fabric-testbed.net
User
IdentityFile /path/to/bastion/private/key
After configuring, you should be able to SSH with the following command:
ssh -i /path/to/vm/private/key [username]@[VM_IP]
For SCP, you can use the command:
scp -i /path/to/vm/private/key [username]@[VM_IP]:file.txt .
Keep in mind that if the VM IP address uses IPv6, colons can interfere with parsing. In this case, you need to escape the characters (escaping is necessary on macOS but not in Bash; however, using escapes works in both environments). Here’s an example:
scp -i /path/to/vm/private/key ubuntu@[2001:400:a100:3030:f816:3eff:fe93:a1a0]:file.txt .Thanks,
KomalAugust 12, 2024 at 3:13 pm in reply to: Maintenance Network AM – 08/12/2024 – 1:00 pm – 2:00 pm #7411Maintenance has been completed!
Thanks,
KomalAugust 12, 2024 at 11:08 am in reply to: Unable to (consistently) reach FABNetv4Ext addresses from outside FABRIC #7409Hi Sunjay,
We did have an dataplane outage at SALT since Friday. It has been resolved now. FabNetv4Ext service on SALT should pass traffic. Thank you for sharing your observations and helping us address this quickly.
Thanks,
KomalAugust 9, 2024 at 5:17 pm in reply to: Unable to (consistently) reach FABNetv4Ext addresses from outside FABRIC #7406Apologies, you are right that sliver is definitely
Active
I did notice dataplane issue for SALT, I will reach out to Tom and share updates.
Reservation ID: 7dafe47e-3058-4282-9d09-66fe74588e1c Slice ID: 1c057890-bc44-450f-b109-995fef665216
Resource Type: FABNetv4Ext Notices: Reservation 7dafe47e-3058-4282-9d09-66fe74588e1c (Slice qbss-tester(1c057890-bc44-450f-b109-995fef665216) Graph Id:b097687e-9cd9-45ac-89bb-d12a73c2da46 Owner:scauligi@icsi.berkeley.edu) is in state (Active,None_)
Start: 2024-08-09 19:42:09 +0000 End: 2025-02-04 20:38:01 +0000 Requested End: 2025-02-04 20:38:01 +0000
Units: 1 State: Active Pending State: None_
-
This reply was modified 9 months, 3 weeks ago by
-
AuthorPosts