Forum Replies Created
-
Posts
-
Hello,
Yesterday some services were affected from an outage and the errors for loading projects and extension might be due to the problem with the affected services. FABRIC team checked all services to confirm they are restored and I also confirmed that slice extension is working well.
Due to the FABRIC team’s schedules, this outage might not have been communicated via the announcements, apologies for this inconvenience.
Hello Sunjay,
I checked the 3 VMs. I could bring the VM node3d2 online manually, but the other two did not succeed. I will suggest re-creating the slice (or modify it to re-create the VMs).
We also need to check ourselves. Can you point out the notebook you used? I’m assuming one of the notebooks in fabric-examples github repo, but if it’s a customized notebook, please let me know, I will reach out to you via email to get the notebook (or you can attach to this thread if it’s fine for you).
I just tested access from a TACC VM and http://linux.mirrors.es.net/ubuntu was reachable. If you’re still having problems, then we need to learn about the specific information from your VM and docker image creation. Please let us know if you still need help on this.
I created a FABRIC ticket on Jira – FIP-2180
If you received an email from this ticket, please “reply to” the email to confirm that we can use it for further communication. If you did not receive an email from the ticket FIP-2180, please notify here on this thread.
That’s not the output that I’m looking for.
The issue that you are reporting has similarities with some previous cases. I just confirmed the status of WASH FPGA for the specific CMAC status. If you want to run your experiment with the WASH, it’s available now and you can check how traffic will work. If it’s working case, then I can perform the same intervention on KANS FPGA. In case, you consider using WASH for testing this, you will be able to fit a VM with 4-8 cores and 100GB disk size (available memory is plenty).
Network services are verified to be working across the sites.
FPGA itself can be the concern. Can you share the output of
command = 'cd esnet-smartnic-fw/sn-stack/ && docker compose exec smartnic-fw sn-cli cmac status'?As an update,
according to our tests and observations since reporting the issue to the hosting site, this issue did not repeat itself and we conclude that it’s resolved.
This problem is caused by some issues within the hosting site’s and/or its upstream provider’s IPv6 routing, specific for the EDUKY node. Work is in progress to address this issue, but we don’t have a time yet for a resolution.
FABRIC provides the following VM images that have docker (and openvswitch) installed in them
– docker_ubuntu_20
– docker_ubuntu_22
– docker_ubuntu_24
– docker_rocky_8On the other hand, the issue may present itself for some other specific servers. As a workaround, the following may help for a temporary solution.
Get the IPv4 address of the server
get.docker.com –> 108.157.150.23
download.docker.com –> 108.157.150.36Adjust NAT64 addresses wrt FABRIC’s NAT64 gateway prefix
get.docker.com –> 2600:2701:5000:5001::6c9d:9617
download.docker.com –> 2600:2701:5000:5001::6c9d:9624Add to /etc/hosts
echo “2600:2701:5000:5001::6c9d:9617 get.docker.com” >> /etc/hosts
echo “2600:2701:5000:5001::6c9d:9624 download.docker.com” >> /etc/hostsThen, you should be able to reach these sites (routed to their IPv4 addresses through FABRIC’s NAT64)
ubuntu@Node1:~$ curl -I https://get.docker.com
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 22592
Connection: keep-alive
Date: Mon, 10 Mar 2025 08:24:07 GMT
Last-Modified: Mon, 23 Dec 2024 12:06:43 GMT
x-amz-version-id: jBhC8udY54KPazAadGzx.egHAVktsNCa
ETag: “1a91bbd5af88e59c539b2f1ac7490347”
Server: AmazonS3
X-Cache: Hit from cloudfront
Via: 1.1 3ce25b408212b0431973764f2829bcd2.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: MCI50-P2
X-Amz-Cf-Id: NtXIyn1U5TQwjsaleKkIXEG7buaVTJN7vPExXgeAiqDGwVdHe9Z6pA==
Age: 17673
