1. Mert Cevik

Mert Cevik

Forum Replies Created

Viewing 15 posts - 16 through 30 (of 189 total)
  • Author
    Posts
  • in reply to: Lost SSH access to some nodes after node.os_reboot() #8559
    Mert Cevik
    Moderator

      Hello Sunjay,

      I checked the 3 VMs. I could bring the VM node3d2 online manually, but the other two did not succeed. I will suggest re-creating the slice (or modify it to re-create the VMs).

      We also need to check ourselves. Can you point out the notebook you used? I’m assuming one of the notebooks in fabric-examples github repo, but if it’s a customized notebook, please let me know, I will reach out to you via email to get the notebook (or you can attach to this thread if it’s fine for you).

      in reply to: Unable to connect to http://linux.mirrors.es.net/ubuntu #8480
      Mert Cevik
      Moderator

        I just tested access from a TACC VM and  http://linux.mirrors.es.net/ubuntu was reachable. If you’re still having problems, then we need to learn about the specific information from your VM and docker image creation. Please let us know if you still need help on this.

        in reply to: FABRIC PSC – Power outage #8465
        Mert Cevik
        Moderator

          Power outage is resolved. PSC node is online, available for experiments.

          in reply to: FABRIC PSC – Power outage #8466
          Mert Cevik
          Moderator

            Power outage is resolved. PSC node is online, available for experiments.

            in reply to: KANS FPGA is not receiving packets #8401
            Mert Cevik
            Moderator

              I created a FABRIC ticket on Jira – FIP-2180

              If you received an email from this ticket, please “reply to” the email to confirm that we can use it for further communication. If you did not receive an email from the ticket FIP-2180, please notify here on this thread.

               

              • This reply was modified 10 months ago by Mert Cevik.
              in reply to: KANS FPGA is not receiving packets #8399
              Mert Cevik
              Moderator

                That’s not the output that I’m looking for.

                The issue that you are reporting has similarities with some previous cases. I just confirmed the status of WASH FPGA for the specific CMAC status. If you want to run your experiment with the WASH, it’s available now and you can check how traffic will work. If it’s  working case, then I can perform the same intervention on KANS FPGA. In case, you consider using WASH for testing this, you will be able to fit a VM with 4-8 cores and 100GB disk size (available memory is plenty).

                in reply to: KANS FPGA is not receiving packets #8395
                Mert Cevik
                Moderator

                  Network services are verified to be working across the sites.

                  FPGA itself can be the concern. Can you share the output of
                  command = 'cd esnet-smartnic-fw/sn-stack/ && docker compose exec smartnic-fw sn-cli cmac status' ?

                  in reply to: FABRIC HAWI – Management network outage #8354
                  Mert Cevik
                  Moderator

                    As an update,

                    according to our tests and observations since reporting the issue to the hosting site, this issue did not repeat itself and we conclude that it’s resolved.

                    in reply to: Cannot connect to download.docker.com #8351
                    Mert Cevik
                    Moderator

                      This problem is caused by some issues within the hosting site’s and/or its upstream provider’s IPv6 routing, specific for the EDUKY node. Work is in progress to address this issue, but we don’t have a time yet for a resolution.

                      FABRIC provides the following VM images that have docker (and openvswitch) installed in them
                      – docker_ubuntu_20
                      – docker_ubuntu_22
                      – docker_ubuntu_24
                      – docker_rocky_8

                       

                      On the other hand, the issue may present itself for some other specific servers. As a workaround, the following may help for a temporary solution.

                      Get the IPv4 address of the server
                      get.docker.com –> 108.157.150.23
                      download.docker.com –> 108.157.150.36

                      Adjust NAT64 addresses wrt FABRIC’s NAT64 gateway prefix
                      get.docker.com –> 2600:2701:5000:5001::6c9d:9617
                      download.docker.com –> 2600:2701:5000:5001::6c9d:9624

                      Add to /etc/hosts

                      echo “2600:2701:5000:5001::6c9d:9617 get.docker.com” >> /etc/hosts
                      echo “2600:2701:5000:5001::6c9d:9624 download.docker.com” >> /etc/hosts

                      Then, you should be able to reach these sites (routed to their IPv4 addresses through FABRIC’s NAT64)

                      ubuntu@Node1:~$ curl -I https://get.docker.com
                      HTTP/1.1 200 OK
                      Content-Type: text/plain
                      Content-Length: 22592
                      Connection: keep-alive
                      Date: Mon, 10 Mar 2025 08:24:07 GMT
                      Last-Modified: Mon, 23 Dec 2024 12:06:43 GMT
                      x-amz-version-id: jBhC8udY54KPazAadGzx.egHAVktsNCa
                      ETag: “1a91bbd5af88e59c539b2f1ac7490347”
                      Server: AmazonS3
                      X-Cache: Hit from cloudfront
                      Via: 1.1 3ce25b408212b0431973764f2829bcd2.cloudfront.net (CloudFront)
                      X-Amz-Cf-Pop: MCI50-P2
                      X-Amz-Cf-Id: NtXIyn1U5TQwjsaleKkIXEG7buaVTJN7vPExXgeAiqDGwVdHe9Z6pA==
                      Age: 17673

                      in reply to: Network outage for FABRIC-TACC #8325
                      Mert Cevik
                      Moderator

                        Network outage is resolved. VMs on TACC node are accessible.

                        in reply to: List of OS supported #8318
                        Mert Cevik
                        Moderator
                          in reply to: Brief maintenance today at 12pm EST #8284
                          Mert Cevik
                          Moderator

                            This maintenance is completed. All testbed services are available.

                            in reply to: Maintenance on the Knowledgebase (learn.fabric-testbed.net) #8276
                            Mert Cevik
                            Moderator

                              Work is completed, learn.fabric-testbed.net is online.

                              in reply to: Cannot login to SALT node from my local machine #8272
                              Mert Cevik
                              Moderator
                                in reply to: Power outage on FABRIC-CLEM #7974
                                Mert Cevik
                                Moderator

                                  Dear Experimenters,

                                  FABRIC-CLEM is back online. VMs are accessible, their PCI devices are re-attached, however IP addresses may need to be re-assigned by the experimenters.

                                  We have an issue on the dataplane that is interrupting traffic for L2STS and Layer3 services. We will post updates.

                                   

                                Viewing 15 posts - 16 through 30 (of 189 total)