1. Mert Cevik

Mert Cevik

Forum Replies Created

Viewing 15 posts - 1 through 15 (of 177 total)
  • Author
    Posts
  • Mert Cevik
    Moderator

      Maintenance is completed.

      Mert Cevik
      Moderator

        Maintenance completed. WASH node is online.

        Mert Cevik
        Moderator

          Maintenance completed. KANS node is online.

          in reply to: Lost SSH access to some nodes after node.os_reboot() #8559
          Mert Cevik
          Moderator

            Hello Sunjay,

            I checked the 3 VMs. I could bring the VM node3d2 online manually, but the other two did not succeed. I will suggest re-creating the slice (or modify it to re-create the VMs).

            We also need to check ourselves. Can you point out the notebook you used? I’m assuming one of the notebooks in fabric-examples github repo, but if it’s a customized notebook, please let me know, I will reach out to you via email to get the notebook (or you can attach to this thread if it’s fine for you).

            in reply to: Unable to connect to http://linux.mirrors.es.net/ubuntu #8480
            Mert Cevik
            Moderator

              I just tested access from a TACC VM and  http://linux.mirrors.es.net/ubuntu was reachable. If you’re still having problems, then we need to learn about the specific information from your VM and docker image creation. Please let us know if you still need help on this.

              in reply to: FABRIC PSC – Power outage #8465
              Mert Cevik
              Moderator

                Power outage is resolved. PSC node is online, available for experiments.

                in reply to: FABRIC PSC – Power outage #8466
                Mert Cevik
                Moderator

                  Power outage is resolved. PSC node is online, available for experiments.

                  in reply to: KANS FPGA is not receiving packets #8401
                  Mert Cevik
                  Moderator

                    I created a FABRIC ticket on Jira – FIP-2180

                    If you received an email from this ticket, please “reply to” the email to confirm that we can use it for further communication. If you did not receive an email from the ticket FIP-2180, please notify here on this thread.

                     

                    • This reply was modified 2 months, 3 weeks ago by Mert Cevik.
                    in reply to: KANS FPGA is not receiving packets #8399
                    Mert Cevik
                    Moderator

                      That’s not the output that I’m looking for.

                      The issue that you are reporting has similarities with some previous cases. I just confirmed the status of WASH FPGA for the specific CMAC status. If you want to run your experiment with the WASH, it’s available now and you can check how traffic will work. If it’s  working case, then I can perform the same intervention on KANS FPGA. In case, you consider using WASH for testing this, you will be able to fit a VM with 4-8 cores and 100GB disk size (available memory is plenty).

                      in reply to: KANS FPGA is not receiving packets #8395
                      Mert Cevik
                      Moderator

                        Network services are verified to be working across the sites.

                        FPGA itself can be the concern. Can you share the output of
                        command = 'cd esnet-smartnic-fw/sn-stack/ && docker compose exec smartnic-fw sn-cli cmac status' ?

                        in reply to: FABRIC HAWI – Management network outage #8354
                        Mert Cevik
                        Moderator

                          As an update,

                          according to our tests and observations since reporting the issue to the hosting site, this issue did not repeat itself and we conclude that it’s resolved.

                          in reply to: Cannot connect to download.docker.com #8351
                          Mert Cevik
                          Moderator

                            This problem is caused by some issues within the hosting site’s and/or its upstream provider’s IPv6 routing, specific for the EDUKY node. Work is in progress to address this issue, but we don’t have a time yet for a resolution.

                            FABRIC provides the following VM images that have docker (and openvswitch) installed in them
                            – docker_ubuntu_20
                            – docker_ubuntu_22
                            – docker_ubuntu_24
                            – docker_rocky_8

                             

                            On the other hand, the issue may present itself for some other specific servers. As a workaround, the following may help for a temporary solution.

                            Get the IPv4 address of the server
                            get.docker.com –> 108.157.150.23
                            download.docker.com –> 108.157.150.36

                            Adjust NAT64 addresses wrt FABRIC’s NAT64 gateway prefix
                            get.docker.com –> 2600:2701:5000:5001::6c9d:9617
                            download.docker.com –> 2600:2701:5000:5001::6c9d:9624

                            Add to /etc/hosts

                            echo “2600:2701:5000:5001::6c9d:9617 get.docker.com” >> /etc/hosts
                            echo “2600:2701:5000:5001::6c9d:9624 download.docker.com” >> /etc/hosts

                            Then, you should be able to reach these sites (routed to their IPv4 addresses through FABRIC’s NAT64)

                            ubuntu@Node1:~$ curl -I https://get.docker.com
                            HTTP/1.1 200 OK
                            Content-Type: text/plain
                            Content-Length: 22592
                            Connection: keep-alive
                            Date: Mon, 10 Mar 2025 08:24:07 GMT
                            Last-Modified: Mon, 23 Dec 2024 12:06:43 GMT
                            x-amz-version-id: jBhC8udY54KPazAadGzx.egHAVktsNCa
                            ETag: “1a91bbd5af88e59c539b2f1ac7490347”
                            Server: AmazonS3
                            X-Cache: Hit from cloudfront
                            Via: 1.1 3ce25b408212b0431973764f2829bcd2.cloudfront.net (CloudFront)
                            X-Amz-Cf-Pop: MCI50-P2
                            X-Amz-Cf-Id: NtXIyn1U5TQwjsaleKkIXEG7buaVTJN7vPExXgeAiqDGwVdHe9Z6pA==
                            Age: 17673

                            in reply to: Network outage for FABRIC-TACC #8325
                            Mert Cevik
                            Moderator

                              Network outage is resolved. VMs on TACC node are accessible.

                              in reply to: List of OS supported #8318
                              Mert Cevik
                              Moderator
                                in reply to: Brief maintenance today at 12pm EST #8284
                                Mert Cevik
                                Moderator

                                  This maintenance is completed. All testbed services are available.

                                Viewing 15 posts - 1 through 15 (of 177 total)