1. Cannot SSH to VMs on newy-w2.fabric-testbed.net

Cannot SSH to VMs on newy-w2.fabric-testbed.net

Home Forums FABRIC General Questions and Discussion Cannot SSH to VMs on newy-w2.fabric-testbed.net

Viewing 15 posts - 1 through 15 (of 16 total)
  • Author
    Posts
  • #6174
    Fraida Fund
    Participant

      slice ID: 14fc881e-5621-4597-82b7-e855d9a029f3

      I can’t connect to my VMs anymore. e.g. (same SSH command worked a few minutes earlier) –

      fabric@fall:fabric-95%$ ssh -i /home/fabric/work/fabric_config/slice_key -F /home/fabric/work/fabric_config/ssh_config ubuntu@2001:400:a100:3040:f816:3eff:fe29:9737 
      Warning: Permanently added 'bastion.fabric-testbed.net' (ED25519) to the list of known hosts.
      channel 0: open failed: connect failed: No route to host
      stdio forwarding failed
      kex_exchange_identification: Connection closed by remote host
      Connection closed by UNKNOWN port 65535
      #6175
      Hussam Nasir
      Moderator

        Due to an issue just discovered in newy-w2, we had to reboot the server. Please try your ssh in about 30 minutes. It should be up by then.

        #6176
        Fraida Fund
        Participant

          They’re back up. thanks!

          #6501
          Sanjana Das
          Participant

            Hello! I am new to Fabric and I have been getting the same error Fraida got. I created the bastion SSH keys using the portal and used the configure file in JupytetHub to configure the sliver keys and the SSH config file. A coworker of mine has also been facing the same issue. Where could we be going wrong?

            #6512
            Komal Thareja
            Participant

              @Sanjana,

              Could you please follow the notebook: jupyter-exanples-1.6.*/configure_and_validate.ipynb to ensure your configuration is correct? Please try creating a slice after that and let us know if you still run into errors.

               

              Thanks,

              Komal

              1 user thanked author for this post.
              #6516
              Sanjana Das
              Participant

                Hello! I have followed that notebook and created a slice! moreover, I also added the public key to the node in the slice using the ‘Add/Remove SSh keys’ notebook. On a side note, I am using the sliver keys generated by jupyterhub.

                #6517
                Komal Thareja
                Participant

                  @Sanjana – glad to hear that you were able to create a slice successfully, are you still facing errors when you SSH to the VMs?

                  Thanks,

                  Komal

                  #6518
                  Sanjana Das
                  Participant

                    Hello! Yes I am still facing the same issue

                    #6519
                    Komal Thareja
                    Participant

                      @Sanjana – Could you please remove this file if it exists ~/work/fabric_config/ssh_config and run the validate_and_config.ipynb  notebook again?

                      Please try SSH to VMs via command line and share the error you see if it still fails.

                      Thanks,

                      Komal

                      #6522
                      Sanjana Das
                      Participant

                        I did what you suggested and I am still getting the same error from the jupyterhub terminal as well as my local machine terminal.

                        Warning: Permanently added ‘bastion.fabric-testbed.net’ (ED25519) to the list of known hosts.
                        channel 0: open failed: connect failed: Connection timed out
                        stdio forwarding failed
                        kex_exchange_identification: Connection closed by remote host
                        Connection closed by UNKNOWN port 65535

                        #6524
                        Komal Thareja
                        Participant

                          @Sanjana,

                          Could you please share the output of following commands:?

                          cat ~/work/fabric_config/fabric_rc

                          cat ~/work/fabric_config/ssh_config

                          ls -ltr ~/work/fabric_config

                          Thanks,

                          Komal

                          #6531
                          Sanjana Das
                          Participant

                            Yes definitely!

                            cat fabric_config/fabric_rc
                            export FABRIC_ORCHESTRATOR_HOST=orchestrator.fabric-testbed.net
                            export FABRIC_CREDMGR_HOST=cm.fabric-testbed.net
                            export FABRIC_CORE_API_HOST=uis.fabric-testbed.net
                            export FABRIC_TOKEN_LOCATION=/home/fabric/.tokens.json
                            export FABRIC_PROJECT_ID=3b5442dc-e800-4aff-991c-3580388c9c2a
                            export FABRIC_BASTION_HOST=bastion.fabric-testbed.net
                            export FABRIC_BASTION_USERNAME=sanjana_das_0000109734
                            export FABRIC_BASTION_KEY_LOCATION=/home/fabric/work/fabric_config/fabbk
                            export FABRIC_SLICE_PUBLIC_KEY_FILE=/home/fabric/work/fabric_config/slice_key.pub
                            export FABRIC_SLICE_PRIVATE_KEY_FILE=/home/fabric/work/fabric_config/slice_key
                            export FABRIC_AVOID=
                            export FABRIC_SSH_COMMAND_LINE=ssh -i {{ _self_.private_ssh_key_file }} -F /home/fabric/work/fabric_config/ssh_config {{ _self_.username }}@{{ _self_.management_ip }}
                            export FABRIC_LOG_LEVEL=INFO
                            export FABRIC_LOG_FILE=/tmp/fablib/fablib.log
                            export FABRIC_BASTION_SSH_CONFIG_FILE=/home/fabric/work/fabric_config/ssh_config

                            cat fabric_config/ssh_config
                            UserKnownHostsFile /dev/null
                            StrictHostKeyChecking no
                            ServerAliveInterval 120

                            Host bastion.fabric-testbed.net
                            User sanjana_das_0000109734
                            ForwardAgent yes
                            Hostname %h
                            IdentityFile /home/fabric/work/fabric_config/fabbk
                            IdentitiesOnly yes

                            Host * !bastion.fabric-testbed.net
                            ProxyJump sanjana_das_0000109734@bastion.fabric-testbed.net:22

                            ls -ltr fabric_config
                            total 36
                            -rw-rw—- 1 fabric users 0 Jan 22 17:43 requirements.txt
                            -rw-rw—- 1 fabric users 121 Jan 22 17:43 fabric_config.json
                            -rw——- 1 fabric users 2575 Jan 22 17:43 slice_key
                            -rw-r–r– 1 fabric users 569 Jan 22 17:43 slice_key.pub
                            -rw-rw-r– 1 fabric users 72 Jan 22 17:46 Untitled.ipynb
                            -rw-rw-r– 1 fabric users 166 Jan 31 15:15 fabbk.pub
                            -rw——- 1 fabric users 492 Jan 31 15:15 fabbk
                            -rw-rw-r– 1 fabric users 3510 Feb 1 20:06 fabric_ssh_tunnel_tools.tgz
                            -rw-rw-r– 1 fabric users 412 Feb 5 17:39 ssh_config
                            -rw-rw—- 1 fabric users 967 Feb 5 17:39 fabric_rc

                            #6540
                            Komal Thareja
                            Participant

                              @Sanjana – your config looks correct. Could you please try creating a slice and try SSH? If the SSH still fails, could you please share your public key: /home/fabric/work/fabric_config/slice_key.pub

                              I can verify the correct key is being passed to the VM or not from backend.

                              Thanks,

                              Komal

                              #6541
                              Sanjana Das
                              Participant

                                okay I will try to do that. I just want to confirm that after creating a slice and adding a node to it I will also need to add the public slice_key to that node using the “add/remove ssh keys” notebook right? and then I will be able to ssh into the vm

                                #6542
                                Komal Thareja
                                Participant

                                  No, you don’t need to run “add/remove SSH keys” notebook. By default, when you create a slice, keys specified in your fabric_rc are already added to the VMs.

                                Viewing 15 posts - 1 through 15 (of 16 total)
                                • You must be logged in to reply to this topic.