1. Node#execute runs but node cannot be accessed from command-line

Node#execute runs but node cannot be accessed from command-line

Home Forums FABRIC General Questions and Discussion Node#execute runs but node cannot be accessed from command-line

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • #2048
    Mason Hicks
    Participant

      I can’t SSH into my nodes on the FABRIC JupyterLab, but using node.execute I am able to remotely access the node.

      For reproducibility, I have used the hello_fabric.ipynb to look into this issue. I have attached a version of the hello_fabric.ipynb exactly as I have ran it, including all of the outputs and environment variable definitions.

      When I try and run the SSH Command associated with my node, I get:

      $ ssh -i /home/fabric/.ssh/id_rsa -J mwhicks2_0037146471@bastion-1.fabric-testbed.net rocky@129.114.110.125
      The authenticity of host ‘bastion-1.fabric-testbed.net (152.54.15.12)’ can’t be established.
      ECDSA key fingerprint is SHA256:AIRhefx5rhgEfSSoO8NIc6g+ohFQuSU0yn0i7qGUkY8.
      Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
      Warning: Permanently added ‘bastion-1.fabric-testbed.net,152.54.15.12’ (ECDSA) to the list of known hosts.
      mwhicks2_0037146471@bastion-1.fabric-testbed.net: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
      kex_exchange_identification: Connection closed by remote host

      Not sure what is going wrong here. I have also attached an ssh -v attempt here: https://pastebin.com/raw/CXEZHziG

      #2165
      Ilya Baldin
      Participant

        Mason,

        Here is a longer explanation of how to properly store your keys and SSH configuration in Jupyter Hub:

        The keys in /home/fabric/.ssh/ are not persistent across the container boots. It’s user’s responsibility to upload their keys at a location under /home/fabric/work directory, which is persistent. They can then point FABLib to use that as below:

        By default a rc file with the required environment variables is created at /home/fabric/work/fabric_config/fabric_rc

        export FABRIC_BASTION_KEY_LOCATION=/home/fabric/work/fabric_config/fabric-bastion
        export FABRIC_SLICE_PRIVATE_KEY_FILE=/home/fabric/work/fabric_config/id_rsa
        export FABRIC_SLICE_PUBLIC_KEY_FILE=/home/fabric/work/fabric_config/id_rsa.pub

        Workflow for SSH:

        1. Update /home/fabric/work/fabric_config/ssh_config to reflect the correct keys and bastion user name
          SSH to the VMs using the following command

        ssh -F ~/work/fabric_config/ssh_config -i ~/work/fabric_config/id_rsa <username>@<mgmt-ip>

        Workflow for Notebooks:

        1. Update the environment variables in /home/fabric/work/fabric_config/fabric_rc
        2. Import the environment variables in the notebooks as below

        fabric_rc_location=os.environ[‘HOME’]+”/work/fabric_config/fabric_rc”
           if os.path.exists(fabric_rc_location):
              with open(fabric_rc_location, ‘r’) as f:
                 for line in f:
                    if line.startswith(‘export’):
                       os.environ[line.split(‘=’)[0].split(‘export’)[1].strip()] = line.split(‘=’)[1].strip()

        ssh_key_file_priv=os.environ[“FABRIC_SLICE_PRIVATE_KEY_FILE”]
        ssh_key_file_pub=os.environ[“FABRIC_SLICE_PUBLIC_KEY_FILE”]

        ssh_key_pub = None
        with open (ssh_key_file_pub, “r”) as myfile:
           ssh_key_pub=myfile.read()
           ssh_key_pub=ssh_key_pub.strip()

      Viewing 2 posts - 1 through 2 (of 2 total)
      • You must be logged in to reply to this topic.