1. Disk-to-Disk network transfer files between Fabric nodes

Disk-to-Disk network transfer files between Fabric nodes

Home Forums FABRIC General Questions and Discussion Disk-to-Disk network transfer files between Fabric nodes

Viewing 5 posts - 1 through 5 (of 5 total)
  • Author
    Posts
  • #4140
    Xusheng Ai
    Participant

      Hello Fabric,

      We are testing the Network performance between two Fabric nodes, and we initially used iperf to transfer files between two nodes, which is a memory-to-memory transformation. However, we are exploring the possibility of using scp to perform disk-to-disk transfers between two Fabric nodes. We were wondering if there is any suggestions for achieving this.

      Thanks,
      Best Regards,
      Xusheng

      #4141
      Gregory Daues
      Participant

        We in CMB-S4 project  have been copying files between Fabric nodes using scp for testing.

        The elements of that setup include
        1) install scp (something ‘yum install openssh-clients’, though it depends on the platform)

        2) create an L3 network on each of  the two nodes

        3) add a route between these

        4) setup ssh keys  ( id_rsa id.rsa.pub ) on nodes, add entry to ~/.ssh/authorized_keys

        We can provide more details on each of these steps if it helps. And some snippets of this setup will be presented in the presentation of Don Petravick et al. Wed Apr 26 in the meeting.

        Beyond copying files with scp, we are also looking for a more performant way to copy files; some testing with bbcp had some brief success but it has not worked consistently, don’t think it supports IPv6, so we continue to look for performant file transfer approaches.

         

         

         

         

         

        #4147
        Xusheng Ai
        Participant

          On step 4, do we need to generate the ssh keys on the node itself or use the keys associated with the slice?

          #4148
          Xusheng Ai
          Participant

            On step 4, do we need to generate the ssh keys on the node itself or use the keys associated with the slice?

            Never mind, thank you very much.

            #4149
            Gregory Daues
            Participant

              Yes I think there is a lot of flexibility with those ssh keys — FABRIC is doing some setup with the Slice key etc,  so probably just best to not interfere with the Slice key (could lock oneself out of the nodes) . Just make a new ssh key pair somewhere, and then stage them into place onto the nodes, and add to ~/.ssh/authorized_keys .

              1 user thanked author for this post.
            Viewing 5 posts - 1 through 5 (of 5 total)
            • You must be logged in to reply to this topic.