HPC Home‎ > ‎User Support‎ > ‎Helpful References‎ > ‎

Transferring Files @HPC


Data transfer servers

Case has configured an alternate route for research data to be transferred with off-campus servers. This is known as our Science DMZ, and data routed here is not subject to packet-level inspection. At this time, only certain servers located within the KSL data center have access to the Science DMZ. Use dtn1.case.edu to use Globus, iCommands or other tools to transfer data via the Science DMZ to external servers. Use hpctransfer.case.edu to move data between your local system and the cluster data storage. The server dtn1.case.edu only supports graphical access through Globus. To login for a shell session, first login to hpctransfer, then ssh to dtn1.

Transferring Files

There are several methods to transfer data with the cluster data storage. 
  • Globus Online (GO) using dtn1 node (Recommended for larger files and faster transfer) - click here for setting-up GO
  • iCommands by iRods; using dtn1 node (Recommended for larger files and faster transfer)
  • For your Windows machines, here are some of the options (you can download):
    • WinSCP - Opens Source GUI Secure File Transfer Protocol (sFTP) Clinet
    • MobaXterm - It has visual enabled SSH, SFTP, RDP sessions under a single hood to enable easy file transfer and remote desktop
    • FileZilla - Open Source GNU/GPL FTP Clinet
    • PSFTP - Putty's secured FTP
    • Window with Cygwin - OpenSSH
  • For your Mac machines, here are some applications you can use:
    • Cyberduck
    • FileZilla
  • For Linux box with OpenSSH: Cyberduck, other open source scp-compatible tools
  • To back up the files on the HPC to Google drive, please use Rclone - Google Drive 
  • WebDrive

scp command

At the Command Line prompt of your desktop or laptop computer, type the command:

Copy from HPC to your local PC
scp -r <username>@<login-node>:/home/<user-id>/<location-for-file-to-transfer> <path-to-transferred-file-PC>

From your PC to HPC
scp -r <file or directory-to-be-copied> <username>@<login-node>:/home/<user-id>/<location-for-file-to-transfer>

Graphical Interface Remote Copy Application

Example: WinSCP which provides user-friendly GUI to drag-drop files from one location to another.

  • Download and Install the WinSCP.
  • Double click on the executable to open the GUI below
  • Click "New"
  • Enter the hostname information: hpctransfer.case.edu and the login information: the CaseID and the SSO password
  • You will see the Graphical Interface similar to the one below
  • You will see a side-side window that points to your desktop/laptop computer and the remote host.
  • You can easily drag-and-drop files between the windows to copy from one location to another.

Example: Cyberduck on MacOS
  • Download and install the Cyberduck
  • Access the cluster via hpctransfer by entering your CaseID and SSO Password

  • You can open the local folder in Finder and this Transfer Window side by side and then drag one file (or folder) from one location to another

Globus Online

GO Enables faster transfer. Click HERE for setting-up GO.

iCommands

A parallel file transfer tool developed for transfers with the CyVerse Data Store. Setup and usage references available here.

WebDrive

WebDrive can help transfer files (SFTP) mapping drive letters to webservers (e.g. hpctransfer.case.edu) on top of providing cloud access (Amazon S3), Google Drive, Dropbox, and more.
  • Download and Install the WebDrive Software from Software Center.
  • Choose SFTP as the type of server you want to connect to
  • In the URL/Address, type the name of the SFTP server (e.g. hpctransfer.case.edu)
  • Type your Case (SSO) username and Password.
  • Choose the drive letter for that server
For details, follow the instruction at https://www.case.edu/webservices/webdav/webdrive.html.


Utilizing Cloud Storage with the HPC

RClone - Google Drive

It is a storage in a google drive. Click here for details

Using Box at HPC

The information for using Box from the HPC cluster can be found at the following page.




Comments