Data Transfer
Data transfer servers
Case has configured an alternate route for research data to be transferred with off-campus servers. This is known as our Science DMZ, and data routed here is not subject to packet-level inspection. At this time, only certain servers located within the KSL data center have access to the Science DMZ.
Use dtn[2-3].case.edu to use Globus, or other tools to transfer data via the Science DMZ to servers external to the Case campus network.
Use hpctransfer node to do internal (Campus) data transfer. For example, from a workstation in the lab (having Globus Personal Endpoint installed) to the Globus Server Endpoint cwru#hpctransfer. Internal means both the source and desitination endpoint are active within the Case campus network.
Research Dedicated Storage transfer to/from '/scratch'
The RDS servers are not mounted to compute nodes. Data from RDS servers needs copied to /scratch or another active mount point for use in computations. To do so programmatically, in a Slurm script, may be accomplished as follows:
# Create temporary scratch space
mkdir /scratch/users/<CaseID>
# Copy data from RDS to /scratch. Suggested nodes to use when copying from/to RDS: dtn[1-3], hpctransfer
ssh dtn2 "cp -r /mnt/rds/<rds name>/<folder1> /scratch/users/<CaseID>"
# Copy data from /scratch back to RDS
ssh dtn2 "cp -r /scratch/users/<CaseID>/<folder1> /mnt/rds/<rds name>/."
Transfer Options
To see more information and examples of the tools utilization below, please go to the Transferring Files @ HPC page.
Parallel Transfers
Globus Online (GO) using dtn[2-3] node (Recommended for larger files and faster transfer) - click here for setting-up GO
iCommands by iRods - specifically for CyVerse
Windows-based Transfer Tools
WinSCP - Opens Source GUI Secure File Transfer Protocol (sFTP) Clinet
MobaXterm - It has visual enabled SSH, SFTP, RDP sessions under a single hood to enable easy file transfer and remote desktop
FileZilla - Open Source GNU/GPL FTP Clinet
PSFTP - Putty's secured FTP
Window with Cygwin - OpenSSH
Mac-based Transfer Tools
Cyberduck
FileZilla
Linux-based Transfer Tools
Cyberduck
Rsync
Scp