Step 1: Being on a secure network

• You must be on a campus network (either TAMU or TAMHSC)

• If you are working from home, you must get on VPN (either tamu or tamhsc)

  • Tamu vpn: 

    • In order to sign in to Tamu vpn, use username and enter your password. The username is your netid. For example, my netid=username is kum. Some people confuse netid as being kum@tamu.edu because some systems will accept the full address, but only the part before @ is the actual netid and your username)

    • After logging into the following page https://it.tamu.edu/services/network-and-internet-access/virtual-private-networks/virtual-private-network-vpn/ 

    • Click on Windows or Mac and then follow the instructions. Go to https://connect.tamu.edu , download cisco

    • You can also use the instructions below for TAMHSC, but instead of putting https://vpn.tamhsc.edu. Write connect.tamu.edu and then login via entering the username and password (keep the mobile with you, it will ask for duo verification).

  • Getting to TAMHSC vpn: https://vpn.tamhsc.edu/

    • Detailed guide: https://docs.google.com/document/d/1n1oYKf3I7eKFiDqGAmZKXckzz7nt0ceqWqObrPCSOEM/

Step 2: Getting to ViDaL

• How to access

  1. Go to webpage: go.vidal.tamu.edu

  2. Click Access ViDaL VMs ( or click icon)

     1. In general, you should look at the node name and use the node that is most appropriate for your need. At the current time, since there are not too many users, resource allocation is self determined. In the future, when there are more users, we will formulate more strict policies with users based on their experience during this phase

     2. General rules

        • Use the regular node (starts with reg-) whenever that meets your need: we have most nodes that are reg (it has 192G RAM and should meet most needs)

        • If you need larger RAM, you should consider using large node (starts with lrg-). These nodes are more restricted. You may need to request access by emailing admin@vidal.tamu.edu

        • If you need gpu, use gpu nodes. You need to request access explicitly. If you need to use them and do not see it on the list, email admin@vidal.tamu.edu

        • You should be able to see how many users are on a node when you select a node, so you may consider going to a node with less users

        • Prioritize using nodes designated to your unit (e.g., TEES, TTI, SPH). If those seem full, use general VIDAL or TAMU nodes next. If those are also full, you may consider using other nodes not in use. But be cautious when using nodes designated for units that you are not a part of. If you find yourself consistently using these nodes, please contact admin@vidal.tamu.edu with a short explanation and we will review adding a node for your unit or more general nodes.

Step 3: Setting up on ViDaL

• Windows setup (only needs to be setup once for the first login to each Windows node)

  1. Remove (Unpin) Internet Explorer from your taskbar

  2. Creating a shortcut to your project folder

     1. Click file explorer

     2. In the address space type in Folder: \\pn-smb (You should see subfolders like home, proj1, proj2, proj3 as in the fig below)

     3. Right click on “Quick access” and “Pin current folder to Quick access” (see fig below) 

Note: If you right click on pin to quick access, the project folder automatically goes on to the desktop.

4. Drag the newly created “pn-smb” link to the desktop for easy access

5. [Optional] You may map network drives to specific folders you use often.

6. IMPORTANT: You MUST use the project folder to store any data from the project. This is so that we can track all data including derived data that must be monitored. We recommend you create a personal folder INSIDE the your project folder for all your work

• E.g. \\pn-smb\proj2\waiver_hck\uc_hck

3. (Optional] Your home directory (you can make short cuts or map drives if you want) 

   1. Location: \\pn-smb\home\yourid 

• ONLY for R users. Renviron for each person to set R_LIBS_USER 

  1. Click file explorer and click document

  2. Turning file name extensions on from view tab

3. Create a text file 

4. Renaming a file to “.Renviron.” (The dot at the end helps to have no extension)

5. Add the following line to the new .Renviron file (while will tell R to use your mapped z: drive as the location for storing any installed R packages): 

R_LIBS_USER=//pn-smb/home/yourid/R/win/3.6

Step 4: Which folders to use for your files and backup

• Folder locations after mapping drives:

  1. What you should be using

     1. Project space: \\pn-smb\proj1\your project or \\pn-smb\proj2\your project or \\pn-smb\proj3\your project

     2. Home space: \\pn-smb\home\yourid

  2. You should NEVER use C:

     1. It is not backed up

     2. It is reserved for systems

     3. This means you should NOT use the default document/download folders. Rather use the network drive so that it is backed up. 

  3. D: or E: (local drive using fast disk. Not a mapped drive) A private folder is provided for each person that is authorized to use the Windows server.  Any data stored on this fast local disk is NOT backed up. Think of this more as temp fast space for heavy computation.

• Remote desktop details:

  1. Remote desktop will be disconnected if idle for 15 minutes

  2. Remote desktop session will be terminated after 2 hours

More details on using Linux

https://docs.google.com/document/d/1PuL_9-Q8UqYr5a3QbGR_7EDyj3Hff8FKKmfg8PjpEeM/edit#heading=h.ftzxzqol2it9

Step 5: Uploading & Downloading Files on ViDaL

There are many ways to do this. Below we have three.

• use any SCP/SFTP program to connect to dtn.vidal.tamu.edu (Duo required)

  1. Home directory will be /home/netid

  2. Project directory will be /project/pinfo

  3. e.g. scp address/filename id@dtn.vidal.tamu.edu:/project/your_folder/filename

  4. Please do NOT save your NetID password in a connection profile in any SCP/SFTP program.

• Use winSCP

  1. In the log-in screen type in “dtn.vidal.tamu.edu” as shown below

2. Click “Login” on the dialogue box screen above.

3. Type in your “User name” into the dialogue box that appears.

4. When the “Authentication Banner” dialogue box pops up hit “Continue” then type your password into the next dialogue box and select 1 or 2 for Duo.

5. The left screen shows the files on the HSC server and the right screen shows files on Vidal.  You are now ready to transfer files.

6. You should be transferring files to the proper project folder in

   1. /gpfs/project/your project folder

• use Mobaxterm (NOT WORKING FOR NOW)

  1. On the top left corner of MobaXterm window, click Session button.

  2. Choose 'SFTP' session type.

  3. Enter remote host (dtn.vidal.tamu.edu) and username (your netID).

  4. Go to 'Advanced Sftp settings' and check '2-steps authentication' checkbox.

  5. After you select 'Ok', you will be prompted with Duo Authentication method (Push, Call, Text, Second device, etc). Select the appropriate option. Do not type your TAMU password.

• Use ssh into data node

  1. On your local linux machine, type “rsync -vah data_directory your_id@dtn.vidal.tamu.edu:/gpfs/project/target_directory”

     • -v : verbose

     • -r : copies data recursively (but don’t preserve timestamps and permission while transferring data

     • -a : archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships and timestamps

     • -z : compress file data

     • -h : human-readable, output numbers in a human-readable format