Guide for Using a DLC Network

Short Introduction

    This guide is for people who need to analyze rearing behavior in videos of rats. 

    To analyze rearing behavior in a video, you will first need to run the video through DeepLabCut. DLC is a deep-learning network that you can train to identify body parts in videos. The DLC network will label rat's body parts (nose, ears, and rump) on the video frames. 

    Then, you can go to MATLAB where these labeled points are further processed to compute the head-body ratio (the ratio between the area of the head and the area of the body). The head-body ratio is somewhat reflective of the rat's position and can show you when the rat is rearing. However, because we don't have a foolproof system to determine whether a rat is rearing or not, you will need to look at the videos and score rearing behaviors manually. 

    This instruction is still under construction. If anything is unclear to you, slack Irene on NewmanLab slack channel. 

To label/analyze videos on DLC using a combined tracker 

Using a batch job on IU high-performance computer (recommended)

There are three main steps: 1) getting your to-be-labeled videos to the high-performance computer; 2) starting the labeling; and 3) harvesting the generated label files. Steps 1 and 3, involving moving files, can be done in several ways:

Moving files (Steps 1 & 3)

1. Using a dedicated app for transferring files. Here are recommended ones:

   1. On Windows, WinSCP

      1. Once installed, open new connection with the following settings:

         1. Transfer protocol: SFTP

         2. Host: quartz.uits.iu.edu

         3. Username: your IU username

         4. Password: your IU password

      2. If this is your first time connecting from this computer, it will put up a security prompt that you have to 'accept'

      3. Enter '1' to send a Duo push 

      4. Navigate to the videos directory of your DLC project

         1. Assuming your DLC project has name fisher_combined_tracker-jic-2022-10-13, this would look like:

            1. /N/project/memLab_deepLabCut/fisher_combined_tracker-jic-2022-10-13/videos

      5. Copy your files into this folder

   2. On Mac, CyberDuck

2. Copying with a terminal command (ONLY WORKS WITH MAC OR LINUX COMPUTER):

   1. Open a terminal

   2. Navigate to the folder with your videos

   3. Now you have two options:

      1. To move files from your local computer to the high-performance computer (assuming your videos are *avi):

         1.  rsync -azP --ignore-existing \*.avi <yourUsername>@quartz.uits.iu.edu:/N/project/memLab_deepLabCut/<yourDLCvideoDir>

            1. where <yourUsername> is your IU username

            2. where <yourDLCvideoDir> is the name of the directories where you want to move the videos (e.g., fisher_combined_tracker-jic-2022-10-13/videos)

      2. To move files back from the high-peformance computer to your local computer:

         1. rsync -azP --ignore-existing <yourUsername>@quartz.uits.iu.edu:/N/project/memLab_deepLabCut/<yourDLCvideoDir>/\*.csv .

or

2. 3. 2. 2. scp <yourUsername>@quartz.uits.iu.edu:/N/project/memLab_deepLabCut/<yourDLCvideoDir>/\*.csv .

            1. where <yourUsername> is your IU username

            2. where <yourDLCvideoDir> is the name of the directories where you want to move the videos (e.g., fisher_combined_tracker-jic-2022-10-13/videos

         3. NOTE: This assumes that the csv files are where you put the videos. However, they may still be in subfolders if you are using AnalyzeVideosBulk.py. To get them back into the videos folder, use the following command on the HPC computer in a terminal that is 'located' at <yourDLCvideoDir>:

            1. To copy all *.csv files in any subdirectory into the local directory:

               1. cp */*.csv . 

            2. To copy *.csv files from a subdirectories with a particular string in their name:

               1. cp *020923*/*.csv .

         4. NOTE: This can work if, on the HPC, the csv files have been compressed into a zip file and then you move the zip file, for example:

            1. zip <destinationFile> <files to zip>

               1. For example: zip 020923DLCfiles.zip *020923*.csv

Launch the batch job (Step 2)

0.     Prepare for batch jobs (**Only needed first time**)

1.     open ThinLinc - red.uits.iu.edu

Links to an external site.

2.     log in using your IU username and passphrase

3.     once you’re logged in, you’ll see a screen that looks like an old computer – open “terminal”

4.     navigate to the subdirectory with the batch job files

cd /N/project/memLab_deepLabCut/slurm_fischerNet

5.     start the job you want:

to train a network:

sbatch slurm_trainNetwork.sh

to label videos:

sbatch slurm_analyzeVideosBulk.sh

6.     monitor how your job is doing

Using Interactive Job on IU high-performance computing (not recommended)

1. open ThinLinc

2.  log in using your IU username and passphrase

3. once you’re logged in, you’ll see a screen that looks like an old computer – open “terminal”

a.     terminal will look somewhat similar to Anaconda Powershell Prompt

4.  Start an interactive job on a GPU compute node with 20GB of RAM, and 20 hours of allowed compute time

srun -A r00258 -p gpu --time=48:00:00 --gpus-per-node=1 --mem=32G --x11 --pty bash

** note ** if you get an 'unable to allocate resources' error, confirm with Ehren that the account number is correct

5. activate virtual machine via conda that has deeplabcut installed

module load conda

conda activate DEEPLABCUT

6. Activate interactive python

ipython

7. Pull deepLabCut tools into the current python session

import deeplabcut

8. Call command to analyze videos

deeplabcut.analyze_videos(<config_path>,<video_path>,save_as_csv=True)

On Google Collab

• 1. Open the Google Colab notebook (links to notebooks can be found under 'List of Networks We Have'.) Make sure you are logged into the lab gmail account (newmanmemorylab@gmail.com).)

  2.  Follow instructions on the Google Colab notebook to analyze your videos.

     • Screenshot attached at the bottom of this page shows an example of how to set up the testVideo_path on Colab.  

     • 'analyze_videos' code will create csv, h5, and meta pickle files.

     • 'create_labeled_videos' code will create an mp4 file (file names will end with '<iteration number>_labeled.mp4'). 

To label/analyze videos on DLC using an individual tracker 

1. Open the Google Colab notebook (links to notebooks can be found under 'List of Networks We Have'. ) Make sure you are logged into the lab gmail account (newmanmemorylab@gmail.com).)

2. Copy over all videos you need to analyze to the 'videos' folder.

• Path to videos folder: ~Projects/DeepLabCut/<project name>/videos

• Make sure ONLY the videos you want to analyze are in the folder. If there are other videos in the folder, make a subfolder inside the videos folder and move them into the subfolder.

To score videos on MATLAB

• 1. Open labelRearing in MATLAB (*Use MATLAB R2021a)

     • file location: ~\Dropbox (NewmanLab)\docs (1)\docs_Dylan\Code\labelRearing.m

  2. Call labelRearing.

     • MATLAB will prompt you to choose the video you'd like to score. Click on the video you want to score. 

  3. Once you chose the video, MATLAB will give you a graph with the frame # in the x-axis and the head-body ratio in the y-axis. Click anywhere on the graph and MATLAB will display the corresponding frame (as shown on the screenshot attached below). 

     • MATLAB will ask you if the rat is rearing in the frame - answer accordingly with y (yes) or n (no).

     • If you are unsure, press s (skip). If you're done scoring the video, press q (quit) to save and exit. 

     • If you made a mistake, click anywhere on the graph, then press d (delete previous point). MATLAB will ask you if you want to delete the last rear or non-rear frame - answer accordingly with u (rear) and d (non-rear).

     • Generally, peaks are when the rat is rearing. Check all peaks. However, don't rely too heavily on the shape of the graph as the head-body ratio is not a perfect representation of a rat's position.

List of Networks We Have (links to google colab pages):

Combined networks (for multiple rats):

• Non-ephys Long Evans in 8-arm radial maze (overhead view)

• Non-ephys Fischer rats in the TCT maze (overhead view) 

Individual networks (for individual rats):

• MsOpto 21

• MsOpto 22

• MsOpto 24

• MsOpto 25

• Rear 1

• Rear 2

• Rear 3

• Rear 4

• Rear 5

(Outdated) Networks Stored in Dropbox:

These are trackers that aren't currently being worked on/with and are stored in Dropbox. Path to folders: Dropbox (NewmanLab)/docs (1)/docs_Irene/.

• msopto21tracker-eln-2021-11-11: individual tracker for msopto21

• msopto22tracker-jic-2022-02-17: individual tracker for msopto22

• msopto23tracker-jic-2022-02-23: individual tracker for  msopto23

• msopto24tracker-jic-2022-02-25: individual tracker for msopto24

• msopto25tracker-jic-2022-03-03: individual tracker for msopto25

• rear1tracker-jic-2021-12-06: individual tracker for Rear1

• rear2tracker-jic-2022-01-26: individual tracker for Rear2

• rear3tracker-jic-2022-01-26: individual tracker for Rear3

• rear4tracker-jic-2021-11-30: individual tracker for Rear4

• rear5tracker2-jic-2022-01-21: individual tracker for Rear5

• rear6tracker-jic-2021-11-16: individual tracker for Rear6

• RC1tracker-SC-2022-01-31: individual tracker for RC1

• RC2tracker-Dylan-2022-02-07: individual tracker for RC2

• RC3tracker-jic-2022-02-15: individual tracker for RC3 

• RC4tracker-jic-2022-03-10: individual tracker for RC4 

• RC5tracker-jic-2022-03-21: individual tracker for RC5

• RC6tracker-jic-2022-03-25: individual tracker for RC6

• RC7tracker-jic-2022-03-28: individual tracker for RC7

• RC8tracker-jic-2022-04-01: individual tracker for RC8

• RC9tracker-jic-2022-04-12: individual tracker for RC9

• RC10tracker-jic-2022-04-13: individual tracker for RC10

• Pierogitracker-aes-2022-05-25: individual tracker for Pierogi (ephys)

• Squashtracker-jic-2022-05-13: individual tracker for Squash (ephys)

• Stuffingtracker-jic-2022-07-08: individual tracker for Stuffing (ephys)

• Cucumbertracker-jic-2022-05-13: individual tracker for Cucumber (ephys)

• rear1_rear2tracker-aes-2022-06-16: combined tracker for Rear1 + Rear2 

• RC3_RC4_singletracker-aes-2022-06-15: combined (single-animal) tracker for RC3 + RC4

• RC3_RC4tracker-aes-2022-06-14: combined multi-animal tracker for RC3 + RC4

• side_combined_tracker-jic-2022-10-13_old: outdated combined tracker for side-view videos ('redo' version in Google Drive has more accurate labels)

• ephys_tracker-aes-2022-07-15 old: outdated combined tracker for top-down vidoes of rats with ephys head drive.