+ Head Tracking with Bonsai

Outlined here are some information and instructions on setting up head tracking with Bonsai.

Software

The Bonsai wiki can be found here: https://bitbucket.org/horizongir/bonsai/wiki/Home. The software can be downloaded via their "Downloads" page. There is an executable file to install the software -- install the latest version. Bonsai is only compatible on a Windows operating system.

6/30/21 - Installed new version on circle track testing room computer only to find out that the Point Grey Capture module doesn't work any more. Found this page: https://groups.google.com/g/bonsai-users/c/ljIQQRNa8cw/m/TQt7G-xzAwAJ which said that you need to have a specific version of the Point Grey drivers installed ( 2.11.3.425_x64) which they provided a link to download from their personal Dropbox (https://www.dropbox.com/s/l40sshr6x0vi5f3/FlyCap2Viewer_2.11.3.425_x64.exe?dl=0) but here is a copy of this file on our Dropbox (stored in Dropbox/Software/.) https://www.dropbox.com/s/aqgud4q67q72m6l/FlyCap2Viewer_2.11.3.425_x64.exe?dl=0

Workflow

To learn more about Bonsai workflows, visit their wiki linked above. 

As of 7/14/2017, two tracking workflows exist for the lab and can be found on Dropbox in the Bonsai folder (Dropbox > Bonsai). Both require a Point Grey Flea3 camera, and will be explained in more detail below. "YYMMDD" indicated the date the workflow was created/updated. Unless specified otherwise, always use the most current date. See at the bottom of this page for copies of these scripts.

YYMMDD_3LEDtrackingWorkflow.bonsai

This workflow tracks a 3 LED array that is fixed atop the animal. Lights are tracked by intensity; therefore, and IR filter needs to be fixed onto the Flea3 camera and the LEDs need to be infrared such that they are the only lights visible to the camera.

Following is a list of elements in the workflow and a brief description.

FlyCapture connects to the Flea3 camera.

Metadata accesses the embedded image information from the Flea3 (in this case GPIO pin state, frame counter, and time stamp) .

Source.Image accesses the camera image.

Crop crops down the camera image to the necessary area. This is not a necessary element, but can be useful if there any weird artifacts in the corners of the image.

BackgroundSubtraction subtracts the background of the image (stable elements in the image) while also applying a threshold to cut out any part of the image that is below a set intensity. This allows the light emitted from the LEDs to be the only thing visible.

VideoWriter writes the video to disk (after background subtraction).

FindContours finds the contour of bright objects detected on the image.

BinaryRegionAnalysis identifies individual elements from FindContours.

PythonTransform takes the top three largest elements and records their x- and y- coordinates.

AverageBinaryRegion finds the average coordinates of all identified elements from BinaryRegionAnalysis.

Source.Centroid allows the centroid from AverageBinaryRegion to be overlayed on the video image (for visual purposes only)

CsvWriter writes the data to a CSV file.

YYMMDD_NoLEDtrackingWorkflow.bonsai

This workflow tracks movement of white areas with background subtraction. The workflow was designed under the assumption that the only white areas that will be moving are the white patches of fur on the rat -- all other non-moving white areas will be normalized out in the background subtraction. Since the white patches of fur need to be visible in the camera image, an IR filter should NOT be used. In general, this tracking workflow is more coarse and should not be used unless a coarse estimate of position is desired without the use of LEDs.

All the elements used in this workflow are also used in the previous workflow; therefore, you can refer to the section above for a description of the elements in this workflow.

Syncing with other software

Since Bonsai is separate from whatever data acquisition software being used, a trigger pulse is necessary to align tracking data from Bonsai with neural recordings from other software. This trigger pulse is simultaneously recorded by Bonsai and the neural recording software to be aligned in post processing.

Generating the trigger pulse

There are currently three methods of generating a trigger pulse, one being specific to the Intan system. 

Method 1: Arduino based timestamp based pulses

The first method, compatible with all current systems (eCube, Intan, TDT/Synapse), is using an Arduino to automatically generate a frequent unique series of TTL pulses. This is done with a custom script (linked to this page) that generates a pulse sequence based on the clocktime of the Arduino. To use, upload the file to an Arduino (tested to work with the Arduino Uno) and attach the ground to the Arduino ground and the signal channel to digital pin 13 on the board.

Method 2: Manual generation

The next method, compatible with all current systems (eCube, Intan, TDT/Synapse), is manually creating the pulse with a button. A simple button can be wired and powered on a breadboard to generate the digital pulse that can be sent to the various recording systems. A single button is currently set up for this purpose.

Soon: image and circuit design.

Method 3: Intan trigger

The second method uses a built in function of the Intan system to generate a digital pulse from an electrophysiological signal. Intan can convert an input channel into a digital pulse by setting the pulse to high once the input channel signal surpasses a given threshold, and back to low once the signal falls back below the threshold. To access this feature, our custom build of Open Ephys must be installed. More info on that soon.

Soon: images and setup instructions.

Sending the trigger pulse

The trigger pulse must be received and recorded by each system being used for alignment to be possible. Instructions for connecting the pulse to various software is outlined below. This pulse must be 5 volts to be reliably detected by the GPIO pins.

Bonsai / Flea3

A trigger pulse can be sent to the Flea3 via a gray 8-pin GPIO cable. The pinout for the cable is shown in the table below and can also be found in the technical reference manual for the Flea3. 

The GPIO 0 pin (pin 1, black wire) is what is used to receive the trigger signal. This pin needs to be connected to wherever the signal pulse is being generated, be it from Intan or a manual button. The ground wire (pin 6, blue wire) must also be connected to ground to complete the circuit.

To make sure the pin is properly configured to receive a trigger, access the camera settings through Point Grey's FlyCap2 software. Once connected to the camera, access the camera control dialog by pressing the button highlighted below.

Click on the Trigger / Strobe tab on the left side of the window, and under the Pin Direction Control panel, make sure GPIO 0 is set to In.

To make sure Bonsai can access the GPIO 0 pin state, go to the Advanced Camera Settings tab, and under the Embedded Image Information panel, make sure GPIO pin state is checked. Make sure that Timestamp and Frame counter are also checked -- these values are necessary for trigger alignment. To save the advanced camera setting, under Memory Channels, select channel 1 in the drop down menu (it is not recommended to save to the default channel), and click Save. This will allow any changes made to the advance camera settings to stay if the camera is ever unplugged or restarted. 

If you're using one of the Bonsai workflows listed above, it is already set up to record the trigger pulse.

Open Ephys / Intan

The Intan system can receive signals from an outside source via one of its ADC channels. Simply wire the output of the trigger pulse to one of the ADC pins found on the Intan system. Make sure the ADC 1-8 button on the Rhythm FPGA Open Ephys widget is activated, otherwise the input into the ADC channels won't be recorded.

Open Ephys / ECube

The ECube has digital in/out channels that can be accessed on the front panel of the ECube or via a breakout panel connected to the ECube. You will need a BNC cable with one end wired to connect to the trigger output and ground, and the other end a BNC connector for the ECube. Simply connect the BNC to the first digital channel on the ECube or breakout panel.

To access the trigger pulse in Open Ephys, initialize a second ECube widget in a separate stream; however, instead of initializing a headstage, use the dropdown menu to change it to Digital in/out. Append an LFP Viewer to the newly added widget. Now all digital signals from the ECube should be saved alongside the headstages.

Synapse / TDT

TDT has a digital BNC breakout which contains channels that can receive and external trigger pulse. The patch panel being used for the digital breakout is the PP16 Patch Panel. You will need a BNC cable with one end wired to connect to the trigger output and ground, and the other end to one of the BNC connectors on the breakout. The digital inputs that can be used to receive the trigger pulse are labels A1-A7 on the patch panel. 

To access the Trigger input in Synapse, the Strobed Data Storage gizmo (shown below) needs to be added to your processing tree.

The channel mapping between RX8 and PP16 is a bit confusing. To access the trigger input from port A1 on the patch panel, Port C0 needs to be set as the primary source for the strobed data storage gizmo. Once you initialize the gizmo, set PortC0 as the input for the Main source, ignoring the second source (shown below).

After initializing the gizmo, under the Capture Mode panel, select the Continuous mode and commit the changes. This should remove the second input to the gizmo, leaving only PortC0 as the primary source.