EyeLink

Maestro 3.2.0 introduces support for an Ethernet connection to and loose integration with the EyeLink 1000+ eye tracker from SR Research. This page discusses the rational behind the new feature, how it was implemented, the new control panel tab dedicated to the EyeLink, and some operational details.

Motivation

In eye movements studies in humans, a high-speed camera-based tracker system is typically used to monitor the subject's eye position during an experiment. These tracker systems typically have analog outputs representing eye position and/or velocity, and in the past we have connected such outputs to Maestro's HGPOS, VEPOS, HEVEL and VEVEL inputs to record eye trajectories of a human subject.

Unfortunately, the analog signals from the lab's old eye tracker were rather noisy. The EyeLink 1000+ eye tracker offers another option: eye position data can be streamed at 1KHz over a fast Ethernet connection. The position trajectories generated by the tracker thus remain in digitized form -- not subject to the system noise to which the analog signals are susceptible (the manufacturer actually includes an analog output option with the EyeLink, but discourages its use because of noise). Of course, to take advantage of this Ethernet link, Maestro must be able to communicate with the EyeLink's Host PC over the TCP/IP connection and "keep up" with the 1KHz data stream when eye position is being recorded.

Design Approach

Given that a new sample is arriving over the link every millisecond, it would seem best to implement an EyeLink service routine in Maestro's RTX driver. However, the EyeLink SDK is not compatible with RTX. Instead, taking advantage of the fact that Windows has sole ownership of 2 CPU cores in the lab’s current quad-core Maestro workstations, the service routine is implemented as a Win32 thread running in the Maestro GUI process. Eye position data is received over the EyeLink Ethernet connection and, in turn, streamed to Maestro’s RTX-based driver (cxdriver) over the existing IPC (interprocess communications) structure. This approach was obviously preferred over adding a “relay” computer between the Eyelink Host PC and the Maestro PC, and testing with the EyeLink in “Mouse Simulation Mode” uncovered no performance issues with the design. The GUI remains responsive while recording (and saving) trials using RMVideo for trial presentation and servicing the 1KHz eye position sample stream from the EyeLink.

While recording and streaming data to cxdriver, the EyeLink service thread runs in a tight loop that constantly checks the link for new eye position samples. It sleeps for 1ms on every other loop iteration to ensure it “keeps up” with the 1KHz stream while leaving some CPU time for other Windows threads -- including a thread internal to the Eyelink SDK that handles the low-level network communications with the EyeLink Host PC. Note that the system MUST support a 1-ms time slice for this approach to work; if not, EyeLink support is disabled.

Design Decisions

• Minimal integration. Maestro does little more than connect to the tracker, configure it to send only raw pupil samples over the Ethernet link, and tell it when to start and stop recording. It ASSUMES that the EyeLink is sampling eye data at 1KHz. No calibration is needed on the EyeLink side, since only raw pupil data is used. No EDF file is recorded.

• EyeLink data is converted to Maestro format. Using gain and offset parameters set by the user on Maestro's EyeLink tab page, raw pupil coordinates (x,y) from the tracker are converted to Maestro’s eye position coordinates -- as 12-bit digitized voltages proportional to visual degrees subtended at the eye. When recording monocularly, the EyeLink position data is placed in Maestro analog input channels HGPOS, VEPOS as you would expect. If recording binocularly, the left eye data is in HGPOS, VEPOS and the right eye in HGPOS2, VEPOS2.

• Eye velocity is computed “on the fly” from EyeLink data by using a central-point difference method [ v(t) = (p(t) - p(t-2)) / 2ms], followed by a sliding-window average to smooth the result. The calculations are independent for X and Y. The length of the sliding window is controlled by a parameter on the EyeLink tab. The resulting eye-velocity signal is stored in Maestro channels HEVEL, VEVEL.

• Checking for delays in the EyeLink data stream. Given that EyeLink data samples are delivered over an Ethernet link, retrieved by a service thread running in Windows, and then delivered over IPC to cxdriver, it is highly likely that there could be occasional delays in the 1KHz EyeLink sample stream. During a data recording in Trial or Continuous mode, Maestro checks the inter-sample interval -- the time in milliseconds between the previous and current EyeLink sample, as measured in “Maestro time”. [We cannot compare EyeLink and Maestro timelines. Even a 0.1% difference in the two system’s sampling rates will lead to significant timestamp differences after only 10 seconds of continuous recording.] If this inter-sample interval reaches 10ms or more, Maestro aborts the data recording. The program also tracks the worst-case observed inter-sample interval, the average interval, and the number of “repeat” samples (when an EyeLink sample is not ready in the buffer, the previous sample is supplied again) over the course of a trial or a Continuous-mode recording. These EyeLink stats are included in the data file.

• Blinks. When the subject blinks, the EyeLink cannot provide position data. The sample stream is uninterrupted, but the sequence of samples during the blink will contain no pupil data. Maestro currently employs a “sample-and-hold” strategy during a blink; when the pupil position coordinates are “missing”, it simply sets the current sample equal to the previous sample. Blink epochs are included in the Maestro data file as pairs of blink start/end "events".

• Minimal data file changes. Since EyeLink data is converted to the Maestro format, eye trajectory data is saved in the usual manner, and downstream analysis code (readcxdata(), JMWork, etc) is unaffected. The data file header includes EyeLink-specific information such as the X and Y calibration parameters, the length of the velocity smoothing filter window, a flag indicating whether or not the EyeLink was in use when the file was recorded, and inter-sample interval statistics (as described above). Blink start and stop events are recorded in the same record type as digital input pulses on DI<2..15>. JMWork and readcxdata() were both revised to expose this EyeLink information.

The EyeLink Tab

An EyeLink tab page was added to the control panel in Idle, Trial, and Continuous modes. As shown in the screen capture below, it includes: a Connect/Disconnect button; sliders for setting the offset and gain parameters that are used to convert the EyeLink raw pupil coordinates to Maestro eye position; and a numeric text field for setting the width of the velocity smoothing filter. The filter width may only be adjusted in Idle mode; the offset and gain parameters can be adjusted at any time.

When Maestro connects to the EyeLink, it starts the background service thread which, in turn, initiates the link. On the EyeLink Host PC, in Output mode, you’ll see a message near the top right corner indicating that the TCP/IP link is open. The service thread merely sleeps in Idle mode, waiting for a command to start collecting eye position data. As soon as Maestro enters Trial or Continuous mode, it wakes up the thread, which sends a command to the EyeLink to begin recording. The EyeLink Host PC will switch from Output to Record mode, and data will begin streaming across the link to Maestro. Only raw pupil data is streamed over the link -- not calibrated gaze position (“calibration” happens on the Maestro side), no eye “events” like saccades and fixations, and no button or input events.

To “calibrate” eye position when using the EyeLink, switch to Continuous mode and have the subject fixate on a target at (0,0), then the slider controls to adjust X Offset and Y Offset as needed. Then put the target at various eccentric locations and adjust X Gain and Y Gain. Because the raw pupil coordinates from the EyeLink may be inverted with respect to Maestro's eye/target coordinate system, push buttons let you toggle the polarity of X Gain and Y Gain; the sliders only control the absolute value.

NOTE that the above description assumes that the EyeLink tracker has already been set-up with the subject in position, the subject's pupil "acquired" by the tracker camera, and the EyeLink Host PC application waiting in Output mode. Consult the EyeLink user manual for details on using the tracker program itself. During experiments using the EyeLink with Maestro, very little configuration happens on the EyeLink side once the pupil is being tracked properly. The EyeLink simply waits idly until it receives a command from Maestro to start recording. It then continually streams raw pupil samples at 1KHz until it gets the stop-record command. This cycle repeats until Maestro disconnects from the tracker.