The "timeline" in Maestro is governed by its data acquisition (DAQ) card. Regardless the operational mode, "time zero" occurs when the card initiates the DAQ sequence, and the fundamental unit of time is the DAQ scan interval -- 1 ms for Trial mode and 2 ms for Continuous. Ideally, we would like to be able to update any Maestro target at the same rate, but this is not possible for targets on the RMVideo display: the update rate is set by the monitor's refresh rate, which is generally 100Hz or less. The different timelines generate important questions: (1) How does Maestro ensure that the first display frame on the RMVideo side begins as closely as possible to Maestro's "time zero"? (2) If an RMVideo target is supposed to turn "on" at an elapsed time T in the Maestro timeline, when does it actually appear on the RMVideo display, in that same timeline? These are important questions because researchers need to be able to compare the time course of presented visual stimuli with the recorded behavioral and neuronal responses they evoke.
To address such questions, we first need to explain how targets are animated on the RMVideo display and how Maestro controls the target animation on a frame-by-frame basis via commands sent over their private Ethernet connection. We'll focus on what happens during a Maestro trial; the process in Continuous mode is similar.
RMVideo has two states: idle and animate. In the idle state it responds to commands to change the background color or the display dimensions, download or otherwise manage movie and image files in its "media store", or load targets for an upcoming animation sequence. In the animate state it prepares the first frame of the animation sequence on a back buffer, waits for the beginning of the monitor's vertical retrace ("vertical sync"), sends a "first frame" signal back to Maestro, then swaps buffers so that the video card begins drawing the first frame to the screen. It then enters a loop in which it checks for a new update vector from Maestro, renders the next frame on the back buffer accordingly, waits for the vertical sync, swaps buffers, and repeats. This cycle continues until it gets the "stop animate" command from Maestro.
In order to control target trajectories on a frame-by-frame basis in this double-buffered scheme, it is vital that the update vector for display frame N+1 be available to RMVideo at the beginning of display frame N -- so that RMVideo has the maximum amount of time (one frame period) to render the next frame on the back buffer without "skipping a frame". Furthermore, Maestro should be preparing and sending the update vector for frame N+2 at the beginning of frame N so that update vector is ready for RMVideo to process at the beginning of frame N+1.
Maestro, for its part, calculates trial target trajectories with millisecond-resolution, the same resolution with which analog data is recorded in Trial mode. For an RMVideo target, it must accumulate any changes in target position or on/off state in the update vector, which is sent at the start of the next display frame. Given that Maestro needs to send the update vector for frame N+2 immediately after display frame N begins on the RMVideo monitor, all RMVideo target computations must LEAD the actual Maestro time line by two full RMVideo frame periods; otherwise, the RMVideo target trajectory will lag the intended trajectory by the same amount.
The above discussion suggests that Maestro "knows" exactly when an RMVideo display frame begins; in fact, it does not. Rather, Maestro and RMVideo were designed to ensure that the trial's DAQ sequence begins ("time zero") as soon as RMVideo begins displaying frame 0 on the monitor. This is the rationale behind that "first frame" signal from RMVideo. Before starting the trial, Maestro calculates the RMVideo target update vectors for the first 3 display frames. It then sends the first two update vectors to RMVideo with the "start animate" command and waits (in a busy loop) for the "first frame" signal from RMVideo. It then immediately starts the DAQ animation sequence. This should guarantee that "time zero" occurs no more than a few hundred microseconds after the RMVideo monitor starts displaying frame 0 (note, however, that we have not done any tests with a photometer in an effort to confirm this). Then, during the first millisecond of trial runtime it sends the update vector for frame 3, and begins accumulating target displacements in the update vector for the next frame. To determine when it needs to send that update vector, Maestro relies on RMVideo's high-resolution measurement of the display frame period. It maintains a variable holding the "elapsed RMVideo time" at which the next update should be sent -- in milliseconds but with microsecond resolution. Each time it sends an update vector to RMVideo, the time for the next update is the current elapsed time plus the high-resolution frame period.
Given this background, we can now address the second question above: If an RMVideo target is programmed to turn on at an elapsed Maestro trial time of T milliseconds, when does it actually appear on the RMVideo display? As a concrete example, suppose that T=120ms and that RMVideo's high-resolution frame period is 11.92 ms. The target must not come on before T=120, and 120/11.92 = 10.06, so the target should turn on during the 11th display frame, which starts at 131.12 ms in Maestro time. In this instance, the target comes on almost a whole frame period after the programmed time of 120 ms.
But this is only true if the target is located in the upper left corner of the display! All of the pixels on the display screen are NOT updated at once; rather, the display is refreshed one pixel at a time, from top to bottom and left to right. Thus, in our example, a small target located near the bottom right corner of the display will actually turn on toward the very end of the display frame 11, or nearly two frame periods after the programmed time of 120ms.
This example illustrates an unavoidable drawback in using RMVideo. However, as long as you know the high-resolution frame period of the RMVideo monitor, you should be able to calculate the actual target trajectories (in Maestro time) taking into account the frame period. In fact, the Matlab utility readcxdata() attempts to do so. Target position and velocity are reported with 1-ms resolution, but the trajectories have a "sample-and-hold" appearance to reflect the fact they are updated only once per RMVideo display frame.