Prior to Maestro version 2.0.0, framebuffer video targets were implemented on the VSG2/4, an ISA-based programmable video card from Cambridge Research Systems. The VSG was an antique compared to today's video adapters, and its slow rendering performance severely restricted the complexity, size, and number of targets that could be simultaneously animated. As a result, framebuffer video targets were rarely used in past versions of Maestro; the most common usage of the VSG-based framebuffer video involved the display of large-field gratings and plaids to study spatio-temporal response properties of cells in V1 and MT.
RMVideo was developed to replace the VSG and significantly enhance the utility and versatility of Maestro's framebuffer video target platform. First integrated into Maestro 2.0.0, RMVideo is a completely independent program running on a separate Linux workstation and communicating with Maestro over a dedicated point-to-point Ethernet link. While its introduction required the addition of a second workstation to the laboratory apparatus, it offers many advantages in performance and capabilities over the VSG-based framebuffer solution:
As an OpenGL application, it can be used with any graphics card that has an adequate OpenGL Linux driver. We are not restricted to one or two video cards on the market, and upgrading to a newer adapter is relatively painless.
All of the original VSG-based framebuffer video targets are supported. Better yet, a grating or plaid can drift under an elliptical Gaussian window -- a stimulus that was not possible with the VSG.
RMVideo targets are updated on a frame-by-frame basis. As a result, they can participate in trials that involve velocity stabilization or any other special feature that requires runtime modification of a target's computed trial trajectory. This was not possible with VSG-based targets.
It is possible to implement any of the now-deprecated XYScope target types with an RMVideo target. In fact, one of the reasons for developing RMVideo in the first place was to guard against the eventuality that our analog XY oscilloscopes will break down for good (they are no longer manufactured and are increasingly difficult to repair). RMVideo offers the additional capability to display random-dot patches in color and with an elliptical Gaussian window. It can also draw more total dots per frame than our XYScope controller, but its frame rate and resolution are fixed and inferior to those of the XYScope platform.
Introducing new kinds of RMVideo targets in the future should be relatively easy in terms of development time and costs.
As of version 2.5.0, it is possible to switch among several alternative video modes (screen resolution + vertical refresh rate). This allows users to maximize temporal resolution (refresh rate) over spatial resolution or vice versa. A typical RMVideo workstation will be capable of many more video modes than are exposed by Maestro; the minimum allowed display resolution is 1024x768 pixels and the minimum refresh rate is 60Hz. All available video modes offer 24-bit RGB color with an 8-bit transparency channel (needed, e.g., for masking a target window with a 2D Gaussian).
The list below describes the different types of RMVideo targets currently supported in Maestro. Each target type is characterized by a variety of parameters; common parameters include window aperture, bounding rectangle, and color. Grating target parameters include spatial frequency in cycles per visual degree subtended at the eye, initial spatial phase in degrees, and the drift axis. The drift axis defines the grating's orientation; it is the angle (in counterclockwise degrees) made with the positive x-axis by a line perpendicular to the grating lines. For the grating and plaid targets, the range of colors in which a grating is rendered are defined by a mean RGB triplet and a contrast RGB triplet. Each component of the RGB mean color are specified with 8-bit resolution, [0..255], with 255 corresponding to the maximum luminance. Assuming the video card and attached monitor have been gamma-corrected, this 8-bit scale should map linearly to the luminance range available. (Users can adjust the monitor's gamma-correction factors directly from Maestro.) For each color component in RGB space, the mean Lmean and contrast C [0..1] define a range of luminance values [Lmin .. Lmax] assigned to the grating, according to:
Lmax = Lmean(1+C) and Lmin = Lmean(1-C).
As of version 3.1.2, you can specify a nonzero contrast for the Random-Dot Patch target. Doing so puts the target in a special two-color contrast mode. Instead of painting all dots in the patch in the same color, half the dots are drawn using Lmax for each color component (typically this mode would be used in grayscale only), while the other half are drawn using Lmin.
The Movie target requires a video source file while the Image target requires an image file. Such media files are stored locally on the RMVideo server, in the "media store": a set of up to 50 folders in $RMVIDEO_HOME/media, each of which can hold up to 50 media files, either videos or images (the maximum capacities are an artifact of the restricted size of packet buffers used in Maestro/RMVideo communications). Media files can be added to or removed from the store via direct interaction with the RMVideo workstation (often a hassle in our experiment rigs!). Alternatively, one can view the store's content, remove a particular file or an entire folder, and download new files directly in Maestro, using the RMVideo Media tab in the Idle mode control panel.
As of version 4.1.0, you can make any RMVideo target flicker periodically whenever it is displayed during a trial or in Continuous mode. The feature was added to support apparent motion studies. The flicker cycle is defined by "on" and "off" phases, and an initial delay period preceding the first "on" phase. All three parameters are specified as a number of video frame periods; flicker is disabled if either the "on" or "off" phase is zero.
Maestro 5.0.2 introduced a new RMVideo target parameter, stereo disparity, to support stereo experiments using any of the "dot" target types Point, Random-Dot Patch, and Flow Field. This feature only works if the graphics card supports stereo mode. In this mode -- instead of a single back buffer that gets "swapped" with the front buffer (which is what is drawn to the screen) at the start of every vertical refresh -- there is both a "left" and a "right" back buffer. On one refresh, the left buffer gets swapped to the front; on the subsequent refresh, the right buffer is displayed, and so on. At the same time, the video card delivers a signal that can be used to drive a polarizer which shields the subject's right eye when the left buffer is displayed, and vice versa -- thereby achieving the stereo effect.
The stereo disparity D defines a horizontal offset in the target's position on every frame: When drawing the target on the left buffer, RMVideo offsets the target's horizontal position by -0.5*D, when drawing on the right buffer, it offsets the target by +0.5*D.
Notes:
Effectively, the "stereo refresh rate" is one-half the actual frame rate of the RMVideo display. Still, RMVideo communicates the actual frame rate to Maestro, so during a trial Maestro updates all targets at that rate, not the stereo rate.
If you are interested in doing stereo experiments, you will need a graphics card (such as the NVidia Quadro) configured to support stereo mode.
When RMVideo launches, it will attempt to create a double-buffered and stereo-enabled visual. If that is successful, then the program will run in stereo mode. If not, it will revert to the usual double-buffered, non-stereo mode. A message is posted to the RMVideo console window indicating whether or not stereo mode is enabled.
Again, the stereo disparity parameter applies only to the "dot" target types. When drawing any other target type in stereo mode, the left and right buffers will have the same content.
Point : A NxN pixel box of a single RGB color. Center of target is at the center of the pixel box. There is no "pattern" associated with this target type. Parameters: N in pixels (limited to [1..25]). RGB color, flicker, stereo disparity.
Random-Dot Patch : Random-dot pattern restricted to a defined target window. Unlike the old XYScope random-dot targets, this target offers adjustable dot size and color, a 2D Gaussian window (if desired), and the ability to apply the "finite dot life", "% coherence", and "directional or speed noise" features simultaneously. Parameters: #dots, dot size in pixels [1..25], flicker, stereo disparity. RGB dot color and contrast (if the contrast for any single color component is nonzero, then half the dots are painted in one color and half in another color, as described above). Window aperture shape (rectangular, elliptical, rectangular annulus, and elliptical annulus), width, and height (both inner and outer dimensions for annular aperture). Standard deviations in x and y for the Gaussian window; a value of zero is treated as infinity, so if both standard deviations are zero, no Gaussian window is applied. Maximum dot lifetime (0 = infinite dot life). Percent coherence (100 = complete coherence). The noise update interval in milliseconds and the per-dot noise range limit, N, defining the range [-N..N]. You can configure the target so that pattern velocity is specified with respect to the display screen rather than the target window. Finally, there is a seed value for the random number generator (RNG) that randomizes dot locations; it also seeds a separate RNG dedicated to dot directional/speed noise.
Random-Dot Flow Field : Simulates an optic flow field, with randomly-positioned dots moving radially away from or toward a central focus of expansion (FOE). The boundaries of the target window are elliptical rather than rectangular, and each dot moves at a different speed depending upon its radial distance from the FOE. Positive pattern velocities correspond to dot motion toward the FOE. Users must specify an inner as well as outer radius for the flow field, since dots will take "forever" to reach the FOE -- leading to clumping of the pattern around the FOE when the pattern velocity is negative. Implementation has been tweaked to work best for pattern velocities in the range 5-60 deg/sec. Parameters: #dots in pattern, dot size in pixels [1..25], flicker, stereo disparity, and dot RGB color. Inner and outer radii of flow field.
Solid Bar/Line : A solid rectangular bar or a pixel-thick line drawn in a single RGB color. This target type does not have a separate target "pattern". Parameters: Width and height of bar in degrees subtended at the eye (a line is drawn instead if the width is zero). RGB color. Flicker. Drift axis angle, i.e., the counter-clockwise angle between the positive x-axis and the vertical dimension of the bar.
Uniform Spot : A single-color patch, optionally masked by a 2D Gaussian window. Parameters: Window aperture shape and dimensions (both inner and outer dimensions for an annular aperture). RGB color. Standard deviations in x and y for the Gaussian window. Flicker.
Grating : A sinusoidal or square-wave grating, optionally masked by a 2D Gaussian window. Note that the grating can only drift along the grating normal, or "drift axis". Parameters: Grating function (square-wave or sinusoidal), spatial frequency, drift axis angle, and initial spatial phase. Window aperture shape (rectangular or elliptical), width, and height. Gaussian standard deviations in x and y. Mean RGB color and RGB contrast of grating, which defines the per-channel luminance range [Lmin .. Lmax] spanned by the grating, as described earlier. The square-wave grating will have just two bands per grating cycle, one at Lmin and one at Lmax. The "dynamic orientation" flag; if set, grating's orientation is offset on a frame-by-frame basis by the direction angle of the target's per-frame pattern velocity vector. Flicker.
Plaid : A sinusoidal or square-wave plaid, optionally masked by a 2D Gaussian window. Parameters: Similar to the Grating target, except that there are two sets of grating parameters. In addition, a flag may be set to treat the two gratings as independent target patterns. In a trial, this target's horizontal pattern velocity is interpreted as the speed of the first grating along its drift axis, while the vertical pattern velocity is interpreted as the second grating's drift speed. If the flag is not set, the gratings act as a single unified "plaid" pattern. The "dynamic orientation" flag; if set, the plaid pattern's orientation is offset on a frame-by-frame basis by the direction angle of the target's per-frame pattern velocity vector. Note that if this flag is set, the two grating components must act as a unified pattern, so the "independent gratings" flag is ignored.
Movie : This target plays a movie extracted from a video file located in RMVideo's local "media store": The target implementation relies on open-source Linux libraries that are part of the FFMPEG project. These libraries support a wide variety of video file formats (e.g., AVI, MPEG) and codecs (e.g., Cinepak, Indeo, H.264); see the FFMPEG website for details. Parameters: The name of the video file and the folder in which it is located within the media store, plus several flags governing movie playback behavior during a trial. Flicker parameters are applicable as well.
Image : (As of V3.3.1 and RMVideo V7) This target displays a static image read from a JPEG, PNG, BMP, or PSD file located in the media store. Image loading relies on the public-domain STB_IMAGE library. Not all versions of these image formats may be supported, so it is always best to test your images before you start an experiment that would use them. Note that it is possible to use images that have transparent regions. Parameters: The name of the image file and the folder in which it is located within the media store. Flicker parameters are applicable as well.