RMVideo Target Types
Prior to Maestro version 2.0.0, framebuffer video targets were implemented on the VSG2/4, an ISA-based programmable video card from Cambridge Research Systems. The VSG was an antique compared to today's video adapters, and its slow rendering performance severely restricted the complexity, size, and number of targets that could be simultaneously animated. As a result, framebuffer video targets were rarely used in past versions of Maestro; the most common usage of the VSG-based framebuffer video involved the display of large-field gratings and plaids to study spatio-temporal response properties of cells in V1 and MT.
RMVideo was developed to replace the VSG and significantly enhance the utility and versatility of Maestro's framebuffer video target platform. First integrated into Maestro 2.0.0, RMVideo is a completely independent program running on a separate Linux workstation and communicating with Maestro over a dedicated point-to-point Ethernet link. While its introduction required the addition of a second workstation to the laboratory apparatus, it offers many advantages in performance and capabilities over the VSG-based framebuffer solution:
As an OpenGL application, it can be used with any graphics card that has an adequate OpenGL Linux driver. We are not restricted to one or two video cards on the market, and upgrading to a newer adapter is relatively painless.
All of the original VSG-based framebuffer video targets are supported. Better yet, a grating or plaid can drift under an elliptical Gaussian window -- a stimulus that was not possible with the VSG.
RMVideo targets are updated on a frame-by-frame basis, in much the same manner as XYScope targets. As a result, they can participate in trials that involve velocity stabilization or any other special feature that requires runtime modification of a target's computed trial trajectory. This was not possible with VSG-based targets.
It is possible to implement any of the current XYScope target types with an RMVideo target. In fact, one of the reasons for developing RMVideo in the first place was to guard against the eventuality that our analog XY oscilloscopes will break down for good (they are no longer manufactured and are increasingly difficult to repair). RMVideo offers the additional capability to display random-dot patches in color and with an elliptical Gaussian window. It can also draw more total dots per frame than our XYScope controller, but its frame rate and resolution are fixed and inferior to those of the XYScope platform.
Introducing new kinds of RMVideo targets in the future should be relatively easy in terms of development time and costs.
As of version 2.5.0, it is possible to switch among several alternative video modes (screen resolution + vertical refresh rate). This allows users to maximize temporal resolution (refresh rate) over spatial resolution or vice versa. A typical RMVideo workstation will be capable of many more video modes than are exposed by Maestro; the minimum allowed display resolution is 1024x768 pixels and the minimum refresh rate is 60Hz. All available video modes offer 24-bit RGB color with an 8-bit transparency channel (needed, e.g., for masking a target window with a 2D Gaussian).
The list below describes the different types of RMVideo targets currently supported in Maestro. Each target type is characterized by a variety of parameters; common parameters include window aperture, bounding rectangle, and color. Grating target parameters include spatial frequency in cycles per visual degree subtended at the eye, initial spatial phase in degrees, and the drift axis. The drift axis defines the grating's orientation; it is the angle (in counterclockwise degrees) made with the positive x-axis by a line perpendicular to the grating lines. For the grating and plaid targets, the range of colors in which a grating is rendered are defined by a mean RGB triplet and a contrast RGB triplet. Each component of the RGB mean color are specified with 8-bit resolution, [0..255], with 255 corresponding to the maximum luminance. Assuming the video card and attached monitor have been gamma-corrected, this 8-bit scale should map linearly to the luminance range available. (Users can adjust the monitor's gamma-correction factors directly from Maestro.) For each color component in RGB space, the mean Lmean and contrast C [0..1] define a range of luminance values [Lmin .. Lmax] assigned to the grating, according to:
Lmax = Lmean(1+C) and Lmin = Lmean(1-C).
As of version 3.1.2, you can specify a nonzero contrast for the Random-Dot Patch target. Doing so puts the target in a special two-color contrast mode. Instead of painting all dots in the patch in the same color, half the dots are drawn using Lmax for each color component (typically this mode would be used in grayscale only), while the other half are drawn using Lmin.
The Movie target requires a video source file while the Image target requires an image file. Such media files are stored locally on the RMVideo server, in the "media store": a set of up to 50 folders in $RMVIDEO_HOME/media, each of which can hold up to 50 media files, either videos or images (the maximum capacities are an artifact of the restricted size of packet buffers used in Maestro/RMVideo communications). Media files can be added to or removed from the store via direct interaction with the RMVideo workstation (often a hassle in our experiment rigs!). Alternatively, one can view the store's content, remove a particular file or an entire folder, and download new files directly in Maestro, using the RMVideo Media tab in the Idle mode control panel.
As of version 4.1.0, you can make any RMVideo target flicker periodically whenever it is displayed during a trial or in Continuous mode. The feature was added to support apparent motion studies. The flicker cycle is defined by "on" and "off" phases, and an initial delay period preceding the first "on" phase. All three parameters are specified as a number of video frame periods; flicker is disabled if either the "on" or "off" phase is zero.
Point : A NxN pixel box of a single RGB color. Center of target is at the center of the pixel box. There is no "pattern" associated with this target type. Parameters: N in pixels (limited to [1..10]). RGB color, flicker.
Random-Dot Patch : This target can emulate any of the XYScope targets except Optical Flow Field, Spot/Rect Dot Array, and Oriented Bar/Line. Unlike the XYScope versions, this target offers adjustable dot size and color, a 2D Gaussian window (if desired), and the ability to apply the "finite dot life", "% coherence", and "directional or speed noise" features simultaneously. Parameters: #dots, dot size in pixels [1..10], flicker, and RGB dot color and contrast (if the contrast for any single color component is nonzero, then half the dots are painted in one color and half in another color, as described above). Window aperture shape (rectangular, elliptical, rectangular annulus, and elliptical annulus), width, and height (both inner and outer dimensions for annular aperture). Standard deviations in x and y for the Gaussian window; a value of zero is treated as infinity, so if both standard deviations are zero, no Gaussian window is applied. Maximum dot lifetime (0 = infinite dot life). Percent coherence (100 = complete coherence). The noise update interval in milliseconds and the per-dot noise range limit, N, defining the range [-N..N]. The meaning of N and its allowed values are the same as described for the XYScope targets Noisy Dots (Direction) and Noisy Dots (Speed). If you want a noise update to occur on every display frame, set the interval to a non-zero value less than the display frame period. You can configure the target so that pattern velocity is specified with respect to the display screen rather than the target window. Finally, there is a seed value for the random number generator (RNG) that randomizes dot locations; it also seeds a separate RNG dedicated to dot directional/speed noise.
Random-Dot Flow Field : This is the RMVideo analog for the XYScope target Optical Flow Field. The algorithmic implementation and parametric definition is essentially the same. However, this target offers the ability to set the dot size and RGB color. Parameters: #dots in pattern, dot size in pixels [1..10], flicker, and dot RGB color. Inner and outer radii of flow field.
Solid Bar/Line : A solid rectangular bar or a pixel-thick line drawn in a single RGB color. This target type does not have a separate target "pattern". Parameters: Width and height of bar in degrees subtended at the eye (a line is drawn instead if the width is zero). RGB color. Flicker. Drift axis angle, i.e., the counter-clockwise angle between the positive x-axis and the vertical dimension of the bar.
Uniform Spot : A single-color patch, optionally masked by a 2D Gaussian window. Parameters: Window aperture shape and dimensions (both inner and outer dimensions for an annular aperture). RGB color. Standard deviations in x and y for the Gaussian window. Flicker.
Grating : A sinusoidal or square-wave grating, optionally masked by a 2D Gaussian window. Note that the grating can only drift along the grating normal, or "drift axis". Parameters: Grating function (square-wave or sinusoidal), spatial frequency, drift axis angle, and initial spatial phase. Window aperture shape (rectangular or elliptical), width, and height. Gaussian standard deviations in x and y. Mean RGB color and RGB contrast of grating, which defines the per-channel luminance range [Lmin .. Lmax] spanned by the grating, as described earlier. The square-wave grating will have just two bands per grating cycle, one at Lmin and one at Lmax. The "dynamic orientation" flag; if set, grating's orientation is offset on a frame-by-frame basis by the direction angle of the target's per-frame pattern velocity vector. Flicker.
Plaid : A sinusoidal or square-wave plaid, optionally masked by a 2D Gaussian window. Parameters: Similar to the Grating target, except that there are two sets of grating parameters. In addition, a flag may be set to treat the two gratings as independent target patterns. In a trial, this target's horizontal pattern velocity is interpreted as the speed of the first grating along its drift axis, while the vertical pattern velocity is interpreted as the second grating's drift speed. If the flag is not set, the gratings act as a single unified "plaid" pattern. The "dynamic orientation" flag; if set, the plaid pattern's orientation is offset on a frame-by-frame basis by the direction angle of the target's per-frame pattern velocity vector. Note that if this flag is set, the two grating components must act as a unified pattern, so the "independent gratings" flag is ignored.
Movie : This target plays a movie extracted from a video file located in RMVideo's local "media store": The target implementation relies on open-source Linux libraries that are part of the FFMPEG project. These libraries support a wide variety of video file formats (e.g., AVI, MPEG) and codecs (e.g., Cinepak, Indeo, H.264); see the FFMPEG website for details. Parameters: The name of the video file and the folder in which it is located within the media store, plus several flags governing movie playback behavior during a trial. Flicker parameters are applicable as well.
Image : (As of V3.3.1 and RMVideo V7) This target displays a static image read from a JPEG, PNG, BMP, or PSD file located in the media store. Image loading relies on the public-domain STB_IMAGE library. Not all versions of these image formats may be supported, so it is always best to test your images before you start an experiment that would use them. Note that it is possible to use images that have transparent regions. Parameters: The name of the image file and the folder in which it is located within the media store. Flicker parameters are applicable as well.