A Brief History
Maestro has evolved over the years to take advantage of performance improvements in the computer workstation and its operating system, to introduce major new functional features, and to replace antiquated legacy hardware.
Cntrlx: 1990s - 2003
Cntrlx was the name given to Maestro's predecessor. It was originally conceived and developed by Dr. Steve Lisberger, amended in various ways by contributions from his graduate students and postdoctoral researchers. In its earliest incarnation, Cntrlx was a distributed network application in which each of three functional modules resided on a different computer. A UNIX workstation hosted an XWindows™-based graphical user interface (cntrlxUNIX), while the experiment control and hardware interface (cntrlxPC) ran as a DOS program on a separate PC. A third module (spikesPC) -- a DOS app running on a second PC -- was dedicated to the 50KHz recording of the amplified signal from an extracellular microelectrode (for offline spike sorting). CntrlxUNIX sent commands over the network to direct the activity of the other two programs, which sent back data packets and/or status messages in return. This distributed design was driven, of course, by the prevailing technology at the time. No single-CPU workstation was fast enough to satisfy the performance requirements imposed by the lab's experimental paradigm.
However, as processor speeds and operating system performance grew by leaps and bounds with each passing year, it became apparent that a redesign was needed. We could no longer enhance the functionality of cntrlxPC because we were bumping up against limitations in the now obsolete DOS. We also wanted to take advantage of the far superior capabilities and resources of the Windows operating system.
In late 1998 Scott Ruffner, a programmer hired to take over development and maintenance of lab software, began the first phase of a "modernization" project: porting the hardware controller from DOS to Windows NT. At the same time, we also upgraded some of the peripheral hardware devices used by the program. The obsolete ISA-based National Instruments AT-MIO-16F5 data acquisition card was replaced by the PCI-MIO-16E1, and the Spectrum TMS320C30 DSP card used to drive our XY oscilloscope displays (for single dot and random-dot patch targets) was replaced by a next-generation C6201 DSP. Much development effort focused on implementing new targets for the XY oscilloscope -- which was fast becoming the most widely used stimulus platform in the laboratory -- and adding entirely new features to support ongoing research projects. Finally, with the integration of the VSG2/3 framebuffer video card from Cambridge Research, it became possible to present color and sinusoidal grating stimuli on a cathode ray tube (CRT) monitor.
Maestro 1.x: 2003 - 2006
Porting the hardware controller from DOS to Windows allowed us to run longer trials (we had much more more memory to work with!) and implement many new features. Still, cntrlxNT (as it was then called) was not taking full advantage of the resources and computing power offered by Windows and the underlying hardware. In addition, by 2003, Windows XP had arrived to take the place of Windows NT. In the second phase of the modernization project, we merged the three functional modules (cntrlxUNIX, cntrlxNT, and spikesPC) into one application running on a single Windows XP workstation. A new name was in order, and Maestro was born. The GUI was completely redesigned as a Windows application, and the hardware controller was modified to assume the duties once handled by spikesPC (albeit at half the sample rate: 25KHz instead of 50). The final product offered significant per-system cost savings, freed up valuable rack space in the lab, and eliminated dependence on network resources.
The port to Windows would not have been possible without the Real-Time Extension (RTX) package from IntervalZero. RTX brought to Windows the real-time performance, fine-grain thread scheduling, and easy hardware access that made our development effort a success. The hardware controller runs as a kernel-level process within the RTX environment, interacts with the Windows GUI over interprocess shared memory, and communicates directly with the hardware using RTX-supplied device management functions.
Besides numerous technical changes, feature additions and enhancements, there are major functional differences between Maestro and CntrlxUNIX/NT. Here are the highlights:
A completely new user interface. A lot of effort was expended to present a well-organized GUI that addressed some of the shortcomings in the CntrlxUNIX interface. Instead of displaying a multitude of overlapping windows all over the screen, the user interface is confined to a single frame window. Use of split panes, tabbed windows, and dockable panels keep the interface compact and clean, while still permitting the user to customize its layout. In addition, Maestro stores the user's layout preferences in the Windows registry -- no more time wasted arranging windows at startup!
Faster access to experimental protocols. Users "create" an experimental protocol by defining one or more sets of trials or stimulus runs, along with any targets, channel configurations, and perturbation objects upon which those definitions depend. CntrlxUNIX stored the definitions of these various "building blocks" in individual ASCII text files. A seasoned CntrlxUNIX user could build up a relatively large set of such files, which could prove difficult to maintain or wade through in search of a particular file. In addition, because the program limited the number of objects resident in memory to 100, switching to a different trial set was a multi-step process: the old set must be cleared, the new set found and loaded, and any targets on which the new trials depend must also be loaded. Saving and accessing experiments is much simpler in Maestro. The user can store many different experiments in a single Maestro "experiment document". The document itself is a hierarchical tree with five major nodes: Trials, Stimulus Runs, Targets, Perturbations, and Channels. A user can easily maintain all of his experimental protocols in one such document, which can store more than 65000 nodes. Furthermore, switching to a different experiment can be simply a matter of selecting the desired trial set or stimulus run set from a drop-down combo box!
All data in a single file. Since Maestro integrates spikesPC functionality into the hardware controller itself, the high-resolution 25KHz spike waveform data is stored in the Maestro data file rather than a separate spikesPC file. All data collected during a single recording is now always stored in a single file.
Test & Calibration Mode. This operational mode, which was not available in CntrlxUNIX, provides some limited support for testing Maestro's device hardware and the electronics in the experiment apparatus.
Support for newer PCI-based hardware. The ISA bus standard for computer peripherals was declared obsolete in the 1990s. As of January 2000, most computer manufacturers no longer built PCs with ISA bus slots -- although it is still possible to find high-performance PCs with ISA bus support. Maestro 1.0 added support for PCI-based replacements for two legacy ISA devices required in most laboratory rigs: the DIO event timer and the analog output board.
Maestro 2.x: 2006 - 2011
In the early years of Maestro, the analog X-Y or vector oscilloscope gradually became our "workhorse" display for visual target stimuli. The XYScope display was preferred over the framebuffer video display because it could be updated at rates as high as 500Hz, while the CRT monitors that were driven by the VSG framebuffer card were limited to ~75-100Hz. However, the large vector scope was already an obsolete piece of technology. As our 1970s-vintage HP1321 and HP1304A scopes began to break down, we found them very difficult to repair and nearly impossible to replace. We realized that we had to prepare for the inevitable demise of the XYScope as a Maestro target display. In addition, interest was growing in the use of drifting sinusoidal grating and plaid targets in experiments, particularly if such stimuli could be windowed by a Gaussian function -- a so-called "Gabor" patch. Unfortunately, our VSG framebuffer graphics card was quite old and not powerful enough to present drifting Gabors. It was also housed on the slow and very outdated ISA bus.
Version 2.0 of the program addressed these concerns by replacing the VSG framebuffer video card with RMVideo (for Remote Maestro Video), an OpenGL application which runs on a separate Linux workstation and communicates with the Maestro hardware controller over a private, dedicated Ethernet link. As an OpenGL application, it takes advantage of the latest and greatest video adapters on the market. It supports drifting Gabor and plaid targets, all the original VSG-based framebuffer video targets, and random-dot pattern targets analogous to those implemented on the XYScope. Better yet, it can animate more targets and larger targets than was possible with the VSG. It can also handle runtime adjustments in a target's trajectory due to velocity stabilization or other special experimental protocols -- something which was again not possible with the VSG. The leap in rendering performance and the greater ease with which new stimuli may be developed far outweigh the disadvantages of once again requiring a second workstation in the laboratory rig.
As more and more of our vector oscilloscopes broke down, RMVideo gradually supplanted the XYScope as the primary visual stimulus platform used in the lab. Other labs that have begun to use Maestro rely solely on RMVideo for their visual stimuli. To give users more flexibility in defining visual stimuli for this platform, Maestro 2.5 introduced support for playing short movies through RMVideo. Future versions of the program will likely introduce more kinds of targets for this display.
It should be noted that the RMVideo display's refresh rates remain inferior to those of the XYScope, and it also faces the danger of technological obsolescence. The LCD flat panel has long replaced the venerable CRT as a computer monitor, but these newer displays are not suitable for neuroscience research. LCD monitor response times are orders of magnitude slower than those of a CRT, and these slow response times lead to artifacts in fast-moving stimuli. Furthermore, adaptive circuitry introduced in an effort to improve response time can cause artifacts in the display and have blurred the meaning of "refresh rate" as it applies to LCD monitors. For these reasons, we continue to use high-performance CRTs for our RMVideo displays. But, as with the vector oscilloscope, such top-flight CRT monitors are nearly impossible to find anymore. There is hope that newer display technologies -- such as OLED and QLED -- will eventually provide a viable alternative to the dinosaur CRT and the unsuitable LCD.
Maestro 3.x: 2011 - 2018
By 2011 Maestro was once again in need of a facelift. Microsoft had announced that support for the Windows XP operating system was finally coming to an end. While we avoided Windows Vista, it was time to make the move to Windows 7. Furthermore, we needed to catch up with changes in PC technology and replace old hardware. Typical workstation CPUs now came standard with 2-4 cores, but Maestro 2 running under the now-deprecated RTX 6.5.1 could not take advantage of the additional cores. The National Instruments' PCI-MIO-16E1 (now known as the PCI-6071) was obsolete and very expensive; newer, more capable data acquisition cards were available for half the price. The Lisberger digital IO and timestamping board was no longer available, and the alternative DSP-based emulation of that board's functionality required a rather expensive M62 card from Innovative Integration. On top of all this, the PCI bus was quickly being replaced by the faster PCI-Express standard, yet another reason to find modern alternatives for Maestro's old ISA- and PCI-based peripherals.
Version 3.0 addressed all of these technical concerns. The code was rebuilt with Microsoft Visual Studio 2010 to run on Windows 7, with the hardware controller running under RTX 2011. While the older devices would still be supported in this version, the intent was to migrate most device functionality to modern PCI-Express boards. There was little point in replacing the Detroit C6x DSP card used to drive the XYScope display, since only one working display remained. The Intel Gigabit CT Ethernet adapter was chosen for communications with RMVideo, as it was one of the few PCI-Express network cards supported by RTX. Finally, Maestro's analog input, analog output, and digital IO timestamping hardware operations were re-implemented on a single physical device, the PCIe-6363 multi-function IO board from National Instruments. With 32 16-bit analog inputs, four 16-bit analog outputs, 48 digital I/O lines, DIO change detection circuitry, and four independent 32-bit counters, the PCIe-6363 had more than enough resources to handle functionality once divided among three separate boards. [NOTE: Maestro 2 actually required nine analog outputs, but eight of these were dedicated to controlling the position of the shuttered fiber-optic spot targets, Fiber1 and Fiber2. These had fallen out of use long ago, and we decided that Maestro 3 would no longer offer them.]
Another reason to migrate to the newer devices and to RTX 2011 is their support for message-signaled interrupts (MSI). Since its inception, one of the most painful tasks in setting up a new PC to run Maestro was configuring the system so that the AI data acquisition card and the RMVideo network adapter each obtained exclusive access to one of the 16 hardware interrupt lines (IRQs). The BIOS and operating system offer little control over IRQ assignments, so we often had to disable many standard devices (like USB and sound card) and move the devices to different PCI slots until each was given exclusive access to an IRQ. With MSI, this problem goes away. As long as you use the new devices, it is much easier to configure a workstation to run Maestro 3.
Significant recent updates/additions in Version 3 include: (1) support for sampling position data from the EyeLink eye tracker over a dedicated Ethernet connection, integrating that eye position data with Maestro-collected data and including it in the Maestro data file; (2) a rebuild of RMVideo on an up-to-date Linux OS; and (3) a new RMVideo target supporting the display of a static image drawn from a JPEG, PNG, BMP, or PSD file.
Maestro 4.x: 2018 - present
By 2017 the Maestro 3.x machines in the Lisberger lab were getting rather old. However, Microsoft had long moved on from Windows 7; mainstream support for the OS had ended in 2015 and extended support would end in 2020. Furthermore, Windows 7 could no longer be installed on new PCs, and 32-bit RTX did not support the latest Window OS, Windows 10. In order to transition the lab to new workstations running Windows 10, we needed to port Maestro to 64-bit Windows and its 64-bit RTX counterpart, RTX64.
Version 4.0 updates Maestro from a 32-bit application to a 64-bit application running under Window 10 1607 LTSB (extended support through 2026) and RTX64 version 3.4. At the same, it ends supports for legacy hardware using the very out-dated ISA and conventional PCI peripheral communication interface standards. The PCIe-6363 multi-function board and an RTX64-supported NIC (like the Intel Gigabit CT Desktop adapter) are the only supported hardware.
As a consequence, Version 4.0 drops support for the XYScope display; the only viable visual stimulus platform at this point is RMVideo. This is of little consequence since there are no longer any suitable analog XY oscilloscopes available. [Note: It is still possible to define XYScope targets in a Maestro 4, but they cannot be displayed as such. We have kept the XYScope-related code base in Maestro in case an alternative XYScope display solution is implemented in the future.]
Like the analog XY scopes, the high-performance CRT monitors we rely on as RMVideo displays are no longer manufactured and are getting quite old. We have avoided using LCD monitors because of undesirable motion artifacts associated with the "sample and hold" nature of these displays. OLED monitors, which may someday offer spatio-temporal performance similar to CRTs, are still not mainstream as of 2019. We have recently tested RMVideo with a Dell S2417DG LCD monitor that supports 2560x1440 @ 144Hz using a DisplayPort 1.4 cable. The monitor also supports NVidia Ultra Low Motion Blur (ULMB) technology at 85-120Hz. While further side-by-side comparisons with a CRT are needed, initial tests suggest it may be feasible to run certain kinds of experiments on a "gaming" monitor like the Dell S2417DG. In anticipation of going this way eventually, Maestro 4.0.5 brought some improvements to the operation of RMVideo and some additional flexibility to tolerate a limited number of "frame drops", since these are much more likely to happen at the higher refresh rates. In Maestro 4.1.0, RMVideo was re-implemented to conform to "modern" OpenGL -- OpenGL 3.3 Core Profile to be specific. This change eliminated RMVideo's dependence on legacy "immediate mode" functions, which have long been considered obsolete and may no longer be supported in future OpenGL drivers.