Components
Nvidia Jetson TX1
The Jetson TX1 is the core of our project, and will handle any and all video processing and 3D mode adjustments. The TX1 is designed specifically for visual computing and processing, and has 256 CUDA cores perfect for video processing. We will utilize OpenCV and Nvidia's Multimedia API to process the interlacing of the two video streams.
For compatibility with the Sony IMX334 camera sensor, this kit requires an additional carrier board which seats the sensors onto the Nvidia Jetson TX1. This setup enables up to three 4-lane MIPI / CSI-2 camera streams, but only two will be necessary to achieve a 3D effect. You can learn more about the 931-LI-JTX1-MIPIADPT carrier board from Leopard Imaging by exploring the datasheet below.
Nvidia Jetson Xavier NX
The Xavier NX features two more CPU cores at a higher clock speed, and more CUDA cores than the Jetson TX1. It is the workhorse for especially demanding processing needs, primarily when processing and interlacing 4K video streams. As the Sony IMX334 is not currently supported by the Jetson Xavier NX, we use this more powerful SOC as a proof of concept, to demonstrate the potential of the system when using video files, or with a compatible camera module.
Sony IMX 334
The Sony IMX334 camera module includes a 4K sensor capable of 30 to 60 frames per second. The sensor and lens elements are assembled by Leopard Imaging, and feature an f/2.2 aperture with a focal length of 5mm. The two camera modules will be placed approximately 4.5 inches apart—mimicking a human’s natural interpupillary distance (IPD)—such that a 3D effect is optimally presented on the monitor. The camera's focus is adjustable with a set screw capable of locking the focus.
Miscellaneous
An FAW-1233-03 Micro Coax I-PEX cable is used to connect our camera modules to the Jetson TX1 carrier board. The cable is uniquely designed by Leopard Imaging for compatibility with the TX1 board and IMX334 sensors. With a 30-pin connector on either end, it enables a high-speed MIPI / CSI-2 data stream from both 4K sensors.
A 3D monitor, courtesy of Alcon, is capable of polarizing individual rows for viewing by the surgeon's left or right eye. When alternating rows with opposite polarity are viewed together, a 3D effect is created in either 4K or 1080p resolution. The monitor may display our own interlaced output or produce its own result when fed with top-bottom or side by side inputs.
Various interface and memory devices are also included, such as a USB hub for the Jetson TX1, several micro SD cards for individual testing on Jetson Nano, and portable flash drives to transfer software to the TX1. Alternatively, the TX1 kit may be connected to the internet via Ethernet for initial transfer of software. An HDMI (preferred) or DisplayPort cable is needed to feed the Jetson's output to the 3D monitor.
The final Opto3D product is enclosed within a custom 3D-printed case, complete with custom buttons to toggle system power, choose the desired 3D mode, and swap left / right polarity. The system case and buttons are designed using Autodesk Fusion 360, and interface with the Jetson TX1 via the many available GPIO pins.
Return to homepage