Drivers

Once your i.MX6TinyRex is up and running, you’ll need to insert a kernel module. To do so, use the command:

modprobe mxc_v4l2_capture

You should also ensure that the adv7610 was recognized, and the kernel module was loaded. Verify with the command

root@imx6q-tinyrex:~# lsmod |grep adv7610

adv7610_video 6202 0

v4l2_int_device 2981 3 ov5647_camera_mipi,adv7610_video,mxc_v4l2_capture

As long as you see an adv7610_video line, you’re good to go!

Gstreamer

Freescale heavily uses Gstreamer in both it’s BSP as well as example code. Gstreamer is a very handy way to create video “pipelines”. A pipeline consists of elements put together in a particular order to convert video or audio data to encoded and/or packaged data or vice-versa. In our case, we want to take video input from the ADV7610 (connected to the i.MX6′s CSI input), convert or encode it and display it on the HDMI output or transport it over a network. A standard Gstreamer h.264 encoding pipeline might look something like:

[ video_source ] -> [ h.264 encoder ] -> [ rtp payloader ] -> [ network sink ]

and simultaneously on the receiving, decoding side:

[ network source ] -> [ rtp depayloader ] -> [ h.264 decoder ] -> [ HDMI output ]

Gstreamer allows you to create these pipelines quickly and easily from the command line using the gst-launch utility and various Gstreamer plugins that make up the elements of the pipeline. To get a list of plugins available, use the gst-inspect command. You can then use gst-inspect on a particular plugin to get more details, e.g:

gst-inspect imxv4l2src

For a much more detailed discussion of Gstreamer, see their gst-launch tutorials here.

Example Gstreamer Pipelines: HDMI input -> HDMI output

The ADV7610 HDMI decoder uses the Freescale v4l2 input module, mxc_v4l2_capture. Freescale’s Gstreamer plugin imxv4l2src takes video from the mxc_v4l2_capture and makes it available to subsequent plugins.

On the output side, there are two available output sinks: imxv4l2sink and mfw_isink. As the name implies, imxv4l2sink uses the v4l2 library to output to the first framebuffer device. Alternatively, mfw_isink writes data directly to the framebuffer, skipping v4l2. It also does other cool things like output to multiple framebuffers simultaneously, and its latency is much less than imxv4l2sink. However, it does require more configuration and can be difficult to work with depending on the window manager being used. Generally, the best output sink to use is autovideosink, which will automatically select the element for you.

So, connect an HDMI source (like your computer, a GoPro, or a BluRay player) to the HDMI input on your TinyRex baseboard, and the HDMI output to a monitor. To display the HDMI input on your monitor, run the pipeline:

gst-launch imxv4l2src ! autovideosink

Example Gstreamer Pipelines: HDMI input -> encoder -> network

If you want to send video over a network, you will need to encode and payload it first. As an example, lets use Freescale’s vpuenc plugin which is capable of using the i.MX6′s hardware encoding engine (the VPU) to encode video into MPEG4, MPEG, h.263 and h.264 formats. To get the video in a form that video players and other Gstreamer clients can understand, we’ll payload the h.264 encoded video using RTP and the rtph264pay plugin. Finally, we’ll send the video over UDP to a client on port 5550.

gst-launch imxv4l2src ! vpuenc codec=avc bitrate=4000000 ! rtph264pay ! udpsink host=192.168.1.1 port=5550 -v

Note that I’ve added arguments to the vpuenc and udpsink elements. Specifically, we use the “avc” codec for vpuenc (telling it to use h.264) at a fixed bitrate of 4Mbps. Additionally, we tell the udpsink to send the data via UDP to 192.168.1.1 on port 5550.

You’ll also notice the -v argument on the end (verbose). This tells gst-launch to spit out a bunch of additional information, including the “capabilities” of each element in the pipeline. These capabilities are important: they tell other elements about the stream, like width/height and the encoded stream type. In order to display the video in a receiving pipeline, we need to tell the receiver pipeline what the capabilities of the stream are. Specifically, we need the capabilities at the udpsink element:

/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)\"Z0JAH6aAUAW5AA\\=\\=\\,aM44gAA\\=\", payload=(int)96, ssrc=(uint)3515352658, clock-base=(uint)3053564270, seqnum-base=(uint)53648

Example Gstreamer Pipelines: Network -> decoder -> HDMI output

We can now create a receiving pipeline, either on your computer or on another i.MX6TinyRex. Lets use the example of another i.MX6TinyRex, where we use the vpudec and rtph264depay elements to again leverage the i.MX6′s hardware h.264 decoder to decode the video and display it on the HDMI output. Note: we need to use the caps from the encoding pipeline in the udpsrc stream to tell the receiving pipeline about the capabilities of the stream. Do this by setting the “caps” property of the udpsrc to the same properties, copied and pasted from above and put in double-quotes:

gst-launch udpsrc port=5550 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)\"Z0JAH6aAUAW5AA\\=\\=\\,aM44gAA\\=\", payload=(int)96, ssrc=(uint)3515352658, clock-base=(uint)3053564270, seqnum-base=(uint)53648" ! rtph264depay ! vpudec low-latency=true ! autovideosink

The vpudec automatically knows what type of stream to decode from the capabilities passed to it from rtph264depay, but I do add the low-latency=true argument to have it produce frames as soon as they are decoded. The autovideosink is again used to output on the HDMI.