Connect a webcam

Sorry, this is in note form and not a simple recipe - it's an ongoing thing for the moment.

Didn't seem much point in connecting a webcam unless I can avoid the USB stick, so the challenge is to see how small a video capture application might be statically linked. I don't much care about JPG or streaming, so I am thinking about just getting hold of a .ppm (or series of PPMs) that I can do some image processing on later - maybe motion detection or object following. I'm sure the CPU power of the S3282 is not going to be up to much, but that's OK for basic stuff.

First things first: Later kernels changed how they worked in terms of image compression. Many webcams only produce JPEG images at the higher resolutions, and the kernel drivers used to do the decompression, thus presenting video4linux applications with an RGB image. Things have now become more complicated for us since someone decided that it's not a good idea to do this decompression in the kernel, so we have to do it in userspace. Fortunately for us, a clever guy called Hans de Goede has produced a library to help, however that does mean rather more bloat in our video capture program.

I had a look at motion, pretty much the defacto standard for embedded systems, however it's still too large to fit in 1MB flash, so it looks like I will have to start from first principles with this one.

In the Documentation directory for most kernels there is at least a v4lgrab.c application, that appears to use the v4l1 API to capture an image (it doesn't say it's v4l1, but I'm guessing). See linux-2.6.30.1/Documentation/video4linux/v4lgrab.c.

My webcam is pac207, incredibly common. I have some others, but I had to tinker to get them to work with Linux on my laptop, so God only knows how much hassle to make them work on Bifferboard. It says this from dmesg when I plug it into my laptop:

usb 2-1: new full speed USB device using uhci_hcd and address 3

usb 2-1: configuration #1 chosen from 1 choice

gspca: probing 093a:2468

pac207: Pixart Sensor ID 0x27 Chips ID 0x00

pac207: Pixart PAC207BCA Image Processor and Control Chip detected (vid/pid 0x093A:0x2468)

gspca: probe ok

usb 2-1: New USB device found, idVendor=093a, idProduct=2468

usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0

usb 2-1: Product: CIF Single Chip

usb 2-1: Manufacturer: Pixart Imaging Inc.

width = 176, height=144

width = 352, height=288

I needed to first install libv4l. I got it from Hans' site. One could alternatively install it via the distribution package manager. Not as easy to find that link as you might hope - it was only referenced on a mailing list, not on his site!

I tried out this v4lgrab.c by compiling it with the instructions in the comments at the top, and then running it with libv4l. Check out the instructions for libv4l, however they basically say, do something like this:

$ export LD_PRELOAD=/usr/local/lib/libv4l/v4l1compat.so

$ ./v4lgrab > file.ppm

$ gimp file.ppm

Hey presto! I've got an image from my webcam, and it's PPM! Now to 'convert' this for Bifferboard.

Unfortunately, I don't really want to have v4l1compat.so hanging around on my Bifferboard, and it probably has dependencies anyhow. This is just a helper intercepting calls to v4l and redirecting them. It's for when you either don't have the source for an app, or can't be bothered to convert it to the work with the latest kernels. Can't be too efficient, so I decided I must surely be able to convert the simple capture app to use the 'proper' way - libv4l1. It turned out this was just a case of search-and-replace:

ioctl -> v4l1_ioctl

read -> v4l1_ioctl

open -> v4l1_open

etc...

Well, obviously only the read/ioctl/open calls relating to video devices! Find attached at the bottom of the page my modified version of v4lgrab.c

I decided to write myself a Makefile to compile this, so I would link the libv4l library:

Makefile

all: v4lgrab

v4lgrab: v4lgrab.o

gcc -o v4lgrab v4lgrab.o -L/usr/local/lib -lv4l1

v4lgrab.o : v4lgrab.c

gcc -Wall -Wstrict-prototypes -I/usr/local/include -c v4lgrab.c

clean:

rm -f v4lgrab *~ *.o

$ ./v4lgrab > file.ppm

$ gimp file.ppm

Huzzah! My PPM is still good. And no pesky preloading this time. But there's a problem. This EXE has zillions of dependencies:

$ ldd v4lgrab

linux-gate.so.1 => (0xffffe000)

libv4l1.so.0 => /usr/local/lib/libv4l1.so.0 (0xb8012000)

libc.so.6 => /lib/libc.so.6 (0xb7ec6000)

libv4l2.so.0 => /usr/local/lib/libv4l2.so.0 (0xb7ebc000)

libpthread.so.0 => /lib/libpthread.so.0 (0xb7ea5000)

/lib/ld-linux.so.2 (0xb803a000)

libv4lconvert.so.0 => /usr/local/lib/libv4lconvert.so.0 (0xb7e3c000)

librt.so.1 => /lib/librt.so.1 (0xb7e32000)

libm.so.6 => /lib/libm.so.6 (0xb7e0c000)

And none of them are going to fit in my 1MB of Flash RAM :-(.

I would like to statically link this file, so I started to convert my Makefile:

static build Makefile

all: v4lgrab

v4lgrab: v4lgrab.o

gcc -static -o v4lgrab v4lgrab.o -L/usr/local/lib \

-lv4l1 -lv4l2 -lv4lconvert -lpthread -lrt -lm

v4lgrab.o : v4lgrab.c

gcc -Wall -Wstrict-prototypes -I/usr/local/include -c v4lgrab.c

clean:

rm -f v4lgrab *~ *.o

Then I realised libv4l doesn't build a static library by default! Aaaggh! Neither is there any documentation telling one how to do this, but I got lucky and realised simply defining a variable 'LINKTYPE=static' does the trick. Both compile and install steps need this set:

$ LINKTYPE=static make

$ LINKTYPE=static make install

Now I have a static binary that can be used for capture.

$ ls -l v4lgrab

-rwxr-xr-x 1 root root 928596 2009-07-16 06:16 v4lgrab

The next step will be to run this on the real Bifferboard.

I added support to my kernel for gspca pac207 webcam, recompiled the kernel+initrd, then flashed and booted. Webcam detected fine, so that's working. Next I setup a web server so I could 'wget' the 'v4lgrab' file across to the Bifferboard. At this point I realised asmutils is a little sucky. I tried:

$ wget http://192.168.1.68/v4lgrab

However that didn't work. After a little guesswork I managed to try:

$ wget 192.168.1.68 v4lgrab

Well, I suppose that will just have to do. No point in messing around:

/ # ./v4lgrab > file.ppm

/dev/video0: Function not implemented

Hmmm... must be something missing from the kernel, but at least the EXE runs.

I decided to add this to my kernel, to see how large it would make it, and was pleasantly surprised to see it wasn't too far over the mark after LZMA compression had done it's thing:

$ ls -l arch/x86/boot/bzImage

-rw-r--r-- 1 wrt users 1144400 2009-07-16 06:51 arch/x86/boot/bzImage

If you're using either the tftp bootloader or MMC boot (or the 8MB version of the Bifferboard!), this would be OK :).

--- intermission ---

To cut a long story short, this was due to lacking this kernel option under 'General setup':

[*] Use full shmem filesystem

What that actually means is.... activate shmem functionality in the kernel, and I thought this was activating just a file system... duh! SHMEM is the mechanism used to transfer an image from camera to application, and needed by the libv4l library.

With this option enabled, I am now able to capture a PPM file on the Bifferboard, although I must first 'wget' the grabbing application from a server somewhere. I have attached the capture app binary in case anyone wants to try it.

Of course, no webcam write-up is complete without the obligatory pointless webcam cap, and this is no exception, welcome to the corner of my laptop, my first ever image captured on the Bifferboard:

Conclusion

Capture takes about 6 seconds, although I suspect most of that is the JPEG conversion. Still, not as slow as I was expecting, remember this is doing floating point math via the kernel emulating a floating point processor (the least efficient way). With a proper integer math library this could be significantly sped up. Hopefully this will be a good starting point for people wanting to play around with image processing on Bifferboard.

TODO

    • We could reduce this application from 1MB to a lot less. I'm sure there is plenty of bloat in the conversion library because it has to deal with all the different cameras.

    • We need to speed up the JPEG decompression, or stream it to another computer to do that. One frame every 6 seconds is a little on the slow side! Perhaps a USB device which outputs RGB would be a lot faster.