Multi-point and tilt-shift focusing

Version 2.0 has the ability to focus on arbitrary planes in the scene, including planes that are angled with respect to the phone. A bellows-style view camera can do this if you tilt the lens. An SLR can do it too, although not as much, by using a tilt-shift lens like the Canon TS E series. SynthCam 2.0 offers similar functionality.

Here are some examples. All images are clickable. For a quick visual tutorial on how to use multi-point focusing, check out the second of these two Videos.

The first image in this pair is a straight photograph, i.e. a single HD resolution video frame, of a model steam-powered water pump.

The second image is a synthetic aperture photograph - the result of a 10-second recording like those on the Examples page, but focused on the entire front bracket that supports the pump cylinder. Notice that all parts of this bracket are sharp, even though it's not parallel to the picture plane. Notice also that objects become blurrier the farther they are from the plane of the bracket.

By the way, it was dark in the room when I took these pictures, so the single frame is noisy, but the synthetic aperture image is an average of dozens of frames, so it's not noisy.

Version 2.0 also offers multi-point focusing.

If you've tried using SynthCam to focus on an extended foreground object - one that nearly fills the field of view - you've probably found that you can make one spot on the object sharp, but the rest remains blurry.

In this example, I've touched several points on the urn and asked the app to keep them all in focus. The second image in this pair shows the result. Click on the image to view it at full resolution. Remember that despite its shallow depth of field, this image was taken with a cell phone camera.

Here's another example of tilt-shift focusing. The subject is George Segal's famous statue, Gay Liberation, on the Stanford University campus.

The first image is a straight photograph. It includes a sandstone column and the iconic Hoover Tower, but because everything is in focus these background elements compete with the statue for attention.

In the second image I've passed a tilted focal plane through the leftmost and rightmost figures. This keeps the statue in focus while softening the background. The difference is subtle, but I think it improves the shot. Click on the images to see them at full resolution.

Here is another pair of shots (straight and synthetic aperture), with a stronger tilt-shift effect.

It was a partly cloudy day with fast-moving clouds, so in both pairs of shots the lighting changes slightly between shots.

Multi-point focusing also helps with night shots.

In this view of the entrance of Memorial Church at Stanford, a straight photograph (first image) is very noisy, but an average of 30 frames, captured during a 3-second recording, looks fine (second image). Of course I can't hold still for 3 seconds, so I placed focus points at several places on the facade, then held as still as I could; SynthCam's tracker did the rest.

While the second image is better than an ordinary cell phone photograph, it's not perfect. Cell phones have poor dynamic range, so the bright chandelier is blown out. I could solve this problem by cycling among 2 or 3 different shutter speeds on successive video frames and computing a high dynamic range (HDR) image, but the iPhone camera's API isn't flexible enough to do this. Maybe they'll take a hint from our lab's Frankencamera project. One can hope...!

Here is SynthCam's user interface for controlling multi-point focusing and tilted focal planes.

When the app first launches, there is a "1" icon displayed on the toolbar, as shown in the first image at left. If you press on the "1", it will cycle to "2", "3", "4", and back to "1". At the same time you'll notice different numbers of white squares appearing on the screen. The solid white square shows where the camera itself is metering and its lens is focusing. The open white reticles show other points the app can track and keep sharp.

You can move or resize these focus points with drag and pinch gestures. You can also move all of them at once, by dragging the small circle in the center. For multi-point focusing to work, all focus points must be placed over scene features that the app can successfully track. This means placing them over spot-like features, corners, or coarse non-repeating textures. In the singleton image at left of an arch through which you can see Rodin's Burghers of Calais, I have placed 4 focus points, 2 on the bases of the columns and 2 on decorative elements of the wall.

The last image in this set shows the effect of recording a synthetic aperture photograph using 4 focus points. In this landscape mode picture, 2 points were placed on the bases of the columns and 2 on the round decorative elements on the wall above them. Notice that the wall is sharp but the background is blurry.

How does multi-point focusing work? With one focus square, the app translates each video frame to keep the features in that square aligned (and therefore sharp). With 2 squares it uses a similarity transform to keep the features in both squares aligned. With 3 squares it uses an affine transformation, and with 4 squares it uses a homography. Don't know what these things are? Start with Wikipedia.

A bellows-style view camera or tilt-shift lens lets you make real-world scenes look like miniature models. Can you do this with SynthCam? Yes and no.

In miniature-model photographs the camera is usually placed high, aimed downward at an angle, and the lens is tilted so that the focal plane passes through the ground at an oblique angle. To obtain this effect with SynthCam, you would need to find features on this oblique, ground-penetrating plane. Scenes usually contain no such features.

That said, you can approximate the effect by placing 2 or 3 focus squares along a line through the scene, then rotate the phone slowly and continuously around its lens while making a recording. Look at the example at left, shot from Hoover Tower at Stanford.

Technically speaking, this is a "photographically produced fake tilt-shift image", as opposed to one produced using digital convolution, for example by applying a gradient blur in Photoshop or another program to an ordinary photograph. But it looks cool.

Page updated

Google Sites

Report abuse