Invisible Marker Tracking for AR

Abstract - We present a novel method for augmented reality that uses an invisible fiducial marker which is drawn with infrared (IR) fluorescent ink. The method has the advantages of the conventional marker-based methods and marker-less methods at the same time: The marker is not intrusive because the marker is transparent in the visible range. Moreover, the method is much more reliable than marker-less methods because it uses a fiducial marker.

The overview of our tracking system is shown in the figure.

1. Introduction
2. Concept
3. Method
4. Application
5. Supplemental video (≒800KB)
6. Publication

Introduction

Augmented Reality (AR) is a technology in which a user’s perception of the real world is enhanced with additional information generated by a computer. In a typical AR system, a user’s view of a real scene is augmented with virtual objects. Thus, accurate measurements of the camera pose relative to the real world are required for the proper registration of virtual objects. For this purpose, many vision-based methods have been proposed and their performance has been improved. The previous methods can be classified into two categories: marker-based and marker-less (or marker-free). Each of the approaches has its own advantages and disadvantages, respectively. Placing a physical marker in the workspace of the user is intrusive while the use of the fiducial marker increases robustness and reduces computation requirement. Using natural features instead of an intrusive physical marker requires user’s intervention or initial offline calibration. To trade the pros and cons, hybrid methods have been recently proposed. However, no method seems to completely resolve the problems.

Concept

The previous marker-based methods require a space for a marker to be attached as shown in Fig. 1(a). This is a significant drawback of the previous marker-based methods against being applied to commercial AR applications. Thus, we present a novel method for augmented reality that uses an invisible fiducial marker which is drawn with infrared (IR) fluorescent ink. The method has the advantages of the conventional marker-based methods and marker-less methods at the same time: The marker is not intrusive because the marker is transparent in the visible range as shown in the left-top image of Fig. 1(b). Moreover, the method is much more reliable than marker-less methods because it uses a fiducial marker. The previous marker-less methods are useful when there are lots of rigid and unchanging features in a scene. If this condition is not satisfied, their performance significantly degrades. In particular, if there should be no features to be tracked in a scene, they do not work any more. However, our method does not require the condition and works well in a scene with no features as shown in Fig. 2.

(a) Using a visible marker

(b) Using an invisible marker

Figure 1. Advantage of using invisible markers.

Figure 2. AR in a scene with no features. An invisible marker is in hiding on the plane.

Method

In the proposed method, it is not required that the marker should be on the simple or solid surface. Even if the markers are on the cluttered surface, they can be tracked using simple image processing algorithms as shown in Fig. 1(b). The image taken by the IR camera has different color or intensity from that by the visible camera but the difference is almost consistent in all the pixels of the images except the pixels occupied by the marker. Thus, if the image taken by the visible is subtracted from the image taken by the IR camera, the pixels occupied by the marker become noticeable as shown in the two left-bottom images of Fig. 1(b). The two images are obtained from applying a simple intensity-based cropping algorithm to the red- and green- channel of the subtraction image. As shown in the rightmost image of Fig. 1(b), no marker can be observable but virtual kettle appears rigidly anchored in the scene.

Two cameras are used: an IR camera for tracking an invisible marker and a visible camera for capturing a scene. The cameras are equipped with an IR filter[1] and a color correction filter, separately. The two cameras are positioned in each side of half-mirror so that their optical centers coincide with each other. Camera calibration techniques can be also used to make the optical centers of the cameras coincident. In case of AR with the cluttered background, however, the calibration error significantly lowers the tracking performance because the result of the subtraction between the IR image and the visible image might not be useful for detecting the marker any more. We track the invisible markers using the IR camera and visualize AR in the view of the visible camera. From a view-point of the visible camera, the system works as an invisible marker based augmented reality system.

[1] No more equipment is required to detect the invisible IR marker.

Application

As an application of the proposed system, the invisible marker can act as a Vision-Based Identity and Geometry (VBIG) tag, which can significantly extend the functionality of RFID. The invisible tag is the same as RFID in that it is not perceivable while more powerful in that the tag information can be presented to the user by direct projection using a mobile projector or by visualizing AR on the screen of mobile PDA.

Publication

1. Hanhoon Park and Jong-Il Park, "Invisible Marker Tracking for AR," Proceedings of The Third IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR) 2004, pp. 272-273, Arlington, VA, USA, Nov. 2004 [paper link]
2. Hanhoon Park and Jong-Il Park, "Invisible Marker Based Augmented Reality System," Proceedings of Visual Communications and Image Processing (VCIP) 2005, vol. 5960, pp. 501-508, Beijing, China, July 2005
   1. Hanhoon Park and Jong-Il Park, "Invisible Marker Based Augmented Reality," International Journal of Human-Computer Interaction, vol. 26, no. 8, pp. xx-xx, Aug. 2010