RealityCheck
A Tool to Evaluate Spatial Inconsistency in Augmented Reality
Authors...
RealityCheck
In augmented reality (AR), virtual objects can drift away from their original locations, significantly impairing a user’s experience. Typically, a virtual object’s drift is approximated by the device localization drift, which is measured using specialized hardware such as3D scanners or laser-based positioning systems. However, with AR rapidly becoming more popular, there is a need for a lightweight, software-based approach to evaluate the drift of virtual objects that is easy for researchers and developers to use, without requiring specialized hardware or extensive environment setup. Towards this, this paper presents RealityCheck, an open-source AR evaluation tool that reports the drift of AR virtual objects in the world coordinate system, requiring only paper printouts and minimal modifications to the AR app. RealityCheck is designed to measure the drift of a virtual object across time of a single user, as well as the positioning differences of the same virtual objects as seen by multiple users. Our prototype is implemented on an Android smartphone running the ARCore platform, and evaluated in indoor and outdoor scenarios under a variety of user mobility patterns. We compared the results of RealityCheck with the ground truth position of the virtual object, and showed that RealityCheck matches the ground truth within 1.5 cm on average.
Note
This Application is intended to run on PC on video captured from external devices like mobile phones.
The test data was captured on a Samsung Galaxy s20 using ARCore.
Download: Here
Requirements
Python 3.7.0+
OpenCV 3.0+
Numpy 1.15.4+
Matplotlib 3.0.0+
Tested on Windows 10 and macOS Catalina 10.15.5
Video
RealityCheck will run on all images in the provided directory in sequence in directory order (see python os.listdir() for details)
Videos must be divided into individual frames for RealityCheck to run.
No other files should be in the directory.
Images may be rotated at runtime (useful for taking videos on devices that are usually rotated like mobile phones)
Annotations
The format for the Hand-Annotations is as follows:
OFFSET: <x-offset> , <y-offset>
GRIDSIZE: <grid-square size>
<frame number>: <x squares>, <y squares>
.
.
.
All in floating point(meters and then square units) values.
RealityCheck assumes that the <Video Directory Name> and annotations file (if present) share the same name with the only difference being the .txt extension on the annotations file
A test Annotations file has been provided for reference.
Command Line
The format for the command to run Reality Check is:
python RealityCheck.py <Video Directory Name> <Config file name (no extension)> <True\False (use Annotations)> <True\False (Rotate images 90 degrees first)>
If the last 2 args are dropped they default to False.
The example (with annotations) provided can be run with the following command:
python RealityCheck.py TestVideo TestConfig True True
Reality check will write all data to <Video Directory Name>Output.txt before terminating correctly
Configuration
The format for the Config File is as follows:
CAMERA: <focal x>,<skew>,<principal x>,<0.0>,<focal y>,<principal y>,<0.0>,<0.0>,<1.0> (in float32)
REALDICT: <ArUco Dictionary Type> (integer)
REALBOARDARRANGEMENT: <x markers>, <y markers> (integers)
REALMARKERSIZE: <size of Marker on board> (float, Meters)
REALBOARDSPACING: <size of gap between Markers> (float, Meters)
VIRTUALDICT: <ArUco Dictionary Type> (integer)
VIRTUALBOARDARRANGEMENT: <x markers>, <y markers> (integers)
VIRTUALMARKERSIZE: <size of Marker on board> (float, Meters)
VIRTUALBOARDSPACING: <size of gap between Markers> (float, Meters)
For the ArUco dictionary types, use the corresponding integer for this enumeration:
PREDEFINED_DICTIONARY_NAME {
DICT_4X4_50 = 0, #0
DICT_4X4_100, #1
DICT_4X4_250, #2
DICT_4X4_1000, #3
DICT_5X5_50, #4
DICT_5X5_100, #5
DICT_5X5_250, #6
DICT_5X5_1000, #7
DICT_6X6_50, #8
DICT_6X6_100, #9
DICT_6X6_250, #10
DICT_6X6_1000, #11
DICT_7X7_50, #12
DICT_7X7_100, #13
DICT_7X7_250, #14
DICT_7X7_1000, #15
DICT_ARUCO_ORIGINAL, #16
DICT_APRILTAG_16h5, #17
DICT_APRILTAG_25h9, #18
DICT_APRILTAG_36h10, #19
DICT_APRILTAG_36h11 #20
}
A test Configuration file has been provided for reference.
It should be noted that RealityCheck only uses ArUco markers now but could be extended to other types.