Semantics in localization

Semantics, i.e., object class, is considered to be important information for the localization. Many researchers have tried to use the semantics for improving localization robustness and accuracy. However, improving the localization performance by using the semantics is not easy because uncertainty of object recognition must be considered. In the general localization model (bottom left), the semantics cannot be treated as a probabilistic variable. Hence, the uncertainty cannot be considered. In this work, I proposed a novel localization model (bottom right) that uses the semantics as an observable variable and is used to model a likelihood distribution. More practically, to model the likelihood distribution, Dirichlet distribution is used and hyperparameters of the distribution are determined using the object recognition result.

Proposed model

In the general model, an ego-vehicle pose, x, is treated as a latent variable, i.e., a target variable to estimate, and control input, u, sensor measurements, z, and map, m, are treated as observable variables, i.e., can be used for the estimate. In this model, we predict how the sensor measurements are obtained from the given pose and map. Hence, the semantics cannot be handled. In the proposed model, object classes, c, is explicitly introduced. In addition, I assumed that the object classes are obtained from a supervised learning-based method such as deep learning. Hence, hyperparameters of the learning model, θ, and training dataset {Z, S} are also introduced. In the proposed model, we predict how the object classes are obtained.

Robustness to inaccurate object recognition

I conducted simulation experiments. The bottom figure shows a result in inaccurate object recognition accuracy case (recognition accuracy is approximately 20 %). The left figure is the object recognition result. The remaining three figures are likelihood calculation results around the ground truth, (0, 0), using the likelihood field model (left), semantic likelihood field model (middle), and the proposed model (right).

The likelihood field model uses only geometric information. The semantic likelihood field model simply uses the semantics, namely it does not consider uncertainty of the object recognition. Hence, its estimation accuracy is quite bad. The proposed method uses the semantics; however, its accuracy is better more than the likelihood field model. These results show that the proposed method enables to appropriately use the semantics for improving the accuracy and robust localization can be performed even though the object recognition is noisy.

Publications

I first presented the localization method considering sensor observation classes in [1]. Additionally, I compared the localization method with other traditional methods and confirmed its robustness to environment changes in [2]. However, the method cannot treated object recognition results because the recognition result is not introduced in the model. Therefore, I extended the model and proposed the semantic localization described in this article in [3].

[1] Naoki Akai, Luis Yoichi Morales, and Hiroshi Murase. "Mobile robot localization considering class of sensor observations," In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3159-3166, 2018. (ResearchGate)

[2] Naoki Akai, Luis Yoichi Morales, Takatsugu Hirayama, and Hiroshi Murase. "Toward localization-based automated driving in highly dynamic environments: Comparison and discussion of observation models," In Proceedings of the IEEE International Conference on Intelligent Transportation Systems (ITSC), pp. 2215-2222, 2018. (ResearchGate)

[3] Naoki Akai, Takatsugu Hirayama, and Hiroshi Murase. "Semantic localization considering uncertainty of object recognition," IEEE Robotics and Automation Letters, vol. 5, no. 3, pp. 4384-4391, 2020. (ResearchGate)