Room: Cascades
Call For Contributions
Privacy-Aware Human State Estimation for MR/AR Applications
This workshop focuses on Privacy-Aware Human State Estimation for various Mixed and Augmented Reality (MR/AR) applications, including human pose estimation, human eye gaze tracking, and human mesh reconstruction, which are crucial core technologies for MR/AR technologies. For dealing with human data, ensuring personal data protection-aware technology could highly contribute to the widespread adoption of these technologies in society. Therefore, under the premise that protecting privacy is essential, we would like to explore the theme of Privacy-Preserving Human State Estimation based on P3 (Pseudo Privacy Preserving) data. P3 is an original term made by the organizers.
What can be P3 data?
Here, P3 data refers to data that are "less likely to lead to individual identification," such as
silhouette images 👤
acoustic signals 🔊
event data 📷
transient images ✨
and wireless signals 📡
that do not contain semantic information, or measurement information in settings where users are not directly measured from the front.
In short, the P3 data include less personal information than "information that easily leads to individual identification," such as image information that includes the state of a person's face or clothing as texture, speech, and acoustic signals during actions where the user's voice tone, conversation content, and actions can be easily inferred.
This topic of research encompasses many important technical challenges, such as estimating human state from sparse measurement information, addressing noise commonly encountered in estimation methods based on sparse measurement information, challenges stemming from low spatial resolution when using modalities other than visible light, and methods for visualizing the estimated results.
Topics
Key topics of the workshop include, but are not limited to:
Human state (pose, gaze, mesh, etc.) estimation using P3 data (such as acoustic signals, wireless signals, event-based data)
Devices for capturing P3 data
Multimodal estimation
Scene estimation for P3-based human state estimation
Trustworthy methods for P3-based human state estimation (such as noise-aware estimation method)
Robust estimation methods
Data synthesis
Visualizations, MR/AR applications for P3 data-based human state estimation
Submitted papers will be peer-reviewed. Accepted papers will be published on IEEE Xplore as ISMAR2024 adjunct proceedings.
For your reference, here are some relevant work on this topic. Please note that these are just examples.
We welcome a wide range of submissions, not limited to these topics!
Examples of relevant papers
[1] Ryosuke Hori, Ryo Hachiuma, Mariko Isogawa, Dan Mikami, Hideo Saito, "Silhouette-based 3D Human Pose Estimation Using a Single Wrist-mounted 360° Camera", IEEE Access, vol. 10, pp. 54957-54968, 2022. [Open Access Journal] [Project Page]
[2] Yuto Shibata, Yutaka Kawashima, Mariko Isogawa, Go Irie, Akisato Kimura, Yoshimitsu Aoki. "Listening Human Behavior: 3D Human Pose Estimation with Acoustic Signals", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.13323-13332, 2023. [Project page]
[3] Christen Millerdurai, Hiroyasu Akada, Jian Wang, Diogo Luvizon, Christian Theobalt, Vladislav Golyanik, "EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.1186-1195, 2024. [Project page]
[4] Kotaro Amaya, Mariko Isogawa. "Adaptive and Robust mmWave-based 3D Human Mesh Estimation for Diverse Poses", IEEE International Conference on Image Processing (ICIP), pp.455-459, 2023. [Paper]
[5] Mariko Isogawa, Ye Yuan, Matthew O'Toole, and Kris Kitani, "Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7013-7022, 2020. [Project page]
[6] Mariko Isogawa, Dorian Chan, Ye Yuan, Kris Kitani, and Matthew O'Toole, "Efficient Non-Line-of-Sight Imaging from Transient Sinograms", 16th European Conference on Computer Vision (ECCV), pp. 193-208, 2020. [Project page]
Important Dates
Paper submission deadline: July 17th (23:59 AoE), 2024
Acceptance notification: August 12th, 2024
Camera-ready deadline: August 26th, 2024
Submission
Submitted papers will be peer-reviewed. Accepted papers will be published on IEEE Xplore as ISMAR 2024 adjunct Proceedings.
Page length: 2 - 8 pages
Template: All paper submissions should be formatted using the IEEE Computer Society VGTC format described here
Submission site: https://cmt3.research.microsoft.com/IWP3HSE2024
NOTE: At least one author must register for the conference and present the work in-person!!
Organizers
General Co-Chairs
Mariko Isogawa
Keio Univ.
Hideo Saito
Keio Univ.
Shohei Mori
Univ. of Stuttgart
Dieter Schmalstieg
Univ. of Stuttgart
Workshop Advisory Board
Invited Speaker
Ryosuke Hori (Keio Univ. / CMU)
Title: Privacy Preserving Human State Estimation
Abstract: This talk provides an overview of human state estimation and introduces techniques that allow for accurate human state estimation without compromising the privacy in everyday environments. The presentation will cover the structure of human state estimation as a field and explore how privacy can be preserved using pseudo privacy-preserving (P3) data. Four key projects will be introduced, each demonstrating how P3 data can be applied to achieve privacy preserving human state estimation. Through the use of silhouette images, event data, IMU data, and foot pressure sensor data, these projects showcase how human pose, gait, and other states can be accurately estimated while ensuring the privacy of individuals in shared spaces. These foundational techniques hold potential for applications in areas such as sports, healthcare, robotics, and mixed reality.
Program Committee Members
Peter Mohr-Ziak (TUGraz)
Maki Sugimoto (Keio University)
Daisuke Iwai (Osaka University)
Verena Biener (University of Stuttgart)
Ana Stanescu (TUGraz)
Yuki Uranishi (Osaka University)
Ryo Hachiuma (NVIDIA)
Ryosuke Hori (Keio University)
