Embedded Files
Sunday June 16, 2019, Room 201B
Organized in conjunction with CVPR 2019 in Long Beach CA
Head Tracking and SLAM
Head Tracking and SLAM
- What is headpose tracking and what are the key challenges in getting headpose tracking to run on embedded platforms for AR applications?
- We describe the SLAM pipeline architecture with an emphasis on temporal processing (sensor fusion), and spatial processing (mapping).
- We will also present examples of challenging scenarios and discuss future work.
World Reconstruction
World Reconstruction
- What is world reconstruction and what are its primary uses: Occlusion, Physics, and Environment Reasoning (e.g. for content placement or navigation)
- What we do in World Reconstruction: Mesh, Planes, Raycast
- Our Volumetric reconstruction approach to world reconstruction
- Handling Dynamics (Fast add, remove, dynamic artifact removal)
- We will also present examples of challenging scenarios and discuss future work.
Content Persistence and Sharing
Content Persistence and Sharing
- High level description of the content persistence workflow.
- Demonstrate the computer vision and deep learning problems required to ensure that content is persisted across multiple sessions.
- What is content sharing? and the computer vision involved to make it a reality.
- Collaborative map building and sharing of sparse and dense maps.
Eye Tracking
Eye Tracking
- Uses of gaze tracking in Magic Leap One for user interaction / understanding and graphics.
- We will present a description of the eye tracking system focussing on two main aspects:
- Hybrid segmentation and model based gaze estimation
- Recovering 3D eye data and estimating 3D fixation point
- Finally, we will go over challenges and future directions
Hand Tracking
Hand Tracking
- The problem of Classification (identifying among a predefined “dictionary” of static hand poses and Regression (find keypoint locations in the presence of highly variable hand shape and pose, self-occlusion and environmental clutter).
- Challenges involved in training and implementing deep learning models on device.
- Low latency, compute, memory usage, and power consumption on (existing) embedded HW
- High throughput (support 45-60 FPS sensors without dropping frames)
- Data collection and labeling
- We will then go over our End-to-End Deep Learning solution and present our work on model optimization and results.
Scene Semantics
Scene Semantics
- What is Object recognition and scene understanding and what are its usecases.
- Challenges in building a real system: Hardware, Algorithm, Data.
- How do we build a scalable system that would produce multiple types of outputs that developers want: 2D bounding boxes, 3D bounding boxes, semantic meshes, semantic planes.
- We will cover the deep learning approaches we have considered and present early results and future work.
Page updated
Google Sites
Report abuse