Research

Depth Estimation for Indoor Panoramas through Merging Multiple Perspective Monocular Depth Predictions

WACV Nov 2022  

We propose a method to estimate depths for indoor panoramas by merging multiple perspective depth maps produced by modern monocular depth estimation methods such as LeReS. The challenge is that the perspective depth maps, each covering a different subset of the panorama, tend to have different scales and shifts of the predicted depth values. Simply stitching them together led to inconsistent depth values for the whole panorama with visible seams. To address the challenge, we propose a novel approach to solve a single depth map for the whole panorama with information taken from each perspective depth estimations and a common panoramic depth map served as the reference for merging. 

Distortion Reduction for Off-Center Perspective Projection of Panoramas 

Best Short Paper Award    NICOGRAPH Nov 2021  

In this paper, we propose modifications to the equirectangular-to-perspective (E2P) projection that significantly reduce distortions when the camera position is away from the origin. This enables users to not only "look around" but also "walk around" virtually in a single panorama with more convincing renderings. We compare with other techniques that aim to augment panoramas with 3D information, including 1) panoramas with depth information and 2) panoramas augmented with room layouts, and show that our approach provides more visually convincing results.

Photoacoustic Image Classification and Segmentation of Breast Cancer: A Feasibility Study 

Journal   2018 December  IEEE Access    [paper] 

We used a pre-processing algorithm to enhance the quality and uniformity of input breast cancer images and a transfer learning method to achieve better classification performance. Besides, by comparing the area under the curve, sensitivity, and specificity of support vector machine with AlexNet and GoogLeNet, it can be concluded that the combination of deep learning and photoacoustic imaging has the potential to achieve important impact on clinical diagnostics. Finally, according to the breast imaging reporting and data-system levels, we divided breast cancer images into six grades and designed a segmentation software for identifying the six grades of breast cancer. Then, we tested based on MAMMOGRAPHYC IMAGES DATABASE FROM LAPIMO EESC/USP to verify the accuracy of our segmentation method, which showed a satisfactory result. 

Action detection based on tracklets with the two-stream CNN  

Springer  March 2017   [paper]   

We used a pre-processing algorithm to enhance the quality and uniformity of input breast cancer images and a transfer learning method to achieve better classification performance. Besides, by comparing the area under the curve, sensitivity, and specificity of support vector machine with AlexNet and GoogLeNet, it can be concluded that the combination of deep learning and photoacoustic imaging has the potential to achieve important impact on clinical diagnostics. Finally, according to the breast imaging reporting and data-system levels, we divided breast cancer images into six grades and designed a segmentation software for identifying the six grades of breast cancer. Then, we tested based on MAMMOGRAPHYC IMAGES DATABASE FROM LAPIMO EESC/USP to verify the accuracy of our segmentation method, which showed a satisfactory result. 

Game AI Generative Content

Speed up artists' work efficiency

• Embed Meta Segment Anything in SD

• Create a stable Lora and Dreambooth model for control

• Encapsulate the Stable Diffusion plug-in to support multi-person queuing and automatic expansion

• Embed SD in Unreal Engine and verify its robustness

• Stable AI Animation videos for Posters and  Advertisements

• Custom batch download online any pictures and use Controlnet  for style transfer

Optimized Unreal Engine 5 Mesh Simplification Methods

Mesh Simplification Plugin for PC and Mobile usages. [Patent]

• Consider Visual appearance and physics functions and Nanite

• Consider various PC and different mobile limits

• Consider Different objects' screen sizes on different platforms

• Consider Max Triangles and Min Triangles for all assets

• Using recursive times and recursive error to adaptive adjust each asset 

UE Chaos scene queries and rigid body engine

Deal with all physics problems and bugs in UE

• Cloth simulation and vehicle running in real-time

• RAII memory management 

• Template metaprogramming

• Compiler optimization from an assembly perspective 

• Multithreaded programming 

• Onnx Model with Machine Learning Deformer Graph