Embedded Files
Github here
We devised a novel methodology to detect objects in 3D by combining the Frustum-PointPillars architecture with PSP-Net (Pyramid Scene Parsing Network) segmentation models.
We devised a novel methodology to detect objects in 3D by combining the Frustum-PointPillars architecture with PSP-Net (Pyramid Scene Parsing Network) segmentation models.
After detecting objects using YOLOv7 (You Only Look Once), we utilized a segmentation model (PSP-Net) to derive the class masks for the detected objects.
After detecting objects using YOLOv7 (You Only Look Once), we utilized a segmentation model (PSP-Net) to derive the class masks for the detected objects.
This additional step facilitated more precise localization and delineation of object boundaries, leading to more accurate 3D bounding box generation.
This additional step facilitated more precise localization and delineation of object boundaries, leading to more accurate 3D bounding box generation.
Illustration of 2D detections that are taken from Ground Truth + Gaussian Mask of objects are extruded to corresponding 3D bounding frustum and masked point cloud.
Illustration of 2D detections that are taken from Ground Truth + Gaussian Mask of objects are extruded to corresponding 3D bounding frustum and masked point cloud.
Through rigorous experimentation and fine-tuning of our model, we achieved a 3% enhancement in performance on the challenging KITTI-hard dataset for cars.
Through rigorous experimentation and fine-tuning of our model, we achieved a 3% enhancement in performance on the challenging KITTI-hard dataset for cars.
This improvement paves the way for advancements in the accuracy of 3D object detection systems, particularly in scenarios with complex and cluttered environments.
This improvement paves the way for advancements in the accuracy of 3D object detection systems, particularly in scenarios with complex and cluttered environments.
Figure shows the bounding boxes generated by YOLOv7 and the ground truth provided as input to the model. Green boxes are the ground truth and blue boxes are the YOLOv7 predictions.
SegMask FP model output visualized in KITTI-hard dataset for cars that show 3D bounding boxes on 2D image and the bird's eye view (BEV).
We improved on Frustum-PointPillars which leverages both RGB and LiDAR data for 3D detection. We added a YOLOv7 object detector for generating a prior for the frustum proposal. We altered the naive Gaussian masking technique with a pre-trained PSPNet segmentation mask, which improved the car AP scores by 3%.
We improved on Frustum-PointPillars which leverages both RGB and LiDAR data for 3D detection. We added a YOLOv7 object detector for generating a prior for the frustum proposal. We altered the naive Gaussian masking technique with a pre-trained PSPNet segmentation mask, which improved the car AP scores by 3%.
The future scope of this work is to improve 2D detection in low-light conditions using data from RGB and LiDAR and we also noticed that to avoid outliers, it is essential to remove reflections in surfaces such as glasses and windows.
The future scope of this work is to improve 2D detection in low-light conditions using data from RGB and LiDAR and we also noticed that to avoid outliers, it is essential to remove reflections in surfaces such as glasses and windows.
Page updated
Google Sites
Report abuse