Project Overview

Motivation

The accessibility of 3D printing has made it possible for more people to create and print their own objects. However, designing and preparing 3D models for printing requires a good understanding of CAD software, which can be challenging and time-consuming for beginners. As a result, there is a need for more user-friendly tools that simplify the process of creating 3D models.

This is where photogrammetry comes in. Photogrammetry allows for the creation of 3D models from photographs, eliminating the need for complex CAD software. By using consumer-grade products like the Einscan-SE and the MakerBot® Digitizer™ Desktop 3D Scanner, anyone can turn physical objects into 3D digital design files without the need for specialized training or knowledge. However, these options are still quite expensive.

By exploring the use of photogrammetry as a tool for creating 3D models, this project aims to increase accessibility and ease of use of 3D printing technology. It could potentially lower the barrier to entry for 3D printing, making it more accessible to a wider audience.

Einscan-SE Desktop 3D Scanner: $1199

MakerBot® Digitizer™ Desktop 3D Scanner: $1400

Previous Research on Photogrammetry

Photogrammetry was originally used to create accurate maps and terrain models for the military. It is also used to create realistic 3D models and environments for film and video game production, as well as for virtual and augmented reality experiences. It is also used in forensic science to create 3D models of crime scenes and accident scenes, aiding investigation and analysis.

"Modeling the World from Internet Photo Collections" by Snavely et al. proposes a method for creating 3D models of large-scale scenes using photos available on the Internet. Photo Tourism has enabled the reconstruction of numerous well-known world sites like Notre Dame and the Great Wall.

Inspired by this research where photogrammetry techniques can be applied to create 3D models of large-scale scenes using photos from the Internet, we want to also see the applications on small scale objects with a simple, low-cost setup.

Photo connectivity graph of the Trevi Fountain

Project Goal

The goal of the project is to capture a sequence of 2D images on a turntable and process them to form a point cloud, which is a 3D representation of the original object. The 3D representation should accurately capture the shape of the object so that the object’s geometry can be used for fabrication using a 3D printer or another fabrication technique.

Methods

The process involves various steps including physical setup, sequence image capture, image downsizing, camera calibration, image masking, keypoint identification and matching, and point cloud construction. Different approaches are available for image masking and keypoint identification, and the choice of algorithm affects the quality and density of the resulting point cloud. These are the final design choices we made that gave us the best results.

Physical Setup

We created the physical photography setup with a white background, a turntable, a phone stand, and a smartphone with a camera.

3. Image Downsizing

The image is compressed by sampling every other point in the image to reduce the number of pixels in the height and the width by half.

5. Masking

Edge-based contour masking cleanly isolates the object without removing any white portions on the object.

7. Keypoint Matching

Keypoint matching where the keypoints are matched between every neighboring pair of images that are rotated by ten degrees

9. Point Cloud

The geometry of each keypoint matched to create a point in a specific location relative to the other keypoints.

2. Sequence Image Capture

We took a sequence of image captures to photograph the object from every angle. This is done by placing the object in the middle of the turntable and rotating the table 20 degrees between each of 18 images.

4. Camera Calibration

We calibrate the camera to record the distortion of the lens and the stretching of the image using OpenCV.

6. SIFT Keypoint Detection

The SIFT algorithm identifies more keypoints but worse matches which creates a denser point cloud. Keypoints are only accepted if they are within the object portion of the image mask.

8. Y-Coordinate Filtering

Y-coordinate filtering which uses the conditional equation y-coordinate difference < image height * threshold to accept keypoint only if the change in the y-coordinate of a keypoint across images is within an acceptable amount.

Results

Here are the results from the 6 objects we worked with:

Analysis of Results

Overall, we chose a broad range of objects of different sizes and shapes to better understand the holistic capabilities of our algorithm. The results analysis indicates that the algorithm performed well in some cases but had limitations in others. The cactus light and pincushion resulted in cylindrical point clouds that did not accurately capture the shapes of the objects. The Starbucks tin had a lot of keypoints on the words but not much on the cap of the tin, and the penguin salt shaker had a glare that made keypoint matching more difficult. The point cloud of the Poptart box captured the flat sides of the box but did not outline the borders of the box as expected. And the charger case produced the same, unfavorable results. It is important to note that the penguin salt shaker's point cloud captured a rough general shape of the penguin. Overall, the algorithm's performance was varied, and the object's features such as glare, details, and shape could affect the results.

Cactus Light

The point clouds generated for the cactus light did not accurately capture its cylindrical shape. However, it is worth noting that the back of the cactus, which was a bit dirty, had more keypoints and matches, resulting in a clustering effect in the point cloud.

Starbucks Tin

For the Starbucks tin, there were many keypoints on the text but almost none on the cap of the tin. The point cloud had curved plane cluster of points in the cylindrical shape from the top view of the point cloud, which was a result of each of the 18 turntable images.

Pincushion

To test a simpler object without any text, we tried the pincushion, which has a spherical shape. However, the resulting point cloud was oval and had a noisy cylindrical shape. This suggests that the shape of the object and its features can significantly impact the quality of the resulting point cloud.

Penguin

The penguin salt shaker presented a challenge due to glare, which made keypoint matching more difficult. As a result, keypoints tended to cluster towards the upper half of the penguin, as there were fewer details and the stomach of the penguin was almost entirely white. Despite these challenges, the resulting point cloud did capture a rough general shape of the penguin.

Poptart Box

For the Poptart box, the resulting point cloud captured the flat sides of the box, but the planes were not perpendicular to outline the border of the box as expected.

Charger Case

Lastly, the charger case resulted in a point cloud similar to that of the Poptart box, which is consistent since these objects essentially have the same shape.

Conclusion

The project did not produce results that can be used to construct the geometry of an object for fabrication. The project allowed the team to learn a lot about photogrammetry and different filtering techniques for keypoint matching.

The overall performance of the project can be improved further by improving the relative geometry calculations of the keypoint matches, masking of the object, and keypoint match filtering so that the point cloud better matches the dimensions of the object. After that, surface construction can be used to turn the point cloud into a three-dimensional surface.

The techniques learned in the project are applicable to object masking, keypoint detection, and keypoint masking. These are important methods in computer vision for image editing and reconstruction.

References

MakerBot, | by. (2019, May 7). Makerbot digitizer desktop 3D scanner: Order Today! MakerBot. https://www.makerbot.com/stories/news/makerbot-digitizer-desktop-3d-scanner-order-today/

Photogrammetry software: Photos to 3D scans. Autodesk. (2021, December 11). https://www.autodesk.com/solutions/photogrammetry-software#:~:text=Photogrammetry%20is%20the%20art%20and,2D%20or%203D%20digital%20models

Se powerful desktop 3D scanner. EinScan. (2023, February 7). https://www.einscan.com/einscan-se/

Snavely, N., Seitz, S. M., & Szeliski, R. (2008). Modeling the world from internet photo collections. International Journal of Computer Vision, 80(2), 189-210.

Harvey Mudd College | CS153 Computer Vision

Page updated

Report abuse