Embedded Files
Investigating PointNet++ for Point Cloud Generative Adversarial Networks
Investigating PointNet++ for Point Cloud Generative Adversarial Networks
February - May 2022
My final course project for CS 674: Intelligent Visual Computing. In this project, I developed and trained a Wasserstein Generative Adversarial Network with Gradient Penalty using Python and PyTorch.
I trained models on point clouds sampled from 3D models belonging to the airplane and chair categories of the ShapeNetCore dataset. Using these point clouds as data, I trained a GAN (running on my desktop GTX 1060 6Gb GPU) consisting of a generator, which creates original point clouds, and a discriminator, which aims to learn which point clouds belong to a certain class of shape (for example, a chair) and which are not of that class (such as random noise). The generator and discriminator both learn at the same time, and eventually the generator is able to generate original point clouds belonging to that class of object.
My goal was to determine if using a different type of algorithm (PointNet++) for the discriminator would allow the model to more accurately determine whether a point cloud belonged to a given class or not, and hopefully help the generator achieve better results. However, I found that PointNet++ causes the discriminator to be overpowered, and this prevents the generator from "learning" to produce better point clouds. Since this approach did not show to improve 3D GANs, I didn't seek to publish my report, but I did receive an A for my project.
This graph shows the loss at each epoch of the experiment; both the generator and discriminator loss decreased when using PointNet as the discriminator, showing that both the generator and discriminator learned. This part of the experiment replicated prior research in 3D machine learning.
The graph on the left was produced using standard PointNet for the discriminator, while the graph on the right was produced using PointNet++ for the discriminator. This visualizes that , when PointNet is used as a discriminator, the generator is able to learn along with the discriminator to produce point clouds that can "deceive" the discriminator. This means that the generator is able to learn along with the discriminator in this GAN. However, when PointNet++ is used as a discriminator, the discriminator learns to quickly determine the difference between models of a given class (for example, chairs) and random noise. In this case, the generator is not able to keep pace and is unable to learn in a GAN environment. These results reflect those found in prior experiments in the field. Please see GitHub for references.
Google Sites
Report abuse