The traditional way of aneurysm detection is based on 2-D convolution neural network using a series of 2-D images [4-8]. However, the detection of an aneurysm heavily relies on its 3D structures. In this study, the 3D models of blood vessels are adopted for better representation of its 3D structural features. The 3D modeling of an object includes Polygonal mesh, Voxel(Volumetric pixel), Point Cloud. There are several neural networks tailored for 3D datasets [9-12].

Polygonal mesh is a collection of vertices, edges, and faces which define the shape of the polyhedral object [13].

Voxel(Volumetric pixel) only represents a value for a regular grid in 3-D spaces. For 2-D images, pixels do not typically have their position which is explicitly encoded with their value. its' position in the data structure always makes up a single volumetric image by rendering system [14].

Point Cloud is a set of data points in space which represent the 3-D shape of an object. Each point has its set of X, Y, and Z coordinates. which is generally produced by 3-D scanners or photogrammetry software, which measures many points on the external surfaces of objects around them. As the output of 3-D scanning processes, point clouds are used for computer-aided design models or for multiple visualizations, animation, and rendering applications [15].

Training set up

The first version of the training pipeline.

1. Data Preparation ( I will use the Generated file, 1694 vessel samples, and 215 aneurysm samples)

Sampling: 2D neural networks requires the training images have the same dimensions. This is also the same for 3D PointNet. But the "dimension" here refers to the number of points in an object, not the size. The 3D model of blood vessel are different in two aspects, first, the volume of blood vessel; second, the density of dots/points. To make the samples have the same points, random sampling was conducted to get the same number of points from the point cloud model. To make the sampling reflects the density of points in different regions of blood vessels, the sampling is based on the density of triangle mesh. If there are more triangles (which means more complex structure), more points are extracted from that region.

2. Data Normalization

The data normalization is to normalize the coordinates of points: X, Y, Z to [-1, 1].

3. Data Augmentation

Data augmentation is commonly used for increasing sample size and reduce the overfitting of neural networks. The Data augmentation involves the rotation of the 3D object and random Noise. 3D rotation is to rotate the point cloud along the Z axis to a random degree. Adding random noise is to add random numbers (0, 0.02) to all the X, Y, Z coordinates of the point cloud. Therefore the total number of points in the point cloud will not be changed.

4. Neural Network Modification

The neural network architecture of PointNet was constructed using Pytorch. The parameters like input size and output size are adjusted according to the blood vessel point cloud.

5. Training

I did some basic hyperparameter tuning. The epoch number is now set to 15. the initial learning rate is set to 0.001. Now I am using Adam as the optimizer which will change the learning rate adaptively.

6. Validation

Data characterization

Data Sampling

Sample show with 250 points Sample show with 500 points

Data Augmentation

1. Rotation 2. Adding Noise 3. OverSampling

Hyperparameter tuning

• Loss function: NLLoss - Negative Log-Likelihood

• optimizer: torch.optim.Adam

• Learning rate: 0.001

• Epoch: 160~

Model Training Results

Point Cloud Transformer(PCT)

Transformers were first introduced by Goolge in 2017 for natural language processing [17]. Now, transformers have become a de-facto standard for NLP. Recently, transformers have been used as a deep learning model to bridge two domains - natural language processing and computer vision [18]. Vision transformer demonstrates excellent results compared to convolutional networks in image classification while requiring less computational resources.

Unlike the typical 2D images, the 3D point clouds can not take advantage of the traditional convolutional network because of its unstructured dataset with irregular domain and lack of order. The permutation invariant nature of transformers makes them more suitable for 3D point clouds. Guo et al. has developed Point Cloud Transformer (PCT), which introduced coordinate-based embedding, offset-attention and neighbor embedding. PCT achieves excellent performance on shape classification [19-20].

Although PCT has been applied in classification tasks of 3D point clouds, its application in 3D medical imaging has not been investigated. I have used PointNet to conduct aneurysm detections. Here I used a transformer, PCT for aneurysm detection and would like to see if the performance can be improved.

Overview of PCT model

PCT follows the fundamental architecture of a typical transformer, which includes three main modules: Input or (word embedding), Positional or(order) encoding, Self-attention. Some novel features have been built into PCT to better accommodate the characteristics of point cloud datasets, for example, the positional embedding is discarded in PCT since the point’s coordinates already contain this information. These novel features are embedded in the following modules, Coordinate-based input embedding module, Optimized offset attention module, and Neighbor embedding module.

The PCT has three pipelines - encoder, classification, and segmentation/normal estimation. I focused on the classification in my project. The PCT transforms the input points into a new dimensional feature space, which characterizes the semantic affinities between points as a basis for the various point cloud processing tasks. Then, the embedded features are fed into 4 stacked attention modules (offset attention) to learn a semantically rich and discriminative representative for each point, followed by several linear layers to generate the output features [26].