Art-style Classifiers

We learned to apply digital signal processing techniques to solve problems like image classification and image style transfer. We gained experience with machine learning algorithms (SVM and CNN) and using libraries such as PyTorch, TensorFlow, and opencv. We got more comfortable with using GitHub as version control and documenting our results in the markdown language. 

DSP Tools Used:

In class:

1. Gram Matrix

2. Convolution 

3. Moving Average Filter

4. Applying 2dfft to images (although not adopted in the end)

Out of Class:

1. GAN (Generative Adversarial Networks)

2. Other Machine learning algorithms/concepts, such as Support Vector Machine and Convolutional Neural Network. 

For image classification of the 10 art styles of the ArtBench-10 dataset, we were able to achieve accuracies of 48.36% and 53.9% for SVM and CNN respectively. This result is better than what we initially anticipated, and we have a few ideas on how we can improve our classification system.

1. Using probability and the confusion matrix obtained to chain the Support Network Machine and Convolutional Neural Network together, making them "vote" on the most probable art style based on the strengths and weaknesses of each model.

2. Using more data to train the CNN, and extract more features from images for the SVM. Machine learning needs a lot of data and computing power to achieve the best results. Training for longer with more data, and a better CNN model have the potential to drastically increase the accuracy.

For style transfer, we can also train models with more image sets to get better results. However, our current approach is limited by the algorithm, which typically only detects the overall style of the image given. We can incorporate features like facial recognition to improve the results for details like a person's face. This approach could increase the stability of the generated image, making it more realistic and visually appealing.

For Support Vector Classifier, we can explore more abstract features in the future. Those features are not so obvious to humans, but they might mean something important to machines. We may also explore texture features, such as Gabor filters or Local Binary Patterns. Also, there is a new Segment Anything developed by Meta, so we can use it to identify objects inside the images. Based on the information provided by those objects, we can classify the images. This approach is close to how a human classifies an image. We know that some art-styles have special elements, so if certain elements appear in an image, then they must belong to a particular style.