Art-style Classifiers
Joshua Ning, Jason Ning, Jichi Wang, Diwen Zhu
OverView
Our team decided to take on the challenge of classifying 10 distinct art styles from the ArtBench-10 dataset with support vector machine and convolutional neural networks. We will compare the two machine-learning algorithms in accuracy and training time.
Additionally, we will be experimenting with 2 style transfer techniques, Neural Style Transfer and Generative Adversarial Networks. We will also compare the two machine-learning algorithms in the achieved results and training time.
Dataset: ArtBench-10
60000 images from 10 distinctive artistic styles
Image size: 256*256
5000 training images, and 1000 testing images per style.
Categories:
Art_nouveau
Baroque
Expressionism
Impressionism
Post-Impressionism
Realism
Renaissance
Romanticism
Surrealism
Ukiyo-e
Classification Projects
Style Transfer Projects
What We learned
We learned to apply digital signal processing techniques to solve problems like image classification and image style transfer. We gained experience with machine learning algorithms (SVM and CNN) and using libraries such as PyTorch, TensorFlow, and opencv. We got more comfortable with using GitHub as version control and documenting our results in the markdown language.
DSP Tools Used:
In class:
Gram Matrix
Convolution
Moving Average Filter
Applying 2dfft to images (although not adopted in the end)
Out of Class:
GAN (Generative Adversarial Networks)
Other Machine learning algorithms/concepts, such as Support Vector Machine and Convolutional Neural Network.
Conclusion and NEXT STEPS
For image classification of the 10 art styles of the ArtBench-10 dataset, we were able to achieve accuracies of 48.36% and 53.9% for SVM and CNN respectively. This result is better than what we initially anticipated, and we have a few ideas on how we can improve our classification system.
Using probability and the confusion matrix obtained to chain the Support Network Machine and Convolutional Neural Network together, making them "vote" on the most probable art style based on the strengths and weaknesses of each model.
Using more data to train the CNN, and extract more features from images for the SVM. Machine learning needs a lot of data and computing power to achieve the best results. Training for longer with more data, and a better CNN model have the potential to drastically increase the accuracy.
For style transfer, we can also train models with more image sets to get better results. However, our current approach is limited by the algorithm, which typically only detects the overall style of the image given. We can incorporate features like facial recognition to improve the results for details like a person's face. This approach could increase the stability of the generated image, making it more realistic and visually appealing.
For Support Vector Classifier, we can explore more abstract features in the future. Those features are not so obvious to humans, but they might mean something important to machines. We may also explore texture features, such as Gabor filters or Local Binary Patterns. Also, there is a new Segment Anything developed by Meta, so we can use it to identify objects inside the images. Based on the information provided by those objects, we can classify the images. This approach is close to how a human classifies an image. We know that some art-styles have special elements, so if certain elements appear in an image, then they must belong to a particular style.