VisArt

"Everything you can imagine is real." —— Pablo Picasso

"I dream of painting and then I paint my dream." —— Vincent van Gogh

This page is for introduction, progress update, and resource collection of the VisArt project. It was inspired by my wife, Xiaoyue Hu, who is a studio artist.

With the explosion of deep learning applications, people get used to hearing that "deep learning is killing every problem in machine learning".

Is there anything AI can't do? Still too many for now. (Yoshua Begio's Quora answer) I believe art belongs to one of the last frontiers. Differ from learning objective fact such as object recognition, the appreciation and composition of artworks are more subjective with relatively latent invariant features since even human experts can't agree with each other. Also, understanding art is a type of recreation, which is the first step of creation and innovation in art. It may lead to complex generative models for emotion and intuition. Our AI systems had been trained well enough to deal with the objective facts (even better than human), but lack of these "feeling" stuff.

There already exist a few related works that use deep learning technique for artwork creation.

Google Deep Dream:

http://googleresearch.blogspot.com/2015/06/inceptionism-going-deeper-into-neural.html

http://deepdreamgenerator.com/

Inception Net, a fancy algorithm that visualizes the image interpretation of each layer in a deep neural network and now becomes a popular and useful tool for artists to create new works. It would be interesting if the model is trained on any digital art gallery.

Neural style:

arxiv.org/abs/1508.06576

github.com/jcjohnson/neural-style

An algorithm that extracts and apply art style to photos with CNN.

Test Image (Cathedral of Learning at Pitt):

Style Images:

(1. The Starry Night - Vincent van Gogh)

(2. Women with a Hat - Henri Matisse)

(3. Seated Nude - Pablo Picasso)

(4. My Window in Peggy Guggenheim Palazzo - Brion Gysin)

Implementation Results:

The last two are fantastic! This technique also provides a new perspective for people to appreciate art style. If you feel weird about some post-modern art style, transferring it to other sense may be an excellent way to understand them better.

I start my exploration from a supervised learning task, artistic style recognition. The mission is more difficult than expected. In this work, I traditional features like color histogram, HoG, SIFT, GIST, also combined with features extracted from ImageNet pre-trained convolutional neural network (Caffe).

Here is a report

Vincent van Gogh's Starry Night and its SIFT bag of words features.

Update of Machine for Learning

A TITAN X Graphic Card on a Dell workstation.

Thanks to the support from my adviser and NSF (System Configuration)

Another PC DIY project: Red Fang.

powered by GTX 1070