Van Gogh Landscapes

Vincent Willem van Gogh (1853-1890), a towering figure of European Post-impressionism and the creator of some of the most recognizable, iconic works in the western canon, did not work as an artist until the last decade of his short life.  Following desultory careers as an art dealer and then a Protestant missionary, he began painting in 1881.  From then until his death in 1890 he painted about 2,100 landscapes, still lifes, portraits, and self-portraits on canvas and panel.  Although he was never commercially successful during his lifetime, his daring, expressive brushstrokes and dramatic juxtapositions of color gained the attention of his more renowned contemporaries, influencing their work and that of successors for decades.  Many painters among the Fauves and the Expressionists noted their admiration for the emotive character of van Gogh’s work, and used similar techniques to convey on canvas their intuitive response to their subject matter.

But sometimes admiration for van Gogh’s work elicited more than homage.  John Rewald, the distinguished historian of Impressionism and Post-Impressionism, remarked that van Gogh may well have been forged “more frequently than any other modern master.”[1]  Perhaps the most scandalous instance of fraud occurred in 1928, when the Berlin art dealer Otto Wacker exhibited a series of previously unknown van Gogh works.  Wacker claimed to represent a mysterious Russian aristocrat on the run from the Bolsheviks, forced to sell his precious collection to save his family.  Although the fakes fooled Baart de la Faille, author of the first catalogue raisonné of van Gogh’s works, others questioned the works’ authenticity as soon as the exhibit opened.  Wacker was prosecuted and served 19 months in prison.

Debate over the authenticity of works attributed to van Gogh did not end with the Wacker affair.  Scholars cite numerous reasons why the artist has been such tempting fodder for forgers.  Foremost is, of course, money. Although van Gogh sold few paintings during his lifetime, his work was in high demand within just decades of his death, and prices for canvases and drawings by the artist climbed steadily for the next century.  Today, prices fetched for his work regularly break auction records.  Because van Gogh sent most of his paintings to his brother Theo during his lifetime, there are no dealers’ records from that period.  

Although van Gogh wrote about specific works in his frequent correspondence with Theo and others, his letters provide forgers with descriptions that can be used to create new “van Goghs,” especially when authenticated works matching the artist’s written descriptions are not otherwise known.  Van Gogh’s oft-noted practices of giving away paintings to friends and exchanging work with other artists can provide cover for a murky early provenance.  And van Gogh didn’t always sign his work, so the absence of a signature might not raise the suspicions of a dealer or collector.

Academic scholars, professional connoisseurs, and Internet cranks have questioned various works, some quite famous, on stylistic or provenance grounds.  Van Gogh’s style and working methods were far from static, however, complicating judgments based on perceived technique or composition.  And van Gogh’s prodigious output and chaotic habits preclude any definitive catalogue raisonné.

Newly discovered works by van Gogh emerge from time to time — rarely but with regularity.  Sometimes such works can be excluded based on scientific analysis, but even here the ready availability of period materials and canvases can limit the certainty of conclusions.  The van Gogh Museum in Amsterdam receives, and usually dismisses, a regular stream of inquiries.  The precise boundaries of van Gogh’s oeuvre, it seems, will remain indistinct in a starry night of contradictory evidence and doubt.

The A-EyeTM for van Gogh

The dataset we employed to train The A-Eye for van Gogh consisted of high-resolution images of 77 landscape paintings by van Gogh, selected to be representative of his evolving landscape painting styles, and 75 landscape paintings by other artists, intended to span a range of visual similarity to the van Gogh landscapes — from very close to evocative but readily distinguishable.  Our objective was to train The A-Eye to make fine distinctions with good generalization properties and without overfitting to the training set.

The “other artists” whose work is included in the dataset includes older artists whom van Gogh admired and learned from (Charles-François Daubigny, Claude Monet, and Camille Pissarro); contemporaries of van Gogh (Emile Bernard, Paul Cézanne, Paul Gauguin, Georges Seurat, and Paul Signac); and later artists — especially the Fauves and Expressionists (André Derain, Erich Heckel, Wassily Kandinsky, Ernst Ludwig Kirchner, Gustav Klimt, Henri Matisse, Gabriele Munter, Emil Nolde, Max Pechstein, Karl Schmidt-Rottluf, and Maurice de Vlaminck) — whose use of color and untraditional brushwork to express the artist’s emotional response to a subject seems indebted to van Gogh.  We also included several paintings by Cuno Amiet (1868-1961), a Swiss painter who explored and worked in numerous styles over the course of his long career.  While a member of the Pont-Aven School in the early 1890s, Amiet became an admirer of van Gogh’s work, and many of Amiet’s paintings from the next several decades seem to consciously incorporate many elements of the Dutch artist’s style.  In 1907, Amiet painted a widely known copy of van Gogh’s 1890 canvas Two Children (Private Collection).  The Van Gogh Museum acquired Amiet’s copy in 2001.

We trained and tested five different sets of images drawn from our collection of 152 van Gogh and non-van Gogh images.  Regrettably, we were unable to obtain for use in our dataset high-quality images of any of the Wacker forgeries.

These are samples of van Gogh and comparative images included in our training set:

Left: Vincent van Gogh, By the Seine, 1887 (oil on canvas; 49.4 x 65.3 cm.).  Van Gogh Museum s0055V1962, Amsterdam.  Right: Claude Monet, The Church at Vétheuil, 1881 (oil on canvas; 59.7 x 73.3 cm.).  Sold at auction by Christie’s in 2018.

Left: Vincent van Gogh, Path in the Woods, 1887 (oil on canvas; 45.3 x 37.7 cm.).  Van Gogh Museum s0080V1962, Amsterdam.  Right: Gustav Klimt, Avenue in the Park of Schloss Kammer, 1910s (oil on canvas; 110 x 100 cm.).  Österreichische Galerie Belvedere 2892, Vienna.

Left: Vincent van Gogh, Farmhouse in Provence, 1888 (oil on canvas; 46.1 x 60.9 cm.).  National Gallery of Art 1970.17.34, Washington, DC.  Right: Wassily Kandinsky, Planegg I -- Landscape near Munich, ca. 1901 (oil on canvas laid down on cardboard; 23.5 x 32.8 cm.).  Sold at auction by Daxer & Marschall, ca. 2018.

Left: Vincent van Gogh, Worker in a Field, 1889 (oil on canvas; 50.3 x 64.9 cm.).  Private Collection.  Right: Maurice de Vlaminck, The Regatta at Bougival, 1905 (oil on canvas; 60.5 x 73.5 cm.). Sold by Christie’s at auction in 2012.

What we Found

Our “Salient Slices” technique, described in detail here, first divides a source image into overlapping tiles of a given size.  These tiles are then sifted in accordance with a discriminator that identifies the tiles likely to contribute meaningfully to classification.  Our discriminator computes image entropy, which corresponds to the degree of visual diversity exhibited by a tile, and retains only those tiles whose image entropy equals or exceeds that of the entire source image.

The A-Eye analyzes each qualifying tile of a test image and assigns it a probability between zero and one:  values equal to or exceeding 0.5 correspond to classification as van Gogh, while values below 0.5 correspond to a non-van Gogh classification.  We classify the image based on both the average probability across all tiles and “majority vote,” i.e., the majority tile classification determines the image classification.  The closer a classification probability is to one, the greater is the likelihood, according to the trained CNN model, that the tile represents van Gogh’s work; conversely, classification probabilities approaching zero are strongly suggestive of a different artist’s work.

We trained and tested our CNN model at tile sizes ranging from 100 x 100 to 650 x 650 pixels.  Our 100 x 100 training set consisted of 10,700 tiles, while our 650 x 650 training set contained 398 tiles.  We obtained maximum classification accuracy using 100 x 100 and 150 x 150-pixel tiles.  Accuracy decreased steadily at larger tile sizes.  We interpret these results to suggest that van Gogh’s uniqueness — what sets his work apart, for classification purposes, from imitators and the artists he influenced — is most pronounced at the brushstroke level; while other artists might indistinguishably imitate his compositions and large-scale features, in other words, they do not (or cannot) duplicate his brushwork.  (Interestingly, we found precisely the opposite to be true for Rembrandt’s portraits.)

Following training, our 100 x 100-pixel CNN successfully classified 88% of the test images and our 150 x 150 pixel CNN properly classified 90%.  We found, however, that only about 5% of the tiles we generated passed the entropy criterion — about 1/3 the pass rate for Rembrandt.  Analyzing the entropy distribution of the van Gogh tiles revealed the reason:  they are strongly skewed toward high entropy (whereas the Rembrandt tile entropies have a bell-curve distribution).  Van Gogh’s style, in other words, makes him a “high-entropy artist.”  To counter this skew, we relaxed the entropy criterion by 1% (i.e., our discriminator accepts a tile if its entropy is at least 99% of the entropy of the overall image).  This small adjustment tripled the number of tiles accepted by the discriminator and reduced the classification error[2] by half.

We find that classification error can be reduced further by combining the probabilities associated with different tile sets, since the different CNN models “see” differently scaled features; the combination can mitigate errors associated with each tile set considered separately.  Rather than taking a straight average, we computed weights for the 100 x 100 and 150 x 150 probabilities that minimize the overall classification error.  We found that the optimized weightings reduce the already low error by another 50% and improve overall accuracy to 94%.[3]  Moreover, once weighted, most of the erroneous classifications fall within 10% of the decision boundary; misclassified paintings, in other words, are more often close calls than glaring mistakes.

We can explore the regions important for classification using “probability maps,” which color-code the probabilities assigned to the examined regions of an image at a granular level:  red corresponds to high-likelihood (≥ 0.65) classification as van Gogh, gold to moderate-likelihood (0.5 ≤ p < 0.65) classification as van Gogh, green to moderate-likelihood (0.5 > p > 0.35) classification as not van Gogh, and blue to high-likelihood (≤ 0.35) classification as not van Gogh.  Gray image regions correspond to tiles that did not pass the image-entropy selection criterion and were not examined.

Below are Farmhouse in a Wheatfield (1888, Van Gogh Museum, Amsterdam), which The A-Eye properly classified as a van Gogh, and the corresponding probability map. 

What we first see is how much of the painting did not pass the entropy criterion, even when relaxed.  The best-performing tiles — the ones that we used — are small and confine the CNN’s attention to detail rather than composition.  Close inspection reveals the examined regions to be visually “busier” than the unexamined areas, and The A-Eye classified nearly all of these as very likely to have been painted by van Gogh.

A less-decisively classified painting was Green Wheatfield with Cypress (1889, National Gallery, Prague), shown with its probability map below.  The A-Eye examined even less of this painting and prominently classified the house as not by van Gogh.  One possible interpretation of this result is that, while the house may have been painted by van Gogh, its brushwork is not characteristic of van Gogh — at least insofar as our training set is concerned.  That is, more non-van Gogh landscapes in our training set had passages with this brushstroke style than did van Gogh paintings.

As an additional validation exercise, we chose three van Gogh works that have been subject to attribution controversy to see whether the A-Eye’s classification would match the current scholarly consensus:

View of Auvers-sur-Oise (1890) is owned by the Museum of the Rhode Island School of Design (RISD) in Providence, RI.  The museum once expressed doubt over its authenticity and described it as “after van Gogh,” and in fact, the eminent van Gogh authority Jan Hulsker noted that “the number of paintings attributed to Van Gogh far exceeds the amount of work he could have done in the seventy days he stayed [in Auvers-sur-Oise] before his death.”[4]  But in 2008 the museum brought the painting to the Van Gogh Museum for scientific analysis, which re-established its authenticity.[5]  (Image courtesy of the RISD Museum, Providence, RI.)

A few years after the earlier-noted Bailey article questioned Garden of the Asylum (1889), curators at the Van Gogh Museum in Amsterdam, which owns the painting, published a detailed scholarly analysis[6] in support of authenticity.  In our view this comprehensive article represents a solid refutation of earlier doubts, and appears to have been the last word.

Finally, in the 1990s, the German art historian Roland Dorn and Zurich-based dealer Walter Feilchenfeldt questioned the authenticity of Three Pairs of Shoes (1886-87).[7]  The Fogg Art Museum in Cambridge, MA, which owns the painting, considers it genuine and cites provenance evidence and an extensive publication history.  Again, no serious attribution challenges appear to have been raised subsequently.

The A-Eye classified all three paintings in accord with the current consensus:

Note that Three Pairs of Shoes is classified as having been painted by van Gogh with considerably lower probability than the other two paintings.  In assigning meaning to this result, it is important to remember that we trained the A-Eye on van Gogh landscapes, and much of Three Pairs of Shoes differs considerably in style and content from our training set.  While it is perhaps unsurprising that van Gogh’s detail-level “signature” persists across genres and enough similarities exist at the brushstroke level to support a successful overall classification, cross-genre analyses like this should be considered risky business. 

[1] Quoted in M. Bailey, “At least 45 van Goghs may well be fakes,” The Art Newspaper, Jul.-Aug. 1997.

[2] We use the L1 measure for classification error, defined as the sum of the absolute difference between 0.5 and the probability associated with an incorrect classification (so that, for example, a van Gogh incorrectly classified with a probability of 0.4 would have an L1 error of 0.5 - 0.4 = 0.1).

[3] Specifically, there were two false negatives (i.e., true van Goghs not classified as such) and one false positive (a Nolde work erroneously classified as as van Gogh). 

[4] Quoted in Bailey,  supra

[5] L. van Tilborgh and O. van Maanen, "Van Gogh's View of Auvers sur Oise Revisited," Manual, Spring 2015, p. 22.

[6] E. Hendriks and L. van Tilborgh, “Van Gogh’s ‘Garden of the Asylum’:  genuine or fake?,” The Burlington Magazine, 143:1176 (Mar. 2001), pp. 145-156.

[7] M. Granade, “Van Gogh Painting May Be Forgery,” Harvard Crimson, Aug. 8, 1997.