Art Description Generation

Have you ever looked at a painting and wondered what is the story behind it?

We present a framework to bring art closer to people by generating comprehensive descriptions of fine-art paintings.

Art Descriptions

There are three main topics that are used to talk about art:

  • Content: what the artwork is about.

  • Form: how the work looks.

  • Context: in what circumstances was conceived.

Topic Annotations

We manually annotated the SemArt dataset:

  • 33,543 sentences

  • 17,249 images

  • 3 topics

Download the annotations from this repository.

Our Model

  • Multi-topic: it modules the generated sentences according to the three topics: content, form, context.

  • Knowledgeable: Enhances each description with external knowledge, such as artist, year, location, etc.

Check the code in this repository.





author = {Zechen Bai and Yuta Nakashima and Noa Garcia},

title = {Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation},

booktitle = {International Conference in Computer Vision},

year = {2021},