Tutorial objective

This tutorial explores how AI advancements can enrich the understanding, enjoyment, and accessibility of art, offering new and intriguing ways to engage with artistic works.

Background

In recent years, the digitization of artistic heritage has witnessed significant progress, offering a wealth of opportunities for computational solutions in art exploration and analysis. Digital archives, online collections, and high-resolution imaging have made artworks more accessible and available for in-depth examination. These digitized resources are valuable inputs for AI-based algorithms and techniques, enabling novel computational approaches to analyze, understand, and derive insights from this heritage. The primary audience for this tutorial includes researchers, practitioners, and enthusiasts in AI, digital humanities, and e-heritage.

Tutorial outline

Art & knowledge graphs

Knowledge graphs are powerful tools for organizing and encoding historical and contextual information related to artworks. Attendees can delve into the concept of knowledge graphs and explore ArtGraph, a specific knowledge graph developed by the tutorial team. ArtGraph is implemented using Neo4j, a NoSQL database, which provides a robust graph query language for information retrieval and knowledge discovery. With knowledge graphs, attendees can navigate and explore art-related information in a structured and interconnected manner. ArtGraph allows for the organization of diverse data, including artist biographies, movements, stylistic characteristics, and relationships between artworks and artists. By leveraging the capabilities of knowledge graphs, attendees can gain insights into how to harness the power of structured data to enhance art exploration, contextual understanding, and analysis in the digital humanities domain.

Artwork captioning

Artwork captioning is an essential task in art exploration and appreciation, aiming to generate descriptive textual descriptions that capture the essence of an artwork, its visual elements, and underlying concepts.  In this tutorial, attendees can delve into the emerging field of artwork captioning and explore how it can be achieved using large language models, such as ChatGPT. When applied to artworks, traditional image captioning models often face challenges due to the unique characteristics and complexities of artistic creations. Traditional captioning models may struggle to accurately describe artworks' intricate details, emotional nuances, and underlying meanings. However, large language models have the potential to go beyond the surface-level visual features and incorporate broader contextual knowledge, art history, and cultural references. Artwork captioning has significant potential to enrich art experiences, promote inclusivity, and facilitate cultural exchange. Furthermore, it can allow individuals with visual impairments or limited artistic knowledge to access and understand artworks more comprehensively.

Artwork inpainting

Artwork inpainting is a fascinating application of AI that involves restoring and completing missing or damaged parts within an artwork. It is a challenging task due to the fine-grained details, textures, and artistic styles present in artworks. In this tutorial, attendees can explore the application of recent advances in diffusion models, inspired by techniques such as DALL-E, for artwork inpainting. By leveraging these models, attendees can learn how to generate realistic and coherent content seamlessly integrating with the original artwork. Artwork inpainting does not involve simple image interpolation; it aims to capture the artistic intent and style, ensuring that the generated content aligns harmoniously with the existing elements of the artwork. Artwork inpainting has significant implications for the restoration of damaged artworks, the completion of unfinished pieces, and the preservation of artistic heritage. 

Resources

• Slides

• ArtGraph, an artistic knowledge graph developed by us (available on Zenodo)

• A GitHub repository containing the code of tutorials and demos related to the topics of the tutorial

Team