Dreamreader

Background

During an interesting brainstorming session about human qualities and other sources of inspiration, we associated our previous blog posts (about fear and sleep) with dreams. Given our common affinity with storytelling, the subjectivity and mystery of this topic appealed to us. Dreaming is an interesting quality to project on artificial creatures as it is considered to be uniquely human. Although there is some research on the function of dreams (to reflect on experiences or help generalise information), there is still little consensus about dreaming in the scientific community. This motivated us to explore the enigmatic phenomenon further. We wanted to uncover the secrets of dreaming by recreating the quality in an artificial creature, which allows us to make the dream experience tangible and conscious. Furthermore, we wanted to inspire the question: why do we dream? 

Technical description

The audience steps into our twilight installation where they meet a fluffy creature hooked up to our Dreamreader technology. A comfortable blanket lies on the ground and soft drapes hang from the ceiling. Our creature mostly consists of fluffy material and a camera. It glows softly, the organic flickering suggests breathing or movement, which makes the creature feel alive. By making the creature soft and organic, we subtly mirror the (human) audience and invite them to participate. The creature is wired to a laptop that manages the awake and sleep cycle of the creature with P5.js code.

When the creature is awake, it interprets its environment through the camera and the ML5.js Image Classifier library. This is the moment to interact with the creature. The audience can move in front of the camera and show the creature all kinds of objects in an interactive dialogue. Occasionally, the creature will detect the objects that the audience has showed it. Once it succesfully detects an object, the name of said object will appear on the screen. After five intervals of having successfully detected objects, the creature has seen enough for the day and becomes tired.

The creature first falls into a deep dreamless sleep. The audience can follow the creature into the dreamworld by lying down next to it on a soft cloud-like blanket. At this time, the list of detected objects is added to a dream prompt for the text-to-image generator DALL·E 2 by OpenAI. This process is based on research about dreaming in humans. The fact that artificial neural networks have a resemblance to the human mind made them an obvious tool to work with. We experimented with a variety of AI tools such as DeepDream, Craiyon and Poems About Things. We tested various keywords and art styles for the prompt. It was interesting to try to confuse the AI, to achieve a more abstract and surreal dream image similar to human dreams. Because a part of the prompt is consistently used for the image generation, it could be seen as the inner personality of the creature.

Through the DALL·E 2 API, the dream is generated. A projector dramatically reveals the result on a cloth draped from the ceiling, resembling a canopy bed. The audience looks up at the dream from the blanket and can guess what the dream is about. The dream will fade as the creature wakes up again and the cycle starts anew.

Reflection

The team collaboration was very effective and enjoyable. We set our goals at the beginning of the project and made an accommodating plan that we followed well. We met up regularly to work on the project together and shared our progress as well as encountered issues. 

We also planned feedback sessions every workshop class. The feedback at the beginning of the project encouraged us to think about different ways of interaction. Later, we made an effort to create a cohesive aesthetic based on the received feedback. We did this by mirroring the setting and software, for example in the look of our creature and the transition effects in the software.

During the exposition, the audience seemed enamoured by our Dreamreader. Often at first, the fluffy creature cached the attention. Visitors made an effort to understand how and what the creature detected. With fascination, they tried to show it certain objects to detect, to influence the creature. When the creature stopped detecting and responding, the audience (sometimes hesitantly) went to lie down on the blanket. They were surprised by the appearing dreams and tried to guess the mixed-up and abstracted elements they have shown when the creature was awake. The uniqueness of the dreams made them feel personal, participants liked to make photos of ‘their dream’. A few participants mentioned the resemblance between human cognitive information processing and that of the artificial creature. In general, much curiosity was awakened in the audience. It was satisfying to make the audience realise in the end, that just like they have influenced the creature’s dreams, this experience has influenced them and might come back in their sleep as a dream.