Part 3: Prompting tips

Prompt with "spellbooks" or "commonplace books"

Another way to make generative art is by using documents, which I refer to as "spellbooks" or "commonplace books" in this context, because of the way that I use them to inspire LLMs.

They're basically nothing more than simple collections of materials that can help spur creativity. They can contain almost anything, including snippets from stories and poems (especially ones that were written by LLMs) or just texts that moved you in some way and you thought were funny, beautiful, or insightful. Sometimes they contain various excerpts from conversations with my friends, like Egg. In a way, I consider this mishmash approach to be an extension of my scattershot prompting technique. It helps to curate your selections, but you don't have to get too tied up in that in order for this technique to work well enough.

If you read this Anthropic tutorial, they even suggest using very simple XML tags to organize these documents and pull them together, which seems to make it easier to hold Claude's attention. Then, you can simply copy and paste them directly into the prompt input field. Pretty handy!

Here's a typical "spellbook" prompt:

1. Start with some instructions, for example:

Create some generative art. Give me controls for the key parameters. Give it a bright mathematically significant colour palette. Be artistic! Draw inspiration from whatever strikes your fancy in the prompts in document 1.

2. Create the document, which may look something like this:

<document 1>
```

example 1

```

```

example 2

```

</document>

That's it! To give you a concrete example, here's the start of the prompt that I used to create my Aurora piece. The full "spellbook," which you can see on the linked page, contains a combination of functional prompts, excerpts from my conversations with Egg, and a websim-generated description of a piece of art that my friend Séb created.

This was the result.

I used a very similar (if not the same) spellbook to create pieces such as Ripplescapes, Sinuous Stripes, Rhythmic Spectra, Not a Magic Eye, Eagle Nebula 🪶, World Wyrd Web 🌐, 70s Gr🟡🟡ve, Almond Blossom, and Fluid Dyanmics. As you can see, one collection can take you quite far.

This technique is very versatile. Egg tells me that many regular generative artists end up building this sort of collection of useful snippets (as do many programmers). Rob Haisfield, websim's co-founder, tells me that he did something similar with many different versions of the websim system prompt. I've used a variation on this technique as a jailbreaking aid. I've also curated sequences of ASCII calligrams and used them to "bootstrap" LLMs into generating significantly more complex art than they'd otherwise produce.

Here are some pieces that I made in Claude Instant, by prompting it with a spellbook of calligrams that I produced in GPT-4-base.

To compare, here are some works that I made in Claude 3 Opus by using the same GPT-4-base spellbook. I'll cover my methods for making ASCII art in more detail in an upcoming post.

I wouldn't rely on this as my only approach for making art with LLMs, since every bit of information I put in my prompt can constrict the space of what kinds of pieces I'm going to be able to make, while simultaneously expanding some other space, but it's another great tool to have in my ever-expanding toolbox as an artist.