Part 4: Things I've made

Collaboration with Egg Syntax

Prompts found here.

This tutorial wouldn't be complete without talking about how I've been collaborating with my friend Egg Syntax.

Egg and I met on the Slack server for the MATS research program, where we became quick friends through our mutual love of weird AI behaviour. Over the past few weeks, we've had an ongoing conversation about the generative art that I've been making. I share where I'm trying things and show my results, and Egg gives me feedback, suggests other interesting directions, puts my work in some historical context, and generally encourages me to keep messing around.

More recently, we've started taking that same easy flow and turning into a more direct artistic collaboration. I start by doing what I've been doing all along – prompting Claude to create generative art for me – and when I find a result that we both like, this triggers Egg's creativity. They’ll grab the code and start tweaking it in various directions, creating versions that they find even more exciting. When Egg finds something that we both like, I'll plug their code back into websim and continue prompting to make further variations. All of this happens organically, while we're exchanging DMs. It's great.

Egg told me that they imagine us shortening our feedback cycle until it becomes "like pair programming, only pair arting." It reminds me of how two musicians might collaborate on a track, where one starts off by drafting an initial outline and then the other adds their own perspective. They end up producing a song together that neither one of them could have penned independently. In this case, I get to combine my prompting skills with Egg's coding expertise and background as a generative artist, which not only elevates my work, but it also seems to open up new possibilities for Claude to explore. I never feel like I'm stuck in a dead end when I'm working this way. I also find it really helpful to hear Egg’s feedback on whether I should keep going in a certain direction, or if they feel like we’ve exhausted the realm of possibilities in a particular space, and they think that we should probably move on.

This form of collaboration is still evolving, but here's a sketch of the story so far:

It all started when I was messing around with the final "InfraZen System" generator in the fluid dynamics series that I described in the previous section. I was lazily prompting Claude to "create a new and completely different piece of generative art" over and over again. In previous months, when I was making ASCII calligrams with Claude, I discovered that a command like this one could sometimes serve as a launch point to explore the surrounding space of possibilities. I wondered if the same approach might work here in my generative art explorations.

When I'm using this approach, I often have to go through a couple of "meh" results before I hit on something that looks good, so I try not to give up so easily and keep the conversation going for a few turns, even when my outputs are underwhelming. Once I hit on a high quality output, I can "dig here" and elicit variations within the same patch of promising space, before eventually moving on; or, I can go back in time and try something else.

Interestingly, not all LLMs are capable of doing this kind of iterative exploration with successive prompts. When I made ASCII art with Bing, it would often "collapse" into the first result that it produced. That's to say, it would usually create the same image, or nearly the same image, whenever I prompted it to make new art (or even create variations on an existing piece) in subsequent turns. This is partially how I came to write such informationally dense "scattershot" prompts; I found that it was necessary to end my conversations with Bing after one turn, so I had to make my opening prompt count, since it was the only one I had.

In this case, the results weren't mind-blowing, even when I tried a similar prompt later on from a different launch point. My outputs looked pretty basic compared to what I now know that Claude is capable of creating. Still, something drew me to this one, which reminds me of a sea anemone whose tentacles are waving gently back and forth underwater. You can check it out on websim here.

Naturally, my next move was to prompt Claude to "make it look more like a sea anemone."

In response, Claude gave me a rotating mandala with really interesting motion and curly protrusions that it labelled as "tentacles" in the controls. You can see it for yourself here.

Boom. A hit. Just like that, we were off to the races.

Just for fun, I asked Claude to "give this more eyes," which produced some hilarious iterations, including this little guy (pictured below) and this one with adjustable eyes.

Egg liked my tentacle mandala so much (albeit without the eyes) that they decided to jump in and add their own code. This marked the beginning of our direct collaboration. Egg downloaded the code for my websim page and tweaked it to enrich the lines between the nodes of the mandala, redrew the strokes as outlines, and gave it an Arabic calligraphy feel overall.

These are Egg's stills of this version.

Egg sent me several versions in quick succession. When I played with a slightly earlier one, I was able to capture this mandala, which has a gilded Baroque look that reminds me of Versailles.

I was blown away that this experiment had gone so well. Egg explained to me some of what they did (and encouraged me to learn how to do it myself):

"This is the advantage of learning to code just enough that you can make tweaks like this.

I don't know much about JS canvas functions, I just looked them up,

eg https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/strokeStyle

https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/lineWidth

One change I made was to tweak the values for these two lines (I forget what they were originally) to make them different; that way it'll take them different amounts of time to cycle through, which means over time you'll get every possible combination (note that I made them relatively prime to maximize that time).

      const curveX = midX + Math.cos(angle + Math.PI / 2) * dist * 0.013;

      const curveY = midY + Math.sin(angle + Math.PI / 2) * dist * 0.011;

That's generally a highly useful trick. I am *all about* creating a wide range of variation by having different things out of phase with each other.

I don't know if you've heard Steve Reich's early work, but that's basically all it is -- taking multiple versions of a sample and putting them slightly out of phase with each other so that over time they cycle into every possible relationship: https://www.youtube.com/watch?v=vugqRAX7xQE"

For my part, I felt moved to prompt over Egg's work. This was my first time doing that. I asked Claude to turn Egg's code into a new websim page, which worked like a charm, and then, once I had that going, I asked Claude to "give it more of a rorschach inkblot effect," since I had so much success with those instructions the last time I used them, when I was building out the fluid dynamics series.

This time, Claude generated a new effect that I'd never seen before. It reminds me of the aesthetic of witches in the anime Madoka. You can check out that version here.

Meanwhile, Egg continued iterating on their own version with code and made the motion even more hypnotic.

Egg produced the above video based on this version and accompanied it with their own original music from back in the day.

They also sent me these still images from the same generator, along with the lyrics to The Charioteers' version of "Ezekiel Saw the Wheel," from this compilation of pre-WWII gospel:

'Ezekiel saw the wheel
Way in the middle of the air
A wheel within a wheel
Way up in the middle of the air'

Biblically correct angel, only it's a biblically correct LLM?

Somewhere along the way, Egg commented that our shape had become recognizably 3-dimensional. I had taken it for granted that it had been a 2-dimensional shape all along, so this development really surprised me.

That's this version, which Egg coded right before they made their Ezekiel one. I continued to build on it, again, by re-using some old prompts that had worked for me before: "make the motion even more complex and interesting and draw inspiration from rorschach inkblots" and "beautiful. make the motion even MORE complex and interesting!" Simple but powerful words.

The image below was made by combining two stills from the resulting generator.

I posted the link to this generator on the websim Discord, where Sirkitree saw it and felt inspired to riff on it. They hit the "back" button on my websim page to travel backwards in time from my latest iteration to Egg's tubular 3-dimensional version, which I had plugged into websim so that I could prompt over it, and then iterated from that point forward to make their own kaleidoscope-inspired variation.

Meanwhile, Egg told me that they liked a different branch that I was working on, and described an image that I sent over as "Islamic calligraphy as a predatory alien bird."

Interestingly, to make this version, I used the same starting point as before (Egg's tubular 3-dimensional generator) and almost the same prompts ("give this more of a rorschach inkblot effect" and "beautiful! make the motion even more complex and interesting! be artistic!").

Not wanting to waste a good description, I plugged "predatory alien bird" into Claude, and got a new generator that Egg had a lot of fun playing with. They sent me back these stills.

This time, Egg said that they look like ribcages, and of course, I passed the memo along to Claude, which yielded this variation.

When I posted this image to the websim Discord, Kat Winter said that she'd like to get it as a tramp stamp, and Rob Haisfield said that he thought that it looked like two birds kissing.

From here, I took it even further by asking Claude to "create additional controls for even more kaleidoscopic motion," because I was inspired by Sirkitree’s use of the word "kaleidoscope." This time, I got a generator whose outputs range from shapes that ressemble Celtic knotwork to a graffiti style vibe.

Egg loved it, prompted over my work, and asked Claude to "remove the circles in the background, decrease the 'animation speed' maximum, and eliminate any sliders that have no effect." They made this video with the resulting generator.

I had fun playing with Egg's version as well and produced this still image, which I sent right back to Egg.

Egg was surprised by my image, because it was so different from anything that they'd seen on their end while playing with the same generator, and asked me for a screenshot of my settings. Upon closer inspection, we realized that we each prefer to use different parameters: I spend more time working with really low wave speeds and high amplitudes, while Egg tends to explore quite low-amplitude spaces with one or both of the wave speed controls in the top half of its range.

Upon having this realization, Egg wrote to me,

This is part of what I love so much about gen art: it starts to become like you're wandering across a vast landscape of possible art, and it has all kinds of nooks and crannies and deep valleys that you may not run across for a very long time...

At this point, I prompted over Egg's version by asking Claude to "give this additional controls for 3d transformative motion." This produced the final generator in this series, which gave me some very nice textures.

But wait, there's more! Simultaneously, while we were making this series of generators, we also created an entirely different branch that was inspired by our initial foray into the world of sea anemones.

I kicked off this branch by splitting from the node that came just before the predatory alien bird. If you recall, I got the alien bird by iterating on Egg's tubular 3-dimensional mandala generator with the prompts "give this more of a rorschach inkblot effect" and "beautiful! make the motion even more complex and interesting! be artistic!" This time, I used the latter prompt again to move forward, but instead of it leading to the alien bird, Claude generated something with the capacity to be much softer and more delicate.

I asked Claude to "make the motion even more like a sublime jellyfish" because that's what this output looked like to me, and got this result next. Even now, I feel amazed that Claude was able to turn my instructions into meaningful code changes that actually look jellyfish-esque! The first two stills are mine; the third one is Egg's.

Then, I asked Claude to "make the shape even more jellyfish like with wider tentacles" and "make it possible to change the angle of the jellyfish." As you can hopefully tell by now, these are very simple prompts that vaguely grasp at what I'm trying to point at (which I often don't have a clear sense of myself), but Claude usually finds a way to make it work.

Meanwhile, Egg prompted over my "sublime jellyfish" edition by asking Claude to "give it a much more delicate and aquatic feel, with a sense of alien aquatic lifeforms" and then following up with "I like that but take away the circles you added, or make them move much more slowly" to create a dreamy version with little bubbles floating in the background, which you can find here. This is one of my favourite generators and I had a lot of fun making these stills.

Yet another branch reminded me of a snowglobe, so I started using that as a keyword. Often I'll get ideas about what’s possible by looking at different completions that I get from the same prompt, and then inserting my own descriptions of what I see back into Claude. That’s how I got the idea of using words like "fog," "moon," "hill," and "house" to anchor the continuing evolution of this piece.

Interestingly, whenever Claude generated trees inside the snowglobe, it almost always made them upside down, and asking it to turn them right side up resulted in all sorts of shenanigans that did not, in fact, match my idea of what a right side up tree is supposed to look like... although it did eventually get one right.

These results remind me of a figure from the paper "Reasoning or Reciting?" on LLMs' ability to reason about counterfactual worlds where the rules are different, which Egg recently sent me.

All in all, it's amazing how many possibilities there are in the space around my initial art generator. The range of effects are so wild and diverse that it becomes hard to prove that each could have arisen from the same starting point, but websim allows us to trace this evolution, and also, to branch it out endlessly.