Prompt Styles and Weights
Experiments
Baseline Images
Lets start with a baseline. For this writeup, we’ll be using the following prompt format:
e.g.
We’ll be relying on the seed to produce consistent images for comparison, so let’s see what happens if we run the same prompt three times
Very minor Differences! Looks like we can reliably use this for our experiment!
Prompt Separators
First, let's look at how separators could affect a simple prompt (We wont get into how NLP parses lists and grammar, or how to trick it into creating pseudo words / compound words)
Utah Teapot Wood
Utah teapot wood
UTAH TEAPOT WOOD
Utah Teapot Wood!
Utah Teapot? Wood
UtAh TeApOt WoOd
Utah Teapot, Wood
Utah Teapot + Wood
Utah Teapot - Wood
Utah Teapot | Wood
Utah Teapot || Wood
Utah Teapot : Wood
Utah Teapot. Wood
Utah Teapot < Wood
Utah Teapot > Wood
Utah Teapot & Wood
Utah Teapot && Wood
Utah Teapot or Wood
Utah Teapot and Wood
Utah Teapot xor Wood
Utah Teapot in the form of wood
Utah Teapot made of Wood
Utah Teapot made by Wood
Utah Teapot in the style of wood
Utah_Teapot_Wood
Welp, that convinces me, separator doesn't do much of anything in this context!
Prompt Weights
Regular Weight Variation
Next, let's look at weights. We're using an image prompt, so that image gets a default weight of .25, unspecified weights get a default value of 1.
Utah Teapot::1 Wood::.1
Utah Teapot::9 Wood::.2
Utah Teapot::8 Wood::.3
Utah Teapot::7 Wood::.4
Utah Teapot::6 Wood::.5
Utah Teapot::5 Wood::.5
Utah Teapot:: Wood::
Utah Teapot::5 Wood::.6
Utah Teapot::4 Wood::.7
Utah Teapot::3 Wood::.8
Utah Teapot::2 Wood::.9
Utah Teapot::.1 Wood::1
Negative Weights
Next let's examine a few interesting weights! Note: The total of all weights needs to be non negative or you will get an error.
Utah Teapot::-1 Wood::1
Utah Teapot::0 Wood::0
Utah Teapot::-.125 Wood::-.125
*--iw defaults to .25
Utah Teapot::0 Wood::-.125
Utah Teapot::-.25
Utah Teapot Wood:: Wood::-1
Image Weights
Image Examples
Aspect Ratios
Uplight And Stop
If you’ve ever upscaled your image, you may find it super grainy, full of noise, or drastically different. Some people use external upscalers, but you can also attempt playing with the –uplight and –stop parameters. Let’s take a look! We’ll use the prompt /imagine Utah Teapot --ar 16:9 --seed 1 play with various stop and uplight combinations. We’ll specifically look at variation #2 (The top right image on the grid).
Seeds
// TODO: Simple example showing how simply incrementing the result doesn't create simple variations, but sameseed does
Default Prompt Weights
Is the same as
/imagine <URL> Prompt::1/imagine <URL> Prompt --iw .25/imagine <URL> Prompt::1 --iw .25/imagine <URL> Prompt::4 --iw 1/imagine https://upload.wikimedia.org/wikipedia/commons/thumb/9/93/Utah_teapot_(solid).stl/1200px-Utah_teapot_(solid).stl.png
Utah Teapot --ar 16:9 --seed 1
/imagine https://upload.wikimedia.org/wikipedia/commons/thumb/9/93/Utah_teapot_(solid).stl/1200px-Utah_teapot_(solid).stl.png
Utah Teapot::1 --ar 16:9 --seed 1
/imagine https://upload.wikimedia.org/wikipedia/commons/thumb/9/93/Utah_teapot_(solid).stl/1200px-Utah_teapot_(solid).stl.png
Utah Teapot --iw .25 --ar 16:9 --seed 1
/imagine https://upload.wikimedia.org/wikipedia/commons/thumb/9/93/Utah_teapot_(solid).stl/1200px-Utah_teapot_(solid).stl.png
Utah Teapot::1 --iw .25 --ar 16:9 --seed 1
/imagine https://upload.wikimedia.org/wikipedia/commons/thumb/9/93/Utah_teapot_(solid).stl/1200px-Utah_teapot_(solid).stl.png
Utah Teapot::4 --iw 1 --ar 16:9 --seed 1
Colors with Weights
// TODO: You can use smaller prompts with low weights to influence the color such as `A girl with black hair and a red streak:: girl with black hair::.2 hair with red streak::.1
Guiding Through Variations
// TODO: You can use the variation buttons to fix the composition, anatomy, position, color, etc.. think of it like blood lines, if a child doesn't improve on the parent image, reroll
Pseudo Compound Words
So what does MJ think of random words we make up? Can it interpret custom compound words such as blightwood or terratron? Maybe! Lets see how words can be smooshed together and how MJ might interpret them. For this, we wont be using the image prompt.
// TODO: UtahTeapot vs Utah Teapot vs Utah-Teapot vs Utah+Teapot (notice the lack of space - makes a big difference in MJs understanding)