Embedded Files
You can run SD online at DreamStudio dot AI, just need to make a login and register.
OR you can install locally using Github, Huggingface, or Visions of Chaos, they all require Nvidia GPU and numerous software installation.
The Stable Diffusion webUI from automatic1111 (we focus on this) - there are other webUI to choose, such as InvokeAI
https://github.com/AUTOMATIC1111/stable-diffusion-webui
https://github.com/invoke-ai/InvokeAI
A very complete list of guides, tools, and papers to help you create the best possible text prompts - Prompt Engineering Guide
Text-to-Image systems/models are machine learning based models that take a descriptive phrase as input and attempt to generate images to match that phrase.
List of 4 different text-to-image diffusion models and the companies that Maintain them.
Stable Diffusion - open source software, maintained by private company called Stability AI, trained on LAION-5B database, the largest freely accessible dataset of image-text pairs
MidJourney - private company, runs on a Discord bot, not open source model
DALL-E (2) - private company called Open AI, also makes ChatGPT, with Microsoft as a large investor
Imagen - Google maintained, not open source, trained on the COCO data set
Clear video description of how text-to-image works, start at 5:58
https://www.shinyobjects.gg/blog/Text-to-Image-Diffusion-Models
More technical programming description of the SD latent diffusion model
https://huggingface.co/blog/stable_diffusion
Understanging SD and diffustion based generative models work (more metaphorical)
1) Training Data - sets of image/text pairs, examples are LAION-5B, COCO, WIT (Wikipedia-based multi-lingual image/text pairs)
2) Deep Learning - Machine Learning process to generate the model by finding variables for every possible image characteristic
3) The Multi Dimensional Latent Space - like a comparison graph with over 500 dimensions (axes with variables) a web of vectors, in webUI this is the model that we download and use, like SD_1.5, SD_2.1, protogen, and others. (the input text prompt naviagates to multiple points inside this math latent space)
4) Diffusion Generation - to translate the point in math space to an image in pixel space we use a process called diffusion which starts with noise and over a series of iterations arranges pixels into a compisition that makes sense as an image. (choosing the Sampling Method here changes output image)
List of top used Datasets for Text to Image models
https://analyticsindiamag.com/top-used-datasets-for-text-to-image-synthesis-models/
SD webUI again, to show X/Y/Z plot feature to wedge or bracket variables against each other and compare results
https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features
Live Demos (WIP - these sections I will actually generate images, cannot drag on too long they are like little renders)
1) show Auto1111 - generate a couple pictures, Varible Grid, then show decent looking finals (do this within the first minute of speaking)
2) show InvokeAI webUI - generate some fast pictures, then show InPainting
3) show interface for model training in dreambooth along with some final results
Text2Light - HDR panoramics
Text2Light - HDR panoramics
This software can make High Dynamic Range 360 panoramic images for lighting 3D scenes in Maya. The images that it creates are fairly generic or typical, but the great part is the light values are very strong and accurate, with the sun clipping at 65k, and around 10k for interior lights.
Page updated
Report abuse