Sign to Image is an application that aims to create a PC or VR experience where the user uses hand gestures to create images.
The project team is composed of four students :
Florian CASTEL - Image generation
Arthur AUDRAIN - Gesture recognition
Ayoub NEMMASSI - Design and storytelling
Hippolyte DUBOIS - Grammar
For the beginning of the project, I focused on the image generation.
State of the art of text-to-image models.
Choice of the model and its differents parameters.
Adaptation of the code of an implementation of the model to create an API accessible on the Internet.
Communication between Unity and the API.
Progressive display of images during their generation.
Options allowing the user to choose the style of the generated images.
Saving and loading images, filtering and ordering images according to a query.
Examples of images generated by the application in different styles from the sign "fox" (From left to right and top to bottom: No style, photo, oil painting, pencil sketch, colored pencil drawing, digital art, abstract, watercolor, cubism) :
Stable diffusion Web UI by AUTOMATIC1111: https://github.com/AUTOMATIC1111/stable-diffusion-webui