Sign to Image

Sign to Image is an application that aims to create a PC or VR experience where the user uses hand gestures to create images.

The team

The project team is composed of four students : 

Florian CASTEL - Image generation
Arthur AUDRAIN - Gesture recognition
Ayoub NEMMASSI - Design and storytelling
Hippolyte DUBOIS - Grammar

My contribution to the project

For the beginning of the project, I focused on the image generation.

• State of the art of text-to-image models.

• Choice of the model and its differents parameters.

• Adaptation of the code of an implementation of the model to create an API accessible on the Internet.

• Communication between Unity and the API.

• Progressive display of images during their generation.

• Options allowing the user to choose the style of the generated images.

• Saving and loading images, filtering and ordering images according to a query.

Examples of images generated by the application in different styles from the sign "fox" (From left to right and top to bottom: No style, photo, oil painting, pencil sketch, colored pencil drawing, digital art, abstract, watercolor, cubism) :

Credits:

Stable diffusion Web UI by AUTOMATIC1111: https://github.com/AUTOMATIC1111/stable-diffusion-webui