Final Project
Legend of Wrong Mountain: Chinese Traditional Opera Generating with Machine Learning Methods
Lingdong Huang, Zheng Jiang, Giada Sun, Tong Bai
Kunqu, which is one of the greatest opera forms of Chinese Art, embodies the Chinese cultural background, literary aesthetics, music and performance. Most of scripts and music of Kunqu are written in several hundreds of years ago, which constructed one of the Masterpieces of the Oral and Intangible Heritage of Humanity. Through applying the cutting-edge Machine Learning methods, our group produced a number of aesthetically and technically interesting results about Kunqu. Among the scripts generated by Machine Learning, we selected one of them called The Legend of Wrong Mountain and also generated accompanying music according to the tonal rules. Our work gives an attempt to explore the relationship with Machine Learning and Chinese Traditional Art and tried to recreate some new art elements mixing with the traditional art and modern technology.
Evolving Method of Perception
Jacqui Fashimpaur, Tatyana Mustakos, Char Stiles, Tait Wayland
We wanted to explore how computers visualize and perceive the world. Embracing naïvety, we imagine what a neural net might dream on some random night during its training to become more aware of its surroundings. In the process of creating and exploring this VR experience, we asked ourself how this computer perception relates to our own, and whether or not there is something universal about the nature of perception.
PUTTI: An Art Crawl in a Post-Human World
Mary Beth Kery, Kenneth Holstein, Hanyuan Zhang, Angela Liang
Terabytes of paintings; no humans left to interpret them... In the future, all that remains of humans are their data (stored away in vast archives) and their algorithms (which crawl the archives, trying to interpret the data). The National Gallery of Art is now home to billions of fledgling algorithms, who feed on the data it provides. “Semantic rot” of digital art archives over time... But the interpreted spaces the algorithms navigate bear little resemblance to human interpretations. Without humans to interpret these vast archives, has the art lost its meaning? Has it acquired new meanings? An intrepid explorer... In this experience, you explore as one of these algorithms. You traverse multiple eras of human art, trying to make sense of these treasured cultural artifacts
GAN Theft Auto
Oscar Dadfar & Hizal Celik
In this research, we analyze the possibility of procedurally generating a
canvas reflective of the Grand Theft Auto style. Using a semantic segmented
GTA V dataset, we train the pytorch implementation of Pix2Pix, as well as
Pix2PixHD. We merge a procedurally-generated Unity environment with the
Pix2Pix style transfer algorithm to get the generative GTA environment, and feed
the data back into Unity to produce a real-time style transfer of the procedurally
generated environment.
Machine Knitting & Machine Learning
Lea Albaugh
As part of an ongoing research project, the CMU Textiles Lab is compiling a dataset of knitting patterns that produce textured swatches: that is, flat rectangles of knitting with such stitch-level details as ribbing, lace eyelets, and cables.
The Textiles Lab's research code parses these textual patterns into a graph-based representation, with each stitch as a node that is connected to its row-wise neighbors and its column-wise parents and children. For machine knitting, each stitch is then allocated to a particular position and time of construction: in other words, the graph is laid out as a planar grid.
For this project, I introduce an intermediate representation of the graph structure that can be encoded as three channels of information over a grid. I then use machine learning methods to generate more patterns in this representation, and decode them back to the graph representation to pass through the rest of the pipeline.
Finding a Latent Space for the Virgin Mary
Nico Zevallos
In the Catholic faith some images of the Virgin Mary are allowed to be venerated. These are images or sculptures to which miraculous power is ascribed. These images are copied and reproduced and in turn are ascribed their own miracles. As a whole, they represent a deep archetype that crosses cultural boundaries and reaches beyond its scope in Catholicism.
My goal was to visualize a latent space for a complex archetype. What underlying structure exists for such a ubiquitous image? Is there enough variety for the computer to learn to produce new images? Would these images even be new?
DeepCloud
Ardavan Bidgoli, Pedro Veloso, and Shenghui Jia
One of the most interesting topics in Machine Learning (ML) for design are generative models. However, the potential of generative models in design is still unexplored. Most of the advancements in generative design systems are problem-oriented (search, optimization, etc.) or rule-oriented (shape grammars, swarm models, cellular automata, etc.). There are almost no design application based on big data and few researches investigate design exploration with data-driven generative systems. As a consequence of this gap, there are no standards for modes of interaction, performance, and design representation with ML generative models for design.
To explore the potential of ML models as a main component of generative systems, we developed
DeepCloud. It is a general design application that incorporates recent advancements in deep
generative models to conceive 3d point-cloud objects in real-time.
Visualizing 3D Loss Landscapes for High-Dimensional Machine Learning Models
Shouvik Mani, Hai Pham, Yang Yang
The goal of this project is to compute and visualize 3D loss landscapes for high-dimensional models. We propose two methods to reduce the dimensionality of the parameters in a model. In both methods, we reduce a model to just two parameters and visualize the loss against those parameters over iterations of gradient descent. The resulting 3D loss landscape can be used to diagnose models (e.g. to identify local optima) and also serves as an artistic representation of a complex ML model.
cellulaire en colère
Jeena Yin, Anirudh Mani, Joseph Gibli, Zaria Howard
Machine learning has given technology a human voice. Even though models can produce human-like audio, computer generated speech has almost solely been used to serve end-users as neutral sounding voice assistants. Our project grants technology the power of free speech by training various neural networks on emotional audio recordings. We allow these models to produce emotive speech and then direct their frustration towards each other. Human spectators to their argument are faced with learned rage communicated through tools typically meant to mindlessly serve.
Monster GAN
Sydney Ayers
Science Fiction and Horror films often act as a mirror for what truly scares a culture. Not just in the literal sense, but also through themes such as fear of invasion, nuclear warfare, communism etc. In the 50s to 60s, oversized insects and animals appeared often, like in Them! and The Deadly Mantis. This reflected the fears of nuclear proliferation and the effect on the environment. I began to wonder what scifi and horror films say not contextually in those time periods, but also as an overarching history of the 20th to 21st century. If you took similar features from every scifi/horror film in the past 70 years, what would the monsters look like? What would they represent?
I chose to use a DCGAN (Deep Convolutional Generative Adversarial Network) because I wanted to create new monsters that still had the influences and style of previous monsters. The dataset used to train the machine learning models contains every movie poster in those genres from the 1950s to now (about 1,760 images). A GAN has 2 parts: a generator and a discriminator. The Generator creates fake samples that along with real samples are put into the Discriminator. The Discriminator tries to guess if this was a real image or a generated image. The Discriminator will attempt to improve the Generator’s parameters until it fools the Discriminator. After running it through the GAN, I also used an online generator to add generative horror titles. I matched these with the images I thought represented them best, and arranged these into a movie poster.
Bred Agents
David Gordon and Aman Tiwari
We create and visualize the interaction of two completely different approaches to creating "intelligent agents". We train some agents to learn to communicate and compete, using MADDPG. We train some others using a variety of RL algorithms. The final population are untrained, but instead evolved using genetic programming. These agents represent different forms of being in the world and various facade dichotomies created in it. We put them into one aquarium for the audience's pleasure.
Alien Walk challenges a trained machine learning agent named Zelboc to reach his spaceship by traversing a treacherous landscape. In order to safely reach his destination, Zelboc must be skilled in running, jumping, and platforming. As Zelboc navigates the world, his decision making is sonified by the triggering of audio samples. Dictated by the layout of the terrain, Zelboc's actions retain rhythmic patterns that lend themselves to procedural music generation. This project explores how an audience might empathize with an automated agent, rather than trying to defeat one as most games require. Zelboc's tireless dedication is surely something we can all admire.