Weekly Workshop
W10-Styly
Concept
Recently I noticed that there's a trend on the internet in China where a lot of people start "meditating" in online, specifically speaking, playing the Muyu (an instrument usually used in Buddhism temples to calm people down).
I think this is a very interesting phenomenon that something that seems so traditional and religious could be combined with digital media. Thus, I wanted to create an AR work that explores meditation in the space where virtual space and reality intersect.
Describe what you did.
I found an avatar that is sitting as if it is sitting in meditation. I changed the avatar's texture by adding a "[Style change] Crystal frozen" modifier to make it looks as if a wooden buddha sculpture. The modifier "[Animation] Go and back" is also deployed to make the figure floats in the air. I also added an "[Interaction] Look at you" modifier to make it look straightly at the player when moving the mobile phone. I wanted to use this feature to add some uncanny feeling to it so that I could raise the question of whether this meditation in the digital space really helps.
There're 3 lotus floating around the figure to further deliver the feeling of meditation. I added "[Animation] Go and back" to make one of the lotus float up and down in the air. I also added "[Style change] Crystal anim rough" to give them a reflective and vibrant texture and pink color. "[Animation] Heartbeat" is also deployed to make the flower become bigger and then smaller, which is meant to make them look more vibrant.
There're also two humanoid behind the user dancing. I exported the prefab from unity and added "[Humanoid Animation] Rumba dancing" to make them dance, indicating the distracting part of the meditation in the digital space.
I also exported a giant wheel circling around the user, which has a mantra written on it. The mantra is extracted from Tibetan Buddhism mantra and is about finding the inner piece. I added "[Style change] Crystal anim rough" to it to give it an illusion of the digital space.
What did you find challenging?
I found it very challenging to work with Styly. It crashes from time to time and I have to refresh the page every now and then. Also, I created a shader for the wheel. However, when I wanted to export it to Styly, it always gave me errors and I wasn't able to export the prefab with the shader. Also, I found that the modifiers are too limited and I wasn't always able to create the effect that I wanted.
Simultaneously, I wasn't able to figure out what the "effect" modifiers do. I added an "[Effect]TrailRenderer_slashVelocity" modifier to one of the flowers and I didn't see any changes to it.
Meditate digitally.mp4
What did you find exciting or inspiring?
The "[Style change] Crystal xx" modifiers are really surprising in making the very shiny and reflective textures and they look very good in the scene.
What did it prompt you to want to know more about? (ready to talk about the questions in class)
I wonder if we could add some user interaction to the scene. For example, I wonder if it's possible for the user to press something on the screen and then the objects in the scene correspondingly perform some actions. (I found a tutorial on Styly Magazine and had a try, but wasn't able to...)
W8-Post processing
Link to the project: Digital Shrine by Jiapei Yao (itch.io)
W6-Shader Graph
For the first graph, I looked up this shader graph screeshot because I really like its aesthetic that is a bit surreal. Thus I tried it out to use it to make the terrain of my project, which is a plane shaped like a donut ring.
As I was trying to remake a same shader graph, I noticed a lot of places that is quite different. I can't seem to find the "Unlit Master" node at the start and the "SimpleNoise3DLayered" node. Thus I tested with "simple noise" node and it worked similarly. I also created another color pallete to put in the "Sample Gradient" node to make the color more suitable for my project. And I played around with some of the nodes and made the circular partern become horizontal stripes by adding a "split" node between the noise node and the UV node. I think its more interesting this way. However, I do wish that it could actually be a running in concentric circles as the donut ring. I will keep on exploring how to make that.
donut terrain.mp4For the second graph, I tried to make this sun-like glowing material. However, because I can't find many of the nodes that it uses, I fail to create a similar one. I adopted how it uses two different "simple gradient" nodes to create two layers of different colors and textures. I also combined this technique with what we learned in class: I added a picture and time to make the shader move. I also added a "Fresnel Effect" to make the edge of the figure glow (learned from another Youtube tutorial).
It wasn't very satisfying because I wanted to make the figure transparent and glow in a way. However, the transparent feature doesn't seem to work well.
center figure.mp4
I also tried to make this "sci-fi shield" to use it as the shader graph for the human figure. However, I didn't manage to make it work and I still haven't figure out why. The shader graph seems to work well in the "Main Preview" window. However, when I apply it to the object, it is just a transparent figure, the pattern disappears.
Comments
Need access to watch the video.
W4-Animation
Oculus Quest 2 Experience
National Geographic VR
I tried out the Machu Picchu map and enjoyed it pretty well. I think it maintains its sense of enchantment pretty well as there're many missions for me to complete, like taking pictures of a certain location and building houses. I think the design of taking out the camera from my back, as if I'm carrying it with me, is very interesting. It's very interactive, making me feel like I was actually a tourist with a camera. However, I find it a bit off when I have to jump to certain places with my left joystick instead of walking towards the designation. But it gets better after I get used to it. I think it's also a bit off when seeing the environment around me has a relatively low resolution, which looks very unrealistic to me.
This game sets up its narrative by telling you that you're a photographer for the national geographic magazine and giving you tasks to complete. It's like a healing trip where you could explore some interesting places in the world and also take beautiful photos. The storytelling is clearly demonstrated as the narration guides me on what to do next and a local guide tells the culture and history of the place.
Paper Birds
This is quite a different type of storytelling compared to National Geographic VR. Its major purpose is to tell the story and we as users are passively listening to the stories told by the narrator. It's a movie inside the VR headset, where you could look around the environment and make limited interactions.
I think the part where there are stars and lights floating in the sky is very beautiful and enchanting. And with some interactions where I could make some particles floating out of my hands, I was totally attracted by that scenery. However, I think it's a little bit off when a certain narrative is happening and is only happening in one place. I had to turn my head to find where that part of the story takes place. Also, unlike National Geographic VR where it's a first-person perspective, in this story, I'm just a passive observer. Thus, I find it a bit not so logical when suddenly I could interact with it. I don't really see how the interaction would help the narrative. But it does add a bit of fun to the game though.
Plot Actions Project 2 Warm Up
For this assignment, I chose the episode "Jibaro" from Love, Death, and Robots season 3. It's a story about "A deaf knight, and a siren of myth become entwined in a deadly dance. A fatal attraction infused with blood, death, and treasure (IMDb)." It metaphorically talks about toxic relationships.
The first action that I chose is where the deaf knight just arrived by the lake and went to pick up the gold pieces in the lake. And because of his movement to fetch the gold pieces in the water, the siren was awoken.
This action/interaction is unique to this world because of many reasons. First, it serves as an introduction to the greediness of the deaf knight and an introduction to the siren character. Second, it indicates the relationship between them and the massacre that will happen shortly after. Thirdly, and most importantly, it is an indication of the setting of this world: a lake controlled by the siren and the gold is what she is governing.
I think this would be an interesting setting recreated in VR that when the user touches an object (the water in this scenario), certain action of the NPCs or certain reaction of the environment is triggered. But of course, this would rely on some coding I guess.
The second action is when the siren awakes and lures the other knights into killing each other and walking towards the center of the lake, as she dances dramatically and shouted loudly.
This action further establishes the worldview where the siren lures people who invade the lake into death with her compelling voice and dance. And at the same time, as she is visually covered with gold and jewelry and sounds like metal colliding with each other in hearing, this action also indicates the characteristic of the siren: a symbol of luxury and richness.
I think the most interesting part of this action to be recreated in VR would be the visualization of the strength of the siren's voice. In the episode, the knights lured by the siren were pulled towards the center of the lake as their chests are "pulled" forward. An invisible force dragged the knights forward in correlation with the audio. Thus, I think it would be very interesting to recreate this feeling of being "dragged by the sound". Simultaneously, I think it would also be interesting to learn how the movements of the siren were done, which is a very dramatic dance. I imagine that this would be very dramatic and appealing if placed in the VR set.
The third action is when the deaf knight knocked down the siren and peeled off all the jewelry and gold on the siren. The deaf knight violently peeled off everything on the siren as the siren's body being pulled up and down by the knight's movement because the jewelry and gold is a part of her body. The gold and jewelry are spilled onto the floor, dipping into the blood of the siren.
This action is the climax and the biggest dramatic conflict of the whole episode, as the siren was completely ruined, and the very darkness of the deaf knight was appealed.
I think the interaction here when the deaf knight goes insane could be adapted to create the climax if I were to tell a story in VR. The repetitive movements of peeling off gold and jewelry and the close-up focusing shot on the movement would be valuable hint for creating a story with strong emotion.
To recreate these actions in unity, I created a scenario where a male character picks up a chip on the floor, and as soon as he did that, the female character is introduced and started dancing. Then when the female character screams (ideally), the male character is pulled to the air, dropped onto the floor. As shown in the video below, the actions' effect mainly relies on the camera movement, which I adapted some of the techniques from "Jibaro". I created some close-ups and panning movement to show what's going on.
However, I had to abandon using the sound effect because I don't know how to add the audio to certain clips so that the audio is played at certain point, instead of from the beginning. Also it's hard to find a animation that fits perfectly well with my imagination.
W4-workshop - SampleScene - PC, Mac & Linux Standalone - Unity 2020.3.26f1 Personal_ _DX11_ 2022-09-27 10-52-38_Trim.mp4W3-Skybox, First Person Controller, and Camera movement
In this week's workshop, I created story where there are snacks along the way to allure audience and "ghosts" that "chase" the audience in a narrow corridor. It's somewhat similar to a 3d version of a Mario-like game.
To create this scene, I took a 360 picture of the corridor inside our residential building (using Google Street View app), which was later used as the skybox. I also 3d scanned some objects with Scenario. The 3d objects include a pair of shoes, a glasses cloth, cereal, chip, and a chocolate bar.
For the camera movement, as described in the storyboard, there are 4 shots in total. For the first shot, I imagined the camera to move quickly towards the first person controller from far to near, to establish the overview of the whole scene. The second shot is to show that is in "front (by default, the direction towards the 'ghost')" of the user (first person controller). The third shot then is to show the back of the user. Then in the final shot, user could control the camera direction him/herself by switching the camera to the first person controller.
As shown in my final work, I did some changes compared to the storyboard because of some technical issue. In the storyboard, I pictured the objects to be lying on the floor and the user will also be "walking" on the floor. However, when I took the 360 picture, I thought that the user could actually move inside this space made of picture. It was when I set the picture to the skybox and moved the objects inside the skybox that I realized that skybox actually represents a "sky" that is very far away. I can't (or I didn't manage) to move in the space. I could only move the perspectives or move among the objects. Thus, what I pictured in the storyboard didn't work out in Unity as I took the picture when standing up, and the floor looks very far away from the default ground where I could place things. Therefore, I changed the idea and decided to put all the objects in the air.
Simultaneously, I found all the objects that I scanned have some ground arround them. Although I tried to remove them with blender, but then the material won't work somehow and I didn't find out how to deal with this. Thus I created a purple cube below each of the objects.
The purple cubes are made to 1). make the objects feel more steady (not floating in the air) and stand out more (otherwise it looks quite messy as the colors are very different; 2). to hide the ground arround the objects to make them appear more natural. It's not an ideal solution. However, I do think that these cubes make the scene more interesting and suited the palyful concept more.
W3-assignment.mp4 W2 - Creating the Environment
I chose the world from the famous anime "Paprika" because this anime is very creative in constructing a world in-between the reality and the dreams. In the perspective of the concept, I think it's idealogy very similar to what the VR is doing -- recreating or blurring the boundary between virtual and reality. I think this is a very interesting topic to talk about in the VR project as well.
The first scene I chose in this film is the cinema inside Detective Toshimi Konakawa's dream.
I found this dream interesting because it is an environment that displays a dream inside a dream. In other words, the cinema itself is a dream, and the scene happening on the screen is a deeper dream. Thus, if I were to create it in Unity, I could create one scene for the cinema itself, and another (maybe even multiple) 3D scene(s) for the environment on the screen. And if I could embed some coding, there could be an interaction as the user could enter the scene on the screen, or change the scene on the screen as well, which I think is very interesting to work with.
For the components of the scene, it's a pretty realistic scene. The objects and their materials are:
Red curtains --> cotton fabric
Screen --> white aluminized reflective surface
Wall
Seats
Coarse fabric
Plastic
The second scene that I chose is at the end of the film, where the center of the city collapsed into a hole.
The storytelling here is very interesting, as reality turns into dreams, and the boundary between reality and dream vanishes. Thus, in order to express this feeling of blurring reality and fantasy, it's very important to consider, especially how to deal with the transition section. That is, how to create the feeling that these two places are both realistic, but still different.
Inside the anime, the director dealt with this question by 1). distorting the transition section and making it looks as if a fabric is being pulled down in the middle; 2) having the contrast in the materials of the reality (solid brick and asphalt road) and the dream (a paper-like texture that looks unreal); 3) a clear division line in-between that looks like rocks and mountain.
When creating the scene in 3D, I think there are these points that I should pay attention to:
the inclined/distorted edge of reality (material remains realistic)
the choice of the texture of the dream part (I imagine it to be realistic, but with faded color, and relatively flattened)
the transition between the two sections (I think I would not create a rock-like division line as the anime since it would be abrupt, I suppose I would have a smoother transition between the first 2 points)
Thus, for my VR project, I would continue like to continue exploring the idea of VR as a device that connects reality and the virtual space. At the present stage, I'm picturing a space similar to the first scene: an environment where two "layers" of space are present (layers as in a dream inside a dream, or a virtual space inside a virtual space). Simultaneously, there's also a component similar to the second scene, where the two layers overlap and influence each other.
To describe in greater detail, what I'm picturing in mind is (primary thought):
A cinema just like the first scene (a regular cinema with a giant screen in front, some seats on the opposite side)
The screen is a second 3D virtual space that the users could walk into and experience (haven't figured out what will be inside the screen yet)
Then after some time/interactions, the boundary (the flat screen) between the two 3D spaces starts to blend and influence each other.
Maybe like the second scene, where some people or objects inside the screen space extrude out of the screen, and both of the spaces are distorted.
Process
Based on the above scenes from the movie "Paprika", I created 2 scenes.
The first scene is based on the cinema scene, which is a classroom. I first downloaded a classroom scene from the asset store (pic 1) and deleted some of the objects and left with only one classroom. To make a more immersive environment, I created a cube and changed its size to make it the roof of the classroom (pic 2). Then, I created another 2 lights on the top of the ceiling (pic 3). And based on what we learned in class, I made the two "lights" glow.
For the other scene, I created a terrain and used the "Raise or Lower Terrain" tool to make some mountains on the terrain. I also added some trees, flowers, and grass to the terrain (pic 4&5).
Then, as inspired by the movie scenes, I combined the two scenes together by deleting the windows of the classroom and moving the terrain near the windows (pic 6). By doing this, I wish to create a story: when having classes in the classroom, the students fantasized and freed themselves by imagining nature appearing in the classroom. It's a story about healing and comfort.
Difficulties
During the process of making the scenes, I encountered a lot of difficulties and had to make a lot of compromises in the end.
To begin with, I first wanted to make the connections between the two scenes on the projection screen instead of the windows. However, as I was checking the projection screen, I noticed that the whole projection screen is an object on top of the wall. And if I remove the projection screen, there won't be a hole on the wall where I could display the other scene. Simultaneously, I don't know how to create a hole in an object, so I had to give up on that idea.
Secondly, I initially wanted to create a city with a collapsing hole in the middle, just like the scene in the movie, instead of a terrain full of plants. However, when I was creating the terrain for the collapsed city, I noticed that I couldn't make the terrain lower than its default plane. After searching online, I didn't find any feasible solution and was told that it was just impossible to do that. Thus, I switched to the alternative of making a terrain full of plants.
W1 - Basic functions (rotation, position, size of objects)
Description:
For this assignment, I downloaded 2 scenes from the unity asset store: one full of food (a), and the other with a playground (b).
For scene (b), I deleted some elements, such as planes and some circles in the sky. I also scaled down the sky and clouds for a better view.
For scene (a), I selected some foods that I like and dragged them into the scene. I resized them so that they could fit the size of the scene (a) better. I also resized all the foods in the "sky", and placed them at different heights and angles (Move tool & rotate tool) to display a sense of randomness. I also duplicate them to have a better effect.
In addition, I created a lollipop with a sphere and a cylinder with new material. I also changed the color of them. Simultaneously, I also add the "rigidbody" property to all the food, so that when the game is played, they could all fall down from the sky. In this way, I hope to achieve a scene similar to the animation "Cloudy with a Chance of Meatballs)" where foods fall from the sky and people could have a "food carnival".
What did you find challenging?
I found moving the objects to the proper space very challenging. For example, when I created the sphere and cylinder for the lollipop, I couldn't put them together with the move tool. Changing the number of the X, Y, and Z positions in the inspector panel didn't work as well. Since it's 3D, moving the object to the ideal place from one perspective is not enough, I had to switch between different angles to move the objects to the place I wanted them to be.
What did you find exciting or inspiring?
I think it was very interesting when I changed the position of the lights and see the shadows of the food change position. It's also interesting when an object's position shifts entirely when you switch a different angle to look at it. This visual illusion could be a topic to talk about many things in the 3D world I guess.
W1 - Lab - Demo - PC, Mac & Linux Standalone - Unity 2020.3.26f1 Personal_ _DX11_ 2022-09-05 23-06-08.mp4What did it prompt you to want to know more about?
When I added the "Rigidbody" property, I found that the objects will fall endlessly down the scene. I'd like to learn how I could control them to stop at a certain height.
I'd also like to learn more about how to create different materials and textures, as the lollipop that I created doesn't look very realistic because of its texture.
I'd also like to know how to navigate the objects and my perspectives more efficiently. Because now it's pretty hard for me to find a great angle to work with, and locate the objects in the position I want them to be.
Comment: Jiapei, This is striking! Love your scene. You achieved so much in the first week of class. Are you planning to work more in low poly systems? Navigation in Unity at first is a headache but you will use to it (as long is your project not too heavy for your computer to process!).
Response: Thank you for the comment! I think low poly systems are very interesting aesthetically, but since we are at the very beginning stage, I'd love to explore more different styles.