Weekly Workshop
Week 10 AR in Styly
The concept is to create a utopian metaverse that reminds the audience of an oriental town or a scene from a Hayao Miyazaki movie. The humanoid I chose is also a Japanese anime girl with the "sitting and laughing" movement. The humanoid animation provided by Styly has very few options so I could not find a desirable one. The other two dancing movements are more awkward when placed in my scene. I would love to learn how to upload character animations that I found in Mixamo.
The most interesting part is that the shader graph I applied in Unity can also be imported into Styly, which opens up many opportunities and customization to my future work.
The most challenging thing for me was to export the whole unity scene to Styly, I encountered an error message as shown in the screenshot even after I carefully follow the instruction. I modified a lot of settings but still got the same error. It was finally resolved just by restarting Unity and re-upload the Styly plugin. I also experimented with spatial sound in Unity, but when I uploaded it to view in AR on my phone, the spatial feel did not come through.
Week 9 Post Processing
Week 6 Shader Graphs
Dissolve & Distort Shader Effects
Given that my midterm project deals a lot with memory loss, cognitive impairment, and environmental storytelling, I looked for screenshots or tutorials on dissolve and distort effects to (1) simulate the process of memories fading out (2) display the visual world seen by a cognitively disabled person.
My experiment 1: dissolve
I downloaded 3d model objects from Sketchfab, a cat and a scooter, and applied the shader material onto them. They would dissolve and then reappear automatically. I'll definitely use this technique in my midterm project
My experiment 2: distort
For the second shader, I did a moveable distorter on the background image. Its position and size can both be altered to achieve the desired effect.
Jiapei: Both of your shader graph fits perfectly well with your concept. I especially like your dissolve shader graph. The transparency and color and the glitches works really well!
Week 4 Experience 2 Oculus Quest apps + Plot Actions Project Warm Up
National Geographic VR
It is a really fun interactive travel experience. I tried the Antarctic expedition which started with me standing on a float next to a giant cruise. The guide explained background information about this expedition to find the emperor penguins' colony. Then I was guided to sit in a kayak and paddled off on my journey. A prompt throughout was that my identity as a National Geographic Explorer had a mission to document and take photos. I felt it was a very clever way to keep the user enchanted. Throughout the exploration, I was always very keen to look for good scenes worth photographing in my surroundings. Sometimes I even had my hands full trying to control the direction of the kayak and take photos, which was a problem I usually welcome in playing a game. The details of the scenes were very aesthetically pleasing and realistic enough that whenever my kayak was about to hit an iceberg I would get very anxious and try to paddle in the opposite direction. A lot of environmental elements contributed to the immersive sense of the freezing and lonely exploration - the sound of iceberg scrap falling into the sea and the vision shrouded in fog - which all together made me believe that I was alone in the middle of Antarctic icebergs at that moment.
The development of the plot is mainly through voiceover instructions, subtitles, and the cursor reminding should direction to follow. For me, the voiceover and the plot trigger happening at exactly the same time played a crucial part in immersion. The tone of the male guide's voiceover was very appropriate to what I was experiencing. When I climbed to the highest point of the iceberg, for example, the male guide would ask me to give him a wave as if he was really watching me from the cruise ship in the distance. On the way to climb the iceberg cliff, when the ice scrap on the side broke and fell down, almost hitting me, the male guide would then exclaim the thrill of what had just happened.
After a relatively calm kayaking and nerve-racking rock climbing, it was more like an adventure game of survival in a snowstorm. I was guided through the task of fixing the tent and collecting supplies, which could be rather anxious in a snowstorm. But eventually, when I managed to get back inside the warm tent and the howling of the wind and the drifting snow were instantly cut off, I felt a sense of accomplishment and relaxation. This immersive narration felt like an emotional roller coaster that took me up and down to keep me engaged and made me look forward to what would happen afterward.
Paper Birds
What impressed me the most about Paper Birds was how exquisitely made its architectural modelings were and how great the spatial experience was. It reminds me of some of Pixar Studio's animations or an immersive theatrical experience. Even though I was given less freedom and interactivity as a viewer - primarily a God's perspective observer - I was able to turn my head to explore the spaces constructed in the film, such as the nightclub, practice room, staircase, basement, tree house, etc. The transition of scenes is guided by the illuminated component in a dark background. As I poke my head into a space and try to follow the footsteps of the main character Toto, I also became more conscious of the environment and how the character interacted with different objects. Being able to draw starlight in a galaxy-like scene indeed created a spiritual feel. Its hand tracking along the process was also very impressive. Being able to draw starlight in a galaxy-like scene indeed created a spiritual feel.
In terms of the immersive narration, it has well balanced the storytelling using both dialogues between characters and voiceovers to move the plot forward together. But I personally think that while it seemed like a tale building a grand worldview, the specific plot was too fragmented. What the story wanted to express was beyond the scope of the half-hour film, and thus seemed less evoking. The transitions between temporal and spacial scenes also need to be more fluid to not distract the audience.
Plot Actions Project Warm Up
I chose to analyze one iconic clip in La La Land where Seb and Mia tap dancing lovely among the twinkling lights of Los Angeles. The original scene is a six-minute uninterrupted take in the movie, and this coherence may be well suited to a re-presentation in the VR environment as it's usually hard to smoothly transit from one scene to another in VR. In addition, the tap-dance scene was shot at a mountain top during sunset and I am really eager to see what the cotton candy sky and the warm-toned street lamps look like in VR.
The user plays as Mia's role.
(1) The clip starts with Mia not being able to find her car, so she looks for it by pressing her key along the uphill road where there are many cars parked. This action provides a natural and logical precondition for Seb and Mia's first extended time together. Seb accompanies Mia along the road to find her car while they have a great time chatting. If without the context to find a car, their encounter and getting along would have seemed contrived and unprovoked. In VR, finding a car can be designed as a task that interacts with the environment using a prop - the key. The trigger on the joystick can be used to simulate the button on the car key. Then the player watches the environment for changes - headlights on along with unlocking beep sound.
(2) Mia and Seb walk to the top of the hill and then improvise a song with cryptic but expressive lyrics. Mia takes out her tap-dance shoes from her bag and wears them. This interaction between characters and their action progressively lead to the subsequent duo dance.
(3) After the dance, the background music switches from upbeat to soothing as the two get closer and look at each other. This suspense, although rather clichéd, never fails to evoke the audience's anticipation and excitement about the highly likely kiss scene. I thought it would be fun to experience it from the first-person perspective of Mia in VR.
IN VR:
Week 3 Importing a 360 Footage
Storyboard
Unity Project
I started off by scanning a 360 photo of my room and a straw hat as a separate 3D object. My story was inspired by Truman's world and I wanted to simulate its context where the player wakes up and finds himself/herself in the middle of being watched and supervised. The ceiling is equipped with spotlights and monitors, and cameras were placed all around the room to follow the player's movements. I found ideal models l in the Asset Store and added them onto the imported 3D image. One of the challenges I encountered was the limitation of the 3D image in adapting the viewpoint. I could not zoom in and out and could only have a panoramic view of the environment from the angle I took the photo. Another difficulty was operating the first-person view controller. I found that every time I entered the play mode, the scale and position of the game objects I placed would change. I presume this is also affected by the 360 photo, which I had no idea how to fix yet.
Week 2 Reproduce Two Scenes in SpongeBob SquarePants
Original scene in the cartoon
Original scene in the cartoon
For the first scene, I chose to reproduce the neighborhood in SpongeBob SquarePants. I found some free assets in the store and by changing the texture to make them better resemble their equivalent in the cartoon. I then went on to build paths to create an orientation. One highlight of the rightmost burger shop is the burger-shaped ornament on its roof and the burger street light on the left side as a hint about the functionality of the house. The biggest difficulty of the scene was its watery context as an undersea structure. I only managed to place a water plane on its top side (as shown in the bottom left picture), but what I really wanted to achieve was the effect of being fully submerged in water.
The second scene is an interior setting of the burger shop, where I first built a semi-enclosed space through planes and walls. Then I identified the most recognizable part of the scene - Squidward Tentacles's checkout counter. It was in the shape of a boat and I was lucky enough to find a similar asset. It then moved to the very concrete interior building with a lot of furniture that needed to be tightly placed in various locations, which took me forever. One thing I really want to improve about this scene is its lighting. I played with the lights a lot by changing the direction and shadows to get different results. I also noticed how different lighting colors can change the scene's mood drastically, but I failed to achieve a good color scheme because I was really unfamiliar with the lighting techniques.
Week 1 Creating a 3D Scene
My first Unity scene drew inspiration from the Moai monolithic statues. I created a secluded island covered by greenery to simulate an Easter Island vibe, with a lake in the middle half surrounded by mountains. To depict the rugged terrain of the wild island and dig out the central lake, I followed a tutorial to adjust the height of the landscapes, then added textures, trees, and grass. After getting the base built, I found similar-style statues in the asset store. I manipulated their scales and angles and scattered them all over the island. Another challenge was to place them precisely onto the rugged ground. During the process, it was not uncommon to switch from one viewpoint to another and found the statues hovering in the air. I spent a lot of time tuning the location parameters, and I was curious if there were some shortcuts or tricks that would help us quickly place objects. I also tried 3D text to give the island a name, but I ended up deleting it because I found that the placement of the 3D text was even trickier than the objects. It was in completely different positions in each perspective which confused me a lot. I'm looking forward to the future lab session that might cover this later.
Yoki, this is beautiful! I like your scene and story about it. I like that you create some path and enlarged the statutes to create some challenges.