Noriega Personal Project Proposal

Personal Proposal: Photogrammetry in 3D Animation

1.        Brainstorm

a. My favorite hobbies or pastimes are the following: reading, writing, drawing/painting, listening to music, messing around on Photoshop, and watching TV shows/films.

b. I am involved in the following studio processes: 3D animating, stop-motion, modeling, and pre-production in film.  

c. I am attracted to the following items and studio processes.

d. Some of the issues that have captured my attention are not limited to the following: the threat of AI to the legitimacy of the entrainment industry, the strong influence of technology on traditional art mediums and processes, the nationwide banning of literature in schools, the alarming decline in U.S. literacy rates, the highly polarized political environment in the United States, the emergence of cancel culture, and biodiversity loss/pollution. The issues that I am interested in incorporating in my project are the threat of AI (how people can bring back deceased individuals via AI) and the decline in traditional literature in schools.

2.       General Question:

How can laser scanning be used to take a preexisting object and make it into a 3D model for animating?

Background: I am currently making my 3D animation capstone project, in which I must collaborate with a team to create a three-minute animated film. One of the criteria is that we must model our own characters and environment. This initiated my interest in photogrammetry, in which I am curious to see if this 3D scanning method can be utilized to create realistic models that can be eventually animated. Even though I do not intend to incorporate photogrammetry into my capstone project, I do want to test its limitations and its effectiveness in the rigging and animating process.  

Refined Question:

How can photogrammetry be used to create a convincing 3D character for computer animation?

I will create a computer-based character ready for animation based on a pre-existing physical model. More specifically, I will make a short, animated clip of the great William Shakespeare reading a line or two from his famous work, The Tragedy of Hamlet, Prince of Denmark.  To accomplish this, I will use photogrammetry to capture the surface geometry of a head bust of Shakespeare and convert it to a closed manifold object. Next, I will upload it into Maya and rig the face; this will include creating a mouth bag, closable eyelids, eyebrows, facial muscles, etcetera. I will then upload an audio clip into Maya and complete the lip sync for the model. Lastly, I will compile it into a video and upload it as an MP4 file.

3.       Precedents and Literature Review

a.       Historical Precedents and Research That Relates Directly to Project: The Uses of Photogrammetry

Photogrammetry, the process of getting physical object dimensions from 2D images, has been utilized in a variety of different disciplines, such as forensics, gaming, mapping and digital recording terrains, and films and television. In games, like Star Wars Battlefront for example, photogrammetry has been utilized to efficiently capture some of the locations and props from the original series. Aside from being used to build the environment in a timely manner, photogrammetry was used to create the main assets of the game, like spaceships. In films, especially animated films, the photogrammetry has been used in a similar fashion, of taking a desired landscape, architecture, or environment and replicating it digitally in the form of a hyper realistic 3D model. An example of a TV series that utilized photogrammetry was Big Little Lies, in which instead of filming at the Monterey restaurant, they captured the inside of the building through photographs and built a digital version of it- this decision saved time, money and travel.

A stop-motion project that utilized photogrammetry for a different reason was a videogame titled Vokabulantis. The creators physically built the environment and characters and then utilized 3D scanning to maintain the stop-motion aesthetic. Most notably the game’s lead characters were created through this process to keep their congruent looks and to prevent them from looking too digitalized. Furthermore, photogrammetry has been used in VFX to create convincing doubles of actors and characters. The process includes taking hundreds of detailed pictures of the object and converting them into a convincing model that can be animated. In conclusion, photogrammetry allows for having realistic backgrounds and characters without having to reallocate resources in building them from scratch.

Resources:

https://visualskies.com/3d-scanning-for-film-and-tv/photogrammetry-for-vfx/#:~:text=In%20the%20realm%20of%20film,detailed%20and%20realistic%203D%20model.

https://www.artec3d.com/learning-center/what-is-photogrammetry

https://blog.frame.io/2021/06/14/photogrammetry-future-of-filmmaking/

https://www.autodesk.com/au/solutions/photogrammetry-software

b.      Significance Of the Project of Animation

The significance of the project is that it will test the limitations of models created through photogrammetry (instead of traditional modeling) and determine if it is indeed a better alternative to traditional digital modeling and sculpting. If the final product is convincing and can be rigged and animated, it will prove that photogrammetry can serve as a powerful tool when it comes to building environment and characters for 3D animation in a timely and financially- efficient manner. Furthermore, the project will prove that AI, thought quite effective, lacks the hand-made quality of traditional artistic practices.   

d. Criteria/Exemplars by Which the Project Will Be Compared

I would like my final product to be compared to the AI avatars that are available online and through apps and software like Adobe Firefly and Synthesia.

4.       Action Plan

a. Timeline to Finish Project:

October 18th - 25th: Use photogrammetry app to scan bust and convert surface geometry into a watertight 3D model in Maya.

October 25th - November 1st: Prepare the model to be animated- make sure the mouth, eyelids, and eyebrows can be moved. Complete facial rig.

November 1st - November 15th: Find and upload an audio clip. Complete lip sync.

    November 15th – November 29th: Build environment and render animation.

 b. Required Preliminary Research:  

The preliminary research that is necessary to complete this project involves figuring out how to set up and rig a model’s face in Maya, how to efficiently lip sync the model’s mouth with audio, and how to scan an object via photogrammetry.

      c. Organized Process:

1. Concept: Using traditional animation skills- instead of AI- to enable a sculpture to talk.  

2. Exploration of Various Solutions: Figure out how to take scanned geometry, rig it, and incorporate it in an animated sequence with audio. Figuring out how to lip sync the model to match the audio is also important. I will be using Maya tutorials, as well as previous class material, to complete these challenges.  

3. Final Product: A short, animated clip of the scanned object talking.

d. Necessary Processes:

The processes that are necessary for this project include animating, rigging, modeling, lip-syncing, and rendering in Maya, as well as 3D scanning with the photogrammetry app Polycam.  

e. Necessary Equipment:

Access to a quality camera and computer with Maya installed is necessary for this project.

f. Project Feasibility, “Deal Breakers,” And Alternatives:

The project is achievable, but can be time-consuming and difficult, especially the rigging process. If I need to, I can simplify the lip-sync and not go too in depth in accuracy. Rendering the final piece is not necessary, and an alternative that can save time is to PlayBlast the final sequence. In the worst-case scenario, I can scan a different model to create a walk cycle; the idea of testing the efficiency of scanned models in an animated sequence will still be applicable.