Rendering

Once the scene was set, it was time to start rendering. Unfortunately, this process caused me unforeseen troubles once again, mainly in regards to temporal samples and motion blur.

sequencer

The sequencer in Unreal Engine is a similar to the camera sequencer in Maya but better in every way. The Master Sequence (as seen below) contains all of the shots in the film, but unlike the Maya sequencer, each shot is essentially its own scene file.

Master Sequence

 Global items like the environment and lighting etc, persist across all shots, but then specific items can be spawned or animated within based on the shot itself.

To the right is the Shot_0140 sequence. Items with an orange lightning bolt delineate spawnable objects, i.e. objects that only appear for this shot. 

At the top of the list is the camera cut. This determines what's seen in the Master Sequence as well as time for engine warm up frames (more on that later). The shot specific camera is next as well as skeletal mesh actors for the Bow, Sylvanas and the Goliath. This shot contains one of the pillar destructions, so there are many items calling for the chaos cache and destruction FX. 

At the bottom you can see a folder for hiding the original intact pillars

Shot_0140 Sequence

Shot_0140 Bin

Above is the bin for all the shot_0140 items. This includes the shot sequence itself, character animations and blueprints necessary to create the pillar destruction.

The Master Sequence, played from an external camera

This video demonstrates the change in locations and lighting depending on the shot. In fact, until I viewed the master sequence in this way for the video, I didn't realise that Sylvanas retained the incorrect scale for shot_0040. Thankfully this wasn't noticeable in camera. 

Camera settings

There were several camera settings I had to consider for every shot, some that change and some that needed to remain consistent. These included Filmback, Focal Length, Aperture and Aspect Ratio.

I had decided during Previz that in order to achieve a cinematic look I was going to shoot with an aspect ratio of 2.39:1. In Maya, I used a resolution of 1920 x 822 which would create the desired aspect ratio. However in Unreal, changing the resolution would only change the output and not the preview.

Changing the sensor size to 30.3 x 13.3 would correct the aspect ratio but after doing some research into filmback this felt incorrect as well. Eventually I landed on a 36mmx22.25mm sensor size with a 2:39 crop.

I used a universal zoom setting but typically stuck to focal lengths between 25 and 110mm, occasionally bumping to an extreme of up to 200mm. My aperture and focal distance varied on a shot by shot basis.

post processing

In addition to the camera settings, there are certain post processing effects that can be applied to shots either universally or per shot. My goal with post processing was to eliminate the need for any sort of post production VFX, opting to render final shots straight out of unreal. These post processing effects consist of Lens Flares, Bloom, Motion Blur, Film Grain and Chromatic Aberration. Most of these effects were controlled through an unbound post processing volume that applied to every shot, and then changed occasionally in individual camera settings.

I used used the default motion blur setting of 0.5 and tried to keep the other effects subtle. As I will discuss momentarily, achieving a clean motion blur was more difficult than just ticking a box.

movie render queue

The movie render queue is used to adjust settings and render shots. 

Within the settings for each shot, it's possible to choose the export file type, resolution, anti aliasing options, console variables and more.

For my purposes, I opted for an .exr sequence with a temporal anti aliasing method and samples set to 1. Temporal samples and anti aliasing proved to be a massive headache for me when it came time to render, and it took a lot of testing to decide on these settings.

Console variables are specific and advanced commands you can input that affect your final render. I experimented with many but eventually only used  r.TemporalAA.Upsampling with a setting of 0

Render Settings

Anti-Aliasing and Motion Blur:

Shot_0050 with half motion blur and 8 Temporal Samples causes the cloth to act funky. Also affects the flame FX in an odd way

Anti-Aliasing was a big issue for me due to the way in which it affected motion blur and its relationship with my cloth simulations.

In order to achieve good motion blur in Unreal, you need to use a high temporal sample count. Temporal sub sampling evaluates the subframes of your render to more accurately depict the motion blur on the final frames. In most cases, this is not a problem. However for me, temporal sampling created issues in regards to the evaluation of my Skeletal Mesh Cloth.

Unlike other unreal simulations, the Skeletal Mesh cloth cannot be baked, and therefore is evaluated on every frame when it comes time to render. But, with temporal samples, the cloth was getting re-evaluated on every subframe, which resulting in glitchy popping cloth that was unusable.

I was also experiencing serious ghosting within the renders (which I will expand upon in the next section) but between this and my motionblur/cloth issues, I ended up producing over 30 versions of Shot_0050 and around 17 versions for each of Shots 0020, 0030, and 0040. 

The folders to the right show my attempts at shot 0050. MB stands for motion blur and I produced renders testing with full, half and no motion blur. ss and ts stand for Spatial Samples and Temporal samples, whilst the 'NONE' references the anti aliasing method. There are 5 methods available within Unreal including Fast Approximate Anti-Aliasing (FXAA), Temporal Anti-Aliasing (TAA), Multisample Anti-Aliasing (MSAA) and Temporal Super Resolution (TSR). I was testing many different methods across multiple shots trying to find a combination that produced the best results.

The change in frame rate referred to the evaluation tick rate within the shot sequencer. All shots were rendered at 24fps but I experimented extensively with the increased evaluation times hoping that with more frames to evaluate on, I could achieve smoother cloth. I also tried locking the framerate at runtime to hope that the cloth would not evaluate on subframes. Neither of these things worked.

One weird and unexplainable link was between the motion blur and the cloth. I was able to achieve smooth cloth with temporal samples but only when the motion blur was set to a value of 1. This was annoying because whilst I now had smooth motion blur it was set far too high and was equally unappealing.

Shot_0050 Render Attempts

I also experimented with various console variables (CVARs). From my research, people with similar problems said that the CVAR 'p.ChaosCloth.UseTimeStepSmoothing 0' would solve the problem. From my tests it made little difference. Other CVARs I experimented with was Screen Percentage which essentially renders the shot with a higher resolution for more accurate motionblur before compressing it back down. And Anti-Aliasing Upsampling which didn't do anything to help with my motion blur issues, but was very useful in achieving a crisper image.

https://docs.unrealengine.com/5.2/en-US/anti-aliasing-and-upscaling-in-unreal-engine/

Motion blur in nuke with AOVs

As much as I didn't want to do any nuke work, the motion blur was causing me such an issue that I experimented with Motion Vector AOVs. Maybe its because my process was incorrect or the AOVs aren't ideal, but adding the motion blur in post didn't really satisfy me.

It felt like the effected areas were too sharp around the edges and the motion blur seemed to be active even when the character wasn't moving. I could have tried to solve this with Roto shapes but figured that if this was going the the case for every shot that would amount to a lot of work.

Eventually, I landed on a Temporal Anti Aliasing method with 1 spatial sample, 1 temporal sample and AA Upsampling. 

Here is documentation detailing the issues between temporal samples and cloth simulation: Unreal Engine Issues and Bug Tracker (UE-172069) 

Ghosting:

Whilst I was trying to tackle my motion blur issues, I was also trying to solve a serious ghosting issue within my renders. For some reason, after-images were visible as the character moved through the shot. This made the frame look mushy and I initially believed it to be tied to the motion blur issue. This was particularly apparent it shots 0030 and 0050.

Shot_0030_v005_Ghosting

Shot_0030_v017_Fixed

As I performed more render tests, I started to realise that the ghosting issue was not really linked to the motion blur problem, as ghosting still occurred when motion blur was disabled. It was then that I started to think about other potential causes, and why the ghosting was worse on certain shots.

I realised that the ghosting occurred more severely in shots where the camera was closer to the ground, which helped me to uncover the root of the problem. Volumetric Height Fog. This, along with my godrays, was responsible for the ghosting as the character passed through the volumetrics. To solve this, I reduced the intensity of my fog and increased the samples. Unfortunately, there was still some ghosting but it was far less noticeable, and at a point where I could live with it. 

other issues

Once I had solved the initial render issues, things were relatively smooth sailing. There were some minor issues regarding DOF and its relationship with transparent textures, but I solved those fairly quickly.

Other than that, one of the only real rendering issues I faced was a problem with culling. 

In the opening Shot, the hammer is seen on the floor and the Goliath is suspended in the air out of shot. But when it came time to render I found that the hammer would not appear.

After some tests, I realised that the hammer would only appear when the goliath was in the frame.

Looking into the issue, I found that culling occurs when an object is too far away from its root or is lacking collision capsules. Checking the physics version of the mesh, I discovered that the hammer itself was lacking a collision box. After adding a collision box to the head of the hammer, the problem resolved itself.