Runtime Meshes in UE

The Road To Geometry Scripting Plugin

A dive into how to create and update meshes at runtime in Unreal Engine:

UProceduralMeshComponent is the historical option here but now with newer versions of the engine we can generate and manipulate UStaticMesh as well, so the question is what should we use?

Our focus will be on performance and to be able to take the decision and say which component should we use (UProceduralMeshComponent  or UStaticMeshComponent), we have to understand the underlying bits of each component. Generally speaking, to update a mesh at runtime it needs to be built, meaning the rendering representation of that mesh needs to be created or updated. Unreal Engine uses a proxy class to handle that, it doesn't directly use the data inside UProceduralMeshComponent or inside the UStaticMeshComponent classes, that proxy is of the type FPrimitiveScenceProxy which will create one or more FMeshBatch from the component's data.

First Option:  UProceduralMeshComponent

In UProceduralMeshComponent, it somewhat straightforward, the FProceduralMeshSceneProxy converts the FMeshProcSection and initializes FStaticMeshVertexBuffers and FLocalVertexFactory which store data that the GPU will need. To the GPU, the data is not different whether it is a ProceduralMesh or a StaticMesh, the difference lies in how these data gets to the GPU.

Second Option: UStaticMeshComponent

For a UStaticMeshComponent, it references a UStaticMesh which holds the source mesh as FMeshDescription which is cooked into an optimized rendering mesh that is used to initialize the FStaticMeshSceneProxy. The source mesh (FMeshDescription) is not used after cooking and is stripped out of the built game since the cooking process done by UStaticMesh::Build() is dependent on some editor-only functions and data. But in newer engine versions (4.25) a new function was added that allowed a runtime generation of the static mesh UStaticMesh::BuildFromMeshDescriptions() which takes a collection of FMeshDescription and initializes the mesh rendering data. This runtime building differs from the editor-only build in that it skips complex steps that are costy and unnecessary to do at runtime such as light map UVs.

Calling UStaticMesh::BuildFromMeshDescriptions() is more expensive than updating sections in a UProceduralMeshComponent. The trade-off here is that you can re-use the generated UStaticMesh in multiple UStaticMeshComponents and make use of instanced rendering unlike UProceduralMeshComponent of which if you have multiple same instances, they will be copied. Generally, UStaticMeshComponent is better supported throughout the engine which leads us to one more fundamental difference, the draw path that is used in rendering those meshes. UProceduralMeshComponent uses the Dynamic Draw Path which is a add a performance cost as it sends FMeshBatches each frame to be drawn, while UStaticMeshComponent uses the Static Draw Path which tells the renderer that its data are not going to change, allowing caching and optimization to be done.

Now that we've seen some of the differences, we can say that static mesh are better BUT ONLY IF we're interested in loading generated meshes as it gives us better rendering performance with a high up-front cost, meaning it is not the best option if we intend to actively edit that mesh each frame.

A Third Option: UDynamicMeshComponent

The guys at Epic introduced a new mesh component UDynamicMeshComponent, this component is like UProceduralMeshComponent in that it also uses the Dynamic Draw Path but it is designed to update faster. However, unlike UProceduralMeshComponent, the new UDynamicMeshComponent stores more complex mesh representation and internally handles rewriting the mesh data as something suitable for the GPU and that is -to my understanding so far- the key to runtime mesh manipulation. This complex representation is of the type FDynamicMesh3.

In terms of trade-offs, this new component generates larger render buffers which puts some load on the GPU memory and it also doesn't currently have any physics support.

OK! so now we have 3 options (UStaticMeshComponent , UProceduralMeshComponent, and UDynamicMeshComponent), each stores the mesh in a different way (FMeshDescription, FProcMeshSection, and FDynamicMesh3). The best way think about it is how different each one of them renders its data.

That is basically the initial thoughts that led to creating the Geometry Scripting plugin inside Unreal Engine.