Create Primitives as sub-mesh of a combined Mesh instead of separate GameObjects/Mesh

[chrisssssy]

Hi again @atteneder ,

when loading a gltf-file where meshes have multiple materials, the meshes get splitted into multiple submeshes, one for each material. Is this the intended way of handling these meshes? Other importers don't split the meshes as far as i could test.

greetings
Christopher

[atteneder]

Hi @chrisssssy,

Let me paraphrase, just so I don't get it wrong:

You observed that glTFast instantiates one GameObject/Mesh per glTF-Primitive. You'd expect one GameObject/Mesh with one Submesh per glTF-Primitive, right?

What problem of yours would that solve or improve?
Do you have a test object at hand?

Technically this would be doable for all glTF meshes where the primitives have common vertex attributes. The glTF spec allows any wild mix of vertex attributes, so I think the current implementation is like it is, since it works for all cases.
It's save to assume that creating less GameObjects is faster and uses less memory, but I assume the gains are very small and only for certain types of assets.

Open for feedback, but we'd have to balance that when discussing such changes.

Thanks,
Andi

[chrisssssy]

Hi @atteneder ,

i would expect that gltfast doesn't split up the mesh into multiple submeshes so the hierarchy and object/mesh count stays the same as in blender. In the following picture you can see how for example Siccity/GLTFUtility handles my gltf-file. The hierarchy and object/mesh count is the same as in blender.

What problem of yours would that solve or improve?
Do you have a test object at hand?

Here is a test file with 2 meshes (the original file has over 200 of similar objects), each with multiple materials assigned. gltf_multiple_materials.zip

I would like to attach colliders to each object, so i can later raycast the scene, select objects and move them arround. I think adding those colliders is much simpler if there are no submeshes to deal with. Otherwise i have to attach colliders to all the submeshes as well and somehow merge them to a single collider again. Because i dont want to select the submeshes but the whole object. But maybe i'm missing something here and this isn't that big of a hassle as i think.

thanks for your quick help as always.

Christopher

[atteneder]

Oh, I see. I'll have a look if this is a low hanging fruit.
Thanks for giving the details!

[atteneder]

I have looked into this now and while I see that this would make your task easier, unfortunately this is not quick/easy to change.

glTF's mesh structure is more flexible than Unity's. You can have (theoretically) primitives with different vertex attributes data and/or structure. So for some corner cases I'd have to split the submeshes anyways.

Doing this right (so that it stays highly performant) would take a couple of days. I'll come back to this, but right now there are more important goals.

How to solve your particular problem:

1. Combine meshes after instantiation. You'd have to implement a custom instantiator, since after instatiation the relation (is this a primitive gameobject or not) is lost.
2. Do not combine the objects, but preserve the logic relations somehow (e.g. custom monobehaviours) and teach your other scripts (that do selection or moving) how to interpret them.

Both those options do not appeal to me personally. Here's what I would do:

Think about abolishing the limitation that only single-mesh objects can represent a connected, movable object. Instead find a way to flag or mark certain nodes as being root nodes of an object that is selectable/movable. Examples of such indirect rules:

• Nodes with a certain identifier in their name are root nodes
• Only put one object in one glTF file, so you can treat the whole glTF as one object
• Declare scene-root-level nodes to be root nodes
• Use glTF Scenes as separator of individual objects

Create a custom glTF extension is also an option, but for sure the hardest one and I wouldn't recommend that.

That being said, I'll keep this issue open, as I eventually want to tackle this some day.

hth

P.S.: I used your model to compare glTFast to GLTFUtility for the first time. glTFast was between 2 and 10 times faster in the Editor. Even when you're doing a mesh combine after loading, glTFast will probably be faster.

[chrisssssy]

Thank you for looking into this issue and for providing different solutions. I will look into them.
Great to hear about the good performance. Thats one more reason to stick with glTFast.

[atteneder]

While implementing animated morph targets ( #8 ) I noticed that the current behavior makes the task harder.

An weights animation clip drives a glTF node/mesh. Since glTFast splits up primitives into separate meshes and parents them underneath the original mesh node, all animation curves have to get duplicated with a fitting animation path.

Another reason to fix this at some point.

[carking1996]

This is a big one for me, and a current reason of not using it right now. I'm using another GLTF importer which is unfortunately slower but doesn't split the meshes up by material. I'm curious when this will be ready for this project.

[atteneder]

This is a big one for me, and a current reason of not using it right now. I'm using another GLTF importer which is unfortunately slower but doesn't split the meshes up by material. I'm curious when this will be ready for this project.

@carking1996 Sorry that I cannot provide this to you currently. There's no definite plan, so work will for sure not start before July. I'll keep you posted in this issue, so please subscribe.

[RicardoVRTom]

Just to add my voice to the discussion, it would be great to have an import option for preserving the sub-meshes and material assignment.

I work with CAD data almost exclusively, and a fast runtime exporter that can handle highly complex models (often > 5000 mesh components in a single file) like this is amazing to have. CAD data often contains multiple materials on a single mesh to distinguish between different surfaces on the part (Machined faces, untreated faces, different components in a merged assembly etc). I had previously been using Unity's PiXYZ loader tool to load model data at runtime, however since Unity deprecated that tool, gLTFast is now their recommended alternative!

[atteneder]

@RicardoVRTom Thanks for the valuable input!

I hope after milestone release 5.x I'll find time to work on some basics like this again.

[Daniel4144]

I am facing the same problem:
I also use CAD models with different materials on different surfaces. Because I need a meshcollider for each object, I have to combine the primitives before creating the colliders or else the colliders behave unreliable.

At the moment I combine them after instantiation manually with an ugly workaround (searching for nodes starting with "Primitive_", combine them with CombineInstance/Mesh.CombineMeshes and delete the leftover primitive gameobjects).
Would be great to have them combined by default.

[luizcarlosfx]

Any update on this? Keeping the original object hierarchy is extremely important for my app. I'm using GLTFUtility plugin for now, but glTFast is much faster, so I'm really looking for this feature

[luizcarlosfx]

I'm trying to create a custom instantiator, but I'm not sure how to deal with skinned meshes. I mean, I can still combine the meshes, but what would be the correct skinnedMesh.bones value, considering that each submesh have a joints property (that might be different?)

[luizcarlosfx]

I finally decided to work on this, and if anyone is still interested here's my solution

CombineMeshInstantiator.cs

using System.Collections.Generic;
using System.Linq;
using GLTFast;
using GLTFast.Logging;
using Unity.Collections;
using UnityEngine;

namespace GLTFast
{
	internal struct PrimitiveData
	{
		internal uint NodeIndex { get; set; }
		internal string MeshName { get; set; }
		internal Mesh Mesh { get; set; }
		internal int[] MaterialIndices { get; set; }
		internal uint[] Joints { get; set; }
		internal uint? RootJoint { get; set; }
		internal float[] MorphTargetWeights { get; set; }
	}

	public class CombineMeshInstantiator : GameObjectInstantiator
	{
		private readonly Dictionary<uint, List<PrimitiveData>> _primitives = new Dictionary<uint, List<PrimitiveData>>();

		public CombineMeshInstantiator(IGltfReadable gltf, Transform parent, ICodeLogger logger = null, InstantiationSettings settings = null) : base(gltf, parent, logger,
			settings)
		{
		}

		public override void AddPrimitive(uint nodeIndex, string meshName, Mesh mesh, int[] materialIndices, uint[] joints = null, uint? rootJoint = null,
			float[] morphTargetWeights = null,
			int primitiveNumeration = 0)
		{
			if ((m_Settings.Mask & ComponentType.Mesh) == 0)
				return;

			if (!_primitives.TryGetValue(nodeIndex, out List<PrimitiveData> primitives))
			{
				primitives = new List<PrimitiveData>();
				_primitives.Add(nodeIndex, primitives);
			}

			primitives.Add(new PrimitiveData
			{
				NodeIndex = nodeIndex,
				MeshName = meshName,
				Mesh = mesh,
				MaterialIndices = materialIndices,
				Joints = joints,
				RootJoint = rootJoint,
				MorphTargetWeights = morphTargetWeights
			});
		}

		//TODO: check how to handle instanced meshes
		public override void AddPrimitiveInstanced(uint nodeIndex, string meshName, Mesh mesh, int[] materialIndices, uint instanceCount, NativeArray<Vector3>? positions,
			NativeArray<Quaternion>? rotations,
			NativeArray<Vector3>? scales, int primitiveNumeration = 0)
		{
			base.AddPrimitiveInstanced(nodeIndex, meshName, mesh, materialIndices, instanceCount, positions, rotations, scales, primitiveNumeration);
		}

		public override void EndScene(uint[] rootNodeIndices)
		{
			foreach (uint nodeIndex in _primitives.Keys)
			{
				List<PrimitiveData> primitiveList = _primitives[nodeIndex];
				PrimitiveData first = primitiveList[0];

				if (primitiveList.Count > 1)
				{
					var combine = new CombineInstance[primitiveList.Count];
					var materialIndices = new int[primitiveList.Count];
					uint[] joints = null;
					bool hasMorphTargets = first.MorphTargetWeights != null;

					if (first.Joints != null)
						joints = primitiveList.SelectMany(p => p.Joints).ToArray();

					for (var i = 0; i < primitiveList.Count; i++)
					{
						PrimitiveData data = primitiveList[i];
						Mesh primitiveMesh = data.Mesh;
						combine[i] = new CombineInstance { mesh = primitiveMesh };
						materialIndices[i] = data.MaterialIndices[0];
					}

					var mesh = new Mesh { name = first.MeshName };

					mesh.CombineMeshes(combine, false, false);

					if (hasMorphTargets)
						CombineMeshUtils.CombineBlendShapes(mesh, primitiveList);

					first.Mesh = mesh;
					first.MaterialIndices = materialIndices;
					first.Joints = joints;
				}

				base.AddPrimitive(nodeIndex, first.MeshName, first.Mesh, first.MaterialIndices, first.Joints, first.RootJoint, first.MorphTargetWeights, 0);
			}

			base.EndScene(rootNodeIndices);
		}
	}
}

CombineMeshUtils.cs

using System;
using System.Collections.Generic;
using IV.XRProj;
using UnityEngine;

namespace GLTFast
{
	public class CombineMeshUtils
	{
		class BlendShapeFrame
		{
			internal Vector3[] Vertices { get; }
			internal Vector3[] Normals { get; }
			internal Vector3[] Tangents { get; }
			int VertexIndex { get; set; }
			internal float Weight { get; set; }
			internal int Index { get; }
			internal string Name { get; }

			internal BlendShapeFrame(string name, int index, int vertexCount)
			{
				Name = name;
				Index = index;
				Vertices = new Vector3[vertexCount];
				Normals = new Vector3[vertexCount];
				Tangents = new Vector3[vertexCount];
			}

			internal void AddVertices(Vector3[] vertices, Vector3[] normals, Vector3[] tangents)
			{
				Array.Copy(vertices, 0, Vertices, VertexIndex, vertices.Length);
				Array.Copy(normals, 0, Normals, VertexIndex, normals.Length);
				Array.Copy(tangents, 0, Tangents, VertexIndex, tangents.Length);
				VertexIndex += vertices.Length;
			}
		}

		internal static void CombineBlendShapes(Mesh combinedMesh, List<PrimitiveData> primitiveList)
		{
			var frames = new List<BlendShapeFrame>();

			BlendShapeFrame GetBlendShapeFrame(string name, int index)
			{
				foreach (BlendShapeFrame frame in frames)
				{
					if (frame.Name == name && frame.Index == index)
						return frame;
				}

				var newFrame = new BlendShapeFrame(name, index, combinedMesh.vertexCount);
				frames.Add(newFrame);
				return newFrame;
			}

			foreach (PrimitiveData data in primitiveList)
			{
				Mesh primitiveMesh = data.Mesh;
				int vertexCount = primitiveMesh.vertexCount;
				var vertices = new Vector3[vertexCount];
				var normals = new Vector3[vertexCount];
				var tangents = new Vector3[vertexCount];

				for (var shapeIndex = 0; shapeIndex < primitiveMesh.blendShapeCount; shapeIndex++)
				{
					string shapeName = primitiveMesh.GetBlendShapeName(shapeIndex);

					for (var frameIndex = 0; frameIndex < primitiveMesh.GetBlendShapeFrameCount(shapeIndex); frameIndex++)
					{
						float weight = primitiveMesh.GetBlendShapeFrameWeight(shapeIndex, frameIndex);

						primitiveMesh.GetBlendShapeFrameVertices(shapeIndex, frameIndex, vertices, normals, tangents);
						BlendShapeFrame frame = GetBlendShapeFrame(shapeName, frameIndex);
						frame.Weight = weight;
						frame.AddVertices(vertices, normals, tangents);
					}
				}
			}

			foreach (BlendShapeFrame frame in frames)
			{
				combinedMesh.AddBlendShapeFrame(frame.Name, frame.Weight, frame.Vertices, frame.Normals, frame.Tangents);
			}
		}
	}
}

I hope you will still work on this @atteneder, because I'm pretty sure this is not the best solution and certainly not the most performant. it's still far better than the GLTFUtility importer though

[luizcarlosfx]

One more thing to notice, I still don't know why, but this code will not work for blend shapes if gpu skinning is enabled in the project settings

[RicardoVRTom]

@luizcarlosfx Thanks for sharing, I'll have to take another look at this.

[LaneF]

This is a blocker for us too, since the segmented meshes have the same instanceId we can't guarantee the correct mesh is assigned when manually instantiating meshes from a data blob/manifest. It's just not an elegant solution to divide the meshes.

[andybak]

I stumbled across this while working on replacing the old GTLF import code on Open Brush.

I originally added GLTFast for import. Then as UnityGTLF hit a bit milestone I added that for export as it seemed more capable in that department.

I recently added an option to use UnityGLTF for import as well. I plan (for a while at least) to make them both available as they both have different strengths and weaknesses and are both potentially useful for our users. (one day, maybe the two projects could coordinate and join forces a bit - thanks ;-) )

Anyway - for this reason I spotted a discrepancy between how they both handle submeshes and in digging a bit deeper I found this issue. At first glance it feels like UnityGLTF's behaviour is more correct but I do appreciate the edge cases mentioned here: #153 (comment)

[hybridherbst]

Fwiw I believe UnityGltf handles these cases correctly, but of course if you find an issue there please let us know.
(UnityGltf values roundtripping and staying in Unity’s philosophy over raw speed, might not be possible to have both)

[luizcarlosfx]

@andybak My custom instantiator that combines the meshes work perfectly and even using it the performance is still far superior to UnityGLTF. Check in the comments above