Streamable

[atteneder]

Make loading glTFs streamable, so the already received part of the content can be parsed while the rest is loading.

• Detect Binary/JSON
• Parse JSON ahead
• Fill individual BufferViews/Accessors instead of single Buffer
• Start buffer to data conversion ahead
  • Vertex/Index/Matrix Buffers
  • Textures

Related:

• Partial / Selective loading and instantiation #318
• Streamable instantiation: Instantiate right after parsing but before loading is done and have the content (meshes, textures) appear once they are there.
  • Texture streaming
    • Instantiate first and assign textures once they are ready
    • Mipmap streaming: stream in low levels first and improve resolution on the go

[ROBYER1]

Does this apply to animation? I am looking into streaming GLTF animations so that the first frame or two can be previewed before the full animation. Is this feasible?

[atteneder]

Does this apply to animation? I am looking into streaming GLTF animations so that the first frame or two can be previewed before the full animation. Is this feasible?

@ROBYER1 I love that idea! From the top of my head:

• Initial pose (which is not necessarily the first frame): I think that's feasible, yes!
• First frame:
  • Via Unity API (AnimationClip): I doubt it. We'd have to add keyframes as we go, which I'm afraid is not supported
  • Via custom animation system: The sky is the limit, but that is not planned
  • Compromise in-between: Manually set first frame pose and assign the complete AnimationClip later: Totally feasible.

I'm asking myself what kind of performance gains would be possible if animation could be streamed. glTF officially did not consider stream-ability in its design, but it can be somewhat stream friendly. This would only benefit glTFs where the animation payload is a noticeable fraction of the overall load, right? I'm always happy when someone presents to me real-life use-cases that matter to them/many, so feel free to do so.

[ROBYER1]

Mainly looking at this from a mobile app bandwidth perspective as potentially up to 100mb bandwidth usage+ I have been informed is quite pricey 😅

I did some R&D on separating the model and animation so that the animated armature as a gltf can be sent over separately to a locally cached 3d model, which works great. I was looking at reducing the bandwidth usage of the app further by possibly streaming in the animations so that if the user was browsing a selection of animations, we could stream in the first frame or two for a still preview before allowing them to stream the full animation. The animation data I would say is the largest file size of the application as the users can look through various complex animations, which we will of course cache as they are viewed up to an extent.

I am looking into the compression/decompression of the animation and the 3d model (separately) also to reduce potential bandwidth usage in those areas, but streaming is also a consideration at this point.

[atteneder]

@ROBYER1 Thanks for your input.

fwiw: You can already compress animation today via EXT_meshopt_compression

[ROBYER1]

Thanks, I'm looking into both that and Draco tomorrow to test out. I'll report back if there are any other use cases where things can improve but overall very pleased with GLTFast 😎

[atteneder]

Loosely related. The upcoming version 6.7.0 will feature API to load glTF from a Stream. While the implementation underneath does not take full advantage of that yet, it's a humble start. I mention this here in case users who want to work with Streams landed here.