Depth sorting for node faces

[x2048]

This PR aims to implement correct painter's algorithm for transparent nodes on the map.

• Vertex indexes for all transparent faces are captured during mesh generation.
• BSP Tree is generated for transparent triangles.
• When rendering the map, BSP tree is traversed to generate correct (in most cases) rendering order, which is applied by replacing indexes in the mesh buffers as suggested by @hecktest.

This PR supersedes parts of #11130 and fixes most of cases of transparency problems (including transparent nodebox nodes at the cost of performance).

Handling Transparent Nodeboxes

In order to render overlapping nodeboxes correctly, this PR splits each nodebox at the edges of other nodeboxes along the same axis. This process produces a significantly larger number of 'atomic' nodeboxes that are guaranteed not to overlap, and may impact overall rendering performance. The total number of nodeboxes (defined + produced) is currently limited to 100.

To do

This PR is Ready for Review and testing

How to test

Use a combination of semi-transparent nodes such as:

• Alpha testing nodes in devtest
• Water
• Colored glass in Mineclone 2
• Stained glass

Transparent nodes should be rendered correctly with minimal temporary artifacts from all angles.

[x2048]

Tagging @appgurueu @Wuzzy2 @sfan5 @Andrey2470T @hecktest

[Andrey2470T]

Seems it has fixed the main transparency issue, transparent nodes are rendered behind other transparent ones now... but:

I used the colored glass from glass_stained mod and it caused that artifact, some parts of mesh faces of the nodes are black and those, as I understood, that are beyond the node area and also the right glass has a weird texture as if its edge pixels are 'stretched' in all four directions. Here's how it looks like in minetest without the bugfix:

[x2048]

I wonder if and how it is supported that a node's texture is tiled within the node. I replaced texture tiling with clamp-to-edge to catch cases of using tiling incorrectly, and I wonder if this case is a legitimate use of textures or just reliance on a bug.

[x2048]

I've now decoupled decision on tile merging from texture tiling and reverted changes to texture tiling, which fixed the artifacts above, and still allowed transparent water and ice to render correctly.

I've also added support for oversized nodeboxes, up to 3 nodes in each direction, which fixed a number of artifacts when looking through weird-shaped stained glass.

@Andrey2470T

[Andrey2470T]

Works fine, I see it is fixed now. 👍

However, when I was testing the PR on BOOM (by Juri) server where there are plenty of deep underwater cavities I noticed an immediate FPS fall up to '1' as plunging into the ocean:

[x2048]

Thank you for sharing. I have tried the server, and in some places there is a lot of transparent geometry to render.

I can do some optimizations, but correct transparent rendering currently should be expected to be 10 times slower compared to solid rendering due to the extra CPU operations, not using Z-buffer, extra drawcalls and lots of material swaps causing performance drop on the GPU. At some point, optimization will require a rewrite of the material system.

[x2048]

I've managed to go from 2 to 8 FPS on a scene in Boom with 200 nodes distance by only depth-sorting meshes within 75 nodes from the player. This causes minor effect of objects "swapping" or "popping up" at a distance as map blocks switch into full depth sorting mode, which plays into an overall story of "level of detail" when rendering the map.

That said, there are still scenes in Boom that cause 2-4 FPS on my laptop at 200 nodes distance due to the sheer amount of falling transparent water.

[Andrey2470T]

Now after the fix FPS has become higher on that server, even a bit higher 8 (varies within 10 - 13 FPS). Thanks for the fix.

[Andrey2470T]

I've discovered a wonderful bug with this PR (server Tunnelers Abyss):

Looks like as if somebody strongly extruded the colored glass panes to the outside. Also they periodically appear or disappear depending from the player's camera direction or can just blink.

[Andrey2470T]

So is this bug irrelevant to this PR if you told me you didn't discover this on Tunneler's Abyss? I don't see its display on any another builds, but only in your one. I doubt this problem concerns to my hardware or video drivers. So, I can conclude this bug hides somewhere in your PR's code.

[x2048]

I can confirm the bug, but it does not occur in 100% of cases.

[x2048]

@Andrey2470T The bug was caused by an uninitialized memory read due to reallocation inside std::vector and the reference to the current box not being adjusted. It should be fixed now, and I've also added a few improvements to the code to avoid performance problems in edge cases.

[appgurueu]

Could the BSP tree be made to take into account semitransparent particle & entity triangles?

[x2048]

I would not say it is impossible, but it is a significant step up in complexity. There are multiple reasons for this:

• Current BSP tree is per mapblock and is built on assumptions of a mapblock mesh (e.g. axis-aligned cubes). Building a generic BSP tree for free-form objects requires some good math and mesh manipulation on the fly (adding new vertices).
• BSP trees in general are expensive to build and are usually used for static meshes (like the map blocks). Putting a moving object into a BSP tree means rebuilding the tree on each frame, and it's at least O(N logN) on the number of triangles.
• Our current rendering process simply not fit for this at the moment, we would need to extract a deferred transparent rendering pass, and I am not sure Irrlicht is helpful here.

[Andrey2470T]

Is this PR ready for merging? In any case, I didn't found any other noticable bugs, except which I reported about before and which already fixed. The only current downside is a lower FPS, however generally it is not critical and quite playable.

[x2048]

This PR is ready to be reviewed by the core developers, there is no known technical debt / problems to be fixed.

[appgurueu]

Bump. This is very important for semitransparency to work among nodes. The rendering glitches before this are pretty critical.

[erlehmann]

@x2048 please rebase this branch on master so it can be compared to recent Minetest easier in terms of performance.

[ghost]

Yes. I have issues:

https://forum.minetest.net/download/file.php?id=25605

Transparent nodes besides other transparent nodes.=> The face is totally transparent.

[x2048]

@runsy Can you give more detail on how to reproduce it?

[wsor4035]

@x2048 seems it was taken from https://forum.minetest.net/viewtopic.php?p=406861#p406861 which appears to be regular mte and not this PR? follow up https://forum.minetest.net/viewtopic.php?p=407177#p407177

[appgurueu]

in the worst case

I always heard that minetets supported old equipment and blah blah blah. Let's keep it real.

You're the one not keeping it real by arguing using the worst case (the only valid point I could see is griefers abusing this to lower FPS of other players in certain areas). If semitransparent nodes are assumed to be randomly distributed and rather sparse, this worst case definitely won't come into play.

To give another example: Quicksort has a worst-case complexity of O(n²) as well, yet it is widely used, because the expected average running time if randomly picking pivots is O(n log n). Granted, in this case the complexity isn't determined by the data (the list) being "average", but rather the random not constantly picking the worst pivots.

[ghost]

So, what is the final decision?

It is necessary another dev to review this.

[x2048]

@runsy The distance of depth sorting will become adjustable, which will allow the client to avoid performance drops. The PR goes in, and we start work on OIT that would support both map, entities, clouds and particles. And be less sensitive to edge cases.

[sfan5]

@x2048 if you want this merged there's only two open comments from my side.

[SmallJoker]

Basically instant OOM while facing the following building near the spawn of Blocky Survival (mt.krypticbit.com 30005). This definitely need investigation.

Surrounding transparent nodes (possibly customized):

• homedecor:window_plain
• cblocks:glass_black
• default:glass
• default:ladder_wood
• default:steelblock_letter_wu (signlike transparent sign)

[x2048]

@SmallJoker good stuff, thanks for sharing

[x2048]

@SmallJoker OOM should be fixed now.

[x2048]

@runsy i've now added a setting transparency_sorting_distance to control the performance impact of transparency sorting.

[ghost]

@runsy i've now added a setting transparency_sorting_distance to control the performance impact of transparency sorting.

Coolz Thankz.

[SmallJoker]

Works. Code adjustments are welcome.

[sfan5]

Gave it a quick test, I don't have any intensive test scenes so I trust @SmallJoker with that.