Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move malloc_trim invocations to background thread #14744

Merged
merged 2 commits into from
Jun 17, 2024

Conversation

sfan5
Copy link
Member

@sfan5 sfan5 commented Jun 10, 2024

fixes #14737

@sfan5 sfan5 added the Bugfix 🐛 PRs that fix a bug label Jun 10, 2024
Copy link
Member

@Desour Desour left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested with /emergeblocks (-1000,-1000,-500) (0,0,0), by adding a TimeTaker.

In master it does several trims that cost about 4 to 30 ms. With PR, it's only one, but it takes about 400 ms.
Even in not so extreme examples, one would get lag spikes of > 1 frame every MEMORY_TRIM_INTERVAL seconds (if I just stand around in devtest, I easily get 13 ms, with PR). Needless to say, this is not good for a game.

Something else that I saw is that the drawtime doesn't seem to be affected by the lag spike. So other threads don't seem to be heavily affected by the trim call.

src/porting.cpp Outdated Show resolved Hide resolved
src/porting.cpp Outdated Show resolved Hide resolved
@sfan5
Copy link
Member Author

sfan5 commented Jun 12, 2024

With PR, it's only one, but it takes about 400 ms.
Even in not so extreme examples, one would get lag spikes of > 1 frame every MEMORY_TRIM_INTERVAL seconds

Hmm, that's sort of the same as the initial problem.
Where the lag spike happens might depend on which threads ends up calling malloc_trim. I guess we would have to move it to some background thread.

@lhofhansl
Copy link
Contributor

Hmm, that's sort of the same as the initial problem.

Until we have a satisfactory solution, can we please revert the original change and not call malloc_trim at all.
The issue that started all this was about a server that ran for many days, that kind of server one should have a beefy machine anyway.

I'm happy if we call malloc_trim once an hour - that would still satisfy the base issue, and a 400ms blip per hour is OK.
Another alternative is to limit this to the server, i.e. do not track the mesh-updates at all. (Won't make a difference in Singleplayer, though).

In the end we're creating new problems for something that has not been a problem for many years, is only a problem for long running servers, and is at best a nuisance anyway.

Create an admin command to call malloc_trim if admins want this.

@sfan5
Copy link
Member Author

sfan5 commented Jun 14, 2024

Like I argued in #14737 this can't be reduced to "some unimportant problem that happens if you run servers for days".

Anyway, I switched this to a new approach. @lhofhansl @Desour can you two re-test?

@sfan5 sfan5 changed the title Rate-limit malloc_trim calls Move malloc_trim invocations to background thread Jun 14, 2024
@lhofhansl
Copy link
Contributor

I guess we just have to agree to disagree on whether this is useful or not.

With that said, I retested this and it works fine. No new jitter observed. So +1.

@sfan5 sfan5 modified the milestones: 5.10.0, 5.9.0 Jun 15, 2024
Copy link
Member

@Desour Desour left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would make more sense to do the trimming in a separate background thread, so to not cause unnecessary latency in meshgen or mapgen sometimes.

But this is better than in master, so 👍 .

(Also, maybe a purely time-based solution would be simpler and good enough. After all, I just have to walk 10 nodes or so in one direction to trigger a trim call.)

@sfan5 sfan5 merged commit fac9aac into minetest:master Jun 17, 2024
13 checks passed
@sfan5 sfan5 deleted the malloctrim2 branch June 17, 2024 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

After #14707 I observe increased visual jitter, in singleplayer mode
5 participants