Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Close uv_idle handles when we're done with them #39268

Merged
merged 1 commit into from
Jan 15, 2021
Merged

Close uv_idle handles when we're done with them #39268

merged 1 commit into from
Jan 15, 2021

Conversation

Keno
Copy link
Member

@Keno Keno commented Jan 15, 2021

uv_idle handles are a bit dangerous because they essentially prevent
the uv loop from going to sleep, so if there's no work to be done
on the julia side, it just turns into a busy loop. We start such
an idle callback in the threads.jl test. We make sure it goes out
of scope, so the GC will eventually close it, but in the meantime
they keep the loop spinning. Now, unfortunately, the next test in
line is the Distributed test, which runs everything in a subprocess,
so it never builds up enough memory pressure to actually run the
GC, so the idle handles never get closed. This isn't too big a
deal, as the worst thing that happens is that it hogs one CPU
core on CI while the threads test is running, but we don't have
enough parallelism there anyway, so I don't expect a meaningful
impact on CI. It does however blow up our rr traces and apparently
causes significantly problems when trying to replay them on pernosco.
This is easy to fix by just closing these handles once we're done
with them.

Reported by @rocallahan

uv_idle handles are a bit dangerous because the essentially prevent
the uv loop from going to sleep, so if there's no work to be done
on the julia side, it just turns into a busy loop. We start such
an idle callback in the threads.jl test. We make sure it goes out
of scope, so the GC will eventually close it, but in the meantime
they keep the loop spinning. Now, unfortunately, the next test in
line is the Distributed test, which runs everything in a subprocess,
so it never builds up enough memory pressure to actually run the
GC, so the idle handles never get closed. This isn't too big a
deal, as the worst thing that happens is that it hogs one CPU
core on CI while the threads test is running, but we don't have
enough parallelism there anyway, so I don't expect a meaningful
impact on CI. It does however blow up our rr traces and apparently
causes significantly problems when trying to replay them on pernosco.
This is easy to fix by just closing these handles once we're done
with them.
@Keno Keno merged commit 35fcfda into master Jan 15, 2021
@Keno Keno deleted the kf/close_idle branch January 15, 2021 18:50
ElOceanografo pushed a commit to ElOceanografo/julia that referenced this pull request May 4, 2021
uv_idle handles are a bit dangerous because the essentially prevent
the uv loop from going to sleep, so if there's no work to be done
on the julia side, it just turns into a busy loop. We start such
an idle callback in the threads.jl test. We make sure it goes out
of scope, so the GC will eventually close it, but in the meantime
they keep the loop spinning. Now, unfortunately, the next test in
line is the Distributed test, which runs everything in a subprocess,
so it never builds up enough memory pressure to actually run the
GC, so the idle handles never get closed. This isn't too big a
deal, as the worst thing that happens is that it hogs one CPU
core on CI while the threads test is running, but we don't have
enough parallelism there anyway, so I don't expect a meaningful
impact on CI. It does however blow up our rr traces and apparently
causes significantly problems when trying to replay them on pernosco.
This is easy to fix by just closing these handles once we're done
with them.
antoine-levitt pushed a commit to antoine-levitt/julia that referenced this pull request May 9, 2021
uv_idle handles are a bit dangerous because the essentially prevent
the uv loop from going to sleep, so if there's no work to be done
on the julia side, it just turns into a busy loop. We start such
an idle callback in the threads.jl test. We make sure it goes out
of scope, so the GC will eventually close it, but in the meantime
they keep the loop spinning. Now, unfortunately, the next test in
line is the Distributed test, which runs everything in a subprocess,
so it never builds up enough memory pressure to actually run the
GC, so the idle handles never get closed. This isn't too big a
deal, as the worst thing that happens is that it hogs one CPU
core on CI while the threads test is running, but we don't have
enough parallelism there anyway, so I don't expect a meaningful
impact on CI. It does however blow up our rr traces and apparently
causes significantly problems when trying to replay them on pernosco.
This is easy to fix by just closing these handles once we're done
with them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants