Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added new command line option heap_size_hint for greedy GC #45369

Merged
merged 3 commits into from
May 19, 2022

Conversation

rssdev10
Copy link
Contributor

@rssdev10 rssdev10 commented May 19, 2022

The changes add a new command-line option --heap_size_hint for triggering aggressive garbage collection.

The following code might be used for checking that option.

using Dates

const mem_size = 0x1_000_000

counter = let
    counter = 0
    start_t = now()

    while (now() - start_t) / Millisecond(1000) < 15
        counter += 1
        arr = Vector{Int32}(undef, mem_size)
        arr[rand(1:mem_size)] = rand(Int32, 1)[begin]
        # sleep(1)
    end
    counter
end

@show counter, mem_size

Memory consumption might be estimated with % gtime --verbose usr/bin/julia memory_test.jl :

(counter, mem_size) = (348403, 0x01000000)
	Command being timed: "usr/bin/julia memory_test.jl"
	User time (seconds): 20.86
...
	Maximum resident set size (kbytes): 440704
...

Use time for Linux and gtime for MacOS.

And, memory consumption after adding the hint
% gtime --verbose usr/bin/julia --heap-size-hint=100m memory_test.jl:

(counter, mem_size) = (436, 0x01000000)
	Command being timed: "usr/bin/julia --heap-size-hint=100m memory_test.jl"
	User time (seconds): 15.24
...
	Maximum resident set size (kbytes): 267024
...

@oscardssmith oscardssmith requested a review from chflood May 19, 2022 13:48
@oscardssmith oscardssmith added the GC Garbage collector label May 19, 2022
src/gc.c Outdated Show resolved Hide resolved
Copy link
Member

@chflood chflood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

.TP
--heap-size-hint=<size>
Forces garbage collection if memory usage is higher that value. The memory hint might be
specified in megabytes (500M) or gigabytes (1.5G)
Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these in multiples of 500x or 1.5x?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

those are just examples. It arguably should be written specified in megabytes (e.g. 500M) or gigabytes (e.g. 1.5G)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these in multiples of 500x or 1.5x?

long double value is used for the numeric part. With 1e-7 restriction as a minimal value. So, even 0.000123G is working.

@maleadt maleadt added backport 1.8 Change should be backported to release-1.8 and removed backport 1.8 Change should be backported to release-1.8 labels Sep 20, 2022
@MilesCranmer
Copy link
Sponsor Member

Is there an environment variable for this? Would be useful for GitHub actions

@sefffal
Copy link

sefffal commented Mar 9, 2024

I second this: I would like to set a heap size hint via environment variable for a Julia library embedded in Python.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GC Garbage collector
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants