Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GC: enable logging to stderr when GC runs #43511

Merged
merged 14 commits into from
Jan 4, 2022
9 changes: 9 additions & 0 deletions base/experimental.jl
Original file line number Diff line number Diff line change
Expand Up @@ -344,4 +344,13 @@ adding them to the global method table.
"""
:@MethodTable

"""
Experimental.enable_gc_logging(on::Bool)

When turned on, print statistics about each GC to stderr.
"""
function enable_gc_logging(on::Bool=true)
ccall(:jl_enable_gc_logging, Cvoid, (Cint,), on)
end

end
19 changes: 17 additions & 2 deletions doc/src/manual/profile.md
Original file line number Diff line number Diff line change
Expand Up @@ -297,12 +297,18 @@ on the author's laptop).

## Memory allocation analysis

One of the most common techniques to improve performance is to reduce memory allocation. The
total amount of allocation can be measured with [`@time`](@ref) and [`@allocated`](@ref), and
One of the most common techniques to improve performance is to reduce memory allocation. Julia
provides several tools measure this:

### `@time`

The total amount of allocation can be measured with [`@time`](@ref) and [`@allocated`](@ref), and
specific lines triggering allocation can often be inferred from profiling via the cost of garbage
collection that these lines incur. However, sometimes it is more efficient to directly measure
the amount of memory allocated by each line of code.

### Line-by-Line Allocation Tracking

To measure allocation line-by-line, start Julia with the `--track-allocation=<setting>` command-line
option, for which you can choose `none` (the default, do not measure allocation), `user` (measure
memory allocation everywhere except Julia's core code), or `all` (measure memory allocation at
Expand All @@ -321,6 +327,15 @@ you want to analyze, then call [`Profile.clear_malloc_data()`](@ref) to reset al
Finally, execute the desired commands and quit Julia to trigger the generation of the `.mem`
files.

### GC Logging

While [`@time`](@ref) logs high-level stats about memory usage and garbage collection over the course
of evaluating an expression, it can be useful to log each garbage collection event, to get an
intuitive sense of how often the garbage collector is running, how long it's running each time,
and how much garbage it collects each time. This can be enabled with
[`Profile.enable_gc_logging(true)`](@ref), which causes Julia to log to stderr every time a
vchuravy marked this conversation as resolved.
Show resolved Hide resolved
NHDaly marked this conversation as resolved.
Show resolved Hide resolved
garbage collection happens.

## External Profiling
NHDaly marked this conversation as resolved.
Show resolved Hide resolved

Currently Julia supports `Intel VTune`, `OProfile` and `perf` as external profiling tools.
Expand Down
17 changes: 17 additions & 0 deletions src/gc-debug.c
Original file line number Diff line number Diff line change
Expand Up @@ -1380,6 +1380,23 @@ NOINLINE void gc_mark_loop_unwind(jl_ptls_t ptls, jl_gc_mark_sp_t sp, int pc_off
jl_set_safe_restore(old_buf);
}

static int gc_logging_enabled = 0;

JL_DLLEXPORT void jl_enable_gc_logging(int enable) {
gc_logging_enabled = enable;
}

void _report_gc_finished(uint64_t pause, uint64_t freed, int full, int recollect) JL_NOTSAFEPOINT {
if (!gc_logging_enabled) {
return;
}
jl_safe_printf("GC: pause %.2fms. collected %fMB. %s %s\n",
IanButterworth marked this conversation as resolved.
Show resolved Hide resolved
pause/1e6, freed/1e6,
full ? "full" : "incr",
recollect ? "recollect" : ""
);
}

#ifdef __cplusplus
}
#endif
3 changes: 3 additions & 0 deletions src/gc.c
Original file line number Diff line number Diff line change
Expand Up @@ -3205,6 +3205,9 @@ static int _jl_gc_collect(jl_ptls_t ptls, jl_gc_collection_t collection)

uint64_t gc_end_t = jl_hrtime();
uint64_t pause = gc_end_t - t0;

_report_gc_finished(pause, gc_num.freed, sweep_full, recollect);

gc_final_pause_end(t0, gc_end_t);
gc_time_sweep_pause(gc_end_t, actual_allocd, live_bytes,
estimate_freed, sweep_full);
Expand Down
3 changes: 3 additions & 0 deletions src/gc.h
Original file line number Diff line number Diff line change
Expand Up @@ -696,6 +696,9 @@ void gc_count_pool(void);

size_t jl_array_nbytes(jl_array_t *a) JL_NOTSAFEPOINT;

JL_DLLEXPORT void jl_enable_gc_logging(int enable);
void _report_gc_finished(uint64_t pause, uint64_t freed, int full, int recollect) JL_NOTSAFEPOINT;

#ifdef __cplusplus
}
#endif
Expand Down