runtime/pprof: heap profile is blank on go1.23rc1 (go1.22.4 was ok) #68136

S-YOU · 2024-06-23T09:16:51Z

Go version

go1.23rc1

Output of `go env` in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/dummy/.cache/go-build'
GOENV='/home/dummy/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS='-mod=readonly'
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/dummy/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/dummy/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/go'
GOSUMDB='off'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.23rc1'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/home/dummy/.config/go/telemetry'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2527459232=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Taking heap profile for my application using heap profile from runtime package

What did you see happen?

heap profile shows nothing (it was ok in go1.22.4)

What did you expect to see?

show something in heap profile

The text was updated successfully, but these errors were encountered:

gabyhelp · 2024-06-23T09:18:20Z

Similar Issues

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

seankhliao · 2024-06-23T09:22:46Z

Right now this issue isn't very actionable with the lack of information: what were the exact steps you took to generate the profile? what does blank mean? do you have a standalone reproducer? can we see the profile?

S-YOU · 2024-06-23T09:53:34Z

I can share profile for go1.23rc1, here it is. it is only 231 bytes @seankhliao
heap-20240623_094946.out.zip

% xxd heap-20240623_094946.out
00000000: 1f8b 0800 0000 0000 04ff 3c8a 314e c340  ..........<.1N.@
00000010: 1045 b1e3 0d8b a3a0 2d90 b0dc 203a 7723  .E......-... :w#
00000020: cd29 3803 5184 d6cb 0a8c d6bb 91c7 8994  .)8.Q...........
00000030: 6e6a c421 5222 7112 4493 863e 151d 0740  nj.!R"q.D..>...@
00000040: 1446 b645 a699 f7ff 7fb7 898c 549c 2672  .F.E........T.&r
00000050: a292 3491 62e0 e9c0 a72a be79 ff7c fe79  ..4.b....*.y.|.y
00000060: 79fd fabe cccf 65a4 9839 ce98 0fa2 90f9  y.....e..9......
00000070: 858c 15ef f66f 1fdd 55c6 877d ff8b b33c  .....o..U..}...<
00000080: 9793 defa edfa 8b32 defd 6391 e209 0a5a  .......2..c....Z
00000090: 6963 5194 dbd6 12ce b573 c1dc 85f2 c99a  icQ......s......
000000a0: 9650 98b0 f62d cef4 d08e e6bc f26b b247  .P...-.......k.G
000000b0: 6536 c671 bb86 c750 5bf0 bab6 d0d8 5580  e6.q...P[.....U.
000000c0: 8700 d418 68a0 ac3c d4ba f238 5d6c ee29  ....h..<...8]l.)
000000d0: 2c31 5d6c 684b 463b b7fc 0b00 00ff ff77  ,1]lhKF;.......w
000000e0: 6fcf 84f0 0000 00                        o......

seankhliao · 2024-06-23T10:01:41Z

How was it created?

S-YOU · 2024-06-23T10:03:07Z

How was it created?

by calling runtime/pprof.writeHeap directly from my application @seankhliao

S-YOU · 2024-06-23T11:21:05Z

okay, when I change profilerecord.StackRecord back to runtime.StackRecord in some places, it shows something.
Might be releated to that fixed 32 depth (array) to 128 (slice) stuff.

seankhliao · 2024-06-24T12:24:17Z

Note: I could only get this to reproduce on very simple programs, doing anything more such as dual writing through pprof.Lookup("heap").WriteTo eliminated the issue.

rsc · 2024-06-28T20:17:04Z

Why is the profile entirely "<unknown>"? What is your total heap size? Can you post the non-empty profile as well?

S-YOU · 2024-06-29T07:18:57Z

Why is the profile entirely "<unknown>"?

<unknown> is because of my partial changes as I mentioned here. I didn't touch *goroutineProfileWithLabels part, so, I guess names are cannot loaded properly. It do show source file locations in source, so <unknown> part is not the problem I believe. @rsc

okay, when I change profilerecord.StackRecord back to runtime.StackRecord in some places, it shows something.

I am not sure profile from modified version of go1.23rc1 or go1.22.4 benefit anything here.

S-YOU · 2024-06-29T07:57:28Z

btw, this is the temporary patch I applied to go1.23rc1 to get previous behavior back for pprof and also temporarily modified goroutineProfileWithLabelsConcurrent to get unknowns labels to proper one. Some might not related, but this is what I have now.

go-1.23rc1-pprof-tmp.patch.zip

S-YOU · 2024-06-29T08:21:43Z

okay, seems like I understand the issue and what was changed in go1.23rc1

when I put _ = pprof.Lookup("heap") in program startup, the issue seems to be resolved and profile is not blank anymore.

you may close the issue, if it is working as expected. @rsc @seankhliao

cherrymui · 2024-07-03T17:31:25Z

This is actually a real bug. Thanks @S-YOU for reporting.
I'll send a fix soon.

gopherbot · 2024-07-03T18:38:18Z

Change https://go.dev/cl/596538 mentions this issue: cmd/link: don't disable memory profiling when pprof.WriteHeapProfile is used

S-YOU · 2024-07-07T08:36:32Z

Thank you very much the for the fix @cherrymui, didn't realize that its been optimized out.

…is used We have an optimization that if the memory profile is not consumed anywhere, we set the memory profiling rate to 0 to disable the "background" low-rate profiling. We detect whether the memory profile is used by checking whether the runtime.MemProfile function is reachable at link time. Previously, all APIs that access the memory profile go through runtime.MemProfile. But the code was refactored in CL 572396, and now the legacy entry point WriteHeapProfile uses pprof_memProfileInternal without going through runtime.MemProfile. In fact, even with the recommended runtime/pprof.Profile API (pprof.Lookup or pprof.Profiles), runtime.MemProfile is only (happen to be) reachable through countHeap. Change the linker to check runtime.memProfileInternal instead, which is on all code paths that retrieve the memory profile. Add a test case for WriteHeapProfile, so we cover all entry points. Fixes golang#68136. Change-Id: I075c8d45c95c81825a1822f032e23107aea4303c Reviewed-on: https://go-review.googlesource.com/c/go/+/596538 Reviewed-by: Than McIntosh <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

seankhliao changed the title ~~pprof: heap profile is blank on go1.23rc1 (go1.22.4 was ok)~~ runtime/pprof: heap profile is blank on go1.23rc1 (go1.22.4 was ok) Jun 23, 2024

seankhliao added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jun 23, 2024

gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jun 23, 2024

seankhliao added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Jun 23, 2024

seankhliao added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Jun 23, 2024

seankhliao added this to the Go1.23 milestone Jun 23, 2024

rsc added the release-blocker label Jun 28, 2024

aclements assigned cherrymui Jul 2, 2024

gopherbot closed this as completed in 82c1434 Jul 3, 2024

dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jul 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

runtime/pprof: heap profile is blank on go1.23rc1 (go1.22.4 was ok) #68136

runtime/pprof: heap profile is blank on go1.23rc1 (go1.22.4 was ok) #68136

S-YOU commented Jun 23, 2024

gabyhelp commented Jun 23, 2024

seankhliao commented Jun 23, 2024

S-YOU commented Jun 23, 2024

seankhliao commented Jun 23, 2024

S-YOU commented Jun 23, 2024

S-YOU commented Jun 23, 2024 •

edited

Loading

seankhliao commented Jun 24, 2024

rsc commented Jun 28, 2024

S-YOU commented Jun 29, 2024 •

edited

Loading

S-YOU commented Jun 29, 2024

S-YOU commented Jun 29, 2024

cherrymui commented Jul 3, 2024

gopherbot commented Jul 3, 2024

S-YOU commented Jul 7, 2024

runtime/pprof: heap profile is blank on go1.23rc1 (go1.22.4 was ok) #68136

runtime/pprof: heap profile is blank on go1.23rc1 (go1.22.4 was ok) #68136

Comments

S-YOU commented Jun 23, 2024

Go version

Output of go env in your module/workspace:

What did you do?

What did you see happen?

What did you expect to see?

gabyhelp commented Jun 23, 2024

seankhliao commented Jun 23, 2024

S-YOU commented Jun 23, 2024

seankhliao commented Jun 23, 2024

S-YOU commented Jun 23, 2024

S-YOU commented Jun 23, 2024 • edited Loading

seankhliao commented Jun 24, 2024

rsc commented Jun 28, 2024

S-YOU commented Jun 29, 2024 • edited Loading

S-YOU commented Jun 29, 2024

S-YOU commented Jun 29, 2024

cherrymui commented Jul 3, 2024

gopherbot commented Jul 3, 2024

S-YOU commented Jul 7, 2024

Output of `go env` in your module/workspace:

S-YOU commented Jun 23, 2024 •

edited

Loading

S-YOU commented Jun 29, 2024 •

edited

Loading