-
Notifications
You must be signed in to change notification settings - Fork 845
Taking a Stack Dump
If things seem to be "stuck", there may be a smoking gun available in the form of a Goroutine stack dump. This will reveal goroutines that are stuck in suspicious states like IO wait
and how long they've been stuck in that state.
This can be acquired by hopping on the machine running the web
node and curling the following endpoint:
Web
http:https://127.0.0.1:8079/debug/pprof/goroutine?debug=2
Worker
http:https://127.0.0.1:7776/debug/pprof/goroutine?debug=2
This will spit out a ton of text, so you may want to redirect its output to a file (or just copy-paste it all from your terminal output).
Note that this has to be done from the same machine because the debug endpoint only listens on the loopback 127.0.0.1
address.
Any Go process can be sent SIGQUIT
, which will send this very same stack dump to stderr
. The downside of this method is that it kills the process.
This can be done like so:
kill -QUIT $pid
...where $pid
should be replaced with whatever the pid is from ps
.
Note that you'll have to look wherever stderr
is being redirected in order to collect the stack dump.
Once dump is ready then you could put it in clipboard and paste it in swirly for a visualized view of the dump.