It probably should have been called pydbgattach or so but I didn't noticed until I went to bed so I keep this name now. :)
This works in several steps:
-
We are attaching to the running Python process and want to inject some C-code
pyinjectcode.c
.Originally I planned to develop a small tool for this by myself based on
ptrace
. For Mac, I also foundmach_inject
which may have been useful.However, to keep things simple for now, I just use GDB for this.
See the file
pydbattach.sh
which basically does this step. -
pyinjectcode.c
creates a new Python thread and runs a Python script. In our case, it runspyinjectcode.py
. -
pyinjectcode.py
starts a Pdb instance and attaches it to another already running Python process (just the first it founds for now). This is done via a more genericsys.settrace
function which is implemented inpythreadhacks.py
. -
pythreadhacks.py
heavily uses(_)ctypes
to access the underlying CPython objects for the thread state. This way, it reimplementsPyEval_SetTrace
in a more general way.
Similar alternatives, to attach to running CPython process without having it prepared beforehand. Those usually use the OS debugging capabilities (or use gdb/lldb) to attach to the native CPython process and then inject some code or just analyze the native CPython thread stacks.
- Google pyringe, last update 2014
- pyrasite, last update 2017
- py-spy, sampling only, dump stack traces
- Pyflame, sampling only
- PyStack, dump stack traces including locals
There are other alternatives where you prepare your Python script beforehand to listen on some (tcp/file) socket to provide an interface for remote debugging and/or just a Python shell / REPL.
- IPython, by embedding an IPython kernel: background_zmq_ipython package
- winpdb (cross platform) remote debugger
- PyCharm IDE remote debugger, doc
- PyDev IDE remote debugger
- Twisted Conch Manhole, official example, lothar.com example, lysator.liu.se example, related StackOverflow question, blog.futurefoundries.com (2013)
- very simple manhole, has also some overview over related projects
- ispyd
- Eric IDE
- Trepan (based on pydb)
- rpdb
- rconsole (part of rfoo)
- pystuck
- pdb-clone
Some overviews and collected code examples:
- (QGIS) Example code for PyDev, Winpdb, Eric
- Python Wiki: Python debugging tools, Python Wiki: Python debuggers
- StackOverflow question about Python remote shells.
-- Albert Zeyer, https://www.az2000.de