Objex - Export and explore Python object graphs
1- collect data from a running process (this comines well with os.fork)
if not os.fork(): # child process
objex.dump_graph('dump.db')2- create an analysis database (this takes a few minutes as indices are added)
python -m objex make-analysis-db dump.db analysis.db3- browse the extracted object graph
python -m objex analysis.db
WELCOME TO OBJEX EXPLORER
Now exploring "dump.db" collected from a0acf3b86f31 at 2018-09-04 18:25:46
RSS memory was 190.43MiB; 151.0MiB (79.8%) found in 622,108 python objects
(Type "help" for options.)Assuming the process has leaked significantly (e.g. doubled or more in memory footprint since it started),
random objects are likely to be leaks. So, go to a random object.
WELCOME TO OBJEX EXPLORER
Now exploring "dump.db" collected from a0acf3b86f31 at 2018-09-04 18:25:46
RSS memory was 190.43MiB; 151.0MiB (79.8%) found in 622,108 python objects
(Type "help" for options.)
objex> go random
Now at: <lithoxyl.action.Action#580111> (size=64, refcount=5)
Next, use the in command to see what global data structure is referring to the leaked object:
objex> in
...
1 modules transitively refer to <lithoxyl.action.Action#580111>:
5 - <module lithoxyl.context#10871>.LITHOXYL_CONTEXT.loggers[0]._all_sinks[0].begin_events[2].action
...
go random and in can also be tools to learn new things about your processes memory structure:
objex> go random
Now at: <str#4774> (size=81, refcount=4, len=48)
objex> in
1 objects refer to <str#4774>:
1 - <function normpath#4394>.__doc__
1 modules transitively refer to <str#4774>:
2 - <module ntpath#2459>.normpath.__doc__
1 frames transitively refer to <str#4774>:
3 - <frame <module>#8316>.locals['os'].path.normpath.__doc__
objex> go random
Now at: <str#16576> (size=49, refcount=5, len=16)
objex> in
0 objects refer to <str#16576>:
2 modules transitively refer to <str#16576>:
1 - <module objex#2441>.Reader.__dict__[<str#16576>]
2 - <module objex.explorer#2490>.Reader.__dict__[<str#16576>]
1 frames transitively refer to <str#16576>:
3 - <frame <module>#8316>.locals['make_analysis_db'].__module__.Reader.__dict__[<str#16576>]
objex> go random
Now at: <frame stacker#219> (size=432, refcount=5)
objex> in
1 objects refer to <frame stacker#219>:
1 - <frame stacker#218>.f_back
objex> go random
Now at: <function create_string_buffer#14902> (size=112, refcount=6)
objex> in
6 objects refer to <function create_string_buffer#14902>:
1 - <module ctypes#2466>.create_string_buffer
2 - <dict#14968>[<str#15003>]
3 - <module ctypes._endian#2428>.create_string_buffer
4 - <dict#15652>[<str#15003>]
5 - <module ctypes.wintypes#2447>.create_string_buffer
6 - <dict#15723>[<str#15003>]
3 modules transitively refer to <function create_string_buffer#14902>:
7 - <module ctypes#2466>.create_string_buffer
8 - <module ctypes._endian#2428>.create_string_buffer
9 - <module ctypes.wintypes#2447>.create_string_buffer
1 frames transitively refer to <function create_string_buffer#14902>:
10 - <frame <module>#8316>.locals['collections']._sys.modules[<str#2320>].create_string_buffer