Fix file descriptor leak

[pchaigno]

If you keep tools/ustat.py running for more than 17min (with 1sec refresh) it errors out with too many open files. Other tools may be affected.

[goldshtn]

LGTM, thanks for catching this.

[4ast]

Great catch!

[vitalyvch]

Exellent !-)

[drandynisbet]

H'mm I'm getting a too many open files issue with
ustat -l java ...
create_probe_event: open(/sys/kernel/debug/tracing/uprobe_events): Too many open files
Traceback (most recent call last):
File "/usr/share/bcc/tools/lib/ustat", line 305, in
File "/usr/share/bcc/tools/lib/ustat", line 297, in run
File "/usr/share/bcc/tools/lib/ustat", line 246, in _loop_iter
File "/usr/share/bcc/tools/lib/ustat", line 237, in _attach_probes
File "/usr/lib/python2.7/dist-packages/bcc/init.py", line 1023, in attach_uprobe
Exception: Failed to attach BPF to uprobe

[yonghong-song]

A file descriptor leaking issues. Could you debug this a little more? You can hack bcc to stop/wait right after the above error appears, and then check file descriptors usr /proc//fd and /proc//fdinfo. That will give a clue what hold on file descriptors.

[drandynisbet]

Hi it seems to increase by circa 7 file descriptors each time the timeout is reached for ustat - Please see attached file descriptor list and info ... obviously I did not wait for it to exhaust my file descriptor list - but you can see the picture ...
FDINFO.txt
FDLIST.txt

[yonghong-song]

there are a lot of bpf prog/map file descriptors. Maybe they should be closed but did not? I tried simple sudo ./ustat.py -l java and cannot reproduce the issue. Could you post your complete command line which triggers the issue?

[drandynisbet]

My ulimit is set by default for open files to (-n) 1024
Not that it matters which one but you probably need a java program running ....
/usr/bin/java -jar xMandelbrot.jar
I'm using openjdk 8 but that shouldn't matter either.
I also set the dropdown MAXITERATIONS to 50000 on the GUI ... again that shouldn't matter ...
Downloaded the jar (I think) from http:https://math.hws.edu/xJava/MB/xMandelbrot.jar
sudo ./ustat.py -l java
Takes circa 5minutes to exhaust file descriptors on my work laptop ...
I ran that from the /tools/lib pulled from the repo and also my install point ...
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 19.04
Release: 19.04
Codename: disco

uname -a
Linux drandynisbet-XPS-13-9360 5.0.0-29-generic #31-Ubuntu SMP Thu Sep 12 13:05:32 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Thanks,
Andy

[yonghong-song]

@drandynisbet Could you check whether the following change

-bash-4.4$ git diff
diff --git a/src/python/bcc/__init__.py b/src/python/bcc/__init__.py
index 2d6dc350..f3910c6f 100644
--- a/src/python/bcc/__init__.py
+++ b/src/python/bcc/__init__.py
@@ -1355,6 +1355,8 @@ class BPF(object):
         if self.tracefile:
             self.tracefile.close()
             self.tracefile = None
+        for name, fn in list(self.funcs.items()):
+            os.close(fn.fd)
         if self.module:
             lib.bpf_module_destroy(self.module)
             self.module = None
-bash-4.4$

fixed the issue or not?

[drandynisbet]

Hi - yes fixed it - many many thanks ... although I did remove my build directory - recreate it and reconfigure as there seemed to be multiple copies of init.py in the build directory and some were not updated with the change ...

Cheers,
Andy