Merge branch 'master' into patch

.gitignore

@@ -2,6 +2,10 @@
 *.swp
 *.swo
 *.pyc
+.idea
 
-# Build artefacts
+# Build artifacts
 /build/
+cmake-build-debug
+debian/**/*.log
+obj-x86_64-linux-gnu

INSTALL.md

@@ -7,6 +7,7 @@
  - [Arch](#arch---aur)
  - [Gentoo](#gentoo---portage)
 * [Source](#source)
+ - [Debian](#debian---source)
  - [Ubuntu](#ubuntu---source)
  - [Fedora](#fedora---source)
 * [Older Instructions](#older-instructions)
@@ -163,6 +164,90 @@ The appropriate dependencies (e.g., ```clang```, ```llvm``` with BPF backend) wi
 
 # Source
 
+## Debian - Source
+### Jessie
+#### Repositories
+
+The automated tests that run as part of the build process require `netperf`. Since netperf's license is not "certified"
+as an open-source license, it is in Debian's `non-free` repository.
+
+`/etc/apt/sources.list` should include the `non-free` repository and look something like this:
+
+```
+deb http:https://httpredir.debian.org/debian/ jessie main non-free
+deb-src http:https://httpredir.debian.org/debian/ jessie main non-free
+
+deb http:https://security.debian.org/ jessie/updates main non-free
+deb-src http:https://security.debian.org/ jessie/updates main non-free
+
+# wheezy-updates, previously known as 'volatile'
+deb http:https://ftp.us.debian.org/debian/ jessie-updates main non-free
+deb-src http:https://ftp.us.debian.org/debian/ jessie-updates main non-free
+```
+
+BCC also requires kernel version 4.1 or above. Those kernels are available in the `jessie-backports` repository. To
+add the `jessie-backports` repository to your system create the file `/etc/apt/sources.list.d/jessie-backports.list`
+with the following contents:
+
+```
+deb http:https://httpredir.debian.org/debian jessie-backports main
+deb-src http:https://httpredir.debian.org/debian jessie-backports main
+```
+
+#### Install Build Dependencies
+
+Note, check for the latest `linux-image-4.x` version in `jessie-backports` before proceeding. Also, have a look at the
+`Build-Depends:` section in `debian/control` file.
+
+```
+# Before you begin
+apt-get update
+
+# Update kernel and linux-base package
+apt-get -t jessie-backports install linux-base linux-image-4.8.0-0.bpo.2-amd64
+
+# BCC build dependencies:
+apt-get install debhelper cmake libllvm3.8 llvm-3.8-dev libclang-3.8-dev \
+ libelf-dev bison flex libedit-dev clang-format-3.8 python python-netaddr \
+ python-pyroute2 luajit libluajit-5.1-dev arping iperf netperf ethtool \
+ devscripts
+```
+
+#### Sudo
+
+Adding eBPF probes to the kernel and removing probes from it requires root privileges. For the build to complete
+successfully, you must build from an account with `sudo` access. (You may also build as root, but it is bad style.)
+
+`/etc/sudoers` or `/etc/sudoers.d/build-user` should contain
+
+```
+build-user ALL = (ALL) NOPASSWD: ALL
+```
+
+or
+
+```
+build-user ALL = (ALL) ALL
+```
+
+If using the latter sudoers configuration, please keep an eye out for sudo's password prompt while the build is running.
+
+#### Build
+
+```
+cd <preferred development directory>
+git clone https://github.com/iovisor/bcc.git
+cd bcc
+debuild -b -uc -us
+```
+
+#### Install
+
+```
+cd ..
+sudo dpkg -i *bcc*.deb
+```
+
 ## Ubuntu - Source
 
 To build the toolchain from source, one needs:
@@ -219,9 +304,12 @@ sudo pip install pyroute2
 wget http:https://llvm.org/releases/3.7.1/clang+llvm-3.7.1-x86_64-fedora22.tar.xz
 sudo tar xf clang+llvm-3.7.1-x86_64-fedora22.tar.xz -C /usr/local --strip 1
 
-# FC23 and FC24
+# FC23
 wget http:https://llvm.org/releases/3.9.0/clang+llvm-3.9.0-x86_64-fedora23.tar.xz
 sudo tar xf clang+llvm-3.9.0-x86_64-fedora23.tar.xz -C /usr/local --strip 1
+
+# FC24 and FC25
+sudo dnf install -y clang clang-devel llvm llvm-devel llvm-static ncurses-devel
 ```
 
 ### Install and compile BCC

README.md

@@ -73,7 +73,7 @@ Examples:
 - examples/tracing/[vfsreadlat.py](examples/tracing/vfsreadlat.py) examples/tracing/[vfsreadlat.c](examples/tracing/vfsreadlat.c): VFS read latency distribution. [Examples](examples/tracing/vfsreadlat_example.txt).
 
 #### Tools:
-<center><a href="images/bcc_tracing_tools_2016.png"><img src="images/bcc_tracing_tools_2016.png" border=0 width=700></a></center>
+<center><a href="images/bcc_tracing_tools_2017.png"><img src="images/bcc_tracing_tools_2017.png" border=0 width=700></a></center>
 - tools/[argdist](tools/argdist.py): Display function parameter values as a histogram or frequency count. [Examples](tools/argdist_example.txt).
 - tools/[bashreadline](tools/bashreadline.py): Print entered bash commands system wide. [Examples](tools/bashreadline_example.txt).
 - tools/[biolatency](tools/biolatency.py): Summarize block device I/O latency as a histogram. [Examples](tools/biolatency_example.txt).
@@ -89,6 +89,7 @@ Examples:
 - tools/[cpuunclaimed](tools/cpuunclaimed.py): Sample CPU run queues and calculate unclaimed idle CPU. [Examples](tools/cpuunclaimed_example.txt)
 - tools/[dcsnoop](tools/dcsnoop.py): Trace directory entry cache (dcache) lookups. [Examples](tools/dcsnoop_example.txt).
 - tools/[dcstat](tools/dcstat.py): Directory entry cache (dcache) stats. [Examples](tools/dcstat_example.txt).
+- tools/[deadlock_detector](tools/deadlock_detector.py): Detect potential deadlocks on a running process. [Examples](tools/deadlock_detector_example.txt)
 - tools/[execsnoop](tools/execsnoop.py): Trace new processes via exec() syscalls. [Examples](tools/execsnoop_example.txt).
 - tools/[ext4dist](tools/ext4dist.py): Summarize ext4 operation latency distribution as a histogram. [Examples](tools/ext4dist_example.txt).
 - tools/[ext4slower](tools/ext4slower.py): Trace slow ext4 operations. [Examples](tools/ext4slower_example.txt).

SPECS/bcc.spec

@@ -1,4 +1,10 @@
 %bcond_with local_clang_static
+#lua jit not available for some architectures
+%ifarch ppc64 aarch64 ppc64le
+%{!?with_lua: %global with_lua 0}
+%else
+%{!?with_lua: %global with_lua 1}
+%endif
 %define debug_package %{nil}
 
 Name: bcc
@@ -11,10 +17,12 @@ License: ASL 2.0
 URL: https://github.com/iovisor/bcc
 Source0: bcc.tar.gz
 
-ExclusiveArch: x86_64
+ExclusiveArch: x86_64 ppc64 aarch64 ppc64le
 BuildRequires: bison cmake >= 2.8.7 flex make
 BuildRequires: gcc gcc-c++ python2-devel elfutils-libelf-devel-static
+%if %{with_lua}
 BuildRequires: luajit luajit-devel
+%endif
 %if %{without local_clang_static}
 BuildRequires: llvm-devel llvm-static
 BuildRequires: clang-devel
@@ -25,6 +33,11 @@ BuildRequires: pkgconfig ncurses-devel
 Python bindings for BPF Compiler Collection (BCC). Control a BPF program from
 userspace.
 
+%if %{with_lua}
+%global lua_include `pkg-config --variable=includedir luajit`
+%global lua_libs `pkg-config --variable=libdir luajit`/lib`pkg-config --variable=libname luajit`.so
+%global lua_config -DLUAJIT_INCLUDE_DIR=%{lua_include} -DLUAJIT_LIBRARIES=%{lua_libs}
+%endif
 
 %prep
 %setup -q -n bcc
@@ -35,8 +48,7 @@ mkdir build
 pushd build
 cmake .. -DREVISION_LAST=%{version} -DREVISION=%{version} \
  -DCMAKE_INSTALL_PREFIX=/usr \
- -DLUAJIT_INCLUDE_DIR=`pkg-config --variable=includedir luajit` \
- -DLUAJIT_LIBRARIES=`pkg-config --variable=libdir luajit`/lib`pkg-config --variable=libname luajit`.so
+ %{?lua_config}
 make %{?_smp_mflags}
 popd
 
@@ -56,16 +68,20 @@ Requires: libbcc = %{version}-%{release}
 %description -n python-bcc
 Python bindings for BPF Compiler Collection (BCC)
 
+%if %{with_lua}
 %package -n bcc-lua
 Summary: Standalone tool to run BCC tracers written in Lua
 Requires: libbcc = %{version}-%{release}
 %description -n bcc-lua
 Standalone tool to run BCC tracers written in Lua
+%endif
 
 %package -n libbcc-examples
 Summary: Examples for BPF Compiler Collection (BCC)
 Requires: python-bcc = %{version}-%{release}
+%if %{with_lua}
 Requires: bcc-lua = %{version}-%{release}
+%endif
 %description -n libbcc-examples
 Examples for BPF Compiler Collection (BCC)
 
@@ -82,8 +98,10 @@ Command line tools for BPF Compiler Collection (BCC)
 %files -n python-bcc
 %{python_sitelib}/bcc*
 
+%if %{with_lua}
 %files -n bcc-lua
 /usr/bin/bcc-lua
+%endif
 
 %files -n libbcc-examples
 /usr/share/bcc/examples/*

debian/control

@@ -3,7 +3,12 @@ Maintainer: Brenden Blanco <[email protected]>
 Section: misc
 Priority: optional
 Standards-Version: 3.9.5
-Build-Depends: debhelper (>= 9), cmake, libllvm3.7 | libllvm3.8, llvm-3.7-dev | llvm-3.8-dev, libclang-3.7-dev | libclang-3.8-dev, libelf-dev, bison, flex, libedit-dev, clang-format | clang-format-3.7, python-netaddr, python-pyroute2, luajit, libluajit-5.1-dev
+Build-Depends: debhelper (>= 9), cmake, libllvm3.7 | libllvm3.8,
+ llvm-3.7-dev | llvm-3.8-dev, libclang-3.7-dev | libclang-3.8-dev,
+ libelf-dev, bison, flex, libedit-dev,
+ clang-format | clang-format-3.7 | clang-format-3.8, python (>= 2.7),
+ python-netaddr, python-pyroute2, luajit, libluajit-5.1-dev, arping,
+ inetutils-ping | iputils-ping, iperf, netperf, ethtool, devscripts
 Homepage: https://github.com/iovisor/bcc
 
 Package: libbcc

docs/kernel-versions.md

@@ -15,6 +15,7 @@ ARM64 | 3.18 | [e54bcde3d69d](https://git.kernel.org/cgit/linux/kernel/git/torva
 s390 | 4.1 | [054623105728](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=054623105728b06852f077299e2bf1bf3d5f2b0b)
 Constant blinding for JIT machines | 4.7 | [4f3446bb809f](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=4f3446bb809f20ad56cadf712e6006815ae7a8f9)
 PowerPC64 | 4.8 | [156d0e290e96](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=156d0e290e969caba25f1851c52417c14d141b24)
+Constant blinding - PowerPC64 | 4.9 | [b7b7013cac55](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=b7b7013cac55d794940bd9cb7b7c55c9dececac4)
 
 ## Main features
 
@@ -24,7 +25,7 @@ Feature | Kernel version | Commit
 Kernel helpers | 3.15 | [bd4cf0ed331a](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=bd4cf0ed331a275e9bf5a49e6d0fd55dffc551b8)
 `bpf()` syscall | 3.18 | [99c55f7d47c0](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=99c55f7d47c0dc6fc64729f37bf435abf43f4c60)
 Tables (_a.k.a._ Maps; details below) | 3.18 | [99c55f7d47c0](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=99c55f7d47c0dc6fc64729f37bf435abf43f4c60)
-BPF attached to sockets | 3.19 | [89aa075832b0](89aa075832b0da4402acebd698d0411dcc82d03e)
+BPF attached to sockets | 3.19 | [89aa075832b0](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=89aa075832b0da4402acebd698d0411dcc82d03e)
 BPF attached to `kprobes` | 4.1 | [2541517c32be](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=2541517c32be2531e0da59dfd7efc1ce844644f5)
 `cls_bpf` / `act_bpf` for `tc` | 4.1 | [e2e9b6541dd4](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=e2e9b6541dd4b31848079da80fe2253daaafb549)
 Tail calls | 4.2 | [04fd61ab36ec](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=04fd61ab36ec065e194ab5e74ae34a5240d992bb)
@@ -53,9 +54,11 @@ Perf events | 4.3 | [ea317b267e9d](https://git.kernel.org/cgit/linux/kernel/git/
 Per-CPU hash | 4.6 | [824bd0ce6c7c](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=824bd0ce6c7c43a9e1e210abf124958e54d88342)
 Per-CPU array | 4.6 | [a10423b87a7e](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=a10423b87a7eae75da79ce80a8d9475047a674ee)
 Stack trace | 4.6 | [d5a3b1f69186](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d5a3b1f691865be576c2bffa708549b8cdccda19)
+Pre-alloc maps memory | 4.6 | [6c9059817432](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=6c90598174322b8888029e40dd84a4eb01f56afe)
 cgroup array | 4.8 | [4ed8ec521ed5](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=4ed8ec521ed57c4e207ad464ca0388776de74d4b)
-LRU hash | [4.10](https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=29ba732acbeece1e34c68483d1ec1f3720fa1bb3) | [](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=29ba732acbeece1e34c68483d1ec1f3720fa1bb3)
-LRU per-CPU hash | [4.10](https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=8f8449384ec364ba2a654f11f94e754e4ff719e0) | [](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=8f8449384ec364ba2a654f11f94e754e4ff719e0)
+LRU hash | 4.10 | [29ba732acbee](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=29ba732acbeece1e34c68483d1ec1f3720fa1bb3)
+LRU per-CPU hash | 4.10 | [8f8449384ec3](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=8f8449384ec364ba2a654f11f94e754e4ff719e0)
+LPM trie | 4.11 | [b95a5c4db09b](https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=b95a5c4db09bc7c253636cb84dc9b12c577fd5a0)
 Text string | _To be done?_ |
 Variable-length maps | _To be done?_ |