trace

README.md

@@ -108,6 +108,7 @@ Tools:
 - tools/[tcpconnect](tools/tcpconnect.py): Trace TCP active connections (connect()). [Examples](tools/tcpconnect_example.txt).
 - tools/[tcpconnlat](tools/tcpconnlat.py): Trace TCP active connection latency (connect()). [Examples](tools/tcpconnlat_example.txt).
 - tools/[tcpretrans](tools/tcpretrans.py): Trace TCP retransmits and TLPs. [Examples](tools/tcpretrans_example.txt).
+- tools/[trace](tools/trace.py): Trace arbitrary functions, with filters. [Examples](tools/trace_example.txt)
 - tools/[vfscount](tools/vfscount.py) tools/[vfscount.c](tools/vfscount.c): Count VFS calls. [Examples](tools/vfscount_example.txt).
 - tools/[vfsstat](tools/vfsstat.py) tools/[vfsstat.c](tools/vfsstat.c): Count some VFS calls, with column output. [Examples](tools/vfsstat_example.txt).
 - tools/[wakeuptime](tools/wakeuptime.py): Summarize sleep to wakeup time by waker kernel stack. [Examples](tools/wakeuptime_example.txt).

man/man8/trace.8

@@ -0,0 +1,117 @@
+.TH trace 8 "2016-02-18" "USER COMMANDS"
+.SH NAME
+trace \- Trace a function and print its arguments or return value, optionally evaluating a filter. Uses Linux eBPF/bcc.
+.SH SYNOPSIS
+.B trace [-h] [-p PID] [-v] [-Z STRING_SIZE] [-S] [-M MAX_EVENTS] [-o] probe [probe ...]
+.SH DESCRIPTION
+trace probes functions you specify and displays trace messages if a particular
+condition is met. You can control the message format to display function 
+arguments and return values. 
+
+Since this uses BPF, only the root user can use this tool.
+.SH REQUIREMENTS
+CONFIG_BPF and bcc.
+.SH OPTIONS
+.TP
+\-h
+Print usage message.
+.TP
+\-p PID
+Trace only functions in the process PID.
+.TP
+\-v
+Display the generated BPF program, for debugging purposes.
+.TP
+\-z STRING_SIZE
+When collecting string arguments (of type char*), collect up to STRING_SIZE 
+characters. Longer strings will be truncated.
+.TP
+\-S
+If set, trace messages from trace's own process. By default, this is off to
+avoid tracing storms -- for example, if you trace the write system call, and
+consider that trace is writing to the standard output.
+.TP
+\-M MAX_EVENTS
+Print up to MAX_EVENTS trace messages and then exit.
+.TP
+\-o
+Print times relative to the beginning of the trace (offsets), in seconds. The
+default is to print absolute time.
+.TP
+probe [probe ...]
+One or more probes that attach to functions, filter conditions, and print
+information. See PROBE SYNTAX below.
+.SH PROBE SYNTAX
+The general probe syntax is as follows:
+
+.B [{p,r}]:[library]:function [(predicate)] ["format string"[, arguments]]
+.TP
+.B [{p,r}]
+Probe type \- "p" for function entry, "r" for function return. The default
+probe type is "p".
+.TP
+.B [library]
+Library containing the probe.
+Specify the full path to the .so or executable file where the function to probe
+resides. Alternatively, you can specify just the lib name: for example, "c"
+refers to libc. If no library name is specified, the kernel is assumed. Also,
+you can specify an executable name (without a full path) if it is in the PATH.
+For example, "bash".
+.TP
+.B function
+The function to probe.
+.TP
+.B [(predicate)]
+The filter applied to the captured data. Only if the filter evaluates as true,
+the trace message will be printed. The filter can use any valid C expression
+that refers to the argument values: arg1, arg2, etc., or to the return value
+retval in a return probe. If necessary, use C cast operators to coerce the
+arguments to the desired type. For example, if arg1 is of type int, use the
+expression ((int)arg1 < 0) to trace only invocations where arg1 is negative.
+Note that only arg1-arg6 are supported, and only if the function is using the
+standard x86_64 convention where the first six arguments are in the RDI, RSI, 
+RDX, RCX, R8, R9 registers. If no predicate is specified, all function 
+invocations are traced.
+.TP
+.B ["format string"[, arguments]]
+A printf-style format string that will be used for the trace message. You can
+use the following format specifiers: %s, %d, %u, %lld, %llu, %hd, %hu, %c,
+%x, %llx -- with the same semantics as printf's. Make sure to pass the exact
+number of arguments as there are placeholders in the format string. The
+format specifier replacements may be any C expressions, and may refer to the
+same special keywords as in the predicate (arg1, arg2, etc.).
+
+The predicate expression and the format specifier replacements for printing
+may also use the following special keywords: $pid, $tgid to refer to the 
+current process' pid and tgid; $uid, $gid to refer to the current user's
+uid and gid; $cpu to refer to the current processor number.
+.SH EXAMPLES
+.TP
+Trace all invocations of the open system call with the name of the file being opened:
+#
+.B trace '::do_sys_open """%s"", arg2'
+.TP
+Trace all invocations of the read system call where the number of bytes requested is greater than 20,000:
+#
+.B trace '::sys_read (arg3 > 20000) """read %d bytes"", arg3'
+.TP
+Trace all malloc calls and print the size of the requested allocation:
+#
+.B trace ':c:malloc """size = %d"", arg1'
+.TP
+Trace returns from the readline function in bash and print the return value as a string:
+#
+.B trace 'r:bash:readline """%s"", retval' 
+.SH SOURCE
+This is from bcc.
+.IP
+https://github.com/iovisor/bcc
+.PP
+Also look in the bcc distribution for a companion _examples.txt file containing
+example usage, output, and commentary for this tool.
+.SH OS
+Linux
+.SH STABILITY
+Unstable - in development.
+.SH AUTHOR
+Sasha Goldshtein