stackcount - Count function calls and their stack traces. Uses Linux eBPF/bcc.
stackcount [-h] [-p PID] [-i INTERVAL] [-T] [-r] [-s]
[-P] [-v] [-d] pattern
stackcount traces functions and frequency counts them with their entire stack
trace, summarized in-kernel for efficiency. This allows higher frequency
events to be studied. The output consists of unique stack traces, and their
occurrence counts. In addition to kernel and user functions, kernel
tracepoints and USDT tracepoint are also supported.
The pattern is a string with optional '*' wildcards, similar to file globbing.
If you'd prefer to use regular expressions, use the -r option.
This tool only works on Linux 4.6+. Stack traces are obtained using the new
`BPF_STACK_TRACE` APIs. For kernels older than 4.6, see the version under
CONFIG_BPF and bcc.
- Print usage message.
- Allow regular expressions for the search pattern. The
default allows "*" wildcards only.
- Show address offsets.
- Include a timestamp with interval output.
- Show raw addresses.
- Print the source of the BPF program when loading it (for
- -i interval
- Summary interval, in seconds.
- -p PID
- Trace this process ID only (filtered in-kernel).
- A function name, or a search pattern. Can include wildcards
("*"). If the -r option is used, can include regular
- Count kernel stack traces for submit_bio():
- # stackcount submit_bio
- Count kernel stack traces for ip_output():
- # stackcount ip_output
- Show symbol offsets:
- # stackcount -s ip_output
- Show offsets and raw addresses (verbose):
- # stackcount -sv ip_output
- Count kernel stacks for kernel functions matching
- # stackcount 'tcp_send*'
- Same as previous, but using regular expressions:
- # stackcount -r '^tcp_send.*'
- Output every 5 seconds, with timestamps:
- # stackcount -Ti 5 ip_output
- Only count stacks when PID 185 is on-CPU:
- # stackcount -p 185 ip_output
- Count user stacks for dynamic heap allocations with malloc
in PID 185:
- # stackcount -p 185 c:malloc
- Count user stacks for thread creation (USDT tracepoint) in
- # stackcount -p 185 u:pthread:pthread_create
- Count kernel stacks for context switch events using a
- # stackcount t:sched:sched_switch
This summarizes unique stack traces in-kernel for efficiency, allowing it to
trace a higher rate of function calls than methods that post-process in user
space. The stack trace data is only copied to user space when the output is
printed, which usually only happens once. The stack walking also happens in an
optimized code path in the kernel thanks to the new BPF_STACK_TRACE table
APIs, which should be more efficient than the manual walker in the eBPF tracer
which older versions of this script used. With this in mind, call rates of
< 10,000/sec would incur negligible overhead. Test before production use.
You can also use funccount to get a handle on function call rates first.
This is from bcc.
Also look in the bcc distribution for a companion _examples.txt file containing
example usage, output, and commentary for this tool.
Unstable - in development.
Brendan Gregg, Sasha Goldshtein