Get start with eBPF Tracing

TODO: learn how to use bcc/trace and bpftrace

bcc

  1. install bcc on RHEL dnf install -y bcc-tools

  2. The tools list under /usr/share/bcc/tools/, go through the tutorial to familiar each tools.

    1. Some tools that may interested, like bpflist, tcp*, memleak
    2. funccount command traces functions, tracepoints, or USDT probes that match a specified pattern, e.g. funccount ‘tcp_*’
    3. trace command probes functions you specify and displays trace messages, e.g. trace ‘do_sys_open “%s”, arg2’, trace -I ‘net/sock.h’ ‘udpv6_sendmsg(struct sock *sk) (sk->sk_dport == 13568)’

  3. How to develope a new tool, seel developer tutorial

bpftrace

  1. install bpftrace on RHEL dnf install -y bpftrace

  2. The tools list under /usr/share/bpftrace/tools/

    1. Some tools that may interested, like gethostlatency.bt, tcp*,

  3. bpftrace Tutorial

  4. bpftrace networking example

How to deal with the errors

Q: bcc: “Unable to find kernel headers. Try rebuilding kernel with CONFIG_IKHEADERS=m (module)”

A: you need to install kernel-devel package if you use RHEL8

Q: bpftrace got errors like “‘net/sock.h’ file not found”.

A: you need to install kernel-devel package if you use RHEL8

Q: bcc: # ./trace ‘sys_execve “%s”, arg1’ cannot attach kprobe, probe entry may not exist Failed to attach BPF program b’probe_sys_execve_1’ to kprobe b’sys_execve’

A: Check ‘/proc/kallsyms’ and see if it contains the probe you want to attache, in this case it should be ‘__x64_sys_execve’ on x86_64 system.

Reference

learn-ebpf-tracing ebpf