Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge pull request iovisor#4089 from davemarchevsky/davemarchevsky_fi… #4

Merged
merged 17 commits into from
Jul 5, 2022

Conversation

fengjixuchui
Copy link
Owner

No description provided.

chenhengqi and others added 17 commits June 10, 2022 07:21
Signed-off-by: Hengqi Chen <[email protected]>
tools/llcstat: Add TID info support
the bpf verifier would complain on program exit without initing R0 but
the buggy implementation makes up one BPF_EXIT instruction only, which
would be rejected by the verifier, and what's worse is that it does not
log any err about the try-to-load failure, which makes all tools use this
api would silently fall back to kprobe unexpectedly.

this patch fixs it and also prints verbose message about the result of the try.

Signed-off-by: Ze Gao <[email protected]>
Running syscount causes the following error:

    ...
    ; if (measure_latency)
    103: (18) r1 = 0xffffc90000b6a002
    105: (71) r1 = *(u8 *)(r1 +0)
    R0_w=inv(id=0,umax_value=16,var_off=(0x0; 0x1f)) R1_w=map_value(id=0,off=2,ks=4,vs=48,imm=0) R7=map_value(id=0,off=0,ks=4,vs=32,imm=0) R8=inv(id=0,smin_value=-2147483648,smax_value=2147483647) R10=fp0 fp-8=mmmmmmmm fp-16=mmmmmmmm
    ; if (measure_latency)
    106: (15) if r1 == 0x0 goto pc+4
    R0_w=inv(id=0,umax_value=16,var_off=(0x0; 0x1f)) R1_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R7=map_value(id=0,off=0,ks=4,vs=32,imm=0) R8=inv(id=0,smin_value=-2147483648,smax_value=2147483647) R10=fp0 fp-8=mmmmmmmm fp-16=mmmmmmmm
    ; __sync_fetch_and_add(&val->total_ns, bpf_ktime_get_ns() - *start_ts);
    107: (85) call bpf_ktime_get_ns#5
    ; __sync_fetch_and_add(&val->total_ns, bpf_ktime_get_ns() - *start_ts);
    108: (79) r1 = *(u64 *)(r6 +0)
    R6 !read_ok
    processed 181 insns (limit 1000000) max_states_per_insn 1 total_states 19 peak_states 19 mark_read 8
    -- END PROG LOAD LOG --
    libbpf: failed to load program 'sys_exit'
    libbpf: failed to load object 'syscount_bpf'
    libbpf: failed to load BPF skeleton 'syscount_bpf': -13
    failed to load BPF object: Permission denied

Fix this by calculating the latency and store on a local variable.

Signed-off-by: Hengqi Chen <[email protected]>
Running tcpconnect causes the following error:

    ...
    ; for (i = 0; i < filter_ports_len; i++) {
    43: (7d) if r1 s>= r4 goto pc+96
    R0=inv(id=0) R1_w=inv66 R2_w=map_value(id=0,off=280,ks=4,vs=280,imm=0) R3=map_value(id=0,off=0,ks=4,vs=280,imm=0) R4_w=inv(id=0,umin_value=67,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv(id=0) R8=inv(id=0) R9=inv(id=0,umax_value=65535,var_off=(0x0; 0xffff)) R10=fp0 fp-80=mmmmmm??
    ; if (port == filter_ports[i])
    44: (61) r4 = *(u32 *)(r2 +0)
    R0=inv(id=0) R1_w=inv66 R2_w=map_value(id=0,off=280,ks=4,vs=280,imm=0) R3=map_value(id=0,off=0,ks=4,vs=280,imm=0) R4_w=inv(id=0,umin_value=67,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv(id=0) R8=inv(id=0) R9=inv(id=0,umax_value=65535,var_off=(0x0; 0xffff)) R10=fp0 fp-80=mmmmmm??
    invalid access to map value, value_size=280 off=280 size=4
    R2 min value is outside of the array range
    processed 783 insns (limit 1000000) max_states_per_insn 4 total_states 23 peak_states 23 mark_read 6
    -- END PROG LOAD LOG --
    libbpf: failed to load program 'tcp_v4_connect_ret'
    libbpf: failed to load object 'tcpconnect_bpf'
    libbpf: failed to load BPF skeleton 'tcpconnect_bpf': -13
    failed to load BPF object: -13

Fix this by checking i against MAX_PORTS.

Signed-off-by: Hengqi Chen <[email protected]>
tcpconnlat uses fentry in BPF programs which may failed on old kernels
which don't have BPF trampline. Let's check fentry support first and
fallback to kprobe if it is not available.

Signed-off-by: Hengqi Chen <[email protected]>
security_inode_create does NOT exist if CONIFG_SECURITY is not set.
The tool filelife attaches to security_inode_create unconditionally
and result in attach error. Fix it by checking symbol existence.

Signed-off-by: Hengqi Chen <[email protected]>
After kernel commit d5869fdc189f ("block: introduce block_rq_error tracepoint"),
tracepoint block_rq_complete now shares the same argument struct as
`struct trace_event_raw_block_rq_completion` with tracepoint block_rq_error.
Because of that, now biopattern is broken because `struct trace_event_raw_block_rq_complete`
is disappeared from kernel BTF. Fix it by checking type existence.

Signed-off-by: Hengqi Chen <[email protected]>
Add pin support for xsk map.

Signed-off-by: Y7n05h <[email protected]>
`bcc` does various conversions of "global address" to "module offset" and
vice versa. Previous work (#1670) modified the "global address" ->
"module offset" calculation in `ProcSyms::Module::contains` to account
for differences between the file offset a section is loading bytes from
and the requested start address (relative to the base address of the
`.so`). Unfortunately that change didn't also modify "module offset" ->
"global address" calculations, such as the one in
bcc_resolve_global_addr. Update that calculation to account for the
same.

This calculation discrepancy was most apparent for us in production when
trying to attach USDTs to a shared lib with differing requested start
address and file offset. This patch also adds a test w/ comments
describing our specific situation and demonstrating how the patch fixes
the issue.

Signed-off-by: Dave Marchevsky <[email protected]>
bcc/syms: Fix shared lib module offset <-> global addr conversion
@fengjixuchui fengjixuchui merged commit b71a347 into fengjixuchui:master Jul 5, 2022
fengjixuchui pushed a commit that referenced this pull request Nov 15, 2022
…for -v option

Add additional information and change format of backtrace
- add symbol base offset, dso name, dso base offset
- symbol and dso info is included if it's available in target binary
- changed format:
INDEX ADDR [SYMBOL+OFFSET] (MODULE+OFFSET)

Print backtrace of ip if it failed to get syms.

Before:
  # offcputime -v
    psiginfo
    vscanf
    __snprintf_chk
    [unknown]
    [unknown]
    [unknown]
    [unknown]
    [unknown]
    sd_event_exit
    sd_event_dispatch
    sd_event_run
    [unknown]
    __libc_start_main
    [unknown]
    -                systemd-journal (204)
        1

    xas_load
    xas_find
    filemap_map_pages
    __handle_mm_fault
    handle_mm_fault
    do_page_fault
    do_translation_fault
    do_mem_abort
    do_el0_ia_bp_hardening
    el0_ia
    xas_load
    --
failed to get syms
      -                PmLogCtl (138757)
        1

After:
  # offcputime -v
    #0  0xffffffc01018b7e8 __arm64_sys_clock_nanosleep+0x0
    #1  0xffffffc01009a93c el0_svc_handler+0x34
    #2  0xffffffc010084a08 el0_svc+0x8
    #3  0xffffffc01018b7e8 __arm64_sys_clock_nanosleep+0x0
    --
    #4  0x0000007fa0bffd14 clock_nanosleep+0x94 (/usr/lib/libc-2.31.so+0x9ed14)
    #5  0x0000007fa0c0530c nanosleep+0x1c (/usr/lib/libc-2.31.so+0xa430c)
    #6  0x0000007fa0c051e4 sleep+0x34 (/usr/lib/libc-2.31.so+0xa41e4)
    #7  0x000000558a5a9608 flb_loop+0x28 (/usr/bin/fluent-bit+0x52608)
    #8  0x000000558a59f1c4 flb_main+0xa84 (/usr/bin/fluent-bit+0x481c4)
    #9  0x0000007fa0b85124 __libc_start_main+0xe4 (/usr/lib/libc-2.31.so+0x24124)
    #10 0x000000558a59d828 _start+0x34 (/usr/bin/fluent-bit+0x46828)
    -                fluent-bit (1238)
        1

    #0  0xffffffc01027daa4 generic_copy_file_checks+0x334
    #1  0xffffffc0102ba634 __handle_mm_fault+0x8dc
    #2  0xffffffc0102baa20 handle_mm_fault+0x168
    #3  0xffffffc010ad23c0 do_page_fault+0x148
    #4  0xffffffc010ad27c0 do_translation_fault+0xb0
    #5  0xffffffc0100816b0 do_mem_abort+0x50
    #6  0xffffffc0100843b0 el0_da+0x1c
    #7  0xffffffc01027daa4 generic_copy_file_checks+0x334
    --
    #8  0x0000007f8dc12648 [unknown]
    #9  0x0000007f8dc0aef8 [unknown]
    #10 0x0000007f8dc1c990 [unknown]
    #11 0x0000007f8dc08b0c [unknown]
    #12 0x0000007f8dc08e48 [unknown]
    #13 0x0000007f8dc081c8 [unknown]
    -                PmLogCtl (2412)
        1

Fixed: iovisor#3884
Signed-off-by: Eunseon Lee <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
7 participants