Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

print_log2_hist(): check and skip possible paddings #2155

Merged
merged 1 commit into from
Jan 21, 2019

Conversation

boat0
Copy link
Contributor

@boat0 boat0 commented Jan 20, 2019

Issue #2154

When a struct S is used as key to a BPF_HISTOGRAM, it is assumed that the second
member of S holds the slot. But when S is converted to python from bpf C,
a padding may be inserted as a second member. This breaks print_log2_hist().

    root@debian:~/bcc/tools# ./softirqs.py -d
    Tracing soft irq event time... Hit Ctrl-C to end.
    ^C
    Traceback (most recent call last):
      File "./softirqs.py", line 144, in <module>
        dist.print_log2_hist(label, "softirq", section_print_fn=vec_to_name)
      File "/usr/local/lib/python2.7/dist-packages/bcc/table.py", line 326, in print_log2_hist
        vals[slot] = v.value
    TypeError: list indices must be integers, not str

Fix it by skipping the possible padding. Future work would be fixing/working
around in the library where the padding is introduced.

Address issue 2154.

When a struct S is used as key to a BPF_HISTOGRAM, it is assumed that the second
member of S holds the slot. But when S is converted to python from bpf C,
a padding may be inserted as a second member. This breaks print_log2_hist().

    root@debian:~/bcc/tools# ./softirqs.py -d
    Tracing soft irq event time... Hit Ctrl-C to end.
    ^C
    Traceback (most recent call last):
      File "./softirqs.py", line 144, in <module>
        dist.print_log2_hist(label, "softirq", section_print_fn=vec_to_name)
      File "/usr/local/lib/python2.7/dist-packages/bcc/table.py", line 326, in print_log2_hist
        vals[slot] = v.value
    TypeError: list indices must be integers, not str

Fix it by skipping the possible padding. Future work would be fixing/working
around in the library where the padding is introduced.
@yonghong-song
Copy link
Collaborator

[buildbot, test this please]

@jeromemarchand
Copy link
Contributor

Shouldn't be this issue still open given the patch is explicitely a temporary one?
This patch seems very unreliable to me. I'm not going to dive into C standards, but for instance, what guaranty do we have that the padding field is always going to be called '__pad_1'?

Before we come to a proper solution, the format of the key structure should be clearly defined. I couldn't find anything about that in the reference guide. For instance, if the slot field was to be always named 'slot', which seems to be already the case in the tools and examples, then print_log2_hist() could just look for that field.

@jeromemarchand
Copy link
Contributor

Fix it by skipping the possible padding. Future work would be fixing/working
around in the library where the padding is introduced.

The padding is added by the compiler for alignment reason. There is nothing to fix there.

@boat0
Copy link
Contributor Author

boat0 commented Feb 15, 2019

This patch seems very unreliable to me. I'm not going to dive into C standards, but for instance, what guaranty do we have that the padding field is always going to be called '__pad_1'?

result_ += "[\"__pad_" + to_string(F->getFieldIndex()) + "\",\"char\",["

Before we come to a proper solution, the format of the key structure should be clearly defined. I couldn't find anything about that in the reference guide.

I believe help to complete the docs is always welcome :)

@boat0
Copy link
Contributor Author

boat0 commented Feb 15, 2019

Fix it by skipping the possible padding. Future work would be fixing/working
around in the library where the padding is introduced.

The padding is added by the compiler for alignment reason. There is nothing to fix there.

As a matter of fact, the padding is added by libbcc, the C part of BCC itself.

@yonghong-song
Copy link
Collaborator

I agree. Do you want to add some documentation about what C=>python key/value structure padding? Although most people should not care about this, but such information is still useful if users reply on particular fields and padding may change their relative position in the structure.

@boat0 boat0 deleted the fix2154 branch February 17, 2019 03:04
@jeromemarchand
Copy link
Contributor

As a matter of fact, the padding is added by libbcc, the C part of BCC itself.

OK, my bad!

I agree. Do you want to add some documentation about what C=>python key/value structure padding? Although most people should not care about this, but such information is still useful if users reply on particular fields and padding may change their relative position in the structure.

That might be useful too, but I was thinking about documenting the format of the key_type of an histogram when said type is a structure.

@yonghong-song
Copy link
Collaborator

That might be useful too, but I was thinking about documenting the format of the key_type of an histogram when said type is a structure.

This is definitely needed. Thanks!

palexster pushed a commit to palexster/bcc that referenced this pull request Jul 7, 2019
Address issue 2154.

When a struct S is used as key to a BPF_HISTOGRAM, it is assumed that the second
member of S holds the slot. But when S is converted to python from bpf C,
a padding may be inserted as a second member. This breaks print_log2_hist().

    root@debian:~/bcc/tools# ./softirqs.py -d
    Tracing soft irq event time... Hit Ctrl-C to end.
    ^C
    Traceback (most recent call last):
      File "./softirqs.py", line 144, in <module>
        dist.print_log2_hist(label, "softirq", section_print_fn=vec_to_name)
      File "/usr/local/lib/python2.7/dist-packages/bcc/table.py", line 326, in print_log2_hist
        vals[slot] = v.value
    TypeError: list indices must be integers, not str

Fix it by skipping the possible padding. Future work would be fixing/working
around in the library where the padding is introduced.
CrackerCat pushed a commit to CrackerCat/bcc that referenced this pull request Jul 31, 2024
Address issue 2154.

When a struct S is used as key to a BPF_HISTOGRAM, it is assumed that the second
member of S holds the slot. But when S is converted to python from bpf C,
a padding may be inserted as a second member. This breaks print_log2_hist().

    root@debian:~/bcc/tools# ./softirqs.py -d
    Tracing soft irq event time... Hit Ctrl-C to end.
    ^C
    Traceback (most recent call last):
      File "./softirqs.py", line 144, in <module>
        dist.print_log2_hist(label, "softirq", section_print_fn=vec_to_name)
      File "/usr/local/lib/python2.7/dist-packages/bcc/table.py", line 326, in print_log2_hist
        vals[slot] = v.value
    TypeError: list indices must be integers, not str

Fix it by skipping the possible padding. Future work would be fixing/working
around in the library where the padding is introduced.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants