Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dist/tools: provide tool to parse pktbuf output #13221

Merged
merged 1 commit into from
Jun 23, 2020
Merged

dist/tools: provide tool to parse pktbuf output #13221

merged 1 commit into from
Jun 23, 2020

Conversation

miri64
Copy link
Member

@miri64 miri64 commented Jan 28, 2020

Contribution description

This adds an parser for those pktbuf outputs. I hope this helps others to also be able to interpret those.

There is definitely a lot of room for improvements.

  • more subparsers
  • some graphical interface would be nice (currently only the structured data is printed to STDOUT)
  • parsing unmarked parts based on next header (might be tricky)
  • more documentation 😅

However, I don't want to work further on this for now, but I wanted you to know that this exists so I opened a PR.

Testing procedure

Make sure you have gdb installed. Run the script according to the README. I used these dumps to test with binaries from current master (should be working as long the structs don't change with any later version):

For iotlab-m3: 
1579257942.071813;m3-101;pktbuf
1579257942.073407;m3-101;packet buffer: first byte: 0x2000205c, last byte: 0x2000405c (size: 8192)
1579257942.074302;m3-101;  position of last byte used: 7976
1579257942.075239;m3-101;~ unused: 0x2000205c (next: 0x20002104, size:  120) ~
1579257942.076284;m3-101;=========== chunk   0 (0x200020d4 size:   48) ===========
1579257942.078218;m3-101;00000000  BC  20  00  20  04  27  00  20  08  00  00  00  01  00  00  00
1579257942.079313;m3-101;00000010  03  5D  F6  65  10  6B  11  15  BC  20  00  20  34  27  00  20
1579257942.081215;m3-101;00000020  08  00  00  00  01  00  00  00  03  65  FC  5C  22  E3  68  01
1579257942.082222;m3-101;~ unused: 0x20002104 (next: 0x2000270c, size: 1536) ~
1579257942.083242;m3-101;=========== chunk   1 (0x20002704 size:    8) ===========
1579257942.084235;m3-101;00000000  3A  00  04  D0  E3  24  BD  33
1579257942.085219;m3-101;~ unused: 0x2000270c (next: 0x2000273c, size:   40) ~
1579257942.087222;m3-101;=========== chunk   2 (0x20002734 size:    8) ===========
1579257942.088215;m3-101;00000000  3A  00  04  D0  E3  24  C6  8D
1579257942.089210;m3-101;~ unused: 0x2000273c (next: (nil), size: 6432) ~
1579257942.071496;m3-100;pktbuf
1579257942.073157;m3-100;packet buffer: first byte: 0x2000205c, last byte: 0x2000405c (size: 8192)
1579257942.074090;m3-100;  position of last byte used: 4592
1579257942.074989;m3-100;~ unused: 0x2000205c (next: (nil), size: 8192) ~
For native: 
packet buffer: first byte: 0x565a3d60, last byte: 0x565a5d60 (size: 8192)
  position of last byte used: 2248
=========== chunk   0 (0x565a3d60 size:  136) ===========
00000000  00  00  00  00  78  3D  5A  56  0C  00  00  00  01  00  00  00
00000010  03  00  00  00  00  00  00  00  80  00  00  00  19  9A  00  00
00000020  F1  76  E8  01  E0  1F  00  00  60  3D  5A  56  A0  3D  5A  56
00000030  28  00  00  00  01  00  00  00  01  00  00  00  C8  1F  00  00
00000040  60  00  00  00  00  C0  1F  FF  00  00  00  00  00  00  00  00
00000050  00  00  00  00  00  00  00  00  FE  80  00  00  00  00  00  00
00000060  00  00  00  00  00  00  00  01  88  3D  5A  56  E0  3D  5A  56
00000070  08  00  00  00  01  00  00  00  FF  FF  FF  FF  FE  40  49  27
00000080  00  00  05  00  00  00  00  00
~ unused: 0x565a3de8 (next: (nil), size: 8056) ~

> packet buffer: first byte: 0x565b3d60, last byte: 0x565b5d60 (size: 8192)
  position of last byte used: 1840
=========== chunk   0 (0x565b3d60 size:   96) ===========
00000000  78  3D  5B  56  28  3E  5B  56  74  00  00  00  01  00  00  00
00000010  01  00  00  00  00  00  00  00  00  00  00  00  90  3D  5B  56
00000020  14  00  00  00  01  00  00  00  FF  FF  FF  FF  00  01  FE  80
00000030  06  06  05  00  00  00  00  00  6A  B4  A6  76  AC  E5  33  33
00000040  00  00  00  16  00  00  00  00  E8  3D  5B  56  10  3F  5B  56
00000050  60  00  00  00  01  00  00  00  01  00  00  00  00  03  04  00
~ unused: 0x565b3dc0 (next: 0x565b3e18, size:   40) ~
=========== chunk   1 (0x565b3de8 size:   48) ===========
00000000  00  00  00  00  00  3E  5B  56  14  00  00  00  01  00  00  00
00000010  FF  FF  FF  FF  28  00  00  00  06  06  05  00  00  00  00  00
00000020  8A  F9  FB  40  49  26  33  33  00  00  00  16  00  00  00  00
~ unused: 0x565b3e18 (next: 0x565b3ea0, size:   16) ~
=========== chunk   2 (0x565b3e28 size:  120) ===========
00000000  60  00  00  00  00  4C  00  01  FE  80  00  00  00  00  00  00
00000010  68  B4  A6  FF  FE  76  AC  E5  FF  02  00  00  00  00  00  00
00000020  00  00  00  00  00  00  00  16  3A  00  05  02  00  00  01  00
00000030  8F  00  01  08  00  00  00  03  04  00  00  00  FF  02  00  00
00000040  00  00  00  00  00  00  00  00  00  00  00  FB  04  00  00  00
00000050  FF  02  00  00  00  00  00  00  00  00  00  01  FF  76  AC  E5
00000060  04  00  00  00  FF  02  00  00  00  00  00  00  00  00  00  00
00000070  00  00  00  6A  00  00  00  00
~ unused: 0x565b3ea0 (next: 0x565b3f70, size:  112) ~
=========== chunk   3 (0x565b3f10 size:   96) ===========
00000000  60  00  00  00  00  38  00  01  FE  80  00  00  00  00  00  00
00000010  88  F9  FB  FF  FE  40  49  26  FF  02  00  00  00  00  00  00
00000020  00  00  00  00  00  00  00  16  3A  00  05  02  00  00  01  00
00000030  8F  00  57  2F  00  00  00  02  04  00  00  00  FF  02  00  00
00000040  00  00  00  00  00  00  00  00  00  00  00  FB  04  00  00  00
00000050  FF  02  00  00  00  00  00  00  00  00  00  01  FF  40  49  26
~ unused: 0x565b3f70 (next: (nil), size: 7664) ~

Issues/PRs references

None

@miri64 miri64 added Type: new feature The issue requests / The PR implemements a new feature for RIOT Area: tools Area: Supplementary tools labels Jan 28, 2020
@miri64 miri64 mentioned this pull request Apr 14, 2020
Closed
81 tasks
@miri64 miri64 requested a review from jia200x as a code owner April 14, 2020 16:11
@miri64
Copy link
Member Author

miri64 commented Apr 14, 2020 •
edited
Loading

Fixed some bugs I noticed when trying to parse the dump in RIOT-OS/Release-Specs#152 (comment)

@leandrolanzieri leandrolanzieri self-assigned this Apr 28, 2020
@leandrolanzieri
Copy link
Contributor

Using the provided dump for native the output of the tool is as follows.

native dump 
$ dist/tools/pktbuf-stats/pktbuf-stats.py examples/gnrc_networking/bin/native/gnrc_networking.elf native_dump.txt
pktbuf output 1
{'first_byte': 1448820064,
 'last_byte': 1448828256,
 'last_byte_used': 1840,
 'line': 'packet buffer: first byte: 0x565b3d60, last byte: 0x565b5d60 (size: '
         '8192)',
 'segments': [{'content': [{'data': {'data': {'addr': 1448820264,
                                              'name': {'offset': 0,
                                                       'segment': 'chunk 2'}},
                                     'next': {'addr': 1448820088,
                                              'name': {'offset': 24,
                                                       'segment': 'chunk 0'}},
                                     'padding': bytearray(b'\x00\x00\x00\x00'),
                                     'size': 116,
                                     'type': 'GNRC_NETTYPE_IPV6',
                                     'users': 1},
                            'size': 24,
                            'start': 1448820064,
                            'type': 'gnrc_pktsnip'},
                           {'data': {'data': {'addr': 1448820112,
                                              'name': {'offset': 48,
                                                       'segment': 'chunk 0'}},
                                     'next': {'addr': 0},
                                     'padding': bytearray(b'\x00\x01\xfe\x80'),
                                     'size': 20,
                                     'type': 'GNRC_NETTYPE_NETIF',
                                     'users': 1},
                            'size': 24,
                            'start': 1448820088,
                            'type': 'gnrc_pktsnip'},
                           {'data': {'dst_l2addr': (b'33\x00\x00\x00\x16',),
                                     'dst_l2addr_len': 6,
                                     'flags': 0,
                                     'if_pid': 5,
                                     'lqi': 0,
                                     'padding': bytearray(b'\x00\x00\x00\x00'),
                                     'rssi': 0,
                                     'src_l2addr': (b'j\xb4\xa6v\xac\xe5',),
                                     'src_l2addr_len': 6},
                            'size': 24,
                            'start': 1448820112,
                            'type': 'gnrc_nettype_netif'},
                           {'data': {'data': {'addr': 1448820496,
                                              'name': {'offset': 0,
                                                       'segment': 'chunk 3'}},
                                     'next': {'addr': 1448820200,
                                              'name': {'offset': 0,
                                                       'segment': 'chunk 1'}},
                                     'padding': bytearray(b'\x00\x03\x04\x00'),
                                     'size': 96,
                                     'type': 'GNRC_NETTYPE_IPV6',
                                     'users': 1},
                            'size': 24,
                            'start': 1448820136,
                            'type': 'gnrc_pktsnip'}],
               'name': 'chunk 0',
               'size': 96,
               'start': 1448820064,
               'type': 'chunk'},
              {'name': 'unused 0x565b3dc0',
               'next': 1448820248,
               'size': 40,
               'start': 1448820160,
               'type': 'unused'},
              {'content': [{'data': {'data': {'addr': 1448820224,
                                              'name': {'offset': 24,
                                                       'segment': 'chunk 1'}},
                                     'next': {'addr': 0},
                                     'padding': bytearray(b'(\x00\x00\x00'),
                                     'size': 20,
                                     'type': 'GNRC_NETTYPE_NETIF',
                                     'users': 1},
                            'size': 24,
                            'start': 1448820200,
                            'type': 'gnrc_pktsnip'},
                           {'data': {'dst_l2addr': (b'33\x00\x00\x00\x16',),
                                     'dst_l2addr_len': 6,
                                     'flags': 0,
                                     'if_pid': 5,
                                     'lqi': 0,
                                     'padding': bytearray(b'\x00\x00\x00\x00'),
                                     'rssi': 0,
                                     'src_l2addr': (b'\x8a\xf9\xfb@I&',),
                                     'src_l2addr_len': 6},
                            'size': 24,
                            'start': 1448820224,
                            'type': 'gnrc_nettype_netif'}],
               'name': 'chunk 1',
               'size': 48,
               'start': 1448820200,
               'type': 'chunk'},
              {'name': 'unused 0x565b3e18',
               'next': 1448820384,
               'size': 16,
               'start': 1448820248,
               'type': 'unused'},
              {'content': [{'data': {'dst': b'\xff\x02\x00\x00\x00\x00\x00\x00'
                                            b'\x00\x00\x00\x00\x00\x00\x00\x16',
                                     'fl': 0,
                                     'hl': 1,
                                     'len': 76,
                                     'nh': 'HOPOPTS',
                                     'padding': bytearray(b':\x00\x05\x02'
                                                          b'\x00\x00\x01\x00'
                                                          b'\x8f\x00\x01\x08'
                                                          b'\x00\x00\x00\x03'
                                                          b'\x04\x00\x00\x00'
                                                          b'\xff\x02\x00\x00'
                                                          b'\x00\x00\x00\x00'
                                                          b'\x00\x00\x00\x00'
                                                          b'\x00\x00\x00\xfb'
                                                          b'\x04\x00\x00\x00'
                                                          b'\xff\x02\x00\x00'
                                                          b'\x00\x00\x00\x00'
                                                          b'\x00\x00\x00\x01'
                                                          b'\xffv\xac\xe5'
                                                          b'\x04\x00\x00\x00'
                                                          b'\xff\x02\x00\x00'
                                                          b'\x00\x00\x00\x00'
                                                          b'\x00\x00\x00\x00'
                                                          b'\x00\x00\x00j'
                                                          b'\x00\x00\x00\x00'),
                                     'src': b'\xfe\x80\x00\x00\x00\x00\x00\x00'
                                            b'h\xb4\xa6\xff\xfev\xac\xe5',
                                     'tc': {'dscp': 0, 'ecn': 0},
                                     'version': 6},
                            'size': 120,
                            'start': 1448820264,
                            'type': 'gnrc_nettype_ipv6'}],
               'name': 'chunk 2',
               'size': 120,
               'start': 1448820264,
               'type': 'chunk'},
              {'name': 'unused 0x565b3ea0',
               'next': 1448820592,
               'size': 112,
               'start': 1448820384,
               'type': 'unused'},
              {'content': [{'data': {'dst': b'\xff\x02\x00\x00\x00\x00\x00\x00'
                                            b'\x00\x00\x00\x00\x00\x00\x00\x16',
                                     'fl': 0,
                                     'hl': 1,
                                     'len': 56,
                                     'nh': 'HOPOPTS',
                                     'padding': bytearray(b':\x00\x05\x02'
                                                          b'\x00\x00\x01\x00'
                                                          b'\x8f\x00W/'
                                                          b'\x00\x00\x00\x02'
                                                          b'\x04\x00\x00\x00'
                                                          b'\xff\x02\x00\x00'
                                                          b'\x00\x00\x00\x00'
                                                          b'\x00\x00\x00\x00'
                                                          b'\x00\x00\x00\xfb'
                                                          b'\x04\x00\x00\x00'
                                                          b'\xff\x02\x00\x00'
                                                          b'\x00\x00\x00\x00'
                                                          b'\x00\x00\x00\x01'
                                                          b'\xff@I&'),
                                     'src': b'\xfe\x80\x00\x00\x00\x00\x00\x00'
                                            b'\x88\xf9\xfb\xff\xfe@I&',
                                     'tc': {'dscp': 0, 'ecn': 0},
                                     'version': 6},
                            'size': 96,
                            'start': 1448820496,
                            'type': 'gnrc_nettype_ipv6'}],
               'name': 'chunk 3',
               'size': 96,
               'start': 1448820496,
               'type': 'chunk'},
              {'name': 'unused 0x565b3f70',
               'next': None,
               'size': 7664,
               'start': 1448820592,
               'type': 'unused'}],
 'size': 8192}

@leandrolanzieri leandrolanzieri added Reviewed: 1-fundamentals The fundamentals of the PR were reviewed according to the maintainer guidelines Reviewed: 3-testing The PR was tested according to the maintainer guidelines labels Apr 28, 2020
@miri64
Copy link
Member Author

miri64 commented Apr 28, 2020

That's how its supposed to be :-)

@leandrolanzieri leandrolanzieri added this to the Release 2020.07 milestone Jun 23, 2020
leandrolanzieri
leandrolanzieri reviewed Jun 23, 2020
View reviewed changes
Copy link
Contributor

@leandrolanzieri leandrolanzieri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small documentation suggestions. Feel free to squash

dist/tools/pktbuf-stats/pktbuf-stats.py Outdated Show resolved Hide resolved
dist/tools/pktbuf-stats/pktbuf-stats.py Outdated Show resolved Hide resolved
dist/tools/pktbuf-stats/pktbuf-stats.py Outdated Show resolved Hide resolved
@miri64
Copy link
Member Author

miri64 commented Jun 23, 2020

Squashed.

@leandrolanzieri leandrolanzieri added CI: run tests If set, CI server will run tests on hardware for the labeled PR Reviewed: 2-code-design The code design of the PR was reviewed according to the maintainer guidelines Reviewed: 4-code-style The adherence to coding conventions by the PR were reviewed according to the maintainer guidelines Reviewed: 5-documentation The documentation details of the PR were reviewed according to the maintainer guidelines and removed CI: run tests If set, CI server will run tests on hardware for the labeled PR labels Jun 23, 2020
@leandrolanzieri leandrolanzieri added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Jun 23, 2020
leandrolanzieri
leandrolanzieri approved these changes Jun 23, 2020
View reviewed changes
Copy link
Contributor

@leandrolanzieri leandrolanzieri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be useful for the upcoming release. I tested again and the parsing is still working. Documentation looks good. ACK!

@leandrolanzieri leandrolanzieri merged commit d90140e into RIOT-OS:master Jun 23, 2020
@miri64 miri64 deleted the tools/feat/pktbuf-parser branch June 23, 2020 11:50
@miri64
Copy link
Member Author

miri64 commented Jun 23, 2020

Thanks for the review and testing :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@leandrolanzieri leandrolanzieri leandrolanzieri approved these changes

@fjmolinas fjmolinas Awaiting requested review from fjmolinas

@jia200x jia200x Awaiting requested review from jia200x jia200x is a code owner

Assignees

@leandrolanzieri leandrolanzieri

Labels
Area: tools Area: Supplementary tools CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Reviewed: 1-fundamentals The fundamentals of the PR were reviewed according to the maintainer guidelines Reviewed: 2-code-design The code design of the PR was reviewed according to the maintainer guidelines Reviewed: 3-testing The PR was tested according to the maintainer guidelines Reviewed: 4-code-style The adherence to coding conventions by the PR were reviewed according to the maintainer guidelines Reviewed: 5-documentation The documentation details of the PR were reviewed according to the maintainer guidelines Type: new feature The issue requests / The PR implemements a new feature for RIOT
Projects
None yet
Milestone
Release 2020.07
Development

Successfully merging this pull request may close these issues.
3 participants
@miri64 @leandrolanzieri @fjmolinas