Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement] Async Pcap(Ng)Reader implementations #4503

Open
calligraf0 opened this issue Aug 16, 2024 · 0 comments
Open

[enhancement] Async Pcap(Ng)Reader implementations #4503

calligraf0 opened this issue Aug 16, 2024 · 0 comments

Comments

@calligraf0
Copy link

calligraf0 commented Aug 16, 2024

Hi, first time opening an issue hope I do everything right 😃 , I choose not to go for the issue template as it seemed out of context for this since it's not a bug but a proposal.

Intro

As for the current state of Scapy there is no async-aware implementation for the PcapReader/PcapNgReader classes which may instead be useful when parsing many large .pcap* files using Scapy.

Solution?

I am not too familiar with the codebase but I tried to put together a horribly hacked version of possible implementation: at https://github.com/calligraf0/scapy/tree/async-pcap-reader (there is also a junkyard.py file which contains a "benchmark" which currently still lags behind the multithread sync implementation)

The main limitations I stumbled across right now are:

  • this obviously requires a python version which supports asyncio
  • lack of gzip support (the code I smashed together currently does not handle gzip compressed files, as I couldn't find a valid "drop in replacement" for gzip which supports async)
  • pcap/cap only support (I only implemented the bare minimum, so no PcapNgReader for now)
  • ugly defer of header parsing on first packet read (because of async, I couldn't use aiofiles in the __init__ method, a possible solution could be using the AsyncMixin class / rework the way the file is handled?)
  • I am not a great programmer 😅 (that has no immediate solution 😝 )

Questions

  • Is this a valid/interesting enhancement proposal? If so, is the solution I PoC'ed together a viable way (providing an extra class AsyncPcapReader instead of modifying the RawPcapReader class to also support async)?
  • I am happy to try give this a better shot, are there any pointers on what to improve/rework?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant