Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Palo Alto ARP table plugin #2613

Merged
merged 30 commits into from
May 15, 2024
Merged

Conversation

Slenderman00
Copy link
Contributor

Implements a plugin that fetches ARP tables from Palo Alto network equipment
#2343

@CLAassistant
Copy link

CLAassistant commented Apr 27, 2023

CLA assistant check
All committers have signed the CLA.

@Slenderman00 Slenderman00 marked this pull request as draft April 27, 2023 14:33
@github-actions
Copy link

github-actions bot commented Apr 28, 2023

Test results

       8 files         8 suites   7m 20s ⏱️
3 320 tests 3 320 ✔️ 0 💤 0
6 372 runs  6 372 ✔️ 0 💤 0

Results for commit 00bd003.

♻️ This comment has been updated with latest results.

@Slenderman00 Slenderman00 marked this pull request as ready for review May 2, 2023 13:06
@sonarcloud
Copy link

sonarcloud bot commented May 4, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@lunkwill42
Copy link
Member

Implements a plugin that fetches ARP tables from Palo Alto network equipment #2343

Thanks a bunch for your much anticipated contribution, @Slenderman00 !

We seem to currently be having some unrelated issues with our CI test suite, which also causes the tests to fail for your PR (See #2617). We'll try to fix those issues in the master branch before evaluation your contribution properly.

@hmpf hmpf requested review from hmpf and lunkwill42 May 24, 2023 08:47
Copy link
Contributor

@hmpf hmpf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Python-wise this looks fine to me. (If I were a perfectionist I would point at the class name of the hidden inner class but since I've worked hard not to be a perfectionist I won't do that ;) )

@lunkwill42
Copy link
Member

Rebased this on the latest master and ran a force-push. My apologies that it took so long to give this a proper look - summer vacation time snuck up on us really fast...

@codecov
Copy link

codecov bot commented Sep 4, 2023

Codecov Report

Attention: Patch coverage is 76.81159% with 16 lines in your changes are missing coverage. Please review.

Project coverage is 60.23%. Comparing base (a2be786) to head (7ff53ee).
Report is 3 commits behind head on master.

Files Patch % Lines
python/nav/ipdevpoll/plugins/paloaltoarp.py 76.81% 16 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2613      +/-   ##
==========================================
+ Coverage   60.21%   60.23%   +0.02%     
==========================================
  Files         601      602       +1     
  Lines       43981    44050      +69     
==========================================
+ Hits        26481    26534      +53     
- Misses      17500    17516      +16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@lunkwill42 lunkwill42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've not done an in-depth review, but most of it looks ok.

No. 1 issue is that there is no test coverage. As you will see, CodeCov will reject this PR since it has less than 50% coverage, and also lowers the overall coverage number for the entire codebase.

Any thoughts on how this could reasonably tested?

python/nav/ipdevpoll/plugins/paloaltoarp.py Outdated Show resolved Hide resolved
@Slenderman00
Copy link
Contributor Author

Slenderman00 commented Sep 11, 2023

for testing i was thinking of just feeding the class some xml data and then verifying the output.

@lunkwill42
Copy link
Member

for testing i was thinking of just feeding the class some xml data and then verifying the output.

Sounds reasonable to me :)

@sonarcloud
Copy link

sonarcloud bot commented Oct 10, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@mehmetes-sudo
Copy link

Hi,

May I ask you please to add this plugin to the next release ?

We have been waiting for this plugin for long time.

Thank you for your understanding

Copy link

sonarcloud bot commented Nov 27, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@lunkwill42
Copy link
Member

May I ask you please to add this plugin to the next release ?

That would have been nice, but the tests don't work, and there are still issues with the code, so we won't have time to fix this on @Slenderman00's behalf for the next release.

@Slenderman00
Copy link
Contributor Author

Slenderman00 commented Mar 7, 2024

@lunkwill42 1137c1a fixes the tests.

@lunkwill42
Copy link
Member

@lunkwill42 1137c1a fixes the tests.

Thank you - we still won't have time to evaluate this properly for the next release, as this is the last day of our sprint. But we will be putting this at the top of our backlog for the next sprint (about 1 week after Easter, I believe).

@lunkwill42
Copy link
Member

Instead of the eternal ping-pong of getting this PR into shape, I've rebased it on the current master and started working on completing it myself. It's still a work in progress, so I'm converting it to a draft PR for now.

@lunkwill42 lunkwill42 marked this pull request as draft April 29, 2024 19:55

class SslPolicy(client.BrowserLikePolicyForHTTPS):
def creatorForNetloc(self, hostname, port):
return ssl.CertificateOptions(verify=False)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not accustomed to using the Twisted client libraries for this sort of thing. Does this in fact mean that we explicitly turn off TLS certificate verification? If so, is that really what we want when talking to a security-specific product? :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed this at UiT last year. Since we are only extracting ARP data this should be fine as long as the keys and permissions on the Palo Alto device are configured correctly (To only allow for fetching ARP data). But yes we are explicitly turning of TLS certificate verification on all requests made by the plugin.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be better to implement certificate pinning instead of this solution just in case someone manages to misconfigure their permissions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A MITM attack might also allow for data infiltration, certificate pinning is starting to seem like the best option.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To get this PR done with, I would just make note of this fact in the docs. That way, you could submit a new PR with changes for certificate config options (i.e. switch verification on/off or point to a pinned certificate)

Copy link
Contributor

@stveit stveit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not much to comment on, but theres a lack of test coverage

python/nav/ipdevpoll/plugins/paloaltoarp.py Outdated Show resolved Hide resolved
Access to the configuration of PaloAlto API keys is required before
plugin instances are created, so we might as well fetch the global
ipdevpoll config when the plugin class is loaded and cache the details
in the class object itself.

Also, there is no need to inefficiently process the configuration as
a list when a dict will do the trick much faster and with less code.
PyOpenSSL is *not* a requirement for the test suite only: It's required
to make the PaloAlto ARP plugin work in production!
The address doesn't necessarily need to be an IP, it could also
be a hostname, so the more generic `address` name is preferable.
We follow PEP-8: Class names should not begin with lower-case letters
unless there is a really specific or necessary reason for it.
Even if we do want a broad exception handler, never use just `except`,
as that will catch even low-level Python exceptions.  Added a noqa tag
and log the full exception with a slightly more information message.
We're expecting to switch to using towncrier at any moment in NAV, so
might as well add a news fragment for this PR right away.
The intent is to fallback to look up using the sysname if the host isn't
configured using an IP address.
@lunkwill42
Copy link
Member

not much to comment on, but theres a lack of test coverage

I tried to increase the coverage ever so slightly, with the result that whereas CodeCov previously greenlighted my changes, it's now nixing them :P

@lunkwill42 lunkwill42 marked this pull request as ready for review May 2, 2024 11:17
@lunkwill42 lunkwill42 added the arp label May 2, 2024
@lunkwill42 lunkwill42 dismissed their stale review May 2, 2024 11:20

Fixed my requests myself, someone else needs to review.

@hmpf hmpf self-requested a review May 2, 2024 11:26
Copy link
Contributor

@hmpf hmpf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now all that is missing is @Slenderman00 testing this on real hardware?

requirements/base.txt Show resolved Hide resolved
@lunkwill42
Copy link
Member

What do you think, @Slenderman00, do our updates make sense to you? We're ready to merge it, but have no way of testing it against real gear.

@Slenderman00
Copy link
Contributor Author

What do you think, @Slenderman00, do our updates make sense to you? We're ready to merge it, but have no way of testing it against real gear.

It looks really good, I should be able to test it on real hardware later today.

@lunkwill42 lunkwill42 linked an issue May 3, 2024 that may be closed by this pull request
@Slenderman00
Copy link
Contributor Author

I have now tested it on real hardware, everything seems to work fine.

@lunkwill42
Copy link
Member

I have now tested it on real hardware, everything seems to work fine.

Then let's roll with this for NAV 5.10! 🎉

@lunkwill42 lunkwill42 merged commit a52309e into Uninett:master May 15, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Palo Alto ARP table import for Machine Tracker
7 participants