Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Template: Analytics and SIEM #4281

Merged
merged 79 commits into from
Oct 2, 2019
Merged

Template: Analytics and SIEM #4281

merged 79 commits into from
Oct 2, 2019

Conversation

jochman
Copy link
Contributor

@jochman jochman commented Sep 2, 2019

Status

Ready

Description

New integration templates

Does it break backward compatibility?

  • No

Must have

  • Tests
  • Documentation (with link to it)
  • Code Review

Jochman added 11 commits August 19, 2019 16:49
 - Analytics & SIEM
 - Authentication
 - Case Management
 - Analytics & SIEM
 - Authentication
 - Case Management
creadet authentication
added BaseClient and DemistoException to CommonServerPython
@jochman jochman self-assigned this Sep 2, 2019
@jochman jochman requested a review from ronykoz September 2, 2019 07:14
glicht
glicht previously requested changes Sep 2, 2019
Copy link
Contributor

@glicht glicht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Work with @kirbles19 to create the Templates/README.md which explains a bit about each template and how to use it.

Jochman added 3 commits September 2, 2019 14:07
@jochman
Copy link
Contributor Author

jochman commented Sep 2, 2019

Work with @kirbles19 to create the Templates/README.md which explains a bit about each template and how to use it.

I'll work with him also on the Docstring

Copy link
Contributor

@ronykoz ronykoz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have duplicate files, please fix that. And arrange the files to be in the structure we discussed.
Templates/Integrations//files...

I've went only through IntegrationTemplates/AnalyticsAndSIEM/AnalyticsAndSIEM.py and the commonServerPython, if you think that there are some other things in the other files that might be relevant please let me know.

And please note that i've not mentioned repeated comments, please fix them everywhere.

''' GLOBALS/PARAMS '''
INTEGRATION_NAME: str = 'Analytics & SIEM Integration'
# lowercase with `-` dividers
INTEGRATION_NAME_COMMAND: str = 'analytics-and-siem'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
INTEGRATION_NAME_COMMAND: str = 'analytics-and-siem'
INTEGRATION_COMMAND_NAME: str = 'analytics-and-siem'

from CommonServerPython import *
from CommonServerUserPython import *

''' IMPORTS '''
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My own preference which makes the code a bit more readable is to order the imports(And anywhere else where the order doesn't matter) by the line length(like the ones for the demistomock)

INTEGRATION_NAME_CONTEXT: str = 'AnalyticsAndSIEM'


class Client:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add class docstring which will explain it usage, it should also contain the documentation for the class attributes


class Client:
def __init__(self, server: str, use_ssl: bool, fetch_time: Optional[str] = None):
self._server: str = server.rstrip(chars='/')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like an overkill with the argument types, you've mentioned them in the function itself, no need to overdo it.
It makes it less readable and you should use it only when needed.

def _http_request(self, method: str, url_suffix: str, full_url: str = None, headers: Dict = None,
auth: Tuple = None, params: Dict = None, data: Dict = None, files: Dict = None,
timeout: float = 10, resp_type: str = 'json') -> Any:
"""A wrapper for requests lib to send our requests and handle requests
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be one line

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to read the PEP257 guide https://www.python.org/dev/peps/pep-0257/ and https://google.github.io/styleguide/pyguide.html
You have this:

The docstring is a phrase ending in a period. It prescribes the function or method's effect as a command ("Do this", "Return that"), not as a description; e.g. don't write "Returns the pathname ...".

And

Args:
List each parameter by name. A description should follow the name, and be separated by a colon and a space. If the description is too long to fit on a single 80-character line, use a hanging indent of 2 or 4 spaces (be consistent with the rest of the file).
The description should include required type(s) if the code does not contain a corresponding type annotation.
If a function accepts *foo (variable length argument lists) and/or **bar (arbitrary keyword arguments), they should be listed as *foo and **bar.

@@ -0,0 +1,443 @@
import demistomock as demisto
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add module docstring

@@ -0,0 +1 @@

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some description here, and mention the usage of it

verify,
proxy
):
"""Base Client for use in new integrations
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be the class docsting

integration_name_command,
integration_name_context,
verify,
proxy
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add default

self._proxies = handle_proxy()
else:
self._proxies = None
self._base_url = '{}{}'.format(self._server, base_suffix)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move it before the proxy handling

Jochman added 4 commits September 3, 2019 14:41
added @Property to vars in BaseClient
added default to __init__ in BaseClient
 * Added build_dbot_entry
 * Added build_malicious_dbot_entry
 * Added build_dbot_entry
 * Added build_malicious_dbot_entry
@kirbles19 kirbles19 self-assigned this Sep 4, 2019
Copy link
Contributor

@Itay4 Itay4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change readme to README

@orenzohar
Copy link
Contributor

are we no longer going to use @logger? @Itay4 @anara123

@jochman
Copy link
Contributor Author

jochman commented Sep 24, 2019

are we no longer going to use @logger? @Itay4 @anara123

May I should add it to every function in the template then it'll get used much more.
what do you think?

@anara123
Copy link
Contributor

we can use @logger.


# Switch case
commands = {
'test-module': test_module,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it appears that if your return_outputs('ok', None, None) it will fail the integration Test. So change return_outputs to support this option

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}
try:
if command == 'fetch-incidents':
incidents, new_last_run = commands[command](client, last_run=demisto.getLastRun())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to do it generically when you know exactly which function your are going to execute. Replace it with fetch_incidents

return f'{INTEGRATION_NAME} - Could not find any events.', {}, {}


def get_event(client: Client, args: Dict) -> Tuple[str, Dict, Dict]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change get_event to be get_event_command , do the same for other functions. And remove _request from Client functions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we've discussed @ the first presentation and that's what agreed upon all of us.

''' COMMANDS '''


def test_module(client: Client, *_) -> Tuple[str, Dict, Dict]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is *_ and test_module dont have demisto.args(), the user will decide which parameters to pass to test_module

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*_ are ignored args.

def fetch_incidents(
client: Client,
fetch_time: str,
last_run: Optional[datetime] = None) -> Tuple[List, datetime]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

last_run should be str not datetime. you cant save datetime in setLastRun() AFAIK


def main(): # pragma: no cover
params = demisto.params()
base_url = f"{params.get('url', '').rstrip('/')}'/api/v2/'"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont think we need to provide default value of '' url. We should return error in that case
use urljoin(params.get('url'), '/api/v2/')

proxy = params.get('proxy')
client = Client(base_url=base_url, verify=verify_ssl, proxy=proxy)
command = demisto.command()
demisto.info(f'Command being called is {command}')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to info this. change to debug

@jochman jochman mentioned this pull request Sep 25, 2019
@jochman jochman requested review from glicht, ronykoz and anara123 and removed request for glicht, yaakovi and ronykoz October 2, 2019 19:31
@anara123 anara123 dismissed stale reviews from glicht, ronykoz, and yaakovi October 2, 2019 20:25

fixed

@anara123 anara123 merged commit dd73004 into master Oct 2, 2019
@anara123 anara123 deleted the new-templates branch October 2, 2019 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants