Add mypy + some other misc. changes #54
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #53
Changes
Docstring clarification for
FivetranSensor
I added this because I thought I was working on #49 when I first started coding.
In any event, I added language clarifying how
FivetranSensor
works more specifically than what was there.Type-checking
A lot of
hooks.py
was un-annotated, so I made sure to get the whole thing annotated.I then added Mypy to the pre-commit. I also added
mypy
to the CI._do_api_call()
and_do_api_call_async()
API changesI made these more plainly wrap the
requests.request()
andasyncio.ClientSession.request
APIs. It did not seem like there was much reason for these APIs to deviate. Deviations include:json
that maps to either thedata=json
orparams=json
kwargs in therequests.request()
call. This seems less desirable than just passingdata=data
,params=params
, orjson=json
. Specifically, thejson
kwarg is confusing from the perspective of therequests
andaiohttp
APIs: in those APIs,json=
expects a dict anddata=
expects a string, but in the FivetranHook API,json=
is a string that gets mapped todata=
.requests.request()
)requests.request()
can do that for you.That said, changing this is probably annoying for users as there may be many users using this function. So I made sure the old API works and gave a deprecation warning via the following:
Lastly, I removed the test for
"UNKNOWN"
. The reason why is becauserequests.request()
can handle any method passed in, and the Fivetran API itself should be in charge of rejecting bad methods. (Note that this is the only nontrivial test change I made; all other test cases pass with essentially no changes, meaning my code is more or less a refactor.)TLDR:
_do_api_call()
now matchesrequests.request()
DeprecationWarning
"Breaking" changes
I put "breaking" in quotes because these are very minor API features that were not being used by anything. That said they still constitute breaking changes, in some sense, if we consider changing the
FivetranHook
's method return types to constitute breaking changes (even if those return types are not so useful).FivetranHook.check_connector
now returns theconnector_details
dict object. (So thatprep_connector
can avoid making an unnecessary 3rd API call.)FivetranHook.prep_connector
now returnsNone
instead ofLiteral[True]
.Use
hook
as cached propertyIt is more idiomatic in Airflow code for the
hook
for an operator to be called as acached_property
, so I swapped the operator and sensor code to implement this pattern, and added a deprecation warning for_get_hook()
.