Tips on Making compose
Beartype-Checkable
#298
Replies: 3 comments
-
Possibly, I am a @beartype expert. Some may dispute this claim. I curse them all!
...hoo, boy. Fascinating package! Thanks so much for directing our attention to this awesome thing you have made. From one Pythonista that abhors static type-checking to another such Pythonistia, I have possibly bad advice. Nobody likes advice – especially possibly bad advice. Prepare for such advice.
from compose import compose
from collections.abc import Callable
from numbers import Number
def multiply(a: Number, b: Number = 2) -> Number:
'''
Multiply the passed two numbers, defaulting the second passed number to 2
as a fallback for a doubling operation. Why? *Because we can.*
'''
return a*b
def increment(z: Number) -> Number:
'''
Increment the passed number by 1. Look. Just do what I say, Python!
'''
return z+1
increment_then_multiply: Callable[[Number, Number], Number] = compose(multiply, increment)
'''
Callable first multiplying the passed two numbers and then incrementing
the resulting number by 1. Believe in the power of function composition.
'''
assert increment_then_multiply(a=1.2, b=42) == 51.4 # <-- *woah* That is something that @leycec would actually understand. Anything less than that is nothing that @leycec would understand. Admittedly, I am beyond neuro-divergent. I have Asperger's and thus need algorithms to be explicitly laid out for me in literal terms that even a malignant sloth baby would understand.
class ComposeType(object):
'''
Abstract base class (ABC) of all composed functions.
'''
def __init__(self, *functions):
"""Initialize the composed function.
Arguments:
*functions: Functions (or other callables) to compose.
Raises:
TypeError:
If no arguments are given.
If any argument is not callable.
"""
if not functions:
raise TypeError(_name(self) + '() needs at least one argument')
_functions = []
for function in reversed(functions):
if not callable(function):
raise TypeError(_name(self) + '() arguments must be callable')
if isinstance(function, ComposeType):
_functions.extend(function.functions)
else:
_functions.append(function)
self.__wrapped__ = _functions[0]
self._wrappers = tuple(_functions[1:])
def __get__(self, obj, objtype=None):
"""Get the composed function as a bound method."""
wrapped = self.__wrapped__
try:
bind = type(wrapped).__get__
except AttributeError:
return self
bound_wrapped = bind(wrapped, obj, objtype)
if bound_wrapped is wrapped:
return self
bound_self = type(self)(bound_wrapped)
bound_self._wrappers = self._wrappers
return bound_self
class compose(ComposeType):
"""Function composition: compose(f, g)(...) is equivalent to f(g(...))."""
def __call__(self, /, *args, **kwargs):
"""Call the composed function."""
result = self.__wrapped__(*args, **kwargs)
for function in self._wrappers:
result = function(result)
return result
...
But that's probably overkill too, right? Python isn't that slow. Okay. Python is disastrously slow. But the only reason @beartype opted for Those same constraints don't really apply to Moreover, the efficiency cost of type-checking correctness is way less than the QA cost of not type-checking correctness. You pretty much always want to type-check correctness in your case. Since the The easiest way to enforce your API at runtime is probably with inline type hints + the standard from collections.abc import Callable
from inspect import signature
class ComposeType(object):
'''
Abstract base class (ABC) of all composed functions.
'''
def __init__(self, *functions: Callable) -> None:
"""Initialize the composed function.
Arguments:
*functions: Functions (or other callables) to compose.
Raises:
TypeError:
If no arguments are given.
If any argument is not callable.
"""
if not functions:
raise TypeError(_name(self) + '() needs at least one argument')
_functions = []
# "inspect.Signature" objects introspecting the currently and previously
# visited callables in the passed tuple of callables (respectively).
func_curr_signature = None
func_prev_signature = None
for function in reversed(functions):
if isinstance(function, ComposeType):
_functions.extend(function.functions)
else:
_functions.append(function)
func_curr_signature = signature(function)
if func_prev_signature:
if not len(func_curr_signature.parameters) == 1:
raise TypeError(
f'{func_curr_signature} accepts '
f'{len(func_curr_signature.parameters)} arguments '
f'rather than exactly one argument.'
)
# "inspect.Parameter" object introspecting the only argument
# accepted by this callable. Just trust us on this one. Yikes!
func_arg = next(iter(func_curr_signature.parameters.values()))
if func_arg.annotation != func_prev_signature.return_annotation:
raise TypeError(
f'{func_curr_signature} argument '
f'"{func_arg.name}" type {repr(func_arg.annotation)} != '
f'{func_prev_signature} return type '
f'{repr(func_prev_signature.return_annotation)}.'
)
#FIXME: Check for everything else, too. For example, you'll want
#to prohibit variadic arguments, keyword-only arguments...
#yadda, yadda! *sigh*
func_prev_signature = func_curr_signature
self.__wrapped__ = _functions[0]
self._wrappers = tuple(_functions[1:]) As the above sample code demonstrates (and as you yourself already astutely realized), what you want to do is well outside of Python's existing litany of Python Enhancement Proposals (PEPs). But @beartype is based almost entirely on PEPs! This means what you think it means: because PEPs cannot help you much here, @beartype cannot help you much here. I mean, @beartype can validate a few things of interest to both you and your users – but alot less than you'll like. You'll have to do most of the heavy lifting yourself. Thankfully, the standard And... there goes Wednesday evening. How did this happen to us, @beartype!?!? 😮💨 |
Beta Was this translation helpful? Give feedback.
-
Wow that is a very extensive response, with a whole lot of work put into it, thank you. Quick answers to the things that have quick answers:
|
Beta Was this translation helpful? Give feedback.
-
Wowza! Thanks so much for the extensive response. I detect a mind like mine. You exhaustively responded to each of my audacious concerns in bullet-point form, which is both impressive and gratifying. Sadly, I can't return the favour just yet. I apologize! Python 3.12 just dropped a few weeks ago. Since @beartype currently fails to support all the new syntactic goodness bundled in Python 3.12 (e.g., For the moment, let's quietly downgrade this to a discussion topic. This is an incredibly engaging discussion, though. I hope to revisit this in a week or two. When I inevitably fail to live up to my promises, please summon me with @leycec. Oh, and... I Thought of Something ElseAh, ha! That's right. More (ob)noxious fodder for your awesome Stubs, BroI realized that you can probably globally substitute R1 = TypeVarTuple('R1')
R2 = TypeVarTuple('R2')
...
@overload
def compose(f2: Callable[[*R1], *R2], f1: Callable[P, *R1], /) -> Callable[P, *R2]:
... Pretty sure that does what you want. Notably, that should transparently cover these two common cases:
What's the catch? You'll need to refactor your code to support that. Namely, you'll need to refactor your
This is what I am saying: def __call__(self, /, *args, **kwargs):
"""Call the composed function."""
# Current callable to be called.
func_curr = None
# Prior callable that was just called.
func_prev = self.__wrapped__
result = func_prev(*args, **kwargs)
for func_curr in self._wrappers:
# Type hint annotating the return of the prior callable if any
# *OR* "None" otherwise.
func_prev_return_hint = func_prev.__annotations__.get('return')
#FIXME: This is just pseudo-code. Pretend the is_hint_tuple_fixed()
#function exists. Nobody ever said Python was easy. If they did, they lied.
# If the prior function was annotated as returning a fixed-length
# tuple, unpack the fixed-length tuple returned by the prior call
# to that function and pass the resulting values as positional
# parameters to the current callable.
if func_prev_return_hint and is_hint_tuple_fixed(func_prev_return_hint):
result = function(*result)
# Else, pass this return as a single parameter to this callable.
else:
result = function(result)
func_prev = func_curr
return result New
|
Beta Was this translation helpful? Give feedback.
-
I am hoping some of you beartype experts can just instantly answer this.
I've got this package which is a real pain for static type-checking:
compose
. The Python type-hinting system is particularly inhospitable to doing anything interesting with callables.But there was popular demand for type checker support, so I provide almost-complete static type-checking support with brute-force
@typing.overload
for arities 1-16 (and every permutation of regular and awaitable return types, for theasync
composition variants). Truly, more elegant software has never been written. For a mix of reasons, I did this in separate stub files, in a separate package:compose-stubs
.For the purposes of runtime type-checking, we can do better.
When type-checking if callable arguments to a composition constructor are the right type, we could take beartype's randomized check-only-some-of-the-things approach: the single mandatory argument of one callable argument must match the return value of the next - for large argument lists we could sample some pairs of arguments rather than all.
When type-checking if a composed instance matches a callable signature, it's barely more than the checking which correctly covers any callable - the only thing that need to be different for composed instances is that the return type is checked on
.functions[-1]
.When type-checking if an object is a composed instance, a simple
isinstance
check is fine.So, any tips on what my package should do to provide my users a great beartype type-checking experience?
Beta Was this translation helpful? Give feedback.
All reactions