Make create_memory_object_stream generic

[gschaffner]

This supports annotating memory object streams with uninstantiable types that the current item_type hack currently does not work for, e.g.

create_memory_object_stream(item_type=Callable)
# error: No overload variant of "create_memory_object_stream" matches argument type
# "_SpecialForm"  [call-overload]

(See also the discussion around https://matrix.to/#/!JfFIjeKHlqEVmAsxYP:gitter.im/$sGb5WyohiQVFuiPdfb0GZ54h-CqN76sxeKKiCWovCdY?via=gitter.im&via=matrix.org)

@agronholm it turns out I was mistaken—due to a bug(?) in mypy, what I thought was a way to do this in a backwards compatible way isn't quite backwards compatible. While c93f3ec (note: not in this PR) is backwards-compatible at runtime (emitting a deprecation warning currently, but v3 could skip this) and backwards compatible in the sense that mypy will correctly infer types for e.g.

s0, r0 = create_memory_object_stream(item_type=float)
s1, r1 = create_memory_object_stream()

it would be a breaking change since it makes unannotated calls emit

s1, r1 = create_memory_object_stream()
# error: Need type annotation for "s1"  [var-annotated]
# error: Need type annotation for "r1"  [var-annotated]

on mypy. Note that mypy does infer a default item type of Any for s1 and r1, but it still wants a type annotation. This is not affected by --allow-any-generic or --disallow-any-generic. It appears to be a known problem, see python/mypy#3737 (comment).

Given that c93f3ec would produce spurious var-annotated on anyio 4 too, I think you are right that this has to be a breaking change in v4.

[gschaffner]

the CI network failure seems unrelated and likely just needs a rerun

[agronholm]

This looks really good! I took the liberty of making some additional changes. Would you mind checking them to make sure I didn't do anything stupid?

[gschaffner]

thanks! your changes look great. i've also just added a couple more annotations to the docs.

[gschaffner]

i just noticed that trio implemented something similar to this a couple months after I posted this. their implementation is more complicated though—specifically, they only use an implementation like mine if TYPE_CHECKING, with a comment stating that it can't be used at runtime on < 3.9. however in my testing this implementation seems fine on 3.{7,8}.

if you don't mind, can you hold off merging this until I figure out why the trio folks did something different? I want to make sure I haven't done something wrong here.

[agronholm]

if you don't mind, can you hold off merging this until I figure out why the trio folks did something different? I want to make sure I haven't done something wrong here.

Alright, there's no rush.

[agronholm]

As this is a backwards incompatible change, it won't affect the 3.x update I am putting together.

[agronholm]

@gschaffner I'd like to get 4.0 out in a couple weeks. Have you made progress with your investigation?

[agronholm]

I've given this another look, and I can't find any fault in it. Trio's way of doing this is more generic, but potentially also less compatible with type checkers. If there's a problem, we can sort it out during the release candidate phase.

[agronholm]

Anyway, thank you for this!

[gschaffner]

aah, i'm sorry for the delay. thank you for re-reviewing and seeing this in!