RFC: experiments in diagonal constraints in jl_args_morespecific (ambiguity/method sorting)

[timholy]

At the moment this is purely an experiment, as it doesn't even get as far as building inference. But I thought I'd post it as potentially thought-provoking. This is motivated by the assumption that #8974 is julia-0.6 at the earliest, and wondering if we can make some improvements in ambiguity-detection and method-sorting now.

This mostly explores TypeVars and "diagonal" constraints. Over in #11242 (comment) @yuyichao came up with some great examples of interesting cases for our current type-specificity computations. All of his examples are currently resolved using the rule that if any argument is more specific than any other, and no argument is less specific, then the method is more specific.

However, this leads to problems like #10174. I've wondered whether ambiguity should be defined along the lines of "two non-identical signatures, neither of which is consistently more specific than the other over the domain of their intersection." The italicized part is potentially a big change, for example:

f1{T<:Number}(::T, ::T) = 1
f1(::Integer, ::Integer) = 2

Currently f1(1,1) == 2, because Number is broader than Integer. But if one adopts the "over the domain of their intersection" perspective, this doesn't make a lot of sense: both of these methods are candidates only when both args are integers, and when that's true definition 1 is actually more restrictive, since it additionally requires that both arguments are of the same type.

This PR explores the possibility of using type-intersection to "nail down" TypeVars and plug them in before running jl_args_morespecific. It borks building int.jl due to situations like this:

f6{T<:Number}(::T, ::T) = error("No f6 defined for ", T)
f6{T<:Signed}(x::T, y::T) = 2

which are judged as having the same priority, because in both cases T gets set to Signed. So, this isn't right, and may not be a productive direction (esp. at this stage in the julia-0.5 release), but I thought it was worth a couple of hours.

[StefanKarpinski]

I'm a bit confused by this. In the case of f1, if we consider only tuples of arguments that both methods can apply to, doesn't that mean that (by definition) they're always equally restrictive – i.e. not at all?

[timholy]

If you just define the first one it might be clearer:

julia> f1{T<:Number}(::T, ::T) = 1
f1 (generic function with 1 method)

julia> f1(3, 0x03)
ERROR: MethodError: no method matching f1(::Int64, ::UInt8)
Closest candidates are:
  f1{T<:Number}(::T<:Number, ::T<:Number)
 in eval(::Module, ::Any) at ./boot.jl:230

It requires they are of the same type.

[StefanKarpinski]

I'm afraid that doesn't clear it up for me – you can give an example the other direction e.g. f1(1.5, 2.5).

[timholy]

Agreed. Naturally, you asking the question made me think this through a little more. For reference, here's a diagram that might help talk about this:

When I say "domain of intersection", I now realize that as I've been describing this I have not always been consistent whether I mean "full" (diagonal) intersection or per-argument intersection. For the former, the area of intersection consists of the blue squares and there's no distinction between the methods (i.e., this is a useless line of thinking). For the latter, the domain of intersection is the square enclosed in yellow---within that square, the diagonal case is more specific because it applies to fewer methods (I = method defined in terms of Integer, D = diagonal method), and in particular the applicability of the diagonal method implies the applicability of the Integer method (but not vice versa).

So I guess what I'm really wondering is whether one may be less likely to get fewer "spurious" ambiguities (e.g., #10174) if we first do the analysis over the rectangular domain of per-argument intersection, and then basically ask whether within that domain the applicability of one method always implies the applicability of the other. When that's true, then you have a consistent specificity arrangement. But it's clearly not sufficient to define specificity on its own, given my f6 example above.

[StefanKarpinski]

Ah, ok. That clears it up. It also makes it "another heuristic" – albeit, potentially a good/better one. Regarding the f6 example, note that this case doesn't come up if you do "full" specificity comparison first since the f6{T<:Signed}(::T, ::T) case applies to a strict subset of the cases that f6{T<:Number}(::T, ::T) applies to. If you only apply this idea to cases that are otherwise ambiguous, it might work.

[StefanKarpinski]

Also, Tim, I'm super impressed with your diagram (and the making of this PR in the first place).

[timholy]

Yeah, I was originally excited about this mostly because I failed to realize that it probably only changes which heuristics we use, not eliminate some of them.

And for the diagram, just thank Gadfly/Immerse, it was really trivial to make.

[timholy]

Having thought about this a bit more on my bike home, rather than trying to decide which is more specific than the other, I think this is best thought of as a missing ambiguity error. Currently f1 does not throw an error on the blue squares, but it should.