strange array element assignment error

[rwgardner]

It's possible that this is not repeatable, but on my machine with Julia Version 0.2.0-1644.rb3c30f37, Commit b3c30 f373b 2013-05-21 05:03:45, there is the following issue. I believe the function foo below should behave like "convert(Array{typeof(x[1]), 1}, x)" (for some inputs x at least). However, the code:

function foo(x)
    ret=Array(typeof(x[1]), length(x))
    for j = 1:length(x)
        ret[j] = x[j]  
    end
    return ret
end

x = Array(Union(Dict{Int64,String},Array{Int64,3},Number,String,Nothing), 3)
x[1] = 1.0
x[2] = 2.0
x[3] = 3.0

foo(x)

gives non-deterministic, incorrect results like

3-element Float64 Array:
7.38985e-317
3.20647e-316
3.28281e-316

sometimes and also the error

ERROR: no method convert(Type{Array{Int64,3}},Float64)
 in convert at base.jl:9
 in foo at none:4

corresponding to the line

ret[j] = x[j]

The foo code seems to work fine in a global context and convert(Array{typeof(x[1]), 1}, x) seems to work fine.

[JeffBezanson]

I will fix this. But, that is one hell of an array type. At that point I would recommend using Array{Any}.

[rwgardner]

But, that is one hell of an array type. At that point I would recommend using Array{Any}.

Agreed. I didn't create it manually. I had just stripped out a bunch of code from where the problem was really occurring to manually recreate the bug as concisely as possible. (The object of that type actually came from the JSON package.)

Thanks.

[JeffBezanson]

Ah, that makes sense! I filed a pull request with JSON.jl to change it to Any.

[Keno]

I can't reproduce. Was this fixed? @vtjnash @JeffBezanson

[vtjnash]

this wasn't fixed. see comments on my patch. jl_matching_methods can still return incorrect information, _methods currently just works around, that to ensure that that information can't propagate into type inference

[JeffBezanson]

This problem no longer manifests, ergo fixed. If you have a failing case, please post it and reopen.

[vtjnash]

Does my comment above that describes how inference is working around this (by splitting Union types) count?

[JeffBezanson]

No, I'm having trouble extracting a test case from that comment. I agree the change to _methods is a workaround, but if the workaround works then the issue is fixed. What is the "incorrect information"? Is this due to type intersection not working well enough on union types? That will have to wait for #8974.

[vtjnash]

From memory (and iPhone), the failing case took the form:
_methods(abs, Tuple{Union{Int64,Float64}})

The type intersection says we should use abs(::Number) -- aka abs(::Union{Int64,Float64}, aka what invoke should call -- but the runtime should actually be either abs(::Int64) or abs(::Float64).

[JeffBezanson]

That's not how it works; we take the intersection with all methods, not just pick one that matches the inferred type:

julia> ccall(:jl_matching_methods, Any, (Any,Any,Int32), abs, Tuple{Union(Int64,Float64)}, -1)
3-element Array{Any,1}:
 svec(Tuple{Int64},svec(),abs(x::Signed) at int.jl:44)                
 svec(Tuple{Float64},svec(),abs(x::Float64) at float.jl:301)          
 svec(Tuple{Union(Float64,Int64)},svec(),abs(x::Real) at number.jl:26)

[ScottPJones]

Could that be written correctly: Union(Tuple{Float64},Tuple{Int64})? (If not, pardon my interrupting!)

[JeffBezanson]

Those types are equal (although the algorithm we use currently doesn't know it). One is not more correct than the other.

[ScottPJones]

OK, but would that work around this problem as well (i.e. is the problem really that the currently algorithm doesn't know that Tuple{Union(Float64,Int64)} and Union(Tuple{Float64},Tuple{Int64}) are equal, as you stated?), instead of splitting it into two methods? Pardon my ignorance!!!

[JeffBezanson]

That's basically what @vtjnash 's workaround does (factoring out Unions). But it isn't necessary in general, only to cover some rare-ish cases in type intersection. I do expect to be able to retire the workaround after #8974.

[ScottPJones]

Thanks! 👍