stagedfunction-related compilation error when type inference fails #8504
Comments
|
While waiting for something else to compile, I looked at this again. Seems like it's a bug in our implementation of stagedfunctions, because I can use this to segfault julia (check out & build the |
|
Using the test script at the top of this issue, valgrind gives me this: No such This is new territory for me, but does this suggest a missing GC root or something? |
|
(That's in our usage of |
|
As far as I know |
|
|
Updating #8501 to today's master yields the following new behavior: it fails on the first call, and succeeds thereafter. No segfault, either. julia> function as_sub(x::AbstractMatrix)
y = similar(x, eltype(x), tuple(([size(x)...]*2)...))
@show typeof(y)
y = sub(y, 2:2:size(y,1), 2:2:size(y,2))
for j=1:size(x,2)
for i=1:size(x,1)
y[i,j] = x[i,j]
end
end
y
end
as_sub (generic function with 1 method)
julia> as_sub(eye(2))
ERROR: `ndims` has no method matching ndims(::Type{Any})
in ndims at abstractarray.jl:22 (repeats 3 times)
in setindex! at subarray2.jl:34
julia> as_sub(eye(2))
typeof(y) = Array{Float64,2}
2x2 SubArray{Float64,2,Array{Float64,2},(StepRange{Int64,Int64},StepRange{Int64,Int64})}:
1.0 0.0
0.0 1.0
julia> as_sub(eye(2))
typeof(y) = Array{Float64,2}
2x2 SubArray{Float64,2,Array{Float64,2},(StepRange{Int64,Int64},StepRange{Int64,Int64})}:
1.0 0.0
0.0 1.0 |
|
I've finally gotten around to reducing this to a relatively simple test case that can be run on master. module AVDebug
import Base: setindex!
type View{T,N,P<:AbstractArray,I<:(RangeIndex...)} <: AbstractArray{T,N}
parent::P
indexes::I
dims::NTuple{N,Int}
end
stagedfunction subview(A::AbstractArray, I::RangeIndex...)
T = eltype(A)
N = length(I)
@show T
@show N
@show A
@show I
:(AVDebug.View{$T,$N,$A,$I}(A, (I[1],I[2]), (length(I[1]), length(I[2]))))
end
stagedfunction setindex!{T,N,P}(A::View{T,N,P}, v, i1, i2)
println("P = ", P)
println("ndims = ", ndims(P))
:(A.parent[2*i1,2*i2] = v)
end
endwith the test script: function as_sub(x::AbstractMatrix)
y = similar(x, eltype(x), tuple(([size(x)...]*2)...))
@show typeof(y)
Y = AVDebug.subview(y, 2:2:size(y,1), 2:2:size(y,2))
@show typeof(Y)
for j=1:size(x,2)
for i=1:size(x,1)
Y[i,j] = x[i,j]
end
end
Y
end
as_sub(eye(2))which yields the following output: julia> include("views_broadcast_test.jl")
T = Any
N = 2
A = Array{Float64,N}
I = (StepRange{Int64,Int64},StepRange{Int64,Int64})
P = _<:AbstractArray{T,N}
T = Any
N = 2
A = Array{Float64,N}
I = (StepRange{Int64,Int64},StepRange{Int64,Int64})
P = _<:AbstractArray{T,N}
typeof(y) = Array{Float64,2}
typeof(Y) = View{Any,2,Array{Float64,N},(StepRange{Int64,Int64},StepRange{Int64,Int64})}
P = Array{Float64,N}
ERROR: `ndims` has no method matching ndims(::Type{Any})
in ndims at abstractarray.jl:22 (repeats 3 times)
in setindex! at /home/tim/tmp/views_broadcast.jl:23
in as_sub at /home/tim/tmp/views_broadcast_test.jl:8
in include at ./boot.jl:242
in include_from_node1 at ./loading.jl:128
while loading /home/tim/tmp/views_broadcast_test.jl, in expression starting on line 14Interestingly, if you change the stagedfunction subview{T,NP}(A::AbstractArray{T,NP}, I::RangeIndex...)
N = length(I)
@show T
@show N
@show A
@show I
:(AVDebug.View{$T,$N,$A,$I}(A, (I[1],I[2]), (length(I[1]), length(I[2]))))
endthen it works, with the following output: julia> include("views_broadcast_test.jl")
T = Float64
N = 2
A = Array{Float64,N}
I = (StepRange{Int64,Int64},StepRange{Int64,Int64})
P = _<:AbstractArray{T,N}
T = Float64
N = 2
A = Array{Float64,N}
I = (StepRange{Int64,Int64},StepRange{Int64,Int64})
P = _<:AbstractArray{T,N}
typeof(y) = Array{Float64,2}
T = Float64
N = 2
A = Array{Float64,2}
I = (StepRange{Int64,Int64},StepRange{Int64,Int64})
typeof(Y) = View{Float64,2,Array{Float64,2},(StepRange{Int64,Int64},StepRange{Int64,Int64})}
P = Array{Float64,2}
ndims = 2
Error showing value of type View{Float64,2,Array{Float64,2},(StepRange{Int64,Int64},StepRange{Int64,Int64})}:
ERROR: `size` has no method matching size(::View{Float64,2,Array{Float64,2},(StepRange{Int64,Int64},StepRange{Int64,Int64})})
in showarray at show.jl:1097
...(that final error is irrelevant, here I just didn't define a Omitting the |
|
Just to summarize what seems to be wrong:
So this seems to again be a case where method sorting is insufficiently picky; it's willing to tolerate a match to non-concrete types even once the types become concrete. |
This circumvents the inference problem in JuliaLang/julia#8504
|
With the recent SubArray work, I was able to work around this by declaring parameters With a recent julia-0.4 (one that includes the SubArray work), I've already committed a work-around for this problem to Images, but if one modifies these lines "back" to this: function sliceim(img::AbstractImage, I::RangeIndex...)
dimmap = Array(Int, ndims(img))
n = 0
for j = 1:ndims(img)
if !isa(I[j], Int); n += 1; end;
dimmap[j] = n
end
S = slice(img.data, I...)
@show typeof(img.data)
@show I
@show typeof(S)
ret = copyproperties(img, S) then one gets this julia> using Images, TestImages
# unfixable (false) ambiguity warning suppressed
julia> img = separate(testimage("mandrill"))
RGB Image with:
data: 512x512x3 Array{UfixedBase{UInt8,8},3}
properties:
IMcs: sRGB
colorspace: RGB
colordim: 3
spatialorder: y x
pixelspacing: 1 1
julia> imgs = sliceim(img, 1:5, 1:5, 2)
typeof(img.data) = Array{UfixedBase{UInt8,8},3}
I = (1:5,1:5,2)
typeof(S) = SubArray{UfixedBase{UInt8,8},3,Array{UfixedBase{UInt8,8},3},(UnitRange{Int64},UnitRange{Int64},Int64,Int64...),1}
ERROR: `convert` has no method matching convert(::Type{SubArray{UfixedBase{UInt8,8},3,Array{UfixedBase{UInt8,8},3},(UnitRange{Int64},UnitRange{Int64},Int64,Int64...),1}}, ::SubArray{UfixedBase{UInt8,8},2,Array{UfixedBase{UInt8,8},3},(UnitRange{Int64},UnitRange{Int64},Int64),1})
in call at no file
in sliceim at /home/tim/.julia/v0.4/Images/src/core.jl:370Note the very strange |
|
I know folks are busy and these stagedfunction/method sorting issues are not simple, so I may not get any responses to this post. But before getting started on implementing a candidate fix, I thought I'd sketch out an approach and see what people think. CC @Keno, @JeffBezanson. Here are the ingredients:
Perhaps one approach to fix this is to change the Reasonable? I'm worried that there might be circumstances where it's reasonable to want subtype matching. Also, I don't understand the difference between |
timholy
added
status:priority
|
Jeff, since you're reading issues, care to comment on the strategy I proposed above? Basically the question is: for stagedfunction instantiations, is it reasonable to insist that they match only when the types align exactly? I might be able to implement this, but I'm not sure it's the right strategy. |
|
By align exactly do you mean that the staged function argument types can only be concrete? |
|
Probably a bad word choice. I just mean that the types match exactly (down to the parameters, etc). |
|
(Non-concrete types can also match exactly, of course.) |
|
cc @JeffBezanson, @Keno |
|
Probably related, and also caused by stagedfunction g(a::Int...)
N=length(a)
:(NTuple{$N,Int})
endDirectly but then using this inside a function |
|
One dirty workaround I've found---though I haven't tested it for tuples of length 8 or higher---is that you can declare and that forces specialization for each tuple-type (TT). But this does required allocation of the tuple, so it's not nearly as good as if we just fixed this issue. |
|
In response to the eltype(AbstractArray{Float64}) #-> Any
eltype(AbstractArray{Float64,TypeVar(:N)}) #-> Any
eltype(AbstractArray{Float64,:N}) #-> Float64So |
|
I've been surprised by this before, too, so I'd agree that sounds like a reasonable addition. |
|
Hooray! |
|
+1. Was this the last of the remaining bugs with |
|
There's still the name bug. |
I've found a case where compiling a function that calls a stagedfunction fails, due to a failure in type inference. If one checks out #8501 (and preferably, inserts a
println(P)here), here's a simple way to reproduce:This gives the following output:
One can see that it fails at compile-time, not runtime: that
@show typeof(y)never gets called.A modified version of the function (which declares the type of
y):yields this:
The text was updated successfully, but these errors were encountered: