-
-
Notifications
You must be signed in to change notification settings - Fork 5.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Refactor compiler method lookup interface
The primary motivation here is to clean up the notion of "looking up a method in the method table" into a single object that can be passed around. At the moment, it's all a bit murky, with fairly large state objects being passed around everywhere and implicit accesses to the global environment. In my AD use case, I need to be a bit careful to make sure the various inference and optimization steps are looking things up in the correct tables/caches, so being very explicit about where things need to be looked up is quite helpful. In particular, I would like to clean up the optimizer, to not require the big `OptimizationState` which is currently a bit of a mix of things that go into IRCode and information needed for method lookup/edge tracking. That isn't part of this PR, but will build on top of it. More generally, with a bunch of the recent compiler work, I've been trying to define more crisp boundaries between the various components of the system, giving them clearer interfaces, and at least a little bit of documentation. The compiler is a very powerful bit of technology, but I think people having been avoiding it, because the code looks a bit scary. I'm hoping some of these cleanups will make it easier for people to understand what's going on. Here in particular, I'm using `findall(sig, table)` as the predicate for method lookup. The idea being that people familiar with the `findall(predicate, collection)` idiom from regular julia will have a good intuitive understanding of what's happening (a collection is searched for a predicate), an array of matches is returned, etc. Of course, it's not a perfect fit, but I think these kinds of mental aids can be helpful in making it easier for people to read compiler code (similar to how #35831 used `getindex` as the verb for cache lookup). While I was at it, I also cleaned up the use of out-parameters which leaked through too much of the underlying C API and replaced them by a proper struct of results.
- Loading branch information
Showing
14 changed files
with
213 additions
and
129 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| abstract type MethodTableView; end | ||
|
|
||
| struct MethodLookupResult | ||
| # Really Vector{Core.MethodMatch}, but it's easier to represent this as | ||
| # and work with Vector{Any} on the C side. | ||
| matches::Vector{Any} | ||
| valid_worlds::WorldRange | ||
| ambig::Bool | ||
| end | ||
| length(result::MethodLookupResult) = length(result.matches) | ||
| function iterate(result::MethodLookupResult, args...) | ||
| r = iterate(result.matches, args...) | ||
| r === nothing && return nothing | ||
| match, state = r | ||
| return (match::MethodMatch, state) | ||
| end | ||
| getindex(result::MethodLookupResult, idx::Int) = getindex(result.matches, idx)::MethodMatch | ||
|
|
||
| """ | ||
| struct InternalMethodTable <: MethodTableView | ||
| A struct representing the state of the internal method table at a | ||
| particular world age. | ||
| """ | ||
| struct InternalMethodTable <: MethodTableView | ||
| world::UInt | ||
| end | ||
|
|
||
| """ | ||
| struct CachedMethodTable <: MethodTableView | ||
| Overlays another method table view with an additional local fast path cache that | ||
| can respond to repeated, identical queries faster than the original method table. | ||
| """ | ||
| struct CachedMethodTable{T} <: MethodTableView | ||
| cache::IdDict{Any, Union{Missing, MethodLookupResult}} | ||
| table::T | ||
| end | ||
| CachedMethodTable(table::T) where T = | ||
| CachedMethodTable{T}(IdDict{Any, Union{Missing, MethodLookupResult}}(), | ||
| table) | ||
|
|
||
| """ | ||
| findall(sig::Type{<:Tuple}, view::MethodTableView; limit=typemax(Int)) | ||
| Find all methods in the given method table `view` that are applicable to the | ||
| given signature `sig`. If no applicable methods are found, an empty result is | ||
| returned. If the number of applicable methods exeeded the specified limit, | ||
| `missing` is returned. | ||
| """ | ||
| function findall(@nospecialize(sig::Type{<:Tuple}), table::InternalMethodTable; limit::Int=typemax(Int)) | ||
| _min_val = RefValue{UInt}(typemin(UInt)) | ||
| _max_val = RefValue{UInt}(typemax(UInt)) | ||
| _ambig = RefValue{Int32}(0) | ||
| ms = _methods_by_ftype(sig, limit, table.world, false, _min_val, _max_val, _ambig) | ||
| if ms === false | ||
| return missing | ||
| end | ||
| return MethodLookupResult(ms::Vector{Any}, WorldRange(_min_val[], _max_val[]), _ambig[] != 0) | ||
| end | ||
|
|
||
| function findall(@nospecialize(sig::Type{<:Tuple}), table::CachedMethodTable; limit::Int=typemax(Int)) | ||
| box = Core.Box(sig) | ||
| return get!(table.cache, sig) do | ||
| findall(box.contents, table.table; limit=limit) | ||
| end | ||
| end | ||
|
|
||
| """ | ||
| findsup(sig::Type{<:Tuple}, view::MethodTableView)::Union{Tuple{MethodMatch, WorldRange}, Nothing} | ||
| Find the (unique) method `m` such that `sig <: m.sig`, while being more | ||
| specific than any other method with the same property. In other words, find | ||
| the method which is the least upper bound (supremum) under the specificity/subtype | ||
| relation of the queried `signature`. If `sig` is concrete, this is equivalent to | ||
| asking for the method that will be called given arguments whose types match the | ||
| given signature. This query is also used to implement `invoke`. | ||
| Such a method `m` need not exist. It is possible that no method is an | ||
| upper bound of `sig`, or it is possible that among the upper bounds, there | ||
| is no least element. In both cases `nothing` is returned. | ||
| """ | ||
| function findsup(@nospecialize(sig::Type{<:Tuple}), table::InternalMethodTable) | ||
| min_valid = RefValue{UInt}(typemin(UInt)) | ||
| max_valid = RefValue{UInt}(typemax(UInt)) | ||
| result = ccall(:jl_gf_invoke_lookup_worlds, Any, (Any, UInt, Ptr{Csize_t}, Ptr{Csize_t}), | ||
| sig, table.world, min_valid, max_valid)::Union{Method, Nothing} | ||
| result === nothing && return nothing | ||
| (result, WorldRange(min_valid[], max_valid[])) | ||
| end | ||
|
|
||
| # This query is not cached | ||
| findsup(sig::Type{<:Tuple}, table::CachedMethodTable) = findsup(sig, table.table) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.