Very high TTFX

[LilithHafner]

I'd like to use Cassette as a backend for a developer-facing tool to log various events during program execution. However, I'm experiencing a 1–2 minute TTFX with Cassette. The tool I'm making should work for arbitrary user code so I don't have access to it at precompile time. Is there any way to use Cassette as a backend without exposing users to this TTFX for every additional code that they wold like to track?

julia> @time @eval using Cassette
  0.069706 seconds (53.16 k allocations: 3.109 MiB, 11.51% compilation time)

julia> @time @eval Cassette.@context Ctx;
  0.152624 seconds (7.33 k allocations: 429.716 KiB, 82.47% compilation time)

julia> @time @eval Cassette.overdub(Ctx(), sort!, [3,1,2]);
134.977680 seconds (55.06 M allocations: 4.079 GiB, 1.52% gc time, 100.00% compilation time)

julia> @time @eval Cassette.overdub(Ctx(), sort!, [3,1,2]);
  0.001106 seconds (64 allocations: 3.266 KiB)

julia> @time @eval Cassette.overdub(Ctx(), sort!, [3.0,1.0,2.0]);
 72.103045 seconds (51.22 M allocations: 4.053 GiB, 2.25% gc time, 99.99% compilation time)

julia> @time @eval Cassette.overdub(Ctx(), sort!, [3.0,1.0,2.0]);
  0.000747 seconds (64 allocations: 3.266 KiB)

For reference, my existing hack is to redefine the methods I want to track (getindex, setindex!, etc.) which causes about 10 seconds worth of invalidations.

This is the FFTX analogue to #91

[vchuravy]

The issue is that Cassette by definition uses a completely split inference cache. It injects the context definition as the first argument into all calls therefore requiring a new cache.

You could precompile some common codes with your context, but fundamentally there is no way around this.