Host-side CUTENSOR

[kshyatt]

@maleadt @vchuravy this is the PR with problems on cyclops. This should enable us to call CUTENSOR on host memory using memory pinning, and is a port of JuliaGPU/CuArrays.jl#683 (where these tests passed, and I checked that they still do on latest CuArrays).

[maleadt]

Wait, how does this work? This isn't unified memory, so does CUTENSOR just do a cuMemHostGetDevicePointer behind the scenes? Do other APIs work like that? If so, that kinda breaks the design of and protection that CuPtr gives you, because marking all those APIs PtrOrCuPtr now allows passing invalid CPU pointers again...

[kshyatt]

It just needs to be memory pinned by CUDA iirc, not a UnifiedBuffer. A HostBuffer is ok as long as it's pinned in a way that's GPU readable.

[maleadt]

Also, shouldn't we be more strict with the type signatures? e.g. Union{Array, CuArray} instead of all of AbstractArray (this is something I've been doing recently with the GPUArrays stuff, mainly to avoid ambiguities though as CuArray<:AbstractArray). Just a thought.

[kshyatt]

Can do, let's see how this round of tests does.

[maleadt]

CUTENSORError: operation not supported (yet) (code 15, CUTENSOR_STATUS_NOT_SUPPORTED) on 10.1 seems related?

[codecov]

Codecov Report

Merging #243 into master will increase coverage by 0.65%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #243      +/-   ##
==========================================
+ Coverage   80.33%   80.98%   +0.65%     
==========================================
  Files         155      155              
  Lines       10275    10408     +133     
==========================================
+ Hits         8254     8429     +175     
+ Misses       2021     1979      -42     

Impacted Files	Coverage Δ
lib/cutensor/wrappers.jl	93.87% <ø> (+21.01%)	⬆️
test/cudadrv/memory.jl	100.00% <ø> (ø)
lib/cudadrv/memory.jl	84.73% <100.00%> (+0.58%)	⬆️
test/cutensor.jl	99.13% <100.00%> (+0.33%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 027c0dc...fcf0bc5. Read the comment docs.

[maleadt]

Why would these synchronizations matter? Memory copies should be synchronizing already.

[kshyatt]

We discussed this in private, but for the record it's because this is host-side memory.

[kshyatt]

I'm losing my mind at the fact that tests are passing

[kshyatt]

Anyway @maleadt do you mind reviewing this? Now that tests pass can we merge?

[maleadt]

LGTM! Some minor nits.

[maleadt]

🎉

[kshyatt]

finally done! 🚀