Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Work around a CUPTI bug in CUDA 12.4 Update 1. #2330

Merged
merged 1 commit into from
Apr 16, 2024
Merged

Work around a CUPTI bug in CUDA 12.4 Update 1. #2330

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions src/compiler/reflection.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,16 @@ function code_sass(io::IO, job::CompilerJob; raw::Bool=false)
return
end

# NVIDIA bug #4604961: CUPTI in CUDA 12.4 Update 1 does not capture profiled events
# unless the activity API is first activated
if runtime_version() == v"12.4"
cfg = CUPTI.ActivityConfig([CUPTI.CUPTI_ACTIVITY_KIND_CONCURRENT_KERNEL,
CUPTI.CUPTI_ACTIVITY_KIND_INTERNAL_LAUNCH_API])
CUPTI.enable!(cfg) do
# do nothing
end
end

cfg = CUPTI.CallbackConfig([CUPTI.CUPTI_CB_DOMAIN_RESOURCE]) do domain, id, data
# only process relevant callbacks
id == CUPTI.CUPTI_CBID_RESOURCE_MODULE_LOADED || return
Expand Down
6 changes: 3 additions & 3 deletions test/core/execution.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ end
CUDA.code_warntype(devnull, dummy, Tuple{})
CUDA.code_llvm(devnull, dummy, Tuple{})
CUDA.code_ptx(devnull, dummy, Tuple{})
if can_use_cupti() && CUDA.runtime_version() != v"12.4"
if can_use_cupti()
# functions defined in Julia
sass = sprint(io->CUDA.code_sass(io, dummy, Tuple{}))
@test occursin(".text._Z5dummy", sass)
Expand All @@ -94,7 +94,7 @@ end
@device_code_warntype io=devnull @cuda dummy()
@device_code_llvm io=devnull @cuda dummy()
@device_code_ptx io=devnull @cuda dummy()
if can_use_cupti() && CUDA.runtime_version() != v"12.4"
if can_use_cupti()
# functions defined in Julia
sass = sprint(io->@device_code_sass io=io @cuda dummy())
@test occursin(".text._Z5dummy", sass)
Expand All @@ -118,7 +118,7 @@ end
@test occursin("dummy", sprint(io->(@device_code_llvm io=io optimize=false @cuda dummy())))
@test occursin("dummy", sprint(io->(@device_code_llvm io=io @cuda dummy())))
@test occursin("dummy", sprint(io->(@device_code_ptx io=io @cuda dummy())))
if can_use_cupti() && CUDA.runtime_version() != v"12.4"
if can_use_cupti()
@test occursin("dummy", sprint(io->(@device_code_sass io=io @cuda dummy())))
end

Expand Down