Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Relax] Integrate cuDNN attention #17157

Merged
merged 9 commits into from
Jul 22, 2024
Merged

[Relax] Integrate cuDNN attention #17157

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
abbcce9
[Relax] Integrate cuDNN attention
vinx13 Jul 14, 2024
37ef936
update cmake
vinx13 Jul 15, 2024
74ba966
lint
vinx13 Jul 15, 2024
8e2982e
lint
vinx13 Jul 16, 2024
e854fe2
cudnn frontend
vinx13 Jul 16, 2024
5ae9005
lint
vinx13 Jul 16, 2024
e930f9a
lint
vinx13 Jul 16, 2024
79dce1c
fix test
vinx13 Jul 17, 2024
fcddeca
skip test
vinx13 Jul 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Prev Previous commit
skip test
  • Loading branch information
@vinx13
vinx13 committed Jul 21, 2024
commit fcddeca24b8e451a918c49c0c92221f05e99cde7
3 changes: 2 additions & 1 deletion tests/python/relax/test_codegen_cudnn.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ def get_numpy_stacked_attention_ref(b, s, n, h, h_v, bias_shape, qk_scale, dtype

@pytest.fixture(
params=[
# B, S, N, H, bias_shape, scale, single_shape, layout
# B, S, N, H, bias_shape scale, single_shape, layout
(4, 8, 32, (64, 32), "none", 1.0, False, "BS3NH"),
(4, 8, 32, (64, 64), "none", "none", True, "BS3NH"),
(4, 8, 32, (64, 32), "none", 1.0, False, "SBN3H"),
Expand All @@ -282,6 +282,7 @@ def stacked_attention_size(request):
return request.param


@pytest.mark.skip(reason="require cudnn frontend")
def test_stacked_attention_split_offload(stacked_attention_size):
b, s, n, (h, h_v), bias_shape, scale, single_shape, layout = stacked_attention_size
qkv, bias, ref = get_numpy_stacked_attention_ref(
Expand Down