[ROCm] Optionally use hipblaslt

[trixirt]

The hipblaslt package is not available on Fedora.
Instead of requiring the package, make it optional. If it is found, define the preprocessor variable HIPBLASLT Convert the checks for ROCM_VERSION >= 50700 to HIPBLASLT checks

Fixes #119081

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/120551

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• NVIDIA runners have a blacklog of 2000+ jobs

❌ 3 New Failures, 2 Unrelated Failures

As of commit 8edc7b9 with merge base bab4b5a ():

NEW FAILURES - The following jobs have failed:

• linux-binary-manywheel / manywheel-py3_8-cuda11_8-test / test (gh)
  ImportError: libcudnn.so.9: cannot open shared object file: No such file or directory
• linux-binary-manywheel / manywheel-py3_8-cuda12_1-test / test (gh)
  ImportError: libcudnn.so.9: cannot open shared object file: No such file or directory
• trunk / macos-12-py3-arm64 / build (gh)
  /Users/ec2-user/runner/_work/pytorch/pytorch/c10/util/StringUtil.cpp:45:8: error: 'wstring_convert<std::codecvt_utf8_utf16<wchar_t>>' is deprecated [-Werror,-Wdeprecated-declarations]

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• trunk / linux-focal-rocm6.0-py3.8 / test (default, 1, 1, linux.rocm.gpu) (gh) (disabled by #117430 but the issue was closed recently and a rebase is needed to make it pass)
  test_cuda.py::TestCuda::test_lazy_init

UNSTABLE - The following job failed but was likely due to flakiness present on trunk and has been marked as unstable:

• linux-binary-libtorch-pre-cxx11 / libtorch-cpu-shared-with-deps-pre-cxx11-build / build (gh) (#129931)
  Error downloading packages:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[jeffdaily]

@xw285cornell would appreciate your review of this. I'm assuming this PR will break your internal build?

[trixirt]

Update for a couple more hipblaslt usages that were added in main this week.

[FelixSchwarz]

This PR looks pretty straight forward and using a variable instead of "magic" version numbers seem to be much cleaner. It would be nice if this PR wouldn't linger around much longer.

[jeffdaily]

@trixirt I am in favor of this PR. My apologies for adding yet more exposure to hipblaslt APIs that you need to work around again. Please resolve conflicts likely due to #122106.

[jithunnair-amd]

Suggest USE_HIPBLASLT instead of HIPBLASLT as name for define

[IMbackK]

considering that the basis issue of this pr #119081 is now correctly recognized as a bug i think it would be good to not leave this lingering much longer.

[xw285cornell]

Sorry just see this. Thanks @jeffdaily for pinging, this will break our internal codebase but it should be an easy fix. I'm not objecting the idea, if you can ping me before this PR lands, I can put a fix to our internal system easily.

[jithunnair-amd]

@trixirt Please resolve conflicts and I'll request an upstream maintainer to approve.

[jithunnair-amd]

Suggest USE_HIPBLASLT instead of HIPBLASLT as name for define

@trixirt Please do consider this renaming to be aligned with current naming

[trixirt]

I am working on this. Its a bit involved and in a parallel track i trying to get hipblastlt to build on Fedora.

[malfet]

LGTM

[trixirt]

I have submitted the hipBLASLt package for Fedora here
hipBLASLt package review

A prelim refactoring of the above change is here refactored for 2.4

Fedora/My preference is to use the package once it is available.