Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add hyperdrive recipe #280

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

d3v-null
Copy link
Contributor

Here's a cpu-only hyperdrive recipe.
I'd love some help converting this to use HIP.
hyperdrive main (not 0.3.0) has just been fixed, and tests pas on bare metal hip 5.7.3

salloc --nodes=1 --partition=gpu-highmem --account=pawsey0875-gpu -t 08:00:00 --gres=gpu:8 --exclusive
git clone https://github.com/MWATelescope/mwa_hyperdrive.git $MYSOFTWARE/hyperdrive; cd $_
module use /software/setonix/2023.08/modules/zen3/gcc/12.2.0/dependencies/
module load .fontconfig/2.13.94-rykl4uf .libpng/1.6.37-5rosdzt
module use /software/setonix/unsupported
export ROCM_VER=5.7.3
module load rocm/${ROCM_VER}
HYPERBEAM_HIP_ARCH=gfx90a HIP_PATH=/software/setonix/rocm/${ROCM_VER}/ ROCM_PATH=/software/setonix/rocm/${ROCM_VER}/ cargo test --features=hip,cfitsio-static,hdf5-static

hyperbeam also passes the tests and can be hipified.

git clone https://github.com/MWATelescope/mwa_hyperbeam.git $MYSOFTWARE/hyperbeam; cd $_
module use /software/setonix/unsupported
export ROCM_VER=5.7.3
module load rocm/${ROCM_VER}
HYPERBEAM_HIP_ARCH=gfx90a HIP_PATH=/software/setonix/rocm/${ROCM_VER}/ ROCM_PATH=/software/setonix/rocm/${ROCM_VER}/ cargo test --features=hip,hdf5-static

@d3v-null
Copy link
Contributor Author

d3v-null commented Jun 27, 2024

Hey, I added hip to hyperbeam, but I'm encountering an issue.

module load spack/default
spack env activate --temp -p
spack add hyperbeam amdgpu_target=gfx90a +rocm +python
spack install
wget https://github.com/MWATelescope/mwa_hyperbeam/blob/main/examples/analytic_gpu.py
python analytic_gpu.py
Traceback (most recent call last):
  File "/software/projects/mwaeor/dev/hyperbeam/hyperbeam-git/examples/analytic_gpu.py", line 14, in <module>
    import mwa_hyperbeam
  File "/tmp/spack-0we9_87c/.spack-env/._view/fdhkusyqukq5b7oyhq6hdozyfrr35v3i/lib/python3.11/site-packages/mwa_hyperbeam/__init__.py", line 1, in <module>
    from .mwa_hyperbeam import *
ImportError: libnuma-eed8cdad.so.1: ELF load command address/offset not properly aligned

Also, not sure if I would need to add anything to this yaml file to force amdgpu_target=gfx90a https://github.com/PawseySC/pawsey-spack-config/blob/main/systems/setonix/environments/rocm/spack.yaml#L12C5-L12C49

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant