Running model on cloud

[uturuncoglu]

As a part of the UFS Coastal Project we are aiming to run the ufs-coastal (it is a fork of ufs-wearther-model) on cloud (specifically ParallelWorks) and creating some application level workflow to run some tests over there. I just wonder about the current status to use spack-stack in the could environment and running model over there. If there is a documentation that has detailed instructions that would be great.

[uturuncoglu]

@DomHeinzeller @BrianCurtis-NOAA Any suggestion?

[pvelissariou1]

@ulmononian Thank you for the reply. The issue is that we cannot access spack-stack from your space. At this point, I thought to deploy spack-stack in our PW clusters if you could share a tar archive of your spack-stack installation. I believe that somehow spack-stack should be available on PW for all cluster spaces. By the way, I checked spack-stack on hera, orion and hercules using UFS-Coastal and other modeling applications and everything works as expected.

[climbfuji]

The issue with sharing a tar file is that spack-stack by default optimizes for the CPU you are using. For instance, JCSDA uses hpc6id / c6i instances with a fairly recent Intel processor. EPIC wasn't able to use the spack-stack binaries we generated, because they are using a different/older instance family with a different CPU type that doesn't understand AVX-512 for example. Your safest bet would be to install yourselves using the site configs we maintain. Or to coordinate with EPIC to use the same instance type they do and then obtain their tar files.

[pvelissariou1]

@climbfuji Thank you, I will try to coordinate with EPIC on this.

[ulmononian]

@pvelissariou1 since you don't have access to either of jcsda/epic space, and given what @climbfuji described about tarballs/machine architecture, i think it could be best to install spack-stack from scratch in your space using one of our site configs for the noaa cloud (e.g., aws: https://github.com/JCSDA/spack-stack/tree/develop/configs/sites/noaa-aws) and followhttps://spack-stack.readthedocs.io/en/latest/CreatingEnvironments.html. you may have to edit the compilers.yaml and packages.yaml based on the compilers and external packages on your instance.

[pvelissariou1]

It seems compiling from scratch is a great option. First, I will try to coordinate on this with EPIC/UFS because we are developing UFS-Coastal based on UFS weather model and we want to be close with them in terms of development and testing. If this doesn't work, then I will build spack-stack from scratch per your recommendations.

[gspetro-NOAA]

@uturuncoglu Just wanted to check whether this was resolved offline with EPIC folks (in which case I'll close the discussion) or whether your team needs additional assistance.

[uturuncoglu]

@gspetro-NOAA I have no information about it but maybe @pvelissariou1 has. I personally did not part of discussion with the EPIC team. @pvelissariou1 Do you have any update?

[gspetro-NOAA]

@pvelissariou1 I will close this discussion since you seem to have determined a path forward, but feel free to open another discussion or email [email protected] if you have any further questions.