Skip to content

Latest commit

 

History

History
215 lines (148 loc) · 6.84 KB

FAQ.md

File metadata and controls

215 lines (148 loc) · 6.84 KB
Raw

FAQ for BBP Spack Usage

Building Software

Q: How do I build and *load* a piece of software?

We'll install Ed(1), the standard editor.

Make sure you're setup as per this.

Specifically that you have the spack git repo on the develop branch, and have created and linked the files into ~/.spack/

  $ spack install ed

This produces output, and should end with something like: $SPACK_INSTALL_PREFIX/linux-rhel7-x86_64/gcc-6.4.0/ed-1.4-35jlkv

One can then run ed with

  $SPACK_INSTALL_PREFIX/linux-rhel7-x86_64/gcc-6.4.0/ed-1.4-35jlkv/ed

More complex packages will have an environment that needs to be setup by the module system. To find the module that was built, issue:

  $ spack module tcl find --full-path ed

At which point, you should be able to:

  $ module load $path_from_above
Q: How do I add/update my package/module versions?

We want to add a new version 2.0.0 to mypackage.

Make sure you're setup as per the instructions.

Change your package recipe to add or update the version specifying the corresponding tag or commit.

  $ spack edit mypackage
  ...
  version('2.0.0', tag='v2.0.0')
  ...

Then you can edit the packages yaml files depending on the type of package (bbp-packages.yaml, external-libraries.yaml…).

Assuming mypackage is an external library:

  $ vim deploy/packages/external-libraries.yaml

Under the spec section

  - [email protected]

After that you should edit the module file that will be at deploy/config/external-libraries/

  $ vim deploy/config/external-libraries/modules.yaml

Under the whitelist section, ensure that your software is mentioned:

  - mypackage

Now you are ready to create a new branch and a PR with the changes. You can check the Jenkins build of your PR on Blue Ocean.

Q: Why do I have to rebuild the entire world?

If you are on the BB5, you shouldn't need to.

As described here, one can use the system packages available with an appropriate ~/.spack/packages.yaml and ~/.spack/upstreams.yaml.

Q: Why do I see PACK_INSTALL_PREFIX in my install? Things are failing!

As described here, the BlueBrain "default" configuration expects that the environment variable $SPACK_INSTALL_PREFIX is defined.

Q: Why are the module files not being rebuilt?

The spack module tcl refresh command respects a blacklists that are in:

  • ~/.spack/modules.yaml

Examples from our deployment workflow can be found in:

  • spack/deploy/configs/applications/modules.yaml
  • spack/deploy/configs/serial-libraries/modules.yaml

Run spack --debug module tcl refresh and search for the module you expect to be built. Modify the whitelist to have the module built.

Q: Why is it so slow to interact with the Spack repository on GPFS

Make sure the spack repo is checked out in a subdirectory of $HOME. The spack repository is quite large, and when it is checked out under a /gpfs/bbp.cscs.ch/project/* directory, performance can be 10x slower than on the SSD provided storage of $HOME.

Q: Is there a binary cache?

We currently have a binary cache for central deployment only. As universally relocatable binaries are very fragile, we do not support binary caches for end-users.

Please make sure you have setup the correct configurations in:

  • ~/.spack/packages.yaml
  • ~/.spack/upstreams.yaml to avoid rebuilding packages that have already been build centrally.
Q: Why are my local modules broken after installing software manually?

When installing CMake-based software with

  $ spack setup package@version
  $ mkdir build
  $ cd build
  $ ../spconfig.py ..
  $ make
  $ make install

Spack will create a skeleton installation with bogus files to directly generate a module for the package to be installed. This may result in a "fake" library to be picked up when installing subsequent packages. Please use

  $ spack dev-build package@version

to install packages locally, and use spack setup only for local development/testing that other packages do not depend on.

Pull Requests

Q: How do I reproduce the Travis checks of a Pull Request?

You can use the native Spack commands:

  $ spack test
  $ module load unstable py-flake8
  $ spack flake8

For the latter, you can also use the official Spack QA script:

  $ ./share/spack/qa/run-flake8-tests

Similarly, the full unit test suite can be run with

  $ ./share/spack/qa/run-unit-tests

but this requires you to install additional software.

Q: How do I test modules generated by a Pull Request?

If you followed the previous point you should be able to see if your PR was succesfully built on Blue Ocean.

Then you can log into BB5 and run the following commands:

  $ module purge
  $ unset MODULEPATH
  $ source /gpfs/bbp.cscs.ch/apps/hpc/jenkins/pulls/xxx/config/modules.sh

Where xxx is the number of your PR.

At this point you should have the environment ready, so if your module was built correctly you should be able to load it.

  $ module load mypackage

Now you are ready to test mypackage.

Q: How do I debug my Pull Request?

To re-create the environment a Pull Request was built in, let's say #666, and debug failures, it is recommended to create a throw-away shell environment and execute the following commands. Note that the parameters in the first line correspond to the pull request and the stage you wish to debug (as labelled in Jenkins, but lowercase):

$ eval $(${SPACK_ROOT}/deploy/pull_env.sh pulls/666 applications)
$ spacktivate
$ spack install $(grep <my_failed_piece_of_software> ${HOME}/specs.txt)

Evaluating the first line will override local environment variables such as the current $HOME directory. After leaving the shell, this will leave a temporary directory behind, following the pattern spack_*. Please make sure to delete this directory when not needed any longer.