System paths may get into generated module files #12628

weijianwen · 2019-08-28T09:38:02Z

Spack generates modules with /usr/bin for intel compilers and libs, which leads to issues after unloading environment modules. Similar issues are mentioned in #5201 . It is marked as resolved by #5460 . But I can still reproduce this issue on the latest Spack.

Steps to reproduce the issue

$ cd MODULE_FILE_DIR
$ rm -f 19.0.4-gcc-4.8.5
$ spack module tcl refresh [email protected]
==> You are about to regenerate tcl module files for:

-- linux-centos7-x86_64 / [email protected] -----------------------------
zxtvkrs [email protected]

==> Do you want to proceed? [y/n] y
==> Regenerating tcl module files
==> Warning: Quotes in command arguments can confuse scripts like configure.
  The following arguments may cause problems when executed:
      source /lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/bi
n/compilervars.sh intel64 &> /dev/null && python -c "import os, json; print(json.dumps(dict(os.environ)))"
  Quotes aren't needed because spack doesn't use a shell.
  Consider removing them
$ grep '/usr/bin' 19.0.4-gcc-4.8.5
append-path PATH "/usr/bin"
$ module purge; module load intel/19.0.4-gcc-4.8.5; which icc;                                                
/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/bin/intel64/icc
$ module purge; which hostname     
/usr/bin/which: no hostname in (/lustre/spack/tools/linux-centos7-x86_64/gcc-4.8.5/emacs-25.3-pmt2l22orayqtu2q4lajks5cbnxn67iz/bin:/lustre/spack/tools/linux-centos7-x86_64/gcc-4.8.5/vim-8.0.1376-rcaapr4slmd7ewvnp4ebhp2b6q3s7nsv/bin)
$ echo $PATH
/lustre/spack/tools/linux-centos7-x86_64/gcc-4.8.5/emacs-25.3-pmt2l22orayqtu2q4lajks5cbnxn67iz/bin:/lustre/spack/tools/linux-centos7-x86_64/gcc-4.8.5/vim-8.0.1376-rcaapr4slmd7ewvnp4ebhp2b6q3s7nsv/bin

Information on your system

Linux 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

adamjstewart · 2019-08-28T18:37:42Z

I think this bug is also related to what I experienced in #12170. When I build py-numpy ^intel-mkl using an externally installed MKL library, spec['blas'].libs returns /usr/lib as seen in this site.cfg:

[mkl]
libraries = mkl_rt
library_dirs = /opt/intel/compilers_and_libraries_2019.4.233/mac/mkl/lib:/usr/lib
rpath = /opt/intel/compilers_and_libraries_2019.4.233/mac/mkl/lib:/usr/lib
include_dirs = /opt/intel/compilers_and_libraries_2019.4.233/mac/mkl/include

There should be some logic to filter system library locations out of .libs. At least it's at the end instead of the beginning.

jrood-nrel · 2019-08-30T14:46:07Z

I think this is similar to an issue I made too #9639

becker33 · 2019-09-24T15:08:27Z

@weijianwen can you paste in the contents of the offending module?

weijianwen · 2019-09-24T16:00:43Z

@becker33 Sure thing. For example /lustre/share/spack/modules/sandybridge/linux-centos7-x86_64/intel/19.0.4-gcc-4.8.5

#%Module1.0
## Module file created by spack (https://github.com/spack/spack) on 2019-09-09 15:44:33.460081
##
## [email protected]%[email protected]+rpath arch=linux-centos7-x86_64/zxtvkrs
##


module-whatis "Intel Compilers."

proc ModulesHelp { } {
puts stderr "Intel Compilers."
}

conflict intel

prepend-path PATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/bin"
prepend-path MANPATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/man"
prepend-path LD_LIBRARY_PATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/lib"
prepend-path LIBRARY_PATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/lib"
prepend-path CPATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/include"
prepend-path CMAKE_PREFIX_PATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/"
prepend-path CLASSPATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/mpi/intel64/lib/mpi.jar"
prepend-path CPATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/pstl/include:/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/tbb/include:/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/tbb/include"
prepend-path FI_PROVIDER_PATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/mpi/intel64/libfabric/lib/prov"
prepend-path INTEL_LICENSE_FILE "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/licenses:/opt/intel/licenses:/lustre/home/rpm/intel/licenses"
setenv I_MPI_ROOT "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/mpi"
prepend-path LD_LIBRARY_PATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/compiler/lib/intel64_lin:/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/mpi/intel64/libfabric/lib:/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/mpi/intel64/lib/release:/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/mpi/intel64/lib:/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/tbb/lib/intel64/gcc4.7:/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/tbb/lib/intel64/gcc4.7"
prepend-path LIBRARY_PATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/mpi/intel64/libfabric/lib:/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/tbb/lib/intel64/gcc4.7:/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/tbb/lib/intel64/gcc4.7"
prepend-path NLSPATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/compiler/lib/intel64/locale/%l_%t/%N"
setenv PSTLROOT "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/pstl"
setenv TBBROOT "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/tbb"
prepend-path MANPATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/mpi/man"
prepend-path MANPATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/man/common"
append-path PATH "/lustre/spack/bin"
append-path PATH "/lustre/home/rpm/.local/bin"
append-path PATH "/lustre/home/rpm/bin"
append-path PATH "/usr/local/bin"
append-path PATH "/usr/local/sbin"
append-path PATH "/usr/local/bin"
append-path PATH "/usr/bin"
append-path PATH "/lustre/home/rpm/bin"
append-path PATH "/usr/local/sbin"
append-path PATH "/usr/sbin"
append-path PATH "/opt/puppetlabs/bin"
prepend-path PATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/mpi/intel64/bin"
prepend-path PATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/mpi/intel64/libfabric/bin"
prepend-path PATH "/lustre/opt/sandybridge/linux-centos7-x86_64/gcc-4.8.5/intel-19.0.4-zxtvkrsgqvhud4og62e55kwgqvhy26d3/compilers_and_libraries_2019.4.243/linux/bin/intel64"

weijianwen · 2020-01-10T08:18:54Z

I dive into Spack code. It seems that TclModulefileWriter (https://github.com/spack/spack/blob/develop/lib/spack/spack/modules/tcl.py) generates Tcl module files via a Jinjia template named modulefile.tcl (https://github.com/spack/spack/blob/develop/share/spack/templates/modules/modulefile.tcl).

@adamjstewart any suggestion on patching these files so PATHs like /usr/sbin will be removed in the generated module files?

weijianwen · 2020-01-10T08:25:26Z

I think I should check intel.py instead (https://github.com/spack/spack/blob/develop/lib/spack/spack/build_systems/intel.py), in which Tcl module contents are handled.

modkin · 2020-01-20T09:31:37Z

Is anyone still looking into this? I have a similar problem where during the module generation for intel-parallel-studio-cluster.2019.5-gcc-9.2.0 I get the following warning:

==> Warning: Quotes in command arguments can confuse scripts like configure.
  The following arguments may cause problems when executed:
      source /software/spack/opt/spack/linux-ubuntu18.04-skylake/gcc-9.2.0/intel-parallel-studio-cluster.2019.5-74y5yhfm4y3oflke2zrupi6de6asrbqy/parallel_studio_xe_2019.5.075/bin/psxevars.sh intel64 &> /dev/null && PYTHONHOME="/usr" "/usr/bin/python3" -c "import os; import json; print(json.dumps(dict(os.environ)))"
  Quotes aren't needed because spack doesn't use a shell.
  Consider removing them

and loading the module does not work and emits:

Module ERROR: extra characters after close-quote
  In '/software/modules/spack/intel-parallel-studio-cluster.2019.5-gcc-9.2.0-74y5yhf'
  Please contact <root@localhost>

The module file contains the following which I think is the root of the problem:

...
prepend-path BASH_FUNC_switchml%% "() {  typeset swfound=1;
 if [ "" = '1' ]; then
 typeset swname='main';
 if [ -e /usr/lib/x86_64-linux-gnu/modulecmd.tcl ]; then
 typeset swfound=0;
 unset MODULES_USE_COMPAT_VERSION;
 fi;
 else
 typeset swname='compatibility';
 if [ -e /usr/lib/x86_64-linux-gnu/modulecmd-compat ]; then
 typeset swfound=0;
 MODULES_USE_COMPAT_VERSION=1;
 export MODULES_USE_COMPAT_VERSION;
 fi;
 fi;
 if [ wfound -eq 0 ]; then
 echo "Switching to Modules wname version";
 source /usr/share/modules/init/bash;
 else
 echo "Cannot switch to Modules wname version, command not found";
 return 1;
 fi
}"
...

alansill · 2020-05-07T03:36:36Z

I see a similar issue when doing a spack install openfoam:

...
==> openfoam: 1912_200403
==> 58918: Installing openfoam
==> Fetching https://sourceforge.net/projects/openfoam/files/v1912/OpenFOAM-v1912_200403.tgz
######################################################################## 100.0%
==> Staging archive: /tmp/asill/spack-stage/spack-stage-openfoam-1912_200403-76novf4oiu2jvc3tjrnydbro4bj54tg7/OpenFOAM-v1912_200403.tgz
==> Created stage in /tmp/asill/spack-stage/spack-stage-openfoam-1912_200403-76novf4oiu2jvc3tjrnydbro4bj54tg7
==> Added file spack-Allwmake
==> Added file README-spack
==> Ran patch() for openfoam
==> 58918: openfoam: Building openfoam [Package]
==> 58918: openfoam: Executing phase: 'configure'
==> 58918: openfoam: Executing phase: 'build'
==> 58918: openfoam: Executing phase: 'install'
==> Warning: Quotes in command arguments can confuse scripts like configure.
  The following arguments may cause problems when executed:
      source /dev/null &> /dev/null && python3 -c "import os, json; print(json.dumps(dict(os.environ)))"
  Quotes aren't needed because spack doesn't use a shell.
  Consider removing them
==> Warning: Quotes in command arguments can confuse scripts like configure.
  The following arguments may cause problems when executed:
      source /home/asill/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/openfoam-1912_200403-76novf4oiu2jvc3tjrnydbro4bj54tg7/etc/bashrc &> /dev/null && python3 -c "import os, json; print(json.dumps(dict(os.environ)))"
  Quotes aren't needed because spack doesn't use a shell.
  Consider removing them
==> OpenFOAM bashrc env: /home/asill/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/openfoam-1912_200403-76novf4oiu2jvc3tjrnydbro4bj54tg7/etc/bashrc
==> 58918: openfoam: Successfully installed openfoam

h-murai · 2020-09-16T04:49:53Z

I also see a similar problem of openfoam with 0.15.1. What status this issue is in?

alalazo · 2024-01-17T10:55:37Z

We have these lines to avoid this issue when doing automatic inspections:

spack/lib/spack/spack/modules/common.py

Lines 718 to 720 in 2867547

 env = spack.util.environment.inspect_path( 

 spec.prefix, prefix_inspections, exclude=spack.util.environment.is_system_path 

 )

I think anything else that gets injected should be solved on a package by package base in setup_*_run_environment. Closing this issue, but feel free to open a new one if anything still needs fixing.

weijianwen added the bug Something isn't working label Aug 28, 2019

adamjstewart added intel modules labels Aug 28, 2019

scheibelp assigned becker33 Sep 21, 2019

scheibelp added the triage The issue needs to be prioritized label Sep 21, 2019

LDAmorim mentioned this issue Apr 13, 2020

Installation issue: openblas #16028

Open

LDAmorim mentioned this issue Apr 21, 2020

Meta: Scientific Python macOS Stack #16215

Open

alalazo changed the title ~~Spack generates modules with /usr/bin for intel compilers and libs~~ System paths may get into generated module files Jan 17, 2024

alalazo closed this as not planned Won't fix, can't repro, duplicate, stale Jan 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

System paths may get into generated module files #12628

System paths may get into generated module files #12628

weijianwen commented Aug 28, 2019

adamjstewart commented Aug 28, 2019

jrood-nrel commented Aug 30, 2019

becker33 commented Sep 24, 2019

weijianwen commented Sep 24, 2019

weijianwen commented Jan 10, 2020

weijianwen commented Jan 10, 2020

modkin commented Jan 20, 2020

alansill commented May 7, 2020

h-murai commented Sep 16, 2020

alalazo commented Jan 17, 2024

System paths may get into generated module files #12628

System paths may get into generated module files #12628

Comments

weijianwen commented Aug 28, 2019

Steps to reproduce the issue

Information on your system

adamjstewart commented Aug 28, 2019

jrood-nrel commented Aug 30, 2019

becker33 commented Sep 24, 2019

weijianwen commented Sep 24, 2019

weijianwen commented Jan 10, 2020

weijianwen commented Jan 10, 2020

modkin commented Jan 20, 2020

alansill commented May 7, 2020

h-murai commented Sep 16, 2020

alalazo commented Jan 17, 2024