Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to mamba by default. #2137

Merged
merged 3 commits into from
Jun 29, 2023
Merged

Conversation

joshua-cogliati-inl
Copy link
Contributor

@joshua-cogliati-inl joshua-cogliati-inl commented Jun 22, 2023


Pull Request Description

What issue does this change request address?

See #1806

What are the significant changes in functionality due to this change request?

Trying to see what happens if mamba is used by default.


For Change Control Board: Change Request Review

The following review must be completed by an authorized member of the Change Control Board.

  • 1. Review all computer code.
  • 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
  • 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
  • 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
  • 5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options. Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in XML block, the node <internalParallel> to True.
  • 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change. The requirements and the requirements test shall be in sync.
  • 7. The merge request must reference an issue. If the issue is closed, the issue close checklist shall be done.
  • 8. If an analytic test is changed/added is the the analytic documentation updated/added?
  • 9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

@moosebuild
Copy link

Job Mingw Test on 5ef2527 : invalidated by @joshua-cogliati-inl

Failed tests/framework/ROM/MSR/parallel

@moosebuild
Copy link

Job Test qsubs sawtooth on 5ef2527 : invalidated by @joshua-cogliati-inl

Failed tests/cluster_tests/InternalParallel/test_hybrid_model_code

@wangcj05
Copy link
Collaborator

@joshua-cogliati-inl Can you also fix the issue when multiple mambas present in conda environment? As you suggested:
image

@joshua-cogliati-inl
Copy link
Contributor Author

@joshua-cogliati-inl Can you also fix the issue when multiple mambas present in conda environment?

@wangcj05 I pushed that change, can you test it?

Thanks

@moosebuild
Copy link

Job Test qsubs sawtooth on bbea84b : invalidated by @joshua-cogliati-inl

failed in set python environment

@wangcj05
Copy link
Collaborator

@joshua-cogliati-inl It is complains about python, Here is what I got:

~/projects/tests/raven
./scripts/establish_conda_env.sh --install
/Users/wangc/miniconda3/bin/mamba
CONDA
... Run Options:
... ECE Mode: 2
... Verbosity: 0
... Clean: 0
... Install Mode: CONDA
... Conda Defs:
... Installing RAVEN libraries ...
... Detected OS as --os mac ...
... Using Python command python
... $RAVEN_LIBS_NAME set through raven/.ravenrc to raven_libraries_testing
... >> If this is not desired, then remove it from the ravenrc file before running.
... >> RAVEN environment is named "raven_libraries_testing"
... found conda path in ravenrc: /Users/wangc/miniconda3/etc/profile.d/conda.sh
... >> If this is not the desirable path, rerun with argument --conda-defs [path] or remove the entry from raven/.ravenrc file.
... Found conda definitions at /Users/wangc/miniconda3/etc/profile.d/conda.sh
conda 23.3.1
raven_libraries_testing     /Users/wangc/miniconda3/envs/raven_libraries_testing
... Found library environment ...
... Removing old environment ...

Remove all packages in environment /Users/wangc/miniconda3/envs/raven_libraries_testing:


## Package Plan ##

  environment location: /Users/wangc/miniconda3/envs/raven_libraries_testing


The following packages will be REMOVED:

  boltons-23.0.0-pyhd8ed1ab_0
  brotli-1.0.9-hb7f2c08_8
  brotli-bin-1.0.9-hb7f2c08_8
  bzip2-1.0.8-h0d85af4_4
  c-ares-1.19.1-h0dc2134_0
  ca-certificates-2023.5.7-h8857fd0_0
  cached-property-1.5.2-hd8ed1ab_1
  cached_property-1.5.2-pyha770c72_1
  certifi-2023.5.7-pyhd8ed1ab_0
  cffi-1.15.1-py311ha86e640_3
  charset-normalizer-3.1.0-pyhd8ed1ab_0
  colorama-0.4.6-pyhd8ed1ab_0
  conda-23.3.1-py311h6eed73b_0
  conda-package-handling-2.0.2-pyh38be061_0
  conda-package-streaming-0.8.0-pyhd8ed1ab_0
  cryptography-40.0.2-py311h61927ef_0
  fmt-9.1.0-hb8565cd_0
  h5py-3.8.0-nompi_py311hcf977c2_101
  hdf5-1.14.0-nompi_hbf0aa07_103
  icu-72.1-h7336db1_0
  idna-3.4-pyhd8ed1ab_0
  jsonpatch-1.32-pyhd8ed1ab_0
  jsonpointer-2.0-py_0
  krb5-1.20.1-h049b76e_0
  libaec-1.0.6-hf0c8a7f_1
  libarchive-3.6.2-h0b5dc4a_1
  libblas-3.9.0-16_osx64_openblas
  libbrotlicommon-1.0.9-hb7f2c08_8
  libbrotlidec-1.0.9-hb7f2c08_8
  libbrotlienc-1.0.9-hb7f2c08_8
  libcblas-3.9.0-16_osx64_openblas
  libcurl-8.1.1-hbee3ae8_0
  libcxx-16.0.4-hd57cbcb_0
  libedit-3.1.20191231-h0678c8f_2
  libev-4.33-haf1e3a3_1
  libexpat-2.5.0-hf0c8a7f_1
  libffi-3.4.2-h0d85af4_5
  libgfortran-5.0.0-11_3_0_h97931a8_31
  libgfortran5-12.2.0-he409387_31
  libiconv-1.17-hac89ed1_0
  liblapack-3.9.0-16_osx64_openblas
  libmamba-1.4.2-h9d281b0_0
  libmambapy-1.4.2-py311hcc19a12_0
  libnghttp2-1.52.0-he2ab024_0
  libopenblas-0.3.21-openmp_h429af6e_3
  libsolv-0.7.23-hbc0c0cd_0
  libsqlite-3.42.0-h58db7d2_0
  libssh2-1.10.0-h47af595_3
  libxml2-2.11.4-hd95e348_0
  libzlib-1.2.13-hfd90126_4
  llvm-openmp-16.0.4-hff08bdf_0
  lz4-c-1.9.4-hf0c8a7f_0
  lzo-2.10-haf1e3a3_1000
  mamba-1.4.2-py311h8082e30_0
  ncurses-6.3-h96cf925_1
  numpy-1.24.3-py311hc44ba51_0
  openssl-3.1.0-h8a1eda9_3
  packaging-23.1-pyhd8ed1ab_0
  pip-23.1.2-pyhd8ed1ab_0
  pluggy-1.0.0-pyhd8ed1ab_5
  pybind11-abi-4-hd8ed1ab_3
  pycosat-0.6.4-py311h5547dcb_1
  pycparser-2.21-pyhd8ed1ab_0
  pyopenssl-23.1.1-pyhd8ed1ab_0
  pysocks-1.7.1-pyha2e5f31_6
  python-3.11.3-h99528f9_0_cpython
  python_abi-3.11-3_cp311
  readline-8.2-h9e318b2_1
  reproc-14.2.4-hb7f2c08_0
  reproc-cpp-14.2.4-hf0c8a7f_0
  requests-2.31.0-pyhd8ed1ab_0
  ruamel.yaml-0.17.28-py311h2725bcf_0
  ruamel.yaml.clib-0.2.7-py311h5547dcb_1
  setuptools-67.7.2-pyhd8ed1ab_0
  tk-8.6.12-h5dbffcc_0
  toolz-0.12.0-pyhd8ed1ab_0
  tqdm-4.65.0-pyhd8ed1ab_1
  tzdata-2023c-h71feb2d_0
  urllib3-2.0.2-pyhd8ed1ab_0
  wheel-0.40.0-pyhd8ed1ab_0
  xz-5.2.6-h775f41a_0
  yaml-cpp-0.7.0-hf0c8a7f_2
  zstandard-0.19.0-py311hebd4beb_1
  zstd-1.5.2-hbc0c0cd_6


Preparing transaction: done
Verifying transaction: done
Executing transaction: done
... Installing libraries ...
... Installing libraries from conda-forge ...
/Users/wangc/miniconda3/bin/python
conda create -n raven_libraries_testing -y -c conda-forge mamba
Collecting package metadata (current_repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 23.3.1
  latest version: 23.5.0

Please update conda by running

    $ conda update -n base -c conda-forge conda

Or to minimize the number of packages updated during conda update use

     conda install conda=23.5.0



## Package Plan ##

  environment location: /Users/wangc/miniconda3/envs/raven_libraries_testing



Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate raven_libraries_testing
#
# To deactivate an active environment, use
#
#     $ conda deactivate

... Activating environment ...
./scripts/establish_conda_env.sh: line 180: python: command not found
... conda-forge command:
... Activating environment ...
... Installing libraries from PIP-ONLY ...
./scripts/establish_conda_env.sh: line 193: python: command not found
...pip-only command:
... Installing libraries from pyomo ...
./scripts/establish_conda_env.sh: line 199: python: command not found
... pyomo command:
... writing settings to raven/.ravenrc ...
... Setting install variables ...
... python /Users/wangc/projects/tests/raven/scripts/update_install_data.py --write --conda-defs /Users/wangc/miniconda3/etc/profile.d/conda.sh --RAVEN_LIBS_NAME raven_libraries_testing --python-command python --installation-manager CONDA
./scripts/establish_conda_env.sh: line 305: python: command not found
~/projects/tests/raven

…nment

Basically, the mamba environment might not have a working python, so
figure out dependencies
install mamba
install rest of dependencies.
@moosebuild
Copy link

Job Mingw Test on 0322a6c : invalidated by @joshua-cogliati-inl

failed in fetch

Copy link
Collaborator

@wangcj05 wangcj05 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is good.
@joshua-cogliati-inl Can we make some test machine(s) still use conda to install libraries? I think we should have a way to test it.

@joshua-cogliati-inl
Copy link
Contributor Author

This PR is good. @joshua-cogliati-inl Can we make some test machine(s) still use conda to install libraries? I think we should have a way to test it.

I will try and get that setup tomorrow.

@wangcj05
Copy link
Collaborator

PR changes are good, checklist is satisfied.

@wangcj05 wangcj05 merged commit 394a9ed into idaholab:devel Jun 29, 2023
@joshua-cogliati-inl joshua-cogliati-inl deleted the mamba_default branch June 29, 2023 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants