-
Notifications
You must be signed in to change notification settings - Fork 666
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CMake Build #1896
CMake Build #1896
Conversation
…ersions of cmake if even possible
I'd like to get some clarification on how the expected workflow under this build system will look for users. Right now, the Perhaps it's a bit late in the PR process to raise this issue, but we have a large amount of documentation that will need to be updated with the current organization of install and run directories. What if we were to try to mirror the workflow implied by the current build system, but just under the Also, is there a strong argument for dropping the |
Could this be related to the newly added physics_mmm/ directory and several physics modules now residing in that directory? |
This is a bug, everything was meant to be installed under test/em_real. In shuffling things around it seems that the binaries and tables from main/ are getting installed one directory higher than they ought to. I'll fix this as soon as I merge develop into this branch - currently running into weird unforeseen issues.
It's standard practice for *nix binaries to not have an extension, more so the
With fixing the install location of tables & binaries we shouldn't need to link back into run/ or test/. The workflow should ideally be going into the out-of-source build (install/) and then from there everything should be the same - though right now a run/ directory does not exist as I was under the impression that almost all runs are started from test/. |
My vote would be to
1. have executables linked to the existing run and test directories where
the data and namelist files reside.
2. at least have the .exe suffix in the linked files in those directories
to reduce user confusion.
I prefer suffixes if only to distinguish files from directories.
Jimy
…On Mon, Feb 12, 2024 at 8:13 PM Anthony Islas ***@***.***> wrote:
I'd like to get some clarification on how the expected workflow under this
build system will look for users. Right now, the real and wrf binaries
are installed in install/bin with copies in install/test. However, in the
install/test directory, although there are binaries and physics tables,
example namelists are in a em_real subdirectory. Do we expect users to
copy namelists from the install/test/em_real directory into the
install/test directory?
This is a bug, everything was meant to be installed under test/em_real. In
shuffling things around it seems that the binaries and tables from main/
are getting installed one directory higher than they ought to. I'll fix
this as soon as I merge develop into this branch - currently running into
weird unforeseen issues.
Also, is there a strong argument for dropping the .exe suffix from
binaries? I don't personally prefer this suffix (as evidenced by the names
of executable files in a certain other model), but I do think there's value
in trying to minimize unnecessary changes at this point in WRF's lifecycle,
especially given the untold number of users who may have scripts for
running complex WRF workflows, and given the large body of tutorial
material and other documentation.
It's standard practice for *nix binaries to not have an extension, more so
the .exe suffix is most generally applied to Windows binaries. If the
workflow can more or less be replicated in the out-of-source build I think
this is a much smaller change that could be tolerated.
perhaps even linked to the current run/ and test/em_real/ (or any of the
test/em_ case directory
With fixing the install location of tables & binaries we shouldn't need to
link back into run/ or test/. The workflow should ideally be going into the
out-of-source build (install/) and then from there everything should be the
same - though right now a run/ directory does not exist as I was under the
impression that almost all runs are started from test/.
—
Reply to this email directly, view it on GitHub
<#1896 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEIZ77F6UUUOG5RL32R2C2TYTLK6PAVCNFSM6AAAAAA2CNSMTGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBQGM2TMMBTGM>
.
You are receiving this because your review was requested.Message ID:
***@***.***>
|
For me the default intel compiler was successful with qcmd, but I still
have problems with configuring when I switch to gcc.
It hangs after this. Not sure what the loadedmodules message means.
--------------------
Default [0] :
[DM] Use MPI? Default [Y] [Y/n] :
[SM] Use OpenMP? Default [N] [y/N] :
Configure additional options? Default [N] [y/N] :
-- Cray Programming Environment 2.7.23 C
-- NOTE: LOADEDMODULES changed since initial config!
-- NOTE: this may cause unexpected build errors.
-- Cray Programming Environment 2.7.23 CXX
-- Cray Programming Environment 2.7.23 Fortran
…On Tue, Feb 13, 2024 at 9:10 AM Jimy Dudhia ***@***.***> wrote:
My vote would be to
1. have executables linked to the existing run and test directories where
the data and namelist files reside.
2. at least have the .exe suffix in the linked files in those directories
to reduce user confusion.
I prefer suffixes if only to distinguish files from directories.
Jimy
On Mon, Feb 12, 2024 at 8:13 PM Anthony Islas ***@***.***>
wrote:
> I'd like to get some clarification on how the expected workflow under
> this build system will look for users. Right now, the real and wrf
> binaries are installed in install/bin with copies in install/test.
> However, in the install/test directory, although there are binaries and
> physics tables, example namelists are in a em_real subdirectory. Do we
> expect users to copy namelists from the install/test/em_real directory
> into the install/test directory?
>
> This is a bug, everything was meant to be installed under test/em_real.
> In shuffling things around it seems that the binaries and tables from main/
> are getting installed one directory higher than they ought to. I'll fix
> this as soon as I merge develop into this branch - currently running into
> weird unforeseen issues.
>
> Also, is there a strong argument for dropping the .exe suffix from
> binaries? I don't personally prefer this suffix (as evidenced by the names
> of executable files in a certain other model), but I do think there's value
> in trying to minimize unnecessary changes at this point in WRF's lifecycle,
> especially given the untold number of users who may have scripts for
> running complex WRF workflows, and given the large body of tutorial
> material and other documentation.
>
> It's standard practice for *nix binaries to not have an extension, more
> so the .exe suffix is most generally applied to Windows binaries. If the
> workflow can more or less be replicated in the out-of-source build I think
> this is a much smaller change that could be tolerated.
>
> perhaps even linked to the current run/ and test/em_real/ (or any of the
> test/em_ case directory
>
> With fixing the install location of tables & binaries we shouldn't need
> to link back into run/ or test/. The workflow should ideally be going into
> the out-of-source build (install/) and then from there everything should be
> the same - though right now a run/ directory does not exist as I was under
> the impression that almost all runs are started from test/.
>
> —
> Reply to this email directly, view it on GitHub
> <#1896 (comment)>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AEIZ77F6UUUOG5RL32R2C2TYTLK6PAVCNFSM6AAAAAA2CNSMTGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBQGM2TMMBTGM>
> .
> You are receiving this because your review was requested.Message ID:
> ***@***.***>
>
|
While I'm not exactly certain what is under the hood causing this the issue arises from swap/switching modules and not clearing the previous build entirely. Solution: Do |
Update: This worked for configure and compile after switching to gcc. |
Even though gcc compiled there were some errors in copies of 4 files as the target directory looks wrong. Error copying file (if different) from "/glade/u/home/dudhia/cmake-test/WRF/run/p3_lookupTable_1.dat-5.3-2momI" to "/glade/u/home/dudhia/cmake-test/WRF/install/test//p3_lookupTable_1.dat-5.3-2momI". |
@islas I would still recommend removing COAMPS in the list of models to compile. Yes, it is in the list of things to compile in the old make system, but nobody sees it, and it never was a real option (but no one bothered to remove it, true). But in cmake, it shows up as an option for users. It can be confusing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have just two more requests, and then I think any remaining cleanup issues can be ironed out before the release:
- Right now it looks like namelists are placed in
install/test/em_real/em_real
, and I think these should go ininstall/test/em_real
. - Would it make sense to symlink the binaries from
install/bin
to theinstall/test/em_real
directory rather than copying them? I would also propose that we have symlinks to the binaries in theinstall/test/em_real
directory with the.exe
suffix. I know this isn't the Unix way, but I really think there's value in not forcing unnecessary updates to documentation, workflows, etc. As a compromise, I wouldn't object to having both names in theinstall/test/em_real
directory (with and without the.exe
suffix).
…xe link automatically
@rcabell Can you help review and approve this PR? |
TYPE: new feature
KEYWORDS: CMake, build, make
SOURCE: internal
DESCRIPTION OF CHANGES:
Problem:
The current WRF build system is fragile with many pitfalls making it difficult for users to build & add to it without perpetuating existing problems. Many options exist across various layers of files, languages, and option control-flow.
Solution:
This requires CMake version 3.20 or newer
A redesign of the build system from the ground up, maintaining the interfacing feel and knowledge accumulated in
arch/configure.defaults
. Condense option selection and control to single locations and as best as possible reduce the complexity of this control.This will be a work in progress as gaps are identified in reproducing the full functionality of the makefile build system. Currently only
em_real
andem_ideal
have limited supportedBrief how to use:
As this is a work in progress, the original
configure
andcompile
scripts have been left as-is. Alongside them are nowconfigure_new
andcompile_new
which walk a user through a slightly similar experience of configuring & compiling WRF.A simple usage example would be:
Notable differences are :
!!
warnings appear for subconfigurations (MPI) that would not be supported in the current environmentcompile_new
does not take target to build as an argument - parallel-j N
jobs still supportedNETCDF
orLD_LIBRARY_PATH
variables.exe
extension, but symlinks are provided./install
. This means the equivalenttest/em_real/wrf.exe
is now atinstall/test/em_real/wrf.exe
LIST OF MODIFIED FILES:
A CMakeLists.txt
A arch/configure_reader.py
A chem/CMakeLists.txt
A cleanCMake.sh
A cmake/c_preproc.cmake
A cmake/confcheck.cmake
A cmake/gitinfo.cmake
A cmake/m4_preproc.cmake
A cmake/modules/FindJasper.cmake
A cmake/modules/FindRPC.cmake
A cmake/modules/FindnetCDF-Fortran.cmake
A cmake/modules/FindnetCDF.cmake
A cmake/modules/FindpnetCDF.cmake
A cmake/printOption.cmake
A cmake/target_copy.cmake
A cmake/template/WRFConfig.cmake.in
A cmake/template/arch_config.cmake
A cmake/template/commit_decl.cmake
A cmake/wrf_case_setup.cmake
A cmake/wrf_get_version.cmake
A compile_new
A confcheck/CMakeLists.txt
A configure_new
A dyn_em/CMakeLists.txt
A external/CMakeLists.txt
A external/RSL_LITE/CMakeLists.txt
A external/atm_ocn/CMakeLists.txt
A external/esmf_time_f90/CMakeLists.txt
A external/fftpack/fftpack5/CMakeLists.txt
A external/io_adios2/CMakeLists.txt
A external/io_esmf/CMakeLists.txt
A external/io_grib1/CMakeLists.txt
A external/io_grib1/MEL_grib1/CMakeLists.txt
A external/io_grib1/WGRIB/CMakeLists.txt
A external/io_grib1/grib1_util/CMakeLists.txt
A external/io_grib2/CMakeLists.txt
A external/io_grib2/bacio-1.3/CMakeLists.txt
A external/io_grib2/g2lib/CMakeLists.txt
A external/io_grib2/g2lib/utest/CMakeLists.txt
A external/io_grib_share/CMakeLists.txt
A external/io_int/CMakeLists.txt
A external/io_netcdf/CMakeLists.txt
A external/io_netcdfpar/CMakeLists.txt
A external/io_phdf5/CMakeLists.txt
A external/io_pio/CMakeLists.txt
A external/io_pnetcdf/CMakeLists.txt
A external/ioapi_share/CMakeLists.txt
A frame/CMakeLists.txt
A inc/CMakeLists.txt
A main/CMakeLists.txt
A phys/CMakeLists.txt
A share/CMakeLists.txt
A test/em_b_wave/CMakeLists.txt
A test/em_convrad/CMakeLists.txt
A test/em_fire/CMakeLists.txt
A test/em_grav2d_x/CMakeLists.txt
A test/em_heldsuarez/CMakeLists.txt
A test/em_hill2d_x/CMakeLists.txt
A test/em_les/CMakeLists.txt
A test/em_quarter_ss/CMakeLists.txt
A test/em_real/CMakeLists.txt
A test/em_scm_xy/CMakeLists.txt
A test/em_seabreeze2d_x/CMakeLists.txt
A test/em_squall2d_x/CMakeLists.txt
A test/em_squall2d_y/CMakeLists.txt
A test/em_tropical_cyclone/CMakeLists.txt
A tools/CMakeLists.txt
A tools/CodeBase/CMakeLists.txt
A doc/README.cmake_build
M tools/fseek_test.c
M README
M arch/configure.defaults
Makefile
TESTS CONDUCTED:
RELEASE NOTE:
Introduction of a CMake build system for em_real and em_ideal