Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SRW-AQM] Update AQM task scripts with those of production/aqm_dev branch #1060

Merged
merged 29 commits into from
Mar 27, 2024
Merged

[SRW-AQM] Update AQM task scripts with those of production/aqm_dev branch #1060

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
75c24a8
change comin model names
chan-hoo Mar 10, 2024
e94b0c3
comin name change
chan-hoo Mar 10, 2024
7020dc8
add prod_util
chan-hoo Mar 11, 2024
6afb7f7
nco env change
chan-hoo Mar 11, 2024
520483a
rename ex scripts for aqm
chan-hoo Mar 11, 2024
cb60e96
fix nexus_gfs_sfc
chan-hoo Mar 11, 2024
ef1844b
add copy of config dirs from components to main dir
chan-hoo Mar 12, 2024
2b6677e
update scripts to meet nco standards
chan-hoo Mar 14, 2024
419fa60
update ex-scripts with production
chan-hoo Mar 14, 2024
a6e5187
remove some nco test cases
chan-hoo Mar 15, 2024
ef507fc
update j-job and pt src script
chan-hoo Mar 15, 2024
50fc2e6
update build script
chan-hoo Mar 15, 2024
92529d4
copy post stat scripts from aqm_dev
chan-hoo Mar 15, 2024
c7deed1
remove nco mode tests
chan-hoo Mar 15, 2024
ef0f7eb
minor fix
chan-hoo Mar 16, 2024
f1de5f7
minor change in aqm_lbcs
chan-hoo Mar 16, 2024
bd43044
minor fix in aqm_lbcs
chan-hoo Mar 18, 2024
85e515b
fix path for output of aqm_lbcs/ics
chan-hoo Mar 18, 2024
4559a17
update fcst scripts
chan-hoo Mar 19, 2024
c40506a
fix aqm_ics script
chan-hoo Mar 19, 2024
2beb73b
Merge branch 'develop' of https://github.com/chan-hoo/ufs-srweather-a…
chan-hoo Mar 19, 2024
1a96560
Merge branch 'develop' of https://github.com/chan-hoo/ufs-srweather-a…
chan-hoo Mar 20, 2024
ab247fe
update nco version
chan-hoo Mar 21, 2024
0f30ef0
remove nco test case
chan-hoo Mar 21, 2024
007d81b
additional change to remove nco test
chan-hoo Mar 21, 2024
9ca3da5
remove unused calls
chan-hoo Mar 21, 2024
ac70562
fix incorrect path
chan-hoo Mar 21, 2024
0bb3fa1
remove nco opsroot from run_WE2E_tests.py
chan-hoo Mar 22, 2024
015902c
remove nco_dir opsroot from jenkins files
chan-hoo Mar 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update ex-scripts with production
  • Loading branch information
@chan-hoo
chan-hoo committed Mar 14, 2024
commit 419fa60fcffe20c528b63fa19a285511966eaef4
64 changes: 22 additions & 42 deletions jobs/JSRW_AQM_ICS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,24 +10,28 @@
#
date
export PS4='+ $SECONDS + '
set -xue
#
#-----------------------------------------------------------------------
#
# Source the variable definitions file and the bash utility functions.
# Set the NCO standard environment variables (Table 1, pp.4)
#
#-----------------------------------------------------------------------
#
. ${HOMEdir}/ush/source_util_funcs.sh
source_config_for_task "task_aqm_ics" ${GLOBAL_VAR_DEFNS_FP}
export USHsrw="${HOMEdir}/ush"
export EXECsrw="${HOMEdir}/exec"
export PARMsrw="${HOMEdir}/parm"
export SCRIPTSsrw="${HOMEdir}/scripts"
#
#-----------------------------------------------------------------------
#
# Save current shell options (in a global array). Then set new options
# for this script/function.
# Source the variable definitions file and the bash utility functions.
#
#-----------------------------------------------------------------------
#
{ save_shell_opts; set -xue; } > /dev/null 2>&1
export USHdir="${USHsrw}" # should be removed later
. ${USHsrw}/source_util_funcs.sh
source_config_for_task "task_aqm_ics" ${GLOBAL_VAR_DEFNS_FP}
#
#-----------------------------------------------------------------------
#
Expand Down Expand Up @@ -59,11 +63,10 @@ which the model needs.
#
#-----------------------------------------------------------------------
#
# Set the NCO standard environment variables (Table 1, pp.4)
# Define job and jobid by default for rocoto
#
#-----------------------------------------------------------------------
#
# Define job and jobid by default for rocoto
WORKFLOW_MANAGER="${WORKFLOW_MANAGER:-rocoto}"
if [ "${WORKFLOW_MANAGER}" = "rocoto" ]; then
if [ "${SCHED}" = "slurm" ]; then
Expand All @@ -78,20 +81,6 @@ if [ "${WORKFLOW_MANAGER}" = "rocoto" ]; then
fi
jobid="${job}.${PDY}${cyc}.${pid}"
fi

# Set environment variables: job card (or rocoto xml)
export envir="${envir:-${envir_default}}"
export model_ver="${model_ver:-${model_ver_default}}"
export KEEPDATA="${KEEPDATA:-${KEEPDATA_default}}"
export SENDCOM="${SENDCOM:-${SENDCOM_default}}"

# Set environment variables: j-job
export NET="${NET:-${NEW_default}}"
export RUN="${RUN:-${RUN_default}}"
export USHsrw="${HOMEdir}/ush"
export EXECsrw="${HOMEdir}/exec"
export PARMsrw="${HOMEdir}/parm"
export SCRIPTSsrw="${HOMEdir}/scripts"
#
#-----------------------------------------------------------------------
#
Expand All @@ -105,18 +94,13 @@ cd $DATA
#
#-----------------------------------------------------------------------
#
# Set other environment variables.
#
#-----------------------------------------------------------------------
#
export CDATE="${CDATE:-${PDY}${cyc}}"
#
#-----------------------------------------------------------------------
#
# Set COM type definitions.
# Define NCO environment variables and set COM type definitions.
#
#-----------------------------------------------------------------------
#
export NET="${NET:-${NET_default}}"
export RUN="${RUN:-${RUN_default}}"

[[ "$WORKFLOW_MANAGER" = "rocoto" ]] && export COMROOT=$COMROOT
if [ "${MACHINE}" = "WCOSS2" ]; then
export COMIN="${COMIN:-$(compath.py -o ${NET}/${model_ver}/${RUN}.${PDY}/${cyc}${SLASH_ENSMEM_SUBDIR})}"
Expand All @@ -128,6 +112,10 @@ fi

mkdir -p ${COMOUT}

# Create a teomporary share directory
export DATA_SHARE="${DATA_SHARE:-${DATAROOT}/DATA_SHARE}"
mkdir -p ${DATA_SHARE}

# Run setpdy to initialize PDYm and PDYp variables
export cycle="${cycle:-t${cyc}z}"
setpdy.sh
Expand All @@ -145,18 +133,18 @@ if [ $RUN_ENVIR = "nco" ]; then
else
export INPUT_DATA="${COMIN}${SLASH_ENSMEM_SUBDIR}/INPUT"
fi
mkdir_vrfy -p "${INPUT_DATA}"
mkdir -p "${INPUT_DATA}"
#
#-----------------------------------------------------------------------
#
# Call the ex-script for this J-job and pass to it the necessary variables.
# Call the ex-script for this J-job.
#
#-----------------------------------------------------------------------
#
export pgmout="${DATA}/OUTPUT.$$"
env

$SCRIPTSsrw/exsrw_aqm_ics.sh
${SCRIPTSsrw}/exsrw_aqm_ics.sh
export err=$?; err_chk

if [ -e "$pgmout" ]; then
Expand All @@ -173,12 +161,4 @@ if [ "${KEEPDATA}" = "NO" ]; then
rm -rf ${DATA}
fi
date
#
#-----------------------------------------------------------------------
#
# Restore the shell options saved at the beginning of this script/function.
#
#-----------------------------------------------------------------------
#
{ restore_shell_opts; } > /dev/null 2>&1

109 changes: 80 additions & 29 deletions jobs/JSRW_AQM_LBCS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,26 +8,30 @@
#
#-----------------------------------------------------------------------
#

date
export PS4='+ $SECONDS + '
set -xue
#
#-----------------------------------------------------------------------
#
# Source the variable definitions file and the bash utility functions.
# Set the NCO standard environment variables (Table 1, pp.4)
#
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "task_get_extrn_lbcs|task_make_orog|task_make_lbcs|cpl_aqm_parm|task_aqm_lbcs" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
export USHsrw="${HOMEdir}/ush"
export EXECsrw="${HOMEdir}/exec"
export PARMsrw="${HOMEdir}/parm"
export SCRIPTSsrw="${HOMEdir}/scripts"
#
#-----------------------------------------------------------------------
#
# Save current shell options (in a global array). Then set new options
# for this script/function.
# Source the variable definitions file and the bash utility functions.
#
#-----------------------------------------------------------------------
#
{ save_shell_opts; . $USHdir/preamble.sh; } > /dev/null 2>&1
export USHdir="${USHsrw}" # should be removed later
. ${USHsrw}/source_util_funcs.sh
source_config_for_task "task_get_extrn_lbcs|task_make_orog|task_make_lbcs|cpl_aqm_parm|task_aqm_lbcs" ${GLOBAL_VAR_DEFNS_FP}
#
#-----------------------------------------------------------------------
#
Expand Down Expand Up @@ -59,55 +63,102 @@ which the model needs.
#
#-----------------------------------------------------------------------
#
# Set the name of and create the directory in which the output from this
# script will be placed (if it doesn't already exist).
# Define job and jobid by default for rocoto
#
#-----------------------------------------------------------------------
#
if [ $RUN_ENVIR = "nco" ]; then
export INPUT_DATA="${COMIN}"
else
export INPUT_DATA="${COMIN}${SLASH_ENSMEM_SUBDIR}/INPUT"
WORKFLOW_MANAGER="${WORKFLOW_MANAGER:-rocoto}"
if [ "${WORKFLOW_MANAGER}" = "rocoto" ]; then
if [ "${SCHED}" = "slurm" ]; then
job=${SLURM_JOB_NAME}
pid=${SLURM_JOB_ID}
elif [ "${SCHED}" = "pbspro" ]; then
job=${PBS_JOBNAME}
pid=${PBS_JOBID}
else
job="task"
pid=$$
fi
jobid="${job}.${PDY}${cyc}.${pid}"
fi
mkdir_vrfy -p "${INPUT_DATA}"
#
#-----------------------------------------------------------------------
#
# Set the run directory
# Create a temp working directory (DATA) and cd into it.
#
#-----------------------------------------------------------------------
#
export DATA="${DATA:-${DATAROOT}/${jobid}}"
mkdir -p $DATA
cd $DATA
#
#-----------------------------------------------------------------------
#
# Define NCO environment variables and set COM type definitions.
#
#-----------------------------------------------------------------------
#
if [ "${RUN_ENVIR}" = "community" ]; then
DATA="${DATA:-${COMIN}${SLASH_ENSMEM_SUBDIR}/tmp_AQM_LBCS}"
check_for_preexist_dir_file "$DATA" "delete"
mkdir_vrfy -p $DATA
cd_vrfy $DATA
export NET="${NET:-${NET_default}}"
export RUN="${RUN:-${RUN_default}}"

[[ "$WORKFLOW_MANAGER" = "rocoto" ]] && export COMROOT=$COMROOT
if [ "${MACHINE}" = "WCOSS2" ]; then
export COMIN="${COMIN:-$(compath.py -o ${NET}/${model_ver}/${RUN}.${PDY}/${cyc}${SLASH_ENSMEM_SUBDIR})}"
export COMOUT="${COMOUT:-$(compath.py -o ${NET}/${model_ver}/${RUN}.${PDY}/${cyc}${SLASH_ENSMEM_SUBDIR})}"
else
export COMIN="${COMIN:-${COMROOT}/${NET}/${model_ver}/${RUN}.${PDY}/${cyc}${SLASH_ENSMEM_SUBDIR}}"
export COMOUT="${COMOUT:-${COMROOT}/${NET}/${model_ver}/${RUN}.${PDY}/${cyc}${SLASH_ENSMEM_SUBDIR}}"
fi

mkdir -p ${COMOUT}

# Create a teomporary share directory
export DATA_SHARE="${DATA_SHARE:-${DATAROOT}/DATA_SHARE}"
mkdir -p ${DATA_SHARE}

# Run setpdy to initialize PDYm and PDYp variables
export cycle="${cycle:-t${cyc}z}"
setpdy.sh
. ./PDY
#
#-----------------------------------------------------------------------
#
# Call the ex-script for this J-job and pass to it the necessary variables.
# Set the name of and create the directory in which the output from this
# script will be placed (if it doesn't already exist).
#
#-----------------------------------------------------------------------
#
$SCRIPTSdir/exregional_aqm_lbcs.sh || \
print_err_msg_exit "\
Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
if [ $RUN_ENVIR = "nco" ]; then
export INPUT_DATA="${COMIN}"
else
export INPUT_DATA="${COMIN}${SLASH_ENSMEM_SUBDIR}/INPUT"
fi
mkdir -p "${INPUT_DATA}"
#
#-----------------------------------------------------------------------
#
# Run job postamble.
# Call the ex-script for this J-job.
#
#-----------------------------------------------------------------------
#
job_postamble
export pgmout="${DATA}/OUTPUT.$$"
env

${SCRIPTSsrw}/exsrw_aqm_lbcs.sh
export err=$?; err_chk

if [ -e "$pgmout" ]; then
cat $pgmout
fi
#
#-----------------------------------------------------------------------
#
# Restore the shell options saved at the beginning of this script/function.
# Whether or not working directory DATA should be kept.
#
#-----------------------------------------------------------------------
#
{ restore_shell_opts; } > /dev/null 2>&1
if [ "${KEEPDATA}" = "NO" ]; then
rm -rf ${DATA}
fi
date

Loading