Merge branch 'main' into 534_ao_docs

PCMDI · Dec 1, 2023 · c8be55c · c8be55c
2 parents 5c60d89 + b8f818d
commit c8be55c
Show file tree

Hide file tree

Showing 31 changed files with 1,774 additions and 908 deletions.
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -28,6 +28,7 @@ repos:
  rev: 5.12.0
  hooks:
  - id: isort
+ args: ["--honor-noqa"]
 
  # =======================
  # Python linting

diff --git a/README.md b/README.md
@@ -98,6 +98,7 @@ Release Notes and History
 
 | <div style="width:300%">[Versions]</div> | Update summary |
 | ------------- | ------------------------------------- |
+| [v3.1.2] | Technical update
 | [v3.1.1] | Technical and documentation update
 | [v3.1] | New metric added: **Precipitation Benchmarking -- distribution bimodality**
 | [v3.0.2] | Minor patch and more documentation added
@@ -123,6 +124,7 @@ Release Notes and History
 
 
 [Versions]: https://github.com/PCMDI/pcmdi_metrics/releases
+[v3.1.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.1.2
 [v3.1.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.1.1
 [v3.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.1
 [v3.0.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.0.2

diff --git a/conda-env/dev.yml b/conda-env/dev.yml
@@ -11,7 +11,7 @@ dependencies:
  - python=3.10.10
  - pip=23.1.2
  - numpy=1.23.5
- - cartopy=0.21.1
+ - cartopy=0.22.0
  - matplotlib=3.7.1
  - cdat_info=8.2.1
  - cdms2=3.1.5
@@ -21,7 +21,7 @@ dependencies:
  - eofs=1.4.0
  - seaborn=0.12.2
  - enso_metrics=1.1.1
- - xcdat=0.5.0
+ - xcdat>=0.6.1
  - xmltodict=0.13.0
  - setuptools=67.7.2
  - netcdf4=1.6.3

diff --git a/doc/jupyter/Demo/Demo_1b_mean_climate.ipynb b/doc/jupyter/Demo/Demo_1b_mean_climate.ipynb
diff --git a/doc/jupyter/Demo/Demo_4_modes_of_variability.ipynb b/doc/jupyter/Demo/Demo_4_modes_of_variability.ipynb
diff --git a/doc/obs_info_dictionary.json b/doc/obs_info_dictionary.json
@@ -100,12 +100,12 @@
  },
  "GPCP-2-3": {
  "CMIP_CMOR_TABLE": "Amon",
- "MD5sum": "3792901034585d3d495722f10a0dfecb",
+ "MD5sum": "036e14d0124f4d00921d58e3c1c3e9e1",
  "RefTrackingDate": "Wed Aug 4 12:22:10 2021",
- "filename": "pr_mon_GPCP-2-3_PCMDI_gn.200301-201812.AC.v20210804.nc",
+ "filename": "pr_monC_GPCP-2-3_PCMDI_gn_197901-201907.AC.v20230516.nc",
  "period": "200301-201812",
  "shape": "(12, 72, 144)",
- "template": "pr/GPCP-2-3/v20210804/pr_mon_GPCP-2-3_PCMDI_gn.200301-201812.AC.v20210804.nc"
+ "template": "NOAA-NCEI/GPCP-2-3/monC/pr/gn/v20230516/pr_monC_GPCP-2-3_PCMDI_gn_197901-201907.AC.v20230516.nc"
  },
  "TRMM-3B43v-7": {
  "CMIP_CMOR_TABLE": "Amon",

diff --git a/docs/metrics.rst b/docs/metrics.rst
@@ -18,5 +18,5 @@ A suite of demo scripts and interactive Jupyter notebooks are provided with `thi
  metrics_mjo
  metrics_monsoon
  metrics_ext
-
-
+ metrics_precip-variability
+ metrics_precip-distribution
diff --git a/docs/metrics_mean-clim.rst b/docs/metrics_mean-clim.rst
@@ -80,7 +80,7 @@ where the list of variables (vars) to run the analysis on includes 'rlut' (outgo
 * **test_data_path**: the path/template where the test_data resides, e.g.: 
 * **reference_data_set**: a python list that specifies 'default', 'alternate1', 'alternate2' or 'all', e.g., ['default']
 * **reference_data_path**: the root path to the observational climatology database, e.g., '~/demo_data/PCMDIobs2_clims/'
-* **target_grid**: '2.5x2.5' or an actual cdms2 grid object
+* **target_grid**: a string giving the desired horizontal resolution in degrees following the form 'LATxLON', e.g. '2.5x2.5'
 * **regrid_tool**: options include 'esmf' and 'regrid2' 
 * **metric_output_path**: the full path to the metrics output in JSON files, e.g., '~/demo_data/PMP_metrics/' 
 

diff --git a/docs/metrics_precip-distribution.rst b/docs/metrics_precip-distribution.rst
@@ -0,0 +1,73 @@
+.. _metrics_precip-distribution:
+
+**************************
+Precipitation Distribution
+**************************
+
+Overview
+========
+With the global domain partitioned into 62 regions, including 46 land and 16 ocean regions, we apply 10 established precipitation distribution metrics. The collection includes metrics focused on the maximum peak, lower 10th percentile, and upper 90th percentile in precipitation amount and frequency distributions; the similarity between observed and modeled frequency distributions; an unevenness measure based on cumulative amount; average total intensity on all days with precipitation; and number of precipitating days each year. 
+
+Demo
+====
+In preparation
+
+Exampe parameter files
+======================
+A set of example parameter files for models and observations can be viewed at `this link`_.
+
+Required data sets 
+==================
+
+This driver expects daily averaged precipitation.
+
+Input files must use the following name convention: ::
+
+ variable_frequency_model_experiment_ensemble_startdate-enddate.nc 
+
+Because underscores are used to separate these elements, they may not be used anywhere else in the file name.
+
+Start and end dates must use the YYYYMMDD format. 
+
+For example, these are valid input file names: ::
+
+ pr_day_bcc-csm1-1_historical_r1i1p1_19800101-19841231.nc 
+ pr_3hr_IMERG-v06B-Final_PCMDI_2x2_20100401-20100430.nc 
+
+If the time series for a single data set is spread across multiple files, those files must be located in a single directory.
+
+Usage
+=====
+Users will set up a parameter file and run the precipitation variability driver on the command line.
+To run the driver, use: ::
+
+ precip_distribution_driver.py -p parameter_file 
+
+This code should be run for a reference observation initially as some metrics (e.g., Perkins score) need a reference.
+
+After completing calculation for a reference observation, this code can work for multiple datasets at once.
+
+This benchmarking framework provides three tiers of area averaged outputs for i) large scale domain (Tropics and Extratropics with separated land and ocean) commonly used in the PMP , ii) large scale domain with clustered precipitation characteristics (Tropics and Extratropics with separated land and ocean, and separated heavy, moderate, and light precipitation regions), and iii) modified IPCC AR6 regions shown in the reference paper.
+
+Options available to set in the parameter file include:
+
+* **mip**: Name of MIP.
+* **var**: Name of data set variable, e.g. "pr". 
+* **frq**: Frequency of data set, either "day" or "3hr". 
+* **modpath**: Path to directory containing input data files. 
+* **mod**: Name of model file or wildcard "*" to use all files in directory. Symlinks may be used. 
+* **results_dir**: Results directory path.
+* **case_id**: Case id.
+* **prd**: Start and end years for analysis as list, e.g. [start_year, end_year].
+* **fac**: Factor to convert from data set units to mm/day. Set to 1 for no conversion.
+* **res**: List of target horizontal resolutions in degrees for interporation.
+* **ref**: Reference data path.
+* **ref_dir**: Reference directory path.
+* **cmec**: Set to True to output CMEC formatted JSON.
+
+
+.. _this link: https://github.com/PCMDI/pcmdi_metrics/tree/main/pcmdi_metrics/precip_distribution/param
+
+Reference
+=========
+Ahn, M.-S., P. A. Ullrich, P. J. Gleckler, J. Lee, A. C. Ordonez, and A. G. Pendergrass, 2023: Evaluating Precipitation Distributions at Regional Scales: A Benchmarking Framework and Application to CMIP5 and CMIP6. Geoscientific Model Development, 16, 3927–3951, https://doi.org/10.5194/gmd-16-3927-2023
diff --git a/docs/metrics_precip-variability.rst b/docs/metrics_precip-variability.rst
@@ -0,0 +1,92 @@
+.. _metrics_precip-variability:
+
+*******************************************
+Precipitation Variability Across Timescales
+*******************************************
+
+Overview
+========
+This set of metrics is designed to measure precipitation variabilty across multiple timescales, including subdaily.
+
+Demo
+====
+* `PMP demo Jupyter notebook`_
+
+Exampe parameter files
+======================
+A set of example parameter files for models and observations can be viewed at `this link`_.
+
+Required data sets 
+==================
+
+Input files must use the following name convention: ::
+
+ variable_frequency_model_experiment_ensemble_startdate-enddate.nc 
+
+Because underscores are used to separate these elements, they may not be used anywhere else in the file name.
+
+Start and end dates must use the YYYYMMDD or YYYYMMDDHHHH format. 
+
+For example, these are valid input file names: ::
+
+ pr_day_bcc-csm1-1_historical_r1i1p1_19800101-19841231.nc 
+ pr_3hr_IMERG-v06B-Final_PCMDI_2x2_201004010000-201004302100.nc 
+
+If the time series for a single data set is spread across multiple files, those files must be located in a single directory.
+
+Usage
+=====
+
+Spectral averages
+*****************
+
+Users will set up a parameter file and run the precipitation variability driver on the command line.
+To run the driver, use: ::
+
+ variability_across_timescales_PS_driver.py -p parameter_file 
+
+Results are reported on a 2x2 degree latitude/longitude world grid.
+
+Options available to set in the parameter file include:
+
+* **mip**: Name of MIP. Use "obs" for reference datasets.
+* **exp**: Name of experiment. 
+* **var**: Name of data set variable, e.g. "pr". 
+* **frq**: Frequency of data set, either "day" or "3hr". 
+* **modpath**: Path to directory containing input data files. 
+* **mod**: Name of model file or wildcard "*" to use all files in directory. Symlinks may be used. 
+* **results_dir**: Results directory path.
+* **case_id**: Case id.
+* **prd**: Start and end years for analysis as list, e.g. [start_year, end_year].
+* **fac**: Factor to convert from data set units to mm/day. Set to 1 for no conversion.
+* **nperseg**: Length of segment in power spectra.
+* **noverlap**: Length of overlap between segments in power spectra.
+* **ref**: Reference data path.
+* **cmec**: Set to True to output CMEC formatted JSON.
+
+Metric 
+******
+
+The precipitation variability metric can be generated after model and observational spectral averages are made.
+
+A script called `calc_ratio.py`_ is provided in the precip_variability codebase. This script can be called with three arguments to generate the ratio.
+
+* **ref**: path to obs results JSON
+* **modpath**: directory containing model results JSONS (not CMEC formatted JSONs)
+* **results_dir**: directory for calc_ratio.py results
+
+The calc_ratio.py script must be called with python directly. For example, to run this script using files from a directory called "results": ::
+
+ python pcmdi_metrics/pcmdi_metrics/precip_variability/scripts_pcmdi/calc_ratio.py \
+ --ref results/precip_variability/GPCP-1-3/PS_pr.day_regrid.180x90_area.freq.mean_GPCP-1-3.json \
+ --modpath results/precip_variability/GISS-E2-H/ \
+ --results_dir results/precip_variability/ratio/
+
+Reference
+==========
+Ahn, M.-S., P. J. Gleckler, J. Lee, A. G. Pendergrass, and C. Jakob, 2022: Benchmarking Simulated Precipitation Variability Amplitude across Timescales. Journal of Climate. https://doi.org/10.1175/JCLI-D-21-0542.1
+
+
+.. _PMP demo Jupyter notebook: https://github.com/PCMDI/pcmdi_metrics/blob/main/doc/jupyter/Demo/Demo_7_precip_variability.ipynb
+.. _this link: https://github.com/PCMDI/pcmdi_metrics/tree/main/pcmdi_metrics/precip_variability/param
+.. _calc_ratio.py: https://github.com/PCMDI/pcmdi_metrics/blob/main/pcmdi_metrics/precip_variability/scripts_pcmdi/calc_ratio.py