Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modes of variability (branch: 377 variability mode nao) #623

Merged
merged 154 commits into from
Oct 23, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
154 commits
Select commit Hold shift + click to select a range
899f541
First upload
lee1043 Jul 12, 2016
855fb14
Sub-directories
lee1043 Jul 12, 2016
e88837a
Delete get_pcmdi_data.py
lee1043 Jul 12, 2016
6c7f850
Delete durolib.py
lee1043 Jul 12, 2016
a523872
Delete test.py
lee1043 Jul 12, 2016
70bc531
minor change
lee1043 Jul 12, 2016
7e0f158
Merge branch '377_Variability_mode_NAO' of github.com:PCMDI/pcmdi_met…
lee1043 Jul 12, 2016
283fcd1
Advance plot, region option, etc.
lee1043 Jul 13, 2016
276221b
Time axis in NetCDF output corrected
lee1043 Jul 13, 2016
b37a149
Variable ID in NetCDF output corrected
lee1043 Jul 13, 2016
f96bafe
rename..
lee1043 Jul 13, 2016
f5ee4f8
NetCDF output correction and etc.
lee1043 Jul 14, 2016
5cebc85
Split code: separate eof and plotting parts
lee1043 Jul 14, 2016
5caf7c4
Seperate netcdf write
lee1043 Jul 14, 2016
6912cd1
Add observation statistics
lee1043 Jul 14, 2016
873aa3d
Add bar chart for rms and cor, code clean up
lee1043 Jul 15, 2016
1acc81f
Adjust dictionary hierarchy for json writing
lee1043 Jul 15, 2016
da0a0de
Add SAM mode and obs option
lee1043 Jul 19, 2016
55209bf
minor..
lee1043 Jul 19, 2016
4f34df0
Add var. modes
lee1043 Jul 20, 2016
b15ee7b
rename
lee1043 Jul 20, 2016
e9223bb
minor..
lee1043 Jul 20, 2016
bba50ec
minor..
lee1043 Jul 20, 2016
cd3e8d8
Change weight from coslat to area
lee1043 Jul 22, 2016
453d334
update..
lee1043 Jul 25, 2016
5ad77e2
working on pdo..
lee1043 Jul 25, 2016
ca9ff96
landmask array dimension control
lee1043 Jul 25, 2016
1e76d41
working on PDO..
lee1043 Jul 26, 2016
e326a3e
working on PDO...
lee1043 Jul 26, 2016
10b7695
working PDO ...
lee1043 Jul 26, 2016
f33d148
working on PDO... get lf function
lee1043 Jul 27, 2016
0140fdc
working on PDO.. land fraction criteria check and let plot use lower …
lee1043 Jul 27, 2016
5b8ed1a
minor .. code clean-up
lee1043 Jul 27, 2016
f068804
Linear regression for teleconnection
lee1043 Aug 4, 2016
7f7ac00
NetCDF output correction for eof1_lr and some bugs fixed
lee1043 Aug 4, 2016
85b71ab
code clean up
lee1043 Aug 4, 2016
04cbcb4
code clean-up
lee1043 Aug 4, 2016
16e48f1
Linear regression added for PDO
lee1043 Aug 5, 2016
2e5cf85
PDO map projection changed to robinson
lee1043 Aug 5, 2016
72bc714
minor code clear
lee1043 Aug 5, 2016
d0f9f06
minor code clean up
lee1043 Aug 5, 2016
b50ccbd
Longitude range changed for PDO_teleconnection
lee1043 Aug 5, 2016
ac02060
scipy importing part deleted from linear regression
lee1043 Aug 8, 2016
0f1b561
Typo correction
lee1043 Aug 9, 2016
15b76c8
Merge branch '377_Variability_mode_NAO' of https://github.com/PCMDI/p…
lee1043 Aug 9, 2016
74f6bd8
One unified driver script for all modes; pesudo_pcs analysis added; r…
lee1043 Aug 12, 2016
81607f3
code clean-up
lee1043 Aug 12, 2016
1691416
bug fix
lee1043 Aug 12, 2016
1ff72c6
Land mask control part for PDO is now seperated
lee1043 Aug 12, 2016
6fa2ee6
bug fix
lee1043 Aug 12, 2016
1073121
EOF sign control bug fix. Still needs to be tested..
lee1043 Aug 16, 2016
11893b6
RegridTool change: regrid2 to esmf
lee1043 Aug 16, 2016
568c0f2
Save rms and cor for both EOF subdomain and global, as well as for ps…
lee1043 Aug 17, 2016
56e587c
scipy is not used
lee1043 Aug 17, 2016
f332eaf
dimension order clarify
lee1043 Aug 17, 2016
61296e9
Add PNA specified EOF sign control
lee1043 Aug 17, 2016
e59c490
Check EOF sign based on geographical region average
lee1043 Aug 17, 2016
39ae1b2
PNA sign control advance although still not perfect..
lee1043 Aug 19, 2016
79767b4
Merge branch '377_Variability_mode_NAO' of https://github.com/PCMDI/p…
lee1043 Aug 19, 2016
8feed24
regrid2 failure for NAM and SAM fixed
lee1043 Aug 19, 2016
931433d
Archive plotting scripts
lee1043 Aug 23, 2016
33cd4c0
Add multiple realization (consider all available ensemble members of …
lee1043 Aug 23, 2016
178f258
clean-up
lee1043 Aug 23, 2016
fbec789
Clean up, minor
lee1043 Aug 23, 2016
6cb271a
Add observation time range check and code clean up
lee1043 Aug 23, 2016
8b94361
Add output directory
lee1043 Aug 23, 2016
0086375
Delete bar_chart_rms.py
lee1043 Aug 23, 2016
923d1f5
Delete barChart.py
lee1043 Aug 23, 2016
c5e0715
Add multiple realization
lee1043 Aug 23, 2016
b84ea16
Merge branch '377_Variability_mode_NAO' of https://github.com/PCMDI/p…
lee1043 Aug 23, 2016
d67063d
Remove showing fraction for pseudo maps
lee1043 Aug 24, 2016
e6bc8ff
Add try/except, adjust output file name (include historical), clean-up
lee1043 Aug 24, 2016
b0f41fb
minor
lee1043 Aug 24, 2016
d09a9ba
Clean up
lee1043 Aug 24, 2016
4821362
Add sign control double check
lee1043 Aug 30, 2016
8e73a0a
increase title font size
lee1043 Aug 30, 2016
e5290c7
Add centered RMS and bias, and modulize statistics calculations
lee1043 Aug 31, 2016
8eac224
Modulize statistics calculations
lee1043 Aug 31, 2016
94a57f5
Argparse applied for user defining options
lee1043 Sep 2, 2016
6450759
Recover rmsc and modulizations that backworded by mistake
lee1043 Sep 2, 2016
2705e32
Change default season to DJF
lee1043 Sep 4, 2016
c616824
clean up
lee1043 Sep 8, 2016
be5ba95
Add regions file in /share as like master
lee1043 Sep 13, 2016
f45b56d
Merge branch '377_Variability_mode_NAO' of https://github.com/PCMDI/p…
lee1043 Sep 13, 2016
6b9be2c
Apply region selector given from /share
lee1043 Sep 13, 2016
a2d25e9
clean up
lee1043 Sep 13, 2016
9ccacfd
Prepare to merge master to my branch
lee1043 Sep 19, 2016
fb9f355
re-sync default_region with master
lee1043 Sep 19, 2016
3f4a235
Add flexibility for EOF mode selection
lee1043 Sep 22, 2016
a27621d
keep using default_region in /share
lee1043 Sep 22, 2016
f92ecae
code cleaned regarding region selector
lee1043 Sep 22, 2016
09afc77
Add obs frac to be saved in output json
lee1043 Sep 28, 2016
7d8029d
Add model year availale checking
lee1043 Sep 28, 2016
87c27f5
Adjust line for model statistics which is regardness to observation
lee1043 Sep 28, 2016
d6d10c4
Enables existing json file update
lee1043 Sep 28, 2016
688504f
bug fix
lee1043 Sep 28, 2016
a007136
Bug fix
lee1043 Oct 5, 2016
89cfa6f
Merge remote-tracking branch 'origin/master' into 377_Variability_mod…
lee1043 Oct 14, 2016
9d2bed0
JSON writing part update
lee1043 Oct 14, 2016
c9c58f0
EC-EARTH landmask N-S reverse error fix
lee1043 Oct 14, 2016
f226c1b
Add import, generalize pseudo pcs
lee1043 Oct 14, 2016
7dae325
Efficiency correction, minor
lee1043 Oct 14, 2016
9c9754b
Add fraction for pseudo pcs
lee1043 Oct 18, 2016
966eacd
Merge branch 'master' into 377_Variability_mode_NAO
lee1043 Oct 18, 2016
84656f7
Merge remote-tracking branch 'origin/master' into 377_Variability_mod…
lee1043 Oct 18, 2016
1b5683c
clean up
lee1043 Oct 19, 2016
81eb47f
clean-up
lee1043 Oct 19, 2016
ec92998
Update to latest version
lee1043 Jun 26, 2017
dc02b70
Update in a while...
lee1043 Oct 4, 2017
75172c8
Code clean up and archive
lee1043 Nov 9, 2017
4ff6975
rename directory
lee1043 Jan 11, 2018
07c9320
apply parameter file
lee1043 Jan 11, 2018
37441ef
apply parameter file
lee1043 Jan 11, 2018
e69eb67
apply parameter file
lee1043 Jan 11, 2018
81cbaec
clean up
lee1043 Jan 11, 2018
d4d85cc
apply parameter file
lee1043 Jan 11, 2018
8e84d74
clean up
lee1043 Jan 11, 2018
5390ee4
clean up
lee1043 Jan 11, 2018
44a6956
clean up
lee1043 Jan 11, 2018
a1cc813
clean up
lee1043 Jan 11, 2018
ff2a7e4
Update myParam_NAM.py
gleckler1 Jan 11, 2018
fed4b94
Merge branch 'master' into 377_Variability_mode_NAO
lee1043 Oct 22, 2019
6c8f684
move dir to consistent with latest master version dir structure
lee1043 Oct 22, 2019
76a97df
update with latest code
lee1043 Oct 22, 2019
8c114cf
add params from latest repo
lee1043 Oct 22, 2019
5929299
reorg
lee1043 Oct 22, 2019
e461696
reorg
lee1043 Oct 22, 2019
df49fe1
update with latest
lee1043 Oct 22, 2019
b7abaa3
reorg
lee1043 Oct 22, 2019
4acd03a
reorg
lee1043 Oct 22, 2019
5046f07
clean up
lee1043 Oct 22, 2019
837fbea
clean up: flake8
lee1043 Oct 22, 2019
049a37f
clean up
lee1043 Oct 22, 2019
0ac03f0
clean up
lee1043 Oct 22, 2019
da39452
clean up
lee1043 Oct 22, 2019
6b9d856
clean up
lee1043 Oct 22, 2019
818f370
clean up flake8
lee1043 Oct 22, 2019
57b7bd9
update
lee1043 Oct 22, 2019
e0a83bd
importing defined functions
lee1043 Oct 22, 2019
c493e4d
add debug_print repearting: temporary solution...
lee1043 Oct 22, 2019
cda4d55
temporary solution...
lee1043 Oct 22, 2019
34ccc09
clean up
lee1043 Oct 22, 2019
c99e969
clean up
lee1043 Oct 22, 2019
e617843
testcode
lee1043 Oct 22, 2019
81c519f
clean up
lee1043 Oct 22, 2019
a4fdb76
clean up
lee1043 Oct 22, 2019
6a93dd3
clean up
lee1043 Oct 22, 2019
51457b7
circle ci
lee1043 Oct 22, 2019
10192b1
clean up
lee1043 Oct 23, 2019
9302705
clean up: circle ci
lee1043 Oct 23, 2019
49b10c8
clean up: circle ci
lee1043 Oct 23, 2019
dcefa70
bug fix
lee1043 Oct 23, 2019
798b7dc
clean up: circle ci
lee1043 Oct 23, 2019
430c92a
clean up: sort_human to be consistent to that in io/base.py
lee1043 Oct 23, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Archive plotting scripts
  • Loading branch information
lee1043 committed Aug 23, 2016
commit 931433d4babf72df29f702d236d7ffb627f72485
90 changes: 90 additions & 0 deletions src/python/devel/variability_mode_nao/scripts/combine.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Combine multiple images into one.
#
# To install the Pillow module on Mac OS X:
#
# $ xcode-select --install
# $ brew install libtiff libjpeg webp little-cms2
# $ pip install Pillow
#
# Basement of this code is coming from https://gist.github.com/glombard/7cd166e311992a828675

from __future__ import print_function
from PIL import Image
import os
import math

modes = ['NAO', 'NAM', 'SAM', 'PNA', 'PDO']

#debug = True
debug = False

lr = False
#lr = True

pseudo = False
pseudo = True

if lr:
ftail1 = '_lr'
else:
ftail1 = ''

if pseudo:
ftail2 = '_pseudo'
else:
ftail2 = ''

for mode in modes:

print(mode)
if mode == 'PDO': seasons = ['monthly']
else: seasons = ['DJF', 'MAM', 'JJA', 'SON']

for season in seasons:

# List up all image files ---
files = os.popen('ls ../'+mode+'/'+mode+'_*_eof1_'+season+'_*_1900-2005'+ftail1+ftail2+'.png').readlines()
files = sorted(files, key=lambda s:s.lower()) # Sort list alphabetically, case-insensitive

# Bring the observation to beginning ---
ref_file = '../'+mode+'/'+mode + '_*_eof1_'+season+'_obs_1900-2005' + ftail1 + '.png'
ref_file_list = os.popen('ls '+ref_file).readlines()
try:
files.remove(ref_file_list[0])
except:
pass
files = ref_file_list + files

# Calculate optimum row/col ---
num = len(files)
col = int(math.ceil(num * 0.4)) # round up
if col > 6: col = 6
row = num // col
if num % col > 0: row = row + 1
if debug: print(num, col, row)

# Set thumbnail plot size ---
hor = 400
ver = 400

# Open new image ---
result = Image.new("RGB", (col*hor, row*ver), (255,255,255,0))

# Append plots ---
for index, file in enumerate(files):
if debug: print(index, file)
path = os.path.expanduser(file)
#img = Image.open(path)
img = Image.open(path.strip())
img.thumbnail((hor, ver), Image.ANTIALIAS)
x = index % col * hor
y = index // col * ver
w, h = img.size
if debug: print('pos {0},{1} size {2},{3}'.format(x, y, w, h))
result.paste(img, (x, y, x + w, y + h))

# Save image ---
#output_image = 'combined_image_'+mode+'_' + season + ftail1 + ftail2 + '.png'
#result.save(os.path.expanduser('./'+output_image))
output_image = 'Combined_'+mode+'_' + season + ftail1 + ftail2 + '.pdf'
result.save(os.path.expanduser('./'+output_image),"PDF", resoultion = 100.0)
19 changes: 19 additions & 0 deletions src/python/devel/variability_mode_nao/scripts/plot.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/bin/bash

# Temporary example to run plot

set -ax

mode='NAM'
#mode='NAO'
#mode='SAM'
#mode='PNA'

#for mode in 'NAM' 'NAO' 'SAM' 'PNA'; do
python -i read_json_plot_charts.py -j '../'$mode'/var_mode_'$mode'_eof1_stat_cmip5_historical_r1i1p1_mo_atm_1900-2005.json' -s 'all' -v 'psl' -e 'cmip5' -m $mode
#done

mode='PDO'
season='monthly'

#python -i read_json_plot_charts.py -j '../'$mode'/var_mode_'$mode'_eof1_stat_cmip5_historical_r1i1p1_mo_atm_1900-2005.json' -s $season -v 'psl' -e 'cmip5' -m $mode
60 changes: 60 additions & 0 deletions src/python/devel/variability_mode_nao/scripts/plot_barChart.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
import numpy as np
import matplotlib.pyplot as PLT
import sys

class BarChart(object):

def __init__(self, mods, data, fig=None, rect=111):

# Canvas setup
if fig is None:
fig = PLT.figure
ax = fig.add_subplot(rect)

# Enable to control ax and fig options outside of this file
self._ax = ax
self._fig = fig

# Axis setup
unit_adjust = 1.
unit = 'Unit needed here ...'
ymax = max(data)
ymin = min(data)
yint = int((ymax-ymin)/3)

# Array setup
num_mods = len(mods)
x = np.linspace(0,num_mods-1,num_mods)
y = np.array(data) * unit_adjust

# Label setup
index = range(1,num_mods+1)
labels = mods

# Plotting
ax.bar(x,y,bottom=0,align='center')
ax.axhline(0, color='black')

# Title and axis
ax.set_title('Subtitle needed here..')
ax.set_xlabel('Models')
ax.set_ylabel(unit)
ax.set_xlim([-1.,len(y)-0.5])
ax.set_ylim([y.min()*1.1,y.max()*1.1])
ax.grid(True)

# Axis labels
PLT.xticks(x,labels,rotation='vertical') ## Label x-axis as model names
#PLT.xticks(x,index,rotation='vertical') ## Label x-axis as model names
#ax.yaxis.set_ticks(np.arange(int(ymin-yint), int(ymax+yint), yint))

# Multi model mean / std. dev. / min / max
yave=np.mean(y)
ystd=np.std(y)
ymin=np.amin(y)
ymax=np.amax(y)

ax.plot(-0.7,yave,'x',c='red',markersize=12) # ave
ax.plot(-0.7,ymin,'+',c='red',markersize=12) # min
ax.plot(-0.7,ymax,'+',c='red',markersize=12) # max
ax.errorbar(-0.7,yave,yerr=ystd,ls='solid',color='red',linewidth=1) # inter-model std. dev.
120 changes: 120 additions & 0 deletions src/python/devel/variability_mode_nao/scripts/plot_scatter.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
import numpy as NP
import matplotlib.pyplot as plt
import sys
import matplotlib.cm as cm

def plot_scatter(models,data1,data2,file_out,label1,label2):

print 'scatter plot: ', label1, label2

#
# Canvas setup
#
if debug:
plt.ion()

fig = plt.figure(figsize=(13,12))
ax = plt.subplot(111)
plt.subplots_adjust(left=0.1, right=0.75, top=0.9, bottom=0.1, wspace=0, hspace=0)

#
# Array setup and plot
#
if debug:
print len(data1), len(data2)

#
# Simple quality control
#
if len(data1) == len(data2) and len(models)==len(data1):
num_dots = len(data1)
else:
sys.exit("data size dose not match")

# X axis ---
xlabel = str.upper(label1)
#xmax = NP.max(data1)*1.1
xmax = NP.max(data1)
xmin = NP.min(data1)
if xmin < 0: xmin = xmin*1.1
else: xmin = xmin*0.9
xint = (xmax-xmin)/5.
if xint < 0.1: xint = round(xint,2)
elif xint < 1: xint = round(xint,1)
else: xint = int(xint)
xlstart = xmin//1
if debug: print 'xmin, xmax, xint:',xmin, xmax, xint

# Y axis ---
ylabel = str.upper(label2)
#ymax = NP.max(data2)*1.1
ymax = NP.max(data2)
ymin = NP.min(data2)
if ymin < 0: ymin = ymin*1.1
else: ymin = ymin*0.9
yint = (ymax-ymin)/5.
if yint < 0.1: yint = round(yint,2)
elif yint < 1: yint = round(yint,1)
else: yint = int(yint)
ylstart = ymin//1
if debug: print 'ymin, ymax, yint:',ymin, ymax, yint

#colors = cm.rainbow(NP.linspace(0, 1, num_dots))
#colors = cm.gist_rainbow(NP.linspace(0, 1, num_dots))
#colors = cm.gist_ncar(NP.linspace(0, 1, num_dots))
colors = cm.Paired(NP.linspace(0, 1, num_dots))
#colors = cm.Set1(NP.linspace(0, 1, num_dots))
#colors = cm.nipy_spectral(NP.linspace(0, 1, num_dots))
#colors = cm.hsv(NP.linspace(0, 1, num_dots))
#colors = cm.jet(NP.linspace(0, 1, num_dots))

for i, mod in enumerate(models):
x = float(data1[i])
y = float(data2[i])
mod = str(models[i])
ax.scatter(x, y, label=str(i+1)+' '+mod, c=colors[i], s=200, linewidth=0.15)
# Show index numbers
ax.annotate(str(i+1), (x,y), fontsize='small', verticalalignment='center', horizontalalignment='center', color='white')

ax.legend(frameon=True, scatterpoints=1, bbox_to_anchor=(1.05, 1), loc=2,
borderaxespad=0., fontsize=11, labelspacing=0.435, ncol=1)

#
# Title and axis labeling
#
ax.set_title(file_out, fontsize=18)
ax.set_xlabel(xlabel, fontsize=17)
ax.set_ylabel(ylabel, fontsize=17)
#ax.set_xlim([xmin,xmax])
#ax.set_ylim([ymin,ymax])
ax.grid(True)

#ax.xaxis.set_ticks(NP.arange(xlstart, xmax+xint/2., xint))
#ax.yaxis.set_ticks(NP.arange(ylstart, ymax+yint/2., yint))
plt.tick_params(labelsize=15) # Tick label font size

#
# Text test
#
x0, xmax = plt.xlim()
y0, ymax = plt.ylim()
data_width = xmax - x0
data_height = ymax - y0
#plt.text(x0 + data_width * 0.5, y0 + data_height * 0.5, 'Some text')

#
# Linear regression
#
fit = NP.polyfit(data1, data2, 1)
fit_fn = NP.poly1d(fit)
plt.plot(data1, fit_fn(data1), '-')
slope = fit[0]
#ax.text(xmax, ymax, 'Regression slope ='+str(round(slope,2)), va='center', ha='left', color='blue', fontsize=15)
ax.text(x0 + data_width * 0.65, y0 + data_height * 0.95, 'Regression slope ='+str(round(slope,2)), va='center', ha='left', color='blue', fontsize=15)

#
# Save image file
#
fig.savefig(file_out+'_scatter.png',dpi=300)
if debug:
plt.show()
Loading