Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PresRat correction #215

Draft
wants to merge 57 commits into
base: main
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
89fbd29
cfg: Missing to ignore F401
castelao Apr 18, 2024
17f065c
style: Ignore F401 only on __init__
castelao Apr 19, 2024
4d1a49d
Initializing PresRat with reference
castelao Apr 23, 2024
1f490a8
feat: ZeroRateMixin()
castelao Apr 23, 2024
c0f1fcb
refact: Moving ZeroRateMixin to mixins
castelao May 24, 2024
5a8f09a
Adding ipython into dev environment
castelao May 24, 2024
f680a46
fix: ZeroRateMixin shouldn't inherit DataRetrievalBase
castelao May 24, 2024
a31595c
refact: Moving PresRat to qdm sub-module
castelao May 24, 2024
5af4631
fix: Use the given threshold (it was hardcoded before)
castelao May 24, 2024
e9ce57f
PresRat using ZeroRateMixin
castelao May 24, 2024
cbee7c9
doc: Explicit steps for PresRat
castelao May 27, 2024
d6f38e5
Prototype for quantiles on time windows
castelao May 28, 2024
3ceda3f
fix: Missing load PresRat at module
castelao May 28, 2024
c2005cc
Overwritting QDM methods for PresRat requirements
castelao Jun 3, 2024
74facd6
Don't need to overwrite __init__ if defining threshold in the runtime
castelao Jun 3, 2024
f3c4ea9
doc: More details on what is PresRat
castelao Jun 3, 2024
b83108b
Testing apply_zero_precipitation_rate()
castelao Jun 6, 2024
8e2a5c9
test: zero_precipitation_rate()
castelao Jun 11, 2024
a49160a
test: presrat_zero_rate with multiple thresholds
castelao Jun 12, 2024
f66069e
fix, doc: Typo 'monthly'
castelao Jun 12, 2024
4f07699
test: apply_zero_precipitation_rate()
castelao Jun 13, 2024
285015c
test: Minimalist check for test_presrat_transform()
castelao Jun 13, 2024
670ba56
test: parallel results must equal serial
castelao Jun 13, 2024
326133d
style: Fixing code style
castelao Jun 17, 2024
739473b
test: test_presrat_transform_nochanges()
castelao Jun 20, 2024
ecf9f8c
test: Missing to update PresRat standard params for testing
castelao Jun 20, 2024
a2c7a37
test: Adding back a small noise on future projections
castelao Jun 20, 2024
e509792
test: Defining some resources used in the tests
castelao Jun 20, 2024
96eee15
clean, test:
castelao Jun 20, 2024
1c2745b
test: test_presrat_zero_rate with multiple thresholds
castelao Jun 20, 2024
0bb4d60
test, clean: Combining tests
castelao Jun 20, 2024
bdad088
test: A fixture with fut_cc dataset itself for efficiency
castelao Jun 20, 2024
14bd475
test: Standard presrat transform test with dataset from memory
castelao Jun 20, 2024
d0d4fcd
test: test_presrat_transform_nozerochanges()
castelao Jun 20, 2024
d2c8e79
test: test_presrat_transform_nochanges() working
castelao Jun 20, 2024
98360af
style:
castelao Jun 20, 2024
b9272cb
test, doc: test_presrat_transform_nochanges()
castelao Jun 20, 2024
87d5401
Saving used threshold in metadata
castelao Jun 21, 2024
7f8d522
Entry point for presrat functions
castelao Jun 21, 2024
4a7cede
feat: apply_zero_precipitation_rate()
castelao Jun 21, 2024
6af2cf6
feat: get_spatial_bc_presrat()
castelao Jun 21, 2024
4fe27f6
fix, test: test_presrat_transform_nochanges()
castelao Jun 21, 2024
6743a86
feat: Saving requirements to estimate K factor
castelao Jun 21, 2024
b5f21e4
Adding time indices
castelao Jun 21, 2024
811a478
fix: zero_precipitation_rate() dealing with NaN
castelao Jun 21, 2024
df61300
refact: Moved apply_zero_precipitation_rate() into bias_tranforms
castelao Jun 21, 2024
6ef54b8
fix: Missing to load xarray
castelao Jun 21, 2024
c869ab4
refact: get_spatial_bc_quantiles returing params as dict
castelao Jun 24, 2024
0446bb5
feat: local_presrat_bc()
castelao Jun 24, 2024
803a072
style: Mostly moving to single quote which is the standard used in sup3r
castelao Jun 24, 2024
34e4d71
test, clean: PresRat tests were already moved into its own module
castelao Jun 24, 2024
24290c5
Locking numpy to < 2.0
castelao Jun 24, 2024
3d5d5ac
doc: Updating one-line description of PresRat
castelao Jun 24, 2024
7d76c17
test: Updating fut_cc_notrend() to match the 'resource'
castelao Jun 24, 2024
a55262e
test, doc: Better explaing the expected effect of NaNs
castelao Jun 24, 2024
a3f6714
feat: PresRat.write_outputs()
castelao Jun 24, 2024
237d674
fix, doc: Weird string literal issue
castelao Jun 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Prev Previous commit
Next Next commit
test: presrat_zero_rate with multiple thresholds
  • Loading branch information
castelao committed Jun 12, 2024
commit a49160a1af993ef61dda764d0cf4e4304420f6a1
91 changes: 91 additions & 0 deletions tests/bias/test_presrat_bias_correction.py
Original file line number Diff line number Diff line change
Expand Up @@ -136,3 +136,94 @@ def test_zero_precipitation_rate_nan():
r1 = f(arr, threshold=5)
r2 = f(np.concatenate([5*[np.nan], arr]), threshold=5)
assert r1 == r2

"""
breakpoint()

# Physically non sense threshold.
out = calc.run(zero_rate_threshold=0)

assert 'ghi_zero_rate' in out, 'Missing ghi_zero_rate in calc output'
zero_rate = out['ghi_zero_rate']
assert np.all(np.isfinite(zero_rate)), "Unexpected NaN for ghi_zero_rate"
assert np.all(zero_rate==0), "It should be all zero percent"

# Physically non sense threshold.
out = calc.run(zero_rate_threshold=1e6)

assert 'ghi_zero_rate' in out, 'Missing ghi_zero_rate in calc output'
zero_rate = out['ghi_zero_rate']
assert np.all(np.isfinite(zero_rate)), "Unexpected NaN for ghi_zero_rate"
assert np.all(zero_rate==1), "It should be all zero percent"
"""

def test_presrat_zero_rate(fp_fut_cc):
"""Estimate zero_rate within PresRat.run()"""
calc = PresRat(
FP_NSRDB,
FP_CC,
fp_fut_cc,
'ghi',
'rsds',
target=TARGET,
shape=SHAPE,
bias_handler='DataHandlerNCforCC',
)

# Physically non sense threshold.
out = calc.run(zero_rate_threshold=50)

assert 'ghi_zero_rate' in out, 'Missing ghi_zero_rate in calc output'
zero_rate = out['ghi_zero_rate']
assert np.all(np.isfinite(zero_rate)), "Unexpected NaN for ghi_zero_rate"
assert np.all((zero_rate>=0) & (zero_rate<=1)), "Out of range [0, 1]"


def test_presrat_zero_rate_threshold_zero(fp_fut_cc):
"""Estimate zero_rate within PresRat.run(), zero threshold

This should give a zero rate answer, since all values are higher.
"""
calc = PresRat(
FP_NSRDB,
FP_CC,
fp_fut_cc,
'ghi',
'rsds',
target=TARGET,
shape=SHAPE,
bias_handler='DataHandlerNCforCC',
)

# Physically non sense threshold.
out = calc.run(zero_rate_threshold=0)

assert 'ghi_zero_rate' in out, 'Missing ghi_zero_rate in calc output'
zero_rate = out['ghi_zero_rate']
assert np.all(np.isfinite(zero_rate)), "Unexpected NaN for ghi_zero_rate"
assert np.all(zero_rate==0), "Threshold=0, rate should be 0"


def test_presrat_zero_rate_threshold_1e9(fp_fut_cc):
"""Estimate zero_rate within PresRat.run(), zero threshold

This should give a zero rate answer, since all values are lower.
"""
calc = PresRat(
FP_NSRDB,
FP_CC,
fp_fut_cc,
'ghi',
'rsds',
target=TARGET,
shape=SHAPE,
bias_handler='DataHandlerNCforCC',
)

# Physically non sense threshold.
out = calc.run(zero_rate_threshold=1e9)

assert 'ghi_zero_rate' in out, 'Missing ghi_zero_rate in calc output'
zero_rate = out['ghi_zero_rate']
assert np.all(np.isfinite(zero_rate)), "Unexpected NaN for ghi_zero_rate"
assert np.all(zero_rate==1), "Threshold=0, rate should be 0"