Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gb/solar module #92

Merged
merged 23 commits into from
Sep 16, 2022
Merged
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
3967370
started test setup for solar module
grantbuster Sep 13, 2022
14a2dd6
added two weeks of cleasky data for nsrdb solar tests
grantbuster Sep 13, 2022
624224c
added farms requirement for solar module
grantbuster Sep 13, 2022
d46f7b1
added basic solar module that will take clearsky ratio GAN outputs an…
grantbuster Sep 13, 2022
7b1ad5e
added solar irrad output handling
grantbuster Sep 13, 2022
69e1431
more tests
grantbuster Sep 13, 2022
d0b57e0
added get_node_cmd for solar module
grantbuster Sep 13, 2022
15d06ca
added solar file parser with test
grantbuster Sep 13, 2022
3569890
added cli
grantbuster Sep 13, 2022
62ff8aa
finished solar cli with tests
grantbuster Sep 13, 2022
88b2597
fixed up tests
grantbuster Sep 13, 2022
d9b96fe
added solar cli to the main cli and pipeline
grantbuster Sep 13, 2022
b23230e
moved i_t_chunks to optional arg and enabled the solar class method e…
grantbuster Sep 13, 2022
d306db3
fixed node count logic
grantbuster Sep 13, 2022
5e92b11
fixed a stupid assumption that the nsrdb time index would always have…
grantbuster Sep 13, 2022
d973a75
changed i_t_chunk arg -> temporal_id which makes a lot more sense
grantbuster Sep 14, 2022
d785fc1
logging updates
grantbuster Sep 14, 2022
883a77f
linter fixes
grantbuster Sep 14, 2022
2fb8630
added new rex version requirement for sup3r solar multitimeresource i…
grantbuster Sep 14, 2022
0476f26
added the option for the user to specify qa datsets with mapping from…
grantbuster Sep 14, 2022
b99c42a
fixed qa features as list
grantbuster Sep 14, 2022
2833f0f
CC-based clearsky ratio should be set with ceiling not scaled to the …
grantbuster Sep 15, 2022
ca33492
moved solar test utilities
grantbuster Sep 16, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
moved i_t_chunks to optional arg and enabled the solar class method e…
…ntry point to run all t chunks if not provided
  • Loading branch information
grantbuster committed Sep 15, 2022
commit b23230ee2a796bb905133ed40d5992ec9ce061c2
28 changes: 15 additions & 13 deletions sup3r/solar/solar.py
Original file line number Diff line number Diff line change
Expand Up @@ -526,20 +526,17 @@ def write(self, fp_out, features=('ghi', 'dni', 'dhi')):
logger.info(f'Finished writing file: {fp_out}')

@classmethod
def run_temporal_chunk(cls, i_t_chunk, fp_pattern, nsrdb_fp,
def run_temporal_chunk(cls, fp_pattern, nsrdb_fp,
fp_out_suffix='irradiance', tz=-6, agg_factor=1,
nn_threshold=0.5, cloud_threshold=0.99,
features=('ghi', 'dni', 'dhi')):
features=('ghi', 'dni', 'dhi'),
i_t_chunk=None):
"""Run the solar module on all spatial chunks for a single temporal
chunk corresponding to the fp_pattern. This typically gets run from the
CLI.

Parameters
----------
i_t_chunk : int
Index of the sorted list of unique temporal indices to run (parsed
from the files matching fp_pattern). This input typically gets set
from the CLI.
fp_pattern : str
Unix-style file*pattern that matches a set of spatiotemporally
chunked sup3r forward pass output files.
Expand Down Expand Up @@ -576,23 +573,28 @@ def run_temporal_chunk(cls, i_t_chunk, fp_pattern, nsrdb_fp,
features : list | tuple
List of features to write to disk. These have to be attributes of
the Solar class (ghi, dni, dhi).
i_t_chunk : int | None
Index of the sorted list of unique temporal indices to run (parsed
from the files matching fp_pattern). This input typically gets set
from the CLI. If None, this will run all temporal indices.
"""

temp = cls.get_sup3r_fps(fp_pattern, ignore=f'_{fp_out_suffix}.h5')
fp_sets, t_slices, temporal_ids, spatial_ids, target_fps = temp

i_fp_sets = [fp_set for i, fp_set in enumerate(fp_sets)
if temporal_ids[i] == temporal_ids[i_t_chunk]]
i_t_slices = [t_slice for i, t_slice in enumerate(t_slices)
if temporal_ids[i] == temporal_ids[i_t_chunk]]
i_target_fps = [target_fp for i, target_fp in enumerate(target_fps)
if i_t_chunk is not None:
fp_sets = [fp_set for i, fp_set in enumerate(fp_sets)
if temporal_ids[i] == temporal_ids[i_t_chunk]]
t_slices = [t_slice for i, t_slice in enumerate(t_slices)
if temporal_ids[i] == temporal_ids[i_t_chunk]]
target_fps = [target_fp for i, target_fp in enumerate(target_fps)
if temporal_ids[i] == temporal_ids[i_t_chunk]]

zip_iter = zip(i_fp_sets, i_t_slices, i_target_fps)
zip_iter = zip(fp_sets, t_slices, target_fps)
for i, (fp_set, t_slice, fp_target) in enumerate(zip_iter):
fp_out = fp_target.replace('.h5', f'_{fp_out_suffix}.h5')
logger.info('Running temporal index {} out of {}.'
.format(i + 1, len(i_fp_sets)))
.format(i + 1, len(fp_sets)))
kwargs = dict(t_slice=t_slice,
tz=tz,
agg_factor=agg_factor,
Expand Down