Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output Reader abstraction #511

Merged
merged 9 commits into from
Jan 24, 2024
Merged

Output Reader abstraction #511

merged 9 commits into from
Jan 24, 2024

Conversation

bemcdonnell
Copy link
Member

Closes #281, #407

I took this opportunity to build upon what we have in here to remove the need to go to swmm.toolkit to investigate the enum attributes to find what properties are available. I think that made it overly difficult to use... So I fixed it. This now done with properties. As we / if we add new properties to the ENUM later, we will not need to modify anything (but we can update docs if needed).

Example of this is used:

from pyswmm import Output, NodeSeries

with Output('tests/data/model_full_features.out') as out:
    ts1 = SubcatchSeries(out)['S1'].rainfall
    ts2 = SubcatchSeries(out)['S1'].snow_depth
    ts3 = SubcatchSeries(out)['S1'].evap_loss
    ts4 = SubcatchSeries(out)['S1'].infil_loss
    ts5 = SubcatchSeries(out)['S1'].runoff_rate
    ts6 = SubcatchSeries(out)['S1'].gw_outflow_rate
    ts7 = SubcatchSeries(out)['S1'].gw_table_elev
    ts8 = SubcatchSeries(out)['S1'].soil_moisture
    ts9 = SubcatchSeries(out)['S1'].pollut_conc_0

    ts1 = NodeSeries(out)['J1'].invert_depth
    ts2 = NodeSeries(out)['J1'].hydraulic_head
    ts3 = NodeSeries(out)['J1'].ponded_volume
    ts4 = NodeSeries(out)['J1'].lateral_inflow
    ts5 = NodeSeries(out)['J1'].total_inflow
    ts6 = NodeSeries(out)['J1'].flooding_losses
    ts7 = NodeSeries(out)['J1'].pollut_conc_0

    ts1 = LinkSeries(out)['C1:C2'].flow_rate
    ts2 = LinkSeries(out)['C1:C2'].flow_depth
    ts3 = LinkSeries(out)['C1:C2'].flow_velocity
    ts4 = LinkSeries(out)['C1:C2'].flow_volume
    ts5 = LinkSeries(out)['C1:C2'].capacity
    ts6 = LinkSeries(out)['C1:C2'].pollut_conc_0

    ts1 = SystemSeries(out).air_temp
    ts2 = SystemSeries(out).rainfall
    ts3 = SystemSeries(out).snow_depth
    ts4 = SystemSeries(out).evap_infil_loss
    ts5 = SystemSeries(out).runoff_flow
    ts6 = SystemSeries(out).dry_weather_inflow
    ts7 = SystemSeries(out).gw_inflow
    ts8 = SystemSeries(out).rdii_inflow
    ts9 = SystemSeries(out).direct_inflow
    ts10 = SystemSeries(out).total_lateral_inflow
    ts11 = SystemSeries(out).flood_losses
    ts12 = SystemSeries(out).outfall_flows
    ts13 = SystemSeries(out).volume_stored
    ts14 = SystemSeries(out).evap_rate
    ts15 = SystemSeries(out).ptnl_evap_rate

@bemcdonnell
Copy link
Member Author

@karosc If you could review this one first, that would be a little better with the code flow. I meant to branch of develop-v2 for my other PR.

Copy link
Member

@karosc karosc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes largely look fine to me. My only comments are related to developer experience and style choices.

Things I like

  1. Concise property based access of timeseries data
  2. Similar usage to Nodes(sim), Links(sim), Subcatchments(sim) objects, which provides a cohesive feel to the pyswmm coding style with both simulation and output objects
  3. docstrings 🤤
  4. Nearly full coverage of new code with new unit tests (technically the exceptions in the OutAttributeBase class are not tested, but we can let that slide probably 😉)

Things I would improve

  1. Missing type hints. This is not a precedent set in this repo, so I can accept not having them. But in my opinion, python code without type hints in this era is incomplete code. We should consider adding type stubs to inline hints as an improvement in the future. Because, they are not included here now, we will have to create them later if we take on that effort.
  2. Although you can access timeseries as props with these changes, they are not defined in a __dir__ method and so you cannot tab complete them. Providing the timeseries data access as properties is a developer experience improvement, so why don't we take it that extra step to get tab completion working so users don't need to have docstrings open side-by-side with code to know what props are available to each SWMM object.

pyswmm/output.py Show resolved Hide resolved
pyswmm/output.py Show resolved Hide resolved
pyswmm/output.py Show resolved Hide resolved
pyswmm/output.py Show resolved Hide resolved
pyswmm/output.py Outdated Show resolved Hide resolved
pyswmm/output.py Show resolved Hide resolved
pyswmm/output.py Outdated Show resolved Hide resolved
@bemcdonnell
Copy link
Member Author

@karosc exceptional review! I’ll make all the changes you’ve requested. I’ll plan to do it before the end of the week if you can be available again on Friday for round 2.

@bemcdonnell
Copy link
Member Author

@karosc, Ready for review / approval!

@bemcdonnell bemcdonnell requested review from karosc and removed request for abhiramm7 January 23, 2024 16:47
@karosc
Copy link
Member

karosc commented Jan 23, 2024

So it turns out we might need to add type hints for IDEs to recognize the props. The __dir__method will allow autocomplete in a REPL, but the IDE's and static type checkers wont read it. Maybe we can add some property type hints at the top of each class comme ça?

class LinkSeries(OutAttributeBase):
    """
    Get a link time series.  New to PySWMM-v2!

    Note: you can use pandas to convert dict to a pandas Series object with dict keys as index

    :return: dict of attribute values with between start_index and end_index
             with reporting timesteps as keys
    :rtype: dict {datetime : value}

    Examples:

    .. code-block:: python

        from pyswmm import Output, LinkSeries

        with Output('tests/data/model_full_features.out') as out:
            ts1 = LinkSeries(out)['C1:C2'].flow_rate
            ts2 = LinkSeries(out)['C1:C2'].flow_depth
            ts3 = LinkSeries(out)['C1:C2'].flow_velocity
            ts4 = LinkSeries(out)['C1:C2'].flow_volume
            ts5 = LinkSeries(out)['C1:C2'].capacity
            ts6 = LinkSeries(out)['C1:C2'].pollut_conc_0

    """

    flow_rate: dict[datetime.datetime, float] 
    flow_depth: dict[datetime.datetime, float]

    def __init__(self, out_handle):
        super().__init__(out_handle)
        self._attr_group = shared_enum.LinkAttribute
        self._idname = None

    def __getitem__(self, idname):
        self._idname = idname
        return self

    def _series_type(self, attr_select):
        return self._handle.link_series(self._idname, attr_select)

@bemcdonnell bemcdonnell merged commit 56947d0 into develop-v2.x Jan 24, 2024
36 checks passed
@bemcdonnell bemcdonnell deleted the outreader_abstraction branch April 12, 2024 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants