Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pyrenew tutorial review #191

Merged

Conversation

sbidari
Copy link
Collaborator

@sbidari sbidari commented Jun 13, 2024

Some ideas for improving the documentation

  1. Introduction to renewal model framework and some of the model components implemented in the package. The model equations/information from here and here can perhaps be moved to the documentation
  2. Include installation instructions in the beginning of the documentation either in a separate section or within the getting started section.
  3. I am not entirely sure what is the difference between the HospitalAdmissionsModel in Pyrenew demo and the one in Fitting a Hospital Admissions-only model. Perhaps clarify what the two tutorials are trying to convey.
  4. If I understand correctly there are two different model implementation - Hospital Admissions and Reproduction Number Renewal Infections. But it isn't clear from the tutorial, suggesting reorganizing the tutorials to make these two model instances examples clear.
  5. Change the order of tutorial and reference in the documentation

I can make most of these changes (might need some help on 1) if we think these would be useful.

@sbidari sbidari added the documentation Improvements or additions to documentation label Jun 13, 2024
@sbidari sbidari added this to the 🐺 Lycorhinus milestone Jun 13, 2024
@sbidari sbidari self-assigned this Jun 13, 2024
@sbidari sbidari linked an issue Jun 13, 2024 that may be closed by this pull request
3 tasks
Copy link

codecov bot commented Jun 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.71%. Comparing base (7822c37) to head (f8829dd).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #191   +/-   ##
=======================================
  Coverage   92.71%   92.71%           
=======================================
  Files          40       40           
  Lines         906      906           
=======================================
  Hits          840      840           
  Misses         66       66           
Flag Coverage Δ
unittests 92.71% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sbidari sbidari changed the title test commit Pyrenew tutorial and tests review Jun 13, 2024
@damonbayer damonbayer mentioned this pull request Jun 14, 2024
@sbidari sbidari marked this pull request as ready for review June 17, 2024 15:55
@damonbayer damonbayer requested review from damonbayer and removed request for damonbayer June 17, 2024 16:36
@damonbayer
Copy link
Collaborator

damonbayer commented Jun 17, 2024

Some ideas for improving the documentation

  1. This sounds good. Please let us know what specifically you want help with.
  2. Include installation instructions in the beginning of the documentation either in a separate section or within the getting started section.
  3. I am not entirely sure what is the difference between the HospitalAdmissionsModel in Pyrenew demo and the one in Fitting a Hospital Admissions-only model. Perhaps clarify what the two tutorials are trying to convey.
  4. If I understand correctly there are two different model implementation - Hospital Admissions and Reproduction Number Renewal Infections. But it isn't clear from the tutorial, suggesting reorganizing the tutorials to make these two model instances examples clear.
  5. Change the order of tutorial and reference in the documentation

I can make most of these changes (might need some help on 1) if we think these would be useful.

  1. This will be covered (at least partially) by Adding Repository Structure And Resources Sections To The README #197
  2. I think installation instructions can go in both the main website page and the getting started tutorial. Note that installation instructions are also here, which is not on the website: https://github.com/CDCgov/multisignal-epi-inference/tree/main/model#readme. This may need to be updated.
  3. Agreed that these are confusing/redundant. I would lean toward merging the PyRenew Demo and Fitting a Hospital Admissions-only Model tutorials. Probably by keeping Fitting a Hospital Admissions-only Model mostly as is, but bringing over anything useful from PyRenew Demo that it is missing.
  4. That's correct. I agree making this more obvious would be useful. We provide two pre-built models: RtInfectionsRenewalModel and HospitalAdmissionsModel. HospitalAdmissionsModel uses RtInfectionsRenewalModel and builds upon it.
  5. Sounds good.

@damonbayer damonbayer removed their request for review June 17, 2024 17:31
@sbidari
Copy link
Collaborator Author

sbidari commented Jun 17, 2024

  1. That's correct. I agree making this more obvious would be useful. We provide two pre-built models: RtInfectionsRenewalModel and HospitalAdmissionsModel. HospitalAdmissionsModel uses RtInfectionsRenewalModel and builds upon it.

I liked the diagram here: https://github.com/CDCgov/multisignal-epi-inference/tree/main/model#readme and can be effective visualization to show HospitalAdmissionsModel uses RtInfectionsRenewalModel and builds upon it.

@damonbayer
Copy link
Collaborator

@sbidari Would it be appropriate to also address these comments from @cherz4 in this PR? If not, please create additional issues related to the points discussed on Teams.

@sbidari
Copy link
Collaborator Author

sbidari commented Jul 1, 2024

Yes, will include more instructions for poetry installation.

@kgostic
Copy link
Collaborator

kgostic commented Jul 2, 2024

Re: 2 - I think this would be a good place to add some installation instructions:

Specifically, I'd suggest a Prerequisites section before The fundamentals that reads something like this:

Prerequisites

This tutorial assumes some pre-existing knowledge of infectious disease dynamics and Python programming. Before you dive in, we recommend:

  • Installing Python3
  • Familiarity with installing and loading modules in python, and with virtual environment management (we recommend poetry)
  • Familiarity with the concept of a class and metaclass in python
  • Familiarity with Bayesian inference, and a working understanding of MCMC methods used to fit Bayesian models to data (link to resources like statistical rethinking?)

@kgostic
Copy link
Collaborator

kgostic commented Jul 2, 2024

Minor: I think this phrasing makes it sound like we're taking the ratio $\frac{I_t}{I_{t-s}}$, rather than considering all previous timesteps in the denominator.

The pyrenew package models the real-time reproductive number $R_t$, the ratio of new infections at 
time $t$ to previous infections at some time $t-s$, as a renewal process model

https://github.com/CDCgov/multisignal-epi-inference/blob/da3fae6a73e5b16e83cfaf986860d33fa39730ac/model/docs/getting_started.qmd#L65C1-L65C236

@sbidari sbidari requested a review from damonbayer as a code owner July 5, 2024 21:10
@sbidari
Copy link
Collaborator Author

sbidari commented Jul 8, 2024

Changes to the tutorial/documentation in this PR:

  • added a page with Prerequisite and installation information
  • merge/remove pyrenew demo
  • rename 'example_with_dataset' to 'hospital_admissions_model'
  • demonstrate extraction of samples from the fitted model

Copy link
Collaborator

@damonbayer damonbayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR does not address tests, which is fine. However, we should change the title of the PR and make sure we still have an open issue for the tests.

@sbidari sbidari changed the title Pyrenew tutorial and tests review Pyrenew tutorial review Jul 8, 2024
@sbidari sbidari mentioned this pull request Jul 8, 2024
@sbidari sbidari requested a review from damonbayer July 8, 2024 19:28
Copy link
Collaborator

@damonbayer damonbayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the new/renamed tutorials to the website navigation.

model/docs/basic_renewal_model.qmd Outdated Show resolved Hide resolved
model/docs/hospital_admissions_model.qmd Outdated Show resolved Hide resolved
@damonbayer damonbayer mentioned this pull request Jul 8, 2024
@sbidari sbidari requested a review from damonbayer July 9, 2024 19:51
Copy link
Collaborator

@damonbayer damonbayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is still plenty of room for improvement in the tutorials, but this is a big step in the right direction. Thanks @sbidari!

@damonbayer damonbayer merged commit a83551b into main Jul 9, 2024
8 checks passed
@damonbayer damonbayer deleted the 166-zib2-identify-areas-for-improvement-in-tutorials-and-tests branch July 9, 2024 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Identify areas for improvement in tutorials and tests
4 participants