Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add WQP counts feature to data import tab #159

Open
3 tasks
cristinamullin opened this issue Jun 6, 2024 · 0 comments
Open
3 tasks

Add WQP counts feature to data import tab #159

cristinamullin opened this issue Jun 6, 2024 · 0 comments
Labels

Comments

@cristinamullin
Copy link
Collaborator

Describe the solution you'd like

'Warning' the user about the number of results using a pop-up before loading into TADA is in line with what WQP does, and would save a huge amount of bandwidth. Here is the WQP 'warning' pop-up.

image

Potential implementation using USGS Data Retrieval Package
Below are notes from Cristina/Laura D. email from 6/5/24
There are currently ways to get count information without doing a full data pull from the WQP, but neither option currently works with the new (WQX3) profile services (and therefore isn't 100% accurate with newer USGS data).

I'll show you below how to do it currently on the WQX2 profiles....but none of it will work going forward on the 3.0 profiles. I hear that it's going to be a priority to get new summary services implemented, but I don't have any idea how long it will be for that to happen. Therefore, I wouldn't recommend implementing these into TADA until we know how the 3.0 services will provide that information.

Anyway, with WQX2, here's how you could do it:

new_service <- readWQPdata(
statecode = "WI",
countycode = "Dane",
querySummary = TRUE, #<- this is the key to just get the counts back
service = "Result",
dataProfile = "resultPhysChem"
)

The result from that querySummary = TRUE is a list that breaks down the number of sites, activity, etc. The code that does the header request is here:
https://github.com/DOI-USGS/dataRetrieval/blob/main/R/getWebServiceData.R#L144

I usually recommend people use the WQP summary service (which again...doesn't work on the new profiles...)
summary <- readWQPsummary(
statecode = "WI",
countycode = "Dane"
)

Look at that "summary" data frame, you can see all sorts of ways to break up the data. For example:

library(dplyr)

summary_all <- summary |>
group_by(Provider) |>
summarize(total_sites = length(unique(MonitoringLocationIdentifier)),
total_activity = sum(ActivityCount))

To summarize...yes, we can get counts....but I wouldn't recommend implementing this until we find out how the WQX3 services will provide that information.

Additional context

Alternatively, we can consider using the services directly instead of dataRetrieval.

WQP/WQX 2.0 web services documentation: https://www.waterqualitydata.us/data/swagger-ui/index.html?docExpansion=none&url=/data/v3/api-docs#/Result/resultPostCountRequest

WQP/WQX 3.0 summary services including counts service are not yet available.

Reminders for TADA contributors addressing this issue

New features should include all of the following work:

  • Create the function/code.

  • Document all code using comments to describe what is does.

  • Create tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant