Resolve Issue 95: Added SQL to prune 311 data to most recent 30 days #123
Conversation
Updated the SQL query for gathering 311 data, to limit the data returned to be from the most recent 30 days. Also removed the arbitrary limitation that had been on the 311 data, which limited it to gather the 250 most recent records.
main.js
Outdated
| @@ -286,7 +286,7 @@ | |||
| // TODO: would be great to prune 311 data to the last 30 days, like the police data | |||
| "311": { | |||
| id: "40776043-ad00-40f5-9dc8-1fde865ff571", | |||
| primaryFiltering: "WHERE \"NEIGHBORHOOD\" LIKE '%Oakland' ORDER BY \"CREATED_ON\" DESC", | |||
| primaryFiltering: "WHERE \"NEIGHBORHOOD\" LIKE '%Oakland' AND \"CREATED_ON\" >= (NOW() - '30 day'::INTERVAL) ORDER BY \"CREATED_ON\" DESC", | |||
There was a problem hiding this comment.
So this works on all data sets from the WPRDC? That would be awesome
There was a problem hiding this comment.
Haven't tried it on the other data-sets, but I'll try it out and get back to you!
main.js
Outdated
| @@ -455,7 +455,7 @@ | |||
| function fetchAllData() { | |||
| Promise.all([ | |||
| fetchWPRDCData("Police", { limit: 250 }), | |||
| fetchWPRDCData("311", { limit: 250 }), | |||
| fetchWPRDCData("311"), | |||
There was a problem hiding this comment.
Could we rm the arbitrary limitation on the others, too? Or do we need to have the work-around there?
There was a problem hiding this comment.
If the updated ".primaryFilter" that I used for the 311 data-set works for the others, we shouldn't need this limit for any of them. I'll test it out and get back to you!
|
@BenjaminMuscato did you get a chance to run your tests? Before you commit again, you'll need to merge in the recent changes, which separate some constants into another JS file. |
This reverts commit e9824ec.
…95---WPRDC-SQL-Date-Range
|
Looks great! If you take out those TODO comments I can merge it in! |
Removed the "TODO" statements about adding the SQL filtering. I left the JS filtering in place for now, as it's also needed to filter by Day and Week, even after the data is gathered from the database.
|
@vonbearshark I've removed the "TODO" comments, I think the PR is ready to go. I didn't touch any of the JS filtering that's currently in place, since I think we still need it in its current form to filter through the data even after we've gotten it from the database (to show by Day, Week, and Month in the GUI). |
|
Great work, @BenjaminMuscato. Thanks a lot! This is a good addition |
Updated the SQL query for gathering 311 data, to limit the data returned
to be from the most recent 30 days.
Also removed the arbitrary limitation that had been on the 311 data,
which limited it to gather the 250 most recent records.