Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a merge step so that we can move trees into the database based on the geojson files. To do this, I've added a
merge
step to the data pipeline. This would run sometime after normalization using the geojsons from that step.The process does two things:
There are still a few TODOs in terms of async code we can add, albeit with additional libraries.
The performance of this step depends on the size of the file.
ogr2ogr
uploads 20k rows at a time. A file with 600k rows (like NYC) can take on the order of minutes, while a file with 20k rows (like Alameda) is within seconds. This is expected to run monthly, so that is not much of a concern.Not included in here is the
.env
file that will be necessary holding the database info, similar to the wtt_server repo.