Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: migrate to production veda stac catalog and titiler #414

Merged
merged 1 commit into from
Jul 17, 2024

Conversation

anayeaye
Copy link
Collaborator

@anayeaye anayeaye commented Jul 1, 2024

Why are you creating this Pull Request?

To migrate the Dashboard to new openveda.cloud VEDA instance.

How tested

Steps taken to validate the new production VEDA instance are documented here.

Copy link

netlify bot commented Jul 1, 2024

Deploy Preview for visex ready!

Name Link
🔨 Latest commit aaa1672
🔍 Latest deploy log https://app.netlify.com/sites/visex/deploys/66830ac24c9c330008f12dc1
😎 Deploy Preview https://deploy-preview-414--visex.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@hanbyul-here
Copy link
Collaborator

hanbyul-here commented Jul 8, 2024

Thanks for the PR and all the work put into this. @anayeaye 🙇 - If possible, I would like to know what the service URL change means for the dashboard. Is this the exact same service from a different domain? Did openveda.cloud instance introduce any change that the dashboard needs to be aware of - if there are, can you point me those?

I think I am currently unsure what should be reviewed in the PR. Should it mainly check that all the datasets get rendered and get analysis results as before? I would appreciate it if you can give me the pointers 🙇 🙇

@sandrahoang686
Copy link
Collaborator

The endpoint updates are really just a change of pointing to where the data lives now so we should make sure any datasets that may have recently been merged are in the prod DB/endpoints as well. @anayeaye am I missing anything?

For config instance, lets make sure E&A pages works the same as before this change. Our benchmark should be the same behavior as before these endpoint changes.

  • Check Datasets/layers
  • Run analyses and make sure everything runs as like before

@anayeaye
Copy link
Collaborator Author

@hanbyul-here @sandrahoang686 sorry for the delay in reply but I think it is all cleared up already

mainly check that all the datasets get rendered and get analysis results as before

Exactly--we think that the dashboard should not experience any change when transitioned to the new stable production URLs. Behind the scenes we copied the data from veda-data-store-staging to veda-data-store and generated new STAC records.

@hanbyul-here
Copy link
Collaborator

hanbyul-here commented Jul 11, 2024

I did not test every dataset - Based on some radom selections, I think the app is behaving in the same way with the new serviceUrl. (To note, this doesn't. mean that all things are working

ex. https://deploy-preview-414--visex.netlify.app/analysis/results?start=2018-01-01T00%3A00%3A00.000Z&end=2022-12-31T00%3A00%3A00.000Z&datasetsLayers=grdi-filled-missing-values-count&aoi=gsia%40mizlCsthgIqx%60yA%3Fdmf~Dv_hxHzoqeB

This analysis fails, but so does on the old serviceUrl :

https://www.earthdata.nasa.gov/dashboard/analysis/results?start=2018-01-01T00%3A00%3A00.000Z&end=2022-12-31T00%3A00%3A00.000Z&datasetsLayers=grdi-filled-missing-values-count&aoi=gsia%40mizlCsthgIqx%60yA%3Fdmf~Dv_hxHzoqeB)

I will leave a bit more thorough QA to @aboydnw 🙇

@aboydnw
Copy link
Contributor

aboydnw commented Jul 11, 2024

I went through most of the datasets and didn't notice any issues. Facebook population density doesn't load but it also doesn't load for me in production, so I'm guessing it is a separate issue.

I did notice something else weird where in the Data Analysis tab there are duplicates for the MTBS Burn Severity layer. Clicking on one of these layers checks/unchecks all 4 here. In reality, there is only one layer for the Disalexi ET Suppression dataset shown here, but there are multiple MTBS Burn Severity layers, across different datasets. I wonder if there is something with the statistices endpoint that doesn't handle layers with identical names very well? It might not matter for this page since we are replacing it with the new E&A page very soon, but I wanted to call it out in case it is in face something with the statistics endpoint itself that could lead to problems down the road.
image

@anayeaye
Copy link
Collaborator Author

Thanks for the QC, @aboydnw! It looks like the facebook population density dataset is working but perhaps another colormap would be preferable and/or perhaps a min zoom? After seeing no errors I zoomed in and found that it is just hard to see the layer. Here's a working tile https://openveda.cloud/api/raster/mosaic/ba15348a0b850d46b5dc8b3838b8c45a/tiles/WebMercatorQuad/8/141/89?assets=cog_default&resampling=bilinear&bidx=1&colormap_name=ylorrd&rescale=0%2C69

@aboydnw
Copy link
Contributor

aboydnw commented Jul 15, 2024

oh wow, you're totally right 😆 thanks for double checking!

@hanbyul-here hanbyul-here merged commit bca0879 into develop Jul 17, 2024
6 checks passed
@hanbyul-here hanbyul-here deleted the feat/migrate-to-openveda branch July 17, 2024 20:00
@hanbyul-here hanbyul-here mentioned this pull request Jul 18, 2024
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants