Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP Add option to change jacobian dtype on EQL gridders #183

Closed
wants to merge 2 commits into from

Conversation

santisoler
Copy link
Member

Add an extra parameter on EQLHarmonic and EQLSpherical constructors to change the dtype of Jacobian elements.
This may help on reducing memory consumption while might drop accuracy of the predicted coefficients.

Fixes #176

Reminders:

  • Run make format and make check to make sure the code follows the style guide.
  • Add tests for new features or tests that would have caught the bug that you're fixing.
  • Add new public functions/methods/classes to doc/api/index.rst and the base __init__.py file for the package.
  • Write detailed docstrings for all functions/classes/methods. It often helps to design better code if you write the docstrings first.
  • If adding new functionality, add an example to the docstring, gallery, and/or tutorials.
  • Add your full name, affiliation, and ORCID (optional) to the AUTHORS.md file (if you haven't already) in case you'd like to be listed as an author on the Zenodo archive of the next release.

@santisoler
Copy link
Member Author

I'm trying to test the accuracy of the EQL gridder that uses jacobian_dtype="float32" by:

  1. Checking if it can recover the data on the observation points,
  2. Comparing predicted values on a grid with an EQL that uses jacobian_dtype="float64".

And I'm facing some drop on the accuracy (an rtol of ~0.5), which by visual inspection is not huge, but can make npt.assert_allclose to fail for some survey points distributions.

So my main question is: should we add this feature?

If yes, which is the best way to test it?

A few ideas came to my head, but I would like to hear some opinions:

  1. The accuracy problem is probably originated while comparing tiny values (close to zero).
  2. We can then shift the data, so there are no values close to zero.
  3. We can compare them through absolute difference (atol), rather than with relative difference (rtol).

I've tested some of these ideas: 2 and 3 seem to work (but I'm not sure if they are a proper way to test this feature), although they might get problems for a certain random choice of the coordinate points. So another option is to fix the choice of those coordinate points by specifying the random_state on vd.scatter_points.

@santisoler santisoler changed the title Add option to change jacobian dtype on EQL gridders WIP Add option to change jacobian dtype on EQL gridders Jul 29, 2020
@leouieda
Copy link
Member

This needs a lot of work on Verde to allow user input of dtypes throughout the project. There are several places where arrays are created that need to take dtype as input.

@santisoler
Copy link
Member Author

I don't think that changing the dtype of the Jacobian would produce any significant improvement on the memory saving side. In fact, as I saw, the accuracy of the predictions can drop below the desired level. I'm closing this PR then.

@santisoler santisoler closed this Apr 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow changing data type of EQLs Jacobian elements
2 participants