Skip to content

Machine learning applied to interpolate sea-surface height observations from altimetry tracks.

Notifications You must be signed in to change notification settings

jejjohnson/ml4ssh

Repository files navigation

Implicit Neural Representations for Interpolation

Author: J. Emmanuel Johnson


Collaborators:


Overview

Baseline Methods: Optimal Interpolation

$$ \mathbf{x}^a = \mathbf{x}^b + \mathbf{K}\left(\mathbf{y}_{obs} - \mathbf{x}^b \right) $$

Standard Methods: Kriging

$$ \mathbf{y} = \boldsymbol{\mu}(\mathbf{x}\phi) + \mathbf{K}\phi \left(\mathbf{y}{obs} - \boldsymbol{\mu}(\mathbf{X}\phi)\right) $$

Modern Methods: Implicit Neural Representations

$$ \mathbf{y}{obs} = \boldsymbol{f}(\mathbf{x}{\phi};\boldsymbol{\theta}) $$


Experiments


Challenge

Simulated Altimetry Tracks Simulated SSH Field
Animation Animation

For more information, see the experiment page.


Challenge

Altimetry Tracks SSH Field
Animation Animation

For more information, see the experiment page.


Demos

Image Regression (Jupyter Notebook)

A standard image regression problem on a fox image. This is the same experiment as the demo in Tancik et al (2020)

Image Regression + Physics Loss (Jupyter Notebook) (TODO)

The standard image regression problem with the physics informed loss function, i.e. the Poisson constraint (gradient, laplacian). This is the same experiment as the Siren paper Sitzmann et al (2020)

QG Simulation (Jupyter Notebook)

This uses a subset of the QG simulations to demonstrate how each of the networks perform. This application is useful for training INR as potential mesh-free surrogates.

QG Simulation + Physics Loss (Jupyter Notebook)

This uses a subset of the QG simulations to demonstrate how each of the networks perform along with the physics-informed QG loss function.


Installation Instructions

Conda Environment (Preferred)

conda env create -f environments/torch_linux.yaml

Pip Install (TODO)

Download (TODO)


Data Download

Datasets

  • 1.5 Layer QG Simulations
    • 94MB
  • SSH Data Challenge 2021a
    • Train/Test Data - 116MB
    • Results: BASELINE - ~15MB; DUACS - ~4.5MB
  • SSH Data Challenge 2020b (TODO)
  • SSH 5 Year Altimetry Tracks (TODO)

Instructions

Step 1: Go into data folder

cd data

Step 2: Give permissions

chmod +x dl_dc21a.sh

Step 3: Download data (bash or python)

See the detailed steps below.


Option 1: Bash Script

Run the bash script directly from the command line

bash dl_dc21a.sh username password path/to/save/dir

Option 2: Python script + credentials.yaml (Preferred)

Create a .yaml file. You can even append it to your already lon .yaml file.

aviso:
  username: username
  password: password

Download with the python script

python dl_dc21a.py --credentials-file credentials.yaml --save-dir path/to/save/dir

Bonus: M1 MacOS Compatible

I have included some environment files for the new M1 MacOS. This is because I personally use an M1 Macbook and I wanted to test out the new PyTorch M1 compatability which makes use of the M1 GPU. I personally found that the training and inference time for using PyTorch are much faster. This coincides with other users experiences (e.g. here) In addition, Anaconda claims that other packages potentially get a 20 % speedup. To install, use the requirement file:

mamba env create -f environments/torch_macos.yaml

Differences:

  • The training scripts use the skorch distribution. This is because it takes advantage of the M1 GPU and I have seen a substantial speed-up.
  • A different environment file, i.e. torch_macos.yaml.

Known Bugs

Datashader (Mac OS)

I cannot get datashader to work for the M1. But using the new Anaconda distribution works fine.

mamba create -n anaconda
mamba install anaconda=2022.05

Acknowledgements

Data


Discussions


Code

About

Machine learning applied to interpolate sea-surface height observations from altimetry tracks.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published