Skip to content

Commit

Permalink
Readme updates
Browse files Browse the repository at this point in the history
  • Loading branch information
Panagiotou committed Feb 29, 2020
1 parent 5876903 commit 6e2a8b2
Show file tree
Hide file tree
Showing 10 changed files with 22 additions and 5 deletions.
Binary file added Media/animation_3d_n1_dem.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Media/animation_3d_n1_im.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified Media/animation_3d_n1_im2dem.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Media/animation_3d_n2_dem.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Media/animation_3d_n2_im.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified Media/animation_3d_n2_im2dem.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Media/virtual_dem.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Media/virtual_im.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified Media/virtual_im2dem.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
27 changes: 22 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,29 +20,46 @@ Inspired by <span style="color:green">Isola et al., 2017</span>, we used a U-Net
|---|---|
|![unet](./Media/unet_for_im2dem.png)| ![patchgan](./Media/patchgan_for_im2dem.png)

## Dataset Construction

To train the cGAN, we went ahead and created our own dataset. We used the ALOS World 3D database to get the DEMs that would serve as the groundtruth and utilized the Google Earth Engine API to couple them with the RGB satellite image (Sentinel-2)

![dataset](./Media/dataset_im2dem.png)

Our inplementation is in TensorFlow2.0. Our network was trained for 300 epochs, using the Adam optimizer (learning rate 2e-4, beta_1 0.5 and the rest is default) with the loss suggested by <span style="color:green">Isola et al., 2017</span>.
Our python script that downloads an RGB satellite image corresponding to a DEM (or GeoJSON geometry) can be found [here](./Visualization/DEM2rgb.py).

## Training-Results

Our implementation is in TensorFlow2.0. Our network was trained for 300 epochs, using the Adam optimizer (learning rate 2e-4, beta_1 0.5 and the rest is default) with the loss suggested by <span style="color:green">Isola et al., 2017</span>.

The network is able to approximate the test set better and better through time:

![epochs](./Media/epochs_im2dem.gif)

Using [this](https://github.com/zhunor/threejs-dem-visualizer) visualizer, we were able to render out results in 3D:

![vis_3d](./Media/animation_3d_n1_im2dem.gif)
| Input Image | Predicted Image | Visualization
|---|---|------|
|![before](./Media/animation_3d_n1_im.jpg)|![after](./Media/animation_3d_n1_dem.jpg)| ![vis_3d](./Media/animation_3d_n1_im2dem.gif)


![vis_3d](./Media/animation_3d_n2_im2dem.gif)
| Input Image | Predicted Image | Visualization
|---|---|---|
|![before](./Media/animation_3d_n2_im.jpg)|![after](./Media/animation_3d_n2_dem.jpg)| ![vis_3d](./Media/animation_3d_n2_im2dem.gif)

## Inverse problem

Moreover, we used the same network to solve the inverse problem, i.e. predicting the RGB image given a DEM. We received promising results:

![inverse](./Media/inverse_im2dem.png)

This system can be readily used, in collaboration with a simple script producing DEMs based on [Perlin noise](https://en.wikipedia.org/wiki/Perlin_noise) to produce artificial terrains, fully generated without any human supervision, with application in virtual reality environments, among others. Here is an example of such a landscape:
This system can be readily used, in collaboration with a simple [script](./RandomDEM/TerrainGen.py) producing DEMs based on [Perlin noise](https://en.wikipedia.org/wiki/Perlin_noise) to produce artificial terrains, fully generated without any human supervision, with application in virtual reality environments, among others. Here is an example of such a landscape:

| Input Image | Predicted Image | Visualization
|---|---|---|
|![before](./Media/virtual_dem.jpg) |![after](./Media/virtual_im.jpg)| ![virtual](./Media/virtual_im2dem.gif)


![virtual](./Media/virtual_im2dem.gif)

---

Expand Down

0 comments on commit 6e2a8b2

Please sign in to comment.