Skip to content

[ECCV2022] "Unsupervised Night Image Enhancement: When Layer Decomposition Meets Light-Effects Suppression", https://arxiv.org/abs/2207.10564

License

Notifications You must be signed in to change notification settings

eorroot/night-enhancement

 
 

Repository files navigation

night_enhancement (ECCV'2022)

Introduction

This is an implementation of the following paper.

Unsupervised Night Image Enhancement: When Layer Decomposition Meets Light-Effects Suppression. European Conference on Computer Vision (ECCV'2022)

Yeying Jin, Wenhan Yang and Robby T. Tan

arXiv

PWC

Replicate

[Poster] [Slides] [Video]

Abstract

Night images suffer not only from low light, but also from uneven distributions of light. Most existing night visibility enhancement methods focus mainly on enhancing low-light regions. This inevitably leads to over enhancement and saturation in bright regions, such as those regions affected by light effects (glare, floodlight, etc). To address this problem, we need to suppress the light effects in bright regions while, at the same time, boosting the intensity of dark regions. With this idea in mind, we introduce an unsupervised method that integrates a layer decomposition network and a light-effects suppression network. Given a single night image as input, our decomposition network learns to decompose shading, reflectance and light-effects layers, guided by unsupervised layer-specific prior losses. Our light-effects suppression network further suppresses the light effects and, at the same time, enhances the illumination in dark regions. This light-effects suppression network exploits the estimated light-effects layer as the guidance to focus on the light-effects regions. To recover the background details and reduce hallucination/artefacts, we propose structure and high-frequency consistency losses. Our quantitative and qualitative evaluations on real images show that our method outperforms state-of-the-art methods in suppressing night light effects and boosting the intensity of dark regions.

Datasets

Light-Effects Suppression on Night Data

  1. Light-effects data
    Light-effects data is collected from Flickr and by ourselves, with multiple light colors in various scenes: Aashish Sharma, Robby T. Tan. "Nighttime Visibility Enhancement by Increasing the Dynamic Range and Suppression of Light Effects", CVPR, 2021.

  1. LED data
    We captured images with dimmer light as the reference images.

  1. GTA5
    Synthetic GTA5 nighttime fog data: Wending Yan, Robby T. Tan, Dengxin Dai. "Nighttime Defogging Using High-Low Frequency Decomposition and Grayscale-Color Networks", ECCV, 2020.

  1. Syn-light-effects
    Synthetic-light-effects data is the implementation of the paper, S. Metari, F. Deschênes, "A New Convolution Kernel for Atmospheric Point Spread Function Applied to Computer Vision", ICCV, 2017.
    Run the Matlab code to generate Syn-light-effects:
glow_rendering_code/repro_ICCV2007_Fig5.m

Low-Light Enhancement

  1. LOL dataset
    LOL: Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu. "Deep Retinex Decomposition for Low-Light Enhancement", BMVC, 2018. [Baiduyun (extracted code: sdd0)] [Google Drive]

  2. LOL-Real dataset
    LOL-real (the extension work): Wenhan Yang, Haofeng Huang, Wenjing Wang, Shiqi Wang, and Jiaying Liu. "Sparse Gradient Regularized Deep Retinex Network for Robust Low-Light Image Enhancement", TIP, 2021. [Baiduyun (extracted code: l9xm)] [Google Drive]

    We use LOL-real as it is larger and more diverse.

Low-Light Enhancement Results:

Pre-trained Model

  1. Download the pre-trained LOL model, put in ./results/LOL/model/
  2. Put the test images in ./LOL/

Test

python main.py

Results

  1. LOL-Real Results

Get the following Table 4 in the main paper on the LOL-Real dataset (100 test images).

Learning Method PSNR SSIM
Unsupervised Learning Ours 25.51 0.8015
N/A Input 9.72 0.1752

[Update]: Re-train (train from scratch) in LOL_V2_real (698 train images), and test on LOL_V2_real (100 test images).
PSNR: 20.85 (vs EnlightenGAN's 18.23), SSIM: 0.7243 (vs EnlightenGAN's 0.61) pre-trained LOL_V2 model

  1. LOL-test Results

Get the following Table 3 in the main paper on the LOL-test dataset (15 test images).

Learning Method PSNR SSIM
Unsupervised Learning Ours 21.521 0.7647
N/A Input 7.773 0.1259

Light-Effects Suppression Results:

Demo

python demo.py

Feature Results:

  1. run the MATLAB code to adaptively fuse the three color channels, output I_gray
checkGrayMerge.m

  1. Download the fine-tuned VGG model (fine-tuned on ExDark (Exclusively Dark Image Dataset) ), put in ./VGG_code/ckpts/vgg16_featureextractFalse_ExDark/nets/model_best.tar

  2. obtain structure features

python test_VGGfeatures.py

Summary of Comparisons:

Citations

If this work is useful for your research, please cite our paper.

@inproceedings{jin2022unsupervised,
  title={Unsupervised night image enhancement: When layer decomposition meets light-effects suppression},
  author={Jin, Yeying and Yang, Wenhan and Tan, Robby T},
  booktitle={European Conference on Computer Vision},
  pages={404--421},
  year={2022},
  organization={Springer}
}

If light-effects data is useful for your research, please cite our paper.

@inproceedings{sharma2021nighttime,
	title={Nighttime Visibility Enhancement by Increasing the Dynamic Range and Suppression of Light Effects},
	author={Sharma, Aashish and Tan, Robby T},
	booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
	pages={11977--11986},
	year={2021}
}

If GTA5 nighttime fog data is useful for your research, please cite our paper.

@inproceedings{yan2020nighttime,
	title={Nighttime defogging using high-low frequency decomposition and grayscale-color networks},
	author={Yan, Wending and Tan, Robby T and Dai, Dengxin},
	booktitle={European Conference on Computer Vision},
	pages={473--488},
	year={2020},
	organization={Springer}
}

About

[ECCV2022] "Unsupervised Night Image Enhancement: When Layer Decomposition Meets Light-Effects Suppression", https://arxiv.org/abs/2207.10564

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 84.2%
  • MATLAB 15.8%