Zhangkai Ni¹, Juncheng Wu¹, Zian Wang¹, Wenhan Yang², Hanli Wang¹, Lin Ma³

¹Tongji University, ²Pengcheng Laboratory, ³Meituan

This repository provides the official PyTorch implementation for the paper “Misalignment-Robust Frequency Distribution Loss for Image Transformation”, CVPR-2024. Paper

This paper aims to address a common challenge in deep learning-based image transformation methods, such as image enhancement and super-resolution, which heavily rely on precisely aligned paired datasets with pixel-level alignments. However, creating precisely aligned paired images presents significant challenges and hinders the advancement of methods trained on such data. To overcome this challenge, this paper introduces a novel and simple Frequency Distribution Loss (FDL) for computing distribution distance within the frequency domain. Specifically, we transform image features into the frequency domain using Discrete Fourier Transformation (DFT). Subsequently, frequency components (amplitude and phase) are processed separately to form the FDL loss function. Our method is empirically proven effective as a training constraint due to the thoughtful utilization of global information in the frequency domain. Extensive experimental evaluations, focusing on image enhancement and super-resolution tasks, demonstrate that FDL outperforms existing misalignment-robust loss functions. Furthermore, we explore the potential of our FDL for image style transfer that relies solely on completely misaligned data.

TL;DR: We propose a novel Frequency Distribution Loss (FDL) for image transformation models trained with misaligned data, opening up new avenues for addressing the broad issue of misalignment in image transformation tasks.

The quantitative comparison of image enhancement on the DPED dataset demonstrates the superiority of our proposed FDL over all compared loss functions across various evaluation criteria.

We compare our proposed FDL against state-of-the-art loss functions in real-world single image super resolution.

Our proposed FDL is also capable of handling style transfer, since the use of distribution distance measurement in the frequency domain.

pip install fdl-pytorch

• Python>=3.6
• Pytorch>=1.0

from FDL_pytorch import FDL_loss
fdl_loss = FDL_loss()
# X: (N,C,H,W) 
# Y: (N,C,H,W) 
loss_value = fdl_loss(X, Y)
loss_value.backward()

If you find our work useful, please cite it as

@inproceedings{ni2024misalignment,
  title={Misalignment-robust frequency distribution loss for image transformation},
  author={Ni, Zhangkai and Wu, Juncheng and Wang, Zian and Yang, Wenhan and Wang, Hanli and Ma, Lin},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={2910--2919},
  year={2024}
}

Thanks for your attention! If you have any suggestion or question, feel free to leave a message here or contact Dr. Zhangkai Ni ([email protected]) or Juncheng Wu ([email protected]).

MIT License