One-Trimap Video Matting (ECCV 2022)
Hongje Seong, Seoung Wug Oh, Brian Price, Euntai Kim, Joon-Young Lee
Official Pytorch implementation of the ECCV 2022 paper, "One-Trimap Video Matting".
- Ubuntu 18.04
- python 3.8
- pytorch 1.8.2
- CUDA 10.2
conda create -n otvm python=3.8
conda activate otvm
conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch-lts
pip install opencv-contrib-python scikit-image scipy tqdm imgaug yacs albumentationsTo train OTVM, you need to prepare AIM and VideoMatting108 datasets
PATH/TO/DATASET
├── Combined_Dataset
│ ├── Adobe Deep Image Mattng Dataset License Agreement.pdf
│ ├── README.txt
│ ├── Test_set
│ │ ├── Adobe-licensed images
│ │ └── ...
│ └── Training_set
│ ├── Adobe-licensed images
│ └── ...
└── VideoMatting108
├── BG_done2
│ ├── airport
│ └── ...
├── FG_done
│ ├── animal_still
│ └── ...
├── flow_png_val
│ ├── animal_still
│ └── ...
├── frame_corr.json
├── train_videos_subset.txt
├── train_videos.txt
├── val_videos_subset.txt
└── val_videos.txt
Download the pre-trained weights from here and put them in the weight/ directory.
mkdir weights
mv STM_weights.pth weights/
mv FBA.pth weights/
mv s1_OTVM_trimap.pth weights/
mv s1_OTVM_alpha.pth weights/
mv s2_OTVM_alpha.pth weights/
mv s3_OTVM.pth weights/
mv s4_OTVM.pth weights/Note: Initial weights of the trimap propagation and alpha prediction networks were taken from STM and FBA, respectively.
vim config.py
# Change below path
_C.DATASET.PATH = 'PATH/TO/DATASET'# options: scripts/train_XXX.sh [GPUs]
bash scripts/train_s1_trimap.sh 0,1,2,3
bash scripts/train_s1_alpha.sh 0,1,2,3
bash scripts/train_s2_alpha.sh 0,1,2,3
bash scripts/train_s3.sh 0,1,2,3
bash scripts/train_s4.sh 0,1,2,3# options: scripts/eval_s4.sh [GPU]
bash scripts/eval_s4.sh 0# options: scripts/eval_s4_demo.sh [GPU]
# The results will be generated in: ./demo_results
bash scripts/eval_s4_demo.sh 0@inproceedings{seong2022one,
title={One-Trimap Video Matting},
author={Seong, Hongje and Oh, Seoung Wug and Price, Brian and Kim, Euntai and Lee, Joon-Young},
booktitle={European Conference on Computer Vision},
year={2022}
}
This software is for non-commercial use only. The source code is released under the Attribution-NonCommercial-ShareAlike (CC BY-NC-SA) Licence (see this for details)
This code is based on TCVOM (ACM MM 2021): [link]