Skip to content

genforce/sefa

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SeFa - Closed-Form Factorization of Latent Semantics in GANs

image Figure: Versatile semantics found from various types of GAN models using SeFa.

Closed-Form Factorization of Latent Semantics in GANs
Yujun Shen, Bolei Zhou
Computer Vision and Pattern Recognition (CVPR), 2021 (Oral)

[Paper] [Project Page] [Demo] [Colab]

In this repository, we propose a closed-form approach, termed as SeFa, for unsupervised latent semantic factorization in GANs. With this algorithm, we are able to discover versatile semantics from different GAN models trained on various datasets. Most importantly, the proposed method does not rely on pre-trained semantic predictors and has an extremely fast implementation (i.e., less than 1 second to interpret a model). Below show some interesting results on anime faces, cats, and cars.

NOTE: The following semantics are identified in a completely unsupervised manner, and post-annotated for reference.

Anime Faces
Pose Mouth Painting Style
image image image
Cats
Posture (Left & Right) Posture (Up & Down) Zoom
image image image
Cars
Orientation Vertical Position Shape
image image image

Semantic Discovery

It is very simple to interpret a particular model with

MODEL_NAME=stylegan_animeface512
LAYER_IDX=0-1
NUM_SAMPLES=5
NUM_SEMANTICS=5
python sefa.py ${MODEL_NAME} \
    -L ${LAYER_IDX} \
    -N ${NUM_SAMPLES} \
    -K ${NUM_SEMANTICS}

After the program finishes, there will be two visualization pages in the directory results.

NOTE: The pre-trained models are borrowed from the genforce repository.

Interface

We also provide an interface for interactive editing based on StreamLit. This interface can be locally launched with

pip install streamlit
CUDA_VISIBLE_DEVICES=0 streamlit run interface.py

After the interface is launched, users can play with it via a browser.

NOTE: We have prepared some latent codes in the directory latent_codes to ensure the synthesis quality, which is completely determined by the pre-trained generator. Users can simply skip these prepared codes by clicking the Random button.

BibTeX

@inproceedings{shen2021closedform,
  title     = {Closed-Form Factorization of Latent Semantics in GANs},
  author    = {Shen, Yujun and Zhou, Bolei},
  booktitle = {CVPR},
  year      = {2021}
}

About

[CVPR 2021] Closed-Form Factorization of Latent Semantics in GANs

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages