Skip to content

[CVPR 2023] STMixer: A One-Stage Sparse Action Detector

Notifications You must be signed in to change notification settings

MCG-NJU/STMixer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

20 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

STMixer

This repository gives the official PyTorch implementation of STMixer: A One-Stage Sparse Action Detector (CVPR 2023)

Installation

  • PyTorch == 1.8 or 1.12 (other versions are not tested)
  • tqdm
  • yacs
  • opencv-python
  • tensorboardX
  • SciPy
  • fvcore
  • timm
  • iopath

Data Preparation

Please refer to PySlowFast DATASET.md for AVA dataset preparation.

Model Zoo

Backbone Config Pre-train Model Frames Sampling Rate Model
SlowOnly-R50 cfg K400 4 16 Link
SlowFast-R50 cfg K400 8 8 Link
SlowFast-R101-NL cfg K600 8 8 Link
ViT-B(VideoMAE) cfg K400 16 4 Link
ViT-B(VideoMAEv2) cfg K710+K400 16 4 Link

Training

python -m torch.distributed.launch --nproc_per_node=8 train_net.py --config-file "config_files/config_file.yaml" --transfer --no-head --use-tfboard

Validation

python -m torch.distributed.launch --nproc_per_node=8 test_net.py --config-file "config_files/config_file.yaml" MODEL.WEIGHT "/path/to/model"

Acknowledgements

We would like to thank Ligeng Chen for his help in drawing the figures in the paper and thank Lei Chen for her surpport in experiments. This project is built upon AlphaAction, AdaMixer and PySlowFast. We also reference and use some code from SparseR-CNN, WOO and VideoMAE. Very sincere thanks to the contributors to these excellent codebases.

Citation

If this project helps you in your research or project, please cite our paper:

@inproceedings{wu2023stmixer,
      title={STMixer: A One-Stage Sparse Action Detector}, 
      author={Tao Wu and Mengqi Cao and Ziteng Gao and Gangshan Wu and Limin Wang},
      booktitle={{CVPR}},
      year={2023}
}