Skip to content
/ MAC Public

MAC: Mining Activity Concepts for Language-based Temporal Localization

License

Notifications You must be signed in to change notification settings

runzhouge/MAC

Repository files navigation

MAC

By Runzhou Ge, Jiyang Gao, Kan Chen, Ram Nevatia.

University of Southern California (USC).

Introduction

This repository contains the code for the WACV 2019 paper, MAC: Mining Activity Concepts for Language-based Temporal Localization. arXiv

Requirements

  • Python 2.7
  • Tensorflow 1.0 or higher
  • others

Download

The code is for Charades-STA dataset.

After cloning this repo, please donwload:

ref_info contains Charades-STA annotations, semantic activity concepts, checkpoints and others. After downloading ref_info.tar, untar it and move the folder to the root/ directory of this repo.

Please also change the visual feature and visual activity concepts directories in the main.py.

Training

For the paper results on Charades-STA dataset, run

python main.py --is_only_test True \
--checkpoint_path ./ref_info/charades_sta_wacv_2019_paper_ACL_k_results/trained_model.ckpt-10000 \
--test_name paper_results

You will get similar results listed in the row "ACL-K" of the following table.

Model R@1,IoU=0.7 R@1,IoU=0.5 R@5,IoU=0.7 R@5,IoU=0.5
CTRL 7.15 21.42 26.91 59.11
ACL-K 12.20 30.48 35.13 64.84

To train the model from scratch, run

python main.py

The results and checkpoints will appear in root/results_history/ and root/trained_save/, respectively.

Results Visualization

Citation

If you find this work is helpful, please cite:

@InProceedings{Ge_2019_WACV,
  author = {Ge, Runzhou and Gao, Jiyang and Chen, Kan and Nevatia, Ram},
  title = {MAC: Mining Activity Concepts for Language-based Temporal Localization},
  booktitle = {The IEEE Winter Conference on Applications of Computer Vision (WACV)},
  month = {January},
  year = {2019}
}

License

MIT License

Acknowledgements

This research was supported, in part, by the Office of Naval Research under grant N00014-18-1-2050 and by an Amazon Research Award.

About

MAC: Mining Activity Concepts for Language-based Temporal Localization

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages