Skip to content

MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation (TPAMI 2020) - adapted for surgical tool usage and added attention

License

Notifications You must be signed in to change notification settings

ilanit1997/MS-TCN2_surgical

 
 

Repository files navigation

MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation (TPAMI 2020)

This repository provides a PyTorch implementation of the paper MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation.

Environment

Python3, pytorch

Tradeoff exploration:

How can we use learned weights to control the tradeoff between global and local history?

Difference from original model:

  • We added additive attention to model.py using einstein summation
  • Combine train, val, predict and eval into one module named - train_predict_eval.py
  • The different experiments can be seen in - train_predict_eval.sh
  • Adding logs to ClearML
  • train-test-val split is unique to our dataset

Analysis plots:

Tradeoff results

Initial VS Learned weights

Initial VS Learned weights

Cite:

@article{li2020ms,
   author={Shi-Jie Li and Yazan AbuFarha and Yun Liu and Ming-Ming Cheng and Juergen Gall},
    journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, 
    title={MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation}, 
    year={2020},
    volume={},
    number={},
    pages={1-1},
    doi={10.1109/TPAMI.2020.3021756},
}

@inproceedings{farha2019ms,
  title={Ms-tcn: Multi-stage temporal convolutional network for action segmentation},
  author={Farha, Yazan Abu and Gall, Jurgen},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={3575--3584},
  year={2019}
}

About

MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation (TPAMI 2020) - adapted for surgical tool usage and added attention

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 92.3%
  • Shell 7.7%