• 2024-07-22: LocoTrack is released.
• 2024-08-03: PyTorch inference and training code released.
• 2024-08-05: Interactive demo released.

Please stay tuned for an easy-to-use API for LocoTrack, coming soon!

Try our interactive demo on Huggingface. To run the demo locally, please follow these steps:

1. Install Dependencies: Ensure you have all the necessary packages by running:

   pip install -r demo/requirements.txt

2. Run the Demo: Launch the interactive Gradio demo with:

   python demo/demo.py

For detailed instructions on training and evaluation, please refer to the README file for your chosen implementation:

• JAX Implementation
• PyTorch Implementation

First, download the evaluation datasets:

# TAP-Vid-DAVIS dataset
wget https://storage.googleapis.com/dm-tapnet/tapvid_davis.zip
unzip tapvid_davis.zip

# TAP-Vid-RGB-Stacking dataset
wget https://storage.googleapis.com/dm-tapnet/tapvid_rgb_stacking.zip
unzip tapvid_rgb_stacking.zip

# RoboTAP dataset
wget https://storage.googleapis.com/dm-tapnet/robotap/robotap.zip
unzip robotap.zip

For downloading TAP-Vid-Kinetics, please refer to official TAP-Vid repository.

Download the panning-MOVi-E dataset used for training (approximately 273GB) from Huggingface using the following script. Git LFS should be installed to download the dataset. To install Git LFS, please refer to this link. Additionally, downloading instructions for the Huggingface dataset are available at this link

git clone [email protected]:datasets/hamacojr/LocoTrack-panning-MOVi-E

Please use the following bibtex to cite our work:

@article{cho2024local,
  title={Local All-Pair Correspondence for Point Tracking},
  author={Cho, Seokju and Huang, Jiahui and Nam, Jisu and An, Honggyu and Kim, Seungryong and Lee, Joon-Young},
  journal={arXiv preprint arXiv:2407.15420},
  year={2024}
}

This project is largely based on the TAP repository. Thanks to the authors for their invaluable work and contributions.