The Repository contains the First Place Vesuvius Grand Prize solution. This repository is part of the First Place Grand Prize Submission to the Vesuvius Challenge 2023 from Youssef Nader, Luke Farritor and Julian Schilliger.

Check out the automatic segmentation pipeline ThaumatoAnakalyptor of our winning Grand Prize submission by Julian Schiliger. ThaumatoAnakalyptor performs in full 3D and is also capable of segmenting in very mushy and twisted scroll regions.

Our final canconical model was a timesformer small architecture with divided space-time attention. The dataset underwent expansion and cleaning rounds to increase accuracy of the labels and become as accurate as possible, approximately 15 rounds were performed between the first letters and final solution. Our solution also consisted of 2 other architectures, Resnet3D-101 with pretrained weights, I3D with non-local block and maxpooling.

Our implementation uses torch, torch-lightning,the timesformer-pytorch and 3D-ResNets-PyTorch.

I recommend using a docker image like pytorch/pytorch:2.0.1-cuda11.7-cudnn8-devel for your development environment. Kaggle/Colab images should work fine as well.

To install this project, run:

pip install -r requirements
#to download the segments from the server
./download.sh
#propagates the inklabels into the respective segment folders for training
python prepare.py

You can find the weights of the canonical timesformer uploaded here

To run inference of timesformer:

python inference_timesformer.py --segment_id 20231210121321 20231221180251 --segment_path train_scrolls --model_path timesformer_wild15_20230702185753_0_fr_i3depoch=12.ckpt