Please login or register to submit your results.
We strongly encourage all participants to use only the sequences from the training set for finding parameters and report results on the provided detections to enable a meaningful comparison of tracking methods.
Tracking results will be evaluated automatically and made visible only to you. You will be able to make them public at any time.
Important: The evaluation server is not to be used for training. To discourage this, you will have to wait 72 hours (3 days) before you can re-submit your results. Moreover, you cannot submit the results for the same methods more than 4 times. If you want to present results of your method with various settings (e.g. different features, inference method, etc.), please use the training set for this purpose and submit only one result to the test server! Note that only the latest submission will be considered.Please submit your results as a single .zip file. The results for each sequence must be stored in a separate .txt file in the archive's root folder. The file name must be exactly like the sequence name (case sensitive).
The file format should be the same as the ground truth file, which is a CSV text-file containing one object instance per line. Each line must contain 10 values:
<frame>,
<id>,
<bb_left>,
<bb_top>,
<bb_width>,
<bb_height>,
<conf>,
<x>,
<y>,
<z>
The conf value contains the detection confidence
in the det.txt files. For the ground truth,
it acts as a flag whether the entry is to be considered. A value
of 0 means that this particular instance is ignored in the evaluation, while
any other value can be used to mark it as active.
For submitted results, all lines in the .txt file are considered.
The world coordinates x,y,z
are ignored for the 2D challenge and can be filled with -1.
Similarly, the bounding boxes are ignored for the 3D challenge.
However, each line is still required to contain 10 values.
All frame numbers, target IDs and bounding boxes are 1-based. Here is an example:
1, 3, 794.27, 247.59, 71.245, 174.88, -1, -1, -1, -1 1, 6, 1648.1, 119.61, 66.504, 163.24, -1, -1, -1, -1 1, 8, 875.49, 399.98, 95.303, 233.93, -1, -1, -1, -1 ...
1, 3, -1, -1, -1, -1, -1, 123.32, 2342.3, 0 1, 4, -1, -1, -1, -1, -1, 153.12, 2478.2, 0 2, 3, -1, -1, -1, -1, -1, 125.23, 2213.7, 0 ...
1, -1, 794.27, 247.59, 71.245, 174.88, 4.56 1, -1, 1648.1, 119.61, 66.504, 163.24, 0.32 1, -1, 875.49, 399.98, 95.303, 233.93, -1.34 ...
Each line of an annotation txt file is structured like this (where rle means run-length encoding from COCO):
time_frame id class_id img_height img_width rle
An example line from a txt file:
52 1005 1 375 1242 WSV:2d;1O10000O10000O1O100O100O1O100O1000000000000000O100O102N5K00O1O1N2O110OO2O001O1NTga3
Meaning:
time frame 52
object id 1005 (meaning class id is 1, i.e. car and instance id is 5)
class id 1
image height 375
image width 1242
rle WSV:2d;1O10000O10000O1O100O100O1O100O1000000000000000O100O...1O1N
image height, image width, and rle can be used together to decode a mask using cocotools .
Submit your tracking result where each row of your submission file has to contain the following values. The values are defined as in the annotation file, and any other values will be ignored.
Each line of an annotation txt file is structured as follows:
frame: The video frame which the annotation is associated with id: Identity of the fish 3d_x: x coordinate of 3D head position in world coordinates 3d_y: y coordinate of 3D head position in world coordinates 3d_z: z coordinate of 3D head position in world coordinates
Four example lines of a submission txt file:
1, 1, 19.61, 28.313, 7.93 1, 2, 18.317, 28.636, 8.911 2, 1, 19.685, 28.348, 7.886 2, 2, 18.197, 28.625, 8.868
Submit your tracking result as a zipped-up json file, which contains a list of elements in the following format:
[{
"image_id" : int,
"category_id" : int,
"bbox" : [x,y,width,height],
"score" : float,
"track_id": int,
"video_id": int
}, ...]
For more details, see https://github.com/TAO-Dataset/tao/blob/master/docs/evaluation.md, or contact [email protected].
Submit your result as a zip file, which contains a folder per sequence in its root. Each folder must contain one png file per frame. Please follow the exact naming convention, so your results can be evaluated. Each png file must be RGB with the same size as the input. The channels encode the following: (R) contains the semantic class, (G) contains the trackID // 256, (B) contains the trackID % 256. A list of semantic classes can be found here.
results.zip |- STEP-ICCV21-01 |--- 000001.png |--- 000002.png |--- ... |- STEP-ICCV21-07 |--- ...For any questions regarding the dataset or evaluation, please contact Mark Weber.
Please only upload predictions for which there exists ground-truth (1FPS). The detailed instructions can be found here. Please contact the benchmark author (Ali Athar) in case instructions are unclear.
./TUD-Stadtmitte.txt
./TUD-Campus.txt
./TUD-Crossing.txt
./PETS09-S2L1.txt
./PETS09-S2L2.txt
./ETH-Bahnhof.txt
./ETH-Sunnyday.txt
./ETH-Jelmoli.txt
./ETH-Linthescher.txt
./ETH-Pedcross2.txt
./ETH-Crossing.txt
./AVG-TownCentre.txt
./ADL-Rundle-1.txt
./ADL-Rundle-3.txt
./ADL-Rundle-6.txt
./ADL-Rundle-8.txt
./KITTI-13.txt
./KITTI-16.txt
./KITTI-17.txt
./KITTI-19.txt
./Venice-1.txt
./Venice-2.txt
./MOT16-01.txt
./MOT16-02.txt
./MOT16-03.txt
./MOT16-04.txt
./MOT16-05.txt
./MOT16-06.txt
./MOT16-07.txt
./MOT16-08.txt
./MOT16-09.txt
./MOT16-10.txt
./MOT16-11.txt
./MOT16-12.txt
./MOT16-13.txt
./MOT16-14.txt
./MOT17-01.txt
./MOT17-02.txt
./MOT17-03.txt
./MOT17-04.txt
./MOT17-05.txt
./MOT17-06.txt
./MOT17-07.txt
./MOT17-08.txt
./MOT17-09.txt
./MOT17-10.txt
./MOT17-11.txt
./MOT17-12.txt
./MOT17-13.txt
./MOT17-14.txt
./MOT17-01-DPM.txt
./MOT17-02-DPM.txt
./MOT17-03-DPM.txt
./MOT17-04-DPM.txt
./MOT17-05-DPM.txt
./MOT17-06-DPM.txt
./MOT17-07-DPM.txt
./MOT17-08-DPM.txt
./MOT17-09-DPM.txt
./MOT17-10-DPM.txt
./MOT17-11-DPM.txt
./MOT17-12-DPM.txt
./MOT17-13-DPM.txt
./MOT17-14-DPM.txt
./MOT17-01-FRCNN.txt
./MOT17-02-FRCNN.txt
./MOT17-03-FRCNN.txt
./MOT17-04-FRCNN.txt
./MOT17-05-FRCNN.txt
./MOT17-06-FRCNN.txt
./MOT17-07-FRCNN.txt
./MOT17-08-FRCNN.txt
./MOT17-09-FRCNN.txt
./MOT17-10-FRCNN.txt
./MOT17-11-FRCNN.txt
./MOT17-12-FRCNN.txt
./MOT17-13-FRCNN.txt
./MOT17-14-FRCNN.txt
./MOT17-01-SDP.txt
./MOT17-02-SDP.txt
./MOT17-03-SDP.txt
./MOT17-04-SDP.txt
./MOT17-05-SDP.txt
./MOT17-06-SDP.txt
./MOT17-07-SDP.txt
./MOT17-08-SDP.txt
./MOT17-09-SDP.txt
./MOT17-10-SDP.txt
./MOT17-11-SDP.txt
./MOT17-12-SDP.txt
./MOT17-13-SDP.txt
./MOT17-14-SDP.txt
./MOT20-01.txt
./MOT20-02.txt
./MOT20-03.txt
./MOT20-04.txt
./MOT20-05.txt
./MOT20-06.txt
./MOT20-07.txt
./MOT20-08.txt
./MOT20-01.txt
./MOT20-02.txt
./MOT20-03.txt
./MOT20-04.txt
./MOT20-05.txt
./MOT20-06.txt
./MOT20-07.txt
./MOT20-08.txt
./ZebraFish-01.txt
./ZebraFish-02.txt
./ZebraFish-03.txt
./ZebraFish-04.txt
./ZebraFish-05.txt
./ZebraFish-06.txt
./ZebraFish-07.txt
./ZebraFish-08.txt
./MOTS20-01.txt
./MOTS20-02.txt
./MOTS20-05.txt
./MOTS20-06.txt
./MOTS20-09.txt
./MOTS20-11.txt
./MOTS20-12.txt
./MOTS20-07.txt
./TAO_train.json
./TAO_test.json
./TAO_val.json
./3T3-run01.txt
./3T3-run03.txt
./3T3-run05.txt
./3T3-run07.txt
./3T3-run09.txt
./A-10-run01.txt
./A-10-run03.txt
./A-10-run05.txt
./A-10-run07.txt
./A-549-run03.txt
./APM-run01.txt
./APM-run03.txt
./APM-run05.txt
./BPAE-run01.txt
./BPAE-run03.txt
./BPAE-run05.txt
./BPAE-run07.txt
./CRE-BAG2-run01.txt
./CRE-BAG2-run03.txt
./CV-1-run01.txt
./CV-1-run03.txt
./LLC-MK2-run01.txt
./LLC-MK2-run02a.txt
./LLC-MK2-run03.txt
./LLC-MK2-run05.txt
./LLC-MK2-run07.txt
./MDBK-run01.txt
./MDBK-run03.txt
./MDBK-run05.txt
./MDBK-run07.txt
./MDBK-run09.txt
./MDOK-run01.txt
./MDOK-run03.txt
./MDOK-run05.txt
./MDOK-run07.txt
./MDOK-run09.txt
./OK-run01.txt
./OK-run03.txt
./OK-run05.txt
./OK-run07.txt
./PL1Ut-run01.txt
./PL1Ut-run03.txt
./PL1Ut-run05.txt
./RK-13-run01.txt
./RK-13-run03.txt
./U2O-S-run03.txt
./U2O-S-run05.txt
./3T3-run02.txt
./3T3-run04.txt
./3T3-run06.txt
./3T3-run08.txt
./A-10-run02.txt
./A-10-run04.txt
./A-10-run06.txt
./A-549-run02.txt
./A-549-run04.txt
./APM-run02.txt
./APM-run04.txt
./APM-run06.txt
./BPAE-run02.txt
./BPAE-run04.txt
./BPAE-run06.txt
./CRE-BAG2-run02.txt
./CRE-BAG2-run04.txt
./CV-1-run02.txt
./CV-1-run04.txt
./LLC-MK2-run02b.txt
./LLC-MK2-run04.txt
./LLC-MK2-run06.txt
./MDBK-run02.txt
./MDBK-run04.txt
./MDBK-run06.txt
./MDBK-run08.txt
./MDBK-run10.txt
./MDOK-run02.txt
./MDOK-run04.txt
./MDOK-run06.txt
./MDOK-run08.txt
./OK-run02.txt
./OK-run04.txt
./OK-run06.txt
./PL1Ut-run02.txt
./PL1Ut-run04.txt
./RK-13-run02.txt
./U2O-S-run02.txt
./U2O-S-run04.txt
./TAO_VOS_train.json
./TAO_VOS_val.json
./HT21-01.txt
./HT21-02.txt
./HT21-03.txt
./HT21-04.txt
./HT21-11.txt
./HT21-12.txt
./HT21-13.txt
./HT21-14.txt
./HT21-15.txt
./STEP-ICCV21-01
./STEP-ICCV21-07
./BURST_test.json
./BURST_test.json
./MOT17-01.txt
./MOT17-03.txt
./MOT17-06.txt
./MOT17-07.txt
./MOT17-08.txt
./MOT17-12.txt
./MOT17-14.txt
./MOTS20-01.txt
./MOTS20-06.txt
./MOTS20-12.txt
./MOTS20-07.txt
./CU15L1B4In_0.txt
./CU20L1B1Out_0.txt
./CU30L1B6Out_0.txt
./CU15L1B1In_0.txt
./CU20L1B4Out_0.txt
./CU10L1B6In_0.txt