python 3.8
pytorch 1.8.1
pytorchvideo
mmaction2
mmcv-full

Install pytorchvideo and mmaction2，please read [https://github.com/facebookresearch/pytorchvideo/blob/main/INSTALL.md ]、(https://github.com/facebookresearch/pytorchvideo/blob/main/INSTALL.md)[https://github.com/open-mmlab/mmaction2/blob/master/docs_zh_CN/install.md](https://github.com/open-mmlab/mmaction2/blob/master/docs_zh_CN/install.md)

Selecte pre-training model to evaluate their inference performance ,such as, SlowFast,I3D,C3D and TSN. Get the required pre-trained model: [https://github.com/open-mmlab/mmaction2]

• Run run.sh to evaluate the performance of the pre-trained model

./run.sh

• Optional arguments:

config           #test config file path
checkpoints The  #checkpoint file to be evaluated
data             #video file/url or rawframes directory
label            #path to the label set
batch_num
batch_size

• Run run.sh to evaluate the performance of SlowFast:

python run.py configs/slowfast_r50_4x16x1_256e_kinetics400_rgb.py\
    checkpoints/slowfast_r50_256p_4x16x1_256e_kinetics400_rgb_20200728-145f1097.pth \
    example.mp4 map/label_map_k400.txt --batch_size 1 --batch_num 11

The model input data is example.mp4, run.py calls modelcl.py to evaluate the model. There are four indicators that will be acquired, throughput, P50-latency, P95-latency and P99-latency.

Set batch_size=1,batch_num=11,data=example.mp4

• The throughput curves of the above four models are shown in the following figure. It can be seen that the throughput of SlowFast and ID3 is higher than that of TSN and C3D. The specific data is located in mm_latency_throughput.xls

• The latency curves of the above four models are shown in the following figure. It can be seen that the latency of SlowFast and ID3 are lower than TSN and C3D, and the latency time of SlowFast and ID3 is almost the same. From the perspective of latency , SlowFast and ID3 are better than TSN and C3D. TSN is better than C3D. The specific data is located in mm_latency_throughput.xls

• The four indicators of the model (result_mm.xlsx):

Selecte SlowFast to evaluate their inference performance.

• Run runcla.sh to evaluate the performance of the pre-trained model

./runcla.sh

• Optional arguments:

data        #video file/url or rawframes directory
label       #path to the label set
batch_num
batch_size

• Run runcla.sh to evaluate the performance of SlowFast:

python runcla.py example.mp4 lab_map_k400.txt --name slowfast --batch_size 1 --batch_num 11

The model input data is example.mp4, runcla.sh calls modelcl.py to evaluate the model. There are four indicators that will be acquired, throughput, P50-latency, P95-latency and P99-latency.

Set batch_size=1,batch_num=11,data=example.mp4

• The throughput curves of the above model is shown in the following figure. The specific data is located in pv_latency_throughput.xls

• The latency curves of the model is shown in the following figure.The specific data is located in pv_latency_throughput.xls

• The four indicators of the model (result.xlsx):