All package installations should be done in a virtualenv or conda env to prevent package conflict errors.

• Install required requirements for onnx and openvino Inference

pip install --upgrade pip
pip install -r inf_requirements.txt

• Clone and install requirements for yolov5 repository

git clone https://github.com/ultralytics/yolov5                    # clone repo
cd yolov5
pip install -r requirements.txt                                    # base requirements

Export a pre-trained or custom trained YOLOv5 model to generate the respective ONNX, TorchScript and CoreML formats of the model. The pre-trained yolov5s.pt is the lightest and fastest model for CPU inference. Other slower but more accurate models include yolov5m.pt, yolov5l.pt and yolov5x.pt. All available model details at Ultralytics YOLOv5 README.

A custom training checkpoint i.e. runs/exp/weights/best.pt can be used for conversion as well.

• Export a pre-trained light yolov5s.pt model at 640x640 with batch size 1

python export.py --weights yolov5s.pt --include onnx --img 640 --batch 1

• Export a custom checkpoint for dynamic input shape {BATCH_SIZE, 3, HEIGHT, WIDTH}. Note, for CPU inference mode, BATCH_SIZE must be set to 1. Install onnx-simplifier for simplifying onnx exports

pip install onnx-simplifier==0.3.10                                
python export.py --weights runs/exp/weights/best.pt --include onnx  --dynamic --simplify

• Cd to yolov5_export_cpu dir and move the onnx model to yolov5_export_cpu/models directory

mv <PATH_TO_ONNX_MODEL> yolov5_export_cpu/models/

python detect_onnx.py -m image -i <IMG_FILE_PATH/IMG_DIR_PATH>
python detect_onnx.py -m video -i <VID_PATH_FILE>
# python detect_onnx.py -h for more info

Optional: To convert the all frames in the output directory into a mp4 video using ffmpeg, use ffmpeg -r 25 -start_number 00001 -i output/frame_onnx_%5d.jpg -vcodec libx264 -y -an onnx_result.mp4

Instructions for setting OpenVINO available here

# install required OpenVINO lib to convert ONNX to OpenVINO IR
pip install openvino-dev[onnx]

This will create the OpenVINO Intermediate Model Representation (IR) model files (xml and bin) in the directory models/yolov5_openvino.

Important Note: --input_shape must be provided and match the img shape used to export ONNX model. Batching might not supported for CPU inference

# export onnx to OpenVINO IR
mo \
  --progress \
  --input_shape [1,3,640,640] \
  --input_model models/yolov5s.onnx \
  --output_dir models/yolov5_openvino

Full OpenVINO export options

Install docker in your system if not already installed.

Pass the docker run command below in a terminal which will automatically download the OpenVINO Docker Image and run it. The models directory containing the ONNX model must be in the current working directory.

docker run -it --rm \
            -v $PWD/models:/home/openvino/models \
            openvino/ubuntu18_dev:latest \
            /bin/bash -c "cd /home/openvino/; bash"

This will create the OpenVINO Intermediate Model Representation (IR) model files (xml and bin) in the directory models/yolov5_openvino which will be available in the host system outside the docker container.

Important Note: --input_shape must be provided and match the img shape used to export ONNX model. Batching might not supported for CPU inference

# inside the OpenVINO docker container
mo \
  --progress \
  --input_shape [1,3,640,640] \
  --input_model models/yolov5s.onnx \
  --output_dir models/yolov5_openvino
# exit OpenVINO docker container
exit  

Full OpenVINO export options

python detect_openvino.py -m image -i <IMG_FILE_PATH/IMG_DIR_PATH>
python detect_openvino.py -m video -i <VID_PATH_FILE>
# python detect_openvino.py -h for more info

Optional: To convert the all frames in the output directory into a mp4 video using ffmpeg, use ffmpeg -r 25 -start_number 00001 -i output/frame_openvino_%5d.jpg -vcodec libx264 -y -an openvino_result.mp4