This second project uses Meta Group's Deep Learning algorithm for drop detection and segmentation.
Firstly, it is necessary to install Detectron2. The installation procedure can be found on https://detectron2.readthedocs.io/en/latest/tutorials/install.html. It is recommended to install it directly in your Python virtual environment (venv). The models and pre-trained model weights are located in the 'detectron2ModelAndWeight' folder and provided in the literature, which can be modified by visiting https://github.com/facebookresearch/detectron2/blob/main/MODEL_ZOO.md
After installation, open detectron2script.py and specify the path to an image of your choice. For example, img\_path = "\Users{USERNAME}/Downloads/IMG\_1918.jpg". Use cfg.MODEL.ROI\_HEADS.SCORE\_THRESH\_TEST = 0.7 to set the threshold score for object detection. The higher the value, the more objects with high confidence scores will be displayed. This allows you to test Detectron2 on a basic image using a pre-trained model so that detected objects are common everyday items.
To train the model to detect specific objects, use detectron2training.py. First, provide training data, such as segmented droplet images. You need to approximate the coordinates of each droplet's outline in an image and input them to train Detectron2. All these files can be found in the balloon folder, composed of a training set called train and a validation set called val, both having identical structures containing images for training purposes, along with the via\_region\_data.json file containing boundary data for drops detected on different images. Simply run the script detectron2training.py. To adjust the duration of training or increase the precision of the model, modify cfg.SOLVER.MAX\_ITER = 300. The precision of the model can also be adjusted through cfg.MODEL.ROI\_HEADS.BATCH\_SIZE\_PER\_IMAGE = 128. If you have access to a GPU instead of a CPU, then cfg.MODEL.DEVICE = "cpu" becomes obsolete in both detectron2training.py and detectron2TestTraining.py.
Once the model has been trained, you may use it for testing on validation images by running detectron2TestTraining.py. You can still adjust the certainty threshold with cfg.MODEL.ROI\_HEADS.SCORE\_THRESH\_TEST = 0.01. Finally, feel free to use any other desired image for drop detection tests; simply specify its path. For instance, im = cv2.imread('\Users{USERNAME}/Downloads/dcdw1.jpg').
Providing proper training data might prove challenging; however, you can utilize training\_svc\_blackhat.py to achieve this goal. The method involves manual validation of outlines using 'y' or 'n' inputs to automatically generate training data for Detectron2. Thus, edge segmentation occurs using the first algorithm while users manually validate whether they match actual droplets. However, undetected edges cannot serve as training data for Detectron2. This technique merely provides a quick way to evaluate Detectron2's efficiency regarding drop detection problems.