Performance issue

[SilentJCR]

Hi, I've been trying your work and have modified some of the python files for my own use. Things are working well.

I'm just here to point out that there's potentially some performance issue in load_image(), in which cv2.cvtColor() is done before cv2.resize(). Doing cvtColor on the raw image, which is usually larger than the input size of the model, takes more time than on the resized one and it may also influence the memory allocation, impairing the performance of cv2.resize().

I tested it out by modifying the code and sent a 1080P video clip as the input in order to record the average time taken for each frame.

Original steps (cvtColor -> resize)

Modified steps (resize -> cvtColor)

 def preprocess(image, input_w=518, input_h=518):
     resized_image = cv2.resize(image, (input_w, input_h), interpolation=cv2.INTER_CUBIC)
     resized_image = cv2.cvtColor(resized_image, cv2.COLOR_BGR2RGB)
 
     # Normalize image (mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
     mean = np.array([0.485, 0.456, 0.406], dtype=np.float32)
     std = np.array([0.229, 0.224, 0.225], dtype=np.float32)
     normalized_image = (resized_image / 255.0 - mean) / std
 
     # Reorder dimensions from HWC to CHW (Channels, Height, Width)
     input_tensor = np.transpose(normalized_image, (2, 0, 1))
 
     # Add batch dimension (1, Channels, Height, Width)
     input_tensor = np.expand_dims(input_tensor, axis=0).astype(np.float32)
 
     return input_tensor