Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added ability to pass TF model as a config parameter #63

Closed
wants to merge 3 commits into from

Conversation

ivelin
Copy link

@ivelin ivelin commented Aug 23, 2019

@blakeblackshear here is what this PR suggests:

New config section:


################
# Optional Tensorflow settings. Change these only if you are familiar with Tensorflow.
# Intended for developer experimentation.
################
tensorflow:
  # object detection model and label paths
  # model: /config/mobilenet_ssd_v2_coco_quant_postprocess_edgetpu.tflite
  # labels: /config/coco_labels.txt
  # face detection model without labels
  model: /config/mobilenet_ssd_v2_face_quant_postprocess_edgetpu.tflite
  labels: /config/coco_labels.txt

In order to enable this, I also switched from DetectWithInputTensor to DetectWithImage. It saves a few lines of code figuring out the model's input tensor shape. This works for models with image input. In the future if we want to allow video, sound or other models, DetectWithImage won't be sufficient.

I did not notice performance impact by using PIL Image to feed a frame as an image to Coral instead of feeding frame np array as an InputTensor.

Looking forward to feedback.

A potential next step would be to enable chaining of models. If model A detects a label in a region (person), feed the detected box from the original image to the next model (face) and then send that to a third model to recognize the face. This will take a good amount of effort to implement and warrants a community discussion first. I'd be happy to contribute.

@ivelin
Copy link
Author

ivelin commented Aug 24, 2019

One more thing. Because of the use of DetectWithImage, it is no longer necessary for the regions to be square. The function resizes to fit the model input tensor while keeping the ratio of the fed image area.
If its OK, I would suggest allowing regions to be optional. By default the whole image can be passed to the model. Further zooming in can take place by model chaining from more generic to more specialized models.

@blakeblackshear
Copy link
Owner

See my comments on your more recent PR.

@hawkeye217
Copy link
Collaborator

hawkeye217 commented Aug 25, 2019

@blakeblackshear, as you mentioned on the other PR, we can pass in a new model via a -v or in a docker-compose.yaml as such:

    volumes:
      - /dev/bus/usb:/dev/bus/usb
      - /path/on/my/host/config:/config
      - /path/on/my/host/mobilenet_ssd_v2_face_quant_postprocess_edgetpu.tflite:/frozen_inference_graph.pb

The particular model I'd like to use is for face detection (which the current code would not work with), so I'd be interested in what comes out of this particular PR. Thanks for your hard work, @blakeblackshear and @ivelin !

@ivelin
Copy link
Author

ivelin commented Aug 25, 2019

@blakeblackshear, as you mentioned on the other PR, we can pass in a new model via a -v or in a docker-compose.yaml as such:

    volumes:
      - /dev/bus/usb:/dev/bus/usb
      - /path/on/my/host/config:/config
      - /path/on/my/host/mobilenet_ssd_v2_face_quant_postprocess_edgetpu.tflite:/frozen_inference_graph.pb

The particular model I'd like to use is for face detection (which the current code would not work with), so I'd be interested in what comes out of this particular PR. Thanks for your hard work, @blakeblackshear and @ivelin !

Its a nice model in my experience. Works well in my environment.

luoj1 pushed a commit to luoj1/frigate that referenced this pull request Apr 29, 2023
Updated readme to run antlr setup script
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants