Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use new OpenVINO Python API features #2

Merged
merged 4 commits into from
Sep 20, 2023

Conversation

jiwaszki
Copy link

As OpenVINO 2023.1 came out, there are few new features in Python API which project can utilize.

  • There is no need to use infer_new_request. Class CompiledModel handles creation of new request and reuse it between runs, saving some time.
  • Better handling of data, which includes direct use of Pytorch Tensors.
  • Additional performance benefits that come with memory sharing on inputs and outputs, thus reducing copies to minimum (share_inputs and share_outputs flags).
  • openvino.runtime can be easily replaced with openvino flat namespace.

I spend some time benchmarking the solution, I will be happy to share results on request. I can also assist in case of any questions about other OpenVINO features:)

@zhuzilin thanks for creating this solution, always happy to see OV in action!

@zhuzilin zhuzilin merged commit cc2f130 into zhuzilin:main Sep 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants