Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Remove attributes clear on TRT nodes for GetOptimizedSymbol #13703

Merged
merged 1 commit into from
Jan 8, 2019

Conversation

Caenorst
Copy link
Contributor

Description

For debugging reason TRT nodes attributes was cleared on the symbol copy returned by GetOptimizedSymbol.

This modification allow to save and load graph with TRT node using the following procedure:

# Save
mx.contrib.tensorrt.set_use_tensorrt(True)
executor = mx.contrib.tensorrt.tensorrt_bind(sym, ...)
opti_sym = mx.contrib.tensorrt.get_optimized_symbol(executor)
opti_sym.save('my_opti_sym')

# Load
mx.contrib.tensorrt.set_use_tensorrt(False)
opti_sym = mx.sym.load('my_opti_sym')

Comments

  • Once saved, the model can be used with MXNET_USE_TENSORRT set to False.
  • When arg_params / aux_params are integrated to TRT_engine you don't need to reinitialize as they will be directly saved in the TRT node attributes. It's still on the user side to check if the weights are in the TRT_node or not, and initialize them properly on the later case
  • You still can't save using JSON as some serialized attributes may contain "{", "}" or "EOF".

@Roshrini
Copy link
Member

@Caenorst Thanks for the contribution!
@KellenSunderland Can you review this PR?
@mxnet-label-bot Add [pr-awaiting-review]

@marcoabreu marcoabreu added the pr-awaiting-review PR is waiting for code review label Dec 21, 2018
@sandeep-krishnamurthy
Copy link
Contributor

@KellenSunderland - Can you please help review this PR?

@Roshrini
Copy link
Member

Roshrini commented Jan 8, 2019

LGTM. @KellenSunderland for review/merge

@KellenSunderland KellenSunderland merged commit 1f151fb into apache:master Jan 8, 2019
rondogency pushed a commit to rondogency/incubator-mxnet that referenced this pull request Jan 9, 2019
haohuanw pushed a commit to haohuanw/incubator-mxnet that referenced this pull request Jun 23, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
pr-awaiting-review PR is waiting for code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants