Skip to content

The format of tmfile

Jump to bottom
Zihang Li edited this page Apr 15, 2020 · 3 revisions

The format of tmfile

Model Structure

header -> table -> graph -> vector -> node -> op -> type/parameters/input_tensor/output_tensor
                         -> vector -> tensor -> dimension/layout/buffer_id 
                         -> vector -> buffer -> size/data

Graph

The tmfile models are organized by as the following structure.

  	                   node1
  	                     |
  	                     | tensor0
  	                     |
  	                   node2
  	               |           |
  	       tensor2 |           | tensor1
  	       	       |           |
  	       	     node4      node3
  	       	       |           |
  	       tensor4 |           | tensor3
  	       	       |           |
  	        	   node5
  	        	     |
  	                     | tensor5
  	                     |
  	                   node6

Data Format

Header -- 10 Bytes

Header is on the head of tmfile. It shows the version of this model and the root table address.

Header size type description
main version 2 Bytes uint16_t The main version of the model.
sub version 2 Bytes uint16_t The sub version of the model (default 0).
compile version 2 Bytes uint16_t The compiler version of the model (default 0).
offset to root table 4 Bytes tm_uoffset_t The address of the root table.

Tables

①root table -- 16 Bytes

root table size type description
original format 4 Bytes int32_t The original format of the model.
sub format 4 Bytes int32_t The sub format for DLA model.
offset to the vector of subgraph 4 Bytes tm_uoffset_t Only one subgraph is supported.
offset to the string of the model name 4 Bytes tm_uoffset_t The name

②subgraph -- 36 Bytes

subgraph size type description
subgraph id 4 Bytes uint32_t The subgraph id.
graph layout 4 Bytes int32_t The actual data layout.
model layout 4 Bytes int32_t The data layout of original model.
offset to the vector of input indices 4 Bytes tm_uoffset_t The vector of the inputs index.
offset to the vector of output indices 4 Bytes tm_uoffset_t The vector of the outputs index.
offset to the vector of nodes 4 Bytes tm_uoffset_t The vector of the nodes.
offset to the vector of tensors 4 Bytes tm_uoffset_t The vector of the tensors.
offset to the vector of buffers 4 Bytes tm_uoffset_t The vector of the buffers.
offset to the string of subgraph name 4 Bytes tm_uoffset_t The string of the subgraph name.

③node -- 28 Bytes

node size type description
node id 4 Bytes uint32_t The node id.
offset to the vector of input tensors 4 Bytes tm_uoffset_t Input tensors of this node.
offset to the vector of output tensors 4 Bytes tm_uoffset_t Output tensors of this node.
offset to the table of operator 4 Bytes tm_uoffset_t The offset to operator info.
offset to the string of node name 4 Bytes tm_uoffset_t The node name address.
offset to the vector of node attributes 4 Bytes tm_uoffset_t The vector of node attributes.
dynamic shape 1 bit Bool aligned by 4 Bytes.

④tensor -- 32 Bytes

tensor size type description
tensor id 4 Bytes uint32_t The tensor id.
buffer id 4 Bytes uint32_t The buffer of this tensor.
offset to the vector of dims 4 Bytes tm_uoffset_t The vector of input tensors.
offset to the string of tensor name 4 Bytes tm_uoffset_t The vector of output tensors.
offset to the vector of quantparams 4 Bytes tm_uoffset_t The offset to operator info.
layout 4 Bytes int32_t NCHW/NHWC
type 4 Bytes int32_t CONST/INPUT/VAR/DEP/UNKNOWN
data type 4 Bytes int32_t int8/uint8/fp32

⑤operator -- 12 Bytes

operator size type description
operator version 4 Bytes uint32_t The operator version.
operator type 4 Bytes uint32_t The type of this operator.
offset to the table of params 4 Bytes tm_uoffset_t The offset to the table of this operator.

⑥quantization parameters -- 12 Bytes

quantization parameters size type description
zero_point 4 Bytes int32_t
scale 4 Bytes float
width 4 Bytes int32_t

⑦buffer -- 8 Bytes

buffer size type description
size 4 Bytes tm_size_t The size of this buffer (bytes).
offset data 4 Bytes tm_uoffset_t The address of the buffer data.

Vectors

①vector offset -- 4+4N Bytes

vector offset size type description
v_num 4 Bytes tm_size_t The number of this vector.
offsets 4N Bytes tm_uoffset_t The address array in this vector.

②vector indices -- 4+4N Bytes

vector indices size type description
v_num 4 Bytes tm_size_t The number of this vector.
indices 4N Bytes tm_uoffset_t The indices array in this vector.

③vector dims -- 4+4N Bytes

vector dims size type description
v_num 4 Bytes tm_size_t The number of this vector.
dims 4N Bytes int32_t The dimension array.

④vector floats -- 4+4N Bytes

vector floats size type description
v_num 4 Bytes tm_size_t The number of this vector.
data 4N Bytes float

⑤vector anchors -- 4+4N Bytes

vector anchors size type description
v_num 4 Bytes tm_size_t The number of this vector.
data[0] [4] 16 Bytes float x0, y0, x1, y1

String

vector anchors size type description
size 4 Bytes tm_size_t The size of this string.
offset to the string 4 Bytes tm_uoffset_t The address of the string.

File Structure

TM_Header
Model name & TM_String(Optional)
-------------------------------Nodes-------------------------------
Node name & TM_String(Optional)
Vector(indices of input tensors)
Vector(indices of output tensors)
Operator param(Optional)
TM_Operator
TM_Node
……
Node name & TM_String(Optional)
Vector(indices of input tensors)
Vector(indices of output tensors)
Operator param(Optional)
TM_Operator
TM_Node
Vector(offsets of all nodes)
-------------------------------Tensors-------------------------------
Tensor name & TM_String(Optional)
Vector(tensor dims)
TM_Tensor
……
Tensor name & TM_String(Optional)
Vector(tensor dims)
TM_Tensor
Vector(offsets of all tensors)
-------------------------------Buffers------------------------------------
Buffer data(Optional)
TM_Buffer
……
Buffer data(Optional)
TM_Buffer
-------------------------------Subgraphs-------------------------------
Vector(offsets of all buffers)
Vector(indices of input nodes)
Vector(indices of output nodes)
TM_Subgraph
Vector(offsets of all subgraphs)
-------------------------------Root Table-------------------------------
TM_Model

Variables

Variables Variables
int8_t uint8_t 1 byte
int16_t uint16_t 2 bytes
int32_t uint32_t 4 bytes
int64_t uint64_t 8 bytes
float 4 bytes
typedef uint32_t tm_uoffset_t 4 bytes
typedef uint32_t tm_size_t 4 bytes
typedef uint8_t tm_bool_t 1 byte

Rules

Alignment

All tables, structures are aligned by the longest element in this table or structure, for example:

struct A
{
	int8_t  d1;	// 1-byte, aligned by 1-byte
	int32_t d3;	// 4-bytes, aligned by 4-bytes
	int16_t d2;	// 2-bytes, aligned by 2-bytes
	int64_t d4;     // 8-bytes, aligned by 8-bytes
}

Endian

Tengine tmfile uses little-endian.

Offset

All offsets and address are from the root address which is the original address of the file.

Operator parameters

All parameters are stored by node-> offset to the table of operator -> operator -> offset to the table of params -> params.

Clone this wiki locally