CN114492778B

CN114492778B - Operation method of neural network model, readable medium and electronic equipment

Info

Publication number: CN114492778B
Application number: CN202210141274.7A
Authority: CN
Inventors: 鲁若荻; 韩冥生; 余宗桥
Original assignee: ARM Technology China Co Ltd
Current assignee: ARM Technology China Co Ltd
Priority date: 2022-02-16
Filing date: 2022-02-16
Publication date: 2024-09-06
Anticipated expiration: 2042-02-16
Also published as: CN114492778A

Abstract

The application relates to the field of artificial intelligence, and discloses an operation method of a neural network model, a readable medium and electronic equipment. The method is applied to the electronic equipment and comprises the following steps: detecting a first neural network model, wherein the first neural network model is an asymmetrically quantized neural network model, and the first neural network model comprises asymmetric quantization parameters of operators of the first neural network model; converting the asymmetric quantization parameters of each operator into symmetric quantization parameters; and calling the operation logic of the preset symmetrical quantized operators by utilizing the symmetrical quantized parameters of each operator to obtain the reasoning result of the first neural network model. Therefore, the electronic equipment which can only run the symmetrical quantized neural network model can call the operation logic of the preset symmetrical quantized operator to realize the function of the asymmetrical quantized neural network model, and the types of the neural network model which can be run by the electronic equipment are increased.

Description

Operation method of neural network model, readable medium and electronic equipment

Technical Field

The application relates to the field of artificial intelligence, in particular to an operation method of a neural network model, a readable medium and electronic equipment.

Background

With the rapid development of artificial intelligence (ARTIFICIAL INTELLIGENCE, AI), neural network models are becoming more and more widely used in the field of artificial intelligence. Since the operation unit for operating the neural network model, such as the neural network processor (Neural-Network Processing Unit, NPU), is usually a fixed point operation unit, in order to increase the operation speed of the neural network model, each operator of the neural network model is usually quantized to obtain the neural network model for fixed point operation, and then the neural network model is operated by the electronic device. The quantization of each operator of the neural network model comprises asymmetric quantization or symmetric quantization, but in order to save development and manufacturing cost, only the operation logic of the operators aiming at symmetric quantization is preset in part of NPUs, and the NPUs can only operate the symmetric quantized neural network model and cannot operate the asymmetric quantized neural network model.

Disclosure of Invention

In view of this, embodiments of the present application provide a method for operating a neural network model, a readable medium, and an electronic device. The electronic equipment can call the operation logic of the preset symmetric quantization operator according to the obtained symmetric quantization parameter to realize the function of the asymmetric quantized neural network model by converting the asymmetric quantization parameter of the asymmetric quantized neural network model into the symmetric quantization parameter, so that the types of the neural network model capable of operating by the electronic equipment are increased, and the universality of the NPU of the electronic equipment is improved.

In a first aspect, an embodiment of the present application provides a method for operating a neural network model, which is applied to an electronic device, where the method includes: detecting a first neural network model, wherein the first neural network model is an asymmetrically quantized neural network model, and the first neural network model comprises asymmetric quantization parameters of operators of the first neural network model; converting the asymmetric quantization parameters of each operator into symmetric quantization parameters; and calling the operation logic of the preset symmetrical quantized operators by utilizing the symmetrical quantized parameters of each operator to obtain the reasoning result of the first neural network model.

According to the method provided by the embodiment of the application, if the processor (such as NPU) for running the first neural network model in the electronic equipment can only call the operation logic of the symmetrically quantized operators to realize the reasoning of the neural network model, the first neural network model can be reasoning by converting the asymmetric quantization parameters of each operator of the first neural network model into the symmetric quantization parameters and calling the operation logic of the symmetrically quantized operators through the obtained symmetric quantization parameters, so that the reasoning result of the first neural network model is obtained. Therefore, the types of the neural network models capable of running of the electronic equipment are increased, and the universality of the NPU of the electronic equipment is improved. In addition, in the process, an asymmetric quantized neural network model is not required to be converted into a floating-point type neural network model, and then converted into a symmetric quantized neural network model, so that the deployment speed of the neural network model is improved.

In a possible implementation of the first aspect, the electronic device includes a first processor, where the first processor is capable of reasoning about a symmetrically quantized neural network model and is not capable of reasoning about an asymmetrically quantized neural network model; and running, by the first processor, the first neural network model.

That is, the first processor of the electronic device can only call the operation logic of the symmetric quantized operator to infer the neural network model, by the method provided by the embodiment of the application, the first processor can realize the function of the first neural network model by converting the asymmetric quantization parameter of the first neural network model into the symmetric quantization parameter and calling the operation logic of the symmetric quantized operator based on the obtained symmetric quantization parameter, thereby increasing the types of the neural network model which can be operated by the first processor and improving the universality of the first processor.

In a possible implementation of the first aspect, the asymmetric quantization parameter includes at least one of the following parameters: the input data comprises asymmetric quantization parameters of the input data, wherein the asymmetric quantization parameters of the input data comprise asymmetric quantization scaling coefficients of the input data and asymmetric quantization zero points of the input data; the asymmetric quantization parameter of the output data comprises an asymmetric quantization scaling coefficient of the output data and an asymmetric quantization zero point of the output data; the asymmetric quantization parameter of the constant data comprises asymmetric quantization constant data, asymmetric quantization scaling factors of the asymmetric quantization constant data and asymmetric quantization zero points of the asymmetric quantization constant data.

In a possible implementation of the first aspect, the converting the asymmetric quantization parameter of each operator into a symmetric quantization parameter includes: determining the maximum value and the minimum value of floating point numbers corresponding to the input data or the output data according to the data type of the input data or the output data and the asymmetric quantization parameters of the input data or the output data; and determining the symmetrical quantization scaling coefficient of the input data or the output data according to the maximum value and the minimum value of the floating point number corresponding to the input data or the output data.

For example, when the data type of the input data or the output data of one operator is UINT8, if the asymmetric quantization scaling factor of the input data or the output data is 1 and the zero point is 0, it is determined that the maximum value of the corresponding floating point number of the input data or the output data is 255, the minimum value is 0, and further, the symmetric quantization scaling factor of the input data or the output data is 0.498 according to the following formula (2).

In one possible implementation manner of the first aspect, determining a maximum value and a minimum value of a floating point number corresponding to the input data or the output data according to the data type of the input data or the output data, and the asymmetric quantization parameter of the input data or the output data includes: determining the maximum value and the minimum value of the fixed point number of the input data or the output data according to the data type of the input data or the output data of each operator; and determining the maximum value and the minimum value of the floating point number corresponding to the input data or the output data according to the asymmetric quantization parameter of the input data or the output data and the maximum value and the minimum value of the fixed point number of the input data or the output data.

For example, when the data type of the input data or the output data is UINT8, the maximum value of the fixed point number of the input data or the output data is 255 and the minimum value is 0, so that it can be determined that the floating point number corresponding to the input data or the output data has a value range of the floating point number [0, 255], that is, the maximum value of the floating point number corresponding to the input data or the output data is the floating point number 255 and the minimum value is the floating point number 0 according to the following formula (11).

In a possible implementation of the first aspect, the asymmetric quantization constant data includes an asymmetric quantization constant, an asymmetric quantization matrix; and, converting the asymmetric quantization parameter of each operator into a symmetric quantization parameter, comprising: determining the maximum value and the minimum value of the floating point number corresponding to the asymmetric quantized constant data according to the data type of the fixed point number of the asymmetric quantized constant data, the asymmetric quantized scaling coefficient of the asymmetric quantized constant data and the asymmetric quantization zero point of the asymmetric quantized constant data; determining a symmetric quantization scaling coefficient of the floating point number corresponding to the asymmetric quantization constant data according to the maximum value and the minimum value of the floating point number corresponding to the asymmetric quantization constant data; and converting floating point number constant data corresponding to the asymmetric quantized constant data into symmetric quantized constant data according to the determined symmetric quantized scaling coefficient of the asymmetric quantized constant data, wherein the floating point number constant data corresponding to the asymmetric quantized constant data is determined by asymmetric quantization parameters of the constant data.

For example, the constant data of a certain operator includes an asymmetric quantized constant 100, where the asymmetric quantized scaling factor of the asymmetric quantized constant data is 2, and the asymmetric quantized zero point is 0, and then according to formula (11), the floating point number constant corresponding to the constant is 50; based on the data type of the constant data, for example UINT8, the maximum value of the floating point number corresponding to the constant data is 127.5, the minimum value is 0, and further the symmetric quantization scaling factor of the constant data is 0.9961 according to the following formula (2).

In one possible implementation manner of the first aspect, determining the maximum value and the minimum value of the floating point number corresponding to the asymmetric quantization constant data according to the data type of the fixed point number of the asymmetric quantization constant data, the asymmetric quantization scaling coefficient of the asymmetric quantization constant data, and the asymmetric quantization zero point of the asymmetric quantization constant data includes: determining the maximum value and the minimum value of the fixed point number of the asymmetric quantization constant data according to the data type of the asymmetric quantization constant data; and determining the maximum value and the minimum value of the floating point number corresponding to the asymmetric quantization constant data according to the asymmetric quantization scaling coefficient of the asymmetric quantization constant data, the asymmetric quantization zero point of the asymmetric quantization constant data and the determined maximum value and the minimum value of the fixed point number of the asymmetric quantization constant data.

For example, in the case where the data type of the asymmetric quantization constant data is UINT8, if the asymmetric quantization scaling factor of the asymmetric quantization constant data is 2 and the asymmetric quantization zero point is 0, the fixed point number of the asymmetric quantization constant data has a maximum value of 255 and a minimum value of 0, so that it can be determined that the floating point number corresponding to the asymmetric quantization constant data has a value range of floating point number [0, 255], that is, the floating point number asymmetric quantization constant data has a maximum value of 255 and a minimum value of 0.

In one possible implementation manner of the first aspect, the asymmetric quantization constant data further includes an asymmetric quantization lookup table, where the asymmetric quantization lookup table includes an asymmetric quantization lookup table index and an asymmetric quantization lookup table result corresponding to each asymmetric quantization lookup table index; and, converting the asymmetric quantization parameter of each operator into a symmetric quantization parameter, comprising: determining floating point number table look-up indexes corresponding to the table look-up indexes of each asymmetric quantization according to the asymmetric quantization scaling coefficient of the asymmetric quantization constant data and the asymmetric quantization zero point of the asymmetric quantization constant data; determining floating point number table look-up results corresponding to each floating point number table look-up index according to the floating point number table look-up index corresponding to each asymmetric quantized table look-up index and the operation logic of each operator; obtaining a symmetric quantization table lookup index according to the symmetric quantization scaling coefficient of the floating point table lookup index, and obtaining a symmetric quantization table lookup result according to the symmetric quantization scaling coefficient of the floating point data table lookup result, wherein the symmetric quantization scaling coefficient of the floating point table lookup index is determined based on the data type of the asymmetric quantization table lookup index, and the symmetric quantization scaling coefficient of the floating point table lookup result is determined based on the data type of the asymmetric quantization table lookup result; and obtaining a symmetrical quantization lookup table based on the lookup table indexes of each symmetrical quantization and the lookup table results of the corresponding symmetrical quantization.

For example, for the Softmax operator below, the electronic device may determine the floating point number table look-up index corresponding to each table look-up index according to the data type (UINT 8) of the table look-up index in the table LUT, the asymmetric quantization scaling factor (2.2x10 ^-5), the asymmetric quantization zero point (0), for example, for table look-up index [59, 104, 182], the floating point number table look-up index may be obtained according to formula (11) [2681818, 4727273, 8272727]; and then substituting the floating point number table index into the operation logic (formula (12)) of the Softmax operator to obtain a floating point number table result [0.0069,0.042,0.9511] corresponding to the floating point number table index, and quantizing the floating point number table result into fixed point number table results [1,5, 121] according to the symmetric quantization scaling coefficient (1.1X10 ^-5) of the floating point number table result.

In a possible implementation of the first aspect, the method further includes: determining the maximum value and the minimum value of fixed points corresponding to the asymmetric quantization table index or the asymmetric quantization table result according to the data type of the asymmetric quantization table index or the asymmetric quantization table result, and determining the maximum value and the minimum value of symbol point table index corresponding to the asymmetric quantization table index or the maximum value and the minimum value of symbol point table result corresponding to the asymmetric quantization table result according to the asymmetric quantization scaling coefficient of the asymmetric quantization constant data and the asymmetric quantization zero point of the asymmetric quantization constant data based on the determined maximum value and the determined minimum value; and determining the symmetric quantization scaling coefficient of the floating point number table index or the floating point number table result according to the determined maximum value and minimum value of the symbol number table index corresponding to the asymmetric quantization table index or the maximum value and minimum value of the symbol number table result corresponding to the asymmetric quantization table result.

For example, in the case where the data types of the asymmetric quantization table look-up result and the asymmetric quantization table look-up result of the Softmax operator are UINT8, the number of the asymmetric quantization table look-up results is 255 at the maximum value and 1 at the minimum value, the asymmetric quantization scale coefficient (2.2x10 ^-5) of the asymmetric quantization table look-up index, the asymmetric quantization zero point (0) of the asymmetric quantization table look-up index, the asymmetric quantization scale coefficient (255) of the asymmetric quantization table look-up result, and the asymmetric quantization zero point (0) of the asymmetric quantization table look-up result are obtained, the maximum value of the floating point number corresponding to the asymmetric quantization table look-up index is 11590909, the minimum value is 0, the maximum value of the floating point number corresponding to the asymmetric quantization table look-up result is 1, the minimum value is 0, and the symmetric quantization coefficient of the floating point number index is 1.1x ^-5 and the symmetric quantization scale coefficient of the floating table look-up result is 127 based on the following formula (2).

In a second aspect, an embodiment of the present application provides a readable medium, where the readable medium contains instructions, which when executed by a processor of an electronic device, cause the electronic device to implement the above first aspect and any one of the operation methods provided by the various possible implementations of the first aspect.

In a third aspect, an embodiment of the present application provides an electronic device, including: a memory for storing instructions for execution by one or more processors of the electronic device; and a processor, which is one of the processors of the electronic device, for executing instructions to cause the electronic device to implement the above first aspect and any one of the operation methods of the neural network model provided by the various possible implementations of the above first aspect.

Drawings

FIG. 1A illustrates a schematic diagram of 8-bit symmetric quantization, according to some embodiments of the application;

FIG. 1B illustrates a schematic diagram of an 8-bit asymmetric quantization, according to some embodiments of the application;

FIG. 2 illustrates a schematic view of a scenario in which an asymmetrically quantized neural network model is deployed into an electronic device 100, according to some embodiments of the application;

FIG. 3 illustrates a scene graph of an electronic device 100 running an asymmetrically quantized neural network model, according to some embodiments of the application;

FIG. 4 illustrates a schematic diagram of the structure of a neural network model 10, according to some embodiments of the application;

FIG. 5 illustrates a schematic diagram of an electronic device classifying an image 20 using an asymmetrically quantized neural network model 10, according to some embodiments of the application;

FIG. 6 illustrates a flow diagram of a neural network model operation method, according to some embodiments of the application;

FIG. 7 illustrates a schematic diagram of a process by which electronic device 100 invokes the arithmetic logic of the symmetric quantization operator to classify image 20, according to some embodiments of the application;

Fig. 8 illustrates a schematic diagram of an electronic device 100, according to some embodiments of the application.

Detailed Description

Illustrative embodiments of the application include, but are not limited to, methods of operating neural network models, readable media, and electronic devices.

For ease of understanding, the terms involved in the embodiments of the present application will first be described.

(1) Symmetric quantization

Symmetric quantization, i.e., converting floating point numbers into signed shaped numbers (integral numeric types, INT) with a range of values [ -2 ^n-1,2^n-1 -1], where n is the number of bits of symmetric quantization. Assuming that the floating point number to be quantized is x _f, the quantization target is that x _f is subjected to n-bit symmetric quantization, namely the value range of the quantized fixed point number is that the symmetric quantization process is expressed as the following formula (1).

In the formula (1), abs () is an absolute value function, max () is a maximum value function, min () is a minimum value function, round is a rounding function, and x _q is a fixed point number. Further, in equation (1), the term S _c shown in equation (2) below may be referred to as a symmetric quantization scale factor. That is, for a fixed number of symmetric quantization, the floating point number corresponding to the fixed point number may be determined according to the symmetric quantization scaling factor corresponding to the fixed point number.

In particular, FIG. 1A illustrates a schematic diagram of 8-bit symmetric quantization of floating point numbers x _f, according to some embodiments of the application. Referring to fig. 1A, the quantization is aimed at 8-bit symmetric quantization of x _f, that is, converting x _f into INT8 type (the range of values is [ -128, 127 ]), and assuming that the maximum value of the absolute value of x _f is max (|x _f |), the process of 8-bit symmetric quantization of x _f is to map the interval [ -max (|x _f|),max(|x_f |) into the interval [ -128, 127 ].

(2) Asymmetric quantization

Asymmetric quantization, i.e., converting floating point numbers into unsigned shaped (unsigned integral numeric types, UINT) numbers with a range of values of [0,2 ⁿ -1], where n is the number of bits of asymmetric quantization. Assuming that the floating point number to be quantized is x _f, the quantization target is to perform n-bit asymmetric quantization on x _f, that is, the value range of the quantized fixed point number is [0,2 ⁿ -1], the asymmetric quantization process is expressed as the following formula (3).

In the formula (3), max () is a maximum function, min () is a minimum function, round is a rounding function, and x _q is a fixed point number. Further, in formula (3), the term AS _c shown in formula (4) below may be referred to AS an asymmetric quantization scale factor, and the term Z _p shown in formula (5) below may be referred to AS an asymmetric quantization zero. That is, for a fixed point number that is asymmetrically quantized, a floating point number corresponding to the fixed point number may be determined according to an asymmetric quantization scaling factor and an asymmetric quantization zero point corresponding to the fixed point number.

Z_p＝round(-AS_c·min(x_f)) (5)

Based on formulas (3) to (5), another expression of asymmetric quantization shown in formula (6) can be obtained:

x_q＝round(x_f·AS_c+Z_p) (6)

In particular, FIG. 1B illustrates a schematic diagram of 8-bit asymmetric quantization of floating point number x _f, according to some embodiments of the application. Referring to fig. 1B, the quantization is performed for 8-bit asymmetric quantization of x _f, that is, x _f is converted into UINT8 type (the range of values is [0, 255 ]), and if x _f has a maximum value of max (x _f) and a minimum value of min (x _f), the process of 8-bit asymmetric quantization of x _f is to map the interval [ min (x _f),max(x_f) ] into the interval [0, 255 ].

It will be appreciated that the UINT type may be represented as UINTn in some embodiments, where n may take 4, 8, 16, 32, etc., other integers may be taken, and the unsigned data of type UINTn may range in value from 0,2 ⁿ -1. That is, for a given UINT type data, the electronic device may determine the range of values for the type of data according to the value of n, and calculate the range of values for the floating point number corresponding to the data based on equation (6).

(3) Neural network model quantization

The quantization of the neural network model is to convert input data, output data and constant data in each operator of the neural network model from floating point number of large data type (for example, 32-bit floating point number) to fixed point number of smaller data type (for example, 4/8/16-bit fixed point number), and the number of bits of the fixed point number is usually matched with the number of bits of the fixed point number supported by an operation unit running the neural network model, such as an NPU, so as to improve the speed of the NPU running the neural network model.

In general, the quantization process of the neural network model is to determine quantization parameters of floating point number input data, floating point number output data and floating point number constant data of each operator according to the value range of input data of each operator floating point number, output data of the floating point number and type of quantized fixed point number (for example UINT 8) (for example, the quantization parameters of asymmetric quantization include asymmetric quantization scaling coefficients and asymmetric quantization zero points, and the quantization parameters of symmetric quantization include symmetric quantization scaling coefficients). That is, the quantized parameters of the operator include the quantized parameters of the input data, the quantized parameters of the output data, the quantized constant data, and the quantized parameters of the constant data of the operator. In the electronic device for running the quantized neural network model, the operation logic of the quantized operator is preset. The quantized operator arithmetic logic takes fixed point number as input and fixed point number as output, and parameters in the arithmetic logic comprise quantization parameters of input data, quantization parameters of output data, quantized constant data and quantization parameters of constant data, when the electronic equipment runs a quantized operator, the electronic equipment invokes preset arithmetic logic according to the quantization parameters of the operator, and the function of the operator can be realized through fixed point operation.

The following describes the technical scheme of the embodiment of the application with reference to the attached drawings.

For ease of understanding, the following description will be given of the operation logic using operators for symmetric quantization and asymmetric quantization, taking convolution operators as an example.

Assuming that b_f is a floating-point number input matrix, c_f is a floating-point number convolution kernel, and d_f is a floating-point number convolution result, the relationship among b_f, c_f, d_f can be expressed as formula (7).

D_f＝B_f*C_f (7)

In the formula (7), "" is a convolution operation symbol, and a specific process of convolution operation will be described later, which will not be described herein.

From equation (1) and equation (2), b_f=b_q/b_sc, c_f=c_q/c_sc, d_f=d_q/d_sc can be found, wherein: b_q is a symmetric quantized fixed-point number matrix corresponding to b_f, and b_sc is a symmetric quantized scaling factor that quantizes b_f to b_q; c_q is a fixed-point number convolution kernel of symmetric quantization corresponding to C_f, and C_Sc is a symmetric quantization scaling coefficient for quantizing C_f to C_q; d_q is a symmetric quantized fixed-point number matrix corresponding to d_f, and d_sc is a symmetric quantization scaling factor that symmetrically quantizes d_f to d_q. Further, the expression (7) can be expressed as the following expression (8).

The modification of equation (8) results in the arithmetic logic of the symmetrically quantized convolution operator shown in equation (9) below.

That is, the electronic device is preset with an operation logic shown in the formula (9), and inputs of the operation logic include a fixed-point input matrix b_q, a fixed-point convolution kernel c_q, a symmetric quantization parameter b_sc of the fixed-point input data b_q, a symmetric quantization parameter d_sc of the fixed-point output data d_q, and a symmetric quantization parameter c_sc of the convolution kernel, and the output is a fixed-point convolution result d_q.

Similarly, the operation logic of the asymmetrically quantized convolution operator may be expressed as the following formula (10), and the specific derivation process may refer to the operation logic of the symmetrically quantized convolution operator, which is not described herein.

In the formula (10), Z _p _b is an asymmetric quantization zero point of the input data b_q, Z _p _c is an asymmetric quantization zero point of the convolution kernel c_q, Z _p _d is an asymmetric quantization zero point of the output data d_q, b_ ASc is an asymmetric quantization scaling factor of the input data b_q, c_ ASc is an asymmetric quantization scaling factor of the convolution kernel c_q, and d_ ASc is an asymmetric quantization scaling factor of the output data d_q.

As can be seen from equation (9) and equation (10), the arithmetic logic and input parameters for the same operator, using asymmetric quantization and symmetric quantization, are not the same. As previously described, some NPUs can only run symmetrically quantized neural network models, i.e., can only run the arithmetic logic of symmetrically quantized operators (e.g., the arithmetic logic of convolution operators shown in equation (9)), and cannot run asymmetrically quantized neural network models. If the asymmetric quantized neural network model is deployed in the electronic device including the NPU, the asymmetric quantized neural network model needs to be converted into the floating point type neural network model (i.e., the input data, the output data and the constant data of each operator of the neural network model are all floating point numbers), and then the floating point type neural network model is quantized into the symmetric quantized neural network before being deployed in the electronic device.

For example, referring to fig. 2, the NPU of the electronic device 100 can only run a symmetrically quantized neural network model, while the neural network model to be run is an asymmetrically quantized neural network model. Therefore, the electronic device 200 needs to convert the asymmetrically quantized neural network model into a floating-point neural network model, quantize the floating-point neural network model into a symmetrically quantized neural network model, and then deploy the symmetrically quantized neural network model into the electronic device 100, and the NPU of the electronic device 100 operates the symmetrically quantized neural network model. Because the floating point number neural network model is quantized into the fixed point number neural network model (i.e. input data, output data, constant data and the like of each operator of the neural network model are fixed point numbers), a large amount of calculation resources are occupied, the time consumption is long, and the quick deployment of the neural network model is not facilitated.

In order to solve the above-mentioned problems, an embodiment of the present application provides an operation method of a neural network model, after an electronic device 100 detects an asymmetrically quantized neural network model, an asymmetric quantization parameter of each operator of the asymmetrically quantized neural network model is obtained, and the asymmetric quantization parameter of each operator is converted into a corresponding symmetric quantization parameter, and an NPU of the electronic device 100 invokes an operation logic of a preset symmetrically quantized operator according to the symmetric quantization parameter of each operator to implement a related function of the asymmetrically quantized neural network model. Thus, referring to fig. 3, although the NPU of the electronic device 100 cannot operate the asymmetrically quantized neural network model, the electronic device 100 converts the asymmetrically quantized parameters of each operator in the asymmetrically quantized neural network model into the symmetrically quantized parameters, and the NPU of the electronic device 100 can call the operation logic of the preset symmetrically quantized operator to implement the related function of the asymmetrically quantized neural network model, without converting the asymmetrically quantized neural network model into a floating point type neural network model by other electronic devices, converting the floating point type neural network model into the symmetrically quantized neural network model, and then operating the NPU of the electronic device 100, thereby increasing the types of the neural network models that the electronic device 100 can operate, improving the universality of the NPU, and improving the deployment speed of the neural network model.

It will be appreciated that the asymmetric quantization parameter for each operator includes at least one of the following: an asymmetric quantization scaling factor of input data, an asymmetric quantization zero of input data, an asymmetric quantization scaling factor of output data, an asymmetric quantization zero of output data, an asymmetric quantization constant data, an asymmetric quantization zero of asymmetric quantization constant data, an asymmetric quantization scaling factor of asymmetric quantization constant data. The symmetric quantization parameter for each operator includes at least one of the following parameters: symmetric quantization scaling coefficients of input data, symmetric quantization scaling coefficients of output data, symmetric quantization constant data, symmetric quantization scaling coefficients of symmetric quantization constant data.

It will be appreciated that in other embodiments, the asymmetric quantization parameter/symmetric quantization parameter of each operator may include more or less parameters, and is not limited herein.

It will be appreciated that the preset arithmetic logic of the symmetrically quantized operator may be preset in the NPU by the developer of the NPU of the electronic device 100, or may be provided to the developer of the electronic device 100 by the developer of the NPU, and preset in the memory of the electronic device 100 by the developer of the electronic device 100. The NPU of the electronic device 100 may call the operation logic of the symmetric quantized operator according to the symmetric quantization parameter of the symmetric quantized input data and the symmetric quantized input data based on the operator, the symmetric quantization parameter of the output data, and the symmetric quantization constant data, to obtain symmetric quantized fixed-point number output data of the operator.

Specifically, in some embodiments, the transformation of the above equation (6) may result in a calculation equation that converts the asymmetrically quantized fixed-point number into a floating-point number as shown in equation (11).

The electronic device 100 may determine a value range of fixed-point number input data according to a data type of fixed-point number input data of the asymmetric quantization operator (for example, if the input data is UINT8, which indicates that the value range is fixed-point number [0, 255], maximum value is 255, and minimum value is 0), determine a maximum value and a minimum value of a floating point number corresponding to the input data based on the maximum value and the minimum value of the asymmetric quantized input data, and the asymmetric quantization scaling factor and the asymmetric quantization zero point of the asymmetric quantized input data, and based on the formula (11); and determining symmetrical quantization scaling coefficients of the input data according to the formula (2) based on the maximum value and the minimum value of the floating point number corresponding to the input data.

Similarly, the electronic device 100 may determine a value range of fixed-point number output data according to the type of output data of the asymmetrically quantized operator (for example, if the input data is UINT8, which indicates that the value range is fixed-point number [0, 255], the maximum value is 255, and the minimum value is 0), and determine a maximum value and a minimum value of a floating point number corresponding to the output data based on the formula (11) according to the maximum value and the minimum value of the asymmetrically quantized output data, the asymmetrically quantized scaling coefficient and the asymmetrically quantized zero point of the output data; and determining symmetric quantization scaling factors of the output data according to formula (2) based on the determined maximum and minimum values of the floating point number.

Similarly, the electronic device 100 may also determine, according to the data type of the asymmetric quantization constant data in the asymmetric quantization operator, a maximum value and a minimum value of a fixed point number of the asymmetric quantization constant data, and determine, according to the formula (11), the maximum value and the minimum value of the floating point number corresponding to the asymmetric quantization constant data based on the determined maximum value and minimum value, the asymmetric quantization scaling coefficient and the asymmetric quantization zero point of the asymmetric quantization constant data; based on the determined maximum value and minimum value of the floating point number, determining a symmetrical quantization scaling factor of constant data according to a formula (2); then, according to the formula (11), converting the floating-point number corresponding to the asymmetric quantization constant data, and then, based on the determined symmetric quantization scaling factor and the floating-point number constant data corresponding to the asymmetric quantization constant data, converting the floating-point number constant data corresponding to the asymmetric quantization constant data into symmetric quantization constant data according to the formula (1) and the formula (2).

For example, assume that the range of values of each element in the input data matrix b_f of an asymmetrically quantized convolution operator quantized to UINT8 is a floating point number [0,1], a floating point number convolution kernelThe range of values of the elements in the floating-point number convolution result D _ f is found to be 0,0.7.

According to the formula (4) and the formula (5), the asymmetric quantization zero Z _p _b=0, the asymmetric quantization scaling factor b_ ASc =255, the asymmetric quantization zero Z _p _d=0, and the asymmetric quantization scaling factor d_ ASc = 364.29 of the output data can be obtained. Assuming that the quantization parameter of the convolution kernel c_f is the same as the asymmetric quantization parameter of the input data b_f (i.e., c_ ASc =255, z _p _c=0), it can be derived according to equation (6) That is, the asymmetric quantization parameters of the asymmetric quantized convolution operator include: an asymmetric quantization zero Z _p _b, an asymmetric quantization scaling coefficient b_ ASc of the input data, an asymmetric quantization zero Z _p _d, an asymmetric quantization scaling coefficient d_ ASc, an asymmetric quantization constant data c_q, and an asymmetric quantization scaling coefficient c_ ASc, an asymmetric quantization zero Z _p _c of the asymmetric quantization constant data of the output data.

When the electronic equipment detects the asymmetrically quantized volume integrating algorithm, firstly, determining that the value range of fixed point number input data D_q is [0, 255], the maximum value is 255 and the minimum value is 0 according to the data type UINT8 of the input data; substituting x _q＝255、Z_p _b=0 and b_ ASc =255 into the formula (11) to obtain a floating point number maximum value corresponding to the input data of 1; substituting x _q＝0、Z_p _b=0 and b_ ASc =255 into formula (11) to obtain a floating point number minimum value corresponding to the input data is 0, and further obtaining a symmetric quantization scaling coefficient b_sc= (2 ^8-1 -1)/1=127 of the input data according to formula (2).

Similarly, the electronic device may determine, according to the data type UINT8 of the output data, that the value range of the fixed point number output data d_q is [0, 255], the maximum value is 255, and the minimum value is 0; substituting x _q＝255、Z_p _d=0 and d_ ASc = 364.29 into formula (11) to obtain a floating point number maximum value corresponding to output data of (255-0)/364.29 =0.7; substituting x _q＝0、Z_p _d=0 and d_ ASc = 364.29 into formula (11) to obtain the floating point number minimum value corresponding to the output data as (0-0)/364.29 =0, and further obtaining the symmetric quantization scaling factor d_sc= (2 ^8-1 -1)/0.7= 181.43 of the output data according to formula (2).

Since the asymmetric quantization parameter of the convolution kernel c_q is the same as the asymmetric quantization parameter of the input data b_q, the symmetric quantization parameter of the convolution kernel c_q should also be the same as the symmetric quantization parameter of the input data b_q, i.e. the symmetric quantization scaling factor c_sc=127 of the convolution kernel c_q. Substituting c_ ASc =255, Z _p _c=0, and c_q into formula (11) can result inFrom c_f and c_sc=127, equation (1) and equation (2) a symmetrically quantized convolution kernel can be obtained

Furthermore, the NPU of the electronic device 100 may call the operation logic of the symmetric quantized convolution operator shown in the preset formula (9) to implement the function of the asymmetric quantized convolution operator according to the b_sc=127, d_sc= 181.43, and c_sc=127 and the fixed point number input data b_q input to the convolution operator, thereby increasing the types of neural network models that the NPU of the electronic device 100 can operate and improving the versatility of the NPU.

The technical scheme of the embodiment of the application is described below with reference to a specific neural network model.

Fig. 4 illustrates a schematic diagram of the structure of a neural network model 10, according to some embodiments of the application. As shown in fig. 4, the neural network model 10 is an asymmetrically quantized neural network model, and includes an input layer 11, a convolution operator 12, a full join operator 13, a Softmax operator 14, and an output layer 15 for classifying input image data. Wherein the input layer 11 is used for preprocessing an input image, and converting the input image data into asymmetrically quantized input data, such as UINT8 type data; the convolution operator 12 is used for carrying out convolution operation on asymmetrically quantized input data to obtain an asymmetrically quantized feature matrix corresponding to an input image; the full-connection operator 13 is used for performing full-connection operation on the feature matrix of the asymmetric quantization to obtain the score of each preset category of the input image data; the Softmax operator 14 is configured to obtain a probability that the input image data belongs to each preset category according to the score that the input image data belongs to each preset category; the output layer 15 is configured to determine a category of the input image data according to a probability that the input image data belongs to each preset category, for example, a category with a highest probability that the input image data is each preset category is used as the category of the input image data.

Further, FIG. 5 shows a schematic diagram of a process for classifying an image 20 by an electronic device using the neural network model 10, according to an embodiment of the present application.

Referring to fig. 5, the electronic device first pre-processes the image 20 by using the input layer 11 to obtain an image matrix H of UINT8, wherein the asymmetric quantization scaling coefficient of the input layer 11 is ASc _out1=1, and the asymmetric quantization zero is Z _p _out1=0.

Next, the convolution operator 12 is used to perform convolution operation on the image matrix H and the convolution kernels K _i (i=1, 2, 3) respectively, so as to obtain three feature matrices a _i (i=1, 2, 3), wherein an asymmetric quantization scaling factor of input data of the convolution operator 12 is ASc _in2=1, an asymmetric quantization zero Z _p _in2=0 of the input data, an asymmetric quantization scaling factor of output data is ASc _out2=0.0833, an asymmetric quantization zero of the output data is Z _p _out2=0, and asymmetric quantization constant data includes a convolution kernel K _i (i=1, 2, 3) (the asymmetric quantization parameter of the convolution kernel is the same as that of the input data of the convolution operator 12).

And performing full-connection operation on the feature matrix a _i (i=1, 2, 3) by using the full-connection operator 13, for example, performing inner product operation on the feature matrix a _i (i=1, 2, 3) and the weight matrix W to obtain a score of the image 20 in a preset category (rabbit/dog/cat), wherein an asymmetric quantization scaling coefficient ASc _in3= 0.08333 of input data of the full-connection operator 13, an asymmetric quantization zero Z _p _in3=0 of the input data, an asymmetric quantization scaling coefficient ASc _out3=2.2x10: 10 ^-5 of the output data, an asymmetric quantization zero of the output data is Z _p _out3=0, and the asymmetric quantization constant data comprises a weight matrix W (an asymmetric quantization parameter of the weight matrix is the same as an asymmetric quantization parameter of the input data of the full-connection operator 13).

Then, according to the score of the preset class of the image 20, the probability of the preset class of the image 20 is obtained from the lookup table LUT by using the Softmax operator 14, wherein the asymmetric quantization scaling factor ASc _in4=2.2x10 ^-5 of the input data of the Softmax operator 14, the asymmetric quantization zero point Z _p _in4=0 of the input data, the asymmetric quantization scaling factor of the output data is ASc _out4=255, the asymmetric quantization zero point of the output data is Z _p _out4=0, and the asymmetric quantization constant data comprises the lookup table LUT.

Finally, the output layer 15 is utilized to determine the category of the image 20 according to the probability that the image 20 is in each preset category, for example, the probability that the image 20 is in each preset category is compared, and the highest probability of the probabilities that the image 20 is in the preset category is determined as the category (cat) of the image 20.

The following describes a technical solution of an embodiment of the present application in connection with the structure of the neural network model 10 shown in fig. 4 and the process of classifying the image 20 by the neural network model 10 shown in fig. 5.

Specifically, fig. 6 illustrates a flow diagram of a method of operation of a neural network model, according to some embodiments of the application. The method is performed by the electronic device 100, and as shown in fig. 6, the process includes the following steps.

S601: an asymmetrically quantized neural network model is detected.

The electronic device 100 triggers the operation method of the neural network model provided by the embodiment of the present application when detecting that the neural network model to be operated is an asymmetrically quantized neural network model.

In some embodiments, the electronic device 100 may determine whether the neural network is an asymmetrically quantized neural network model based on the data type of the data in the neural network model to be run. Specifically, the electronic device 100 determines that the neural network model to be operated is an asymmetrically quantized neural network model when detecting that the data type of the data in the neural network model to be operated is UINT (UINT includes, but is not limited to UINT4, UINT8, UINT16, UINT32, etc.), for example, when detecting that the data type of the data in the neural network model 10 is UINT 8.

In some embodiments, the electronic device 100 may also determine whether the neural network model to be run is an asymmetrically quantized neural network model according to quantization parameters of operators of the neural network model to be run. For example, the electronic device 100 may determine that the neural network model is an asymmetrically quantized neural network model upon detecting that the quantization parameters of the neural network model to be run include scaling coefficients and zero points.

It can be appreciated that, in other embodiments, the electronic device 100 may also determine whether the neural network model to be operated is an asymmetrically quantized neural network model in other manners, and trigger the operation method of the neural network model provided in the embodiments of the present application if it is detected that the neural network model to be operated is an asymmetrically quantized neural network model.

S602: and acquiring asymmetric quantization parameters of input data/output data of each operator, and converting the asymmetric quantization parameters into symmetric quantization parameters.

That is, the electronic device 100 sequentially acquires the input data and the asymmetric quantization parameter of the output data of each operator, converts the asymmetric quantization parameter of the input data (the asymmetric quantization scaling coefficient of the input data, the asymmetric quantization zero point of the input data) into the symmetric quantization parameter of the input data (the symmetric quantization scaling coefficient of the input data), and converts the asymmetric quantization parameter of the output data (the asymmetric quantization scaling coefficient of the output data, the asymmetric quantization zero point of the output data) into the symmetric quantization parameter of the output data (the symmetric quantization scaling coefficient of the output data).

For example, for the neural network model 10 shown in fig. 4, the asymmetric quantization parameters of the input layer 11 include: the asymmetric quantization scaling factor ASc _out1=1 of the output data, and the asymmetric quantization zero zp_out1=0 of the output data. Since the value range corresponding to UINT8 is [0, 255], the maximum value of the floating point number corresponding to the output data can be obtained according to formula (11) to be the floating point number (255-0)/1=255, and the minimum value is the floating point number (0-0)/1=0, and the electronic device 100 can determine the symmetric quantization scaling coefficient sc_out1= (2 ⁷ -1)/255=0.498 of the output data of the input layer 11 based on formula (2).

For another example, for the neural network model 10 shown in fig. 4, the asymmetric quantization parameters of the convolution operator 12 include: the asymmetric quantization scaling factor ASc _in2=2 for the input data, the asymmetric quantization zero zp_in2=0 for the input data, the asymmetric quantization scaling factor ASc _out2=0.0833 for the output data, the asymmetric quantization zero zp_out2=0 for the input data. Since UINT8 corresponds to the value range of [0, 255], the maximum value of the floating point number corresponding to the input data is floating point number (255-0)/1=255, the minimum value is floating point number (0-0)/1=0, the maximum value of the floating point number corresponding to the output data is floating point number (255-0)/0.0833=3061, and the minimum value is floating point number (0-0)/1=0 according to the formula (11). The electronic device 100 may determine the symmetric quantization scaling factor sc_in2= (2 ⁷ -1)/255=0.498 for the input data and the symmetric quantization scaling factor sc_out2= (2 ⁷ -1)/3061=0.0417 for the output data of the convolution operator 12 based on equation (2). Similarly, the symmetric quantization scaling coefficient sc_in3=0.0417 of the input data and the symmetric quantization scaling coefficient sc_out3=1.1×10 ^-5 of the output data of the full join operator 13 can be obtained.

For another example, for the neural network model 10 shown in fig. 4, the asymmetric quantization parameters of the softmax operator 14 include: the asymmetric quantization scaling factor ASc _in4=2.2x ^-5 of the input data, the asymmetric quantization zero zp_in4=0 of the input data, the asymmetric quantization scaling factor ASc _out4=255 of the output data, and the asymmetric quantization zero zp_out4=0 of the output data. Since UINT8 corresponds to the value range of [0, 255], the maximum value of the floating point number corresponding to the input data is floating point number (255-0)/(2.2x10 ^-5) = 11590909, the minimum value is floating point number (0-0)/(2.2x10 ^-5) =0, the maximum value of the floating point number corresponding to the output data is floating point number (255-0)/255=1, and the minimum value is floating point number (0-0)/255=0 according to the formula (11). The electronic device 100 may determine the symmetric quantization scaling coefficient sc_in4= (2 ⁷-1)/11590909＝1.1×10^-5, the symmetric quantization scaling coefficient sc_out4= (2 ⁷ -1)/1=127 of the input data of the Softmax operator 14 based on equation (2).

S603: it is determined whether the current operator includes asymmetric quantization constant data.

The electronic device 100 determines whether the current operator includes asymmetric quantized constant data, and if so, proceeds to step S604, where it is indicated that the asymmetric quantized constant data needs to be converted into symmetric quantized constant data; otherwise, it is explained that the asymmetric quantization constant data does not need to be converted into symmetric quantization constant data, and the process goes to step S605.

For example, with the neural network model 10, when the current operator is the input layer 11 or the output layer 15, the current operator does not have asymmetric quantization constant data, and the process goes to step S605; for the neural network model 10, the convolution operator 12 has the asymmetric quantized constant data convolution kernel K _i (i=1, 2, 3), the full join operator 13 has the asymmetric quantized constant data weight matrix W, the Softmax operator 14 has the asymmetric quantized constant data look-up table LUT, and when the current operator is the convolution operator 12, the full join operator 13 or the Softmax operator 14, the electronic device 100 may determine that the current operator includes the asymmetric quantized constant data, and go to step S604.

S604: and converting the asymmetric quantization constant data into symmetric quantization constant data according to the asymmetric quantization parameter of the asymmetric quantization constant data.

Under the condition that asymmetric quantization constant data exist in a current operator, the electronic device 100 determines the maximum value and the minimum value of the fixed point number of the asymmetric quantization constant data according to the asymmetric quantization parameter of the asymmetric quantization constant data of the current operator and the data type of the fixed point number of the asymmetric quantization constant data, so as to determine the maximum value and the minimum value of the floating point number corresponding to the asymmetric quantization constant data according to a formula (11); determining a symmetrical quantization scaling factor for converting the constant data of the floating point number into symmetrical quantization constant data according to the formula (2); and then converting the asymmetric quantized constant data into corresponding floating-point constant data according to the formula (11), and converting the floating-point constant data into symmetric quantized constant data according to the formula (1).

For example, when the current operator is the aforementioned convolution operator 12, the asymmetric quantization constant data includes a convolution kernel K _i (i=1, 2, 3). Since the asymmetric quantization parameter of the convolution kernel K _i is the same as the asymmetric quantization parameter of the input data of the convolution operator 12, the symmetric quantization scaling coefficient of the convolution kernel K _i is the same as the symmetric quantization scaling coefficient of the input data of the convolution operator 12 (both sc_in2). The electronic device 100 may first convert K _i to the corresponding floating point number K _i_f＝(K_i -zp_in2)/ASc _in2 according to equation (11); then, according to the formula (1), K _i _f is converted into a symmetrically quantized convolution kernel K _i′＝round(K_i _f×Sc_in2. Specifically, referring to fig. 7, assume thatThen

For another example, when the current operator is the aforementioned full join operator 13, the asymmetric quantization constant data includes a weight matrix W. Since the asymmetric quantization parameter of the weight matrix W is the same as the asymmetric quantization parameter of the input data of the full join operator 13, the symmetric quantization scaling factor of the weight matrix W is the same as the symmetric quantization scaling factor of the input data of the full join operator 13 (both are sc_in3). The electronic device 100 may first convert W into a corresponding floating point number w_f= (W-zp_in3)/ASc _in3 according to formula (11); and then according to the formula (1) and the formula (2), converting w_f into a weight matrix W' =round (w_f×sc_in3) with symmetric quantization.

Specifically, referring to fig. 7, assume that

Then

For another example, where the current operator is the Softmax operator 14 described above, the asymmetric quantization constant data comprises a look-up table LUT. The Softmax operation may be expressed as the following equation (12).

In the formula (12), in _k _f represents floating point number input data of a Softmax operator, namely the floating point number score of the input image data in the kth class; p _k _f is a floating point number output result of the Softmax operator, and represents a probability that input image data is of the kth class, wherein k=1 represents a class of rabbits, k=2 represents a class of dogs, and k=3 represents a class of cats. As can be seen from the formula (12), the range of the value of P _k _f is [0,1].

The lookup table LUT is used for storing the fixed number results of Softmax operations corresponding to different fixed number input data in _k _q. For example, assuming that the asymmetric quantization scaling factor of the output data of the full join operator 13 is 2.2x10 ^-5, the asymmetric quantization zero is 0, in ₁_q＝59,in₂_q＝104,in₃ _q=182, then in ₁_f＝2681818,in₂_f＝4727273,in₃ _f= 8272727 can be obtained, P ₁_f＝0.0069,P₂_f＝0.042,P₃ _f= 0.9511 can be obtained by substituting in ₁_f＝2681818,in₂_f＝4727273,in₃ _f= 8272727 into the foregoing formula (12), and then performing 8-bit asymmetric quantization on P ₁_f＝0.0069,P₂_f＝0.042,P₃ _f= 0.9511 to obtain: p ₁_q＝2,P₂_q＝11,P₃ _q=243, that is, the lookup table LUT stores the lookup table index [59, 104, 182] (corresponding to [ in1, in2, in3 ]) and the lookup table result is [2, 11, 243] (corresponding to P ₁,P₂,P₃). Other table lookup results corresponding to the table lookup index can be obtained by a similar method, and are not described herein.

When the electronic device 100 detects the LUT, it converts the table look-up index of the LUT into a floating point table look-up index according to the formula (11), converts the floating point table look-up index into a symmetric quantized fixed point table look-up index, substitutes the floating point table look-up index into the formula (12) to obtain the floating point P _k _f, and then uses the symmetric quantized table look-up result of the symmetric quantization performed by the floating point P _k _f as the table look-up result of the symmetric quantized fixed point table look-up index in the new LUT'. For example, converting the lookup index [59, 104, 182] into a floating point number to obtain in ₁_f＝2681818,in₂_f＝4727273,in₃ _f= 8272727, performing 8-bit symmetric quantization on in ₁_f＝2681818,in₂_f＝4727273,in₃ _f= 8272727 to obtain a new lookup index [30, 52, 91], substituting in ₁_f＝2681818,in₂_f＝4727273,in₃ _f= 8272727 into the foregoing formula (12) to obtain P ₁_f＝0.0069,P₂_f＝0.042,P₃ _f= 0.9511, and performing 8-bit symmetric quantization on P ₁_f＝0.0069,P₂_f＝0.042,P₃ _f= 0.9511 to obtain P ₁_q＝1,P₂_q＝5,P₃ _q=121, i.e. in the lookup table LUT ' shown in fig. 7, the lookup result corresponding to the lookup index [30, 52, 91] (corresponding to [ in1', in2', in3' ]) is [1,5, 121] (corresponding to P ₁′,P₂′,P₃ '). Other results corresponding to the table lookup index may be obtained by a similar method, and will not be described herein.

It will be appreciated that the above conversion of asymmetric quantized constant data in convolution operators, full join operators, and Softmax operators to symmetric quantized constant data is only an example, and similar methods may be used for converting asymmetric quantized constant data in other operators (including but not limited to pooling operators, activation operators, sorting operators, normalization operators, etc.) to symmetric quantized constant data, and will not be described here.

S605: and judging whether conversion of quantization parameters of all operators is completed or not.

The electronic device 100 determines whether the conversion of all operators is completed, and if so, goes to step S606; otherwise, go to step S602 to perform conversion of quantization parameters of the next operator.

S606: and according to the symmetrical quantization parameters of each operator, invoking the operation logic of the corresponding symmetrical quantization operator to realize the function of the asymmetric quantized neural network model.

After completing the conversion of the quantization parameters of all operators, the electronic device 100 invokes the operation logic of the corresponding symmetric quantization operator through the NPU according to the symmetric quantization parameters of each operator, so as to realize the function of the asymmetric quantized neural network model.

Specifically, referring to fig. 7, after the electronic device 100 converts the asymmetric quantization parameters of each operator of the neural network model 10 into symmetric quantization parameters, each operator may be represented as an input layer 11', a convolution operator 12', a full join operator 13', a Softmax operator 14', and an output layer 15' shown in fig. 7.

The NPU of the electronic device 100 may first quantize the image 20 into an image matrix H' based on equation (1) with a symmetric quantization scaling factor sc_out1=0.0498.

Next, the NPU of the electronic device 100 invokes the operation logic of the symmetric-quantization convolution operator, for example, the operation logic shown in the foregoing formula (9), and if the convolution kernel is K _i '(i=1, 2, 3), the image matrix H' is convolved with K _i 'to obtain the fixed-point feature matrix a _i' (i=1, 2, 3). Namely, the NPU acquires symmetric quantized input data b_q (e.g., H ') described above, symmetric quantized scaling factor b_sc of the input data (e.g., S _c _in2 obtained in step S602 described above), symmetric quantized convolution kernel c_q (e.g., convolution kernel K _i' obtained in step S603 described above), symmetric quantized scaling factor c_sc of the convolution kernel (e.g., S _c _in2 described above), symmetric quantized scaling factor d_sc of the output data (e.g., S _c _out2 obtained in step S602 described above), and obtains d_q according to equation (9) described above. For example, in the case of b_q=h ', c_q=k ₁ ', the aforementioned feature matrix a ₁ ' can be obtained.

It will be appreciated that since there is no circuitry in the NPU that can directly implement the division operation, in some embodiments, the division operation in equation (9) can be implemented by a multiplicative shift to increase the speed at which the NPU operates the convolution operator. For example, assuming that b_sc×c_sc=0.498 ² =0.248, 0.248 may be expressed as 1/1×2 ^-2, and the binary number corresponding to the result of b_q×c_q×d_sc is shifted to the right by-2 bits and multiplied by 1, so that the operation result of b_q×c_q×d_sc/(b_sc×c_sc) is obtained.

Then, the NPU of the electronic device 100 invokes the operation logic of the symmetric quantized full-join operator (for example, the operation logic shown in the following formula (17)), and performs full-join operation on the feature matrix a _i ' and the weight matrix W ', for example, performing inner product, to obtain the scores in1', in2', and in3' of the image 20 for each preset category; and then invoking the operation logic of the symmetrically quantized Softmax operator, namely, searching from the lookup table LUT 'by taking [ in1', in2', in3' ] as a lookup index to obtain the probability that the image 20 belongs to each preset category; and finally, invoking the operation logic of the symmetrical quantization output layer, taking the preset category corresponding to the maximum probability of the probabilities of the image 20 belonging to each preset category as the category of the image 20, for example, determining the category of the image 20 as a cat. The derivation of the arithmetic logic of the symmetric quantization full join operator will be described below, and is not described in detail herein.

It may be appreciated that the arithmetic logic of the symmetrically quantized operators may be preset in the NPU by the developer of the NPU of the electronic device 100, or may be provided to the developer of the electronic device 100 by the developer of the NPU, and preset in the memory of the electronic device 100 by the developer of the electronic device 100.

It will be appreciated that each of the steps S601 to S605 may be all completed by the CPU of the electronic device 100, may be all completed by the NPU of the electronic device 100, or may be part of the steps respectively completed by the CPU and the NPU of the electronic device 100, which is not limited herein.

It should be understood that the above-mentioned operation sequence from step S601 to step S605 is only an example, and in other embodiments, the operation sequence of part of the steps may be adjusted, or part of the steps may be combined or split, which is not limited in the embodiments of the present application.

According to the method provided by the embodiment of the application, the NPU of the electronic equipment 100 can realize the related functions of the asymmetric quantized neural network model by calling the preset symmetric quantized operator, and other electronic equipment is not required to convert the asymmetric quantized neural network model into the floating point type neural network model, and the NPU of the electronic equipment 100 operates after converting the floating point type neural network model into the symmetric quantized neural network model, so that the types of the neural network model which can be operated by the electronic equipment 100 are increased, the universality of the NPU is improved, and the deployment speed of the neural network model is improved.

The specific process of convolution computation and the operation logic of the symmetric quantized fully-connected operator are described below.

First, the calculation process of the convolution operation will be described.

Assuming that the size of the input data matrix B is mxm, the size of the convolution kernel C is nxn, and the convolution step size is k, the convolution result D of the matrix B and the convolution kernel C may be expressed as:

In formula (13), D (m, n) is an element of the mth row and the nth column of the matrix D; m, n satisfies the following relationship:

Wherein the method comprises the steps of For rounding down, i.e.Is the largest integer less than X. Since M-N < M and k is a positive integer, it can be seen thatThat is, the size of the convolution result D is always less than or equal to the size of the matrix B.

To ensure that the size of the matrix corresponding to the convolution result is the same as the size of the input data, and avoid losing the data features of the edges of the data matrix, rows with 0 are filled in before the first row and after the last row of the input matrix and rows or columns with 0 are filled in before the first column and after the last column of the input matrix in the convolution calculation process, that is, rows or columns with 0 are filled in around the input matrix. The number of rows or columns of 0 are filled around the input matrix B, and at this time, the size of the input matrix B becomes (m+2p) × (m+2p). At this time, m, n in the formula (1) satisfies the following relation:

Order the The value of P can be calculated, for example, when the convolution kernel size is n=3 and the step size k=1, then p=1.

The arithmetic logic of the symmetric quantized fully-join operator is described below.

The full join operator is an operator that performs a weighted calculation on input data, and the full join calculation result F of the input matrix E and the weight matrix W can be expressed as the following formula (14).

Wherein E (i, j) is an element of an ith row and a jth column of the input matrix, W (i, j) is an element of an ith row and a jth column of the weight matrix, and the sizes of the input matrix E and the weight matrix W are M multiplied by N.

Assuming that e_f is a floating-point number input matrix, w_f is a floating-point number weight matrix, and f_f is a floating-point number full-connection calculation result, based on equation (14), the relationship among e_f, w_f, and f_f can be expressed as the following equation (15).

From equation (1) and equation (2), e_f=e_q/e_sc, w_f=w_q/w_sc, f_f=f_q/f_sc can be found, where: e_q is a symmetric quantized fixed-point number matrix corresponding to E_f, and E_Sc is a symmetric quantized scaling factor for quantizing E_f to E_q; w_q is a symmetric quantized fixed-point number weight matrix corresponding to W_f, and W_Sc is a symmetric quantized scaling factor for quantizing W_f to W_q; f_q is a fixed point number of symmetric quantization corresponding to f_f, and f_sc is a symmetric quantization scaling factor that quantizes f_f to f_q. Further, the expression (15) can be expressed as the following expression (16).

The transformation of equation (16) results in the arithmetic logic of the symmetric quantized fully-connected operator shown in equation (17) below.

When the NPU executes the arithmetic logic of the full-join operator shown in the formula (17), it obtains the symmetrically quantized input data e_q (e.g., a _i '(i=1, 2, 3)) and the symmetrically quantized scaling coefficient e_sc of the input data (e.g., S _c _in3 obtained in the step S602), the symmetrically quantized weight matrix w_q (e.g., the weight matrix W' obtained in the step S603), the symmetrically quantized scaling coefficient w_sc of the weight matrix (e.g., S _c _in3), the symmetrically quantized scaling coefficient f_sc of the output data (e.g., S _c _out3 obtained in the step S602), and then obtains the full-join calculation result of a _i 'and W' according to the formula (17), for example, in the case of e_q=a ₁ ', the above-mentioned in1' can be obtained.

It will be appreciated that since there is no circuitry in the NPU that can directly implement the division operation, in some embodiments, the division operation in equation (17) can be implemented by shifting and multiplication to increase the speed at which the NPU runs the fully-connected operator. For example, assuming that e_sc×w_sc=0.0417 ² = 0.00174, 0.00174 may be expressed as 1/9×2 ^-6, so that the binary number corresponding to the result of b_q×c_q×d_sc is shifted to the right by-6 bits (i.e., shifted to the left by 6 bits) and multiplied by 9, thereby obtaining the operation result of b_q×c_q×d_sc/(b_sc×c_sc).

It will be appreciated that in other embodiments, the fully-join operator may also employ other arithmetic logic, and embodiments of the present application are not limited thereto.

It will be appreciated that for other symmetrically quantized operators, the arithmetic logic may be obtained by a similar method, and will not be described in detail herein.

Further, fig. 8 illustrates a schematic structural diagram of an electronic device 100, according to some embodiments of the present application. As shown in fig. 8, electronic device 100 includes one or more processors 101A, NPU B, a system Memory 102, a Non-Volatile Memory (NVM) 103, a communication interface 104, an input/output (I/O) device 105, and system control logic 106 for coupling processor 101A, system Memory 102, non-Volatile Memory 103, communication interface 104, and input/output (I/O) device 105. Wherein:

The Processor 101A may include one or more processing units, for example, a processing module or processing circuit, which may include a central Processor CPU (Central Processing Unit), an image Processor GPU (Graphics Processing Unit), a digital signal Processor DSP (DIGITAL SIGNAL Processor), a microprocessor MCU (Micro-programmed Control Unit), an AI (ARTIFICIAL INTELLIGENCE ) Processor, or a programmable logic device FPGA (Field Programmable GATE ARRAY), may include one or more single-core or multi-core processors.

The neural network processor 101B may be configured to invoke the preset arithmetic logic of the symmetrically quantized operator, to implement reasoning of the neural network model. The neural network processor 101B may be a separate processor or may be integrated within the processor 101A. In some embodiments, the NPU may be configured to execute instructions corresponding to an operation method of the neural network model provided by the embodiment of the present application.

The system Memory 102 is a volatile Memory such as Random-Access Memory (RAM), double data rate synchronous dynamic Random Access Memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDR SDRAM), or the like. The system memory is used to temporarily store data and/or instructions, for example, in some embodiments, the system memory 102 may be used to store related instructions, asymmetric/symmetric quantization parameters, asymmetric/symmetric quantization constant data, etc. of the neural network model 10, and may also be used to store the arithmetic logic of the preset symmetric quantization operator.

Nonvolatile memory 103 may include one or more tangible, non-transitory computer-readable media for storing data and/or instructions. In some embodiments, the non-volatile memory 103 may include any suitable non-volatile memory, such as flash memory, and/or any suitable non-volatile storage device, such as a hard disk drive (HARD DISK DRIVE, HDD), compact Disc (CD), digital versatile Disc (DIGITAL VERSATILE DISC, DVD), solid state disk (Solid-state-STATE DRIVE, SSD), and the like. In some embodiments, the nonvolatile memory 103 may also be a removable storage medium, such as a Secure Digital (SD) memory card or the like. In other embodiments, the nonvolatile memory 103 may be used to store the related instructions, asymmetric/symmetric quantization parameters, asymmetric/symmetric quantization constant data, etc. of the neural network model 10, and may also be used to store the arithmetic logic of the preset symmetric quantization operator.

In particular, the system memory 102 and the nonvolatile memory 103 may each include: a temporary copy and a permanent copy of instruction 107. The instructions 107 may include: execution by at least one of the processor 101A and/or the neural network processor 101B causes the electronic device 100 to implement the method of operating a neural network model provided by embodiments of the present application.

The communication interface 104 may include a transceiver to provide a wired or wireless communication interface for the electronic device 100 to communicate with any other suitable device via one or more networks. In some embodiments, the communication interface 104 may be integrated with other components of the electronic device 100, e.g., the communication interface 104 may be integrated in the processor 101A. In some embodiments, the electronic device 100 may communicate with other devices through the communication interface 104, e.g., the electronic device 100 may obtain a neural network model to be run from the other electronic devices through the communication interface 104.

Input/output (I/O) devices 105 may include input devices such as a keyboard, mouse, etc., output devices such as a display, etc., through which a user may interact with electronic device 100.

The system control logic 106 may include any suitable interface controller to provide any suitable interface with other modules of the electronic device 100. For example, in some embodiments, the system control logic 106 may include one or more memory controllers to provide an interface to the system memory 102 and the non-volatile memory 103.

In some embodiments, at least one of the processors 101A may be packaged together with logic for one or more controllers of the system control logic 106 to form a system package (SYSTEM IN PACKAGE, SIP). In other embodiments, at least one of the processors 101A may also be integrated on the same Chip with logic for one or more controllers of the System control logic 106 to form a System-on-Chip (SoC).

It is understood that electronic device 100 may be any electronic device capable of running a neural network model, including, but not limited to, a cell phone, a wearable device (e.g., a smart watch, etc.), a tablet, a desktop, a laptop, a handheld computer, a notebook, an ultra-mobile personal computer (UMPC), a netbook, a cellular telephone, a Personal Digital Assistant (PDA), an augmented reality (augmented reality, AR)/Virtual Reality (VR) device, etc., embodiments of the present application are not limited.

It is to be understood that the configuration of the electronic device 100 shown in fig. 8 is merely an example, and in other embodiments, the electronic device 100 may include more or fewer components than shown, or may combine certain components, or may split certain components, or may have a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Embodiments of the disclosed mechanisms may be implemented in hardware, software, firmware, or a combination of these implementations. Embodiments of the application may be implemented as a computer program or program code that is executed on a programmable system comprising at least one processor, a storage system (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device.

Program code may be applied to input instructions to perform the functions described herein and generate output information. The output information may be applied to one or more output devices in a known manner. For purposes of the present application, a processing system includes any system having a Processor such as, for example, a digital signal Processor (DIGITAL SIGNAL Processor, DSP), microcontroller, application SPECIFIC INTEGRATED Circuit (ASIC), or microprocessor.

The program code may be implemented in a high level procedural or object oriented programming language to communicate with a processing system. Program code may also be implemented in assembly or machine language, if desired. Indeed, the mechanisms described in the present application are not limited in scope by any particular programming language. In either case, the language may be a compiled or interpreted language.

In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried by or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed over a network or through other computer readable media. Thus, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to floppy diskettes, optical disks, read-Only memories (CD-ROMs), magneto-optical disks, read Only Memories (ROMs), random access memories (Random Access Memory, RAMs), erasable programmable Read-Only memories (Erasable Programmable Read Only Memory, EPROMs), electrically erasable programmable Read-Only memories (ELECTRICALLY ERASABLE PROGRAMMABLE READ-Only memories, EEPROMs), magnetic or optical cards, flash Memory, or tangible machine-readable Memory for transmitting information (e.g., carrier waves, infrared signal digital signals, etc.) using the internet in an electrical, optical, acoustical or other form of propagated signal. Thus, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

In the drawings, some structural or methodological features may be shown in a particular arrangement and/or order. However, it should be understood that such a particular arrangement and/or ordering may not be required. Rather, in some embodiments, these features may be arranged in a different manner and/or order than shown in the illustrative figures. Additionally, the inclusion of structural or methodological features in a particular figure is not meant to imply that such features are required in all embodiments, and in some embodiments, may not be included or may be combined with other features.

It should be noted that, in the embodiments of the present application, each unit/module mentioned in each device is a logic unit/module, and in physical terms, one logic unit/module may be one physical unit/module, or may be a part of one physical unit/module, or may be implemented by a combination of multiple physical units/modules, where the physical implementation manner of the logic unit/module itself is not the most important, and the combination of functions implemented by the logic unit/module is only a key for solving the technical problem posed by the present application. Furthermore, in order to highlight the innovative part of the present application, the above-described device embodiments of the present application do not introduce units/modules that are less closely related to solving the technical problems posed by the present application, which does not indicate that the above-described device embodiments do not have other units/modules.

It should be noted that, in the examples and descriptions of this patent, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

While the application has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the application.

Claims

1. An operation method of a neural network model, applied to electronic equipment, is characterized by comprising the following steps:

Inputting image data into a first neural network model, wherein the first neural network model is an asymmetrically quantized neural network model, and the first neural network model comprises asymmetrical quantization parameters of operators of the first neural network model;

Wherein the asymmetric quantization parameters include: the asymmetric quantization parameters of the constant data comprise asymmetric quantization constant data, asymmetric quantization scaling coefficients of the asymmetric quantization constant data and asymmetric quantization zero points of the asymmetric quantization constant data, and the asymmetric quantization constant data further comprises an asymmetric quantization lookup table, wherein the asymmetric quantization lookup table comprises asymmetric quantization lookup table indexes and asymmetric quantization lookup table results corresponding to the asymmetric quantization lookup table indexes;

Converting the asymmetric quantization parameter of each operator into a symmetric quantization parameter;

wherein said converting the asymmetric quantization parameter of each of said operators into a symmetric quantization parameter comprises:

determining floating point number table look-up indexes corresponding to the asymmetric quantization table look-up indexes according to the asymmetric quantization scaling coefficient of the asymmetric quantization constant data and the asymmetric quantization zero point of the asymmetric quantization constant data;

Determining floating point number table look-up results corresponding to each floating point number table look-up index according to the floating point number table look-up index corresponding to each asymmetric quantized table look-up index and the operation logic of each operator;

Obtaining a symmetric quantization table lookup index according to the symmetric quantization scaling coefficient of the floating point table lookup index, and obtaining a symmetric quantization table lookup result according to the symmetric quantization scaling coefficient of the floating point data table lookup result, wherein the symmetric quantization scaling coefficient of the floating point table lookup index is determined based on the data type of the asymmetric quantization table lookup index, and the symmetric quantization scaling coefficient of the floating point table lookup result is determined based on the data type of the asymmetric quantization table lookup result;

obtaining a symmetrical quantization lookup table based on each symmetrical quantization lookup table index and a corresponding symmetrical quantization lookup table result;

invoking operation logic of a preset symmetric quantized operator by using the symmetric quantized parameters of each operator to obtain an inference result of the first neural network model;

a category of the image data is determined based on the inference result.

2. The method of claim 1, wherein the electronic device comprises a first processor capable of reasoning about a symmetrically quantized neural network model, and incapable of reasoning about an asymmetrically quantized neural network model; and

The first neural network model is run by the first processor.

3. The method according to claim 1 or 2, wherein the asymmetric quantization parameter further comprises at least one of the following parameters:

An asymmetric quantization parameter of the input data, wherein the asymmetric quantization parameter of the input data comprises an asymmetric quantization scaling coefficient of the input data and an asymmetric quantization zero point of the input data;

And the asymmetric quantization parameters of the output data comprise an asymmetric quantization scaling coefficient of the output data and an asymmetric quantization zero point of the output data.

4. A method according to claim 3, wherein said converting the asymmetric quantization parameter of each of said operators into a symmetric quantization parameter comprises:

Determining the maximum value and the minimum value of floating point numbers corresponding to the input data or the output data according to the data type of the input data or the output data and the asymmetric quantization parameters of the input data or the output data;

And determining symmetrical quantization scaling coefficients of the input data or the output data according to the maximum value and the minimum value of the floating point number corresponding to the input data or the output data.

5. The method of claim 4, wherein determining a maximum value and a minimum value of a floating point number corresponding to the input data or the output data based on a data type of the input data or the output data, an asymmetric quantization parameter of the input data or the output data, comprises:

determining the maximum value and the minimum value of the fixed point number of the input data or the output data according to the data type of the input data or the output data of each operator;

And determining the maximum value and the minimum value of the floating point number corresponding to the input data or the output data according to the asymmetric quantization parameter of the input data or the output data and the maximum value and the minimum value of the fixed point number of the input data or the output data.

6. A method according to claim 3, wherein the asymmetric quantization constant data comprises an asymmetric quantization constant, an asymmetric quantization matrix; and said converting the asymmetric quantization parameter of each of said operators into a symmetric quantization parameter, comprising:

determining the maximum value and the minimum value of the floating point number corresponding to the asymmetric quantization constant data according to the data type of the asymmetric quantization constant data, the asymmetric quantization scaling coefficient of the asymmetric quantization constant data and the asymmetric quantization zero point of the asymmetric quantization constant data;

Determining a symmetric quantization scaling coefficient of the floating point number corresponding to the asymmetric quantization constant data according to the maximum value and the minimum value of the floating point number corresponding to the asymmetric quantization constant data;

And according to the determined symmetric quantization scaling coefficient of the asymmetric quantization constant data, converting floating-point number constant data corresponding to the asymmetric quantization constant data into symmetric quantization constant data, wherein the floating-point number constant data corresponding to the asymmetric quantization constant data is determined by asymmetric quantization parameters of the constant data.

7. The method of claim 6, wherein determining the maximum value and the minimum value of the floating point number corresponding to the asymmetric quantization constant data according to the data type of the fixed point number of the asymmetric quantization constant data, the asymmetric quantization scaling coefficient of the asymmetric quantization constant data, and the asymmetric quantization zero point of the asymmetric quantization constant data comprises:

determining the maximum value and the minimum value of the fixed point number of the asymmetric quantization constant data according to the data type of the asymmetric quantization constant data;

And determining the maximum value and the minimum value of the floating point number corresponding to the asymmetric quantization constant data according to the asymmetric quantization scaling coefficient of the asymmetric quantization constant data, the asymmetric quantization zero point of the asymmetric quantization constant data and the determined maximum value and minimum value of the fixed point number of the asymmetric quantization constant data.

8. The method as recited in claim 1, further comprising:

Determining the maximum value and the minimum value of fixed points corresponding to the asymmetric quantization table index or the asymmetric quantization table result according to the asymmetric quantization table index or the data type of the asymmetric quantization table result, and determining the maximum value and the minimum value of symbol point table index corresponding to the asymmetric quantization table index or the maximum value and the minimum value of symbol point table result corresponding to the asymmetric quantization table result according to the asymmetric quantization scaling coefficient of the asymmetric quantization constant data and the asymmetric quantization zero point of the asymmetric quantization constant data based on the determined maximum value and the determined minimum value;

And determining the symmetric quantization scaling coefficient of the floating point number table index or the floating point number table result according to the determined maximum value and minimum value of the symbol number table index corresponding to the asymmetric quantization table index or the determined maximum value and minimum value of the symbol number table result corresponding to the asymmetric quantization table result.

9. A readable medium having instructions embodied therein, which when executed by a processor of an electronic device, cause the electronic device to implement the method of operating a neural network model of any one of claims 1 to 8.

10. An electronic device, comprising:

a memory for storing instructions for execution by one or more processors of the electronic device;

And a processor, one of the processors of the electronic device, for executing the instructions to cause the electronic device to implement the method of operating a neural network model of any one of claims 1 to 8.