WO2021027533A1 - Text semantic recognition method and apparatus, computer device, and storage medium - Google Patents

Abstract

A text semantic recognition method and apparatus, a computer device, and a storage medium. The method comprises: calculating a character vector of text characters in a target text and a word vector of each text segmented word; splicing the character vector of each text character with the word vector of the pertaining text segmented words to obtain a splicing vector of the text characters; according to the forward order of occurrence of the text characters in the target text, sequentially inputting the character vectors of the text characters and the splicing vectors into a first neural network to obtain a first text feature; according to the reverse order of occurrence of the text characters in the target text, sequentially inputting the character vectors corresponding to the text characters and the splicing vectors into a second neural network to obtain a second text feature; and inputting a comprehensive text feature obtained by splicing the first text feature and the second text feature into a third neural network to obtain the semantic type of the target text.

Description

Text semantic recognition method, device, computer equipment and storage medium

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on August 13, 2019, the application number is 201910744603.5, and the invention title is "Text Semantic Recognition Method, Device, Computer Equipment and Storage Medium", the entire content of which is incorporated by reference Incorporated in this application.

Technical field

This application relates to the field of artificial intelligence technology, in particular to text semantic recognition methods, devices, computer equipment and storage media.

Background technique

With the development of the Internet, text semantic recognition technology has been more and more widely used. Especially in the field of intelligent question answering, in order to accurately answer the user's inquiry, it is usually necessary to convert the user's input voice into text data, and further semantically recognize the text data to determine the true meaning of the text data, so as to answer the user accurately and quickly Questions consulted.

In terms of online platforms, in order to maintain the civilization of online language and improve the user experience, text semantic recognition technology is usually used to identify the text published on the Internet to identify the semantics of violence, vulgar, sensitive topics, commercial advertisements, etc. The text of the information.

At present, most text semantic analysis technologies use keyword matching methods for processing, and a keyword database needs to be constructed in advance, and the text to be recognized is matched with keywords in the constructed database to identify sensitive words. The inventor realizes that the semantics of keywords that are not recorded in the database cannot be accurately recognized, that is, the coverage of keywords limits the accuracy of text semantic recognition, so that the accuracy of text semantic recognition is low.

Summary of the invention

Based on this, it is necessary to provide a text semantic recognition method, device, computer equipment, and storage medium for the above technical problems.

A text semantic recognition method, the method includes:

Calculate the word vector of each text character in the target text and the word vector of each text segmentation;

Splicing the word vector of each text character with the word vector of the corresponding text segmentation to obtain the splicing vector of the corresponding text character;

According to the forward appearance order of text characters in the target text, the word vectors and splicing vectors corresponding to multiple text characters are sequentially input into different hidden layers of the first neural network to obtain the first order of the target text based on the forward appearance order. Text feature

According to the reverse appearance order of the text characters in the target text, the word vectors and splicing vectors corresponding to multiple text characters are sequentially input into different hidden layers of the second neural network to obtain the second text feature of the target text based on the reverse appearance order ；

The integrated text feature obtained by splicing the first text feature and the second text feature is input to a third neural network to obtain the semantic type of the target text.

A text semantic recognition device, the device includes:

The vector calculation module is used to calculate the word vector of each text character in the target text and the word vector of each text word segmentation;

The vector splicing module is used to splice the word vector of each text character with the word vector of the corresponding text segmentation to obtain the splicing vector of the corresponding text character;

The first text feature acquisition module is used to sequentially input the word vectors and splicing vectors corresponding to multiple text characters into different hidden layers of the first neural network according to the forward appearance order of the text characters in the target text to obtain the target The text is based on the first text feature in the forward order of appearance;

The second text feature acquisition module is used to input word vectors and splicing vectors corresponding to multiple text characters into different hidden layers of the second neural network according to the reverse appearance order of text characters in the target text to obtain the target text The second text feature based on the reverse appearance order;

The semantic type acquisition module is configured to input the integrated text feature obtained by splicing the first text feature and the second text feature into a third neural network to obtain the semantic type of the target text.

A computer device includes a memory and a processor, the processor and the memory are connected to each other, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the The program instructions of the memory, wherein:

Calculate the word vector of each text character in the target text and the word vector of each text segmentation;

Splicing the word vector of each text character with the word vector of the corresponding text segmentation to obtain the splicing vector of the corresponding text character;

According to the forward appearance order of text characters in the target text, the word vectors and splicing vectors corresponding to multiple text characters are sequentially input into different hidden layers of the first neural network to obtain the first order of the target text based on the forward appearance order. Text feature

According to the reverse appearance order of the text characters in the target text, the word vectors and splicing vectors corresponding to multiple text characters are sequentially input into different hidden layers of the second neural network to obtain the second text feature of the target text based on the reverse appearance order ；

The integrated text feature obtained by splicing the first text feature and the second text feature is input to a third neural network to obtain the semantic type of the target text.

A computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, they are used to implement the following steps:

Calculate the word vector of each text character in the target text and the word vector of each text segmentation;

Splicing the word vector of each text character with the word vector of the corresponding text segmentation to obtain the splicing vector of the corresponding text character;

According to the forward appearance order of text characters in the target text, the word vectors and splicing vectors corresponding to multiple text characters are sequentially input into different hidden layers of the first neural network to obtain the first order of the target text based on the forward appearance order. Text feature

According to the reverse appearance order of the text characters in the target text, the word vectors and splicing vectors corresponding to multiple text characters are sequentially input into different hidden layers of the second neural network to obtain the second text feature of the target text based on the reverse appearance order ；

The integrated text feature obtained by splicing the first text feature and the second text feature is input to a third neural network to obtain the semantic type of the target text.

The above-mentioned text semantic recognition method, device, computer equipment and storage medium obtain the word vector corresponding to each text character and the word vector of the word segmentation of the text and perform vector splicing to obtain the splicing vector corresponding to the text character. Vector splicing is used to characterize text through multiple feature vectors, which enhances the feature dimension of text language representation. Furthermore, by inputting word vectors and splicing vectors into different hidden layers of different neural networks in forward and reverse order, relevant information of text characters can be obtained more fully, and the contextual semantics between text characters can be mined, so that the first The first text feature and the second text feature output by the neural network are spliced to obtain a comprehensive feature, which can more fully express the semantic feature of the target text and improve the accuracy of text semantic recognition.

Description of the drawings

Figure 1 is an application scenario diagram of a text semantic recognition method in an embodiment;

FIG. 2 is a schematic flowchart of a text semantic recognition method in an embodiment;

FIG. 3 is a schematic diagram of a process of generating a preset file in an embodiment;

Figure 4 is a structural block diagram of a text semantic recognition device in an embodiment;

Figure 5 is a structural block diagram of a text semantic recognition device in another embodiment;

Fig. 6 is an internal structure diagram of a computer device in an embodiment.

detailed description

In order to make the purpose, technical solutions, and advantages of this application clearer, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the application, and not used to limit the application.

The text semantic recognition method provided in this application can be applied to the application environment as shown in FIG. 1. The text semantic recognition method is applied to the text semantic system. The text semantic system includes a terminal 102 and a server 104. Wherein, the terminal 102 and the server 104 communicate through the network. The text semantic recognition method can be completed in the terminal 102 or the server 104, and the terminal 102 can collect the target text to be recognized and use the above-mentioned text semantic recognition method on the terminal 102 to perform semantic type recognition. Or the terminal 102 may obtain the target text to be recognized, and then transmit the target text to the server 104 through a network connection, and the server 104 uses the above-mentioned text semantic recognition method to recognize the target text in semantic type. The terminal 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server 104 may be implemented as an independent server or a server cluster composed of multiple servers.

In one embodiment, as shown in FIG. 2, a method for text semantic recognition is provided. Taking the method applied to the server in FIG. 1 as an example for description, the method includes the following steps:

Step S202: Calculate the word vector of each text character in the target text and the word vector of each text segmentation.

Among them, the text characters are multiple independent characters obtained by segmenting the target text. The text characters can specifically be letters, numbers, words or symbols. Text segmentation refers to the process of dividing the target text into individual words, that is, recombining consecutive word sequences into word sequences according to certain specifications. Text segmentation can be segmented using string matching-based word segmentation methods, semantic-based word segmentation methods, and statistics-based word segmentation methods. The word vector and word vector are used to represent the multi-dimensional representation of the target text.

Specifically, the server determines each text character contained in the target text and the text segmentation to which each text character belongs according to the acquired target text, and matches the pre-trained word vector library or word vector library to obtain the target text The word vector corresponding to each text character and the word vector corresponding to the text segmentation. The server may also encode the obtained text characters and text word segmentation through preset vector encoding rules to obtain corresponding word vectors and word vectors.

In one of the embodiments, the specific steps of obtaining the target text include: the terminal obtains the target text, where there may be multiple target texts, and the target text may be recognized text obtained by voice recognition, or directly input by the user at the terminal text. The terminal transmits the acquired target text to the server. The target text can also be obtained from the network platform, and the relevant target text is obtained from the network through crawler technology.

In one of the embodiments, the step of determining each text character contained in the target text and the text segmentation to which each text character belongs includes: the server performs character segmentation processing according to the received target text to obtain the target text contained The text characters; arrange the obtained text characters according to the order in which the text characters appear in the target text to obtain the character sequence of the target text, delete the text characters belonging to the stop vocabulary from the character sequence, and obtain the preprocessed After the character sequence. Among them, stop words refer to words or characters with no processing value that need to be filtered in natural language processing tasks; stop words include English characters, numbers, mathematical characters, punctuation marks, and single Chinese characters that are frequently used.

The server detects each character in the character sequence, and identifies the same character to distinguish different words corresponding to the same character; uses the pre-built word segmentation thesaurus to segment the character sequence with the character identifier to obtain A word sequence with character identifiers; based on the preprocessed character sequence, the server determines the text segmentation to which each character belongs from the word sequence.

In one of the embodiments, the word-segmentation thesaurus can be built based on "Xinhua Dictionary" or other similar published books, or the word-segmentation thesaurus can be constructed according to the intelligent customer service scenario. The constructed word segmentation database can be stored in the database of the server or sent to the cloud.

In one of the embodiments, the target text may also be obtained through a server. For example, the server may obtain required text data from a web page as the target text, and further determine each text character of the target text and the text word segmentation to which each text character belongs.

For example, the obtained target text is "Shenzhen city government is in the civic center." First, the server performs word segmentation processing on the target text to obtain the character sequence "Shenzhen/Shen/City/的/市/政/府/在/市/民/中/心/.", delete the characters belonging to the stop vocabulary in the character sequence, and obtain the preprocessed character sequence "Shen/Shen/City/City/Government/Fun/City/民/中/心"; further, the same characters are identified by the character, and the character sequence "Shen/Shen/City 01/市02/Government/Fun/City 03/Min/Zhong/Xin" is obtained. Word segmentation, the word sequence "Shenzhen 01/City 02 Government/City 03 People Center" is obtained. Although the text character is "市", there are three words, but the text segmentation to which the text character belongs can be distinguished according to the character identifier.

Step S204, splicing the word vector of each text character with the word vector of the corresponding text segmentation to obtain the splicing vector of the corresponding text character.

Among them, the splicing vector is a vector formed by splicing multiple text vectors according to a preset rule, and the splicing vector represents the representation dimension of the multiple vectors.

Specifically, based on the acquired word vector and word vector of the target text, the server splices the word vector corresponding to each text character and the word vector to which the text character belongs to obtain the splicing vector corresponding to the text character, thereby obtaining the target The splicing vector of all text characters contained in the text, in which the order in which the word vector and word vector are spliced is not required.

In one of the embodiments, the server adds or multiplies the word vector corresponding to each text character and the word vector of the text segmentation to which the text character belongs to obtain a splicing vector of the corresponding text character.

Step S206: According to the forward appearance order of the text characters in the target text, the word vectors and splicing vectors corresponding to the multiple text characters are sequentially input into different hidden layers of the first neural network to obtain the first text of the target text based on the forward appearance order feature.

Among them, the first neural network is mainly used to generate features that carry the contextual semantics of the target text based on the forward appearance order from the features contained in the target text entered in the forward appearance order of the text characters in the target text; the first neural network The network includes multiple hidden layers, and each hidden layer can have the same or different neuron nodes. The first neural network is a recurrent neural network, which can be a long and short-term memory network LSTM and a recurrent neural network RNN.

Specifically, the target text contains multiple text characters, and the server calculates the word vector and splicing vector corresponding to each text character, and sorts the calculated word vector and splicing vector according to the forward appearance order of the text characters in the target text. Further, The word vectors and splicing vectors sorted in the forward order of appearance are sequentially input into different hidden layers of the first neural network for feature extraction, mutual information of different text characters is obtained, and the first text feature based on the forward order of appearance is obtained.

In one of the embodiments, the word vectors, word vectors, and splicing vectors corresponding to multiple text characters can be sequentially input to different hidden layers of the first neural network to obtain the first text feature of the target text based on the forward appearance order.

Step S208, according to the reverse appearance order of the text characters in the target text, input the word vectors and splicing vectors corresponding to the multiple text characters into different hidden layers of the second neural network in order to obtain the second text feature of the target text based on the reverse appearance order.

Among them, the second neural network is mainly used to generate features that carry the contextual semantics of the target text based on the reverse appearance order from the features contained in the target text entered in the reverse appearance order of the text characters in the target text; the second neural network includes Multiple hidden layers, each hidden layer can have the same or different neuron nodes. The second neural network is a recurrent neural network, which can be a long short-term memory network LSTM and a recurrent neural network RNN.

Specifically, based on the obtained word vectors and splicing vectors corresponding to all text characters in the target text, according to the reverse order of the text characters appearing in the target text, the word vectors and splicing vectors corresponding to the text characters are sequentially input to the second neural network In the different hidden layers of, the input word vector and splicing vector are extracted through the second neural network, and the second text feature based on the reverse appearance order is obtained.

In one of the embodiments, the maximum number of text characters that can be input by the first neural network or the second neural network can be preset. If the number of text characters of the currently input target text is less than the maximum number of text characters, the target text The word vector matrix formed is filled with 0 vectors, and the filled word vector matrix is used as the input of the first neural network or the second neural network.

In step S210, the integrated text feature obtained by splicing the first text feature and the second text feature is input to the third neural network to obtain the semantic type of the target text.

Among them, the integrated text feature is a text feature formed by splicing the output of the first neural network and the output of the second neural network according to preset rules. The third neural network is mainly used to classify the semantic type according to the comprehensive text features corresponding to the input target text, so as to obtain the semantic type of the target text. The semantic type refers to determining the type of the target text according to the semantic relationship of the target text.

Specifically, based on the obtained first text feature and second text feature of the target text, the server splices the first text feature and the second text feature to obtain the integrated text feature of the target text; further, the integrated text feature of the target text is transmitted To the third neural network, the comprehensive text features are classified according to semantic categories through the third neural network, and the semantic type of the target text is obtained, fully considering the semantic understanding of the text context and implicit vocabulary. For example, when recognizing the swear words and polite words of the target text, the semantic types can be set to two categories accordingly, namely category 1: text is swear words, and category 0: text is polite words.

In the above embodiment, the word vector corresponding to each text character and the word vector of the corresponding text segmentation are obtained by calculation and vector splicing is performed to obtain the splicing vector corresponding to the text character. The vector characterizes the text, which enhances the feature dimension of the text language. Furthermore, by inputting word vectors and splicing vectors into different hidden layers of different neural networks in forward and reverse order, relevant information of text characters can be obtained more fully, and the contextual semantics between text characters can be mined, so that the first The first text feature and the second text feature output by the neural network are spliced to obtain a comprehensive feature, which can more fully express the semantic feature of the target text and improve the accuracy of text semantic recognition.

In one embodiment, as shown in FIG. 3, the above method further includes the step of generating a preset file:

Step S302: Obtain sample text.

Step S304: Extract the word vectors and word vectors of the sample text based on the pre-trained first neural network.

In step S306, the character vector and the word vector are respectively numbered.

Step S308: Write the word vector, the word vector, and the corresponding character numbers to the preset file.

Calculating the word vector of text characters and the word vector corresponding to text word segmentation includes: numbering each text character and text word segmentation; based on the character number, reading the word vector corresponding to each text character and each The word vector corresponding to the text segmentation.

Among them, the preset file is a pre-built text with an index, including a word vector and its index, and a word vector and its index.

Specifically, before calculating the word vector of the text character and the word vector corresponding to the text segmentation, a preset file containing the word vector and the index query of the word vector needs to be constructed. The server obtains the sample text and the corresponding known semantic type from the terminal or web page, extracts the word vector and word vector of the sample text based on the pre-trained first neural network, and performs character numbers on the extracted word vector and word vector respectively , Get the mapping relationship between word vectors and numbers, and the mapping relationship between word vectors and numbers. The server writes the word vector, the word vector, and the corresponding character numbers into the preset file to form the word vector and the word vector with a character number index.

Based on each text character contained in the target text and the text segmentation to which the text character belongs, the server performs character numbering for each text character and text segmentation, and obtains the mapping relationship between text characters and character numbers, and text segmentation and character segmentation The mapping relationship between numbers. The word vector of the corresponding text character is queried from the preset file according to the character number of each text character, and the word vector of the corresponding text word segment is queried from the preset file according to the character number of each text word segmentation.

In one of the embodiments, the character number may include a number type. The character vectors and word vectors are respectively numbered according to the numbering type, and the numbering types of the word vector and the word vector can be the same or different. For example, the character vectors are numbered according to natural numbers; the word vectors can be numbered according to natural numbers or English letters.

For example, if the target text is "Shenzhen", if the character number of the text character "Deep" is 01, the word vector corresponding to the character number 01 obtained from the preset file is (1,1,2,2).

In this embodiment, through a pre-built preset file containing word vectors and word vectors, when calculating the word vectors and word vectors of the target text, according to the character numbers corresponding to the text characters and text segmentation, the preset files are The query to obtain the corresponding word vector and word vector can accurately and quickly obtain the word vector of the text character and the word vector of the text word segmentation, thereby improving the rate and accuracy of obtaining the semantic type of the target text.

In one embodiment, according to the forward appearance order of the text characters in the target text, the word vectors and splicing vectors corresponding to the multiple text characters are sequentially input into different hidden layers of the first neural network to obtain the target text based on the forward appearance order The first text feature includes: input the word vector and splicing vector corresponding to the text character in the current sequence to the current hidden layer of the first neural network according to the forward appearance order of the text characters in the target text; The word vector and splicing vector corresponding to a sequence of text characters are input into the next hidden layer of the first neural network; the next hidden layer is used as the current hidden layer to iterate until the last sequential text characters, and the first order of the target text based on the forward appearance order is obtained. A text feature.

Specifically, according to the forward appearance order of the text characters in the target text, the server inputs the word vectors and splicing vectors corresponding to the text characters in the first order into the first hidden layer of the first neural network, through the first hidden layer Project the input word vector and splicing vector to obtain the character feature corresponding to the text character. Among them, the first-order text character means that after all text characters in the target text are sorted according to a preset appearance order, the text character in the first position is the first-order text character. The last-order text character refers to the last text character after all text characters in the target text are sorted according to the preset order of appearance.

The server takes the first hidden layer as the current hidden layer, and obtains the word vector and the splicing vector corresponding to the current hidden layer according to the forward appearance sequence of the text characters in the target text. The weight of each neuron node in the current hidden layer is preset, wherein the weight of each neuron node can be the same or different. Furthermore, the server performs a nonlinear mapping on the features of the current hidden layer input according to the preset weights of each neuron node to obtain the character features output by the current hidden layer. Among them, the non-linear mapping can use activation functions such as a sigmoid (S-type) function, a tanh (hyperbolic tangent) function, and a relu (modified linear unit) function.

The server inputs the character feature output by the current hidden layer and the word vector and splicing vector corresponding to the next sequential text character to the next hidden layer of the first neural network, and the next hidden layer is used as the current hidden layer, and the following steps are executed in a loop: The text characters appear in the forward direction of the target text, and the word vector and splicing vector corresponding to the current hidden layer are obtained. Preset the weight of each neuron node in the current hidden layer. The server uses nonlinear mapping to obtain the character features output by the current hidden layer according to the word vector and the splicing vector corresponding to the current hidden layer, the weight of each neuron node corresponding to the current hidden layer, and the character features output by the previous hidden layer. That is, the server inputs the character features output by the previous hidden layer, the word vector and the splicing vector corresponding to the current hidden layer into the current hidden layer, and performs nonlinear mapping on the input features according to the preset weights of each neuron node to obtain Character characteristics of the next hidden layer. Use the next hidden layer as the current hidden layer, repeat the iterations, and execute the above steps until the word vector and splicing vector corresponding to the last sequential text characters are input to the current hidden layer to obtain the character features output by the current hidden layer, and the output The character feature is the first text feature of the target text based on the forward appearance order.

In one of the embodiments, the current hidden layer includes a first sub-hidden layer and a second sub-hidden layer; inputting the word vector and the splicing vector corresponding to the current sequential text character into the current hidden layer of the first neural network includes: combining the word vector and The output of the previous hidden layer is used as the input of the first sub-hidden layer. The first sub-hidden layer is used to project the word vector according to the weight of each neuron node corresponding to the first sub-hidden layer to obtain the first sub-character feature; A sub-character feature and a splicing vector are used as the input of the second sub-hidden layer. The second sub-hidden layer is used to project the splicing vector according to the weight of each neuron node corresponding to the second sub-hidden layer to obtain the second sub-character feature as The output of the current hidden layer.

Specifically, the current hidden layer includes a first sub-hidden layer and a second sub-hidden layer; the server inputs the word vector corresponding to the text character in the first order of appearance into the first sub-hidden layer, and passes it through the first sub-hidden layer The preset weights of each neuron node project the word vector to obtain the first character feature output by the first sub-hidden layer; further, the server splices the first character feature with the text character whose appearance order is in the first place. The vector is used as the input of the second sub-hidden layer, and the input features are non-linearly mapped according to the preset weights of each neuron node in the second sub-hidden layer to obtain the character features output by the second sub-hidden layer as the first The output of a hidden layer.

In one of the embodiments, the current hidden layer includes a first sub-hidden layer and a second sub-hidden layer; inputting the word vector and splicing vector corresponding to the current sequential text character into the current hidden layer of the first neural network includes: the server will splice the vector And the output of the previous hidden layer as the input of the first sub-hidden layer, the first sub-hidden layer is used to project the word vector according to the weight of each neuron node corresponding to the first sub-hidden layer to obtain the first sub-character feature; The first sub-character feature and the word vector are used as the input of the second sub-hidden layer. The second sub-hidden layer is used to project the splicing vector according to the weight of each neuron node corresponding to the second sub-hidden layer to obtain the second sub-character feature. As the output of the current hidden layer.

In one of the embodiments, according to the forward appearance order of the text characters in the target text, the server sequentially inputs the word vectors, word vectors, and splicing vectors corresponding to multiple text characters into different hidden layers of the first neural network to obtain the target text The first text feature based on the forward order of appearance.

Specifically, according to the forward appearance order of the text characters in the target text, the server inputs the word vectors, word vectors, and splicing vectors corresponding to the text characters in the current order into the current hidden layer of the first neural network, and according to the word vectors, The word vector and splicing vector, the weight of each neuron node corresponding to the current hidden layer and the character feature output by the previous hidden layer, use nonlinear mapping to obtain the character feature output by the current hidden layer; the server compares the character feature output by the current hidden layer with The word vector, word vector, and splicing vector corresponding to the next sequence of text characters are input to the next hidden layer of the first neural network, the next hidden layer is used as the current hidden layer, and the character features output by the current hidden layer are repeated to the next order The word vectors, word vectors, and splicing vectors corresponding to the text characters are input into the next hidden layer of the first neural network, until the final sequence of text characters, and the first text feature of the target text based on the forward appearance order is obtained.

In one of the embodiments, the first hidden layer includes a first sub-hidden layer, a second sub-hidden layer, and a third sub-hidden layer; the word vector, word vector, and splicing vector corresponding to the current sequential text character are input into the first neural network The current hidden layer includes: taking the word vector and the output of the previous hidden layer as the input of the first sub-hidden layer, the first sub-hidden layer is used to perform the word vector according to the weight of each neuron node corresponding to the first sub-hidden layer The first sub-character feature is obtained by projection; the first sub-character feature and word vector are used as the input of the second sub-hidden layer, and the second sub-hidden layer is used to stitch the vector according to the weight of each neuron node corresponding to the second sub-hidden layer Perform projection to obtain the second sub-character feature; use the second sub-character feature and the splicing vector as the input of the third sub-hidden layer, and the third sub-hidden layer is used to splice according to the weight of each neuron node corresponding to the third sub-hidden layer The vector is projected to obtain the third sub-character feature as the output of the first hidden layer. Among them, the projection order of the word vector, word vector, and splicing vector corresponding to the text character is not specified and can be set arbitrarily. For example, the word vector can be projected to obtain the first sub-character feature corresponding to the first sub-hidden layer.

In one of the embodiments, the server uses nonlinear mapping to obtain the next hidden layer according to the word vector and splicing vector corresponding to the current hidden layer, the weight of each neuron node corresponding to the current hidden layer, and the character features output by the previous hidden layer. The character characteristics of can be illustrated by the following examples.

For example, suppose that each neuron node corresponding to the current hidden layer is denoted as W _f , the word vector corresponding to the current hidden layer is x _t , the stitching vector corresponding to the current hidden layer is y _t , and the character feature output by the previous hidden layer is h _{t- 1. The} nonlinear function is tanh, then the character feature f _t output by the next hidden layer can be calculated by the following formula:

f _t =tanh(W _c [h _t-1 ,x _t ,y _t ]+b _f ); where b _f is the bias of the current hidden layer.

In one of the embodiments, the first neural network further includes a random inactivation layer, and the method further includes: using the first text feature as the input of the random inactivation layer, and the inactivation layer is used to use each data in the first text feature at any time. The sparse feature vector is obtained by projection according to the preset sparse probability, which is used as the output of the first neural network.

Among them, the random inactivation layer (dropout) is mainly used to sparse the input first text feature, and to zero some elements of the first text feature to prevent the neural network from overfitting, and also to reduce the neural network's Calculation amount.

Specifically, the server inputs the first text feature to the random inactivation layer, and the random inactivation layer performs sparse processing on the first text feature according to the set sparse probability, and projects each data in the first text feature according to the sparse probability, In this way, the sparse feature vector is obtained, where the sparse probability refers to the probability that the data appears after projection. For example, the first text feature is a one-dimensional sequence [1,2,3,4] ^T , the sparsity probability is set to 0.5, and the probability of each number in the corresponding one-dimensional sequence after projection is 0.5, that is, after random The output result of the deactivation layer can be [0,2,0,4] ^T or [0,0,0,4] ^T.

In one embodiment, according to the reverse appearance order of the text characters in the target text, the word vectors and splicing vectors corresponding to multiple text characters are sequentially input into different hidden layers of the second neural network to obtain the second target text based on the reverse appearance order. The text feature includes: according to the reverse appearance order of the text characters in the target text, input the first text feature, the word vector and the splicing vector corresponding to the text character in the first order to the first hidden layer of the second neural network to obtain The character features output by the first hidden layer; the second hidden layer of the second neural network is used as the current hidden layer, and the second sequential text characters are used as the current sequential text characters; the character features output by the previous hidden layer, The word vector and the splicing vector corresponding to the current sequence of text characters are input into the current hidden layer of the second neural network; the next hidden layer is used as the current hidden layer to iterate until the last sequential text characters, and the target text based on the reverse appearance order is obtained. 2. Text features.

Specifically, the first neural network and the second neural network are connected in series, and the output of the first neural network is used as the input of the second neural network. According to the reverse appearance order of the text characters in the target text, the word vector and the splicing vector corresponding to the first-order text characters are obtained; the server inputs the obtained first text feature output by the first neural network and the word vector and splicing vector to the first The first hidden layer of the second neural network is used to obtain the character features output by the first hidden layer; the second hidden layer of the second neural network is used as the current hidden layer, and the second sequential text character is used as the current sequential text character; further, the previous The character features output by a hidden layer, the word vector and the splicing vector corresponding to the current sequential text characters are input to the current hidden layer of the second neural network, and the input features are non-linearly mapped through the weight of each neuron set in the current hidden layer , Get the character features output by the current hidden layer; the server inputs the character vector and the splicing vector corresponding to the next sequential text character and the character feature output by the current hidden layer into the next hidden layer of the second neural network, and uses the next hidden layer as the current The hidden layer iterates until the last sequence of text characters, and obtains the second text feature of the target text based on the reverse appearance order.

In one of the embodiments, the current hidden layer of the second neural network includes a first sub-hidden layer and a second sub-hidden layer; the character feature output by the previous hidden layer, the word vector corresponding to the current sequential text character, and the splicing The vector input to the current hidden layer of the second neural network includes: taking the word vector and the character features output by the previous hidden layer as the input of the first sub-hidden layer. The first sub-hidden layer is used for each neural network corresponding to the first sub-hidden layer. The weight of the meta node is used to project the word vector to obtain the first sub-character feature; the first sub-character feature and the splicing vector are used as the input of the second sub-hidden layer, and the second sub-hidden layer is used for each corresponding to the second sub-hidden layer. The weight of the neuron node projects the splicing vector to obtain the second sub-character feature as the output of the current hidden layer.

In one of the embodiments, inputting the second text feature to the third neural network to obtain the semantic type of the target text includes: using the second text feature as the input of the attention mechanism layer, and the attention mechanism layer is used to use the second text Each data in the feature is weighted to obtain the weighted feature; the weighted feature is used as the input of the random inactivation layer, and the random inactivation layer is used to project each data in the weighted feature according to the preset sparse probability to obtain the sparse feature; The feature is used as the input of the fully connected layer, and the fully connected layer is used to classify the sparse features to obtain the prediction probability corresponding to each semantic type; select the semantic type with the largest prediction probability as the semantic type of the target text.

In one of the embodiments, the server sequentially inputs the word vectors, word vectors, and splicing vectors corresponding to multiple text characters into different hidden layers of the second neural network according to the reverse appearance order of the text characters in the target text, to obtain the target text based on The second text feature in the reverse order of appearance.

In one of the embodiments, the second neural network further includes a random inactivation layer, and the method further includes: the server uses the second text feature as the input of the random inactivation layer, and the inactivation layer is used to use each of the second text features at any time. The data is projected according to the preset sparse probability to obtain the sparse feature vector, which is used as the output of the second neural network.

In the following, a specific embodiment is used to describe the process of processing the target text by the first neural network and the second neural network. For example, for the target text "Shenzhen", the word vector corresponding to the text character "Deep" is "(1,1,2,2)", and the splicing vector is "(1,1,1,1)", and the corresponding The text character "Zhen" corresponds to "(1,2,3,4)", the splicing vector is "(2,1,1,1)", and the text character "City" corresponds to "(0,0,2,5)" , The splicing vector is "(3,1,1,1)";

According to the forward appearance sequence of the text characters in the target text, the word vector of the target text formed is

The stitching vector is

Input the word vector and splicing vector corresponding to the first-order text characters into the first hidden layer, and use the first hidden layer as the current hidden layer. The current hidden layer includes the first sub-hidden layer and the second sub-hidden layer; The vector (1,1,2,2) is input into the first sub-hidden layer of the current hidden layer to obtain the first sub-character feature output by the first sub-hidden layer; further, the first sub-character feature and the splicing vector (1 ,1,1,1) Input to the second hidden layer, and perform nonlinear mapping on the input features according to the preset weight of each neuron node in the second hidden layer to obtain the output of the second hidden layer Character features, as the output of the current hidden layer.

Further, the text characters are moved in the order of appearance in the forward direction to obtain the corresponding word vector (1,2,3,4) and the splicing vector (2,1,1,1), and the second hidden layer is used as the current hidden layer. Input the output of the previous hidden layer and the word vector (1,2,3,4) into the first hidden layer of the current hidden layer to obtain the output of the first hidden layer of the current hidden layer; The output of the first sub-hidden layer and the stitching vector (2,1,1,1) are input to the second sub-hidden layer of the current hidden layer, according to the preset value of each neuron node in the second sub-hidden layer of the current hidden layer. The weight performs non-linear mapping on the input features to obtain the character features output by the second sub-hidden layer of the current hidden layer as the output of the current hidden layer. Until the output of the previous hidden layer, the word vector (0,0,2,5) and the stitching vector (3,1,1,1) corresponding to the last sequential text characters are input to the current hidden layer, the output of the current hidden layer is obtained , Use the output result as the first text feature of the output of the first neural network.

In one of the embodiments, the first neural network and the second neural network are connected in series, and the second text feature output by the second neural network is input to the third neural network; wherein, the output of the first neural network is used as the second neural network input of.

Based on the reverse appearance order of text characters in the target text, the word vector of the target text formed is

The stitching vector is

According to the reverse appearance order of the text characters, the first text feature, the word vector and the splicing vector corresponding to the text character in the first order are input into the first hidden layer of the second neural network to obtain the character feature output by the first hidden layer , And use the first hidden layer of the second neural network as the current hidden layer, and the second sequential text character as the current sequential text character; that is, the first text feature, the word vector (0, 0, 2, 5) is input to the first In the sub-hidden layer, the output of the first sub-hidden layer of the current hidden layer is obtained. Input the output of the first sub-hidden layer of the current hidden layer and the stitching vector (3,1,1,1) into the second sub-hidden layer of the current hidden layer, according to the preset second sub-hidden layer of the current hidden layer The weight of each neuron node performs a nonlinear mapping on the input feature, so as to obtain the character feature output by the second sub-hidden layer of the current hidden layer, which is used as the output of the current hidden layer.

Move the text characters in the reverse order to obtain the corresponding word vector (1,2,3,4) and the splicing vector (2,1,1,1), and use the second hidden layer as the current hidden layer. Input the output of the previous hidden layer and the word vector (1,2,3,4) into the first hidden layer of the current hidden layer to obtain the output of the first hidden layer of the current hidden layer; The output of the first sub-hidden layer and the stitching vector (2,1,1,1) are input to the second sub-hidden layer of the current hidden layer, according to the preset value of each neuron node in the second sub-hidden layer of the current hidden layer. The weight performs non-linear mapping on the input features to obtain the character features output by the second sub-hidden layer of the current hidden layer as the output of the current hidden layer. Until the output of the previous hidden layer, the word vector (1,1,2,2) and the stitching vector (1,1,1,1) corresponding to the last sequential text characters are input to the current hidden layer, the output of the current hidden layer is obtained , Use the output result as the second text feature output by the first neural network. The second text feature is input to the third neural network to obtain the semantic type of the target text. Through the serial connection of the first neural network and the second neural network, the output of the first neural network is used as the input of the second neural network, that is, the first text feature based on the forward order and the word vector sum based on the reverse appearance order The splicing vectors are input to the second neural network together, which can more fully mine the mutual information between text characters, especially when the text characters are far apart, it can also obtain the context information between the characters.

In one of the embodiments, the first neural network and the second neural network are connected in parallel, and the first text feature output by the first neural network and the second text feature output by the second neural network are spliced and input to the third neural network. Parallel processing of the target text through the first neural network and the second neural network can increase the data processing rate.

In this embodiment, by inputting various features such as character vectors, word vectors, and splicing vectors corresponding to text characters into the first neural network and the second neural network, the first neural network enters each of the second neural network. The hidden layer performs cyclic calculation on the input features to obtain the first text feature and the second text feature representing the semantics of the target text. Through the cyclic calculation of the neural network, the correlation between text characters can be better captured, especially For text characters with far apart characters. That is to say, when the text character interval is large, the relevant information of the predicted position can also be obtained well. The relevant information does not increase with the number of cycles, showing a decay trend.

In one embodiment, the third neural network layer includes an attention mechanism layer and a fully connected layer; splicing and inputting the first text feature and the second text feature to the third neural network to obtain the semantic type of the target text includes: The first text feature and the second text feature are spliced to obtain the integrated text feature of the target text; the integrated text feature is used as the input of the attention mechanism layer, and the attention mechanism layer is used to weight each data in the integrated text feature to obtain the weight Features; the weighted feature is used as the input of the random inactivation layer, and the random inactivation layer is used to project each data in the weighted feature according to the preset sparse probability to obtain the sparse feature; use the sparse feature as the input of the fully connected layer, fully connected The layer is used to classify sparse features to obtain the prediction probability corresponding to each semantic type; select the semantic type with the largest prediction probability as the semantic type of the target text.

Specifically, the server splices the first text feature and the second text feature of the target text to obtain the comprehensive text feature of the target text. Further, the server inputs the obtained comprehensive text features to the attention mechanism layer, and the attention mechanism layer calculates each data in the comprehensive text features according to the pre-trained coefficient weight parameters to obtain the coefficient sequence; uses a nonlinear activation function to activate the coefficient sequence, Obtain the activated coefficient sequence; normalize the activated coefficient sequence through the logistic regression (softmax) function to obtain the coefficient probability corresponding to each data in the integrated text feature; among them, the range of the coefficient probability lies in [0, 1 ]between. The coefficient probabilities obtained are multiplied by the respective data corresponding to the integrated text features, and the weighted features after weighting processing are obtained as the output of the attention mechanism layer.

Further, the server inputs the weighted features output by the attention mechanism layer to the random inactivation layer, and the random inactivation layer sparsely processes the weighted features according to the set sparsity probability, and projects each data in the weighted features according to the sparsity probability to This results in sparse features, where the sparse probability refers to the probability that the data appears after projection.

Further, the server inputs the sparse features to the fully connected layer, performs classification operations on the sparse features through the fully connected layer, and calculates the prediction probability corresponding to each semantic type according to the weight parameters of the trained fully connected layer. Each predicted probability corresponds to a semantic type. The server selects the semantic type with the largest predicted probability as the semantic type of the target text.

In one of the embodiments, the third neural network layer also includes a logistic regression layer (softmax layer), which specifically includes: taking the prediction probability corresponding to each semantic type as the input of the softmax layer, where the softmax layer is used to perform the The normalization process obtains the probability corresponding to each semantic type, and the semantic type with the highest probability is selected as the semantic type of the target text.

For example, the output prediction probability of the fully connected layer is

The semantic type corresponding to a is 1, and the semantic type corresponding to b is 0; after using the softmax function for normalization, the normalized output probability of each semantic type is

The semantic type corresponding to the maximum probability is selected as the semantic type of the target text.

In one of the embodiments, the neural network includes a first neural network, a second neural network, and a third neural network. The training process of the neural network model includes: obtaining sample text and known labels, and determining the sample text characters contained in the sample text And the sample text segmentation to which each sample text character belongs, calculate the sample word vector corresponding to the sample text character and the sample word vector corresponding to the sample text segmentation, and divide the sample word vector corresponding to each sample text character with the sample text segmentation. The sample word vector is spliced to obtain the sample splicing vector of the corresponding sample text characters; according to the forward appearance order of the sample text characters in the sample text, the sample word vectors and sample splicing vectors corresponding to multiple sample text characters are sequentially input to the sample to be trained The first neural network obtains the first text feature of the sample; according to the reverse appearance order of the sample text characters in the sample text, the sample word vectors and sample splicing vectors corresponding to multiple sample text characters are sequentially input to the second neural network to be trained to obtain samples Second text feature;

The sample comprehensive text feature obtained by splicing the sample first text feature and the sample second text feature is input to the third neural network to be trained to obtain the predicted semantic type of the sample text; the loss value is calculated according to the predicted semantic type and the known label, The loss value is transmitted to each layer of the neural network model through the backward gradient propagation method to obtain the gradient of each layer parameter; the parameters of each layer in the neural network model are adjusted according to the gradient until the determined loss value reaches the training stop condition.

Among them, adjusting the parameters of each layer in the neural network model specifically includes adjusting the weight parameters of the fully connected layer, the weight parameters and bias parameters of each hidden layer in the first neural network and the second neural network. The function for calculating the loss value may be a cross-entropy loss function. The back gradient propagation method may be a batch gradient descent method (BGD), a small batch gradient descent method (MGBD), and a stochastic gradient descent method (SGD).

In this embodiment, the comprehensive text features are weighted through the attention mechanism layer of the third neural network layer to highlight features with higher mutual information between text characters, and weaken features with lower mutual information; further, through The random inactivation layer performs sparse processing on the weighted features to obtain sparse features after sparse processing, and performs classification operations on the sparse features through the fully connected layer to obtain the prediction probability corresponding to each semantic type, and selects the semantics corresponding to the maximum prediction probability Type is used as the semantic type of the target text, using comprehensive text features to represent the semantic features of the target text, and through weighting and sparse processing, increasing the contextual semantics of text characters, reducing the amount of computer equipment calculations, and improving the classification of target samples Accuracy.

It should be understood that, although the various steps in the flowchart of Figs. 2-3 are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless specifically stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least some of the steps in Figure 2-3 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. These sub-steps or stages The execution order of is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

In one embodiment, as shown in FIG. 4, a text semantic recognition device 400 is provided, including: a vector calculation module 402, a vector splicing module 404, a first text feature acquisition module 406, a second text feature acquisition module 408, and The semantic type acquisition module 410, where:

The vector calculation module 402 is used to calculate the word vector of each text character in the target text and the word vector of each text segmentation.

The vector splicing module 404 is used for splicing the word vector of each text character with the word vector of the corresponding text segmentation to obtain the splicing vector of the corresponding text character.

The first text feature acquisition module 406 is configured to input word vectors and splicing vectors corresponding to multiple text characters into different hidden layers of the first neural network in sequence according to the positive appearance order of the text characters in the target text, and obtain the target text based on the positive The first text feature in the order of appearance.

The second text feature acquisition module 408 is used to input word vectors and splicing vectors corresponding to multiple text characters into different hidden layers of the second neural network according to the reverse appearance order of the text characters in the target text to obtain the target text based on the reverse appearance The second text feature of the order.

The semantic type obtaining module 410 is configured to input the integrated text feature obtained by splicing the first text feature and the second text feature into the third neural network to obtain the semantic type of the target text.

In one embodiment, the first neural network inputs the word vector and the splicing vector corresponding to the text characters in the current sequence to the current hidden layer of the first neural network according to the forward appearance order of the text characters in the target text, and outputs the current hidden layer The character vector corresponding to the next sequential text character and the splicing vector are input to the next hidden layer of the first neural network, and the next hidden layer is used as the current hidden layer to iterate until the last sequential text character, and the target text is obtained based on the positive The first text feature in the order of appearance.

In one embodiment, as shown in FIG. 5, the above further includes a preset sample generation module 412, which extracts the word vector and word vector of the sample text based on the pre-trained first neural network layer; performs characterization on the word vector and the word vector respectively. Numbering; write the word vector, word vector and the corresponding character number to the preset file; calculating the word vector of the text character and the word vector corresponding to the text word segmentation includes: character numbering for each text character and text word segmentation; based on the character Number, read in the preset file the word vector corresponding to each text character and the word vector corresponding to each text word segmentation.

In one embodiment, the above-mentioned first text feature acquisition module is further configured to input the word vector and the splicing vector corresponding to the text characters in the current sequence into the current hidden layer of the first neural network according to the forward appearance order of the text characters in the target text; Input the character feature output by the current hidden layer and the word vector and splicing vector corresponding to the next sequential text character into the next hidden layer of the first neural network; use the next hidden layer as the current hidden layer, and return the characters output by the current hidden layer The character vector and the splicing vector corresponding to the next sequence of text characters are input into the next hidden layer of the first neural network until the last sequence of text characters, and the first text feature of the target text based on the forward appearance sequence is obtained.

In one embodiment, the above-mentioned second text feature acquisition module is further configured to input the first text feature, the word vector and the splicing vector corresponding to the text character in the first order to the second nerve according to the reverse appearance order of the text character in the target text. The first hidden layer of the network; input the output of the first hidden layer, the word vector and the splicing vector corresponding to the current sequential text character into the current hidden layer of the second neural network; the character feature output by the current hidden layer and the next sequential text character The corresponding word vector and splicing vector are input to the next hidden layer of the second neural network; the next hidden layer is used as the current hidden layer, and the word vector and splicing corresponding to the character feature output by the current hidden layer and the next sequential text character are returned The vector is input to the next hidden layer of the second neural network until the final sequence of text characters, and the second text feature of the target text based on the reverse appearance order is obtained.

In an embodiment, the above-mentioned first text feature acquisition module is further used to use the word vector and the output of the previous hidden layer as the input of the first sub-hidden layer, and the first sub-hidden layer is used for the corresponding The weight of each neuron node projects the character vector to obtain the first sub-character feature; the first sub-character feature and the splicing vector are used as the input of the second sub-hidden layer, and the second sub-hidden layer is used to correspond to the second sub-hidden layer The weight of each neuron node is projected on the splicing vector to obtain the second sub-character feature as the output of the current hidden layer.

In one embodiment, the above-mentioned first text feature acquisition module is also used to use the first text feature as the input of the random inactivation layer, and the inactivation layer at any time is used to set each data in the first text feature according to a preset sparse probability Perform projection to obtain a sparse feature vector as the output of the first neural network.

In one embodiment, the above-mentioned semantic type acquisition module is also used to splice the first text feature and the second text feature to obtain the comprehensive text feature of the target text; the comprehensive text feature is used as the input of the attention mechanism layer, the attention mechanism The layer is used to weight each data in the comprehensive text feature to obtain the weighted feature; the weighted feature is used as the input of the random inactivation layer, and the random inactivation layer is used to project each data in the weighted feature according to the preset sparse probability Sparse features are obtained; the sparse features are used as the input of the fully connected layer, and the fully connected layer is used to classify the sparse features to obtain the prediction probability corresponding to each semantic type; select the semantic type with the largest prediction probability as the semantic type of the target text.

In the above embodiment, the word vector corresponding to each text character and the word vector of the word segmentation of the text are obtained by calculation and vector splicing is performed to obtain the splicing vector corresponding to the text character. The splicing vector corresponding to the text character is obtained through vector splicing of the text character. To characterize the text and enhance the feature dimension of the text language. Furthermore, by inputting word vectors and splicing vectors into different hidden layers of different neural networks in forward and reverse order, relevant information of text characters can be obtained more fully, and the contextual semantics between text characters can be mined, so that the first The first text feature and the second text feature output by the neural network are spliced to obtain a comprehensive feature, which can more fully express the semantic feature of the target text and improve the accuracy of text semantic recognition.

For the specific definition of the text semantic recognition device, please refer to the above definition of the text semantic recognition method, which will not be repeated here. Each module in the above-mentioned text semantic recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The foregoing modules may be embedded in the form of hardware or independent of the processor in the computer device, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the foregoing modules.

In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. 6. The computer equipment includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store the word vector corresponding to the text characters contained in the preset file and the target text and the word vector corresponding to the text word segmentation. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program is executed by the processor to realize a text semantic recognition method.

Those skilled in the art can understand that the structure shown in FIG. 6 is only a block diagram of part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may Including more or less parts than shown in the figure, or combining some parts, or having a different part arrangement.

A computer device includes a memory and a processor, the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the text semantic recognition method provided in any one of the embodiments of the present application are implemented.

A computer-readable storage medium has a computer program stored thereon, and when the computer program is executed by a processor, the steps of the text semantic recognition method provided in any embodiment of the present application are realized.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through a computer program. The computer program can be stored in a computer readable storage medium, where the The computer-readable storage medium may be nonvolatile or volatile. When the computer program is executed, it may include the procedures of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application. Should be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims (20)

1. A text semantic recognition method, the method includes:

   Calculate the word vector of each text character in the target text and the word vector of each text segmentation;

   Splicing the word vector of each text character with the word vector of the corresponding text segmentation to obtain the splicing vector of the corresponding text character;

   According to the forward appearance order of text characters in the target text, the word vectors and splicing vectors corresponding to multiple text characters are sequentially input into different hidden layers of the first neural network to obtain the first order of the target text based on the forward appearance order. Text feature

   According to the reverse appearance order of the text characters in the target text, the word vectors and splicing vectors corresponding to multiple text characters are sequentially input into different hidden layers of the second neural network to obtain the second text feature of the target text based on the reverse appearance order ；

   The integrated text feature obtained by splicing the first text feature and the second text feature is input to a third neural network to obtain the semantic type of the target text.
2. The method according to claim 1, wherein the method further comprises:

   Get sample text;

   Extracting the word vectors and word vectors of the sample text based on the pre-trained first neural network layer;

   Perform character numbers on the word vectors and word vectors respectively;

   Writing the character vector, the word vector, and the corresponding character numbers to a preset file;

   Said calculating the word vector of the text character and the word vector corresponding to the text segmentation includes:

   Perform character numbering on each of the text characters and the text segmentation;

   Based on the character number, the word vector corresponding to each text character and the word vector corresponding to each text word segmentation are obtained from the preset file.
3. The method according to claim 1, wherein said inputting word vectors and splicing vectors corresponding to multiple text characters into different hidden layers of the first neural network in sequence according to the forward appearance order of text characters in said target text, Obtaining the first text feature of the target text based on the forward appearance sequence includes:

   According to the forward appearance sequence of the text characters in the target text, input the word vector and the splicing vector corresponding to the text character in the current sequence into the current hidden layer of the first neural network;

   Input the character vector and the splicing vector corresponding to the character feature of the current hidden layer and the next sequential text character into the next hidden layer of the first neural network;

   The next hidden layer is used as the current hidden layer to iterate until the last sequence of text characters, and the first text feature of the target text based on the forward appearance sequence is obtained.
4. The method according to claim 3, wherein the current hidden layer includes a first sub-hidden layer and a second sub-hidden layer; and the input of the word vector and the splicing vector corresponding to the current sequential text character into the current first neural network Hidden layers include:

   The word vector and the output of the previous hidden layer are used as the input of the first sub-hidden layer, and the first sub-hidden layer is used to perform the word vector according to the weight of each neuron node corresponding to the first sub-hidden layer. Project to obtain the first sub-character feature;

   The first sub-character feature and the splicing vector are used as the input of a second sub-hidden layer, and the second sub-hidden layer is used to adjust the splicing vector according to the weight of each neuron node corresponding to the second sub-hidden layer Perform projection to obtain the second sub-character feature as the output of the current hidden layer.
5. The method according to claim 3, wherein the first neural network further comprises a random inactivation layer, and the method further comprises:

   The first text feature is used as the input of the random inactivation layer, and the instant inactivation layer is used to project each data in the first text feature according to a preset sparse probability to obtain a sparse feature vector as the The output of the first neural network.
6. The method according to claim 1, wherein, according to the reverse appearance order of the text characters in the target text, the word vectors and splicing vectors corresponding to the multiple text characters are sequentially input into different hidden layers of the second neural network to obtain The second text feature of the target text based on the reverse appearance order includes:

   According to the reverse appearance order of the text characters in the target text, the first text feature, the word vector and the splicing vector corresponding to the text character in the first order are input into the first hidden layer of the second neural network to obtain the first Character characteristics of hidden layer output;

   Use the second hidden layer of the second neural network as the current hidden layer, and use the second-order text characters as the current-order text characters;

   Input the character feature output by the previous hidden layer, the word vector corresponding to the text character in the current order, and the splicing vector into the current hidden layer of the second neural network;

   The next hidden layer is used as the current hidden layer to iterate until the last sequential text characters, and the second text feature of the target text based on the reverse appearance order is obtained.
7. The method according to claim 1, wherein the third neural network layer includes an attention mechanism layer and a fully connected layer; the first text feature and the second text feature are spliced and input to the third neural network , Obtaining the semantic type of the target text includes:

   Splicing the first text feature and the second text feature to obtain a comprehensive text feature of the target text;

   Using the integrated text feature as an input of the attention mechanism layer, and the attention mechanism layer is used for weighting each data in the integrated text feature to obtain a weighted feature;

   Using the weighted feature as an input of the random inactivation layer, and the random inactivation layer is used to project each data in the weighted feature according to a preset sparse probability to obtain a sparse feature;

   Using the sparse feature as an input of a fully connected layer, and the fully connected layer is used to perform a classification operation on the sparse feature to obtain a prediction probability corresponding to each semantic type;

   The semantic type with the largest predicted probability is selected as the semantic type of the target text.
8. A text semantic recognition device, wherein the device includes:

   The vector calculation module is used to calculate the word vector of each text character in the target text and the word vector of each text word segmentation;

   The vector splicing module is used to splice the word vector of each text character with the word vector of the corresponding text segmentation to obtain the splicing vector of the corresponding text character;

   The first text feature acquisition module is used to sequentially input the word vectors and splicing vectors corresponding to multiple text characters into different hidden layers of the first neural network according to the forward appearance order of the text characters in the target text to obtain the target The text is based on the first text feature in the forward order of appearance;

   The second text feature acquisition module is used to input word vectors and splicing vectors corresponding to multiple text characters into different hidden layers of the second neural network according to the reverse appearance order of text characters in the target text to obtain the target text The second text feature based on the reverse appearance order;

   The semantic type acquisition module is configured to input the integrated text feature obtained by splicing the first text feature and the second text feature into a third neural network to obtain the semantic type of the target text.
9. A computer device includes a memory and a processor, the processor and the memory are connected to each other, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the The program instructions of the memory, wherein:

   Calculate the word vector of each text character in the target text and the word vector of each text segmentation;

   Splicing the word vector of each text character with the word vector of the corresponding text segmentation to obtain the splicing vector of the corresponding text character;

   According to the forward appearance order of text characters in the target text, the word vectors and splicing vectors corresponding to multiple text characters are sequentially input into different hidden layers of the first neural network to obtain the first order of the target text based on the forward appearance order. Text feature

   According to the reverse appearance order of the text characters in the target text, the word vectors and splicing vectors corresponding to multiple text characters are sequentially input into different hidden layers of the second neural network to obtain the second text feature of the target text based on the reverse appearance order ；

   The integrated text feature obtained by splicing the first text feature and the second text feature is input to a third neural network to obtain the semantic type of the target text.
10. The computer device according to claim 9, wherein the processor is configured to:

    Get sample text;

    Extracting the word vectors and word vectors of the sample text based on the pre-trained first neural network layer;

    Perform character numbers on the word vectors and word vectors respectively;

    Writing the character vector, the word vector, and the corresponding character numbers to a preset file;

    Said calculating the word vector of the text character and the word vector corresponding to the text segmentation includes:

    Perform character numbering on each of the text characters and the text segmentation;

    Based on the character number, the word vector corresponding to each text character and the word vector corresponding to each text word segmentation are obtained from the preset file.
11. The computer device according to claim 9, wherein the processor is configured to:

    According to the forward appearance sequence of the text characters in the target text, input the word vector and the splicing vector corresponding to the text character in the current sequence into the current hidden layer of the first neural network;

    Input the character vector and the splicing vector corresponding to the character feature of the current hidden layer and the next sequential text character into the next hidden layer of the first neural network;

    The next hidden layer is used as the current hidden layer to iterate until the last sequence of text characters, and the first text feature of the target text based on the forward appearance sequence is obtained.
12. The computer device according to claim 11, wherein the current hidden layer includes a first sub-hidden layer and a second sub-hidden layer; the processor is configured to:

    The word vector and the output of the previous hidden layer are used as the input of the first sub-hidden layer, and the first sub-hidden layer is used to perform the word vector according to the weight of each neuron node corresponding to the first sub-hidden layer. Project to obtain the first sub-character feature;

    The first sub-character feature and the splicing vector are used as the input of a second sub-hidden layer, and the second sub-hidden layer is used to adjust the splicing vector according to the weight of each neuron node corresponding to the second sub-hidden layer Perform projection to obtain the second sub-character feature as the output of the current hidden layer.
13. The computer device according to claim 11, wherein the processor is configured to:

    The first text feature is used as the input of the random inactivation layer, and the instant inactivation layer is used to project each data in the first text feature according to a preset sparse probability to obtain a sparse feature vector as the The output of the first neural network.
14. The computer device according to claim 9, wherein the processor is configured to:

    According to the reverse appearance order of the text characters in the target text, the first text feature, the word vector and the splicing vector corresponding to the text character in the first order are input into the first hidden layer of the second neural network to obtain the first Character characteristics of hidden layer output;

    Use the second hidden layer of the second neural network as the current hidden layer, and use the second-order text characters as the current-order text characters;

    Input the character feature output by the previous hidden layer, the word vector corresponding to the text character in the current order, and the splicing vector into the current hidden layer of the second neural network;

    The next hidden layer is used as the current hidden layer to iterate until the last sequential text characters, and the second text feature of the target text based on the reverse appearance order is obtained.
15. The computer device according to claim 9, wherein the third neural network layer includes an attention mechanism layer and a fully connected layer; and the processor is configured to:

    Splicing the first text feature and the second text feature to obtain a comprehensive text feature of the target text;

    Using the integrated text feature as an input of the attention mechanism layer, and the attention mechanism layer is used for weighting each data in the integrated text feature to obtain a weighted feature;

    Using the weighted feature as an input of the random inactivation layer, and the random inactivation layer is used to project each data in the weighted feature according to a preset sparse probability to obtain a sparse feature;

    Using the sparse feature as an input of a fully connected layer, and the fully connected layer is used to perform a classification operation on the sparse feature to obtain a prediction probability corresponding to each semantic type;

    The semantic type with the largest predicted probability is selected as the semantic type of the target text.
16. A computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, they are used to implement the following steps:

    Calculate the word vector of each text character in the target text and the word vector of each text segmentation;

    Splicing the word vector of each text character with the word vector of the corresponding text segmentation to obtain the splicing vector of the corresponding text character;

    According to the forward appearance order of text characters in the target text, the word vectors and splicing vectors corresponding to multiple text characters are sequentially input into different hidden layers of the first neural network to obtain the first order of the target text based on the forward appearance order. Text feature

    According to the reverse appearance order of the text characters in the target text, the word vectors and splicing vectors corresponding to multiple text characters are sequentially input into different hidden layers of the second neural network to obtain the second text feature of the target text based on the reverse appearance order ；

    The integrated text feature obtained by splicing the first text feature and the second text feature is input to a third neural network to obtain the semantic type of the target text.
17. The computer-readable storage medium according to claim 16, wherein when the program instructions are executed by the processor, they are further used to implement the following steps:

    Get sample text;

    Extracting the word vectors and word vectors of the sample text based on the pre-trained first neural network layer;

    Perform character numbers on the word vectors and word vectors respectively;

    Writing the character vector, the word vector, and the corresponding character numbers to a preset file;

    Said calculating the word vector of the text character and the word vector corresponding to the text segmentation includes:

    Perform character numbering on each of the text characters and the text segmentation;

    Based on the character number, the word vector corresponding to each text character and the word vector corresponding to each text word segmentation are obtained from the preset file.
18. The computer-readable storage medium according to claim 16, wherein when the program instructions are executed by the processor, they are further used to implement the following steps:

    According to the forward appearance sequence of the text characters in the target text, input the word vector and the splicing vector corresponding to the text character in the current sequence into the current hidden layer of the first neural network;

    Input the character vector and the splicing vector corresponding to the character feature of the current hidden layer and the next sequential text character into the next hidden layer of the first neural network;

    The next hidden layer is used as the current hidden layer to iterate until the last sequential text characters, and the first text feature of the target text based on the forward appearance order is obtained.
19. The computer-readable storage medium according to claim 18, wherein the current hidden layer includes a first sub-hidden layer and a second sub-hidden layer; when the program instructions are executed by the processor, they are also used to implement the following steps and methods :

    The word vector and the output of the previous hidden layer are used as the input of the first sub-hidden layer, and the first sub-hidden layer is used to perform the word vector according to the weight of each neuron node corresponding to the first sub-hidden layer. Projection to obtain the first sub-character feature;

    The first sub-character feature and the splicing vector are used as the input of a second sub-hidden layer, and the second sub-hidden layer is used to adjust the splicing vector according to the weight of each neuron node corresponding to the second sub-hidden layer Perform projection to obtain the second sub-character feature as the output of the current hidden layer.
20. The computer-readable storage medium according to claim 18, wherein when the program instructions are executed by the processor, they are further used to implement the following steps:

    The first text feature is used as the input of the random inactivation layer, and the instant inactivation layer is used to project each data in the first text feature according to a preset sparse probability to obtain a sparse feature vector as the The output of the first neural network.

Images