CN114416947A - Relation perception similar problem identification and evaluation method, system, equipment and storage medium - Google Patents

Relation perception similar problem identification and evaluation method, system, equipment and storage medium Download PDF

Info

Publication number
CN114416947A
CN114416947A CN202210047822.XA CN202210047822A CN114416947A CN 114416947 A CN114416947 A CN 114416947A CN 202210047822 A CN202210047822 A CN 202210047822A CN 114416947 A CN114416947 A CN 114416947A
Authority
CN
China
Prior art keywords
question
semantic
vector
relationship
relation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210047822.XA
Other languages
Chinese (zh)
Other versions
CN114416947B (en
Inventor
陈恩红
刘淇
陈彦敏
王皓
黄振亚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology of China USTC
Original Assignee
University of Science and Technology of China USTC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology of China USTC filed Critical University of Science and Technology of China USTC
Priority to CN202210047822.XA priority Critical patent/CN114416947B/en
Publication of CN114416947A publication Critical patent/CN114416947A/en
Application granted granted Critical
Publication of CN114416947B publication Critical patent/CN114416947B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0281—Customer communication at a business location, e.g. providing product or service information, consulting

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Business, Economics & Management (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Development Economics (AREA)
  • Strategic Management (AREA)
  • Accounting & Taxation (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Finance (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • General Business, Economics & Management (AREA)
  • Human Computer Interaction (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Machine Translation (AREA)

Abstract

The invention discloses a relation perception similar problem identification and evaluation method, a system, equipment and a storage medium. The results of the prediction are improved to some extent by a plurality of evaluation indexes.

Description

Relation perception similar problem identification and evaluation method, system, equipment and storage medium
Technical Field
The invention relates to the field of natural language processing, in particular to a method, a system, equipment and a storage medium for identifying and evaluating relationship perception similar problems.
Background
Similar question identification is a core question in the field of intelligent customer service question-and-answer research. When a user proposes a new question, the intelligent customer service needs to understand the new question of the user, finds out similar questions possibly matched with the user question, obtains the best matched question through a matching model, and feeds back the best matched question to the corresponding answer of the user. The user affinity question recognition task can also be modeled as an affinity text matching task. Similar question recognition tasks are applied to many fields, such as community question and answer queries, information retrieval and intelligent customer service systems, and similar question recognition is researched as a core question of the application field. Therefore, how to solve the identification of similar problems becomes a very important basic problem.
Around this research topic, researchers have proposed a variety of solutions, and most of the related researches mainly focus on lexical, syntactic or problem structures between two problems, and the similarity of the two problems is determined by modeling the semantic relationship of the two problems.
However, similar problem matching faces a diverse challenge due to the short length of the problem and the flexibility and breadth of natural language expressions. In order to solve the diversity of the problem, some researches have introduced external knowledge such as knowledge graph, problem answer, etc. to solve the situation of insufficient diversity. However, these external knowledge have a wide range of fields or are not targeted, and cannot completely meet the supplement of problem diversity expression, so that the recognition accuracy still needs to be improved.
Disclosure of Invention
The invention aims to provide a relation perception similar problem identification and evaluation method, a system, equipment and a storage medium, which can fully utilize semantic relation information among a plurality of problem pairs to solve similar problem identification among problems and have higher prediction precision.
The purpose of the invention is realized by the following technical scheme:
a relation-aware similar problem identification and evaluation method comprises the following steps:
extracting matched semantically related question data from the data set under the condition of multiple semantic relations, wherein each matched semantically related question data is text data and comprises a verification question pair QuAnd QaAnd problem QaA plurality of semantic related problem sets T under different corresponding matching relations;
constructing a relation perception neural network-based similar problem recognition model, performing joint representation on each matched semantic-related problem data, and utilizing the obtained problem QaIs associated with the problem QuThe expression vector is identified and evaluated, in a training stage, a loss function is constructed by using an identification and evaluation result and an identification label, and model parameters are updated;
and in the testing stage, for a given problem pair, the trained recognition model based on the relation perception neural network similarity problem is used for recognition and evaluation.
A relationship-aware similarity problem identification and evaluation system for implementing the foregoing method, the system comprising:
a data extraction unit for extracting matched semantically related question data from the data set under multiple semantic relations, wherein each matched semantically related question data is text data and contains verification question pair QuAnd QaQuestion QaAnd problem QaA plurality of semantic related problem sets T under different corresponding matching relations;
a model construction and training unit for constructing a relation perception neural network-based similar problem recognition model, performing joint characterization on each matched semantically related problem data, and utilizing the obtained problem QaIs associated with the problem QuThe representative vector of (a) is subjected to an identification evaluation,in the training stage, a loss function is constructed by using the identification evaluation result and the identification label, and model parameters are updated;
and the recognition evaluation testing unit is used for performing recognition evaluation on a given problem pair by using the trained relation perception-based neural network similarity problem recognition model.
A processing device, comprising: one or more processors; a memory for storing one or more programs;
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the aforementioned methods.
A readable storage medium, storing a computer program, characterized in that the computer program realizes the aforementioned method when executed by a processor.
According to the technical scheme provided by the invention, the relation perception neural network-based similar problem recognition model is used for recognizing the similar problems of the problem pairs, and compared with the traditional model, the semantic relation related information of a plurality of semantic matches is used. The results of the prediction are improved to some extent by a plurality of evaluation indexes.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a flowchart of a method for identifying and evaluating relationship-aware similarity problems according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a relationship-aware similarity problem identification and evaluation system according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a processing apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
Terms that may be used herein are first described as follows.
The terms "comprising," "including," "containing," "having," or other similar terms of meaning should be construed as non-exclusive inclusions. For example: including a feature (e.g., material, component, ingredient, carrier, formulation, material, dimension, part, component, mechanism, device, process, procedure, method, reaction condition, processing condition, parameter, algorithm, signal, data, product, or article of manufacture), is to be construed as including not only the particular feature explicitly listed but also other features not explicitly listed as such which are known in the art.
Next, the existing records of similar results are analyzed and explained.
The analysis shows that the existing data set has the following condition that the same problem and a plurality of problems have semantic relations. For example, as shown in table 1, there is a semantic relationship of such multiple matching problems in a customer service problem data set of a certain bank. Wherein Q1Respectively and 4 questions (i.e., Q in Table 1)2～Q5) There is a semantic relationship. A classification label of 1 indicates that the two questions have the same semantics, and a classification label of 0 indicates that the two questions are similar, but the semantics are not the same. Specifically, Q2And Q3And Q1The label of (1) is given, and Q is obtained by analyzing the question content2And Q3Although the expressions are different in terms of sentence patterns, the expressed basic semantics are the same and are all information about the "telephone audit time", so that Q2And Q3Is Q1Multiple semantic matching problems. Q4And Q5And Q1Is labeled 0, assay Q4And Q5It is known that Q4And Q5These two problems and Q1Having the same key words but different semantics of expression, so Q4And Q5Is Q1Multiple semantically similar but not matching problems.
Figure BDA0003473066610000041
TABLE 1 multiple semantic matching problem examples
Thus, by observing the data set, it can be seen that under a label-based classification relationship, there is a pair of Q1A plurality of semantically matched semantically related questions. The semantic influence of the semantic matching relations on the basic problem is reflected on finer granularity by the semantic matching situations under the two different relations. In contrast to matching labels used only to distinguish problems in previous studies, in fact the matching labels of the problems reflect the implicit semantic relationship that the problem objects have between them. The matching relations have strong semantic relevance and can help the semantic understanding of the problem. In order to show the positive effects of multiple matching semantic relations on similar problem identification, the matching semantic relations are used for representing semantic information between problem pairs, and the method is the key point of research.
The relationship-aware similar problem identification and evaluation method provided by the invention is described in detail below. Details which are not described in detail in the embodiments of the invention belong to the prior art which is known to the person skilled in the art. Those not specifically mentioned in the examples of the present invention were carried out according to the conventional conditions in the art or conditions suggested by the manufacturer. The instruments used in the examples of the present invention are not indicated by manufacturers, and are all conventional products that can be obtained by commercial purchase.
As shown in fig. 1, a method for identifying and evaluating similar problems of relationship perception mainly includes the following steps:
step 1, extracting matched semantically related problem data from a data set under the condition of a plurality of semantic relations, wherein each matched semantically related problem data is text dataIncluding verification problem pair QuAnd QaAnd problem QaAnd a plurality of semantic related problem sets T under corresponding different matching relations.
The preferred embodiment of this step is as follows:
1) the problem of finding that there are multiple semantic matches.
In the embodiment of the invention, problems related to matching are extracted from collected data, each problem and a plurality of other problems form a similar problem matching relation, the similar problem matching relation comprises two categories, and a first category label is 1, which indicates that the two problems have the same semantic meaning and belong to a semantic matching relation; the second classification label is 0, which indicates that the two questions have the same keywords but different semantics and belong to a semantic similarity relationship.
2) The data set is partitioned.
In the embodiment of the invention, in order to further divide a plurality of semantic matching problem sets and verification model data sets, the existing data is divided into two parts: the first part is used as a matching problem knowledge base; the other part is used as the training data of the verification model; wherein the matching problem knowledge base provides QaWith set T, the training data of the verification model, provide problem Qu(ii) a During training, the verification data finds a plurality of semantic matching problem sets corresponding to the problems from the matching problem knowledge base to realize semantic uniform expression of the corresponding problems.
Specifically, the method comprises the following steps: problem Q in training datauAnd matching individual questions Q in the question knowledge baseaWhen matching, for problem QaIf the matching problem knowledge base M ═ S (S)1,S2,..) of any other problem or problem QaWith a classification label of 1, put the corresponding question into the set
Figure BDA0003473066610000051
TpRepresents a problem QaN is the set TpThe number of questions in; if the classification label is 0, put into the set
Figure BDA0003473066610000052
TqRepresents a problem QaM is a set TqThe number of questions in; set T ═ Tp,TqGet it as the question QaAnd a plurality of semantically related problem sets under corresponding different matching relations.
3) And processing text content.
In the embodiment of the invention, each matched semantic related question data obtained in the process is participled to obtain a verification question pair QuAnd QaThe above two types of word segmentation results are used as input of model training and used for evaluation of similar problem identification of each problem pair.
Step 2, constructing a relation perception neural network-based similar problem recognition model, performing joint characterization on each matched semantically related problem data, and utilizing the obtained problem QaIs associated with the problem QuAnd in the training stage, a loss function is constructed by using the identification and evaluation result and the identification label, and the model parameters are updated.
The preferred embodiment of this step is as follows:
1) and constructing a relation-aware neural network-based similar problem identification model.
In the embodiment of the invention, the relation perception neural network-based similar problem identification model mainly comprises the following steps: the system comprises a problem representation layer, a relation perception representation layer and a problem identification evaluation layer; wherein: a) the question representation layer is used for extracting each verification question pair QuAnd QaIs represented vector, and question QaRepresenting vectors of each question in a plurality of semantically related question sets under corresponding different matching relations; illustratively, the processing stages may use the sequence-BERT model. b) The relation perception representation layer is used for combining the question QaObtaining the semantic relation with each question in the question set T to obtain the question QaThe relationship-aware representation vector of (a) and, in particular,this stage is to integrate multiple semantic matching relationship information into question QaThe expression vector (2) is a unified expression vector for obtaining the problem of relation perception. c) The question recognition and evaluation layer is used for utilizing the question QuAnd problem QaIs represented vector, and question QaThe relationship perception of (1) represents a vector, for a verification problem pair QuAnd QaAnd (4) performing identification and evaluation on the semantic relationship, and predicting a similar problem identification label of the verification problem pair.
The following is a description of the principles and associated processes for each layer.
a) A problem presentation layer.
All questions need to be input to the question representation layer, including the question Q mentioned earlieruQuestion matching problem Q in the knowledge baseaAnd about problem QaProblem set T ═ Tp,Tq}. At the question representation level, the question is represented as a dense vector in d-dimension by having pre-trained the initial word vector. Thus, for any question to capture the context between question words, the Sennce BERT model is used to encode each question independently and map them to a dense vector, generating a semantic representation of the question.
h＝Sentence-BERT(Q)
Wherein the sequence-BERT represents a sequence BERT model, h is a representation vector of the problem Q,
Figure BDA0003473066610000064
| Q | is the length of the problem Q, dhIs the dimension of the output.
For verification problem pair QuAnd QaObtaining a corresponding representation vector h by a problem representation layeruAnd ha(ii) a For problem set T ═ Tp,TqObtaining semantic matching relation vector set through problem representation layer
Figure BDA0003473066610000061
Vector set with semantic similarity relation
Figure BDA0003473066610000062
b) A relationship-aware representation layer.
In the relation perception representation layer, the plurality of semantic relations comprise semantic matching relations and semantic similar relations, and each semantic relation is related to the question QaMultiple semantic matching problems. For multiple semantic matching problems, RNs (relational networks) are used first to capture the relationship vectors of two problems, and then RNs are expanded to discuss the integration of multiple relationship labels to semantic matching and semantic similarity.
In order to clarify the relation semantics in the question and its associated sentences, an RNs relation network structure is used to represent the relation between two objects. Expressing question Q by RNs with question as relational objectaMultiple relation vector h corresponding to the problemk. According to the RNs model, the function of model relationship characterization:
RNs(Hr)＝fφ(∑gθ(ha,hk))
wherein, gθAnd fφIs a Multi-Layer Perceptron network (MLP), haIs a problem QaRepresents a vector of hkA representative vector representing a single problem in the set of problems T; h isk∈Hpor Hq，Hp、HqAre respectively a set Tp、TqThe set of representation vectors of the question.
To obtain haAnd hkThe interaction characteristics between the elements are realized by adopting an element-wise product method and utilizing haAnd hkAnd (3) performing product operation on corresponding position elements:
Figure BDA0003473066610000063
thus, the function of the relational representation of the problem can be rewritten as:
Figure BDA0003473066610000071
wherein HrObtaining Q about a problem through an RNs networkaThe multi-semantic matching vector obtains similar semantic features of corresponding words between two sentences by using dot product operation of elements, and obtains core interactive features expressed by a plurality of relations by using an RNs network structure.
In order to embody the implicit semantics of the matching relationship between different labels, the part extends the function of the basic problem relationship to the multi-relationship semantic matching representation. Combining the label of each question in the question set T, calculating the following matching expression function under multiple labels:
Figure BDA0003473066610000072
wherein,
Figure BDA0003473066610000073
o∈|Nr|，Nrset of semantic vectors for multiple semantic matching problems under the r-th relationship, | NrI represents a semantic vector set NrR represents a set of categories of labels. Intuitively, the above formula encodes the label feature vectors of multiple sentences by normalization.
Set H of vectors based on representationpAnd HqAnd expanding the matching expression function under the multiple labels to obtain:
Figure BDA0003473066610000074
wherein, WpiAnd WqjThe learning weights under two relationships, bpiAnd bqjIs a deviation parameter;
Figure BDA0003473066610000075
Figure BDA0003473066610000076
each being a representative vector of the corresponding problem.
Computing by adopting an element-wise product method to respectively obtain haAnd
Figure BDA00034730666100000712
h betweenaAnd
Figure BDA00034730666100000711
the interaction characteristics between:
Figure BDA0003473066610000077
Figure BDA0003473066610000078
obtaining:
Figure BDA0003473066610000079
wherein v isrIs a problem QaThe relationship perception of (a) represents a vector. From the above principle, the problem QaSemantic matching characterizations of multiple relationships are obtained through the RNs network.
c) And a problem identification and evaluation layer.
The goal of the question recognition evaluation layer is to evaluate the language match of each question pair. First, question QuAnd problem QaIs represented vector, and question QaRepresents a vector join, represented as:
Figure BDA00034730666100000710
wherein h isu、ha、vrSequentially represents the problem QuIs a representative vector, question QaIs a representative vector, question QaThe relationship perception of (a) represents a vector.
Then, by RELU excitationOperation of live function and sigmoid function to obtain verification problem pair QuAnd QaSemantic relation R (Q)u,Qa) Expressed as:
oau＝ReLU(W1zau+b1),
R(Qu,Qa)＝σ(W2oau+b2)
wherein o isauσ () is sigmoid function, W, which is the result of the RELU activation function1，W2，b1，b2Are network parameters. Semantic relationship R (Q)u,Qa) I.e. the identification and evaluation result, and the similar label of the verification problem pair.
2) And (5) training a model.
In the embodiment of the present invention, all W and b parameter matrices or vectors (i.e., model parameters) involved in all the above descriptions in the last part of the constructed relationship-aware-based neural network model are trained, and a cross entropy loss function is used as a final optimization target, where the loss function is expressed as:
Figure BDA0003473066610000081
where θ is the model parameter to be updated, ylIs the true identification label for the ith validation question pair, N is the number of training instances (i.e., validation question pairs);
Figure BDA0003473066610000082
indicating the ith pair of verification questions,
Figure BDA0003473066610000083
representing the identification evaluation result of l verification problem pairs, and adding l 2-norm as a training parameter considering the complexity of the model, wherein lambda isθIs a regularization hyper-parameter.
By optimizing the above loss function, an optimum state can be learned. Illustratively, in the whole training process, Adam is used as an optimizer, and the learning rate is 0.0005; in the training process, firstly, obtained data is divided into a training data and a problem matching knowledge base according to the principle of 20% and 80%, then, in the training data, a training set, a verification set and a test set are divided according to the proportion of 60%, 20% and 20%, the training set and the verification set are used for optimizing parameters of a model, and the test set is used for verifying the model.
And 3, in the testing stage, for a given problem pair, carrying out identification evaluation by using a trained recognition model based on the relation perception neural network similarity problem.
After model training is completed through the foregoing steps, Q is paired for a given new problemu’And Qa’Likewise, for problem Qa’Searching a plurality of semantically related problem sets T' under different matching relations through a matching problem knowledge base, processing text contents, inputting the processed text contents into a trained recognition model based on relation perception neural network similarity problems, and processing the processed text contents according to the above mode. And obtaining the similar labels of the new problem pairs, and realizing the similar problem identification and evaluation of the problem pairs.
According to the scheme of the embodiment of the invention, the message is subjected to problem pair similarity problem identification based on the relation perception neural network similarity problem identification model, and compared with the traditional model, a plurality of semantic relation related information matched with semantics are used. The results of the prediction are improved to some extent by a plurality of evaluation indexes.
Another embodiment of the present invention further provides a relationship-aware similar problem identification and evaluation system, which is mainly used to implement the method provided in the foregoing embodiment, as shown in fig. 2, the system mainly includes:
a data extraction unit for extracting matched semantically related question data from the data set under multiple semantic relations, wherein each matched semantically related question data is text data and contains verification question pair QuAnd QaAnd problem QaA plurality of semantic related problem sets T under different corresponding matching relations;
a model construction and training unit for constructing a recognition model based on the relation perception neural network similarity problemJointly characterizing each matching semantically related question data and utilizing the obtained question QaIs associated with the problem QuThe expression vector is identified and evaluated, in a training stage, a loss function is constructed by using an identification and evaluation result and an identification label, and model parameters are updated;
and the recognition evaluation testing unit is used for performing recognition evaluation on a given problem pair by using the trained relation perception-based neural network similarity problem recognition model.
It will be clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be performed by different functional modules according to needs, that is, the internal structure of the system is divided into different functional modules to perform all or part of the above described functions.
In addition, the specific technical details related to each unit of the system have been described in detail in the previous embodiment of the method, and thus are not described again.
Another embodiment of the present invention further provides a processing apparatus, as shown in fig. 3, which mainly includes: one or more processors; a memory for storing one or more programs; wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the methods provided by the foregoing embodiments.
Further, the processing device further comprises at least one input device and at least one output device; in the processing device, a processor, a memory, an input device and an output device are connected through a bus.
In the embodiment of the present invention, the specific types of the memory, the input device, and the output device are not limited; for example:
the input device can be a touch screen, an image acquisition device, a physical button or a mouse and the like;
the output device may be a display terminal;
the Memory may be a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as a disk Memory.
Another embodiment of the present invention further provides a readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the method provided by the foregoing embodiment.
The readable storage medium in the embodiment of the present invention may be provided in the foregoing processing device as a computer readable storage medium, for example, as a memory in the processing device. The readable storage medium may be various media that can store program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (10)

1. A relation-aware similar problem identification and evaluation method is characterized by comprising the following steps:
extracting matched semantically related question data from the data set under the condition of multiple semantic relations, wherein each matched semantically related question data is text data and comprises a verification question pair QuAnd QaAnd problem QaA plurality of semantic related problem sets T under different corresponding matching relations;
constructing a relation perception neural network-based similar problem recognition model, performing joint representation on each matched semantic-related problem data, and utilizing the obtained problem QaIs associated with the problem QuThe expression vector is identified and evaluated, in a training stage, a loss function is constructed by using an identification and evaluation result and an identification label, and model parameters are updated;
and in the testing stage, for a given problem pair, the trained recognition model based on the relation perception neural network similarity problem is used for recognition and evaluation.
2. The relationship-aware similar problem identification and assessment method according to claim 1, wherein said extracting matching semantically related problem data under multiple semantic relationship conditions from a data set comprises:
extracting problems related to matching from collected data, wherein each problem and a plurality of other problems form a similar problem matching relation, the similar problem matching relation comprises two categories, and a first category label is 1, indicates that the two problems have the same semantic meaning and belong to a semantic matching relation; the second classification label is 0, which indicates that the two problems have the same keywords but different semantics and belong to a semantic similarity relationship;
the extracted problem is divided into two parts: the first part is used as a matching problem knowledge base; the other part is used as the training data of the verification model; extracting problem Q in training datauAnd matching individual questions Q in the question knowledge baseaForm a verification question pair for question QaIf any other questions in the question knowledge base are matched with QaIf the classification label of (1) is set, the set T will be put inpIf the classification label is 0, put into the set Tq(ii) a Set T ═ Tp,TqGet it as the question QaA plurality of semantic related problem sets under corresponding different matching relations;
then, each matched semantically related question data is participled to obtain a verification question pair QuAnd QaAnd the text content word segmentation result of each question in the question set T.
3. The relationship-aware similar problem identification and assessment method according to claim 1, wherein the relationship-aware neural network-based similar problem identification model comprises: the system comprises a problem representation layer, a relation perception representation layer and a problem identification evaluation layer; wherein:
the question representation layer is used for extracting each verification question pair QuAnd QaIs represented vector, and question QaRepresenting vectors of each question in a plurality of semantically related question sets under corresponding different matching relations;
the relation perception representation layer is used for combining the question QaObtaining the semantic relation with each question in the question set T to obtain the question QaThe relationship-aware representation vector of (1);
the question recognition and evaluation layer is used for utilizing the question QuAnd problem QaIs represented vector, and question QaThe relationship perception of (1) represents a vector, for a verification problem pair QuAnd QaThe semantic relationship of the user is identified and evaluated.
4. The relationship-aware similar problem identification and evaluation method according to claim 3, wherein the problem representation layer performs independent coding on each problem to generate a representation vector of the problem:
h＝Sentence-BERT(Q)
wherein h is a representative vector of the problem Q; the sequence-BERT represents the sequence BERT model;
for verification problem pair QuAnd QaObtaining a corresponding representation vector h by a problem representation layeruAnd ha(ii) a For problem set T ═ Tp,TqObtaining a semantic matching relationship vector set H through a problem representation layerpVector set H with semantic similarity relationq(ii) a Wherein, the set TpAnd TqProblem of and QaThe classification labels of (1) and (0) respectively represent that two problems belong to a semantic matching relationship, and the classification label of (1) represents that the two problems belong to a semantic similar relationship.
5. The relation-aware similar problem identification and evaluation method according to claim 3, wherein the combined problem Q isaExtracting the semantic relation with each question in the question set T to obtain a question QaStep package of relationship-aware representation vectorComprises the following steps:
representing a network characterization problem Q using RNs relationshipsaThe relationship to each question in the set of questions T represents:
Figure FDA0003473066600000021
wherein, gθIs a multi-layer perceptron network and is,
Figure FDA0003473066600000022
represents h obtained by adopting element-wise product method to carry out operationaAnd hkInter-working characteristics between haIs a problem QaRepresents a vector of hkA representative vector representing a single problem in the set of problems T; h isk∈Hpor Hq，Hp、HqAre respectively a set Tp、TqSet of representation vectors, set T, of questionpEach problem in (1) and problem QaThe classification label of (1) represents that the two problems belong to a semantic matching relationship; set TqEach problem in (1) and problem QaThe classification label of (2) is 0, which represents that the two problems belong to semantic similarity relation;
combining the label of each question in the question set T, calculating the following matching expression function under multiple labels:
Figure FDA0003473066600000023
wherein,
Figure FDA0003473066600000024
Nra semantic vector set which is a plurality of semantic matching questions under the r-th relation; | NrI represents a semantic vector set NrR represents a set of categories of labels;
set H of vectors based on representationpAnd HqPerforming the matching representation function under the multiple labelsAnd (3) opening to obtain:
Figure FDA0003473066600000025
wherein, WpiAnd WqjThe learning weights under two relationships, bpiAnd bqjIs a deviation parameter;
Figure FDA0003473066600000026
Figure FDA0003473066600000031
each being a representative vector of the respective problem;
computing by adopting an element-wise product method to respectively obtain haAnd
Figure FDA0003473066600000032
h betweenaAnd
Figure FDA0003473066600000033
the interaction characteristics between:
Figure FDA0003473066600000034
Figure FDA0003473066600000035
obtaining:
Figure FDA0003473066600000036
wherein v isrIs a problem QaThe relationship perception of (a) represents a vector.
6. According to claim 3The relation-aware similar problem identification and evaluation method is characterized in that the problem Q is useduAnd problem QaIs represented vector, and question QaThe relationship perception of (1) represents a vector, for a verification problem pair QuAnd QaThe identification and evaluation of the semantic relationship comprises the following steps:
will question QuAnd problem QaIs represented vector, and question QaRepresents a vector join, represented as:
Figure FDA0003473066600000037
wherein h isu、ha、vrSequentially represents the problem QuIs a representative vector, question QaIs a representative vector, question QaThe relationship-aware representation vector of (1);
obtaining verification problem pairs Q through calculation of RELU activation function and sigmoid functionuAnd QaSemantic relation R (Q)u,Qa) Expressed as:
oau＝ReLU(W1zau+b1),
R(Qu,Qa)＝σ(W2oau+b2)
wherein o isauσ () is sigmoid function, W, which is the result of the RELU activation function1，W2，b1，b2Are network parameters.
7. The relationship-aware similarity problem identification and assessment method according to any one of claims 1 to 6, wherein said loss function is expressed as:
Figure FDA0003473066600000038
wherein, ylIs the true identification tag for the ith verification problem pair, N is the trainingVerifying the number of problem pairs;
Figure FDA0003473066600000039
indicating the ith pair of verification questions,
Figure FDA00034730666000000310
denotes the result of the identification evaluation of the l verification problem pairs, lambdaθTo regularize the hyper-parameters, θ is the model parameter to be updated.
8. A relationship-aware similarity problem identification and assessment system for implementing the method of any one of claims 1 to 7, the system comprising:
a data extraction unit for extracting matched semantically related question data from the data set under multiple semantic relations, wherein each matched semantically related question data is text data and contains verification question pair QuAnd QaQuestion QaAnd problem QaA plurality of semantic related problem sets T under different corresponding matching relations;
a model construction and training unit for constructing a relation perception neural network-based similar problem recognition model, performing joint characterization on each matched semantically related problem data, and utilizing the obtained problem QaIs associated with the problem QuThe expression vector is identified and evaluated, in a training stage, a loss function is constructed by using an identification and evaluation result and an identification label, and model parameters are updated;
and the recognition evaluation testing unit is used for performing recognition evaluation on a given problem pair by using the trained relation perception-based neural network similarity problem recognition model.
9. A processing device, comprising: one or more processors; a memory for storing one or more programs;
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-7.
10. A readable storage medium, storing a computer program, characterized in that the computer program, when executed by a processor, implements the method according to any of claims 1 to 7.
CN202210047822.XA 2022-01-17 2022-01-17 Method, system, equipment and storage medium for identifying and evaluating relationship perception similarity problem Active CN114416947B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210047822.XA CN114416947B (en) 2022-01-17 2022-01-17 Method, system, equipment and storage medium for identifying and evaluating relationship perception similarity problem

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210047822.XA CN114416947B (en) 2022-01-17 2022-01-17 Method, system, equipment and storage medium for identifying and evaluating relationship perception similarity problem

Publications (2)

Publication Number Publication Date
CN114416947A true CN114416947A (en) 2022-04-29
CN114416947B CN114416947B (en) 2024-10-18

Family

ID=81273685

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210047822.XA Active CN114416947B (en) 2022-01-17 2022-01-17 Method, system, equipment and storage medium for identifying and evaluating relationship perception similarity problem

Country Status (1)

Country Link
CN (1) CN114416947B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110765240A (en) * 2019-10-31 2020-02-07 中国科学技术大学 Semantic matching evaluation method for multiple related sentence pairs
CN111639171A (en) * 2020-06-08 2020-09-08 吉林大学 Knowledge graph question-answering method and device
CN112100348A (en) * 2020-09-01 2020-12-18 武汉纺织大学 Knowledge base question-answer relation detection method and system of multi-granularity attention mechanism
CN112100351A (en) * 2020-09-11 2020-12-18 陕西师范大学 Method and equipment for constructing intelligent question-answering system through question generation data set
CN112800203A (en) * 2021-02-05 2021-05-14 江苏实达迪美数据处理有限公司 Question-answer matching method and system fusing text representation and knowledge representation
WO2021184311A1 (en) * 2020-03-19 2021-09-23 中山大学 Method and apparatus for automatically generating inference questions and answers

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110765240A (en) * 2019-10-31 2020-02-07 中国科学技术大学 Semantic matching evaluation method for multiple related sentence pairs
WO2021184311A1 (en) * 2020-03-19 2021-09-23 中山大学 Method and apparatus for automatically generating inference questions and answers
CN111639171A (en) * 2020-06-08 2020-09-08 吉林大学 Knowledge graph question-answering method and device
CN112100348A (en) * 2020-09-01 2020-12-18 武汉纺织大学 Knowledge base question-answer relation detection method and system of multi-granularity attention mechanism
CN112100351A (en) * 2020-09-11 2020-12-18 陕西师范大学 Method and equipment for constructing intelligent question-answering system through question generation data set
CN112800203A (en) * 2021-02-05 2021-05-14 江苏实达迪美数据处理有限公司 Question-answer matching method and system fusing text representation and knowledge representation

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
吴炎;王儒敬;: "基于BERT的语义匹配算法在问答系统中的应用", 仪表技术, no. 06, 15 June 2020 (2020-06-15) *

Also Published As

Publication number Publication date
CN114416947B (en) 2024-10-18

Similar Documents

Publication Publication Date Title
Ashraf et al. A globally regularized joint neural architecture for music classification
CN115688024A (en) Network abnormal user prediction method based on user content characteristics and behavior characteristics
Farazi et al. Accuracy vs. complexity: a trade-off in visual question answering models
CN112579666A (en) Intelligent question-answering system and method and related equipment
CN112287239B (en) Course recommendation method and device, electronic equipment and storage medium
CN114662652A (en) Expert recommendation method based on multi-mode information learning
Sangeetha et al. Sentiment analysis of amazon user reviews using a hybrid approach
Azam et al. Multivariate‐bounded Gaussian mixture model with minimum message length criterion for model selection
Lin et al. Lifelong Text-Audio Sentiment Analysis learning
CN112541091A (en) Image searching method, device, server and storage medium
CN117648916A (en) Text similarity recognition model training method and text related information acquisition method
CN113468311B (en) Knowledge graph-based complex question and answer method, device and storage medium
CN114416947A (en) Relation perception similar problem identification and evaluation method, system, equipment and storage medium
Omarov et al. Bidirectional long-short-term memory with attention mechanism for emotion analysis in textual content
CN115455286A (en) Self-attention recommendation method, device and equipment based on data blood margin characteristics and behavior semantic characteristics
Naithani et al. Novel ABC: Aspect Based Classification of Sentiments Using Text Mining for COVID-19 Comments
CN116010563A (en) Multi-round dialogue data analysis method, electronic equipment and storage medium
Sujithra et al. An intellectual decision system for classification of mental health illness on social media using computational intelligence approach
Назаренко et al. Neural network approach for emotional recognition in text
CN116662536B (en) Abstract extraction method and related device
Mansur et al. Text Analytics and Machine Learning (TML) CS5604 Fall 2019
Manshani et al. Sentiment Analysis: A comparative study of Deep Learning and Machine Learning
Karpagam et al. Multimodal Fusion for Precision Personality Trait Analysis: A Comprehensive Model Integrating Video, Audio, and Text Inputs
Anh et al. Artificial Intelligence for Automated Pricing Based on Product Descriptions
AlGhalibi Emotion and Sentiment Detection in Unstructured Social Data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant