JP6002598B2 - Emphasized position prediction apparatus, method thereof, and program - Google Patents

Description

  The present invention relates to a voice synthesis technique, and more particularly to a technique for predicting a voice enhancement position.

  For example, “expressive” naturally uttered in scenes where speech is spoken according to movie scenes, scenes of storytelling of fairy tales, scenes of product promotion through TV and other media, and telephone reception scenes at call centers, etc. In “speech”, the emphasis is frequently made. Even in the synthesized sound generated by speech synthesis, the naturalness of the synthesized sound is enhanced by performing appropriate emphasis.

  As described in Non-Patent Document 1, when a specific section is emphasized and uttered, the fundamental frequency of the emphasized section is higher than that of a part uttered in a reading tone. Even if a conventional speech synthesizer constructs a speech synthesis model from various everyday expressive speech different from reading aloud, and uses that model to synthesize speech, the voice in such an emphasis section is high. I can't reproduce it enough.

  In Non-Patent Document 1, a section where the difference between the fundamental frequencies of the original voice and the synthesized voice is larger than a threshold is estimated as an emphasis section, a mark (emphasis mark) is given to the estimated emphasis section, and the emphasis marks are included. By reconstructing a speech synthesis model using learning data, the performance of voice pitch reproduction is improved.

Maeno, Nose, Kobayashi, Ijima, Nakajima, Mizuno, Yoshioka, "Study of automatic assignment of local prosodic context for enhanced speech synthesis", IEICE Technical Report, Speech Study Group, 112 (81), pp.1- 6, 2012.

  In order to perform speech synthesis by the method of Non-Patent Document 1, it is necessary to determine whether or not to add an emphasis mark to each section of the input text to be synthesized. However, in the method of Non-Patent Document 1, in order to determine whether or not to add an emphasis mark based on the difference between the original speech and the synthesized speech, the nature of the section to which the emphasis mark is attached is based on conventional language processing. And the nature of the emphasis interval based on the linguistic information of the text, which has been estimated in phonetics research, is not always the same.

  This invention is made in view of such a subject, and it aims at providing the technique which can estimate the emphasis area of an input text based on the linguistic information of an input text.

  In the present invention, an emphasized position prediction model representing the relationship between the feature amount composed of at least one of the text morphological analysis result and the dependency analysis result and the emphasized position of the text is stored. When speech synthesis is performed, a feature quantity of the same type as the feature quantity composed of at least one of the morphological analysis result and the dependency analysis result of the input text is obtained, and input using the feature quantity and the emphasized position prediction model. Identifies the highlighted position of the text.

  In the present invention, it is possible to predict the emphasis section of the input text based on the language information of the input text. Further, in the method as described in Non-Patent Document 1, the emphasis interval changes depending on the learning data at the time of model creation. In the present invention, it is possible to predict from the text an emphasis section that can cope with such a change.

The figure which shows the function structural example of an emphasis position prediction apparatus. The figure which shows the operation | movement flow of an emphasis position prediction apparatus. An example of the group of the feature-value comprised by a feature-value structure part.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram of an emphasized position prediction apparatus 100 according to the present embodiment, and FIG. 2 shows an operation flow thereof.

  The emphasis position prediction apparatus 100 of this embodiment includes a control unit 111, a feature amount configuration unit 112, an emphasis position prediction unit 113, and an emphasis position prediction model storage unit 114. The emphasized position prediction apparatus 100 may further include a category name dictionary storage unit 115 that stores a category name dictionary. The emphasized position prediction apparatus 100 is an apparatus configured by reading a predetermined program into a known computer including, for example, a CPU (central processing unit), a RAM (random-access memory), and the like. At least a part of the emphasized position prediction apparatus 100 may be configured by hardware such as an integrated circuit. The emphasized position prediction apparatus 100 executes each process under the control of the control unit 111.

  An analysis result storage unit 101 outside the emphasized position prediction apparatus 100 stores input text to which morphological analysis results and dependency analysis results are assigned as language information. The input text is a sequence composed of one or more words to be synthesized with speech, and this sequence is divided for each emphasis occurrence range.

  Morphological analysis is a technique for dividing text into words and adding lexical information such as parts of speech and reading to each word. For example, Reference 1 (Yuji Matsumoto, morphological analysis system “tea bowl”, information processing, 41 (11), pp. 1208-1214, 2000).

  Dependency analysis is a technology that uses morpheme analysis results as input, constructs a phrase consisting of multiple word strings, and predicts the dependency relation between phrases based on the part-of-speech, the appearance of words, and the relationship between words. Yes, for example, it can be implemented by the method of Reference 2 (Taku Kudo, Yuji Matsumoto, Japanese Dependency Analysis by Chunking Stage Application, IPSJ Transactions, 43 (6), pp.1834-1842, 2002) . These are conventional techniques, and detailed description thereof is omitted.

  In prosodic processing such as voice pitch, processing is often performed in units of accent phrases. An accent phrase is a sequence composed of one or more phrases. The dependency information of the first phrase and the dependency information of the last phrase of the accent phrase can be used as dependency information between accent phrases. The prediction of the accent phrase boundary can be found in, for example, Reference 3 (Nakajima, H., Miyazaki, N., Yoshida, A., Nakamura, T., Mizuno, H., “Creation and Analysis of a Japanese Speaking Style Parallel Database for Expressive Speech Synthesis ”, in Proc. Oriental COCOSDA, 2010, https://desceco.org/O-COCOSDA2010/proceedings/paper_30.pdf.).

  The emphasis occurrence range is a section composed of words or word strings determined based on a predetermined standard. Any section may be used as the emphasis occurrence range. For example, a word or a word string in a section (intonation phrase) between the pause position and the pause position can be set as one emphasis occurrence range. The pose position can be predicted by the method described in Reference Document 4 (supervised by Kogure, edited by Yamamori, “Future Netto Technology Series 4 Media Processing Technology”, pp. 76-77, Telecommunications Association). Alternatively, an accent phrase may be used as the emphasized occurrence range.

  The feature quantity constructing unit 112 reads at least one of the morpheme analysis result and the dependency analysis result corresponding to each emphasis occurrence range of the input text from the analysis result storage unit 101 (steps S11 and S12), and corresponds to the read information. Generate and output feature quantities. The feature quantity configuration unit 112 generates a feature quantity for each of the emphasis occurrence ranges of the input text. The feature amount is information that can be extracted from at least one of the morphological analysis result and the dependency analysis result. For example, a column (for example, a combined value of vectors or elements) including all of the following elements a) to e) or a combination of some of the elements is used as the feature amount. In this case, from the viewpoint of emphasized position prediction performance, it is desirable that a column including both elements a) and b) be a feature amount. Preferably, it is essential that the feature amount is a column including both elements a) and b). Moreover, it is good also as a feature-value by adding the row | line | column containing the one part or all elements of the following c-1) -c-4).

  a) Information indicating the word of interest within the emphasis generation range and the appearance form (or surface form) of N words before and after the word may be used as the feature amount element. This is information that can be extracted from the morphological analysis result. Which position is the word to be focused on with respect to the emphasized occurrence range, how many words are to be focused on with respect to each emphasized occurrence range, and what value is N Such conditions are determined in advance. For example, a preliminary experiment using learning data is performed in advance, and the condition that shows the highest performance is adopted there. For example, only the word of the main word among the words having a specific relationship with respect to the emphasized occurrence range may be set as “word of interest”, or all the words having a specific relationship with respect to the emphasized occurrence range may be referred to as “word of interest”. It is good also as. The “word having a specific relationship with the emphasized occurrence range” may be a word belonging to the emphasized occurrence range, a word within a predetermined range from the emphasized occurrence range, or the emphasized occurrence range. The word may be a predetermined distance away from. N is an integer greater than or equal to 0, and may be the same value for all input data, or may differ depending on the type of input data. These conditions are set in the same manner for the following other elements. However, the conditions in a) to e) may be the same or different.

  b) Information representing the part-of-speech of the word of interest and N words before and after it (ie, 2N + 1 words of the above a)) may be used as an element of the feature amount. This is also information that can be extracted from the morphological analysis result.

c) Information indicating the relationship between the word of interest regarding the words before and after the emphasis occurrence range and other words that are in a dependency relationship with the word of interest may be used as the element of the feature amount. This is information that can be extracted from the dependency analysis result. For example, the following can be used as elements of the feature amount.
c-1) Information representing the distance counted by the number of words from the focused word to other words at the end of the sentence related to the focused word.
c-2) Information representing the minimum distance counted by the number of words from the focused word to other words at the beginning of the sentence related to the focused word. That is, information representing the distance counted by the number of words from the focused word to the nearest other word on the sentence head related to the focused word.
c-3) Information representing the maximum distance counted by the number of words from the focused word to other words at the beginning of the sentence related to the focused word. That is, information representing the distance counted by the number of words from the focused word to the other word farthest on the sentence head related to the focused word.
c-4) Information representing the number of words on the sentence side related to the focused word.
In c-1), if the word of interest is not the word at the end of the sentence and there is no word related to the direction of the sentence end, the value of the element is set to 0. In c-2) and c-3), if the word of interest is not the word at the beginning of the sentence and there is no word at the beginning of the sentence related to the word, the value of the element is set to 0. Information obtained by replacing “end of sentence” in “c-1) with“ beginning of sentence ”may be used as an element, and information obtained by replacing“ start of sentence ”in c-2) to c-4) with“ end of sentence ”. May be an element. In addition, the dependency in c) is usually obtained in units of phrases, but when obtaining dependency information in units of accent phrases frequently used in speech synthesis, the word of interest is extracted from words before and after the accent phrase boundary. The above-mentioned distance and number to "can be used as an element of the feature amount. Note that the accent phrase boundary may be specified by the information indicating the accent phrase boundary and the presence / absence of a pose input to the feature quantity construction unit 112, or predicted by the feature quantity construction unit 112 based on the above-mentioned Reference 3 and the like. May be. The “word of interest” is an individual word within the emphasized occurrence range.

  d) Information indicating the presence or absence of a pose at positions before and after the accent phrase of interest within the emphasis generation range may be used as an element. This is information obtained based on the results of References 3 and 4. The presence / absence of a pose at the position before the accent phrase M1 before the accent phrase of interest and the presence / absence of a pose at the position behind the accent phrase M2 behind the accent phrase of interest may be included as elements. M1 and M2 are determined in advance. For example, a preliminary experiment using learning data is performed in advance, and the condition that shows the highest performance is adopted there. The “accent phrase of interest” is an accent phrase including the word of interest within the emphasized occurrence range.

  e) When the emphasized position prediction apparatus 100 includes the category name dictionary storage unit 115, information representing “word category” obtained by searching the category name dictionary using the word corresponding to the emphasized occurrence range as a key is used as the feature amount. It may be an element. For example, information representing all or a partial combination of category names other than the part of speech of the main part of the 2N + 1 words in the above a) may be used as the element of the feature amount. For example, information representing a subcategory category such as an organization name or company name to which a word belongs may be used as an element of the feature amount. Such a subcategory category can be assigned using, for example, Reference 5 (Japanese Vocabulary System, supervised by NTT Communication Science Laboratories, edited by Ikehara et al., 1997, Iwanami Shoten). The category name includes, for example, a foreign word or a Japanese word. In addition, information representing a category of a word obtained by referring to an independently constructed category name dictionary may be used as an element of the feature amount. For example, a positive category such as “bright” or “fun”, or a negative category such as “dark” or “hard” can be set. “Main part” means “main part”. That is, the “main part” means a word (word on the receiving side) that is involved in the dependency. When there are a plurality of main parts in the above 2N + 1 words, a feature amount element may be configured for only one of the main part words, or a feature amount for a plurality of main part words. The elements may be configured. When the main part does not exist in the above 2N + 1 words, information indicating that the main part does not exist may be used as the element of the feature amount.

  FIG. 3 shows an example in which a column composed of a plurality of elements is used as a feature amount. In the example of FIG. 3, each accent phrase is set as an emphasis occurrence range, the information (2) column indicating the appearance of the main word corresponding to the accent phrase), and the part of speech of the main word (3) Column), information indicating whether or not there is a pose on the front side of the accent phrase (column of 4), information indicating whether or not there is a pose on the back side of the accent phrase (column of 5), and the main part word Column of information (6) indicating the minimum value of the distance from the word to the front word related to the word by dependency, and a column of information (7) indicating the category name other than the part of speech of the main word ) As an element of feature quantity. For example, the feature quantity corresponding to accent phrase number 2 is a) as a) information representing the word “soft” and the words “clean” and “warm air” before and after the 2) column in FIG. ) "Adjective" in the column and information indicating "adjective" and "noun" before and after, c) as information 6) as information "1" in column, as d) 4) and 5) "no" and "in column It is a vector whose elements are information representing “no” and information representing “foreign word” in column 7).

  The feature value output from the feature value configuration unit 112 is input to the enhancement position prediction unit 113. The enhancement position prediction unit 113 identifies the enhancement position of the input text using the enhancement position prediction model stored in the enhancement position prediction model storage unit 114 and the feature amount obtained by the feature amount configuration unit 112 (step S114). . The emphasis position prediction model is a model that represents the relationship between the feature amount composed of at least one of the text morphological analysis result and the dependency analysis result and the text emphasis position. That is, the emphasized position prediction model is a model that represents the relationship between whether the emphasized occurrence range (section) constituting the text is the emphasized position and the feature amount, and the emphasized position predicting unit 113 is obtained by the feature amount configuring unit 112. The feature amount is used to identify whether each emphasis occurrence range constituting the input text is an emphasis position.

ただし、Ｐ（ｘ，ｙ）はｘ＝（ｘ_０，…，ｘ_Ｉ−１），ｙ＝（ｙ_０，…，ｙ_Ｉ−１）の同時確率、Ｐ（ｙ_ｉ｜ｘ_ｉ）はｘ_ｉのもとでのｙ_ｉの条件付き確率、Ｐ（ｘ_ｉ｜ｘ_ｉ−１）はｘ_ｉ−１のもとでのｘ_ｉの条件付き確率である。ｘ_−１は文の先頭の単語よりも前にあると仮定する文の開始位置という便宜上の強調予測範囲に関して得られる特徴量で定数を要素とするベクトルである。 Details are illustrated below. The feature amount obtained for each emphasis occurrence range i = 0,..., I−1 is x _i, and identification information indicating whether each emphasis occurrence range i is an emphasis position (whether an emphasis mark is added) (Identification information indicating whether or not) is y _i . However, I is an integer of 1 or more. For example, I is the number of all emphasized occurrence ranges belonging to the input text. The feature quantity x _i is a vector or the like. An example of the identification information y _i is binary information in which y _i = 1 when the emphasis occurrence range i is an emphasis position and y _i = 0 when the emphasis occurrence range i is not an enhancement position. A sequence of feature values x _i is x = (x ₀ ,..., X _I-1 ), and a sequence of identification information y _i is y = (y ₀ ,..., Y _I-1 ). In this case, the emphasized position prediction model is a model that associates the feature amount series x of the input text with the identification information series y. For example, the enhancement position prediction model is a model in which a feature quantity series x is an input variable and an identification information series y is an output variable. The enhanced position prediction model is not limited, but a specific example of the enhanced position prediction model is a probabilistic model such as a hidden Markov model used for sequence labeling or the like. The enhancement position prediction model is, for example, a feature amount composed of at least one of a morphological analysis result and a dependency analysis result of learning text, and an identification indicating whether each enhancement occurrence range of the text is an enhancement position. It can be constructed by implementing general machine learning techniques using learning data consisting of pairs with information. The configuration of the feature amount used for learning the emphasized position prediction model is the same as the configuration of the feature amount obtained by the feature amount configuration unit 112. The identification information of the learning data may be provided manually or may be automatically extracted by a conventional technique such as Non-Patent Document 1. Details of the machine learning method are described in Reference Document 6 (Daiya Takamura et al., “Introduction to Machine Learning for Language Processing”, Corona). The enhancement position prediction model, for example, takes an enhancement occurrence range as a processing unit, and inputs a series of feature amounts composed of the processing unit and feature amounts corresponding to the preceding and following units, and the emphasis mark in the processing unit is input. It is a model for predicting a series of identification information indicating presence or absence. An example of an emphasized position prediction model constructed by a hidden Markov model is shown below.

Where P (x, y) is x = (x ₀ ,..., X _I-1 ), y = (y ₀ ,..., Y _I-1 ), and P (y _i | x _i ) is x The conditional probability of y _i under _i , P (x _i | x _i−1 ) is the conditional probability of x _i under x _i−1 . x ₋₁ is a vector having a constant as an element, which is a feature amount obtained with respect to an enhanced prediction range for convenience, which is the start position of a sentence that is assumed to be before the first word of the sentence.

  The enhancement position prediction unit 113 reads the enhancement position prediction model from the enhancement position prediction model storage unit 114, sets the feature amount series x output from the feature amount configuration unit 112 as an input variable of the enhancement position prediction model, and identifies The information sequence y is predicted, and the identification information sequence y is output as an emphasized position prediction result. For example, when the emphasized position prediction model is configured by a hidden Markov model, a sequence y of identification information that maximizes P (x, y) is output with respect to the input feature amount sequence x. Such a search for the identification information sequence y can be performed using, for example, a known Viterbi algorithm. With the emphasis occurrence range as a processing unit, a series of identification information representing the presence / absence of an emphasis mark in the processing unit is input as a sequence of feature amounts composed of the unit and feature amounts corresponding to the preceding and following units. Using the emphasis position prediction model, enumerate all the possibilities with and without emphasis marks for each unit from the beginning of the sentence to the end of the sentence, and emphasize the position prediction model globally from the beginning of the sentence to the end of the sentence. A sequence having the maximum probability given by may be selected by the Viterbi algorithm. Alternatively, instead of the emphasis occurrence range, a series y of identification information may be obtained using a focused word, phrase, or accent phrase as a processing unit. That is, the processing unit for obtaining the feature amount and the processing unit for obtaining the identification information may be the same or different. Alternatively, the search range may be narrowed down to a series of identification information having a high probability between several units before and after the processing unit. It is also possible to predict the emphasized position only for the range related to the configuration of the feature quantity, and to predict the emphasized position itself. These search methods are well known, and can be performed by the method described in Reference 6, for example. Further, the emphasized position prediction model is not limited to the hidden Markov model. For example, the emphasized position prediction model may be configured by any model as long as it can provide a score and a probability value. It may be constituted by a decision tree, may be constituted by a logarithmic linear model, or may be constituted by a neural network.

  As described above, in the present embodiment, the emphasized position of the input text can be identified using the emphasized position prediction model that represents the relationship between the feature amount corresponding to the language information of the text and the emphasized position of the text. Further, the emphasized position predicted using the emphasized position prediction model of the present embodiment has a property consistent with the model of Non-Patent Document 1. As a result, it is possible to synthesize natural speech in which the pitch of the voice is accurately reproduced.

  The present invention is not limited to the embodiment described above. For example, the emphasis position prediction device may include a unit that performs morphological analysis and dependency analysis of input text, or the emphasis position prediction device includes an analysis result storage unit that stores morphological analysis results and dependency analysis results. You may have. The various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capacity of the apparatus that executes the processes. Needless to say, other modifications are possible without departing from the spirit of the present invention.

  When the above configuration is realized by a computer, the processing contents of the functions that each device should have are described by a program. By executing this program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. An example of a computer-readable recording medium is a non-transitory recording medium. Examples of such a recording medium are a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, and the like.

  This program is distributed, for example, by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, this computer reads a program stored in its own recording device and executes a process according to the read program. As another execution form of the program, the computer may read the program directly from the portable recording medium and execute processing according to the program, and each time the program is transferred from the server computer to the computer. The processing according to the received program may be executed sequentially. The above-described processing may be executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by an execution instruction and result acquisition without transferring a program from the server computer to the computer. Good.

  In the above embodiment, the processing functions of the apparatus are realized by executing a predetermined program on a computer. However, at least a part of these processing functions may be realized by hardware.

100 Emphasized position prediction device

Claims (9)

An emphasis position prediction model storage unit that stores an emphasis position prediction model representing a relationship between a feature amount composed of at least one of a text morpheme analysis result and a dependency analysis result and the emphasis position of the text;
A feature quantity component that obtains a feature quantity composed of at least one of a morphological analysis result and a dependency analysis result of the input text;
An emphasized position prediction unit for identifying an emphasized position of the input text using the feature amount obtained by the emphasized position prediction model and the feature amount configuration unit,
The feature amount includes information representing information and part of speech of the words in the head word representing the appearance form of the words in the head word, or, each occurrence form of the word of words and a predetermined number before and after a word of the head word of the head word As well as information representing each part of speech of the word,
An emphasized position predicting apparatus characterized by that. The emphasized position prediction apparatus according to claim 1,
The feature amount includes information indicating a minimum value of a distance from the word of the main word to a front word that is related to the word of the main word.
An emphasized position predicting apparatus characterized by that.   The emphasized position prediction apparatus according to claim 1 or 2,
  The feature amount includes information representing a word category,
  An emphasized position predicting apparatus characterized by that. The emphasized position prediction apparatus according to any one of claims 1 to 3 ,
The emphasized position prediction model is a model that represents a relationship between whether the section constituting the text is an emphasized position and the feature amount,
The emphasis position prediction unit identifies whether each section constituting the input text is an emphasis position.
An emphasized position predicting apparatus characterized by that. An emphasized position prediction model representing a relationship between a feature amount composed of at least one of a text morpheme analysis result and a dependency analysis result and the emphasized position of the text is stored in the emphasized position prediction model storage unit,
A feature quantity configuration step for obtaining a feature quantity composed of at least one of a morphological analysis result and a dependency analysis result of the input text;
Emphasis position prediction step for identifying an emphasis position of the input text using the feature amount obtained in the enhancement position prediction model and the feature amount configuration step;
Run
The feature amount includes information representing information and part of speech of the words in the head word representing the appearance form of the words in the head word, or, each occurrence form of the word of words and a predetermined number before and after a word of the head word of the head word As well as information representing each part of speech of the word,
An emphasized position prediction method characterized by the above.   The emphasized position prediction method according to claim 5,
  The feature amount includes information indicating a minimum value of a distance from the word of the main word to a front word that is related to the word of the main word.
  An emphasized position prediction method characterized by the above.   The emphasized position prediction method according to claim 5 or 6, comprising:
  The feature amount includes information representing a word category,
  An emphasized position prediction method characterized by the above.   The emphasized position prediction method according to any one of claims 5 to 7,
  The emphasized position prediction model is a model that represents a relationship between whether the section constituting the text is an emphasized position and the feature amount,
  The emphasized position predicting step identifies whether each section constituting the input text is an emphasized position.
  An emphasized position prediction method characterized by the above. Program for causing a computer to function claims 1 as either stress position prediction device 4.

Images