JP4829477B2 - Voice quality conversion device, voice quality conversion method, and voice quality conversion program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method of converting free voice quality which does not depend upon voiced contents of a conversion target speaker. <P>SOLUTION: A speech signal of the target speaker is inputted to an input part 11 and a pronunciation symbol string of voiced contents which are the same with or similar to the target speaker's voice is inputted to a pronunciation symbol string input part 12. A speech synthesis part 14 generates a synthesized sound by using a database for speech synthesis in a data storage part 13 for speech synthesis according to the inputted pronunciation symbol string. A feature parameter extraction part 15 extracts feature parameters by analyzing the target speaker's voice and a feature parameter extraction part 16 extracts feature parameters by analyzing the generated synthesized voice. A conversion function generation part 17 uses both the extracted feature parameters to identify a function of converting a spectrum shape of the synthesized voice into a spectrum shape of the target speaker's voice. A voice quality conversion part 18 converts the voice quality of the input signal with the identified conversion function. <P>COPYRIGHT: (C)2005,JPO&amp;NCIPI

Description

  The present invention relates to a voice quality conversion device, a voice quality conversion method, and a voice quality conversion program.

  Conventionally, research has been conducted on a voice quality conversion technique for converting a voice uttered by a speaker into a voice having a voice quality of another speaker. For example, Patent Document 1 discloses a technique for converting the voice quality of the speaker A into the voice quality of the speaker B. The voice quality conversion method in Patent Document 1 is shown in FIG. In this voice quality conversion method, the voice of the speaker A is analyzed by the LPC analysis 101 and quantized by the vector quantization 105 using the codebook 103 of the speaker A. Further, the voice of the speaker B is analyzed by the LPC analysis 102, and is quantized by the vector quantization 106 using the codebook 104 of the speaker B. Each quantized data is matched (mapped) by matching 107 by DP matching with time axis matching, and a spectrum conversion code book 109 is created by a conversion code book creation 108 using a histogram based on the mapping. To do. The voice quality of the speaker A is converted using the spectrum conversion code book 109.

  Furthermore, in recent years, in a speech synthesizer that outputs the contents of input text as speech using a speech synthesis database generated and accumulated from speech data of one or more speakers, speech having a desired voice quality can be obtained. Voice quality conversion technology for synthesis is also being studied. The advantage of this method is that when a synthesized sound having a certain voice quality is generated, it is not necessary to create and store a database by recording voice data of a speaker having that voice quality each time, and one or a plurality of standard stories. If a user database is stored in advance, a synthesized sound having a desired voice quality can be generated by converting the voice quality of the database so as to have a desired voice quality.

  For example, in Patent Document 2, a mapping code book between a speech database stored in advance and a target speaker's speech is created, and a speech database converted as a conversion function is used as a text speech synthesis database. Is disclosed. The voice quality conversion method in Patent Document 2 is shown in FIG. A speech database including acoustic feature parameters of a plurality of registered speakers is stored in advance. This voice quality conversion device stores a speech database 203 including acoustic feature parameters of a plurality of registered speakers and its code book 204 in advance. Based on the input speech signal of at least one word of the target speaker, a selection unit 201 that selects a speaker closest to the target speaker to be subjected to voice quality conversion from a plurality of registered speakers, and a selection unit 201 Generating means 202 for calculating a mapping codebook 205 from the selected speaker to the target speaker by calculating a difference between the acoustic space of the speaker selected by and the acoustic space of the target speaker; Based on the input character string to be synthesized, the acoustic feature parameters of the selected speaker's speech stored in the speech database 203 are quantized using the selected speaker's codebook and selected. Mapping processing means 206 for generating an acoustic feature parameter of the target speaker's speech signal corresponding to the character string based on the correspondence between the speaker codebook and the mapping codebook, and mapping processing means 206 Therefore, based on the acoustic feature parameters of the generated target speaker's voice signal, and a voice synthesis section 207 which generates a sound signal of a target speaker corresponding to the character string output. The generation unit 202 calculates a mapping code book from the selected speaker to the target speaker by using the moving vector field smoothing method.

  Note that non-patent literature 1 and non-patent literature 2 describe general techniques of speech signal processing and speech recognition.

JP-A-1-97997 (FIG. 4) JP-A-8-248994 (FIG. 1) By Sadahiro Furui, "Digital Audio Processing", Tokai University Press, 1985 Seiichi Nakagawa, "Speech recognition using probabilistic models", IEICE, 1988

  In the voice quality conversion method disclosed in Patent Document 1, when voice quality conversion is performed from the conversion source speaker A to the conversion target speaker B, voice data of the original sound in which A and B are uttering exactly the same content ( Alternatively, it is necessary to create a conversion function using feature amount data. For this reason, in order to perform voice quality conversion by this method, voice data having exactly the same contents of speaker A and speaker B is required each time, and free voice quality conversion cannot be performed.

  Further, in the voice quality conversion and speech synthesis method disclosed in Patent Document 2, the difference between the acoustic space of the target speaker's speech and the acoustic space of the speech database is taken and mapped using the moving vector field smoothing method. In order to obtain a code book, when the amount of information in the database is overwhelmingly larger than the amount of information of the target speaker, it is necessary to smooth the very sparse association to make the entire association. For this reason, there is a problem that the accuracy of association is low, the sound quality of the converted speech is deteriorated, or the similarity with the voice quality of the target speech is low.

  Therefore, a main object of the present invention is to provide an apparatus, a method, and a program for realizing free voice quality conversion that does not depend on the utterance content of the conversion target speaker.

  Another object of the present invention is to provide an apparatus, a method, and a program that enable identification of a conversion function that can be converted into a desired voice quality with high accuracy.

  Still another object of the present invention is to provide an apparatus, a method, and a program capable of automatically generating a synthesized sound having the same or similar utterance content as the conversion target speaker voice.

  In order to achieve the above object, according to the first aspect, a voice quality conversion apparatus according to the present invention is a target speaker voice input for inputting a voice of a speaker as a conversion target (referred to as “target speaker voice”). A speech synthesis data storage unit that stores data for speech synthesis, and a phonetic symbol input unit that inputs a phonetic symbol string describing the utterance content that is the same as or similar to the input target speaker voice. . In addition, a speech synthesis unit that synthesizes and outputs speech based on speech synthesis data stored in the speech synthesis data storage unit according to the input phonetic symbol string, and a speech output from the speech synthesis unit A synthesized sound feature parameter extracting unit that extracts a feature parameter of a signal (referred to as “synthesized sound”), and a target speaker voice feature parameter extracting unit that extracts a feature parameter of the target speaker voice. Further, the synthesized speech feature parameter from the synthesized speech feature parameter extraction unit and the target speaker speech feature parameter from the target speaker speech feature parameter extraction unit are input, and in the space representing the feature parameter, 1 to obtain a correspondence relationship on the time axis between the first portion and the second portion of the target speaker voice, and change the spectrum shape in the first portion of the synthesized speech to the spectrum shape in the second portion of the target speaker voice. A conversion function generation unit that identifies a conversion function to be converted.

  According to the second aspect, the voice quality conversion method according to the present invention is a method for converting voice quality by a voice quality conversion device. The voice quality conversion method includes a step of inputting a voice of a speaker as a voice quality conversion target (referred to as “target speaker voice”), a phonetic symbol string describing the utterance content of the target speaker voice, Creating synthesized speech from a phonetic symbol string using speech synthesis data stored in advance. A step of analyzing the synthesized sound and the target speaker voice and extracting each feature parameter; and a step of correlating the feature parameter of the target speaker voice and the feature parameter of the synthesized sound on the time axis; Generating a conversion function for converting the spectrum shape of the synthesized speech into the spectrum shape of the target speaker speech based on the feature parameters of the target speaker speech and the synthesized speech associated with each other.

  Further, according to the third aspect, the voice quality conversion method according to the present invention is a method for converting voice quality by a voice quality conversion device. The voice quality conversion method includes a step of inputting a voice of a speaker as a voice quality conversion target (referred to as “target speaker voice”), a step of recognizing the input target speaker voice, and a target from the result of the voice recognition. Generating a phonetic symbol string describing the utterance content of the speaker voice. Furthermore, (a) a step of creating a synthesized sound from a phonetic symbol string using speech synthesis data stored in advance in the storage unit; (b) analyzing the synthesized sound and the target speaker voice; A step of extracting each feature parameter, (c) a step of associating the feature parameter of the target speaker speech with the feature parameter of the synthesized sound on the time axis, and (d) two feature parameters associated with each other And a step of generating a conversion function for converting the spectrum shape of the synthesized sound into the spectrum shape of the target speaker speech, and (e) converting the voice quality of the speech to be converted using the generated conversion function And (f) the step of storing the voice quality conversion result in the storage unit as data for speech synthesis is repeated until the convergence condition is reached.

  According to the fourth aspect, a voice quality conversion program according to the present invention is a program that is executed by a computer constituting the voice quality conversion device. This program inputs a voice of a speaker as a voice quality conversion target (referred to as “target speaker voice”) and a phonetic symbol string describing the utterance content of the target speaker voice, and stores them in advance in the storage unit. Using the stored speech synthesis data, a process of creating a synthesized sound from a phonetic symbol string is executed. A process of analyzing the synthesized sound and the target speaker voice and extracting each feature parameter; a process of associating the feature parameter of the target speaker voice and the feature parameter of the synthesized sound on the time axis; A process of generating a conversion function for converting the spectrum shape of the synthesized speech into the spectrum shape of the target speaker speech is executed based on the characteristic parameters of the target speaker speech and the synthesized speech associated with each other.

  Furthermore, according to the fifth aspect, the voice quality conversion program according to the present invention is a program that is executed by a computer constituting the voice quality conversion device. This program inputs the voice of the speaker that is the voice quality conversion target (referred to as “target speaker voice”), the process of recognizing the input target speaker voice, and the result of the speech recognition. Generating a phonetic symbol string describing the utterance content of the person's voice. Furthermore, (a) a process of creating a synthesized sound from a phonetic symbol string using speech synthesis data stored in advance in the storage unit, and (b) analyzing the synthesized sound and the target speaker voice, (C) processing for extracting feature parameters of the target speaker voice, and processing for associating the feature parameters of the target speaker speech with the feature parameters of the synthesized speech on the time axis, and (d) two feature parameters associated with each other And a process for generating a conversion function for converting the spectrum shape of the synthesized sound into the spectrum shape of the target speaker voice, and (e) a process for converting the voice quality of the voice to be converted using the generated conversion function; (F) A process of storing the voice quality conversion result in the storage unit as data for voice synthesis and a process of repeating until the convergence condition is reached.

  According to the present invention, conversion to target voice quality can be performed without preparing a conversion source voice having the same or similar utterance content as the target speaker voice in advance. Therefore, the burden on the user is reduced. This is because a synthesized sound having the same or similar utterance content as the target speaker voice can be generated by inputting a phonetic symbol string representing the same or similar utterance content as the target speaker voice. .

  In addition, it is possible to identify a conversion function for performing learning speech association and voice quality conversion with high accuracy. Accordingly, it is possible to obtain a high-quality sound having a voice quality close to a desired voice quality as compared with the prior art. One reason for this is that by estimating the data used to create the synthesized speech from the information of the target speaker speech, the synthesized speech can be created so that the conversion function to the target speaker speech can be easily identified. Because. Another reason is that by storing the data used when creating the synthesized sound, this data can be used for generating a conversion function in addition to the feature parameter extracted from the speech.

  Furthermore, a synthesized sound having the same or similar utterance content as that of the target speaker voice can be created without preparing a phonetic symbol string representing the same or similar utterance content as that of the target speaker voice. Therefore, the processing is automated and the processing speed is improved. This is because a phonetic symbol string representing the utterance content identical or similar to the target speaker voice can be automatically generated by recognizing the target speaker voice.

[First Embodiment]
FIG. 1 shows a block diagram of a voice quality conversion apparatus according to the first embodiment of the present invention. The voice quality conversion apparatus includes a target speaker voice input unit 11, a phonetic symbol string input unit 12, a voice synthesis data storage unit 13, a voice synthesis unit 14, a target speaker voice feature parameter extraction unit 15, and a synthesized sound. A feature parameter extraction unit 16, a conversion function generation unit 17, and a voice quality conversion unit 18 are provided.

  The target speaker voice input unit 11 inputs voice data having a target voice quality. The phonetic symbol string input unit 12 inputs a phonetic symbol string in which the utterance content of the voice to be created is described. At this time, the input phonetic symbol string is described so as to synthesize voice having the same or similar utterance content as the target speaker voice input from the target speaker voice input unit 11. The speech synthesis data storage unit 13 stores data such as units of speech and syllables, time length information, and the like used by the speech synthesis unit 14. The voice synthesis unit 14 calculates synthesis data from the data stored in the voice synthesis data storage unit 13 according to the data corresponding to the phonetic symbol string, and synthesizes and outputs the voice using the data.

  Further, the target speaker voice feature parameter extraction unit 15 performs spectrum analysis on the voice data input from the target speaker voice input unit 11 and extracts feature parameters. The synthesized sound feature parameter extraction unit 16 performs spectrum analysis on the speech data input from the speech synthesis unit 14 to extract feature parameters. Here, the characteristic parameters extracted from the target speaker voice and the synthesized sound include, for example, LPC (linear predictive coding) coefficients, formant frequencies, band bandwidths, prosodic data, and the like described in Non-Patent Document 1. Including at least one. The conversion function generation unit 17 identifies a function for converting the voice quality of the synthesized sound into the voice quality of the target speaker voice based on the feature parameter extracted from the target speaker voice and the feature parameter extracted from the synthesized voice. The voice quality conversion unit 18 converts the converted input signal to be converted using the identified conversion function, and outputs a converted output signal.

  Next, the operation of the voice quality conversion apparatus according to the first embodiment of the present invention will be described with reference to FIG. 1 and FIG. FIG. 13 is a flowchart showing the operation of the voice quality conversion apparatus according to the first embodiment of the present invention. First, voice data of a speaker having a voice quality desired by the user is prepared as a target voice and input to the target speaker voice input unit 11 (step S11). Next, a phonetic symbol string is created so as to have the same or similar utterance content as the target speaker voice input to the target speaker voice input unit 11 and input to the phonetic symbol string input unit 12. Data for synthesis is calculated from the voice synthesis data storage unit 13 in accordance with the data corresponding to the inputted phonetic symbol string, and the voice synthesis unit 14 generates a synthesized sound. At this time, the time length of the speech of the target speaker voice and the synthesized sound may be shifted. The rules for associating speech synthesis data with phonetic symbols are based on the speech synthesis data such as speech segment data and time length information stored in the speech synthesis data storage unit 13 when a phonetic symbol is designated. It is created in advance so as to calculate speech synthesis data corresponding to phonetic symbols (step S12).

  Further, the target speaker voice feature parameter extraction unit 15 analyzes the target speaker voice and extracts feature parameters of the target speaker voice. In addition, the synthesized sound feature parameter extraction unit 16 analyzes the synthesized sound and extracts a feature parameter of the synthesized sound (step S13). In order to correct the time lag between the synthesized speech and the target speech, the time axis is correlated between the synthesized speech and the target speech using the parameters extracted from the target speaker speech and the feature parameters extracted from the synthesized speech. Do. As a method of association, time axis expansion and contraction by DP matching can be considered (step S14). After all learning data (a set of the target speaker voice and synthesized sound) is associated with the time axis, the conversion function generation unit 17 uses the characteristic parameters already associated with the time axis to determine the voice quality of the synthesized sound. A function for converting the voice quality of the target speaker voice is created (step S15).

  Further, in order to determine which conversion function is applied to which part of the converted input signal, the converted input signal (element data) is associated with the created conversion function (step S16). ). The converted input signal is converted using the conversion function associated with the converted input signal (step S17). The converted data is output as a post-conversion output signal (step S18).

  According to the first embodiment of the present invention, since the synthesized sound having the same or similar utterance content as that of the target speaker voice is generated by the voice synthesis, the analysis is performed. Therefore, regardless of the utterance content of the target speaker voice, It is possible to generate a conversion function after associating accuracy. For this reason, free voice quality conversion independent of the utterance content of the conversion target speaker can be realized. Therefore, it is not necessary to input the conversion source voice each time according to the utterance content of the target speaker voice, so that the burden on the user can be reduced and the processing can be speeded up.

[Second Embodiment]
FIG. 2 shows a block diagram of a voice quality conversion apparatus according to the second embodiment of the present invention. 2, the same reference numerals as those in FIG. 1 denote the same or corresponding parts, and the description thereof is omitted. The voice quality conversion device according to the second embodiment further includes a voice recognition unit 19 in addition to the voice quality conversion device according to the first embodiment. The voice recognition unit 19 recognizes the target speaker voice input by the target speaker voice input unit 11a, and outputs the utterance content of the target speaker voice to the utterance symbol string input unit 12a together with the phonetic symbols.

  Next, the operation of the voice quality conversion apparatus according to the second embodiment of the present invention will be described with reference to FIG. 2 and FIG. However, since steps S21 and S23 to S28 in FIG. 14 perform the same operations as S11 and S13 to S18 in FIG. 13, respectively, only the operations of the newly added steps will be described. The target speaker voice input from the target speaker voice input unit 11a in step S21 is voice-recognized by the voice recognition unit 19 (step S221) to generate a phonetic symbol string (step S222). A synthesized sound is created by using it (step S223).

  According to the second embodiment of the present invention, since the speech recognition result of the target speaker voice is input as a phonetic symbol string for generating a synthesized sound, there is no need for the user to input a phonetic symbol string as text, Processing can be automated. The present invention is effective in increasing the accuracy of association using many sentences as target speaker voices.

[Third Embodiment]
The voice quality conversion apparatus according to the third embodiment of the present invention stores the segmentation information used in the speech synthesis unit in addition to the configuration of the voice quality conversion apparatus according to the first embodiment, and outputs the segmentation information to the conversion function generation unit An information storage unit is used. By adopting such a configuration and using segmentation information in the conversion function generation unit, it is possible to identify a conversion function that can be converted to a desired voice quality with high accuracy. In the following description, the segmentation information may mainly be phonemes (phoneme labels) corresponding to time information, and may be added with prosodic information such as a pitch pattern or power.

  FIG. 3 is a block diagram of a voice quality conversion apparatus according to the third embodiment of the present invention. In FIG. 3, a target speaker voice input unit 11, a phonetic symbol string input unit 12, a voice synthesis data storage unit 13, a target speaker voice feature parameter extraction unit 15, a synthesized voice feature parameter extraction unit 16, The voice quality conversion unit 18 is equivalent to that shown in FIG. The voice quality conversion device according to the third embodiment of the present invention further includes a synthesized sound segmentation information storage unit 20. The synthesized sound segmentation information storage unit 20 stores segmentation information used when the speech synthesizer 14a synthesizes speech according to the phonetic symbol string, and outputs the segmentation information to the conversion function generator 17a.

  Next, the operation of the voice quality conversion apparatus according to the third embodiment of the present invention will be described with reference to FIG. 3 and FIG. However, in FIG. 15, the same reference numerals in FIG. 13 perform the same or equivalent processing, the description thereof is omitted, and only the operation of the newly added step will be described. In step S12a, the segmentation information 23a used when the speech synthesizer 14 synthesizes the speech is stored in the synthesized speech segmentation information storage unit 20 and is output to the conversion function generator 17a. The segmentation information 23a input to the conversion function generation unit 17a is used for time axis association (step S14a) and generation of a conversion function (step S15a).

  According to the third embodiment of the present invention, since the segmentation information used when generating the synthesized sound is used when generating the conversion function, the process of associating the synthesized sound feature parameter with the target speaker voice feature parameter is performed. Simplification and high accuracy can be achieved.

[Fourth Embodiment]
FIG. 4 shows a block diagram of a voice quality conversion apparatus according to the fourth embodiment of the present invention. 4, the same reference numerals as those in FIG. 2 denote the same or corresponding parts, and the description thereof is omitted. The voice quality conversion apparatus according to the fourth embodiment of the present invention further includes a synthesized sound segmentation information storage unit 20 as compared to FIG. The synthesized sound segmentation information storage unit 20 is the same as that described with reference to FIG.

  Next, the operation of the voice quality conversion apparatus according to the fourth embodiment of the present invention will be described with reference to FIG. 4 and FIG. However, in FIG. 16, the same reference numerals in FIG. 13 perform the same or equivalent processing, the description thereof is omitted, and only the operation of the newly added step will be described. In step S223a, the segmentation information 23b used when the speech synthesizer 14a synthesizes speech is output and input to the conversion function generator 17a. The input segmentation information 23b is used in time axis association (step S24a) and creation of a conversion function (step S25b).

  According to the fourth embodiment of the present invention, the target speaker's speech is synthesized with speech to automatically generate a phonetic symbol string, and the segmentation information used when generating the synthesized speech using this is used when generating the conversion function. Therefore, it is possible to simplify the processing and increase the accuracy when associating the synthesized speech feature parameter with the target speaker speech feature parameter while automating the processing.

[Fifth Embodiment]
FIG. 5 shows a block diagram of a voice quality conversion apparatus according to the fifth embodiment of the present invention. 5, the same reference numerals as those in FIG. 1 denote the same or corresponding parts, and the description thereof is omitted. The voice quality conversion device according to the fifth embodiment of the present invention further includes a segmentation information input unit 22. The segmentation information input unit 22 analyzes the target speaker voice input by the target speaker voice input unit 11 by means (not shown), extracts the segmentation information 23c, and outputs it to the voice synthesis unit 14b.

  Next, the operation of the voice quality conversion apparatus according to the fifth embodiment of the present invention will be described with reference to FIGS. However, in FIG. 17, the same reference numerals in FIG. 13 perform the same or equivalent processing, the description thereof is omitted, and only the operation of the newly added step will be described. In step S12b, segmentation information 23c based on the target speaker voice input by the target speaker voice input unit 11 is input. The input segmentation information 23c is output to the speech synthesizer 14b, and has the utterance contents of the phonetic symbol string input by the phonetic symbol string input unit 12, and the voice whose utterance time length is the same as that of the target speaker voice. The speech synthesizer 14b synthesizes it.

  According to the fifth embodiment of the present invention, since the segmented information based on the target speaker voice is input to generate the synthesized sound, the time length information of the generated synthesized sound is the time length of the target speaker voice. It becomes equal to information, and it becomes unnecessary or can simplify the time axis association processing in the conversion function generation unit.

[Sixth Embodiment]
FIG. 6 shows a block diagram of a voice quality conversion apparatus according to the sixth embodiment of the present invention. 6, the same reference numerals as those in FIG. 2 denote the same or corresponding parts, and the description thereof is omitted. The voice quality conversion device according to the sixth embodiment of the present invention further includes a segmentation information input unit 22. The segmentation information input unit 22 analyzes the target speaker voice input by the target speaker voice input unit 11a by means (not shown), extracts the segmentation information 23c, and outputs it to the voice synthesis unit 14b.

  Next, the operation of the voice quality conversion apparatus according to the sixth embodiment of the present invention will be described with reference to FIG. 6 and FIG. However, in FIG. 18, the same reference numerals in FIG. 13 perform the same or equivalent processing, the description thereof is omitted, and only the operation of the newly added step will be described. In step S222a, segmentation information 23c based on the target speaker voice input by the target speaker voice input unit 11a is input. The input segmentation information 23c is output to the speech synthesizer 14b, and the speech having the utterance content of the phonetic symbol string input by the phonetic symbol string input unit 12a and having the same utterance time length as the target speaker voice is synthesized. Is done.

  According to the sixth embodiment of the present invention, since the segmentation information based on the target speaker voice is input to generate the synthesized sound, the time length information of the generated synthesized sound is the time length of the target speaker voice. It becomes equal to information, and it becomes unnecessary or can simplify the time axis association processing in the conversion function generation unit. Further, in the present invention, since the phonetic symbol string is generated from the target speaker voice by voice recognition, the consistency between the phonetic symbol string and the time length information extracted from the target speaker voice is achieved while automating the process. Can be easily removed.

[Seventh Embodiment]
FIG. 7 shows a block diagram of a voice quality conversion apparatus according to the seventh embodiment of the present invention. 7, the same reference numerals as those in FIG. 2 denote the same or corresponding parts, and the description thereof is omitted. The voice quality conversion device according to the seventh embodiment of the present invention further includes a target speaker voice segmentation information storage unit 21. The target speaker voice segmentation information storage unit 21 stores the segmentation information of the target speaker voice input by the target speaker voice input unit 11a and outputs it to the conversion function generation unit 17b.

  Next, the operation of the voice quality conversion apparatus according to the seventh embodiment of the present invention will be described with reference to FIGS. However, in FIG. 19, the same reference numerals in FIG. 14 perform the same or equivalent processing, the description thereof is omitted, and only the operation of the newly added step will be described. In step S221a, segmentation information 23d extracted as a result of speech recognition of the target speaker speech signal by the speech recognition unit 19 is output. The segmentation information 23d is output to the conversion function generation unit 17b, and is used for time axis association (step S24a) and creation of the conversion function (step S25a).

  According to the seventh embodiment of the present invention, when the target speaker voice is recognized, the segmentation information of the target speaker voice is simultaneously extracted and input to the conversion function generation unit. In addition to this, segmentation information can be used for generating a conversion function, and it is possible to simplify and increase the accuracy of the process when associating the synthesized speech feature parameters with the target speaker speech feature parameters.

[Eighth Embodiment]
FIG. 8 shows a block diagram of a voice quality conversion apparatus according to the eighth embodiment of the present invention. 8, the same reference numerals as those in FIG. 2 denote the same or corresponding parts, and the description thereof is omitted. The voice quality conversion device according to the eighth embodiment of the present invention further includes a target speaker voice segmentation information storage unit 21a. The target speaker voice segmentation information storage unit 21a stores the segmentation information of the target speaker voice input by the target speaker voice input unit 11a and outputs it to the voice synthesis unit 14b.

  Next, the operation of the voice quality conversion apparatus according to the eighth embodiment of the present invention will be described with reference to FIGS. However, in FIG. 20, the same reference numerals in FIG. 14 perform the same or equivalent processing, the description thereof is omitted, and only the operation of the newly added step will be described. In step S221a, segmentation information 23e extracted as a result of speech recognition of the target speaker by the speech recognition unit 19 is output. In step S223b, the speech synthesizer 14b inputs the segmentation information 23e, synthesizes speech based on the segmentation information 23e of the target speaker speech having the utterance content of the phonetic symbol string input by the phonetic symbol string input unit 12a. To do.

  According to the eighth embodiment of the present invention, when the target speaker voice is recognized, the segmentation information of the same voice is simultaneously extracted and input to the voice synthesizer. Thus, a synthesized sound can be generated, and it is possible to simplify and increase the accuracy of the process when associating the synthesized sound feature parameter with the target speaker voice feature parameter.

[Ninth Embodiment]
FIG. 9 shows a block diagram of a voice quality conversion apparatus according to the ninth embodiment of the present invention. 9, the same reference numerals as those in FIG. 2 denote the same or corresponding parts, and the description thereof is omitted. The voice quality conversion device according to the ninth embodiment of the present invention further includes a target speaker voice segmentation information storage unit 21b. The target speaker voice segmentation information storage unit 21b stores the segmentation information of the target speaker voice input by the target speaker voice input unit 11a and outputs the segmentation information to the voice synthesis unit 14b and the conversion function generation unit 17b.

  Next, the operation of the voice quality conversion apparatus according to the ninth embodiment of the present invention will be described with reference to FIG. 9 and FIG. However, in FIG. 21, the same reference numerals in FIG. 14 perform the same or equivalent processing, the description thereof is omitted, and only the operation of the newly added step will be described. In step S221a, segmentation information 23f extracted as a result of speech recognition of the target speaker by the speech recognition unit 19 is output. The segmentation information 23f is output to the speech synthesizer 14b, has the utterance contents of the phonetic symbol string input by the phonetic symbol string input unit 12a, and synthesizes speech based on the segmentation information 23f of the target speaker speech. Further, the segmentation information 23f is also output to the conversion function generation unit 17b, and is used for time axis association (step S24a) and generation of the conversion function (step S25a).

  According to the ninth embodiment of the present invention, when the target speaker speech is recognized, the segmentation information of the same speech is simultaneously extracted and input to the speech synthesizer and the conversion function generator. The synthesized speech can be generated using the segmentation information based on the segmentation information, and the segmentation information can be used for generating the conversion function in addition to the feature parameter of the target speaker voice. Therefore, it is possible to simplify and increase the accuracy of the process when associating the synthesized speech feature parameter with the target speaker speech feature parameter.

[Tenth embodiment]
FIG. 10 shows a block diagram of a voice quality conversion apparatus according to the tenth embodiment of the present invention. 10, the same reference numerals as those in FIG. 2 denote the same or corresponding parts, and the description thereof is omitted. The voice quality conversion device according to the tenth embodiment of the present invention further includes a synthesized speech segmentation information storage unit 20 and a target speaker speech segmentation information storage unit 21. The synthesized sound segmentation information storage unit 20 is the same as the synthesized sound segmentation information storage unit 20 described in FIG. The target speaker voice segmentation information storage unit 21 is the same as the target speaker voice segmentation information storage unit 21 described with reference to FIG.

  Next, the operation of the voice quality conversion apparatus according to the tenth embodiment of the present invention will be described with reference to FIG. 10 and FIG. However, in FIG. 22, the same reference numerals in FIG. 13 perform the same or equivalent processing, the description thereof is omitted, and only the operation of the newly added step will be described. In step S221a, segmentation information 23d extracted as a result of speech recognition of the target speaker by the speech recognition unit 19 is output. The segmentation information 23d is output to the conversion function generation unit 17c. In step S223a, segmentation information 23g used when the speech synthesizer 14a synthesizes speech is stored and output to the conversion function generator 17c. The segmentation information 23f and 23g input to the conversion function generation unit 17c is used for time axis association (step S24b) and generation of a conversion function (step S25b).

  According to the tenth embodiment of the present invention, since the segmentation information of both the synthesized sound and the target speaker voice can be used in the conversion function generation unit, the segmentation information is converted in addition to the characteristic parameters of the synthesized sound and the target speaker voice. It can be used for function generation, and it is possible to simplify and increase the accuracy of the process when associating the synthesized speech feature parameters with the target speaker speech feature parameters.

[Eleventh embodiment]
FIG. 11 shows a block diagram of a voice quality conversion apparatus according to the eleventh embodiment of the present invention. 11, the same reference numerals as those in FIG. 2 denote the same or corresponding parts, and the description thereof is omitted. The voice quality conversion device according to the eleventh embodiment of the present invention further includes a synthesized speech segmentation information storage unit 20 and a target speaker voice segmentation information storage unit 21a. The synthesized sound segmentation information storage unit 20 and the target speaker voice segmentation information storage unit 21a are the same as the synthesized sound segmentation information storage unit 20 of FIG. 3 and the target speaker voice segmentation information storage unit 21a of FIG.

  Next, the operation of the voice quality conversion apparatus according to the eleventh embodiment of the present invention will be described with reference to FIG. 11 and FIG. However, in FIG. 23, the same reference numerals in FIG. 20 perform the same or equivalent processing, and the description thereof will be omitted, and only the operation of the newly added step will be described. In step S223c, segmentation information 23g used when the speech synthesizer 14c synthesizes speech is stored and input to the conversion function generator 17a. The input segmentation information 23g is used for time axis association (step S24a) and creation of a conversion function (step S25a).

  According to the eleventh embodiment of the present invention, when the target speaker voice is recognized, the segmentation information of the same voice is simultaneously extracted and input to the voice synthesizer. Can be used to generate synthesized sounds. Further, since the segmentation information used when generating the synthesized sound is used when generating the conversion function, the segmentation information can be used for generating the conversion function in addition to the characteristic parameter of the synthesized sound. Therefore, it is possible to simplify and increase the accuracy of the process when associating the synthesized speech feature parameter with the target speaker speech feature parameter.

[Twelfth embodiment]
FIG. 12 shows a block diagram of a voice quality conversion apparatus according to the twelfth embodiment of the present invention. 12, the same reference numerals as those in FIG. 2 denote the same or corresponding parts, and the description thereof is omitted. The voice quality conversion device according to the twelfth embodiment of the present invention further includes a synthesized speech segmentation information storage unit 20 and a target speaker voice segmentation information storage unit 21b. The synthesized sound segmentation information storage unit 20 and the target speaker voice segmentation information storage unit 21b are the same as the synthesized sound segmentation information storage unit 20 of FIG. 3 and the target speaker voice segmentation information storage unit 21b of FIG. 9, respectively.

  Next, the operation of the voice quality conversion device according to the twelfth embodiment of the present invention will be described with reference to FIGS. However, in FIG. 24, the same reference numerals in FIG. 21 perform the same or equivalent processing, and the description thereof is omitted, and only the operation of the newly added step will be described. In step S223c, segmentation information 23g used when the speech synthesizer 14c synthesizes speech is stored and input to the conversion function generator 17c. The input segmentation information 23g is used for time axis association (step S24b) and creation of a conversion function (step S25b).

  According to the twelfth embodiment of the present invention, when the target speaker speech is recognized, the segmentation information of the same speech is simultaneously extracted and input to the speech synthesizer and the conversion function generator. A synthesized sound can be generated using the segmentation information based thereon. In addition to the feature parameters of the target speaker's voice, segmentation information can be used for generating a conversion function. Further, since the segmentation information of both the synthesized sound and the target speaker voice can be used in the conversion function generation unit, the segmentation information can be used for generating the conversion function in addition to the characteristic parameters of the synthesized sound and the target speaker voice. Therefore, it is possible to simplify and increase the accuracy of the process when associating the synthesized speech feature parameter with the target speaker speech feature parameter.

  FIG. 25 is a block diagram of the voice quality conversion apparatus according to the first embodiment of the present invention. The voice quality conversion device includes a target speaker voice input unit 11, a phonetic symbol string input unit 12, a voice synthesis data storage unit 13, a voice synthesis unit 14, LPC analysis units 25, 26 and 30, a DP matching unit 27, a spectral shape conversion. A conversion function deriving unit 29, a corresponding frame searching unit 31, and a spectrum converting unit 32. In FIG. 25, the same reference numerals as those in FIG. 1 denote the same or equivalent components, and the description thereof will be omitted unless otherwise specified.

  In the present embodiment, it is assumed that a database for speech synthesis is created in advance from at least one speaker and is stored in the speech synthesis data storage unit 13. The database includes data such as speech segments, durations, and pitch patterns. However, it is not always necessary to store all these data in the database, and only one or two of them may be stored.

  Now, the utterance content C of the target speaker voice has been clarified, and a phonetic symbol string that can generate a synthesized sound having the same or similar utterance content as the utterance content is input from the phonetic symbol string input unit 12 in advance. Suppose that it is prepared. In accordance with this phonetic symbol string, the speech synthesizer 14 synthesizes the synthesized sound B of the utterance having the voice quality of the segments in the speech synthesis data. Examples of the target speaker voice input from the target speaker voice input unit 11 include a case where a voice is stored in a recording device at the start of operation, and a case where voice data stored in advance is used. There may be a plurality of target speaker voices.

  Next, the LPC analysis unit 25 inputs the voice of the utterance content C of the speaker A (hereinafter referred to as target A) input from the target speaker voice input unit 11 for each analysis frame having a length of 20 msec, for example. LPC analysis is performed, and the LPC coefficient 35a of the target A is extracted. In addition, the LPC analysis unit 26 similarly performs LPC analysis on the speech of the utterance content C of the synthesized sound B created by the speech synthesis unit 14 (hereinafter referred to as the synthesized sound B), and extracts the LPC coefficient 35b of the synthesized sound B. To do. In this embodiment, LPC analysis will be described as an analysis method (feature parameter extraction). In addition, spectrum analysis such as LSP (line spectrum pair) analysis, PARCOR (partial auto-correlation) analysis, zero crossing counting method, self A pitch extraction method such as a correlation method and an analysis based on a combination of these may be considered. In addition to LPC coefficients, prosody parameters such as autocorrelation coefficients, cepstrum coefficients, and pitch frequencies can be used as feature parameters to be extracted.

  Next, the flow of processing in the DP matching unit 27, the spectrum shape conversion unit 28, the conversion function derivation unit 29, the corresponding frame search unit 31, and the spectrum conversion unit 32, which are main parts of voice quality conversion, will be described. FIG. 26 is a diagram for explaining the flow of processing of voice data in the main part of voice quality conversion. The target speaker voice (target A) and the synthesized sound B subjected to the LPC analysis are subjected to DP matching by the DP matching unit 27 to obtain a correspondence relationship. A conversion function from the synthesized sound B to the target A is generated by the spectrum shape conversion unit 28 and the conversion function derivation unit 29 using the obtained correspondence relationship.

  The speech unit to be converted is associated with the synthesized speech unit by the corresponding frame search unit 31. In this case, the association is performed, for example, by selecting a feature space having a smaller distance in the LPC coefficients of the speech unit and the synthesized speech unit to be converted. The speech unit to be converted is subjected to spectral conversion by the conversion function previously obtained corresponding to the synthesized speech unit that is associated, and a speech unit having the target voice quality is generated. Hereinafter, each part will be described in more detail.

  The DP (dynamic programming) matching unit 27 uses DP (dynamic programming) matching to match the time axes of the target A and the synthesized sound B using the LPC coefficient 35a of the target A and the LPC coefficient 35b of the synthesized sound B. Perform time axis expansion and contraction. Thereby, the correspondence for each analysis frame between the target A and the synthesized sound B is created. At this time, as a scale representing the distance in the feature amount space used in DP matching, a sum of squares of a difference vector, a weighted likelihood ratio (WLR) scale, a maximum likelihood spectral distance, an LPC cepstrum distance, and the like can be used.

  FIG. 27 is a diagram schematically illustrating DP matching for each analysis frame. Frames of the target speaker voice (target A) are A1, A2, A3, A4, A5,... Am, .., and a frame of the synthesized sound B is B1, B2, B3, B4, B5,.・ Let's say. The target speaker voice (target A) and the synthesized sound B are subjected to LPC analysis for each frame, and then, for example, the distance in the feature amount space between the frames A1 and B1, and the feature amount space between the frames A1 and B2. A correspondence relationship having the shortest distance among the internal distances and the distances within the feature amount space between the frames A2 and B1 is selected. Here, assuming that the distance in the feature amount space between the frame A2 and the frame B1 is selected, then, for example, the distance in the feature amount space between the frame A3 and the frame B1, and the feature amount space between the frame A2 and the frame B2. The correspondence with the shortest distance is selected from the internal distance and the distance in the feature amount space between the frame A3 and the frame B2. In this way, correspondence of the minimum distance of the feature amount space distance is sequentially obtained, and DP matching between the target speaker voice (target A) and the synthesized sound B is performed.

  Next, the spectrum shape conversion unit 28 converts the voice quality of the synthesized sound B into the voice quality of the target A using the LPC coefficient 35a of the target A and the LPC coefficient 35b of the synthesized sound B that are associated with each analysis frame. Identify the transformation function as In order to realize a conversion in which the voice quality is converted from the synthesized sound B to the target A, the frequency region of the synthesized sound B is set so that the frequency characteristics of each analysis frame of the synthesized sound B are as equal as possible to the frequency characteristics of the target A. Can be converted with a conversion function. When this is considered for each analysis frame, the frequency characteristic of one analysis frame Bn (referred to as the nth analysis frame Bn) of the synthesized sound B is the analysis frame of the target A associated with this analysis frame on the time axis. This means that it is only necessary to identify a conversion function that is converted into the frequency characteristic of Am (m-th analysis frame Am). Therefore, the analysis frame is subjected to Fourier transform to obtain the shape (spectrum envelope) of the frequency domain of the waveform, and expansion / contraction by DP matching or the like on the frequency axis is performed to obtain a conversion function from the synthesized sound B to the target A. That is, conversion is performed so that the spectrum shape of the analysis frame Bn of the synthesized sound B matches the spectrum shape of the analysis frame Am of the target A.

  Further, spectral shape conversion will be described. FIG. 28 is a diagram schematically illustrating DP matching on the frequency axis. The spectral envelopes of the target speaker voice (target A) and synthesized sound B are SA and SB, respectively. A correspondence relationship between the spectrum envelope SA and the spectrum envelope SB is obtained on the frequency axis by DP matching. The paths P0,..., Pi,... Pn representing the correspondence relationship correspond to transformation functions, and the target speaker's voice ((Pi,... Pn) is non-linearly mapped by the paths P0,. A spectrum of target A) will be obtained. Note that pre-processing such as high-frequency emphasis or low-frequency emphasis may be performed on the spectrum envelopes SA and SB in advance, and DP matching may be performed on the pre-processed spectrum envelope.

  As described above, in the conversion function identification, the spectrum envelope SBn is derived from the result of LPC analysis of the nth analysis frame Bn of the synthesized sound B by the LPC analysis unit 26. Similarly, the spectrum envelope SAm of the analysis frame Am of the target A associated with Bn on the time axis is identified. Then, a conversion function is created so that SBn is converted to SAm. In this way, since the conversion function from the nth analysis frame Bn of the synthesized sound B to the analysis frame Am of the target A associated on the time axis is obtained, this is applied to all the frames and the synthesized sound is applied. A conversion function for the entire A can be obtained.

  In the above description, the conversion function is obtained for each frame using only one sentence such as the target A and the synthesized sound B as learning data. However, when the learning data is further increased, the target A and the synthesized sound B are obtained. It is also possible to cluster the LPC coefficients in the feature amount space, for example, for each phoneme, and identify a conversion function for each representative point of the cluster. Also, a method of expressing the feature amount space with a probability density distribution such as GMM (Guassian Mixture model) and associating it with the density distribution state is conceivable.

  FIG. 29 is a diagram schematically showing a conversion function between phonemes clustered in the feature amount space. In the feature parameter space of the synthesized sound B and the feature parameter space of the target A, for example, phonemes (a, i, u, e, o, s) are classified into clusters, respectively. The representative points in the cluster are converted into the representative points in the cluster in the feature amount space of the target A by the conversion function.

  Next, the speech synthesis segment data to be converted is associated with the conversion function. For this purpose, the LPC analysis unit 30 performs an LPC analysis on a speech synthesis unit signal in advance to obtain an LPC coefficient for each analysis frame of the unit data.

  The corresponding frame search unit 31 corresponds to the LPC coefficient for each analysis frame of the segment data output from the LPC analysis unit 30 that has the closest distance in the feature amount space in the LPC coefficient 35b for each frame of the synthesized sound B. Ask as a frame.

  The conversion function deriving unit 29 sets the conversion function for each frame of the synthesized sound B obtained by the spectrum shape conversion unit 28 as the conversion function of the segment data. At this time, when the learning data is increased, the conversion function can be set not for each frame but for each cluster, as in the case of identification of the conversion function described with reference to FIG.

  The spectrum conversion unit 32 selects a conversion function for the previous corresponding frame from the conversion functions for the segment obtained by the conversion function deriving unit 29, and uses the selected conversion function to generate a segment signal (segment data). Convert the spectrum of. Thereby, the post-conversion segment signal (the segment data which can produce a synthetic sound) with the voice quality of the target A can be obtained.

  Further, by replacing this segment data with the segment data in the speech synthesis database before conversion stored in the speech synthesis data storage unit 13, synthesis having the voice quality of the target A for any text A database that can create sounds is completed.

  In the first embodiment, it is assumed that the phonetic symbol string for generating the synthesized sound is known in advance. However, the utterance content of the target speaker voice is expressed as text data such as kanji-kana mixed sentences, and morphological analysis is performed. Etc., and a phonetic symbol string can be generated based on the result.

  In addition to the above-described unit data, other input unit data can be used as the input signal to be converted. For example, there is a case where the segment data stored in the voice synthesis data storage unit 13 is travel conversation data, and the segment data serving as an input signal is general conversation data.

  Furthermore, a synthesized sound created with the above-mentioned segment data, a synthesized sound created with other segment data, a voice uttered by the user, or the like can be used as an input signal to be converted.

  Furthermore, as a modification of the first embodiment, a method of identifying the conversion function without performing the time axis association processing of the synthesized sound and the target speaker voice in the conversion function generation unit is also conceivable. The reason why this method can be realized will be described below. In the present invention, processing is performed with the utterance contents of the synthesized sound and the target speaker voice being the same or similar, so that the phoneme information of the synthesized sound and the target speaker voice has many identical parts, The distributions of the feature parameters are similar to each other. Therefore, for example, when a conversion function is obtained by clustering for each phoneme, the conversion function can be generated without taking the time axis correspondence between the synthesized sound and the target speaker voice. If this method is used, the time-axis association process is not performed, so that the processing speed can be greatly improved.

  FIG. 30 is a block diagram of a voice quality conversion apparatus according to the second embodiment of the present invention. The voice quality conversion apparatus includes a target speaker voice input unit 11, a phonetic symbol string input unit 12, a voice synthesis data storage unit 13, a voice synthesis unit 14a, a voice recognition unit 19, a synthesized voice segmentation information storage unit 20, and a target speaker voice. A segmentation information storage unit 21, LPC analysis units 25, 26 and 30, a DP matching unit 27 a, a spectrum shape conversion unit 28, a conversion function derivation unit 29, a corresponding frame search unit 31, and a spectrum conversion unit 32 are provided. In FIG. 30, the same reference numerals as those in FIG. 25 denote the same or equivalent components, and the description thereof is omitted unless otherwise specified.

  The voice recognition unit 19 generates a phonetic symbol string for generating a synthesized sound having the same or similar utterance content as the target speaker voice signal by performing voice recognition on the input target speaker voice signal. To the phonetic symbol string input unit 12. Assuming that the speech having the utterance content C is the target speaker speech, the phonetic symbol string having the utterance content C can be automatically generated by recognizing the speech. For this reason, even if the phonetic symbol string is not clear in advance, it is possible to generate a synthesized sound having the same or similar content as the target speaker voice, and if only the target speaker voice is input, the first Processing similar to that in the embodiment can be automatically performed. As a speech recognition method at this time, for example, there is a speech recognition method by HMM (hidden Markov model) as described in Non-Patent Document 2.

  The synthesized speech segmentation information storage unit 20 outputs the segmentation information calculated and used from the speech synthesis data when the speech synthesizer 14a synthesizes speech to the DP matching unit 27a to use it when generating the conversion function. To do. Assuming that the speech having the utterance content C is the target speaker speech, the speech recognition unit 19 recognizes the speech, thereby automatically generating a pronunciation symbol string having the utterance content C and generating a pronunciation symbol string input unit. 12 is output. If a phonetic symbol string is input to the voice synthesis unit 14a, the voice synthesis unit 14a calculates segmentation information corresponding to the utterance content C from the voice synthesis data storage unit 13 according to the phonetic symbol string. The synthesized sound segmentation information storage unit 20 stores this segmentation information, outputs it to the DP matching unit 27a as the segmentation information 23i, and uses it when generating the conversion function. For example, by synthesizing segmentation information and temporal changes of feature parameters extracted by analyzing synthesized speech to temporal changes of feature parameters extracted by analyzing target speaker voices, synthesis with target speaker voices It is possible to simplify and increase the accuracy of sound time axis association. As the segmentation information, as shown in FIG. 31, a phoneme is associated with a table in which frame numbers and phonemes are associated (for example, frame numbers 1, 2,... 10, 11,... 50, 51,. “I”, “i”,... “I”, “e”,... “U”, “s”,. As another usage method, a method of clustering using phoneme segmentation information when the combination of the target speaker speech and the synthesized speech is a plurality of sentences is also conceivable.

  The target speaker voice segmentation information storage unit 21 inputs segmentation information of the target speaker voice that is output when the voice recognition unit 19 recognizes the target speaker voice signal. Assume that the target speaker voice having the utterance content C is input to the target speaker voice input unit 11. The voice recognition unit 19 recognizes the target speaker voice, generates a phonetic symbol string describing the utterance content C, and outputs it to the phonetic symbol string input unit 12, while segmentation information as a result of voice recognition Is output to the target speaker voice segmentation information storage unit 21, and the target speaker voice segmentation information storage unit 21 stores the segmentation information. The segmentation information 23h is output to the DP matching unit 27a and used for the process of associating the synthesized sound with the target speaker voice. As a usage method, the target speaker is analyzed by associating the temporal change of the feature parameter extracted by analyzing the segmentation information and the target speaker voice with the temporal change of the feature parameter extracted by analyzing the synthesized sound. It is possible to simplify and improve the time-axis association between speech and synthesized speech. As the segmentation information, the same information as the table shown in FIG. 31 is used.

  Similarly to the DP matching unit 27 described in the first embodiment, the DP matching unit 27a uses the LPC coefficient 35a of the target A and the LPC coefficient 35b of the synthesized sound B to match the time axes of the target A and the synthesized sound B. Therefore, time axis expansion / contraction by DP matching is performed. Further, the DP matching unit 27a uses the segmentation information 23h output from the synthesized speech segmentation information storage unit 20 and the segmentation information 23i output from the target speaker voice segmentation information storage unit 21 to perform time-axis association by DP matching. Simplification and high precision are being achieved.

  Next, an example of time axis association by DP matching using segmentation information will be described. FIG. 32 is a diagram illustrating a first example of time axis association by DP matching using segmentation information. A target A (phoneme “a” “s” “u”) analyzed in the vertical axis direction and a synthesized sound B (phoneme “a” “s” “u”) analyzed in the horizontal axis direction are arranged. As the segmentation information 23h, a phoneme boundary P1 is set between the phonemes “a” and “s” of the target A, and a phoneme boundary Q1 is set between the phonemes “s” and “u” of the target A. Yes. Further, as segmentation information 23i, the phoneme boundary P2 between the phonemes “a” and “s” of the synthesized sound B and the phoneme boundary Q2 between the phonemes “s” and “u” of the target A are Is set. At this time, the intersection between the phoneme boundary P1 and the phoneme boundary P2 is defined as a constraint point P, and the intersection between the phoneme boundary Q1 and the phoneme boundary Q2 is defined as a constraint point Q. DP matching is sequentially performed on the phonemes “a”, “s”, and “u” to obtain a DP path. At this time, a restriction is imposed so that the DP path passes through the restriction point P and the restriction point Q, and the DP path is obtained. That is, by determining the DP path so as to pass through the constraint point, DP matching with a constraint condition is performed so that phoneme boundaries are associated with each other. Note that the constraint point does not necessarily pass, and a gentle constraint that passes in the vicinity of the constraint point may be imposed.

  The segmentation information is information in which the start time and end time of each phoneme, the phoneme label, and the like are described for a certain utterance content. Since the segmentation information indicates the phoneme label and the start time and end time of the phoneme, the phoneme boundary in the target A or the synthesized sound B is clearly shown. Therefore, when obtaining the correspondence (DP path) between each frame of the target A and the synthesized sound B on the time axis, by using the segmentation information, the association by DP matching is performed near the phoneme boundary. Even if it is ambiguous, it is possible to realize highly accurate association.

  Next, another example of time axis association by DP matching using segmentation information will be described. FIG. 33 is a diagram illustrating a second example of time axis association by DP matching using segmentation information. The target A and the synthesized sound B are arranged in the same manner as described with reference to FIG. However, in FIG. 33, the difference from FIG. 32 is that segmentation information 23h is not input. That is, the target speaker voice segmentation information storage unit 21 does not exist and the target A has no phoneme boundary. In this case, after obtaining the DP path by normal DP matching, the estimated phoneme boundary P3 of the target A corresponding to the point where the DP path passes through the phoneme boundary P1 of the synthesized sound B is obtained. Further, the estimated phoneme boundary Q3 of the target A corresponding to the point where the DP path passes through the phoneme boundary Q1 of the synthesized sound B is obtained. That is, after DP matching is performed without a constraint, a location associated with a phoneme boundary of speech with segmentation information can be estimated as a phoneme boundary of speech without segmentation information. Normally, a phoneme boundary cannot be determined only by performing DP matching. Therefore, this method of estimating a phoneme boundary is an effective means when selecting a conversion function.

In the above description, an example in which segmentation information is applied to DP matching has been described. However, it is also possible to apply segmentation information when the converted input signal (element signal) is converted in the spectrum conversion unit 32. is there. That is, when determining which conversion function is used to convert each frame of the input signal to be converted, to which set (for example, a cluster for each phoneme), the label information attached to the segmentation information of the frame is used. It can be easily determined whether it belongs. This is shown in FIG. In FIG. 34, label information (“i”, “i”, “e”, “u”) is associated with each cluster in the feature parameter space, and the representative in the cluster in the feature amount space directly from the label information. Can be converted to a point.

  Next, an example in which the voice quality conversion method is repeated to increase the accuracy will be described. FIG. 35 is a flowchart showing a voice quality conversion method according to the third embodiment of the present invention. 35, steps S31 to S39 are steps for performing processing equivalent to steps S21, S221, S222, S223, S23, S24, S25, S26, and S27 of FIG. 14, respectively, and description thereof is omitted. In step S40, the converted signal is registered in the voice synthesis data storage unit 13 as voice synthesis data.

  In step S40, it is determined whether or not the repetition has reached a predetermined convergence condition. If not, the process returns to step S34 to generate a synthesized sound using the converted voice synthesis data. If it has reached, a series of processing ends in step S42. As the convergence condition, for example, there is a case where the distance in the LPC coefficient space between the target speaker voice and the synthesized sound becomes a certain value or less, or a state where this certain value or less continues for a certain number of times. Further, the convergence condition may be a case where the total value of the difference values from the previous time such as spectrum and power becomes equal to or less than a certain value, or a case where the condition below this certain value continues for a certain number of times. Furthermore, the convergence condition may be a case where the number of repetitions is a fixed number of times.

  In the third embodiment, as in the first embodiment, when converting the voice quality of speech synthesis data using a segment as a converted input signal, the target speech is again generated using the speech synthesis data after conversion. A synthesized sound having the same or similar utterance content as the person's voice is generated, a conversion function is generated in the same manner as in the first embodiment, and the process of converting the segment is repeated a plurality of times. Furthermore, by using the second embodiment in combination, if the target speaker voice is first input, all of the repeated processing can be performed automatically, improving the conversion accuracy (similarity of voice quality). It is possible to go. In addition, although the example which does not use segmentation information was shown in FIG. 35, of course, you may use segmentation information.

  In the first to third embodiments described above, speech synthesis data consisting of speech data for a plurality of people is stored in the speech synthesis data storage unit 13 and used for the voice quality of the input target speaker. It is also possible to select the data for synthesis. For example, when the target speaker voice is a male voice, a male voice segment data is used, and when the target speaker voice is a female voice, a female voice segment data is used. If this method is used, it is possible to avoid extreme conversion and reduce deterioration in sound quality due to the conversion process.

  Next, the voice quality conversion apparatus, voice quality conversion method or apparatus for generating synthesized sound using the synthesized voice data generated by the voice quality conversion program will be described. FIG. 36 is a block diagram showing a synthesized sound generating apparatus according to the fourth embodiment of the present invention. 36, a symbol string input unit 41, a speech synthesis output unit 42, and a data storage unit 43 have the same functions as the phonetic symbol string input unit 12, the speech synthesis unit 14, and the speech synthesis data storage unit 13 of FIG. It is what you have. However, the speech synthesis output unit 42 has a function of outputting the generated synthesized sound. The data storage unit 43 stores the converted output signal generated by the voice quality conversion device, voice quality conversion method, or voice quality conversion program described above as synthesized sound data. The speech synthesis output unit 42 generates a synthesized sound using the synthesized sound data read from the data storage unit 43 based on the symbol string input to the symbol string input unit 41 and outputs it. Since the synthesized sound to be output is generated based on the converted output signal, that is, the data converted into the voice quality of the target speaker, the synthesized voice has the voice quality of the target speaker.

  Next, another apparatus for generating a synthesized sound using the synthesized sound data generated by the voice quality conversion apparatus, voice quality conversion method or voice quality conversion program of the present invention will be described. FIG. 37 is a block diagram showing a synthesized sound generating apparatus according to the fifth embodiment of the present invention. In FIG. 37, the voice input unit 51 inputs a voice signal emitted by the user. The conversion data storage unit 53 stores an LPC coefficient for each analysis frame and a corresponding conversion function generated by the voice quality conversion device, voice quality conversion method, or voice quality conversion program described above. The voice conversion output unit 52 performs, for example, LPC analysis on the voice signal output from the voice input unit 51 for each frame, and searches the converted data storage unit 53 for a frame having a parameter space distance closest to the analyzed frame. The corresponding conversion function is read out, and the spectrum of the audio signal is converted and output by this conversion function. Since the output synthesized sound is generated based on the data converted into the voice quality of the target speaker, it becomes a synthesized sound having the voice quality of the target speaker.

  INDUSTRIAL APPLICABILITY According to the present invention, it can be applied to uses such as changing the voice quality freely in communication equipment such as a telephone and a transceiver. In addition, the present invention can also be applied to a case where the voice quality of the synthesized sound when reading a text of an e-mail or chat on a personal computer or a mobile phone is set to a voice quality desired by the user. Furthermore, when voice recording of animation, dubbing of a foreign movie, or the like is performed by text voice synthesis, the present invention can be applied to the use of generating voice quality of character voice that matches a character.

It is a block diagram which shows the structure of the voice quality conversion apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the voice quality conversion apparatus which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the voice quality conversion apparatus which concerns on the 3rd Embodiment of this invention. It is a block diagram which shows the structure of the voice quality conversion apparatus which concerns on the 4th Embodiment of this invention. It is a block diagram which shows the structure of the voice quality conversion apparatus which concerns on the 5th Embodiment of this invention. It is a block diagram which shows the structure of the voice quality conversion apparatus which concerns on the 6th Embodiment of this invention. It is a block diagram which shows the structure of the voice quality conversion apparatus which concerns on the 7th Embodiment of this invention. It is a block diagram which shows the structure of the voice quality conversion apparatus which concerns on the 8th Embodiment of this invention. It is a block diagram which shows the structure of the voice quality conversion apparatus which concerns on the 9th Embodiment of this invention. It is a block diagram which shows the structure of the voice quality conversion apparatus which concerns on the 10th Embodiment of this invention. It is a block diagram which shows the structure of the voice quality conversion apparatus which concerns on the 11th Embodiment of this invention. It is a block diagram which shows the structure of the voice quality conversion apparatus which concerns on the 12th Embodiment of this invention. It is a flowchart which shows operation | movement of the voice quality conversion apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the voice quality conversion apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the voice quality conversion apparatus which concerns on the 3rd Embodiment of this invention. It is a flowchart which shows operation | movement of the voice quality conversion apparatus which concerns on the 4th Embodiment of this invention. It is a flowchart which shows operation | movement of the voice quality conversion apparatus which concerns on the 5th Embodiment of this invention. It is a flowchart which shows operation | movement of the voice quality conversion apparatus which concerns on the 6th Embodiment of this invention. It is a flowchart which shows operation | movement of the voice quality conversion apparatus which concerns on the 7th Embodiment of this invention. It is a flowchart which shows operation | movement of the voice quality conversion apparatus which concerns on the 8th Embodiment of this invention. It is a flowchart which shows operation | movement of the voice quality conversion apparatus which concerns on the 9th Embodiment of this invention. It is a flowchart which shows operation | movement of the voice quality conversion apparatus which concerns on the 10th Embodiment of this invention. It is a flowchart which shows operation | movement of the voice quality conversion apparatus which concerns on the 11th Embodiment of this invention. It is a flowchart which shows operation | movement of the voice quality conversion apparatus which concerns on the 12th Embodiment of this invention. It is a block diagram which shows the structure of the voice quality conversion apparatus which concerns on 1st Example of this invention. It is a figure explaining the flow of a process of the audio | voice data in the principal part of voice quality conversion. It is the figure which represented DP matching for every analysis frame typically. It is the figure which represented DP matching on a frequency axis typically. It is a figure which shows typically the conversion function between the phonemes clustered within the feature-value space. It is a block diagram of the voice quality conversion apparatus which concerns on 2nd Example of this invention. It is a figure which shows matching with a frame number and a phoneme. It is a figure which shows the 1st example of the time-axis matching by DP matching using segmentation information. It is a figure which shows the 2nd example of the time-axis matching by DP matching using segmentation information. It is a figure which shows matching with label information and a cluster. It is a flowchart figure showing the voice quality conversion method which concerns on 3rd Example of this invention. It is a block diagram showing the synthetic sound production | generation apparatus which concerns on the 4th Example of this invention. It is a block diagram showing the synthetic sound production | generation apparatus which concerns on the 5th Example of this invention. It is a block diagram showing the voice quality conversion method of a 1st prior art example. It is a block diagram showing the voice quality conversion method of the 2nd prior art example.

Explanation of symbols

11, 11a Target speaker voice input unit 12, 12a Phonetic symbol string input unit 13 Speech synthesis data storage unit 14, 14a, 14b, 14c Speech synthesis unit 15 Target speaker voice feature parameter extraction unit 16 Synthetic sound feature parameter extraction unit 17, 17a, 17b, 17c Conversion function generation unit 18 Voice quality conversion unit 19 Speech recognition unit 20 Synthetic speech segmentation information storage units 21, 21a, 21b Target speaker speech segmentation information storage unit 22 Segmentation information input units 23a, 23b, 23c, 23d, 23e, 23f, 23g, 23h, 23i Segmentation information 25, 26, 30 LPC analysis unit 27, 27a DP matching unit 28 Spectrum shape conversion unit 29 Conversion function derivation unit 31 Corresponding frame search unit 32 Spectrum conversion unit 35a, 35b LPC Coefficient 41 symbol Input unit 42 speech synthesizing output unit 43 data storage unit 51 the audio input unit 52 speech conversion output unit 53 converts the data storage unit

Claims (27)

A target speaker voice input unit for inputting a voice of a speaker to be converted (referred to as “target speaker voice”);
A voice synthesis data storage unit for storing voice synthesis data;
A phonetic symbol input unit for inputting a phonetic symbol string describing the same or similar utterance content as the input target speaker voice;
A speech synthesis unit that synthesizes and outputs speech based on speech synthesis data stored in the speech synthesis data storage unit according to the input phonetic symbol string;
A synthesized sound feature parameter extracting unit that extracts feature parameters of a speech signal (referred to as “synthesized sound”) output from the speech synthesizer;
A target speaker voice feature parameter extraction unit for extracting feature parameters of the target speaker voice;
The synthesized sound feature parameter extraction unit from the synthesized sound feature parameter extraction unit and the target speaker voice feature parameter extraction unit from the target speaker voice feature parameter extraction unit are input, and the synthesized sound is expressed in a space representing the feature parameter. Corresponding to the second part of the target speaker voice on the time axis, and the spectrum shape of the first part of the synthesized sound is determined as the first part of the target speaker voice. A conversion function generation unit for identifying a conversion function to be converted into a spectral shape in the part 2;
A voice quality conversion device characterized by comprising:   A voice signal to be converted is input, and the voice signal to be converted is converted into a voice signal having the voice quality of the target speaker voice using the conversion function identified by the conversion function generation unit and output. The voice quality conversion device according to claim 1, further comprising a voice quality conversion unit that performs the operation.   A voice recognition unit that inputs and recognizes the target speaker voice input by the target speaker voice input unit, generates the phonetic symbol string, and outputs the phonetic symbol string to the voice synthesis unit; The voice quality conversion device according to claim 1, wherein   A first segmentation information storage unit that stores segmentation information including at least a phoneme sequence corresponding to time information, which is used when the synthesized sound is generated by the speech synthesis unit, and outputs the segmentation information to the conversion function generation unit The voice quality conversion device according to claim 1, further comprising:   A segmentation information input unit that inputs segmentation information including at least a phoneme sequence corresponding to time information of the synthesized sound created by the speech synthesis unit and outputs the segmentation information to the speech synthesis unit; The voice quality conversion device according to any one of claims 1 to 3, wherein   Second segmentation information including a phoneme string corresponding to at least time information of the target speaker voice recognized by the voice recognition unit is stored and output to at least one of the conversion function generation unit and the voice synthesis unit. The voice quality conversion device according to claim 3, further comprising a segmentation information storage unit.   The voice quality conversion device according to any one of claims 1 to 3, wherein the voice synthesis data stored in the voice synthesis data storage unit includes voice data of a plurality of different speakers. . A data storage unit that stores a converted signal output from the voice quality conversion unit of the voice quality conversion device according to claim 2 as voice synthesis data;
A symbol string input unit for inputting a symbol string describing the utterance content;
A speech synthesis output unit that synthesizes and outputs speech from speech synthesis data in the speech synthesis data storage unit according to the input symbol string;
A synthesized sound generating device comprising: A conversion data storage unit that stores in association with claim 1 conversion function identified to correspond to the characteristic parameters and the characteristic parameters of the synthesized speech in voice quality conversion device according,
An audio input unit for inputting the converted audio signal;
The feature parameter of the input converted speech signal is obtained, the feature parameter of the synthesized sound in the converted data storage unit corresponding to the obtained feature parameter is searched, and the conversion function corresponding to the feature parameter of the synthesized sound is obtained. A voice conversion output unit for reading and converting the converted voice signal by the conversion function;
A synthesized sound generating device comprising:
A method for converting voice quality by a voice quality conversion device,
Inputting the voice of the speaker that is the voice quality conversion target (referred to as "target speaker voice");
Inputting a phonetic symbol string describing the utterance content of the target speaker voice, and creating synthesized speech from the phonetic symbol string using data for speech synthesis stored in advance in a storage unit;
Analyzing the synthesized sound and extracting characteristic parameters of the synthesized sound;
Analyzing the target speaker voice and extracting feature parameters of the target speaker voice;
Correlating the feature parameter of the target speaker voice and the feature parameter of the synthesized sound on the time axis;
Generating a conversion function for converting the spectrum shape of the synthesized speech into the spectrum shape of the target speaker speech based on the target speaker speech and the synthesized speech feature parameters that are associated with each other; and
A voice quality conversion method comprising:   11. The voice quality conversion method according to claim 10, further comprising: converting a voice signal to be converted into a voice signal having a voice quality of the target speaker voice using the conversion function. Recognizing the input target speaker voice,
The voice quality conversion method according to claim 10, wherein the phonetic symbol string is generated from a result of the voice recognition.   The segmented information including a phoneme string corresponding to at least time information of the synthesized sound is input, and the synthesized sound is generated based on the segmentation information. Voice quality conversion method.   The step of performing the association on the time axis according to first segmentation information including at least a phoneme string corresponding to time information, which is used when generating the synthesized sound, includes: The voice quality conversion method according to any one of the above.   11. The step of associating on the time axis with second segmentation information including at least a phoneme sequence corresponding to time information of the target speaker voice recognized by the voice is included. The voice quality conversion method according to any one of?   The step of generating the synthesized sound based on second segmentation information including at least a phoneme string corresponding to time information of the target speaker voice recognized by the voice is included. 15. The voice quality conversion method according to any one of claims 14 and 14.   Based on the second segmentation information, the synthesis is performed based on the second segmentation information based on the second segmentation information including at least the phoneme string corresponding to the time information of the target speaker voice recognized by the voice. The voice quality conversion method according to claim 10, further comprising a step of generating a sound. A method for converting voice quality by a voice quality conversion device,
Inputting the voice of the speaker that is the voice quality conversion target (referred to as "target speaker voice");
Recognizing the input target speaker voice;
Generating a phonetic symbol string describing the utterance content of the target speaker voice from the result of the voice recognition;
Including,
(A) creating synthesized speech from the phonetic symbol string using speech synthesis data stored in advance in the storage unit;
(B) analyzing the synthesized sound and the target speaker voice and extracting respective characteristic parameters;
(C) associating the feature parameter of the target speaker voice with the feature parameter of the synthesized sound on the time axis;
(D) generating a conversion function for converting the spectrum shape of the synthesized sound into the spectrum shape of the target speaker voice based on the two corresponding feature parameters;
(E) converting the voice quality of the voice to be converted using the generated conversion function;
(F) storing the conversion result of the voice quality in the storage unit as the data for voice synthesis;
Is repeated until a predetermined convergence condition is reached. In the computer that composes the voice quality conversion device,
Input the voice of the speaker that is the voice quality conversion target (referred to as "target speaker voice"),
A process of inputting a phonetic symbol string describing the utterance content of the target speaker voice and creating a synthesized sound from the phonetic symbol string using data for speech synthesis stored in a storage unit in advance,
A process of analyzing the synthesized sound and the target speaker voice and extracting respective characteristic parameters;
A process for associating the feature parameter of the target speaker voice and the feature parameter of the synthesized sound on the time axis;
A process for generating a conversion function for converting a spectrum shape of the synthesized sound into a spectrum shape of the target speaker sound based on the target speaker voice and the characteristic parameter of the synthesized sound that are associated with each other;
A program that executes The program according to claim 19, wherein
A program for causing the computer to further execute a process of converting an audio signal to be converted into an audio signal having a voice quality of the target speaker voice using the conversion function. The program according to claim 19, wherein
Processing for recognizing the input target speaker voice;
A program for causing the computer to execute processing for generating the phonetic symbol string from the result of the speech recognition. In the program according to any one of claims 19 to 21,
A program for causing the computer to execute a process of inputting segmentation information including a phoneme string corresponding to at least time information of the synthesized sound and generating the synthesized sound based on the segmentation information. In the program according to any one of claims 19 to 21,
A program for causing the computer to execute a process of performing association on the time axis by using first segmentation information including at least a phoneme string corresponding to time information, which is used when generating the synthesized sound. The program according to any one of claims 19 to 21 and 23,
A program for causing the computer to execute a process of performing association on the time axis by using second segmentation information including at least a phoneme string corresponding to time information of the target speaker voice recognized by the voice. The program according to any one of claims 19 to 21 and 23,
A program for causing the computer to execute a process of generating the synthesized sound based on second segmentation information including at least a phoneme string corresponding to time information of the target speaker voice recognized by the voice. The program according to any one of claims 19 to 21 and 23,
Corresponding on the time axis is performed by second segmentation information including at least a phoneme sequence corresponding to time information of the target speaker speech recognized by the speech, and the synthesized sound is based on the second segmentation information. The program which makes the said computer perform the process which produces | generates. In the computer that composes the voice quality conversion device,
Input the voice of the speaker that is the voice quality conversion target (referred to as "target speaker voice"),
Processing for recognizing the input target speaker voice;
Generating a phonetic symbol string describing the utterance content of the target speaker voice from the result of the voice recognition, and
(A) a process of creating a synthesized sound from the phonetic symbol string using speech synthesis data stored in advance in the storage unit;
(B) a process of analyzing the synthesized sound and the target speaker voice and extracting respective characteristic parameters;
(C) a process of associating the feature parameter of the target speaker voice and the feature parameter of the synthesized sound on the time axis;
(D) generating a conversion function for converting the spectrum shape of the synthesized sound into the spectrum shape of the target speaker voice based on the two associated feature parameters;
(E) a process of converting the voice quality of the voice to be converted using the generated conversion function;
(F) a process of storing the conversion result of the voice quality in the storage unit as the data for voice synthesis;
Is repeated until a predetermined convergence condition is reached,
A program that executes processing.

Images