JP2007114355A - Voice synthesis method and apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a high-quality synthesized voice, and to provide a voice synthesis method that is superior in processability. <P>SOLUTION: A speech spectrum is expressed with few parameters, by approximating a speech spectral envelope with a mixed Gaussian distribution function, and analysis parameters are obtained. Superposition of Gabor functions which are the inverse Fourier transform of the mixed Gaussian distribution function is made into fundamental waveform, and voiced sound is synthesized, by arranging it for each pitch period. Voiceless sound is also synthesized, when the pitch period is rendered random. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

The present invention relates to a speech synthesis technique.

With the progress of speech information processing in computers, speech synthesis can generate high-quality and various styles of speech that can be applied not only to reading text but also to various requirements such as interactive synthesized speech. Speech synthesis is awaited.

PSOLA and waveform-connected speech synthesis methods (Non-Patent Document 1) can synthesize high-quality synthesized speech if there is sufficient variety of speech segment data, but synthesize speech with conditions not included in the database. However, the processability of manipulating voice features that adapt to the speaker is not high. Even if it is dealt with by supplementing the voice data according to the style of voice to be synthesized, it is expected that it is difficult to collect data corresponding to various utterance styles. For this reason, it is thought that it is inefficient to generate emotional speech, dialogue speech, and the like.

In contrast, in filter-type speech synthesis, which is a representative example of a parametric speech synthesis method, the spectral envelope and the fine structure are separated (approximately). For this reason, F ₀ can be arbitrarily changed, and the filter characteristics are controlled with a relatively small number of parameters to generate a speech spectrum, so that high workability is expected. LPC (Non-Patent Document 2), PARCOR (Non-Patent Document 4), LSP (Non-Patent Document 4), Cepstrum (Non-Patent Document 5), and the like have been proposed as parameters that give filter characteristics, and each has a relatively high quality. A speech analysis and synthesis method has been established. However, in these methods, since the relationship between the filter parameters and the voice quality and style of the voice is not uniquely determined, it is not easy to freely control the nature of the voice.

The filter-type speech synthesis exhibits considerably high quality when used as a speech analysis / synthesis system. However, the voice quality is generally lower than that of waveform-connected speech synthesis, such as when driving at F ₀ different from the time of analysis. As one factor, attention can be paid to the gain characteristic and time characteristic of the filter described below.

We consider the analysis and synthesis of voiced sound in the speech analysis and synthesis method (LPC system) using all-pole filters. In general, there is often a large level difference (spectrum dynamic range) reaching several tens of dB between the peaks and valleys of the speech spectrum envelope, and in order to express this with a model using a small number of parameters, Such an all-pole filter having a relatively low order is used. The all-pole filter is a multiple resonance system, but for this reason, the Q value of the resonance characteristic of each pole tends to take a larger value than the actual vocal tract characteristic.

In the time characteristic of such a frequency characteristic filter, a gain substantially proportional to the Q value is generated with respect to the signal component of the resonance frequency, and the output amplitude rises and attenuates with a time constant substantially proportional to the Q value. Considering the case of synthesizing speech by controlling the accent (pitch), the all-pole filter is driven with F ₀ different from the analysis time, and the harmonic component of the driving sound source signal coincides with the resonance frequency of high Q value. In such a case, it takes time to rise and fall (fall) the output amplitude, and as a result, the time control characteristic of the synthesized speech is deteriorated. Then, by superimposing such a sound on the subsequent sound, there is a possibility that it becomes a cause of a “crisp” sound with an impression that an echo is applied.

Figure 1 shows an impulse train (corresponding to voiced sound drive) of one frame length (30 msec) to an all-pole filter obtained by LPC analysis of a section corresponding to phoneme / o / in a certain voice data. This is the output waveform when input. The output amplitude continues to increase with respect to the input (it takes time to reach a steady state), and the output continues for several tens of milliseconds after the input is completed. Further, since the gain of the output signal is proportional to the Q value in the filter, the gain greatly varies depending on the pitch frequency of the driving sound original signal. Because of this phenomenon, it is difficult to control the power of synthesized speech in the filter type speech synthesis.

These problems are by no means a special situation and can often occur in LPC systems. In order to show this experimentally, we prepared a long voice (about 1 minute) and analyzed and synthesized it with the LPC system. First, an experiment was conducted to investigate the time control characteristics. The pitch cycle was changed from 0.8 to 1.2 in steps of 0.02, and input to the analyzed filter for 30 msec. After that, synthesis was continued without input, and the decay time was examined at each frame and pitch period. However, the decay time is defined as the time from when input is stopped until the power of the synthesized speech drops by 30 dB. In addition, power was defined as the sum of squares of amplitude for 10 msec in order to follow fast changes. In FIG. 1, 55 msec is the decay time. FIG. 2 shows the decay time as a histogram in units of 5 ms. It can be said that the more the distribution is biased to the right, the longer the decay time. Furthermore, in order to investigate the gain characteristics, the pitch frequency was similarly changed to synthesize the entire speech, and the power of each frame in the voiced section was examined. The power changes by changing the pitch frequency of the driving sound source in the same frame. The difference in power between the maximum and the minimum is shown by the histogram in FIG. It can be said that the more the distribution is biased to the right, the greater the change in gain. From these results, it is possible to confirm the problem of time characteristics and the phenomenon that the gain changes greatly in the LPC filter.

For these reasons, in the analysis and synthesis of the LPC system, if the pitch frequency of the original voice is used, a relatively high quality analysis and synthesis sound can be obtained. However, when the pitch frequency is different from that of the original sound, the quality deteriorates. Conceivable. This problem is rooted in the nature of the polar filter (cyclic digital filter) and is difficult to solve. If the Q value is lowered to improve it, the level difference between the peaks and valleys of the envelope cannot be formed, and a buzzy impression sound with low clarity is generated.

In the CSM method (Non-Patent Document 6), a plurality of sinusoidal frequencies (CSM frequencies) substantially corresponding to the formant frequency are obtained by CSM speech analysis, which is a speech analysis method based on a line spectrum model. Then, using the sum of the sine waves of those frequencies as a basic waveform, the phase is reset to 0 for each basic period to synthesize speech. The amplitude was also multiplied by an exponential decay to broaden the line spectrum. Since this is a method in which speech synthesis can be performed parametrically without using a recursive filter, the amplitude control is extremely easy, so that “crisp” speech synthesis can be expected. However, since the CSM method corresponds to approximating the speech spectrum with a line spectrum as shown in FIG. 4, there remains room for study as a method for reproducing the spectrum.
JP-B 61-13600 Nick Campbell, Alan Black: “CHATR: Natural Speech Waveform-Connected Arbitrary Speech Synthesis System,” Signal Processing Society of Japan Technical Report, vol.96, no. 39, pp. 45-52, 1996. F. Itakura and S. Saito: “AnalysisSynthesis Telephony Based on the Maximum Likelihood Method,” Proc. 6th Int. Congresson Acoustics, 1968 Nobuhiko Kitawaki, Fumitada Itakura, Shuzo Saito: “Optimum Code Construction in PARCOR Type Speech Analysis and Synthesis System,” IEICE Transactions, J61-A, pp.119-126, 1978 Noboru Tsumura and Fumita Itakura: “Linear Spectrum Pair (LSP) Speech Information Compression by Speech Analysis and Synthesis,” IEICE Transactions, J64-A, pp. 599-606, 1981. Sei Imai, Tadashi Kitamura, Hiroyuki Takeya: “Speech analysis using two-dimensional cepstrum,” IEICE Transactions, J59-A, pp. 1096-1103, 1976. Shigeki Hiyama and Fumitada Itakura: “Speech synthesis using composite sine waves,” Spoken Papers, S79-39, pp.293-300, 1979. ParhamZolfaghari, Tony Robinson, “Formant Analysis Using Mixture of Gaussians,” Proc.ICSLP 96, vol. 2, pp. 1229. 1232, 1996. Shigeki Kajiyama, Sadahiro Furui: “A method of pitch extraction using lag windows,” Proceedings of the IEICE National Convention, 1235, Vol. 5, p. 263, 1978. Hirokazu Kameoka, Takuya Nishimoto, Shigeki Hiyama, “Simultaneous estimation of acoustic features of music using harmonic time structured clustering (HTC),” IPSJ SIG, 2005-MUS-61-12, pp. 71-78, 2005. Hirokazu Kameoka, Junki Ono, Shigeki Hatakeyama: “Speech analysis using synthetic function model of spectral envelope and harmonic structure,” Proceedings of the 2005 Autumn Meeting of the Acoustical Society of Japan, 2-6-4, 2005.

It is an object of the present invention to provide a high-quality synthesized speech and a speech synthesis method with excellent processability.

The speech synthesis method employed by the present invention is based on speech spectral features obtained by speech analysis that approximates the spectral envelope of speech for each frame with a mixed distribution of a predetermined number of unimodal functions, in the time domain. A composite function obtained by superimposing a predetermined number of functions corresponding to peak functions is used as a basic waveform, and the basic waveform is arranged at a predetermined driving time point. Voiced sound has a substantially periodic waveform, and a certain waveform is repeated periodically. In the present specification, the waveform that is periodically repeated is referred to as a basic waveform. In the present invention, the basic waveform can be obtained based on the voice spectrum feature amount. The basic waveform may be obtained based on the processed (changed) voice spectrum feature amount.

In one preferable aspect, the speech spectrum feature amount is acquisition of a model parameter of a mixture distribution of a unimodal function. The target basic waveform can be generated from the parameters of the mixture distribution. Typically, the model parameters include the mean, variance, and weight of each unimodal function. Note that the number of mixtures of unimodal functions may be included in the parameters. In one preferred aspect, the model parameters are obtained using an EM algorithm. In this specification, the “EM algorithm” is used to include an algorithm that is substantially equivalent to the EM algorithm as used in Non-Patent Document 9.

The basic waveform in the time domain is considered to correspond to the mixed distribution in the frequency domain, and the basic waveform corresponds to an inverse Fourier transform of the mixed distribution. In the present invention, the inverse Fourier transform is not indispensable, and if the correspondence between a certain function in the time domain and a certain function in the frequency domain is known, the basic function is directly used using the parameters of the frequency domain function (speech spectrum feature). Waveform can be calculated. In one preferred embodiment, a Gaussian distribution function in the frequency domain is associated with a Gabor function in the time domain. Therefore, in this case, the mixed distribution is a mixed Gaussian distribution including a predetermined number of Gaussian distribution functions, and the basic waveform is a composite Gabor function formed by superimposing a predetermined number of Gabor functions.

In one aspect, the speech synthesis method includes a speech analysis step of approximating a spectrum envelope of speech for each frame with a mixed distribution of a predetermined number of unimodal functions to obtain a speech spectrum feature amount. The acquisition of the spectrum envelope is not essential, and the spectrum envelope acquired and stored in advance may be analyzed. In one aspect, the spectral envelope is obtained by smoothing a speech spectrum using a lag window.

In one aspect, the speech synthesis method includes pitch extraction. In one aspect, the speech synthesis method includes determination of voiced / unvoiced sound.

In one aspect, the voice is a voiced sound, and the driving time point is set for each pitch period. That is, a voiced sound is synthesized by arranging basic waveforms at a pitch period. In one aspect, when setting the driving time point, the superimposition time point is shifted for each component of the composite waveform and superimposed at a pitch period. Even in the case of voiced sound, the arrangement at the time of driving is not limited for each period. In one aspect, there are a plurality of driving time points within a pitch period. In accordance with the multi-pulse method in LPC, large and small driving time points may be appropriately arranged, and a “composite Gabor function” may be arranged there. This is expected to improve the synthesized speech quality.

In one aspect, the voice is an unvoiced sound, and the driving time is set at a random interval. For example, convolution of a random signal and a composite Gabor function can be considered. As for the unvoiced sound, a conventional unvoiced sound generation method may be employed.

The speech synthesizer employed by the present invention is based on speech spectral features obtained by speech analysis that approximates the spectral envelope of speech for each frame with a mixed distribution of a predetermined number of unimodal functions. A composite function obtained by superimposing a predetermined number of functions corresponding to peak functions is used as a basic waveform, and the basic waveform is arranged at a predetermined driving time point. In one preferred embodiment, the mixed distribution is a mixed Gaussian distribution composed of a predetermined number of Gaussian distribution functions, and the basic waveform is a composite Gabor function formed by superimposing a predetermined number of Gabor functions.

In one aspect, the speech synthesizer includes: a speech analysis unit that obtains a speech spectrum feature amount by approximating a spectrum envelope of speech for each frame by a mixed distribution of a predetermined number of unimodal functions; And a speech synthesizer that uses a composite function formed by superimposing a predetermined number of functions corresponding to the peak function as a basic waveform, and arranges the basic waveform at a predetermined driving time point.

In one aspect, the speech synthesizer stores a speech spectrum feature obtained by speech analysis that approximates a spectrum envelope of speech for each frame with a mixture distribution of a predetermined number of unimodal functions, and a time And a speech synthesizer that uses a composite function obtained by superimposing a predetermined number of functions corresponding to the unimodal function in a region as a basic waveform, and places the basic waveform at a predetermined driving time point.

All the speech synthesis methods according to the present invention can be executed by a computer. The speech synthesizer according to the present invention can be configured by a computer (including an input unit, an output unit, a display unit, a calculation unit, and a storage unit). Therefore, the present invention further relates to a computer program for causing a computer to perform speech synthesis according to the present invention, or a computer-readable recording medium on which the computer program is recorded.

In one aspect, the invention provides a computer for performing speech synthesis based on speech spectral features obtained by speech analysis that approximates the spectral envelope of speech for each frame with a mixed distribution of a predetermined number of unimodal functions. Is a computer program for speech synthesis for functioning as a basic waveform a composite function obtained by superimposing a predetermined number of functions corresponding to the unimodal function in the time domain, and for arranging the basic waveform at a predetermined driving time point .

In one aspect, the present invention is obtained by speech analysis that approximates a speech spectral envelope per frame with a mixed distribution of a predetermined number of unimodal functions for speech synthesis based on speech spectral features. Storage means for storing the voice spectrum feature value, and means for arranging a basic function at a predetermined driving time point with a complex function obtained by superimposing a predetermined number of functions corresponding to the unimodal function in the time domain as a basic waveform And a computer program for speech synthesis for functioning.

In one aspect, the present invention provides a speech spectrum by speech analysis that approximates a speech spectral envelope for each frame with a mixed distribution of a predetermined number of unimodal functions for speech synthesis based on speech spectrum features. Means for acquiring a feature value, and means for setting a basic function as a composite waveform obtained by superimposing a predetermined number of functions corresponding to the unimodal function in the time domain, and disposing the basic waveform at a predetermined driving time point, A computer program for speech synthesis to function.

According to the present invention, high-quality synthesized speech can be obtained compared to filter-type speech synthesis. The speech synthesis method according to the present invention performs speech synthesis based on speech spectrum feature amounts, and is superior in workability compared to waveform-connected speech synthesis. Therefore, according to the present invention, it is possible to perform speech synthesis capable of generating high-quality and various styles of speech suitable for generating conversational speech.

The speech analysis and synthesis method according to the present invention will be described based on a composite wavelet model (CWM) which is one preferred embodiment.

[A] Speech Synthesis by Connection of Basic Waveforms First, speech synthesis by connection of basic waveforms, which is a premise of the composite wave model according to the present invention, will be described. Considering the synthesis of voiced sound in the method described in the conventional example as a linear system that inputs an impulse train of pitch period, and organizing it by the impulse response of the linear system, the waveform period type itself is the pitch period waveform of the speech waveform. In contrast to the impulse response, the all-pole filter corresponds to the inverse Fourier transform of the estimated spectral envelope.

When this is interpreted as a repetition of the basic waveform and compared, the pitch periodic waveform in the waveform connection type can be seen as a superposition of the basic sine wave and its many harmonic sine waves that compose this, but these individual amplitude phases are the pitch. Because it depends largely on itself, it is not suitable for control independent of pitch.

On the other hand, in the CSM synthesis, the sine wave fragment corresponding to the formant frequency is the basic waveform, and in the all-pole filter, the exponential damped sine wave that is the impulse response of the simple vibration (secondary system) is the basic waveform, Either can be interpreted as a superposition of these basic waveforms. A necessary property for these basic waveforms is to have a spectrum that closely approximates the spectral envelope of the speech. From this point of view, a long basic waveform as in the case of the cyclic filter is not necessarily required, and the cyclic filter is simply a factor of deteriorating the time characteristic.

Therefore, for speech synthesis with good parametric and time characteristics, at least for synthesis of voiced sound, it is advantageous to use a method that repeats the inverse Fourier transform of the spectral envelope at the pitch period and multiplies it by the desired amplitude without using a recursive filter. is there.

[B] Modeling of basic waveform If the basic waveform of synthesized speech can be expressed by a small number of easy-to-handle parameters, processing such as manipulating voice quality and emotion of synthesized speech may be facilitated. The requirements include:
(1) A parametric method capable of expressing various voices with a small number of parameters.
(2) In order to express a large dynamic range of the speech spectrum and keep the Q value low, the system should not use a cyclic filter.
Is required.

が成り立つ。すなわち、周波数領域のガウス分布関数は、図5(a)に示すように、時間領域ではガウス分布関数と正弦波の積であるGabor 関数で表される。ガウス分布関数はdB尺度で見れば下に開いた放物線であり、これを共振特性と考えるとQ値を抑えつつ、かつ大きな山と谷を形成するのに都合がよい。これらの関数対は、スペクトル領域でも時間領域でも大きく拡がらない利点を持つ。これを音声のフォルマントに対応づけて考える。 Therefore, pay attention to the following Fourier transform formula. If ω is frequency, t is time, and a, b, and c are arbitrary real numbers,

Holds. That is, the Gaussian distribution function in the frequency domain is represented by a Gabor function that is the product of a Gaussian distribution function and a sine wave in the time domain, as shown in FIG. 5 (a). The Gaussian distribution function is a parabola that opens downward on the dB scale. Considering this as a resonance characteristic, it is convenient to form large peaks and valleys while suppressing the Q value. These function pairs have the advantage that they do not expand significantly in the spectral or time domain. This is considered in association with the voice formant.

Therefore, as shown in FIG. 6, if the speech spectrum envelope is approximated by a mixed Gaussian distribution function model (GMM), a basic waveform can be generated from the spectrum envelope represented by the GMM. When the (amplitude) spectrum envelope is approximated by superimposing a plurality of Gaussian distribution functions as shown in FIG. 5B, the basic waveform is a superposition of a plurality of Gabor functions. Therefore, this method is modeled on Composite Sinusoidal Modeling and Gabor is used instead of sine wave.
The composite wavelet model (CWM: Composite Wavelet Model) is used in the sense that superposition of wavelets is used as a basic waveform. In general, GMM is an abbreviation for Gaussian Mixture Model, which means a mixed Gaussian distribution density model, and its integral value must be equal to 1. However, in this specification, GMM means a mixed Gaussian distribution function model as a model of a spectrum (power spectrum or zero-phased amplitude spectrum).

[C] Speech analysis method by approximation of GMM using EM algorithm The spectrum envelope is approximated by a small number of Gaussian distribution functions to obtain speech spectrum features (average, variance, weight), and the average of each mixture component Can be expected to correspond to the formant frequency, and the dispersion corresponds to the spread of the formant, and there is a possibility that the formant structure of the speech can be directly manipulated by the analysis parameter (speech spectral feature). This is considered to be advantageous in terms of voice quality conversion utilizing the knowledge of phonetics as in the formant speech synthesis. On the contrary, when the spectral envelope is approximated with a large number of Gaussian distribution functions, it is difficult to process, but it is expected that the accuracy of the approximation is improved and the voice quality is improved.

Non-Patent Document 7 discloses a method of approximating a spectrum envelope with a mixed Gaussian distribution function for formant analysis of a speech spectrum. In the speech analysis in the present invention, the method disclosed in Non-Patent Document 7 can also be used. However, it is not obvious whether the EM (Expectation-Maximization) algorithm relating to the distribution density function estimation can be used for power spectrum modeling as it is. This is discussed in Non-Patent Document 9, and the KL scale of model parameters (Kullback-LeiblerL information amount and pseudo distance between functions of the same type) for the observed spectrum is minimized by an algorithm of the same type as the EM algorithm ( Or it can be minimized). Here, based on the principle, spectrum parameters are extracted by GMM estimation of the speech spectrum in units of analysis frames based on an algorithm that is the same type as the EM algorithm. In this specification, the “EM algorithm” is used to include an algorithm that is substantially equivalent to the EM algorithm as used in Non-Patent Document 9.

Non-Patent Document 7 points out the case where GMM convergence converges to a pitch structure, not an envelope. In this embodiment, the problem is avoided by obtaining and using a smoothed power spectrum by multiplying the autocorrelation function by a lag window and performing Fourier transform. For the calculation of the spectral envelope using the lag window, Patent Document 1 can be referred to.

[D] Speech Analysis / Synthesis Procedure Using CWM The procedure of the speech synthesis method of the present invention will be described. The speech synthesis method according to the present invention includes speech analysis steps that are executed in advance and speech synthesis steps that are performed through storage, transmission, and processing of the analysis results. Hereinafter, one preferred embodiment of the speech analysis step and one preferred embodiment of the speech synthesis step will be exemplified.

[D-1] Analysis System Procedure (1) A speech spectrum feature amount is calculated for each frame.
The details are, for example,
(1a) Perform difference processing of speech waveform (for example, pass through high frequency emphasis filter);
(1b) Cut out speech waveform every short time and apply data window (Hamming window etc.);
(1c) Find the autocorrelation function of the audio signal;
(1d) Multiply the autocorrelation function by a window (lag window);
(1e) Fourier transform (by an algorithm such as FFT);
(1f) Find the square root for each frequency point (this gives a smoothed amplitude spectrum that is zero-phased);
Do that.
Patent Document 1 can be referred to for the calculation of the speech spectrum here. In addition, various other known methods can be employed for calculation of the voice spectrum (acquisition of the voice spectrum envelope).

(2) The speech spectrum for each frame is approximated by a mixed Gaussian function.
The details are, for example,
The model parameters (mean, variance, weight) of the mixed Gaussian function (GMM) are determined by an algorithm similar to the EM algorithm, starting from appropriate initial values.
That is, if the number of mixing is m,
Each mean μ _i (i = 1,2,3, ..., m),
Each variance σ _i ² (i = 1,2,3, ..., m),
Each weight w _i (i = 1,2,3, ..., m),
Is the spectrum analysis result.
For details of an algorithm similar to the EM algorithm used here, Non-Patent Document 9 can be referred to. The method of approximating the speech spectrum envelope with the GMM is not limited to that described here, and the method described in Non-Patent Document 7 and other methods may be used. For example, spectral envelope estimation described in Non-Patent Document 10 can be used.

Depending on the purpose, voiced / unvoiced sound may be determined. In the case of voiced sound, a fundamental frequency (pitch frequency) may be obtained and added to the analysis result. An existing pitch extraction method can be used for voiced / unvoiced sound determination and F ₀ estimation. Moreover, nonpatent literature 8 can be referred about the pitch extraction using a lag window.

[D-2] Synthesis System Procedure (1) Determine the driving time. In the case of voiced sound, the driving time is determined at every pitch period, and in the case of unvoiced sound, the driving time is determined randomly.
In this specification, in the case of synthesis of voiced sound, the basic waveform is periodically repeated (actually, the basic waveform gradually changes shape), but the position at which the basic waveform is arranged is called a driving time point. In the case of CWM synthesis, the CWM basic waveform is a symmetrical waveform with the origin as the center, but this is periodically arranged on the time axis to create a periodic waveform. Such a time point at which the center of the basic waveform is placed is called a drive time point.

(2) Driving a “composite Gabor function” (superimposed of multiple Gabor functions) corresponding to the inverse Fourier transform of the mixed Gaussian function obtained (or processed) from the analysis of the frame corresponding to the driving time point Place at the time. A weighted sum of Gabor functions corresponding to the inverse Fourier transform of the GMM is obtained from the average μ _i of the Gaussian function for each frame, the variance σ _i ² , and the weights w _i (where i = 1,..., M). This is periodically arranged at pitch cycle intervals to obtain a speech synthesis output.
(2-1) The Gabor functions are not all centered, but if they are shifted appropriately, the overall energy can be increased (improvement of the crest factor) while lowering the peak. At the same time, the quality may be improved by adjusting the phase of the synthesized speech waveform.
(2-2) In the case of voiced sound, it is not limited to each cycle, and a large and small driving time point may be appropriately arranged in accordance with the multi-pulse method in LPC, and a “composite Gabor function” may be arranged there. . Expected to improve synthesized speech quality. By continuously changing the multi-pulse corresponding to the driving time point from a single pulse to a composite pulse, and further to a random pulse, it is possible to continuously generate voiced to unvoiced sound, and smoother and more natural sound. Can be synthesized.
(2-3) In the case of unvoiced sound, convolution of a random signal and a composite Gabor function may be used. In addition, various modifications such as using the conventional method only for the unvoiced sound generation method are conceivable.

[E] Speech synthesis experiment In order to confirm the effectiveness of the speech analysis and synthesis method of the present invention, it was confirmed whether speech was reproduced by analysis and synthesis. In addition, in order to solve the problems of the conventional method, we compared it with the LPC method.

[E-1] Experimental conditions First, in order to verify the operation of the speech synthesis method according to the present invention and the approximation by GMM, speech was analyzed into low-dimensional parameters by the method of the present invention, and synthesis was performed from the parameters. In the experiment, about 5 sentence speeches by female speakers in 3-5 seconds were selected from the ATR speech database and used. Spectral envelopes were extracted by the lag window method for audio with a sampling frequency of 16 kHz and a sample size of 16 bits. Furthermore, the spectral envelope was approximated to the sum of five Gaussian functions. Therefore, the analysis parameters are 15 dimensions per frame. This time, F ₀ was extracted by the F ₀ extraction tool attached to Snack Sound Toolkit (“The Snack Sound Toolkit,” https://www.speechkth.se/snack/). The analysis was performed with a frame length of 30 ms and a frame shift of 10 ms.

First, the speech was synthesized without changing the pitch period and analysis parameters. The unvoiced sound was synthesized by a method that gives a random pitch period. In addition to the comparison by listening, the spectra were compared. Furthermore, the average of the pitch period and analysis parameters was changed to about 0.7 times to 1.3 times, the speech was synthesized, and it was confirmed by listening whether it was broken as speech. The latter corresponds to changing the formant frequency.

In order to show that the time characteristic is improved by the speech synthesis method according to the present invention, an experiment similar to the experiment for the LPC filter described in the conventional example was performed, and the time characteristic and the gain characteristic were examined.

[E-2] Experimental results and discussion It was confirmed by listening experiments that a good voice was synthesized, but a buzzer-like noise was heard in the background. Fig. 8 shows the spectrum of the original speech at the beginning of "I'm glad ..." and the synthesized speech of the proposed method. As can be seen from the figure, the synthesized speech can reproduce the characteristics of the original speech considerably, but the concentration of energy is remarkable because the basic waveform is zero-phased. FIG. 8 shows a part of the sound “A” synthesized by the speech synthesis method according to the present invention. It has a waveform that is clearly different from the original speech, but the spectrum is almost the same. When tests were conducted to change the pitch period and formant frequency, it was possible to synthesize speech without failing under any conditions. 9 and 10 show time characteristics and gain characteristics of the method according to the present invention. Comparison with FIG. 2 and FIG. 3 shows that the time characteristic is improved and the gain is stabilized by the method according to the present invention.

The present invention provides a speech synthesis technology capable of generating voices with high sound quality and various voice qualities, including singing voice synthesis, conversation voice and emotion voice generation, voice dialogue system, car navigation system, HMM voice synthesis system, It can be used in various situations such as combining anthropomorphic agents, robots, and visually impaired support.

An example of time characteristics of LPC filter output is shown below. The trend of the time characteristic of the LPC filter output is shown. Shows the trend of LPC filter output gain characteristics. The line spectrum representation of speech spectrum by CSM method is shown. Here is a Fourier transform pair of Gaussian functions. An example of speech spectrum approximation by GMM is shown. An example of spectrograms of original speech (top) and synthesized speech (bottom) is shown. Waveform examples of the original speech (top) and synthesized speech (bottom) are shown. The time characteristic of the method based on this invention is shown. The gain disparity of the method according to the present invention is shown.

Claims (22)

  A function corresponding to the unimodal function in the time domain based on a speech spectral feature obtained by speech analysis that approximates a spectral envelope of speech for each frame with a mixed distribution of a predetermined number of unimodal functions, A speech synthesis method, wherein a composite function formed by superimposing a predetermined number is used as a basic waveform, and the basic waveform is arranged at a predetermined driving time point.   The speech synthesis method according to claim 1, wherein the speech spectrum feature amount is a model parameter of a mixed distribution of unimodal functions.   The speech synthesis method according to claim 2, wherein the model parameters include an average, a variance, and a weight of each unimodal function.   The speech synthesis method according to claim 3, wherein the model parameter is acquired using an EM algorithm.   The speech synthesis method according to claim 1, wherein the basic waveform corresponds to an inverse Fourier transform of the mixture distribution.   6. The mixed distribution according to claim 1, wherein the mixed distribution is a mixed Gaussian distribution including a predetermined number of Gaussian distribution functions, and the basic waveform is a composite Gabor function formed by superimposing a predetermined number of Gabor functions. Speech synthesis method.   The speech synthesis method includes a speech analysis step of obtaining a speech spectrum feature amount by approximating a spectrum envelope of speech for each frame with a mixed distribution of a predetermined number of unimodal functions. Voice synthesis method.   The speech synthesis method according to claim 7, wherein the spectrum envelope is acquired by smoothing a speech spectrum using a lag window.   The speech synthesis method according to claim 1, wherein the speech synthesis method includes pitch extraction.   The speech synthesis method according to claim 1, wherein the speech synthesis method includes determination of voiced / unvoiced sound.   The speech synthesis method according to claim 1, wherein the voice is voiced sound, and the driving time point is set for each pitch period.   The speech synthesis method according to any one of claims 1 to 11, wherein the voice is a voiced sound, and when the driving time point is set, the superposition time point is shifted for each component of the composite waveform and is superposed at a pitch period.   The speech synthesis method according to claim 1, wherein the voice is a voiced sound, and there are a plurality of the driving time points within a pitch period.   The speech synthesis method according to claim 1, wherein the voice is an unvoiced sound, and the driving time is set at a random interval.   A function corresponding to the unimodal function in the time domain based on a speech spectral feature obtained by speech analysis that approximates a spectral envelope of speech for each frame with a mixed distribution of a predetermined number of unimodal functions, A speech synthesizer characterized in that a composite function formed by superimposing a predetermined number is used as a basic waveform, and the basic waveform is arranged at a predetermined driving time point. The speech synthesizer
A speech analysis unit that obtains a speech spectrum feature amount by approximating a spectrum envelope of speech for each frame with a mixed distribution of a predetermined number of unimodal functions;
A speech synthesizer for placing a function corresponding to the unimodal function in a time domain as a basic waveform with a composite function formed by superimposing a predetermined number of the functions, and arranging the basic waveform at a predetermined driving time;
The speech synthesizer according to claim 15. The speech synthesizer
A storage unit for storing a speech spectrum feature obtained by speech analysis that approximates a spectrum envelope of speech for each frame by a mixed distribution of a predetermined number of unimodal functions;
A speech synthesizer for placing a function corresponding to the unimodal function in a time domain as a basic waveform with a composite function formed by superimposing a predetermined number of the functions, and arranging the basic waveform at a predetermined driving time;
The speech synthesizer according to claim 15, comprising:   The mixed mixture is a mixed Gaussian distribution including a predetermined number of Gaussian distribution functions, and the basic waveform is a composite Gabor function formed by superimposing a predetermined number of Gabor functions. Speech synthesizer. A computer to perform speech synthesis based on speech spectral features obtained by speech analysis that approximates the spectral envelope of speech for each frame with a mixed distribution of a predetermined number of unimodal functions;
A computer program for speech synthesis for causing a complex function obtained by superimposing a predetermined number of functions corresponding to the unimodal function in a time domain as a basic waveform and functioning as means for arranging the basic waveform at a predetermined driving time point. A computer to synthesize speech based on speech spectral features,
Storage means for storing a speech spectrum feature obtained by speech analysis that approximates a spectrum envelope of speech for each frame by a mixed distribution of a predetermined number of unimodal functions;
A function corresponding to the unimodal function in the time domain, a complex function formed by superimposing a predetermined number of times as a basic waveform, and means for arranging the basic waveform at a predetermined driving time point;
A computer program for speech synthesis to make it function. A computer to synthesize speech based on speech spectral features,
Means for acquiring a speech spectrum feature amount by speech analysis that approximates a spectral envelope of speech for each frame by a mixed distribution of a predetermined number of unimodal functions;
A function corresponding to the unimodal function in the time domain, a complex function formed by superimposing a predetermined number of times as a basic waveform, and means for arranging the basic waveform at a predetermined driving time point;
A computer program for speech synthesis to make it function.   A computer-readable recording medium on which the speech synthesis computer according to any one of claims 19 to 21 is recorded.

Images