KR101065188B1 - Apparatus and method for speaker adaptation by evolutional learning, and speech recognition system using thereof - Google Patents

Abstract

The present invention relates to a speaker adaptation technology in a speech recognition system. The apparatus for speaker adaptation by evolutionary learning according to the present invention recognizes a speech using a predetermined environment parameter in a recognition mode in which the speech recognition system performs speech recognition. A feature transform unit for performing feature transformation on the feature vector of the target speech data; A voice database storing the recognition target voice data; And a model converter configured to perform model transformation on an existing acoustic model using voice data stored in the voice database in a standby mode in which the voice recognition system does not perform voice recognition. In addition to improving the recognition performance of the speech recognition system, it provides the advantage of simultaneously performing speaker adaptation and environmental adaptation.

Description

Apparatus and method for speaker adaptation by evolutional learning, and speech recognition system using according to the present invention.

The present invention relates to a speaker adaptation technique in a speech recognition system, and more particularly, by performing both feature transformation and model transformation through an unsupervised adaptation method by evolutionary learning, thereby improving user convenience and recognition performance of the speech recognition system. The present invention relates to a speaker adaptation apparatus and method for simultaneously performing speaker adaptation and environment adaptation and a speech recognition system using the same.

Recently, voice recognition technology has been applied to various fields such as home appliances, mobile phones, security and authentication, and its application fields and demands are rapidly increasing.

Here, speech or voice recognition refers to linguistic meaning and content from speech spoken by humans through an automatic speech recognition system. Specifically, a speech waveform is input to identify and process a word or word string. The process of doing that.

Speech recognition systems that perform such speech recognition generally include word recognition systems, continuous speech recognition systems, speaker recognition systems, and the like. Voice recognition algorithms that can be classified and commonly applied to the voice recognition systems include a voice activity detection process, a feature extraction process, and a matching process.

As described above, the speech recognition technology is a kind of pattern recognition technology. In order to finally recognize the meaning of the input speech data through the matching process, a "training process" and a "test process" are required. .

The training process is a process of generating an acoustic model by collecting specific speech samples from a speaker, and the test process is a process of matching a pattern of the generated acoustic model with a series of feature vectors extracted from input speech data. . Therefore, the recognition performance of the speech recognition system depends on how high accuracy or reliability can be obtained when the acoustic model generated through the training process is used for pattern matching with the input speech samples for testing.

On the other hand, the acoustic model used in the speech recognition system includes a speaker independent acoustic model and a speaker dependent acoustic model. The speaker independent acoustic model is a general acoustic model trained using speech data uttered from multiple speakers, while the speaker dependent acoustic model is a unique acoustic model trained using speech data uttered from a specific speaker.

In the case of speaker independent acoustic model, it is designed to show the best recognition performance on average for a wide range of unspecified speakers. However, the speaker-independent acoustic model has a problem in that it cannot exhibit the best recognition performance for a single specific speaker. Therefore, in a speech recognition system applied to a personal mobile device such as a mobile phone or a PDA using only a single specific speaker, a speaker adaptation method of converting a speaker-independent acoustic model into a speaker-dependent acoustic model using a specific speaker's voice is required. Do.

The existing speaker adaptation methods include supervised adaptation and unsupervised adaptation.

However, the map adaptation method has a problem in that the user's convenience is impaired because the user must make a specific sentence or receive transcription data (label or transcription) of the spoken sentence before performing the speech recognition in the actual speech recognition service environment. .

In addition, the unsupervised adaptation method requires considerable adaptation time to generate a speaker-dependent acoustic model, and in particular, the transcription data for adaptation must be predicted by the speech recognition system itself, resulting in inaccurate speech data. There is a problem that causes degradation.

On the other hand, when the speech recognition is actually performed, since the speech is mixed with noise, the speech recognition rate is lowered because the environment does not match during the learning and the speech recognition test. Therefore, a technique for improving speech recognition performance in a noise environment by matching a learning environment with a speech recognition environment in a speech recognition system has been developed.

However, the conventional speaker adaptation technique has a problem that does not provide a speaker adaptation technique capable of efficiently performing speaker adaptation and environmental adaptation simultaneously in a real noise environment.

Accordingly, the first technical problem to be solved by the present invention is to improve both user convenience and recognition performance of the speech recognition system by performing both feature transformation and model transformation through an unsupervised adaptive method by evolutionary learning. The present invention provides a speaker adaptation apparatus that simultaneously performs adaptation and environmental adaptation.

The second technical problem to be solved by the present invention is to improve both user convenience and recognition performance of the speech recognition system, as well as speaker adaptation and performance by performing both feature transformation and model transformation through an unsupervised adaptive method by evolutionary learning. It is to provide a speaker adaptation method that simultaneously performs environmental adaptation.

A third technical problem to be solved by the present invention is to provide a speech recognition system using the speaker adaptation apparatus or the speaker adaptation method.

In order to solve the first technical problem as described above, the present invention is a device for performing speaker adaptation by evolutionary learning in a speech recognition system, in a recognition mode in which the speech recognition system performs speech recognition, a predetermined environment A feature conversion unit for performing feature transformation on the feature vector of the speech data to be recognized using the parameter; A voice database storing the recognition target voice data; And a model converter configured to perform model transformation on an existing acoustic model using voice data stored in the voice database in a standby mode in which the voice recognition system does not perform voice recognition. to provide.

In one embodiment, the feature converter determines the environment parameter in advance using a maximum likelihood method.

In an embodiment, the speaker adaptation apparatus extracts a feature of the ambient noise in a non-voice section other than a voice section detected through voice activity detection in the voice recognition system, so that the recognition target voice data is extracted. It further includes an environment identification unit for identifying the generated environment.

In one embodiment, the voice database stores voice data generated in the environment for each of the environments identified by the environment identification unit.

In one embodiment, the model conversion unit, using the voice data stored in the voice database performs a model conversion for the existing acoustic model for each identified environment.

In order to solve the second technical problem as described above, the present invention provides a method for performing speaker adaptation by evolutionary learning in a speech recognition system, wherein a predetermined environmental parameter is determined in a recognition mode in which the speech recognition system performs speech recognition. A feature transformation step of performing a feature transformation on the feature vector of the speech data to be recognized using; A voice data storage step of storing the recognition target voice data in a database; And a model transformation step of performing a model transformation on an existing acoustic model using voice data stored in the database in a standby mode in which the speech recognition system does not perform speech recognition. Ball.

In order to solve the third technical problem as described above, the present invention provides a speech recognition system that recognizes speech by performing speaker adaptation by evolutionary learning, in a recognition mode in which the speech recognition system performs speech recognition. A feature converter configured to perform feature transformation on the feature vector of the target speech data using a predetermined environment parameter; A voice database storing the recognition target voice data; A model converter configured to perform model transformation on an existing acoustic model using voice data stored in the voice database in a standby mode in which the voice recognition system does not perform voice recognition; And a recognition unit configured to perform voice recognition using the feature vector converted by the feature conversion unit and the acoustic model converted by the model conversion unit in the recognition mode.

In one embodiment, the feature converter determines the environment parameter in advance using a maximum likelihood method.

In an embodiment, the voice recognition system may identify an environment in which the voice data to be recognized is generated by extracting features of the ambient noise in a non-voice section other than the voice section detected through voice activity detection. It further includes an environment identification unit.

In one embodiment, the voice database stores voice data generated in the environment for each of the environments identified by the environment identification unit.

In one embodiment, the model conversion unit, using the voice data stored in the voice database performs a model conversion for the existing acoustic model for each identified environment.

In one embodiment, the speech recognition system further includes a model database for storing the acoustic model for the environment model-specific conversion by the identified environment by the model conversion unit.

In one embodiment, the speech recognition system further includes a model repositioning unit for limiting the number of acoustic models stored in the model database by a predetermined threshold.

In one embodiment, the recognition unit, the voice recognition using the acoustic model corresponding to the environment in which the voice data to be recognized from the acoustic model stored in the model database.

The present invention provides an advantage of improving user convenience by applying an unsupervised adaptation method by evolutionary learning for speaker adaptation of a speech recognition system.

In addition, by appropriately performing the feature conversion and model conversion according to the operating state of the speech recognition system, it provides an advantage of improving the efficiency of the speaker adaptation method and the recognition performance of the speech recognition system.

Furthermore, by generating a speaker-dependent acoustic model for each speech recognition environment, it provides an advantage of simultaneously performing speaker adaptation and environmental adaptation.

Prior to the description of the specific contents of the present invention, the technical outline of the present invention will be described first for ease of understanding.

The basic purpose of the speaker-to-environment adaptation technique is to reduce speech recognition performance due to inconsistencies between noise-free training data of unspecified majority speakers used in generating acoustic models of speech recognition systems and noise-mixed speech data of specific speakers used in actual tests. It is to be prevented.

Such an adaptation technique may be classified into an adaptation technique based on a “feature transformation” and an adaptation technique based on a “model transformation” according to a target to perform an adaptation transformation.

The feature transformation adaptation technique removes noise from real speech data and maps it to clean speech data by using feature vector compensation, and performs speech recognition quickly through a relatively small amount of computation. There is an advantage to adaptation.

However, the feature transformation adaptation technique has a disadvantage in that the degree of improvement in recognition performance is limited compared to the model transformation adaptation technique.

The model transformation adaptation technique is a technique for performing speech recognition by adapting a previously learned acoustic model to an acoustic model dependent on a real speech recognition environment through model compensation using adaptation data. There is a big advantage.

However, the model transformation adaptation technique requires a relatively large amount of adaptation data and computation time because only model variables related to specific adaptation data are transformed.

Accordingly, the present invention provides a new speaker adaptation technique that effectively applies both techniques in speech recognition and at the same time compensates for the disadvantages. That is, in the recognition mode in which the speech recognition system is performing the speech recognition, the feature conversion to enable rapid environment adaptation is performed (online adaptation), and in the standby mode in which the speech recognition system does not perform the speech recognition, it is stored in advance. A sufficient amount of speech data is used as adaptive data to perform model transformation (offline adaptation). As described above, the present invention ensures stable and high recognition performance regardless of the amount of adaptive data accumulated by repeatedly performing feature transformation and model transformation as an evolutionary learning process for speaker adaptation.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to clarify a solution of the technical problem of the present invention. However, in describing the present invention, when it is determined that the description of the related known technology may make the gist of the present invention unclear, the description thereof will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to intentions or customs of users, operators, and the like. Therefore, the definition should be based on the contents throughout this specification.

1 is a block diagram illustrating an example of a speech recognition system to which the speaker adaptation apparatus 100 according to the present invention is applied.

2 shows an example of a speaker adaptation method by evolutionary learning according to the present invention in a flowchart.

1 and 2, the speaker adaptation apparatus 100 according to the present invention includes a feature converter 102, a voice database 106, and a model converter 108, and an environment identifier 104. It may further include.

First, the speech recognition system initially generates and stores a speaker and an environment-independent acoustic model through a training step (S210). The adaptive technique is to reestimate the speaker and environment-independent acoustic model into a model suitable for a specific speaker and environment.

3 shows the basic principle of the speaker adaptation technique.

Referring to FIG. 3, the speaker adaptation method includes a process of estimating a speaker independent acoustic model 300 and a process of re-estimating the speaker independent acoustic model estimated through the speaker independent acoustic model 310. That is, a multidimensional feature vector such as MFCC (Mel-Frequency Cepstral Coefficient) is extracted from a large amount of speech data obtained from an unspecified number of speakers, and EM (Expectaion-Maximization) technique is applied to the extracted feature vectors. Estimate the Hidden Markov Model (HMM). The estimated HMM has speaker-independent characteristics as an acoustic model for an unspecified number of speakers. On the other hand, if the voice data used in the training of the HMM is collected in a variety of different environments (various kinds of sound equipment, noise environment, etc.), the HMM will also have environmentally independent characteristics at the same time.

When a specific user performs speech recognition using the trained speaker independent acoustic model as described above, the speech characteristics of the specific user may not be reflected and thus satisfactory recognition performance may not be exhibited. Therefore, in order to use voice recognition-based functions smoothly in personal mobile devices such as mobile phones and PDAs, it is necessary to reestimate (adapt) the acoustic model optimized for the acoustic characteristics of a specific user. As will be described again below, the present invention introduces a new adaptation scheme and uses the estimated speaker and environment independent acoustic model and the relatively small amount of speech data obtained from a specific speaker (user) to generate the speaker and environment dependent acoustic model. Estimate.

In the recognition mode in which the speech recognition system performs speech recognition, the feature conversion unit 102 performs a feature transformation on the feature vector of the speech data to be recognized by using a predetermined environment parameter when the speech data is input. It performs (S220, S230). In other words, while the mobile phone, PDA, and other mobile devices to which the voice recognition system is applied are receiving the user's voice command, the feature transformation is performed to enable rapid environment adaptation (online adaptation).

More specifically, the feature transformation is a method of transforming feature vectors extracted from input speech data using environment parameters predicted in advance. That is, the feature conversion process can be basically expressed as Equation 1.

X '= A  + B

In Equation 1, X denotes a feature vector of the input voice data, and A and B denote predetermined environmental parameters. As a result, the feature conversion is to improve the performance of speech recognition by changing only the feature vector X extracted from the input speech data using predetermined environmental parameters A and B. The environmental parameters A and B may be predicted by using a maximum likelihood method and the like, such as voice data obtained in a short section of the front part of the input voice before recognition of the input voice data, and quickly using the predetermined environment parameters. By performing the feature transformation, it is possible to perform the adaptation in real time according to the environment change. More specifically, the environmental parameters A and B correspond to a kind of conversion parameter used to reduce the difference between the input voice and the existing acoustic model. The environmental parameter may be predicted using an existing acoustic model, that is, an existing acoustic model before the feature conversion and data acquired for a short time of the front part of the input voice. For example, the environment parameter may be predicted using a technique of maximizing a likelihood in which a voice may be generated using a Baum's auxiliary function when a specific acoustic model is given, and the predicted environment parameter is converted into the input voice data. It can be applied to the feature vector of, so that the speech recognition performance can be compensated by quickly converting the feature vector into a feature vector that is closer to the actual speech recognition environment.

Of course, various schemes may be applied for the feature conversion in actual implementation. For example, cepstral mean subtraction, mean-variance normalization: On real-time mean-variance normalization of speech recognition features, P.Pujol, D See Macho and C. Nadeu, ICASSP, 2006, pp. 773-776), and RASTA algorithm (ReelAtive SpecTrAl algorithm: Data-driven RASTA filters in reverberation, MLShire et al, ICASSP, 2000, pp. 1627-1630). Histogram normalization: Quantile based histogram equalization for noise robust large vocabulary speech recognition, F. Hilger and H. Ney, IEEE Trans. Audio, Speech, Language Processing, vol. 14, no. 3, pp. 845-854 Augmenting delta feature: On the use of high order derivatives for high performance alphabet recognition, J. diMartino, ICASSP, 2002, pp.953-956). In addition, as a technique for linearly transforming feature vectors, Linear Discriminant Analysis (LDA) and Principal Component Analysis: Optimization of temporal filters for constructing robust features in speech recognition, Jeih-Weih Hung et. Al, IEEE Trans.Audio , Speech, and Language Processing, vol. 14, No. 3, 2006, pp. 808-832) may be applied. In addition, as a method using a non-linear neural network, TRAP (TempoRAl Patterns: Temporal patterns in ASR of noisy speech, H. Hermansky and S. Sharma, ICASSP, 1999, pp. 289-292), ASAT (Automatic Speech Attribute Transcription: A study on knowledge source integration for candidate rescoring in automatic speech recognition, Jinyu Li, Yu Tsao and Chin-Hui Lee, ICASSP, 2005, pp.837-840).

Next, the voice database 106 stores the recognition target voice data as adaptation data to be used for acoustic model adaptation (S240). In this case, in a resource-constrained mobile environment such as a mobile phone or a robot, only a limited amount of voice data may be stored through a first-in, first-out method, rather than storing all voice data.

Then, the model conversion unit 108, in the standby mode in which the speech recognition system does not perform speech recognition (S250), a model for the existing acoustic model using the speech data stored in the speech database 106 The conversion is performed (S260). That is, when the amount of data calculation of the device to which the speech recognition system is applied is not large (for example, at night or during charging), a sufficient amount of adaptive data accumulated in the speech database 106 while in the recognition mode is used. Perform model transformation (offline adaptation). The model transformation refers to a technique of directly converting parameters of an HMM which is an acoustic model. In other words, the acoustic model consists of a mean vector, a covariance matrix, and a weight, wherein the model transformation changes them all or selectively. To this end, parameters are predicted using the speech data stored in the speech database 106 and model parameters are converted using a linear regression tree. This process corresponds to a process of performing unsupervised adaptation in which the system predicts and trains transcription for acoustic model training in a mobile environment.

As described above, in the standby mode of the speech recognition system, although the amount of computation is relatively large, model transformation is performed to maximize recognition performance. At this time, the feature of the stored speech data used by the model converter 108 when the model is transformed may use MFCC (Mel-Frequency Cepstral Coefficient) as in the case of speaker-independent acoustic model estimation. In addition to the typical expectation-maximization (EM) technique, a maximum likelihood linear regression (MLLR) or maximum a posterior (MAP) technique may be used.

When there is the newly stored voice data in the voice database 106, the model converting unit 108 performs model conversion on the final acoustic model using the newly stored voice data (S270). On the other hand, when the voice recognition system is in the recognition mode again (S250), the above-described steps S220 to S240 are repeated.

As will be described again below, in one embodiment of the present invention, since the recognition target voice data has different characteristics according to a user's speech environment (eg, subway, indoor, street, etc.), the speaker adaptation apparatus 100 may be used. The environment identification unit 104 extracts a feature of ambient noise in a non-voice section other than the voice section detected through voice activity detection in the voice recognition system to identify an environment in which the voice object to be recognized is generated. ) May be further included. In this case, the voice database 106 stores voice data generated in the environment for each environment identified by the environment identification unit 104. The model converting unit 108 may perform model transformation on the existing acoustic model for each identified environment by using the voice data stored in the voice database 106. The resulting acoustic models for each environment have speaker-dependent features in that the stored voice data are all acquired from a specific user, and have environmentally-dependent features for training acoustic models for each environment. do.

As a result, the speaker adaptation apparatus according to the present invention simultaneously performs speaker adaptation and environmental adaptation through evolutionary learning that repeatedly performs feature transformation adaptation and model transformation adaptation.

4 shows an example of a speech recognition method by evolutionary learning according to the present invention in a flow chart.

1 and 4, the speech recognition system according to the present invention includes the speaker adaptation apparatus 100, and includes a speech input unit 110, a speech section detector 120, a feature extractor 130, and a model. The database 140 further includes a model rearrangement unit 150 and a recognition unit 160.

First, the voice input unit 110 receives an analog voice signal (S400). In addition, the voice input unit 110 includes a pre-processing process of converting the voice signal into digital voice data through filtering through an anti-aliasing filter and conversion through an analog to digital converter (ADC). Can be performed.

Next, the voice section detection unit 120 performs voice activity detection (VAD) to determine whether the voice data correspond to human voice (S402). In this case, in order to perform the speech section detection, the speech section detection unit 120 uses a machine learning method for detecting a speech section after learning a model using learning data about speech and sounds other than the speech, or Modeling features related to speech characteristics (zero crossing rate, spectral entropy, etc.) and searching for the appearance of the features may be used to detect speech segments.

Next, the feature extracting unit 130 receives the recognition target voice data of the voice section and extracts a feature vector from the recognition target voice data (S404). In this case, the extracted feature vector generally has frequency information over time in a form in which only the components necessary for speech recognition are compressed in the speech object to be recognized. The feature vector includes MFCC (Mel-Frequency Cepstral Coefficients), LPCC (Linear Prediction Cepstral Coefficients), EIH (Ensenble Interaval Histogram), etc. In an embodiment of the present invention, MFCC is used as a feature vector. In addition, the feature extractor 130 performs various preprocessing processes, such as frame unit configuration, hamming window, Fourier transform, filter bank, and cepstrum transform, to extract a feature vector from the speech data to be recognized. Can be done.

Next, as described above, the feature converting unit 102 of the speaker adaptation apparatus 100 extracts the extracted feature using a predetermined environment parameter in a recognition mode in which the speech recognition system is performing speech recognition. Feature transformation is performed on the vector (S406). Then, the recognition unit 160 performs speech recognition using the transformed feature vector and the acoustic model.

On the other hand, the environment identification unit 104 of the speaker adaptation apparatus 100 extracts the feature of the ambient noise in a non-voice section other than the voice section detected by the voice section detection unit 120 to generate the voice data. Identifies the environment (S408). At this time, the environment identification unit 104, the same principle as the feature extraction unit 130 can extract the features of the ambient noise in the non-speech section, and also separate ambient noise estimation algorithms (eg SF Boll, "Suppression of acoustic noise in speech using spectral subtraction", IEEE Transactions on Acoustics, Speech, and Signal Processing, ASSP-27 (2), pp. 113-120, 1979. It may be.

Next, the recognition unit 160 is dependent on the feature vector transformed by the feature transform unit 102 and the environment-dependent transformed by the model transform unit 108 of the speaker adaptation apparatus 100 in the recognition mode. Among the acoustic models, voice recognition is performed on the speech data to be recognized using an optimal acoustic model corresponding to the identified environment (S410).

Next, the voice database 106 of the speaker adaptation apparatus 100 stores the voice data generated in the environment for each of the environments identified by the environment identification unit 104 as adaptation data to be used for acoustic model adaptation ( S412).

On the other hand, if the voice recognition system is still in the recognition mode (S414), the above-described steps S400 to S412 are repeatedly performed.

On the other hand, when the voice recognition system is in the standby mode without performing the voice recognition (S414), the model conversion unit 108 of the speaker adaptation apparatus 100, the voice data stored in the voice database 106 Using the model conversion for the existing acoustic model is performed (S416). In particular, the model converting unit 108 performs model transformation on the existing acoustic model for each of the identified environments by using the voice data stored in the voice database 106. That is, an acoustic model is generated that depends on each of the identified environments.

Next, the model database 140 stores the acoustic model for each environment, which is model-converted for each environment identified by the model converter 108 (S418). That is, the model database 140 may initially store speaker independent or speaker and environment independent acoustic models. However, through the above-described evolutionary learning process, the model database 140 stores speaker and environment dependent acoustic models.

5 shows a process of generating a speaker and an environment dependent acoustic model through evolutionary learning.

Referring to FIG. 5, a speaker and an environment independent acoustic model 500 are initially generated and stored in the speech recognition system. The speaker and the environment-independent acoustic model performs environment transformation of the environment-dependent acoustic models 520 corresponding to each environment by performing model transformation for each environment using the environment-specific voice data 510 stored in the voice database 106. Create When the voice data 530 of the new environment is stored in the voice database 106, model conversion is performed for each of the environment dependent acoustic models 520 using the corresponding voice data for each new environment. Create new environmentally dependent acoustic models 540.

In one embodiment, if the newly stored voice data in the voice database 106 is generated in an environment in which an environment dependent acoustic model has already been generated, a model for the previously generated environment dependent acoustic model using the newly stored voice data. By performing the transformation again, you can adapt it to your specific environment. In addition, if the newly stored voice data is generated in a new environment, as described above, model transformation is performed on all of the already generated environment dependent acoustic models using the newly stored voice data, and the resultant environment dependent The likelihood of the models can be used to generate an acoustic model dependent on the new environment.

Next, the model rearrangement unit 150 limits the number of acoustic models stored in the model database 140 by a predetermined threshold (S420). That is, the number of acoustic models stored and used may be limited in consideration of the storage capacity or the amount of calculation, rather than generating and storing all of the environment dependent acoustic models in a resource-constrained mobile device.

The model repositioner 150 may apply various methods to limit the number of acoustic models below the threshold. For example, in one embodiment, when the number of acoustic models stored in the model database 140 exceeds the threshold, the model rearrangement unit 150 simply selects the newly generated acoustic model from the model database 140. You can restrict it from being stored in. In another embodiment, the model rearrangement unit 150 may use the likelihood of the generated acoustic models or the weighted sum or linear combination of each acoustic model in consideration of the amount of stored data for each environment. linear combination) or the like can reduce the number of acoustic models below the threshold. In addition, in another embodiment, the model rearrangement unit 150 causes the environment identification unit 104 to classify and identify environments in which voice data is generated by only a predetermined number of types. The number of acoustic models generated by 108 may be adjusted below the threshold.

Then, when there is newly stored voice data in the voice database 106 (S422), the voice recognition system repeats the above steps S416 to S420. However, when the voice recognition system is switched to the recognition mode for performing voice recognition, the above-described steps S400 to S412 are repeated.

As described above, each environment-specific acoustic model has speaker-dependent characteristics in that all of the stored voice data are obtained from a specific user, and the environment-dependent characteristics in terms of training the acoustic model for each environment are described. Will be carried.

As a result, the speech recognition system according to the present invention simultaneously performs speaker adaptation and environmental adaptation through evolutionary learning that repeatedly performs feature transformation adaptation and model transformation adaptation.

On the other hand, the recognition unit 160, when performing the voice recognition for the recognition target voice data, by the environment identification unit 104 identifies the generation environment of the recognition target voice data, the model database 140 in the Search and use the acoustic model corresponding to the identified environment. In other words, the recognition unit 160, the recognition mode of the speech recognition system and the feature vector converted by the feature converting unit 102 and the speaker and the environment-dependent acoustic model converted by the model converting unit 108 Voice recognition is performed by using.

In one embodiment, the recognition unit 160, when performing the voice recognition for the voice data to be recognized, the sound having the highest value by calculating the acoustic models and likelihood stored in the model database 140 The model may be determined to be dependent on a generation environment (ie, a current speech recognition environment) of the speech data to be recognized, and speech recognition may be performed using the corresponding acoustic model.

On the other hand, the present invention can be implemented in a program code that can be read by a computer on a computer-readable recording medium. When the present invention is executed through software, the constituent means of the present invention are code segments for performing necessary tasks. In addition, the program or code segments may be stored in a computer readable medium of a computer or transmitted as a computer data signal coupled with a carrier via a transmission medium or a communication network.

Computer-readable recording media include all kinds of recording devices for storing data that can be read by a computer system. For example, the computer-readable recording medium may include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer can store and execute the code that is readable.

As described above, the present invention provides an advantage of user convenience by applying an unsupervised adaptation method by evolutionary learning for speaker adaptation of a speech recognition system. In addition, by appropriately performing the feature conversion and model conversion according to the operating state of the speech recognition system, it provides an advantage of improving the efficiency of the speaker adaptation method and the recognition performance of the speech recognition system. Furthermore, by generating a speaker-dependent acoustic model for each speech recognition environment, it provides an advantage of simultaneously performing speaker adaptation and environmental adaptation.

So far, the present invention has been described with reference to the embodiments. However, one of ordinary skill in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential technical spirit of the present invention. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. That is, the true technical scope of the present invention is shown in the appended claims, and all differences within the equivalent scope will be construed as being included in the present invention.

1 is a block diagram showing an example of a speech recognition system to which the speaker adaptation apparatus according to the present invention is applied.

2 is a flowchart illustrating an example of a speaker adaptation method by evolutionary learning according to the present invention.

3 is a diagram illustrating the basic principle of a speaker adaptation technique.

4 is a flowchart illustrating an example of a speech recognition method by evolutionary learning according to the present invention.

5 is a diagram illustrating a process of generating a speaker and an environment dependent acoustic model through evolutionary learning.

Claims (19)

Apparatus for performing speaker adaptation by evolutionary learning in speech recognition system, A feature conversion unit for performing feature transformation on a feature vector of speech data to be recognized using a predetermined environment parameter in a recognition mode in which the speech recognition system is performing speech recognition; A voice database storing the recognition target voice data; And In a standby mode in which the speech recognition system does not perform speech recognition, a model conversion unit for performing a model conversion for the existing acoustic model using the speech data stored in the speech database, And the speech recognition system performs speaker adaptation and environment adaptation simultaneously by repeating the feature transformation and the model transformation according to a performance mode. The method of claim 1, And the feature transform unit determines the environment parameter in advance using a maximum likelihood method. The method of claim 1, The speaker adaptation apparatus is configured to extract an ambient noise feature in a non-voice section other than a voice section detected through voice activity detection in the voice recognition system to identify an environment in which the voice object to be recognized is generated. Speaker adaptation apparatus by evolutionary learning, further comprising an identification unit. The method of claim 3, wherein And the voice database stores voice data generated in a corresponding environment for each of the environments identified by the environment identification unit. 5. The method of claim 4, And the model converter is configured to perform model transformation on the existing acoustic model for each of the identified environments using the speech data stored in the speech database. In the method of speaker adaptation by evolutionary learning in speech recognition system, A feature transformation step of performing a feature transformation on a feature vector of speech data to be recognized using a predetermined environment parameter in a recognition mode in which the speech recognition system is performing speech recognition; A voice data storage step of storing the recognition target voice data in a database; And And a model conversion step of performing a model transformation on an existing acoustic model using the voice data stored in the database in the standby mode in which the voice recognition system does not perform voice recognition. And the speech recognition system performs speaker adaptation and environment adaptation simultaneously by repeating the feature transformation and the model transformation according to a performance mode. The method of claim 6, The feature conversion step includes the step of pre-determining the environmental parameters using a maximum likelihood method (speaker). The method of claim 6, In the speaker adaptation method, an environment in which the voice recognition system extracts features of ambient noise in a non-speech section other than the voice section detected through voice activity detection to identify an environment in which the voice data to be recognized is generated. Speaker adaptation method by evolutionary learning, characterized in that it further comprises an identification step. The method of claim 8, The voice data storing step is a step of storing the speech data generated in the environment for each environment identified through the environment identification step, speaker adaptation method by evolutionary learning. 10. The method of claim 9, And the model conversion step is a step of performing a model transformation on the existing acoustic model for each of the identified environments by using the voice data stored in the database. A computer readable recording medium having recorded thereon a program for executing a method according to any one of claims 6 to 10 by a computer. In a speech recognition system that recognizes speech by performing speaker adaptation by evolutionary learning, A feature converter configured to perform feature transformation using a predetermined environment parameter on a feature vector of speech data to be recognized in a recognition mode in which the speech recognition system is performing speech recognition; A voice database storing the recognition target voice data; A model converter configured to perform model transformation on an existing acoustic model using voice data stored in the voice database in a standby mode in which the voice recognition system does not perform voice recognition; And In the recognition mode, a recognition unit for performing speech recognition using the feature vector converted by the feature conversion unit and the acoustic model converted by the model conversion unit, The speech recognition system is a speech recognition system by evolutionary learning to perform speaker adaptation and environment adaptation by repeating the feature transformation and the model transformation in accordance with the performance mode. The method of claim 12, The feature conversion unit, the speech recognition system by evolutionary learning, characterized in that for determining the environment parameters in advance using a maximum likelihood method (maximum likelihood method). The method of claim 12, The voice recognition system may further include an environment identification unit for extracting a feature of ambient noise in a non-voice section other than the voice section detected through voice activity detection to identify an environment in which the voice object to be recognized is generated. Speech recognition system by evolutionary learning, characterized in that. The method of claim 14, The voice database, the voice recognition system by the evolution learning, characterized in that for storing the voice data generated in the environment for each environment identified by the environment identification unit. The method of claim 15, The model conversion unit, the speech recognition system by the evolution learning, characterized in that for performing the model transformation for the existing acoustic model for each identified environment using the speech data stored in the speech database. The method of claim 16, The speech recognition system, the speech recognition system by evolution learning further comprises a model database for storing the acoustic model of the environment model-specific conversion by the model conversion unit by the identified environment. The method of claim 17, The speech recognition system further includes a model repositioning unit configured to limit the number of acoustic models stored in the model database by a predetermined threshold. The method of claim 17, The recognition unit, the speech recognition system by the evolutionary learning, characterized in that for performing the speech recognition using the acoustic model corresponding to the environment in which the speech data to be recognized from the acoustic model stored in the model database.

Images