WO2002032208A2

WO2002032208A2 - Method for determining an acoustic environment situation, application of the method and hearing aid

Info

Publication number: WO2002032208A2
Application number: PCT/CH2002/000049
Authority: WO
Inventors: Silvia Allegro
Original assignee: Phonak Ag
Priority date: 2002-01-28
Filing date: 2002-01-28
Publication date: 2002-04-25
Also published as: JP2005504325A; WO2002032208A3; JP3987429B2; EP1470735B1; CA2439427A1; AU2472202A; EP1470735A2; CA2439427C; AU2002224722B2

Abstract

The invention relates to a method and a device for determining an acoustic environment situation. The method consists of processing an acoustic input signal (IN) that is preferably picked up with the help of at least one microphone in at least two processing stages (S1, ..., Sn). At least one of the two processing stages (S1, ..., Sn) comprises an extraction phase in which characteristic features are extracted from the input signal (IN) and each of the processing stages (S1, ..., Sn) comprises an identification phase in which extracted characteristic features are classified. Class information (KI1, ..., KIn; KI1', ..., KIn') characterising or identifying the acoustic environment situation is generated based on the classification of said features in at least one processing stage (S1, ..., Sn). The invention also relates to applications of the inventive method in hearing aids and to a hearing aid.

Description

Method for determining an acoustic environmental situation, application of the method and a hearing aid

The present invention relates to a method for determining an acoustic environmental situation, an application of the method, a device for determining the acoustic environmental situation and a hearing aid.

Today, modern hearing aids can be adapted to different acoustic environments with the aid of different hearing programs. The hearing aid is intended to offer the user optimum benefit in every situation.

The hearing program can be selected either using the remote control or using a switch on the hearing aid itself. Switching between different hearing programs is, however, troublesome, if not impossible, for many users. It is not always easy for experienced hearing aid users to determine which program offers the best comfort and the best speech intelligibility at which time. Automatic detection of the acoustic environment and the associated automatic switching of the hearing program in the hearing aid is therefore desirable.

BESTATIGUNGSKOPIE Various methods for the automatic classification of acoustic environmental situations are currently known. In all of these methods, the input signal, which can come from one or more microphones in the hearing aid, is different

Characteristics extracted. Based on these features, a pattern recognizer uses an algorithm to make a decision about the belonging of the analyzed input signal to a certain acoustic environment. The different known methods differ on the one hand by the different features that are used in the description of the acoustic environment (signal analysis), and on the other hand by the pattern recognizer used that classifies the features (signal identification).

From the publication of the international patent application with the publication reference WO 01/20 965 are a method and an apparatus for

Determination of the acoustic surrounding situation known. It is a one-step processing of an acoustic input signal in a feature extraction unit and a classification unit connected downstream thereof, in which the extracted features are classified to generate class information. With the known teaching, good results are obtained, particularly when using auditory-based features. An improvement is particularly in the

Areas of application for hearing aids wanted, because straight In this area of application, the classification of acoustic environmental situations must be extremely accurate. At the same time, the existence of several, very general classes of noise, such as music or noise, presents some difficulties. It is in the nature of these noise classes that they can be very general and broad, ie they can occur in many different ways. The noise class noise, for example, contains a wide variety of sounds such as background conversations, train station noise, hair dryers, and the noise class music includes pop music, classical music, individual instruments, vocals, etc.

However, due to the very general nature of these noise classes, it is very difficult to get a good detection rate using the known ones

Obtain processing method in a feature extraction unit and a downstream classification unit. Although the robustness of the recognition system can be improved by a suitable choice of features, as was first shown in WO 01/20965 by using auditory-based features, it is difficult, in particular because of a high variance between general noise classes, to make these clear and to distinguish them from one another without a doubt.

The present invention is therefore based on the object of specifying a method for determining an acoustic ambient situation which is more robust and more precise than the known methods. This object is achieved by the measures specified in claim 1. Advantageous embodiments of the invention, an application of the method, a device and a hearing aid are specified in further claims.

By an acoustic input signal in a multi-stage process, which consists of at least two classification levels, each level preferably consisting of one

Extraction phase and an identification phase is processed, an extremely robust and accurate classification of the current acoustic environment is obtained. For example, the method according to the invention can successfully avoid incorrect classification of pop music into the "speech in noise" category. The method according to the invention also enables a general noise class, for example noise, to be subdivided into subclasses, such as traffic noise or background noise for conversations. Special situations, such as those that occur in the interior of an automobile ("in-the-car noise"), can also be recognized. In general, room properties can be identified and taken into account accordingly in the further processing of important signal components. It has been shown that the method according to the invention also makes it possible to localize the sources of noise, which has made it possible to determine the presence of a detect specific noise source in several other noise sources.

The invention is explained in more detail below with reference to drawings, for example. Show:

Fig. 1 shows a known single-stage device for

Determination of an acoustic environment,

2 shows a first embodiment of a device according to the invention with two processing stages,

3 shows a second, general embodiment of a multi-stage device according to the invention,

4 shows a third, general embodiment of a multi-stage device according to the invention,

5 shows a fourth, general embodiment of a multi-stage device according to the invention,

6 shows a simplified embodiment compared to the two-stage embodiment according to FIG. 2 and

7 shows a hearing aid with a multi-stage device according to the invention according to FIGS. 2 to 6. 1 shows a known single-stage device for determining the acoustic environmental situation, the device consisting of a series connection of a feature extraction unit F, a classification unit C and a post-processing unit P.

An acoustic input signal IN, which was recorded with a microphone, for example, is the

Feature extraction unit F is applied, in which characteristic features are extracted.

For the feature extraction in audio signals, an analysis of the was proposed in the article by JM Kates entitled "Classification of Background Noises for Hearing-Aid Applications" (1995, Journal of the Acoustical Society of America 97 (1), pages 461 to 469) fluctuations in level and spectrum. Furthermore, in the European patent specification number EP-B1-0 732 036 an analysis of the

Amplitude histogram proposed to achieve the same goal. Finally, the feature extraction was also examined and applied by analyzing different modulation frequencies.

In this regard, the two articles by Ostendorf et. al. with the titles "Empirical Classification of Different Acoustic Signals and Speech Using a Modulation Frequency Analysis" (1997, DAGA 97, pages 608 to 609) and "Classification of Acoustic Signals based on the analysis of modulation spectra for use in digital hearing aids "(1998, DAGA 98, pages 402 to 403). A similar approach is also found in an article by Edwards et al. entitled" Signal processing algorithms for a new software-based, digital hearing device "(1998, The Hearing Journal 51, pages 44 to 52). Further possible features are the level itself or the zero crossing rate, as described, for example, in HL Hirsch," Statistical Signal Characterization "(Artech House 1992) The features previously used for audio signal analysis are therefore purely technical.

Furthermore, in the already cited publication of international patent application WO 01/20965, in addition to the technical features mentioned, the advantageous use of auditory-based features was mentioned for the first time.

1, the features M extracted in the feature extraction unit F are applied to the classification unit C, which is basically one of the known ones for the noise classification

Pattern identification methods are used. So-called distance estimators, Bayesian classifiers, fuzzy logic systems or neurons are particularly suitable

Networks as pattern recognizers. Further information on the first two methods can be found in the publication "Pattern Classification and Scene Analysis" by Richard 0. Duda and Peter E. Hart (John Wiley & Sons, 1973). Regarding neural networks, the Standard work by Christopher M. Bishop entitled "Neural Networks for Pattern Recognition" (1995, Oxford University Press). Furthermore, reference is made to the following publications: Ostendorf et. al. , "Classification of acoustic signals based on the analysis of modulation spectra for use in digital hearing aids" (Zeitschrift für Audiologie, 1998, pages 148 to 150); F. Feldbusch, "Noise Detection Using Neural Networks" (1998, Journal of Audiology, pages 30 to 36); European patent application with the

Publication number EP-A1-0 814 636; and US patent with the publication number US-5 604 812. In addition to the explained pattern recognition methods, in which only the static properties of the noise classes of interest are modeled, the already cited publication of international patent application WO 01/20965 also mentions methods in which dynamic Properties are taken into account.

1 by the in the

Classification unit C performed

Processing steps receive class information KI, which may be fed to a post-processing unit P for any cleanup of the class membership. Subsequently, adjusted class information KI ^{1 is obtained} .

2 shows a first embodiment variant of a device according to the invention. It is a device with two process stages S1 and S2, with each feature level S1, S2 each containing a feature extraction unit F1 or F2 and a classification unit C1 or C2. The original input signal IN is supplied to both process stages S1 and S2, namely both the feature extraction unit F1 and the feature extraction unit F2, which are each operatively connected to the corresponding classification unit C1 or C2 in the sequence. It is now important that class information KI1, which is obtained on the basis of calculations in the classification unit C1 of the first method step S1, the

Classification unit C2 of the second method stage S2 is influenced, in such a way that, for example, one of several possible pattern identification methods is selected and used for the noise classification in the classification unit C2 of the second method stage S2.

The general embodiment variant of the invention shown in FIG. 2 is further explained below using a concrete example:

The characteristic extraction unit F1 provides the characteristics tonality, spectral center of gravity (CGAV: spectral center of gravity) and fluctuation of the spectral

Center of gravity (CGFS) and spectral latitude and

Settling time extracted and in the

Classification unit Cl, in which an HMM (Hidden Markov Model) classifier is used, classified, wherein the input signal IN is divided into one of the following classes with the aid of the HMM classification: "speech", "speech in noise", "noise" or "music". This is called class information KI. The result of the first processing stage S1 is applied to the classification unit C2 of the processing stage S2, in which a second feature set is extracted using the feature extraction unit F2. In addition to the characteristics of tonality, spectral center of gravity and fluctuation of the spectral center of gravity (CGFS), the additional characteristic variance of the harmonic structure (pitch) - also referred to below as Pitchvar - is extracted. On the basis of these features, the result of the first processing stage S1 is checked and, if necessary, corrected using a rule-based classifier in the classification unit C2. The rule-based classifier contains only a few simple heuristic decisions, which are based on the four characteristics and are based on the following considerations:

The tonality characteristic is used for correction in each class if the characteristic values lie completely outside an admissible value range of the class information KII, which is in the first classification unit C1 - i.e. by the HMM classifier. As expected, the tonality for "music" is high, for "speech" in the middle range, for "speech in noise" a little lower and for "noise" deep. For example, if an input signal IN by the

Classification unit Cl falls into the "language" class, it is then expected that corresponding features, which have been determined in the feature extraction unit F1, have indicated to the classification unit Cl that the relevant signal component fluctuates in the input signal IN scark. If, on the other hand, the tonality for this input signal IN is very low, it is most likely not "speech" but "speech in noise". Similar considerations can be made for the other three characteristics, namely the variance of the harmonic structure

(Pitchvar), the spectral shear point (CGAV) and for the fluctuation of the spectral center of gravity (CGFS). Accordingly, the rules for the rule-based classifier, which are used in the classification unit C2, can be formulated as follows:

.2 -

In this embodiment of the invention, it has surprisingly been found that almost the same features are used in the second processing stage S2 as in the first processing stage S1. It has also been shown that the tonality characteristic is best suited to correcting a correction of the errors generated by the classification unit C1. It can therefore be said that when using the rule-based classifier, the tonality is of crucial importance.

When testing the embodiment described above, it was found that even with this simple two-step process Hit rate could be improved by at least 3% compared to the one-step process. In some cases, the hit rate improved by 91%.

A further embodiment is shown in a general representation in FIG. 3. It is a processing method with n stages. In continuation of the above considerations, each of the processing stages Sl to Sn has a feature extraction unit Fl to Fn and a classification unit Cl to Cn connected downstream thereof for generating the respective class information KIl to KIn. If necessary, Sl to Sn is in each or in individual processing stages

Post-processing unit Pl to Pn for generating cleaned class information KIl 'to KIn ¹ available.

In a continuation of the embodiment variant according to FIG. 2, the embodiment variant shown in FIG. 3 is particularly suitable for a so-called coarse-fine classification. In a rough-fine classification, a result obtained in processing stage i is refined in a subsequent processing stage i + 1. A rough classification is thus carried out in a higher processing level, with a rough classification based on more specific feature extractions and / or classification methods being carried out in a lower processing level on the basis of the rough classification. This process can can also be viewed as generation of hypotheses in a higher process level, which are checked, ie confirmed or rejected in a lower process step. It is expressly pointed out that the hypotheses that have been created in a higher level of procedure (rough classification) can also be entered with other information, in particular with manual means such as remote controls or switches. In Fig. 3, this is indicated, representative of the first processing stage S1, by means of a manipulated variable ST, by means of which, for example, the calculations in the classification unit C1 can be overridden. Of course, the manipulated variable can also be a classification unit C2 to Cn or one

Postprocessing units Pl to Pn are fed to another process stage S1 to Sn.

In a classification system according to the invention with several processing stages S1 to Sn everyone can

Processing level Sl to Sn, which, however, is not mandatory, can be assigned to a task, such as, for example: a rough classification, a fine classification, a localization of a noise source, a check whether a specific noise source, e.g. B. automotive noise in a vehicle, or an extraction of certain signal components from an input signal, for. B. Elimination of echo taking into account spatial characteristics. The individual process steps S1 to Sn are therefore individual in the sense that in them different characteristics are extracted and different classification methods are used.

In a further application example, an individual one is provided in a first processing stage S1

Localize the signal in a mixture of different signal components, carry out a rough classification of the localized signal source in a second processing stage S2 and carry out a fine classification of the rough classification obtained in the second processing stage S2 in a third processing stage S3.

The localization of the sound source carried out in the first processing stage can be followed by directional filtering, for example using multi-microphone technology.

Of course, a feature extraction unit Fl to Fn can be divided among several classification units Cl to Cn, i.e. the results of one

Feature extraction units F1 to Fn can be used by several classification units C1 to Cn. It is also conceivable that a classification unit C1 to Cn is used in several processing stages S1 to Sn. Finally, it can be provided that the class information KIl to KIn or the cleaned class information KIl 'to KIn' obtained in the different processing stages S1 to Sn are weighted differently in order to maintain the final classification. 4 shows a further embodiment of the invention, in which a plurality of processing stages S1 to Sn are used. In contrast to the embodiment according to FIG. 3, the

Class information KIl to KIn not only used in the immediately following processing stages, but possibly in all subordinate processing stages. In an analogous manner, the results from previous processing stages S1 to Sn can also have their effects on the subsequent feature extraction units F1 to Fn or on the features to be extracted.

4, postprocessing units Pl to Pn are also conceivable, prepare the intermediate results of the classification and make them available as cleaned class information KIl ¹ to KIn '.

5 shows a further embodiment variant of a multi-stage device for determining the acoustic ambient situation, again in a general form. As already shown for the embodiment variants according to FIGS. 3 and 4, several processing stages S1 to Sn with feature extraction units Fl to Fn and classification units Cl to Cn are shown. The class information KIl to KIn obtained in each processing step Sl to Sn becomes one Decision unit FD supplied, in which the final classification is carried out by generating the class information KI. In the decision unit FD it is optionally provided to generate feedback signals which are based on the

Feature extraction units F1 to Fn and / or act on the classification units Cl to Cn, for example in order to adapt individual parameters in these processing units or to be able to exchange entire classification units Cl to Cn.

It is expressly pointed out that the repercussions and links between the processing units of the embodiment variants according to FIGS. 3 to 5 are not limited to the variants shown. It is conceivable that individual repercussions or links are omitted. In principle, any combination of processing units and structures is possible.

In addition, it is conceivable that - when using the invention with hearing aids - the different processing stages between two hearing aids, i.e. with one hearing aid each for the left and right ear. In this embodiment, the information is exchanged via a wired or a wireless transmission link.

6 again shows a simplified embodiment variant of the invention to illustrate the above general explanations of the possible ones Structures. Although only one

If feature extraction unit F1 is provided, two processing stages S1 and S2 are provided. The first processing stage S1 consists of the feature extraction unit F1 and the

Classification unit Cl. In the second processing stage S2, the same features are used that have already been used in the first method stage S1. A recalculation of the features in process stage S2 is therefore unnecessary, and the results of the feature extraction unit F1 of the first process stage S1 can be used in the second process stage S2. In the second method stage S2, only the classification method is changed, depending on the class information KII of the first processing stage S1.

FIG. 7 shows the use of the device according to the invention in a hearing aid device which essentially has a transmission unit 200. 100 denotes a multi-stage processing unit which is implemented according to one of the embodiment variants shown in FIGS. 2 to 6. The input signal IN is applied to both the multi-stage processing unit 100 and the transmission unit 200, in which the acoustic input signal IN is processed with the aid of the class information KI1 to KIn or KIl 'to KIn' generated in the multi-stage processing unit 100. It is preferably provided to select a suitable hearing program based on the acoustic environment situation determined, as is the case in FIG Introduction and has been described in the international patent application WO 01/20 965.

300 denotes a manual input unit, with the aid of which - for example via a radio link, as can be seen schematically in FIG. 7 - may act both on the multi-stage processing unit 100 in the manner already explained or on the transmission unit 200. In the case of the hearing aid 200, reference is made to the explanations in WO 01/20 965, whose

Content is to be regarded as an integral part of this description.

In particular, one of the following methods is used as a possible classification method in all of the explained embodiment variants of the invention:

- Hidden Markov models;

- fuzzy logic; - Bayesian classification;

- rule-based classification;

- Neural networks;

- Minimum distance.

Finally, it is expressly pointed out that in the feature extraction units F1 to Fn (FIGS. 2 to 7) technical and / or auditory-based features can be extracted. For detailed explanations of the technical and audio-based features, reference is again made to the explanations in the international patent application WO 01/20 965.

The preferred application of the method according to the invention for determining the acoustic ambient situation is the selection of a hearing program in a hearing aid device. However, it is also conceivable to use the device according to the invention or to use the method according to the invention for speech recognition.

Claims

Claims:

1. A method for determining an acoustic ambient situation, the method consisting in processing an acoustic input signal (IN), preferably recorded with the aid of at least one microphone, in at least two processing stages (S1, ..., Sn), in such a way that that in at least one of the at least two processing stages (Sl, ..., Sn) an extraction phase is provided in which characteristic features are extracted from the input signal (IN), and - that in each processing stage (Sl, ..., Sn) an identification phase is provided in which extracted characteristic features are classified, the classification of the features in at least one processing stage (Sl, ..., Sn)

Class information (KIl, ..., KIn; KIl ', ..., KIn') are generated which characterize or identify the acoustic environment.

2. The method according to claim 1, characterized in that an extraction phase is provided in each processing stage (Sl, ..., Sn), in which characteristic features are extracted from the input signal (IN).

3. The method according to claim 1 or 2, characterized in that on the basis of class information received in a processing stage (Sl, ..., Sn) (KIl, ..., KIn; KIl ', ..., KIn'), Art processing in a different processing level (Sl, ..., Sn) is selected.

4. The method according to claim 2 or 3, characterized in that received in the identification phase of a processing stage i (Sl, ..., Sn)

Class information (KIl, ..., KIn; KIl ', ..., KIn') determine the type of processing in the subsequent or subordinate processing level i + 1 (Sl, ..., Sn).

5. The method according to claim 4, characterized in that on the basis of the class information obtained in processing stage i (Sl, ..., Sn) (KIl, ..., KIn; KIl ', ..., KIn'), specific features in the extraction phase of the subsequent or subordinate processing stage i + 1 (Sl, ..., Sn) and / or specific

Classification procedures in the identification phase of the subsequent or subordinate processing level i + 1 (Sl, ..., Sn) can be selected.

6. The method according to any one of the preceding claims, characterized in that one of the following classification methods is used in the identification phase: - Hidden Markov models;

- fuzzy logic;

- Bayesian classification;

- rule-based classification;

- Neural networks;

- Minimum distance.

7. The method according to any one of the preceding claims, characterized in that technical and / or auditory-based features are extracted in the extraction phases.

8. The method according to any one of the preceding claims, characterized in that in at least one processing stage (Sl, ..., Sn) following the

Extraction phase, a post-processing phase is provided, in which the class information (KIl, ..., KIn) is reworked to generate cleaned class information (KIl ', ... KIn').

9. Application of the method according to one of claims 1 to 8 for adapting at least one hearing aid to a current acoustic ambient situation.

10. Application according to claim 9, characterized in that a hearing program or a due to the determined current acoustic environmental situation Transfer function between at least one microphone and a listener is set in the at least one hearing aid.

11. Application of the method according to one of claims 1 to 8 for speech recognition.

12. Device for determining an acoustic environmental situation with a feature extraction unit (M), which with a classification unit (C)

Processing of an input signal (IN) is operatively connected, characterized in that at least two processing stages (Sl, ..., Sn) are provided, one in at least one of the at least two processing stages (Sl, ..., Sn)

Feature extraction unit (Fl, ..., Fn) is included and in each processing stage (Sl, ..., Sn) a classification unit (Cl, ..., Cn) is included that the input signal (IN) to the feature extraction units (Ml , ..., Mn) and that with the

Classification units (Cl, ..., Cn) class information (KIl, ..., KIn) can be generated.

13. The apparatus according to claim 12, characterized in that in each processing stage (Sl, ..., Sn)

Feature extraction unit (Ml, ..., Mn) is provided.

14. The apparatus according to claim 12 or 13, characterized in that the class information (KIl, ..., KIn; KIl ', ..., KIn') of a different processing level (Sl, ..., Sn).

15. Device according to one of claims 12 to 14, characterized in that the class information

(KIl, ..., KIn; KIl ', ..., KIn ¹ ) of a processing stage i (Sl, ..., Sn) of the subsequent or higher processing stage i + 1 (Sl, ..., Sn) are.

16. The apparatus according to claim 15, characterized in that the class information (KIl, ..., KIn; KIl ¹ , ..., KIn ') of a processing stage i (Sl, ..., Sn) of the feature extraction unit (Ml,. .., Mn) of a subsequent or subordinate processing level i + 1 (Sl, ..., Sn) and / or the classification unit (Cl, ..., Cn) of the subsequent or subordinate processing level i + 1 (Sl,. .., Sn) are applied.

17. Device according to one of claims 12 to 16, characterized in that in at least one

Processing stage (Sl, ..., Sn) applies the received class information (KIl, ..., KIn) to a post-processing unit (Pl, ..., Pn) for generating cleaned class information (KIl ¹ , ..., KIn ') are.

18. The apparatus according to claim 12 or 13, characterized in that the class information (KIl, ..., KIn) of all processing stages (Sl, ..., Sn) of a decision unit (ED).

19. The device according to claim 18, characterized in that the decision unit (ED) is operatively connected to at least one feature extraction unit (Fl, ..., Fn) and / or to at least one classification unit (Cl, ..., Cn).

20. Hearing aid with a transmission unit (200) that is operatively connected on the input side at least to a microphone and on the output side to a transducer unit, in particular a receiver, and with a device according to one of claims 12 to 17 for generating class information (KIl, ... , KIn; KIl ', ..., KIn'), the class information (KIl, ..., KIn; KIl ', ..., KIn') of the transmission unit (200) being applied.

21. Hearing aid according to claim 20, characterized in that an input unit (300) is provided which is operatively connected, preferably via a radio link, to the transmission unit (200) and / or to the device according to one of claims 12 to 17.