KR20180073167A - Method of dangerous sound detection for hearing impaired persons - Google Patents

Abstract

According to the present invention, a dangerous situation recognizing system comprises: a database storing sound data that occurs during a dangerous situation to guide dangerous situations of hearing impaired persons who can not recognize a sound or can not distinguish a sound well; a sound collection part collecting a surrounding sound; a feature extraction part extracting a feature of the sound collected by the sound collection part; a danger sound detection engine detecting a danger sound from the feature input from the feature extraction part using an artificial neural network pattern recognition algorithm; and an alarm part generating a notification signal instead of a sound depending on a detection result of the danger sound detection engine.

Description

TECHNICAL FIELD The present invention relates to a method for detecting a dangerous sound or a specific sound for a hearing-impaired person,

The present invention relates to sound recognition, and more particularly, to a method and system for recognizing and recognizing various sounds in daily life.

Due to the aging society, the elderly population is increasing and the number of hearing impaired people is increasing due to excessive use of headphones, and the number of hearing-impaired people is also increasing to about 3% of the normal population. Hearing-impaired people who are completely unable to hear or can not distinguish their sounds are hard to hear and understand the situation, so they have many difficulties in everyday life, and use sound information to detect dangerous situations in indoor and outdoor environments Immediate action is not possible. Therefore, when a person with hearing impairment is in danger, it is necessary to convert the dangerous situation into visual and tactile information through sensors attached to the body or handheld devices.

However, it is necessary to have an empirical and concrete daily life sound recognition method and system which can accurately identify the dangerous situation by being able to grasp various sounds of daily life with high reliability.

In order to solve the above-described problems, it is an object of the present invention to provide a method and system for accurately grasping a sound that can be blown in a dangerous situation in indoor and outdoor.

It is also an object of the present invention to provide a dangerous situation countermeasure method and system that catches a situation from a detected sound and warns it in a manner other than auditory sense such as visual or tactile sense.

The present invention relates to a sound processing apparatus, including a database for storing sound data generated in a dangerous situation, a sound collecting unit for collecting sound around the sound collecting unit, a feature extracting unit for extracting characteristics of sounds collected by the sound collecting unit, And a dangerous sound detection engine for detecting a dangerous sound by using an artificial neural network pattern recognition algorithm from characteristics input from the input unit.

And a warning unit for generating a sound replacement notification signal according to the detection result of the dangerous sound detection engine.

The database includes an indoor and outdoor risk database, a congestion noise database, and a simulation DB that is a mixture of the indoor and outdoor risk database and the congestion noise database, and the sound collector collects the collected sounds.

The risk context extractor extracts features of the sound collected using the LPCC feature vector, and extracts features of the sound using the MFCC feature vector according to the size and type of the sound, thereby enabling optimum sound recognition in each situation.

The above-mentioned database may include a contrast judgment unit for discriminating a dangerous situation sound by comparing the data transmitted from the sound collection unit and the stored data.

According to another aspect of the present invention, there is provided a sound processing method comprising the steps of: collecting sound of a surrounding situation; extracting characteristics of the collected sounds; classifying sounds using the artificial neural network algorithm; And a step of determining whether or not a risk situation is determined on the basis of the determination result.

And extracting the feature of the sound, the step of discriminating the size and the pattern of the sound, and the step of selecting the feature vector according to the discrimination result.

According to the present invention, it is possible to accurately detect a sound that may occur in a dangerous situation, determine whether or not a dangerous situation exists, and transmit sound substitute information to a hearing-impaired person so that the user can escape from a dangerous situation.

1 is a structural diagram of a dangerous sound detection and notification system.
FIGS. 2 to 4 are diagrams showing the results of experimenting recognition rates for each dangerous situation by differentiating feature vectors. FIG.

A study on the risk detection technology for the hearing impaired people has been carried out using the Mel-Frequency Cepstral Coefficient (MFCC) feature vector-based GMM (Gaussian Mixture Model) pattern recognition algorithm to detect motorcycle, car horn, thunder, (HMCC) based on the Mel-Frequency Cepstral Coefficient (MFCC), and the study of differences in the spectral characteristics of the sound.

In the present invention, a risk detection engine for a wearable device for the hearing impaired is designed using an artificial neural network (ANN).

1 is a structural diagram of a dangerous situation recognition and alarm system according to the present invention.

As shown in the figure, a database (a dangerous situation sound DB) storing sounds generated in a dangerous situation, a database (ambient noise DB) storing everyday sounds in everyday life, a characteristic of each sound from the sounds stored in the database A second feature extracting unit (feature extracting unit) for extracting features from the synthesized sound stored in the simulation DB, a first feature extracting unit (feature extracting unit) (Not shown) for generating a sound substitution notification signal (visual signal, tactile signal, etc.) according to the detected result and a dangerous sound detection engine for detecting a dangerous sound from the features input from the feature extraction unit and the second feature extraction unit, .

The Dangerous Situation DB is a database of sounds that occur during a dangerous situation that may occur in indoor or outdoor environments, and the environmental noise database is a database of sounds in general indoor and outdoor situations such as a quiet room, a busy street, an alleyway, .

Basically, the database serves as a population for extracting minutia points, and the dangerous situation recognition system according to the present invention can be continuously used to collect sounds of each case while being operated.

The first feature extraction unit extracts the feature of each sound from the sound stored in the dangerous situation sound DB and the environmental noise database, and extracts the feature of the sound using the feature vectors such as LPB, LPCC, and MFCC.

The simulation DB stores the result of synthesizing the dangerous sound and the environmental noise in order to learn the dangerous sound detection engine and stores the sound data in which the dangerous sound occurs under the environmental noise in daily life. The second feature extraction unit extracts features from the synthesized speech stored in the simulation DB.

The dangerous sound detection engine proceeds to pattern classification using the ANN classifier, and finally determines whether or not a dangerous situation exists in a decision block according to the result.

The alarm unit informs the user of the dangerous situation by a tactile signal such as a visual signal or vibration according to the judgment result of the dangerous sound detection engine.

In addition, a sound input unit (not shown) collects surrounding sounds and transmits them to the first and / or second feature extracting unit, so that the dangerous sound detecting engine judges actual situations.

On the other hand, the output of the sound input unit is transmitted to at least one of the dangerous situation sound database, the environmental noise database, and the simulation DB to update each DB, or to compare with the stored data stored in the DB, (Not shown) included in the DB may be used to determine whether or not the sound is a dangerous sound.

That is, in the present invention, the dangerous sound detection engine determines the dangerous situation through the feature extraction unit. Alternatively, or in place of the dangerous sound detection engine, the contrast determination unit may determine the dangerous situation.

In the above description, the outputs of the first and second feature extraction units are input to the dangerous sound detection engine. However, only the output of the first feature extraction unit or the output of the second feature extraction unit can be input to the dangerous sound detection engine Of course.

Hereinafter, a process of grasping a dangerous situation from surrounding sounds according to the present invention will be described in detail.

The dangerous situation sound database stores representative and most frequent dangerous situation sounds. Specifically, it divides the dangerous situation into an indoor risk situation and an outdoor dangerous situation. In the indoor risk situation, it stores the boiling sound, the fire alarm, the doorbell and the telephone sound. In the outdoor risk situation, Stores star data.

The environmental noise database defines environmental noise as 4 kinds of quiet one way, alley road, highway, and roadway, and stores about 5 minutes of data.

The first and second feature extraction units compares and analyzes the accuracy of detecting a dangerous situation using LPC, LPCC, and MFCC, which are feature vectors most commonly used in the field of signal processing.

- Environmental noise DB

The indoor environment and the outdoor environment are different from each other because the environment in which the risk of hearing-impaired persons may occur is the indoor environment and the outdoor environment and the noise that can be generated in the two environments is different. Therefore, The DB is defined as a quiet room for the indoor environment. The outdoor environment is defined as three kinds of alleys, highways, and roads in consideration of the locations where dangerous situations may occur, and these sounds are stored.

All four environmental noises are recorded for 5 minutes each and sampled at 16kHz.

- Risk situation sound DB

There are different sounds of dangerous situations that can occur depending on the surroundings of hearing-impaired people. In the indoor environment, there are boiling water of kettle, fire alarm, telephone ringtone, doorbell, etc. There are car horn sound and emergency siren sound in outdoor environment. Therefore, the dangerous situation sound DB according to the present invention defines four kinds of dangerous sounds generated in the indoor environment as boiling water, fire alarm, telephone ringing sound, and doorbell. In the outdoor environment, , And each dangerous situation sound data is recorded for 30 seconds to 60 seconds on a mono channel, and is sampled and stored at 16 kHz.

- feature vector

The first and second feature extracting units will be described in detail.

The Mel-Frequent Cepstrum Coefficient (MFCC) is a value obtained by taking DCT (Discrete Cosine Transform) in the Mel-scale frequency domain in which the power spectrum calculated for the voice signal in the frame is simulated with the frequency response of the hearing instrument.

LPC is a method of predicting a current signal [n] in a past signal by linear combination, and can be represented in the form of a differential equation as shown in Equation (1) using an all-pole model.

은 예측신호, a _i 는 선형예측계수이며, p는 예측계수의 차수이다. Where Sn is the input signal,

A _i is a linear prediction coefficient, and p is a degree of a prediction coefficient.

 The prediction error of the current signal and the predicted signal is expressed by Equation (2).

Equation 3 is the mean square error (MSE) J for the prediction signal.

To find a linear prediction coefficient that minimizes the error of the prediction signal, the equation (3) can be partially differentiated with respect to a _i to obtain p linear simultaneous equations (4).

(4) can be expressed by Equation (5) because E [ s ( ni ) s ( nj )] in Equation 4 is an autocorrelation function of the input signal, and the linear prediction coefficient can be expressed by Equation Can be obtained by using an inverse matrix. The Levinson-Durvin algorithm, which is generally well known, is used to solve the time complexity problem of finding the inverse of the autocorrelation function.

? ? ? ?

LPCC is defined as an inverse z-transform of C (z) and is expressed by Equation (6).

??? ??

If the all-pore z = z _i is in the unit cycle and the gain value is 1, the LPCC (c _ip (n)) is defined by Equation (7).

LPCC is obtained from linear prediction coefficients by recursive. The recursive procedure is shown in Equation (8).

? ? ?

- Separation degree comparison

The Bhattacharyya distance measurement was used to check the performance of the three feature vectors of MFCC, LPC and LPCC used in the present invention.

Bhattacharyya distance measurement is the method of calculating the distance by measuring the error rate, which is the best evaluation criterion when the distribution of each class has Gaussian form. Bhattacharyya The feature vector with the greatest distance is the most distant from the class, so it is most suitable for detecting the dangerous situation. A total of 6 hazardous situations sound data are stored in a window size of 150ms

Ging vectors were extracted and a total of 720 vectors were extracted for each 120 points. The Bhattacharyya distances were obtained by comparing the extracted feature vectors.

After extracting LPC feature vectors, LPCC feature vectors, and MFCC feature vectors from six hazardous sounds, the average separation of LPC feature vectors was 2.79, and the average of MFCC feature vectors The separation was 3.77. The average separation of the highest LPCC feature vectors was 3.79. This means that when the LPCC feature vector is used, the LPCC feature vector is a feature vector that can accurately reflect the characteristics of each dangerous situation statistically, meaning that the distance between the sounds of the dangerous situations is the longest on average. The LPCC feature vector is adopted as the main feature vector.

- Risk awareness

The dangerous sound detection engine uses Artificial Neural Network (ANN) pattern recognition algorithm to recognize the dangerous situation using extracted feature vectors. It is a pattern recognition algorithm designed to adjust the weight value, which means the connection strength between each perceptron through ANN training, and to train the nonlinear function of input.

Since the ANN neural network used in the present invention does not complicate the structure in consideration of the computing environment of the wearable device and the nature of the signal is encoded when extracting the feature vector, the characteristics of the signal are encoded between the input layer and the hidden layer There is no preprocessing that can be done. This can reduce large amounts of parameters in the neural network.

The neural network structure of the present invention is composed of an input layer, a hidden layer and an output layer. In consideration of the embedded environment, the hidden layer is not complicated as one layer. The number of input neurons in the input layer is determined by the degree of the extracted feature vector We set it to ten together. The number of neurons in the hidden layer was determined experimentally by repeating several times. One of the six neurons has a bias value.

The output layer consists of six neurons, each of which outputs a class score for each of the six hazardous situations. The coefficients of the whole neurons in the neural network update the coefficients minimizing the error using error back propagation according to the error of the output layer output during the learning process.

Finally, the active function used in the ANN neural network is a bipolar sigmoid with α = 1, and the learning rate of the coefficient is set to p 0.6.

- Optimum configuration

Experiments were carried out to obtain the optimal configuration for the recognition of the dangerous situation by using the above-described configuration and setting the sound volume to 5 db, 10 db, 15 db, and 20 db in each case of the indoor environment and the outdoor environment (siren and car horn) The results are shown in Figs.

3, the LPCC feature vectors were generally used. However, in the outdoor environment, the recognition rate of the MFCC was large in the siren and the vehicle horn, and the MFCC was 55.07% But the recognition rate of 20dB or more showed the highest 100% accuracy.

On the other hand, considering the complexity of each feature vector extraction method, the MFCC has a complexity equivalent to the square of LPC and LPCC, and therefore, LPC and LPCC feature vectors are more suitable in an embedded environment such as wearable device.

Therefore, in the present invention, the LPCC is used as a basic feature vector in constructing the first and second feature detectors, and when the MFCC is high in performance, the MFCC feature vector And extracts a feature using the extracted feature.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. It is needless to say that the objects and effects of the present invention also include those other than those described herein within a reasonable range in consideration of the technical sense. For example, changes in the activation function and the learning rate of the ANN are included in the technical scope of the present invention.

Accordingly, the scope of the present invention should be determined by the description of the following claims.

Claims (7)

A database for storing sound data generated in a dangerous situation,
A sound collecting unit for collecting sounds of the surroundings,
A feature extraction unit for extracting features of sounds collected by the sound collection unit;
A danger sound detection engine for detecting a danger sound using an artificial neural network pattern recognition algorithm from the features input from the feature extraction unit;
Wherein the risk identification system comprises:
The method according to claim 1,
And a warning unit for generating a sound replacement notification signal according to the detection result of the dangerous sound detection engine.
The system according to claim 1,
An indoor and outdoor risk DB, a congestion noise DB, and a simulation DB in which the indoor and outdoor risk DB and the congestion noise DB are mixed,
Wherein the sound collector collects the collected sounds.
The risk assessment system according to claim 1,
And extracts characteristics of the sound collected using the LPCC feature vector, and extracts features of the sound using the MFCC feature vector according to the size and type of the sound.
4. The system according to claim 3,
And a contrast determination unit for determining a sound of a dangerous situation by comparing the data transmitted from the sound collecting unit and the stored data.
Collecting sound of a surrounding situation,
Extracting features of the collected sounds;
Classifying sounds using the artificial neural network algorithm,
Determining whether or not a risk situation exists based on a result of the classifying step
The method comprising the steps of:
7. The method of claim 6, wherein extracting the feature of the sound comprises:
Determining a size and a pattern of the sound;
And selecting a feature vector according to the determination result
Risk identification method.

Images