CN113314143B - Method and device for judging apnea and electronic equipment - Google Patents
Method and device for judging apnea and electronic equipment Download PDFInfo
- Publication number
- CN113314143B CN113314143B CN202110629654.0A CN202110629654A CN113314143B CN 113314143 B CN113314143 B CN 113314143B CN 202110629654 A CN202110629654 A CN 202110629654A CN 113314143 B CN113314143 B CN 113314143B
- Authority
- CN
- China
- Prior art keywords
- snore
- loudness
- frame
- suspected
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 208000008784 apnea Diseases 0.000 title claims abstract description 111
- 238000000034 method Methods 0.000 title claims abstract description 59
- 206010041235 Snoring Diseases 0.000 claims abstract description 250
- 239000012634 fragment Substances 0.000 claims abstract description 129
- 238000001228 spectrum Methods 0.000 claims abstract description 72
- 230000005236 sound signal Effects 0.000 claims abstract description 41
- 238000013528 artificial neural network Methods 0.000 claims abstract description 27
- 230000003595 spectral effect Effects 0.000 claims description 25
- 230000029058 respiratory gaseous exchange Effects 0.000 claims description 23
- 230000009466 transformation Effects 0.000 claims description 6
- 238000012544 monitoring process Methods 0.000 abstract description 9
- 230000015654 memory Effects 0.000 description 36
- 230000008569 process Effects 0.000 description 11
- 238000004590 computer program Methods 0.000 description 10
- 230000009471 action Effects 0.000 description 9
- 238000001514 detection method Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 238000005070 sampling Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000036541 health Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000000241 respiratory effect Effects 0.000 description 3
- 230000033764 rhythmic process Effects 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 201000002859 sleep apnea Diseases 0.000 description 3
- 208000037656 Respiratory Sounds Diseases 0.000 description 2
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 239000008280 blood Substances 0.000 description 2
- 210000004369 blood Anatomy 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 229910052760 oxygen Inorganic materials 0.000 description 2
- 239000001301 oxygen Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 206010021079 Hypopnoea Diseases 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 208000035475 disorder Diseases 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000000135 prohibitive effect Effects 0.000 description 1
- 208000023504 respiratory system disease Diseases 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 208000019116 sleep disease Diseases 0.000 description 1
- 208000020685 sleep-wake disease Diseases 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/08—Detecting, measuring or recording devices for evaluating the respiratory organs
- A61B5/0826—Detecting or evaluating apnoea events
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
- G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
- G10L25/66—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for extracting parameters related to health condition
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Physiology (AREA)
- Pulmonology (AREA)
- Epidemiology (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Evolutionary Computation (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Veterinary Medicine (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The application discloses an apnea judging method, an apnea judging device and electronic equipment, wherein the method comprises the following steps: acquiring an audio signal; determining the loudness of background noise according to the loudness of each frame, and determining the sound fragment according to the loudness of each frame and the loudness of the background noise; based on the neural network snore prediction model, identifying a snore segment from the voiced segments; recognizing continuous snore fragments and snore fragment intervals, and extracting normal snore interval characteristics according to the continuous snore fragments and the snore fragment intervals; determining snore segment intervals greater than an apnea threshold as suspected apnea segments; and determining the effectiveness of the suspected apnea fragments according to the loudness of each frame of the suspected apnea fragments, the linear spectrum energy and a preset rule. By monitoring the snore interval, whether the user breathes smoothly or not is determined, and the method is strict in logic, small in calculated amount and high in identification accuracy.
Description
Technical Field
The application relates to the technical field of artificial intelligence, in particular to an apnea judging method, an apnea judging device and electronic equipment.
Background
The apnea syndrome is a sleep disorder seriously endangering health, the sleep state of a user can be influenced by wearing equipment through detecting nasal airflow in medicine, and the user faces a psychological threshold for monitoring, so that the popularity of screening is greatly influenced.
At present, respiration is monitored on the market based on a piezoelectric principle and electromagnetic waves/radar waves, and the problem is that under the condition of apnea, a user with non-neural apnea has chest movement, so that the accuracy of identification is affected. The smart watch finds out the apnea by monitoring blood oxygen, and the difficulty faced is that the accuracy of continuous blood oxygen monitoring is contradicted with the design of power consumption, so that a user can wear the smart watch at night and feel uncomfortable. Yet another significant problem faced by these smart devices is that the purchase cost of the hardware itself is prohibitive for most users, and the apnea syndrome primary screening market requires a convenient and inexpensive solution.
The primary screening of the apnea syndrome by adopting the sound is an efficient means, so that on one hand, a user has no psychological problem of wearing discomfort, and on the other hand, the popularization of the smart phone greatly reduces the worry of the user on the monitoring cost. But how to accurately identify snore and respiratory sounds is a key of the technology.
At present, a plurality of related products/patents focus on identification of frequency spectrum characteristics before and after the occurrence of apnea or hypopnea, but the problems of large calculated amount, low identification accuracy and the like generally exist.
Disclosure of Invention
The embodiment of the application provides an apnea judging method, an apnea judging device and electronic equipment, so as to solve or at least partially solve the problems.
According to a first aspect of the present application, there is provided a method for determining apnea, comprising:
acquiring an audio signal, and determining the loudness, linear spectrum energy and mel spectrum energy of each frame in the audio signal;
determining the loudness of background noise according to the loudness of each frame, and determining a sound fragment according to the loudness of each frame and the loudness of the background noise;
inputting a neural network snore prediction model according to the Mel frequency spectrum energy of each frame, and identifying a snore segment from the voiced segments;
identifying continuous snore fragments and snore fragment intervals, and extracting normal snore interval characteristics according to the continuous snore fragments and the snore fragment intervals, wherein the normal snore interval characteristics comprise average snore intervals, minimum snore intervals and maximum snore intervals;
determining an apnea threshold according to a definition of medical apnea; determining snore segment intervals greater than an apnea threshold as suspected apnea segments;
And determining the effectiveness of the suspected apnea fragments according to the loudness of each frame of the suspected apnea fragments, the linear spectrum energy and a preset rule.
According to a second aspect of the present application, there is provided a device for determining apnea, comprising:
an acquisition unit for acquiring an audio signal, and determining the loudness, linear spectrum energy and mel spectrum energy of each frame in the audio signal;
the first identification unit is used for determining background noise loudness according to the loudness of each frame and determining a sound fragment according to the loudness of each frame and the background noise loudness;
the second identification unit is used for inputting a neural network snore prediction model according to the Mel frequency spectrum energy of each frame, and identifying a snore segment from the voiced segments;
the third recognition unit is used for recognizing the continuous snore fragments and the snore fragment intervals, and extracting normal snore interval characteristics according to the continuous snore fragments and the snore fragment intervals, wherein the normal snore interval characteristics comprise average snore intervals, minimum snore intervals and maximum snore intervals;
a fourth recognition unit for determining an apnea threshold according to a definition of medical apnea; determining snore segment intervals greater than an apnea threshold as suspected apnea segments;
And the judging unit is used for determining the effectiveness of the suspected apnea fragments according to the loudness of each frame of the suspected apnea fragments, the linear spectrum energy and a preset rule.
According to another aspect of the present application, there is provided an electronic device including: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the method of any of the above.
The above-mentioned at least one technical scheme that this application embodiment adopted can reach following beneficial effect:
firstly, determining the background noise loudness of the current environment according to the loudness of each frame composing an audio signal, marking the segment above the background noise loudness as a sound segment, further, identifying snore segments from the sound segments based on a neural network snore prediction model, identifying segments with snore segments exceeding a certain threshold value in the snore segments as suspected apnea segments, and judging whether breathing is stopped according to the loudness of each frame of a spacing segment, linear spectrum energy and a preset rule. According to the snore segment identification method based on the neural network snore prediction model, the snore segment is accurately identified, the snore interval is further identified in the snore segment, whether the user breathes smoothly is determined through monitoring of the snore interval, a solid foundation is provided for monitoring the health state of the user, and the snore segment identification method based on the neural network snore prediction model is strict in logic, small in calculated amount and high in identification accuracy.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
fig. 1 is a flow chart of a method for determining apnea according to an embodiment of the present application;
fig. 2 is a schematic structural view of an apnea judging device according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an electronic device in an embodiment of the present application.
Detailed Description
For the purposes, technical solutions and advantages of the present application, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The following describes in detail the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.
Fig. 1 shows a flow chart of a method for determining apnea according to an embodiment of the present application, and as can be seen from fig. 1, the method at least includes steps S110 to S160:
it should be noted that, the present application is mainly directed to judging the apnea occurring in the snoring process, because the breathing disorder of people is usually in the snoring process, and because the airway is blocked during snoring, the apnea is easy to occur.
Step S110: an audio signal is acquired and linear spectral energy, mel spectral energy and loudness of each frame in the audio signal are determined.
The audio signal may, but is not limited to, collecting the user's sound signal while sleeping through the intelligent terminal. For example, the audio signal of the microphone of the intelligent terminal is acquired at 16000Hz, the audio signal is (2,4,100,120,140,60, -60, -130, …) with the interval time of each point=1/16000 second, and the audio signal is acquired at 16000Hz, and the size of one acquisition point signal is represented by 16 bits and is mono.
The linear spectral energy, mel spectral energy, for each frame in the audio signal may be obtained according to the following steps: collecting sound pressure amplitude values of sampling points in an audio signal at a preset frequency; and carrying out frame division, frame shifting, fourier transformation and Mel frequency transformation on the audio signal, and determining a plurality of linear frequency points, corresponding Mel frequencies, corresponding linear frequency spectrum energy and corresponding Mel frequency spectrum energy which form each frame.
The sampling time sequence of the audio signal is taken out and continuous N data are taken as a group, the group of data is called a frame of data, for example, 512 continuous data are taken out each time, which is called a frame of data, the process is frame division, and the number of the data taken out each time can be set according to the calculated amount, and is usually 512 or 1024. In particular, in a frame of data, which frequency points can be obtained, this is related to the frequency point resolution, for example, taking the frequency point resolution as 16000, taking 512 pieces of data each time as an example, since 16000/512=31.25 Hz, that is, in the frequency domain of 0-8000Hz, only the information of the frequency points of 31.25×n can be obtained, and n=an integer of 1-256.
In this application, in order to improve the detection accuracy, in the process of framing, each value is not started from the tail of the previous frame data, but from the middle position of the previous frame data, in this embodiment, the second frame is started from the middle position of the first frame, that is, 257 frequency points, and the magnitudes corresponding to 512 frequency points are taken. For convenience of processing, a frame number is set for each frame of audio signal, sequentially increasing.
The sound pressure amplitude corresponding to each sampling point in a frame of audio signal is subjected to short-time Fourier transform and is combined according to the time sequence, so that the spectrum energy of the frame of audio signal is formed. I.e. the spectral energy of each frame can be represented using a one-dimensional array (a 1, a2, a3, …, a 256), corresponding to spectral energy magnitudes of 31.25Hz,62.5Hz,93.75Hz, …,8000Hz, respectively, and then using a mel filter bank to obtain mel spectral energy, the relationship of mel spectrum to linear spectrum can be referred to in the prior art.
Further, the loudness of each frame can be determined according to the sound pressure amplitude of each sampling point, and the specific calculation formula can refer to the prior art.
Step S120: determining the background noise loudness according to the loudness of each frame; and determining the voiced fragments according to the loudness of each frame and the loudness of the background noise.
Background noise is usually present in the environment in which people sleep, is usually continuous and of a smooth loudness, and therefore the influence of the background noise is first excluded. Because the environments of different users are different, the sizes of the background noise are also different, and in order to accurately exclude the background noise, the loudness of the background noise is calculated first.
Specifically, the background noise loudness can be obtained by calculating the loudness of each frame, for example, the loudness of all frames in the audio signal in 5 continuous seconds is arranged into a 1-dimensional array, the mean value and the variance of the loudness are calculated, if the variance is smaller than a preset upper limit threshold value of stability, the loudness is used as the background noise, and the mean value of the loudness is used as the background noise loudness.
On the other hand, the background noise loudness can be determined according to the loudness of each frame, for example, if the loudness of a certain frame is greater than the background noise loudness, the frame is marked as a voiced frame, and if the loudness of a certain frame is less than or equal to the background noise loudness, the frame is marked as a background frame, and the voiced frame or continuous voiced frames are used as voiced fragments.
It should be noted that the above description is merely an example, and the rule for detecting the voiced sound segment may be formulated more strictly for improving the detection accuracy, which is not limited in the present application.
Step S130: and inputting a neural network snore prediction model according to the Mel frequency spectrum energy of each frame, and identifying the snore segment from the voiced segments.
The voiced segments may be a dream, a snore, or others. Thus, there is a need to identify snore pieces among the sound pieces.
The snore can be identified through a neural network snore prediction model, the model is a two-class model, the mel spectrum energy of each frame is used as an input value and is input into the neural network snore prediction model, and the output result is that the sound segment is a snore segment or a non-snore segment.
In the present application, the neural network snore prediction model is built based on a multi-layer neural network, such as a neural network snore prediction model built by combining a 1-layer full-connection layer, a 1-layer Long Short-Term Memory (LSTM), a 1-layer full-connection layer, and a 1-layer logistic regression layer (softmax).
Step S140, recognizing continuous snore fragments and snore fragment intervals, and extracting normal snore interval characteristics according to the continuous snore fragments and the snore fragment intervals, wherein the normal snore interval characteristics comprise average snore intervals, minimum snore intervals and maximum snore intervals. And determining continuous snore fragments in the sound fragments, wherein the parts between the discontinuous snore fragments are snore fragment intervals, sleeping and breathing habits of each person are different, snoring has certain regularity, and according to the continuous snore fragments and the intervals between the continuous snore fragments, extracting features to obtain normal snore interval features of a user, wherein the normal snore interval features comprise average snore intervals, minimum snore intervals and maximum snore intervals.
Step S150, determining an apnea threshold according to the definition of medical apnea; the snore segment interval above the apnea threshold is determined to be a suspected apnea segment.
The apnea threshold is 10s, as defined by medical apnea.
Further, it may be determined that the snore segment interval greater than the apnea threshold is a suspected apnea segment, such as a segment in which the snore segment interval exceeds 10 seconds, and the segment in which the snore segment interval exceeds 10 seconds is used as the suspected apnea segment.
When people snore continuously, it is indicated that people do not have respiratory disorder, and only when snore and snore are separated by more than 10 seconds, the occurrence of apnea is possible. Therefore, the snore segment is cut into continuous snore frames and intervals of more than 10 seconds, including but not limited to judging according to the loudness of each frame constituting the snore segment and the number of each frame, for example, a preset snore threshold is set, when the loudness of a certain frame is greater than the preset snore threshold, the frame is considered to be a snore frame, and if the frame numbers of multiple frames of snore frames are continuous, the multiple frames of snore frames are determined to be continuous snore segments. One or more continuous frames with the loudness smaller than the preset snore threshold value form a snore interval section, namely the interval between one sound of snore and another sound of snore, and the interval exceeds 10 seconds to be a suspected apnea fragment.
Step S160, determining the validity of the suspected apneas according to the loudness of each frame of the suspected apneas, the linear spectrum energy and the preset rule.
And determining the suspected apnea fragments, judging the effectiveness of the suspected apnea fragments, if the suspected apnea fragments are ineffective, considering that the apnea fragments do not occur, and if the suspected apnea fragments are effective, considering that the apnea fragments occur.
In some embodiments of the present application, the effectiveness of an apneic segment may use the following 3 conditions, but is not limited to these 3 conditions:
firstly, using the loudness and the 0-1K linear frequency point spectrum energy and two conditions, finding out the loudness or the sound fragments with small fluctuation of the 0-1K spectrum energy. Setting a loudness threshold value of background noise plus smaller loudness as a loudness fluctuation threshold value; finding out that the variance of the 0-1K linear frequency point spectrum energy sum within 5 seconds is smaller than a preset threshold value to be used as the background noise 0-1K linear frequency point spectrum energy sum, and superposing the preset threshold value to be used as the 0-1K linear frequency point spectrum energy sum and the steady background noise upper limit threshold value; one of the more than 2 conditions is identified as a voiced frame; the sound fragments are formed by 3 suspected sound frames;
first, if there is sound of body movement in the bed in a suspected apneic segment, the suspected apneic segment is not effective. In the application, the spectrum energy of the linear frequency point of the initial frame of the voiced segment is formed into a 1-dimensional array, the spectrum energy of the linear frequency point of the voiced frame in the subsequent voiced segment is continuously overlapped, and finally divided by the number of the voiced frames to obtain the spectrum energy of the linear frequency point of the average single frame of the voiced segment. If the fluctuation of the frequency spectrum energy of each linear frequency point of 0-8K is smaller than the preset threshold value of the fluctuation of white noise, and the time length of the sound fragment exceeds more than 2 seconds, recognizing the sound as the sound of the body moving on the bed;
Second, the time interval of the voiced segments corresponds to the respiratory characteristics obtained by continuous snoring statistics, i.e. the interval is satisfied within the minimum and maximum respiratory interval range, while the time length of the voiced segments is within the respiratory preset length range. If the condition is met, the suspected apnea segment is invalid;
third, preset rules: the duration of suspected apneas reaches or exceeds 10 seconds, otherwise it is ineffective.
It should be noted that the above description is merely an example, and the detection rule for the effectiveness of the suspected apneas may be formulated more strictly for the purpose of improving the detection accuracy, and the present application is not limited thereto.
As can be seen from the method shown in fig. 1, the present application first determines the background noise loudness of the current environment according to the loudness of each frame constituting the audio signal, the segments above the background noise loudness are marked as voiced segments, further, based on the neural network snore prediction model, the snore segments are identified from the voiced segments, in the snore segment interval, the suspected apnea segments are identified, and whether the breathing is stopped is determined according to the loudness of each frame, the linear spectral energy and the preset rules. According to the snore segment identification method based on the neural network snore prediction model, the snore segment is accurately identified, the snore interval is further identified in the snore segment, whether the user breathes smoothly is determined through monitoring of the snore interval, a solid foundation is provided for monitoring the health state of the user, and the snore segment identification method based on the neural network snore prediction model is strict in logic, small in calculated amount and high in identification accuracy.
In some embodiments of the present application, in the above method, determining the background noise loudness from the loudness of each frame includes: intercepting a background noise sample fragment from an audio signal according to a preset duration; determining the loudness of each frame that makes up a segment of background noise samples; the loudness mean and loudness variance of the background noise sample segments are determined from the loudness of the frames that make up the background noise sample segments. And comparing the loudness variance with a preset background noise variance upper limit threshold value, and taking the loudness average value of the background noise sample fragments as the background noise loudness under the condition that the loudness variance is smaller than the preset background noise variance upper limit threshold value. When the background noise loudness is determined, if the audio signal is taken as a whole sample, the calculation amount is very large, and because the sound of people's dream or snoring usually does not last the whole sleeping process, the data of the current moment forward for a preset time period can be intercepted in the audio signal as a background noise sample fragment, so that the environment that the background noise is continuously changed can be adapted, and the noise of a dawn district is more common than the noise of a district at late night.
Specifically, a background noise sample segment is cut out from the audio signal according to a preset duration, and the latest audio part is cut out, for example, the audio signal in the latest continuous 5s duration is cut out to be used as the background noise sample segment.
For the determination of the background noise, reference may be made to the prior art, or a method recommended in the present application may be adopted, specifically, the loudness of each frame that constitutes a background noise sample segment is determined, and according to the loudness of each frame, the average value of the loudness and the variance of the loudness of the whole background noise sample segment are determined, where the variance in statistics is the average of square values of differences between each sample value and the average of all sample values, and the variance can characterize the smoothness. The loudness variance of the entirety of the background noise sample segment may be calculated from the definition of the variance and the loudness of each frame.
And under the condition that the loudness variance is smaller than the preset background noise variance upper threshold, no snore, dream and other voiced fragments exist in the background noise sample fragments, only background noise exists, and further, the loudness average value of the background noise sample fragments is used as the background noise loudness. The mean and variance are defined in a general sense, and please refer to the prior art for the method of calculating the mean and variance of loudness.
And under the condition that the loudness variance is larger than a preset background noise variance upper limit threshold value, considering that the background noise sample fragment has other sounds except the background noise, discarding the background noise sample fragment, and re-intercepting.
In some embodiments of the present application, in the above method, determining the voiced segments according to the loudness of each frame and the background noise loudness includes: under the condition that the loudness of one frame is larger than the sum of the background noise loudness and the fluctuation loudness, determining the frame as a starting frame of the voiced fragments; under the condition that the loudness of the preset number of continuous frames is smaller than the sum of the background noise loudness and the fluctuation loudness, determining that a first frame of the continuous frames is a cut-off frame of a sound fragment; each frame between the start frame and the stop frame is taken as a voiced segment.
For the purpose of improving detection accuracy, in this embodiment, a fluctuation loudness is added on the basis of the background noise loudness, where the fluctuation loudness may be, for example, 4-6 db, and if one or more frames in front of a certain frame do not conform to the rule of recognizing the voiced segment, the frame is considered to be the starting frame of the voiced segment if the loudness of the frame is greater than the sum of the background noise loudness and the fluctuation loudness; for the judgment of the cut-off frame, under the condition that the loudness of the preset number of continuous frames is smaller than the sum of the background noise loudness and the fluctuation loudness, the first frame of the continuous frames is determined to be the cut-off frame of the voiced segment, for example, the loudness of all continuous 3 frames is smaller than the sum of the background noise loudness and the fluctuation loudness, the voiced segment is considered to be cut-off, and the first frame of the continuous 3 frames is regarded as the cut-off frame of the voiced segment. And finally, taking all frames between the starting frame and the cut-off frame as the voiced fragments. In some embodiments of the present application, in the above method, inputting a neural network snore prediction model according to mel spectrum energy of each frame, identifying a snore segment from the voiced segments includes: the Mel frequency spectrum capability of each frame forming the sound segment is used as an input value to be input into a neural network snore prediction model to obtain a snore probability value of each frame; and under the condition that the snore probability value is larger than a preset probability threshold value, setting the frame as a snore frame, and marking the sound fragments containing the snore frame as snore fragments.
When the snore fragments are identified, the snore prediction model of the neural network can be used, the input value is the Mel frequency spectrum energy of each frame forming the sound fragments, and the output value is the snore probability value of each frame of the sound fragments.
And comparing the snore probability value with a preset probability threshold, if the preset probability threshold is set to be 0.35, marking the frame as a snore frame and marking the sound fragment containing the snore frame as a suspected snore fragment under the condition that the snore probability value is greater than or equal to the preset probability threshold.
For the sound segments which are not marked as suspected snore segments, namely the sound segments which do not contain the snore frames, the sound segments are not marked as snore segments by the neural network snore prediction model, but in order to improve the detection accuracy, the embodiment further detects and extracts the snore by combining the rhythm characteristics of the snore and the average linear spectrum energy similar characteristics. Firstly, determining a continuous suspected snore segment from the suspected snore segments, specifically, identifying whether adjacent snores accord with breathing characteristics or not, and if the frame number of the starting frame of the previous snore segment and the frame number of the starting frame of the next snore segment do not exceed a preset threshold, for example, 10 seconds, then the continuous suspected snore segment can be considered.
Under the condition that the continuous suspected snore fragments meet the preset snore rule, marking the continuous suspected snore fragments as snore fragments, wherein the preset snore rule is as follows: adjacent suspected snore starting frames are within a preset breath threshold, such as 2-7 seconds.
For the sound section which is not identified as the suspected snore, in order to improve the detection accuracy, the snore is further detected and extracted by adopting a preset snore rule.
The preset snore rule can be understood as two sub-snore rules, namely a first snore rule and a second snore rule, wherein the first snore rule characterizes the rhythm characteristics of the snore, and specifically comprises that if the time of the initial frame interval frame of the continuous sound segment is in a preset duration range, the range is a normal range of human breathing, the preset duration range can be but is not limited to 2-7s, and 3 continuous time intervals are kept stable, if the variance of the time intervals is less than 1 second, the first snore rule is satisfied, otherwise, the first snore rule is not satisfied.
The snore rule II characterizes the similar characteristics of linear spectrum energy, the latest 4 continuous sound fragments are selected as samples, the linear spectrum energy from the starting frame to each frame of the cut-off frame of each sound fragment is overlapped, the frame average linear spectrum energy of the sound fragments is obtained by dividing the frame number from the starting frame to the cut-off frame, the local maximum value of the linear spectrum energy at the linear frequency points is calculated, the corresponding frequency points are placed in the characteristic frequency point array, if the characteristic frequency points of the 4 continuous sound fragments are consistent, the snore rule II is satisfied, otherwise, the snore rule II is not satisfied.
In some embodiments of the present application, the determining the validity of the suspected apneas fragment according to the loudness of each frame of the suspected apneas fragment, the linear spectral energy, and the preset rule includes:
and determining a suspected action segment according to the loudness of each frame of the suspected action segment and the linear spectrum energy, wherein the loudness of each frame forming the suspected action segment is larger than a first preset loudness threshold, meanwhile, the frame mean linear spectrum energy is kept stable in a full frequency band, and the suspected action segment is determined to be invalid under the condition that the duration of the suspected action segment is larger than a first preset duration threshold.
In the snoring process, actions such as turning, moving and the like can also occur, and snoring is stopped when the actions occur normally, so that the influence of the actions is preferably eliminated when the effectiveness of suspected apnea fragments is detected. Specifically, the spectral energy of the linear frequency points of the initial frame of the voiced segment is formed into a 1-dimensional array, the spectral energy of the linear frequency points of all frames in the voiced segment is overlapped, and finally the spectral energy of the linear frequency points of the average single frame of the voiced segment is obtained by dividing the spectral energy by the number of the voiced frames. If the spectral energy fluctuation of each linear frequency point of 0-8K is smaller than the preset threshold value of white noise fluctuation, and the time length of the sound fragment exceeds more than 2 seconds, the body is identified to move in the bed, and the body is not in an apnea state.
In some embodiments of the present application, the determining the validity of the suspected apneas segment according to the loudness of each frame of the suspected apneas segment and the preset rule further includes: and determining a suspected breathing segment according to the loudness of each frame of the suspected apnea segment, wherein the loudness of each frame forming the suspected breathing segment is larger than a second preset loudness threshold, the time interval of the sound segment accords with the breathing characteristics obtained by continuous snore statistics, namely the interval is satisfied within the range of the minimum and maximum breathing intervals, and the time length of the sound segment is within the range of the breathing preset length. And if the condition is met, determining that the suspected apnea fragment is marked as invalid.
Further, people may stop snoring suddenly due to actions, environmental changes and the like to enter a normal breathing state in the snoring process, and in this case, people breathe uniformly with little breathing sound, so that the influence of people entering the normal breathing state needs to be eliminated.
Similarly, it is also possible to set a suitable second preset loudness threshold, where the value of the second preset loudness threshold is relatively small, such as 4 db, and the second preset loudness threshold is smaller than the first preset loudness threshold. And further, determining that the suspected breathing pause segment is invalid when the duration of the suspected breathing segment is smaller than a second duration threshold.
The second duration threshold may be calculated according to a rule of snoring of the user, for example, counting duration of discontinuous snoring frames, i.e. snoring intervals, in the snoring segment, obtaining a minimum duration and a maximum duration, and obtaining a weighted average according to the minimum duration and the maximum duration.
Fig. 2 shows an apparatus for determining apnea according to an embodiment of the present application, and as can be seen from fig. 2, the apparatus 200 includes:
an obtaining unit 210, configured to obtain an audio signal, and determine loudness, linear spectral energy and mel spectral energy of each frame in the audio signal;
a first identifying unit 220, configured to determine a background noise loudness according to the loudness of each frame, and determine a voiced segment according to the loudness of each frame and the background noise loudness;
a second identifying unit 230, configured to input a neural network snore prediction model according to mel spectrum energy of each frame, and identify a snore segment from the voiced segments;
a third identifying unit 240, configured to identify a continuous snore segment and a snore segment interval, and extract a normal snore interval characteristic according to the continuous snore segment and the snore segment interval, where the normal snore interval characteristic includes an average snore interval, a minimum snore interval, and a maximum snore interval;
A fourth recognition unit 250 for determining an apnea threshold according to a definition of medical apnea; determining snore segment intervals greater than an apnea threshold as suspected apnea segments;
the judging unit 260 is configured to determine the validity of the suspected apneas according to the loudness of each frame of the suspected apneas, the linear spectrum energy and a preset rule.
In some embodiments of the present application, in the above apparatus, the acquiring unit 210 is configured to acquire an audio signal at a preset frequency; performing frame division, frame shifting and short-time Fourier transformation on the audio signal to obtain linear spectrum energy of each frame, and performing Mel spectrum transformation on the linear spectrum energy to obtain Mel spectrum energy; and determining the loudness of each frame according to the sampled microphone sound pressure signal data. .
In some embodiments of the present application, in the above apparatus, the method is configured to intercept a background noise sample segment in the audio signal according to a preset duration; determining the loudness of each frame that makes up the background noise sample segment; determining the loudness mean and loudness variance of the background noise sample fragments according to the loudness of each frame constituting the background noise sample fragments; and comparing the loudness variance with a preset snore threshold, and taking the loudness mean value of the background noise sample fragments as the background noise loudness under the condition that the loudness variance is smaller than the preset snore threshold.
In some embodiments of the present application, in the foregoing apparatus, the first identifying unit 220 is configured to determine that a frame is a start frame of the voiced segment if a loudness of the frame is greater than a sum of the background noise loudness and the fluctuation loudness; under the condition that the loudness of a preset number of continuous frames is larger than the sum of the background noise loudness and the fluctuation loudness, determining that a first frame of the continuous frames is a cut-off frame of the voiced segment; and taking the amplitude spectrum of each frame between the starting frame and the cut-off frame as a voiced segment.
In some embodiments of the present application, in the above apparatus, the second identifying unit 230 is configured to input mel spectrum energy of each frame that forms the voiced segments as an input value to the neural network snore prediction model, to obtain a snore probability value of each voiced frame; and under the condition that the snore probability value is greater than or equal to a preset probability threshold value, marking the sound frame as a snore frame, and marking the sound segment containing the snore frame as a snore segment.
In some embodiments of the present application, in the above device, the second identifying unit 230 is further configured to determine a continuous suspected snore segment from the snore segments; and marking the continuous suspected snore fragments as snore fragments under the condition that the continuous suspected snore fragments meet the preset snore rule, wherein the preset snore rule is as follows: the time of the initial frame interval of the adjacent snore segments in the continuous suspected snore segments is in a preset duration range; in addition, in order to improve the detection accuracy, the snore is further detected and extracted in the continuous sound section by combining the rhythm characteristic of the snore and the average linear spectrum energy similar characteristic, and besides the preset snore rule, the local maximum value of the single-frame average linear spectrum energy of each sound section forming the continuous sound section is completely consistent, namely the average linear spectrum energy similar characteristic is met, and the snore section is marked.
In some embodiments of the present application, in the foregoing apparatus, the determining unit 250 is configured to determine that the suspected apneas fragment is a suspected body moving fragment in bed when the loudness and the linear spectral energy of each frame of the suspected apneas fragment satisfy a first preset rule, where the first preset rule is: the loudness of each frame composing the suspected body moving section on the bed is larger than the sum of the background noise loudness and a first preset loudness threshold value, and the linear spectrum energy is uniformly distributed in the whole frequency band; and determining that the mark of the suspected apnea fragment is invalid under the condition that the time length of the suspected body moving fragment in the bed is greater than a first preset time length threshold value.
In some embodiments of the present application, in the foregoing apparatus, the determining unit 250 is configured to determine that the suspected apneas fragment is a suspected apneas fragment if the loudness and the linear spectral energy of each frame of the suspected apneas fragment satisfy a second preset rule, where the second preset rule is: the loudness of each frame forming the suspected breathing fragment is larger than the sum of the background noise loudness and a second preset loudness threshold, or the sum of the linear frequency point spectrum energy of 0-1K is larger than the sum of the linear frequency point spectrum energy of 0-1K of background noise and the sum of the linear frequency point spectrum energy of the superimposed first preset 0-1K;
And under the condition that the duration of the suspected breathing fragments is smaller than a second duration threshold, the interval time between the initial frames of adjacent suspected breathing fragment features is in a second preset duration range, and the suspected apnea fragments are determined to be invalid.
It can be understood that the above apparatus can implement the steps of the method provided in the foregoing embodiments, and the relevant explanation about the method is applicable to the apparatus and will not be repeated herein.
Fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Referring to fig. 3, at the hardware level, the electronic device includes a processor, and optionally an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, network interface, and memory may be interconnected by an internal bus, which may be an ISA (Industry Standard Architecture ) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus, or EISA (Extended Industry Standard Architecture ) bus, among others. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 3, but not only one bus or type of bus.
And the memory is used for storing programs. In particular, the program may include program code including computer-operating instructions. The memory may include memory and non-volatile storage and provide instructions and data to the processor.
The processor reads the corresponding computer program from the nonvolatile memory to the memory and then runs, and an apnea judging device is formed on a logic level. And the processor executes the program stored in the memory.
The method executed by the above-described apnea judging device according to the embodiment shown in fig. 3 of the present application may be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the signals in the memory and, in combination with its hardware, performs the steps of the above method.
The electronic device may further execute the method executed by the apnea judging device in fig. 3, and implement the function of the apnea judging device in the embodiment shown in fig. 3, which is not described herein.
The embodiments of the present application also provide a computer-readable storage medium storing one or more programs, where the one or more programs include instructions, which when executed by an electronic device that includes a plurality of application programs, enable the electronic device to perform a method performed by the apnea determination device in the embodiment shown in fig. 3.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement signal storage by any method or technology. The signals may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store signals that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. which are within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.
Claims (10)
1. A method for determining apnea, comprising:
acquiring an audio signal, and determining the loudness, linear spectrum energy and mel spectrum energy of each frame in the audio signal;
determining the loudness of background noise according to the loudness of each frame, and determining a sound fragment according to the loudness of each frame and the loudness of the background noise;
Inputting a neural network snore prediction model according to the Mel frequency spectrum energy of each frame, and identifying a snore segment from the voiced segments;
identifying continuous snore fragments and snore fragment intervals, and extracting normal snore interval characteristics according to the continuous snore fragments and the snore fragment intervals, wherein the normal snore interval characteristics comprise average snore intervals, minimum snore intervals and maximum snore intervals;
determining an apnea threshold according to a definition of medical apnea; determining snore segment intervals greater than an apnea threshold as suspected apnea segments;
and determining the effectiveness of the suspected apnea fragments according to the loudness of each frame of the suspected apnea fragments, the linear spectrum energy and a preset rule.
2. The method of claim 1, wherein the acquiring the audio signal, determining the loudness, linear spectral energy, and mel spectral energy of each frame in the audio signal comprises:
collecting audio signals at a preset frequency;
performing frame division, frame shifting and short-time Fourier transformation on the audio signal to obtain linear spectrum energy of each frame, and performing Mel spectrum transformation on the linear spectrum energy to obtain Mel spectrum energy;
And, a step of, in the first embodiment,
and determining the loudness of each frame according to the sampled microphone sound pressure signal data.
3. The method of claim 1, wherein said determining a background noise loudness from the loudness of the frames comprises:
intercepting a background noise sample fragment from the audio signal according to a preset duration;
determining the loudness of each frame that makes up the background noise sample segment;
determining the loudness mean and loudness variance of the background noise sample fragments according to the loudness of each frame constituting the background noise sample fragments;
and comparing the loudness variance with a preset steady noise upper limit threshold value, and taking the loudness mean value of the background noise sample fragments as the background noise loudness under the condition that the loudness variance is smaller than the preset steady noise upper limit threshold value.
4. The method of claim 1, wherein said determining the voiced segments based on the loudness of the frames and the background noise loudness comprises:
under the condition that the loudness of a frame is larger than the sum of the background noise loudness and the fluctuation loudness, determining the frame as the initial frame of the voiced segment;
under the condition that the loudness of the preset number of continuous frames is smaller than the sum of the background noise loudness and the fluctuation loudness, determining that a first frame of the continuous frames is a cut-off frame of the voiced segment;
And taking each frame between the starting frame and the cut-off frame as a sound fragment.
5. The method of claim 1, wherein extracting mel-frequency spectral energy for each frame, inputting a neural network snore prediction model, and identifying a snore segment from the voiced segments comprises:
the Mel frequency spectrum energy of each frame forming the sound segment is used as an input value to be input into the neural network snore prediction model to obtain a snore probability value of each frame, and the frame higher than a snore probability threshold is marked as a snore frame;
the voiced segments containing the snore frames are marked as snore segments.
6. The method of claim 5, wherein said inputting a neural network snore prediction model based on mel-frequency spectral energy of said frames, identifying a snore segment from said voiced segments further comprises:
determining a continuous suspected snore segment from the voiced segment;
and marking the continuous suspected snore fragments as snore fragments under the condition that the continuous suspected snore fragments meet the preset snore rule, wherein the preset snore rule is as follows: the interval time between the initial frames of adjacent consecutive suspected snore segments is within a first preset duration range.
7. The method of claim 6, wherein said inputting a neural network snore prediction model based on mel-frequency spectral energy of said frames, identifying a snore segment from said voiced segments further comprises:
determining normal adjacent continuous suspected snore fragments according to the sound fragments and the snore fragments;
and determining the average value, the minimum value and the maximum value of the intervals between the initial frames of the normal adjacent continuous suspected snore fragments, and storing the average value, the minimum value and the maximum value as personalized breathing characteristic data.
8. The method of claim 1, wherein determining the validity of the suspected apneas according to the loudness of the frames of the suspected apneas, the linear spectral energy, and the preset rules comprises:
under the condition that the loudness and linear spectrum energy of each frame of the suspected apnea fragment meet a first preset rule, determining the suspected apnea fragment as a suspected body moving fragment in a bed, wherein the first preset rule is as follows: the loudness of each frame composing the suspected body moving section on the bed is larger than the sum of the background noise loudness and a first preset loudness threshold value, and the linear spectrum energy is uniformly distributed in the whole frequency band;
And determining that the mark of the suspected apnea fragment is invalid under the condition that the time length of the suspected body moving fragment in the bed is greater than a first preset time length threshold value.
9. The method of claim 1, wherein determining the validity of the suspected apneas according to the loudness of the frames of the suspected apneas, the linear spectral energy, and the preset rules further comprises:
and under the condition that the loudness and linear spectrum energy of each frame of the suspected apnea fragment meet a second preset rule, determining that the suspected apnea fragment is a suspected apnea fragment, wherein the second preset rule is as follows: the loudness of each frame forming the suspected breathing fragment is larger than the sum of the background noise loudness and a second preset loudness threshold, or the sum of the linear frequency point spectrum energy of 0-1K is larger than the sum of the linear frequency point spectrum energy of 0-1K of background noise and the sum of the linear frequency point spectrum energy of the superimposed first preset 0-1K;
and under the condition that the duration of the suspected breathing fragments is smaller than a second duration threshold, the interval time between the initial frames of adjacent suspected breathing fragment features is in a second preset duration range, and the suspected apnea fragments are determined to be invalid.
10. An apnea judging device, comprising:
an acquisition unit for acquiring an audio signal, and determining the loudness, linear spectrum energy and mel spectrum energy of each frame in the audio signal;
the first identification unit is used for determining background noise loudness according to the loudness of each frame and determining a sound fragment according to the loudness of each frame and the background noise loudness;
the second identification unit is used for inputting a neural network snore prediction model according to the Mel frequency spectrum energy of each frame, and identifying a snore segment from the voiced segments;
the third recognition unit is used for recognizing the continuous snore fragments and the snore fragment intervals, and extracting normal snore interval characteristics according to the continuous snore fragments and the snore fragment intervals, wherein the normal snore interval characteristics comprise average snore intervals, minimum snore intervals and maximum snore intervals;
a fourth recognition unit for determining an apnea threshold according to a definition of medical apnea; determining snore segment intervals greater than an apnea threshold as suspected apnea segments;
and the judging unit is used for determining the effectiveness of the suspected apnea fragments according to the loudness of each frame of the suspected apnea fragments, the linear spectrum energy and a preset rule.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110629654.0A CN113314143B (en) | 2021-06-07 | 2021-06-07 | Method and device for judging apnea and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110629654.0A CN113314143B (en) | 2021-06-07 | 2021-06-07 | Method and device for judging apnea and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113314143A CN113314143A (en) | 2021-08-27 |
| CN113314143B true CN113314143B (en) | 2024-01-30 |
Family
ID=77377478
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110629654.0A Active CN113314143B (en) | 2021-06-07 | 2021-06-07 | Method and device for judging apnea and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113314143B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115206329B (en) * | 2022-09-15 | 2023-01-24 | 深圳市倍轻松科技股份有限公司 | Method, device, electronic equipment and storage medium for determining snore signals |
| CN116312634A (en) * | 2023-03-10 | 2023-06-23 | 漳州松霖智能家居有限公司 | Snore detection method and device based on multi-model superposition |
| CN116570239A (en) * | 2023-04-14 | 2023-08-11 | 漳州松霖智能家居有限公司 | Snore detection method and device |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102499637A (en) * | 2011-09-26 | 2012-06-20 | 大连理工大学 | Obstructive sleep apnea-hypopnea syndrome screening method and device thereof |
| CN104688229A (en) * | 2015-01-28 | 2015-06-10 | 中国人民解放军理工大学 | Method for monitoring sleep respiration based on snore signals |
| CN104739412A (en) * | 2013-12-29 | 2015-07-01 | 中国移动通信集团公司 | Method and equipment for monitoring sleep apnea |
| CN106691382A (en) * | 2016-12-26 | 2017-05-24 | 赛博龙科技(北京)有限公司 | Snore detection method and device based on time frequency similarity |
| CN109431470A (en) * | 2018-12-20 | 2019-03-08 | 西安交通大学医学院第二附属医院 | Sleep breath monitoring method and device |
| CN111166297A (en) * | 2020-02-19 | 2020-05-19 | 赛博龙科技(北京)有限公司 | Method and device for evaluating sleep quality based on user sleep audio |
| CN111345782A (en) * | 2020-03-12 | 2020-06-30 | 深圳数联天下智能科技有限公司 | Snore type identification method and electronic equipment |
| CN111696575A (en) * | 2020-06-19 | 2020-09-22 | 杭州电子科技大学 | Low ventilation and apnea detection and identification system based on hybrid neural network model |
| CN111938650A (en) * | 2020-07-03 | 2020-11-17 | 上海诺斯清生物科技有限公司 | Method and device for monitoring sleep apnea |
| CN111938649A (en) * | 2019-05-16 | 2020-11-17 | 医疗财团法人徐元智先生医药基金会亚东纪念医院 | Method for predicting sleep apnea from snore by using neural network |
| CN112472066A (en) * | 2020-11-25 | 2021-03-12 | 陈向军 | Breathing disorder monitoring terminal, monitor and system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010066008A1 (en) * | 2008-12-10 | 2010-06-17 | The University Of Queensland | Multi-parametric analysis of snore sounds for the community screening of sleep apnea with non-gaussianity index |
| WO2017029317A1 (en) * | 2015-08-17 | 2017-02-23 | Resmed Sensor Technologies Limited | Screener for sleep disordered breathing |
| US11793453B2 (en) * | 2019-06-04 | 2023-10-24 | Fitbit, Inc. | Detecting and measuring snoring |
-
2021
- 2021-06-07 CN CN202110629654.0A patent/CN113314143B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102499637A (en) * | 2011-09-26 | 2012-06-20 | 大连理工大学 | Obstructive sleep apnea-hypopnea syndrome screening method and device thereof |
| CN104739412A (en) * | 2013-12-29 | 2015-07-01 | 中国移动通信集团公司 | Method and equipment for monitoring sleep apnea |
| CN104688229A (en) * | 2015-01-28 | 2015-06-10 | 中国人民解放军理工大学 | Method for monitoring sleep respiration based on snore signals |
| CN106691382A (en) * | 2016-12-26 | 2017-05-24 | 赛博龙科技(北京)有限公司 | Snore detection method and device based on time frequency similarity |
| CN109431470A (en) * | 2018-12-20 | 2019-03-08 | 西安交通大学医学院第二附属医院 | Sleep breath monitoring method and device |
| CN111938649A (en) * | 2019-05-16 | 2020-11-17 | 医疗财团法人徐元智先生医药基金会亚东纪念医院 | Method for predicting sleep apnea from snore by using neural network |
| CN111166297A (en) * | 2020-02-19 | 2020-05-19 | 赛博龙科技(北京)有限公司 | Method and device for evaluating sleep quality based on user sleep audio |
| CN111345782A (en) * | 2020-03-12 | 2020-06-30 | 深圳数联天下智能科技有限公司 | Snore type identification method and electronic equipment |
| CN111696575A (en) * | 2020-06-19 | 2020-09-22 | 杭州电子科技大学 | Low ventilation and apnea detection and identification system based on hybrid neural network model |
| CN111938650A (en) * | 2020-07-03 | 2020-11-17 | 上海诺斯清生物科技有限公司 | Method and device for monitoring sleep apnea |
| CN112472066A (en) * | 2020-11-25 | 2021-03-12 | 陈向军 | Breathing disorder monitoring terminal, monitor and system |
Non-Patent Citations (1)
| Title |
|---|
| 鼾声检测研究综述;孙井鹏;胡晰远;彭思龙;马彦;;世界睡眠医学杂志(第03期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113314143A (en) | 2021-08-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113314143B (en) | Method and device for judging apnea and electronic equipment | |
| US20240050027A1 (en) | Apparatus and method for diagnosing sleep quality | |
| Sun et al. | SymDetector: detecting sound-related respiratory symptoms using smartphones | |
| US6290654B1 (en) | Obstructive sleep apnea detection apparatus and method using pattern recognition | |
| CN104739412B (en) | A kind of method and apparatus being monitored to sleep apnea | |
| Hoyos-Barceló et al. | Efficient k-NN implementation for real-time detection of cough events in smartphones | |
| Lin et al. | Automatic wheezing detection using speech recognition technique | |
| CN110853605B (en) | Music generation method and device and electronic equipment | |
| Chang et al. | DAG-SVM based infant cry classification system using sequential forward floating feature selection | |
| Reggiannini et al. | A flexible analysis tool for the quantitative acoustic assessment of infant cry | |
| CN112155560B (en) | Apnea detection method and system based on real-time cardiac shock signal | |
| JP2020185390A (en) | Method for predicting sleep apnea | |
| Lu et al. | Research on sports video detection technology motion 3D reconstruction based on hidden Markov model | |
| CN109767784B (en) | Snore identification method and device, storage medium and processor | |
| US11382534B1 (en) | Sleep detection and analysis system | |
| CN111685731B (en) | Sleep data processing method, device, equipment and storage medium | |
| Markandeya et al. | Smart phone based snoring sound analysis to identify upper airway obstructions | |
| Fischer et al. | Classification of breath and snore sounds using audio data recorded with smartphones in the home environment | |
| CN116211256A (en) | Non-contact sleep breathing signal acquisition method and device | |
| Bi et al. | FamilyLog: monitoring family mealtime activities by mobile devices | |
| CN115137310A (en) | Sleep data management method and system based on music identification of Internet of things | |
| Montazeri Ghahjaverestan et al. | Sleep/wakefulness detection using tracheal sounds and movements | |
| WO2024188177A1 (en) | Snore detection method and apparatus based on multi-model superimposition | |
| Messaoud et al. | A cry-based babies identification system | |
| Abushakra et al. | Efficient frequency-based classification of respiratory movements |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |