JP6464853B2 - Audio playback apparatus and audio playback program - Google Patents

Description

  The present invention relates to an audio playback apparatus and an audio playback program for playing back music (music, accompaniment, etc.) based on audio data representing an acoustic waveform.

  2. Description of the Related Art Conventionally, as described in Patent Document 1, for example, an electronic musical instrument (for example, an electronic piano) having a keyboard performance practice function is known. This electronic musical instrument stores music data representing the performance of music. The music data is so-called MIDI data. In other words, the music data includes note information including the key number and key pressing (key release) strength, and timing information indicating the timing of key pressing (key release). The music data is composed of a plurality of track data corresponding to each performance part. This electronic musical instrument has a keyboard device composed of a plurality of keys and a plurality of guide lamps provided corresponding to the plurality of keys. In the practice mode, when the user plays the main melody part, the control unit (sound source circuit) of the electronic musical instrument reproduces the performance of the accompaniment part based on the track data of the accompaniment part constituting the music data. Further, the control unit emits a guide lamp corresponding to a key to be depressed by the user at each timing based on the track data of the main melody part constituting the music data. When the key corresponding to the emitted guide lamp is not pressed, the control unit pauses the reproduction of the music data. Thereby, the performance of the accompaniment part is paused. When the user presses the key corresponding to the emitted guide lamp, the control unit resumes the reproduction of the music data.

  Conventionally, as described in Patent Documents 2 and 3, for example, electronic musical instruments that reproduce the performance of the accompaniment part in synchronization with the performance progress of the main melody part by the performer are known. The control unit of the electronic musical instrument reproduces the performance of the accompaniment part based on the accompaniment data representing the performance of the accompaniment part. Accompaniment data is so-called audio data. That is, the accompaniment data is composed of a plurality of sample values obtained by sampling a sound (phrase) generated by playing a musical instrument (for example, drum, bass, guitar, etc.) at a predetermined sampling period. The controller reproduces the performance of the accompaniment part by sequentially reading out the sample values and converting them into acoustic signals.

  Further, the electronic musical instrument has main melody data corresponding to the score of the main melody part. The main melody data is so-called MIDI data. The main melody data consists of a plurality of note data. The note data includes a key number indicating the key number to be pressed (released) and timing information indicating the timing to press (release key). The tempo of the main melody part represented by the main melody data matches the tempo of the accompaniment part. The controller reproduces main melody data and accompaniment data simultaneously. Specifically, the control unit sequentially reads the sample values at the predetermined sampling period, converts the sample values into acoustic signals, and sequentially selects note information based on the timing information. When the key corresponding to the selected note information is not pressed (released) within a predetermined time, the control unit repeatedly reproduces a part of the accompaniment data. In this way, the control unit synchronizes the performance progress of the main melody part and the performance progress of the accompaniment part.

  An audio playback device (for example, a CD player) that plays back music (music) based on audio data representing an acoustic waveform is known. In the conventional audio playback apparatus, music playback can be paused.

JP 2004-101979 A JP 2013-47713 A JP 2012-220593 A

  When a plurality of performers play an acoustic instrument, each performer adjusts the progress (or tempo) of his / her performance while listening to the performances of other performers. For example, when practicing the performance of a song, if one player temporarily stops playing during the performance, the other performers also stop playing temporarily, and the paused performance progress position (for example, temporary The practice may be resumed from the beginning of the stopped measure. In this case, for example, even if the percussion instrument player pauses the performance immediately after playing the percussion instrument, unless the performer intentionally stops the percussion instrument's percussion vibration (choke performance), The sound continues to be emitted until it decays and its volume is “0”.

  The control unit of the electronic musical instrument of Patent Document 1 pauses the reproduction of the music data when the key corresponding to the lighted guide lamp is not pressed. Here, a situation is assumed in which the generation of a musical sound is started by supplying one note information to the sound source circuit, and the reproduction of the music data is paused immediately after that. If the musical sound is set to attenuate at a predetermined attenuation rate, the control unit (sound source circuit) does not immediately mute the musical sound but attenuates it at the predetermined attenuation rate. Therefore, in this case, it is possible to simulate the sound generation mode when the performance of an acoustic instrument (a musical instrument that emits a decaying sound) is paused.

  On the other hand, even if the performance of the main melody is paused, the control unit of the electronic musical instruments of Patent Documents 2 and 3 does not pause the reproduction of the accompaniment data, but does not suspend a part of the accompaniment data. Play repeatedly. That is, even if the performer pauses the performance of the main melody, the accompaniment sound continues to be generated without being attenuated. In many cases, the sound generated by repeating a part of the accompaniment data is a sound that cannot be generated by an acoustic instrument. Therefore, the performer may feel that the accompaniment sound is generated in an unnatural manner.

  Further, in the above-described conventional audio reproduction apparatus, the sound volume is gradually reduced in the section where the sound is being attenuated (for example, the section after the attack portion of the sound emitted by percussion is performed). When the playback is paused in (interval), the volume becomes “0” at the moment when the pause is instructed, as indicated by a broken line in FIG. 6B. When the music reproduction is resumed, the music is reproduced from the paused position. That is, as shown in the figure, the sound is reproduced from a portion where the sound is slightly attenuated. As described above, when music playback is paused and when music playback is resumed in the above-described conventional audio playback device, the sound generation mode is when the performance of an acoustic instrument (a musical instrument that emits attenuated sound) is paused. And the sound generation mode when the performance is resumed. Therefore, the performer may feel that the sound generation mode at the time of pause and restart of music playback by the conventional audio playback device is unnatural.

  The present invention has been made to address the above-described problems, and an object of the present invention is to provide an audio playback apparatus that can naturally feel the sound generation mode when music playback is paused. In addition, in the description of each constituent element of the present invention below, in order to facilitate understanding of the present invention, reference numerals of corresponding portions of the embodiment are described in parentheses, but each constituent element of the present invention is The present invention should not be construed as being limited to the configurations of the corresponding portions indicated by the reference numerals of the embodiments.

  In order to achieve the above object, a feature of the present invention is reproduction means for reproducing music based on an audio signal representing an acoustic waveform of music, wherein the music based on the audio signal is at any timing of the audio signal. Based on the change in the frequency characteristics in the section before the music pause timing and the playback means (RD, WF, TR, SG, SW, IT, SY, 16, 17) Estimating means (ES) for estimating the attenuation characteristic of each frequency component of the signal, and the reproducing means attenuates each frequency component in an interval after the pause timing based on the attenuation characteristic. An audio reproduction device (10) provided with attenuation sound reproduction means (DG, SW, IT, SY, 16, 17) for reproducing the attenuation sound representing the state of going; It lies in the fact was. Note that the playback means may temporarily stop the music playback when a predetermined trigger condition is satisfied. For example, in an apparatus for reproducing an accompaniment pattern in synchronization with a user's manual performance, when the manual performance is temporarily stopped, the playback means may temporarily stop the music playback. Also, for example, in an apparatus in which audio signals are sequentially supplied to the playback means, the playback means may temporarily stop playing music when the supply of audio signals is temporarily stagnant.

  The audio playback apparatus configured as described above attenuates the sound that has started to be sounded immediately before the paused portion during the pause of music playback. As a result, it is possible to simulate the sound generation mode when the performance of the acoustic instrument (the instrument that emits the attenuated sound) is paused.

  Another feature of the present invention is that, when the playback unit releases the pause and restarts the playback of the music, the playback unit determines whether or not the audio signal corresponds to the attenuation characteristic and the length of the attenuated sound. The present invention has a correction means (SG) for correcting a section after the pause timing.

  According to this, it is possible to prevent the attenuation component generated during the pause period from being reproduced again when the music reproduction is resumed. Therefore, the sound generation mode at the time of resuming reproduction can be felt naturally. In the case where the pause period is relatively long and the musical sound that has started to sound immediately before the paused position is completely attenuated, when the music playback is resumed, the music playback of the audio data is resumed. The decay component of the musical tone that starts sounding in the section immediately before the paused position is deleted from the portion corresponding to the section immediately after the paused position.

  Another feature of the present invention resides in that the reproduction control means has a determination means (CT) for determining the temporary stop timing based on the estimation error of the attenuation characteristic.

  According to this, the start of the pause can be postponed until the reproduction progress position suitable for the pause position is detected. That is, for example, it is possible to prevent a pause from being started at a portion where the frequency component is fluctuating violently and where it is difficult to determine an attenuation rate (a portion where the error is large), such as a sound rising portion (attack).

  Note that the present invention is not limited to implementation as an audio playback device, but can also be implemented as a computer program (audio playback program) applied to a computer provided in the audio playback device.

1 is a block diagram illustrating a configuration of an electronic musical instrument to which an audio playback device according to an embodiment of the present invention is applied. It is a block diagram which shows the state from which the computer part functioned as an audio reproduction apparatus. It is a conceptual diagram which shows the windowing process performed with respect to the frame cut out from audio data. It is a graph showing the amplitude characteristic of each frame. It is a graph which shows the fluctuation | variation of the amplitude regarding one frequency bin of audio data, and the approximate curve which represents the fluctuation | variation most likely. It is a graph which shows the fluctuation | variation of the amplitude regarding one frequency bin of audio data. It is a graph which shows the fluctuation | variation of the amplitude regarding one frequency bin of a reproduction | regeneration signal.

  An electronic musical instrument 10 to which an audio playback device according to an embodiment of the present invention is applied will be described. First, an outline of the electronic musical instrument 10 will be described. The electronic musical instrument 10 stores a plurality of music data corresponding to a plurality of types of music. The music data is composed of main melody data corresponding to the score of the main melody part of music and accompaniment data representing the acoustic waveform of the performance sound of the accompaniment part. The main melody data is so-called MIDI data. That is, the main melody data is composed of a plurality of note data. The note data includes a key number indicating the key number to be pressed (released) and timing information indicating the timing to press (release key). Accompaniment data is so-called audio data. In other words, each accompaniment data is obtained by sampling a sound generated by playing an instrument of an accompaniment part (for example, drum, bass and guitar) at a predetermined sampling period (for example, 1/444100 seconds). Consists of sample values. In the performance, a plurality of musical sounds may be pronounced simultaneously. The tempo of the main melody part represented by the main melody data matches the tempo of the accompaniment part. The accompaniment data corresponds to the audio signal of the present invention.

  The electronic musical instrument 10 includes a performance operator (for example, a keyboard device) and a setting operator. The user operates the setting operator to select one piece of music data, starts playing the accompaniment part, and operates the keyboard device to perform the main melody part (manual performance). The electronic musical instrument 10 refers to the timing information, and selects note data corresponding to the current timing (performance progress) from the note data constituting the main melody data. If the key corresponding to the key number of the selected note data is not pressed (not released) within a predetermined time (for example, within one second), the electronic musical instrument 10 determines that the manual performance by the user has been paused. Then, playback of the accompaniment part is paused. At this time, the electronic musical instrument 10 does not immediately set the volume of the accompaniment part to “0”, but reproduces a state in which the musical sound of the accompaniment part is attenuated. Specifically, the electronic musical instrument 10 attenuates the musical sound of the accompaniment part based on a plurality of sample values constituting a section (a plurality of frames) immediately before the paused portion of the accompaniment data. Is estimated, and the estimated attenuation component is reproduced.

  Further, the electronic musical instrument 10 detects that the manual performance by the user has been resumed, and resumes the reproduction of the performance of the accompaniment part from the paused position. At that time, when the accompaniment part performance is resumed using the accompaniment data as it is, as shown by the broken line in FIG. ) Will be played. When the electronic musical instrument 10 pauses the reproduction of the performance of the accompaniment part, the attenuation component is estimated and reproduced. Therefore, when the reproduction of the accompaniment part is resumed using the accompaniment data as it is, the player (or listener) ) Feels that the attenuation component has been reproduced twice. Therefore, the electronic musical instrument 10 corrects the data in the section immediately after the paused portion of the accompaniment data so that the attenuation component is not reproduced twice. In other words, when the electronic musical instrument 10 resumes the playback of the accompaniment part, the section corresponding to the section immediately after the position where the playback of the accompaniment part is paused in the accompaniment data is the section immediately before the paused position. The musical tone that has started to be sounded is corrected so as to represent a state in which the musical sound has attenuated during the pause period. Then, the electronic musical instrument 10 resumes the reproduction of the accompaniment part based on the modified accompaniment data.

  Next, a specific configuration of the electronic musical instrument 10 will be described. As shown in FIG. 1, the electronic musical instrument 10 includes an input operator 11, a computer unit 12, a display 13, a storage device 14, an external interface circuit 15, and a sound source circuit 16, which are connected via a bus BUS. It is connected. A waveform memory WM and a sound system 17 are connected to the sound source circuit 16.

  The input operator 11 includes a setting operator and a performance operator. The setting operator includes a switch corresponding to an on / off operation, a volume or rotary encoder corresponding to a rotation operation, a volume or linear encoder corresponding to a slide operation, a mouse, a touch panel, and the like. These setting operators are used, for example, when selecting music data. The performance operator includes a keyboard device, a pedal device, and the like. When the performer operates the input operator 11, operation information indicating the operation content is supplied to the computer unit 12 described below via the bus BUS.

  The computer unit 12 includes a CPU 12a, a ROM 12b, a RAM 12c, and a timer 12d connected to the bus BUS. The CPU 12a reads various programs from the ROM 12b and executes them. For example, the CPU 12a executes a manual performance program that supplies note information (key number, key pressing strength, etc.) relating to a key pressed by the performer to a tone generator circuit 16 described later to generate a musical sound. Further, the CPU 12a executes an accompaniment reproduction program for reproducing the accompaniment part. When the CPU 12a executes the accompaniment playback program, the electronic musical instrument 10 functions as an audio playback device. The accompaniment part playback sequence will be described later. In addition to the various programs, the ROM 12b stores initial setting parameters, graphic data for generating display data representing an image displayed on the display 13, character data, and the like. Various data are temporarily stored in the RAM 12c when various programs are executed. The timer 12d includes a counter circuit that measures time. The timer 12d is used, for example, when determining the timing for supplying note information to the tone generator circuit 16, the timing for reading the sample values constituting the accompaniment data, and the like.

  The display 13 is configured by a liquid crystal display (LCD). The computer unit 12 generates display data representing contents to be displayed using graphic data, character data, and the like, and supplies the display data to the display unit 13. The display device 13 displays an image based on the display data supplied from the computer unit 12. For example, the name of the currently selected music data is displayed.

  The storage device 14 includes a large-capacity nonvolatile recording medium such as an HDD, a CD, and a DVD, and a drive unit corresponding to each recording medium. The accompaniment data is stored in the storage device 14.

  The external interface circuit 15 includes a connection terminal that allows the electronic musical instrument 10 to be connected to an external device such as another electronic musical instrument or a personal computer. The electronic musical instrument 10 can be connected to a communication network such as a LAN (Local Area Network) or the Internet via the external interface circuit 15.

  The waveform memory WM stores a plurality of timbre waveform data representing the acoustic waveforms of musical sounds such as piano, organ, violin, trumpet and the like. The tone generator circuit 16 reads the timbre waveform data designated by the CPU 12a from the waveform memory WM, generates acoustic waveform data representing the acoustic waveform of the musical sound corresponding to the note information using the read timbre waveform data, and generates a sound system. 17 is supplied. The sound source circuit 16 includes a mixing circuit that acquires acoustic waveform data from the CPU 12a, and superimposes and adds the acoustic waveform data corresponding to the note information to the sound system 17.

  The sound system 17 includes a D / A converter that converts a digital sound signal into an analog sound signal, an amplifier that amplifies the converted analog sound signal, and a left and right pair that converts the amplified analog sound signal into an acoustic signal and outputs the sound signal. The speaker is provided. Note that the sound system 17 may include an effect circuit that imparts effects such as reverb and chorus to various musical sounds.

  Next, an accompaniment part playback sequence by the electronic musical instrument 10 configured as described above will be described. In particular, the operation of the computer unit 12 when pausing reproduction of the accompaniment part and the operation of the computer unit 12 when releasing the pause and restarting reproduction of the accompaniment part will be described. When the CPU 12a executes the accompaniment playback program, as shown in FIG. 2, the computer unit 12 has a reading unit RD, a windowing unit WF, a conversion unit TR, a buffer unit BF, an estimation unit ES, an attenuation signal generation unit DG, It functions as an accompaniment playback device (audio playback device) including a playback signal generation unit SG, a switching unit SW, an inverse conversion unit IT, a resynthesis unit SY, and a playback control unit CT.

  The reading unit RD sequentially reads sample values constituting the accompaniment data from the storage device 14 and supplies them to the windowing unit WF. However, the reading unit RD temporarily stops the sample value reading process and the supply process when a pause signal ps is supplied from a reproduction control unit CT described later. Thereby, the reproduction of the accompaniment part is paused. In addition, when the reproduction start signal ss is supplied from the reproduction control unit CT, the reading unit RD restarts the sample value reading process and the supply process.

The windowing unit WF stores the sample values supplied from the reading unit RD in the order of supply. Each time a predetermined number (for example, 512) of sample values is supplied, the windowing unit WF selects the latest predetermined number (for example, 2048) of sample values. That is, the windowing unit WF cuts out part of the accompaniment data as shown in FIG. Hereinafter, the cut out portion is referred to as a frame FR. Hereinafter, when it is necessary to distinguish and explain each frame, the frame number t (= 1, 2,...) Is used and described as “frame FR _t ”. The frame number t corresponds to the accompaniment playback progress position. The cut-out position of the frame FR is sequentially shifted backward (end side of accompaniment data). Hereinafter, the period for shifting the cutout position of the frame FR is referred to as a frame period. Frame _{FR t = n} top-side portion (e.g., a portion corresponding to 3/4 of the frame _{FR t = n),} the previous frame _{FR t = n-1} at the end side portion thereof (e.g., And a portion corresponding to three quarters of the frame FR _{t = n−1} ). Each time the windowing unit WF cuts out the frame FR, it performs a windowing process on the cutout frame FR (applies a window function (for example, Hanning window)) and supplies the calculation result to the conversion unit TR. Note that the frame period and the length of one frame (that is, the number of samples constituting the frame) are not limited to the above example.

  The conversion unit TR performs a Fourier transform (FFT) on the frame FR supplied from the windowing unit WF, and supplies the result to the buffer unit BF and the reproduction signal generation unit SG. That is, the conversion unit TR calculates the amplitude X and the phase φ related to each frequency bin f and supplies them to the buffer unit BF and the reproduction signal generation unit SG (see FIG. 4). “F” is an index of the frequency bin. That is, the “frequency bin f” does not represent the frequency itself, but the “f” -th frequency counted from the frequency bin having the lowest (or highest) frequency among the plurality of frequency bins in the frequency characteristic diagram. Represents a bin. In the following explanation, when it is necessary to distinguish between the frequency bin f and the frame number t of the amplitude X and the phase φ, they are expressed as “amplitude X (f, t)” and “phase φ (f, t)”. To do. That is, “amplitude X (f, t)” and “phase (f, t)” are “f” -th counted from the low frequency side in the frequency characteristic diagram of the “t” -th frame counted from the head of the accompaniment data. Represents the magnitude and phase of the amplitude for each frequency bin.

The buffer unit BF temporarily stores the amplitude X and the phase φ supplied from the conversion unit TR (for a processing time of a predetermined number (for example, 50) frames). As will be described later, the buffer unit BF supplies the amplitude X and the phase φ to the estimation unit ES. For example, the buffer unit BF estimates the amplitude X for the last 50 frames and the phase φ for the two most recent frames in each frame (when the amplitude X and the phase φ for one frame are newly supplied). Supply to the ES. That is, when the value of the current frame number t is “n”, the buffer unit BF sets the frequency bins f of the frames FR _{t = n} , FR _{t = n−1} ,..., FR _{t = n−49.} And the phase φ related to the frames FR _{t = n} and FR _{t = n−1} are supplied to the estimation unit ES.

The estimation unit ES estimates the attenuation rate of the amplitude X with respect to each frequency bin f in each frame. Specifically, as illustrated in FIG. 5, when the value of the current frame number t is “n”, the estimation unit ES receives the frames FR _{t = n} and FR _{t = n−1} acquired from the buffer unit BF. ,..., FR _{t = n−49,} an approximate curve representing the variation of the amplitude X most likely is estimated. In FIG. 5, the frame _FR t _{= n} as observed _{values, FR t = n-1,} ···, a variation in the amplitude X of frequency bins f (= k) between _{FR t = n-49} dashed lines It shows with. In the figure, the approximate curve is indicated by a solid line. In the present embodiment, it is assumed that the aspect of attenuation of the amplitude X is exponential. Therefore, the approximate curve F (f, t) is expressed as the following equation (1) using the attenuation coefficient a (f) and the gain g (f) regarding the frequency bin f.

  The estimation unit ES calculates an attenuation coefficient a (f) and a gain g (f) using a regression analysis method. This attenuation coefficient a (f) corresponds to the attenuation rate of the amplitude X. The estimation unit ES supplies the attenuation coefficient a (f) to the attenuation signal generation unit DG, the reproduction signal generation unit SG, and the reproduction control unit CT. Further, the estimation unit ES supplies the gain g (f) to the reproduction control unit CT. In the calculation of the attenuation coefficient a (f), a constraint “a (f)> 0” may be imposed.

Further, in each frame, the estimation unit ES obtains the phase φ related to each frequency bin f of the two most recent frames from the buffer BF, and calculates the phase difference Δφ (f) between the two most recent frames. When the value of the current frame number t is “n”, the phase difference Δφ (f) is expressed as the following equation (2).

  The estimation unit ES supplies the phase difference Δφ (f) to the attenuation signal generation unit DG and the reproduction signal generation unit SG.

Based on the latest attenuation coefficient a (f) supplied from the estimation unit ES, the attenuation signal generation unit DG calculates the amplitude X ′ for each frame during the period in which the reproduction of the accompaniment part is paused, Supply to the switching unit SW. Specifically, when the pause is started when the value of the frame number t is “n”, the attenuation signal generation unit DG acquires the amplitude X (f, n) regarding each frequency bin f from the estimation unit ES. To do. Then, the attenuation signal generation unit DG uses each frequency when the elapsed time from the pause start corresponds to the time obtained by multiplying the frame period by the number of frames m (that is, the “m” th frame counted from the pause start). The amplitude X ′ (f, n + m) for the bin f is calculated based on the following equation (3).

In addition, the attenuation signal generation unit DG, based on the latest phase difference Δφ (f) supplied from the estimation unit ES, the phase φ ′ (f, f, f) for each frame during the period in which the reproduction of the accompaniment part is paused. t) is calculated and supplied to the switching unit SW. The attenuation signal generation unit DG uses each frequency bin f when the elapsed time from the pause start corresponds to a time obtained by multiplying the frame period by the number of frames m (that is, the “m” th frame counted from the pause start). The phase φ ′ (f, n + m) is calculated based on the following equation (4).

  The attenuation signal generation unit DG supplies the calculated amplitude X ′ and phase φ ′ to the switching unit SW.

The reproduction signal generator SG calculates the amplitude X ′ for each frame in the period in which the pause is cancelled. Here, consider the first frame in which the value of the frame number t is “t ₁ ” and the second frame in which the value of the frame number t is “t ₂ ” in the original waveform data (that is, accompaniment data) (FIG. 6A). The second frame is a frame after the first frame. Then, the amplitude X (f, t ₂ ) of the frequency bin f of the second frame is obtained by using the amplitude X (f, t ₁ ) of the frequency bin f of the first frame and the attenuation coefficient a (f) in the first frame. Is expressed as the following equation (5).

In Equation (5), it is assumed that the amplitude X attenuates at a speed corresponding to the attenuation coefficient a (f) from the first frame to the second frame. That is, the second term on the right side of Equation (5) corresponds to the amplitude after attenuation. “S (f, t ₂ )” is a difference between the amplitude represented by the second term on the right side and the actual amplitude (amplitude obtained by Fourier transform of a frame cut out from accompaniment data).

Next, a case where a temporary stop is started in the first frame will be considered (see the figure). It is assumed that the playback is resumed when the elapsed time from the start of the pause corresponds to the time obtained by multiplying the frame period by the number of frames m. In this case, before reaching the second frame of the original waveform data (that is, until all sample values constituting the second frame are read), the amplitude X is attenuated during the pause as compared with the case where the pause is not performed. It is greatly attenuated by the amount (see FIG. 6B). Therefore, when the reproduction of the accompaniment part is resumed, an amplitude X ′ (f, t ₂ + m) as shown in the following equation (6) may be generated.

Then, Expression (7) that does not depend on the difference s (f, t ₂ ) is derived from Expression (5) and Expression (6).

The reproduction signal generator SG calculates the amplitude X ′ (f, t ₂ + m) based on the equation (7). As will be described later, the frame number m representing the length of the paused period is supplied from the reproduction control unit CT to the reproduction signal generation unit SG. When the calculated value of the amplitude X ′ (f, t ₂ + m) is smaller than “0”, the reproduction signal generating unit SG sets the value of the amplitude X ′ (f, t ₂ + m) to “0”. The reproduction signal generator SG supplies the amplitude X ′ (f, t ₂ + m) to the switching unit SW.

  Further, the reproduction signal generation unit SG supplies the phase φ supplied from the conversion unit TR as the phase φ ′ to the switching unit SW.

  The switching unit SW reverses the amplitude X ′ and the phase φ ′ supplied from the attenuation signal generation unit DG during the pause (from when the pause signal ps is supplied to when the reproduction start signal ss is supplied). Supplied to the conversion unit IT. On the other hand, the switching unit SW displays the amplitude X ′ and the phase φ ′ supplied from the reproduction signal generation unit SG during reproduction (from the time when the reproduction start signal ss is supplied to the time when the pause signal ps is supplied). Supply to the inverse transform unit IT.

  The inverse transform unit IT performs inverse Fourier transform on the amplitude X ′ and the phase φ ′ supplied from the switching unit SW to generate acoustic waveform data (a plurality of sample values) corresponding to one frame.

The resynthesizing unit SY superimposes and adds the acoustic waveform data of the most recent predetermined number of frames sequentially supplied from the inverse converting unit IT by shifting the number of samples corresponding to the frame period. Then, of the acoustic waveform data obtained by the superposition addition, a portion where all the frames used for the superposition addition overlap is supplied to the mixing circuit of the sound source circuit 16. For example, when each frame is composed of 2048 sample values and the number of samples corresponding to the frame period is 512, the latest four frames (FR _{t = n−3} , FR _{t = n−2} , FR _{t = N−1} , FR _{t = n} ) are shifted by 512 samples and superimposed. Then, of the acoustic waveform data obtained by the superposition addition, the portion where the above four frames overlap is supplied to the mixing circuit of the sound source circuit 16. The sound source circuit 16 superimposes and adds the acoustic waveform data supplied from the re-synthesis unit SY and other acoustic waveform data (for example, acoustic waveform data representing a musical sound corresponding to note information), and supplies the resultant to the sound system 17. To do.

As described above, the reproduction control unit CT determines whether or not the manual performance by the user has been paused and resumed. When the manual performance by the user is paused, it is determined whether or not the playback progress position (current frame) of the current accompaniment part is suitable for pause. Specifically, the reproduction control unit CT calculates the estimation error ε based on the following equation (8). For example, the range of “i” is “0” to “5”. That is, the estimation error for the section from the current frame to the 5th previous frame is calculated.

  If the estimation error ε is smaller than a predetermined threshold (for example, “1.0”), it is considered that each frequency component is steady to some extent in the vicinity of the current frame. In this case, the playback control unit CT determines that the current frame (actually, the timing of the boundary between the current frame and the next frame) is suitable for pausing playback of the accompaniment part, and pauses. The signal ps is supplied to the reading unit RD, the attenuation signal generation unit DG, and the switching unit SW. The reproduction control unit CT counts up the number m of frames every time one frame period elapses. On the other hand, if the estimation error ε is equal to or greater than the predetermined threshold, it is considered that each frequency component fluctuates violently in the vicinity of the current frame. In this case, the reproduction control unit CT advances the reproduction progress position of the accompaniment part to the subsequent frame, and reads the pause signal ps in the frame where the estimation error ε is smaller than the predetermined threshold value RD, the attenuation signal generation unit Supply to DG and switching unit SW. As described above, the reproduction control unit CT determines a frame (timing) to be paused. Note that the reproduction control unit CT corresponds to the determining means of the present invention.

  When the manual performance by the user is resumed, the reproduction control unit CT ends the counting of the number of frames m. Then, the reproduction control unit CT supplies the reproduction start signal ss to the reading unit RD, the attenuation signal generation unit DG, and the switching unit SW, and supplies the number m of frames to the reproduction signal generation unit SG.

  As described above, the electronic musical instrument 10 attenuates the musical sound that starts to be sounded immediately before the paused part during the pause period of the accompaniment part. As a result, it is possible to simulate the sound generation mode when the performance of the acoustic instrument (the instrument that emits the attenuated sound) is paused.

  Also, the start of the pause is postponed until a playback progress position suitable for the pause position is detected. That is, the accompaniment part is not temporarily stopped and the decay sound starts to be generated at the moment when it is determined that the manual performance by the user is paused, but the accompaniment part is paused and the decay sound is generated with a slight delay. Sometimes it starts. According to this, the accompaniment part is paused at a more appropriate position, and an attenuation sound starts to be generated. In other words, for example, a pause is started at a portion where each frequency component fluctuates violently and the approximate curve (formula (1)) is difficult to determine, such as a sound rising portion (attack). Can be prevented.

  Also, at the time of resuming playback, the portion of the accompaniment data corresponding to the section immediately after the position where the playback of the accompaniment part is paused is the musical sound whose sound is started in the section immediately before the paused position. It is corrected to represent the attenuated state. In the case where the pause period is relatively long and the musical sound that has started to sound immediately before the paused position is completely attenuated, when the accompaniment part is resumed, the accompaniment part of the accompaniment data is reproduced. The decay component of the musical tone that starts sounding in the section immediately before the paused position is deleted from the portion corresponding to the section immediately after the paused position. According to this, it is possible to prevent the attenuation component generated during the pause period from being reproduced even when the accompaniment part is resumed. Therefore, the sound generation mode at the time of resuming reproduction can be felt naturally.

  Furthermore, in carrying out the present invention, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the object of the present invention.

  For example, a switch for temporarily stopping the reproduction of the accompaniment part may be provided as one of the input operators 11. In this case, the playback control unit CT may detect that the switch has been pressed during playback of the accompaniment part, and supply the pause signal ps to the reading unit RD, the switching unit SW, and the like. Further, the reproduction control unit CT may detect that the switch has been pressed while the accompaniment part is paused, and supply the reproduction start signal ss to the reading unit RD, the switching unit SW, and the like.

  The electronic musical instrument 10 may store audio data representing the entire music (that is, performance of all performance parts including the main melody part), and may reproduce the music based on the audio data. Also in this case, similarly to the above embodiment, the electronic musical instrument 10 only needs to pause and resume playback of the music.

In the above embodiment, the buffer unit BF stores the amplitude X and the phase φ related to the most recent predetermined number of frames (that is, the current frame and the past several frames). However, the amplitude X and the phase φ related to several frames after the frame currently being reproduced (the frame for generating the acoustic waveform data and supplying it to the sound source circuit 16) are calculated in advance and temporarily stored in the buffer unit BF. You may remember. For example, the reading unit RD reads in advance from the frame being played back to five frames ahead, and the amplitude X and the phase φ obtained by processing them in the windowing unit WF and the conversion unit TR are stored in the buffer unit BF. It is good to memorize temporarily. In this case, the amplitude X and the phase φ related to the frames FR _{t = n + 5} , FR _{t = n + 4} ,..., FR _{t = n} ,..., FR _{t = n−44} may be stored in the buffer unit BF. Then, the reproduction control unit CT determines whether or not the frame being reproduced is suitable for pausing based on the amplitude X relating to a predetermined number of frames positioned before and after the frame FR _{t = n} being reproduced. May be. That is, in the above formula (8), the range of “i” may be set to “−5” to “+5”. Even in this case, the calculation method of the attenuation coefficient a (f), the gain g (f), and the phase difference Δφ (f) by the estimation unit ES is the same as that in the above embodiment. That is, the estimation unit ES calculates the attenuation coefficient a (f), the gain g (f), and the phase difference Δφ (f) using the amplitude X and the phase φ related to the frame before the frame currently being reproduced.

  In the above embodiment, the phase difference Δφ between the two most recent frames is calculated, and the phase of each frame after being temporarily stopped is estimated using this phase difference Δφ. However, instead of this, the phase of each frame after the pause may be estimated based on the phase transition between frames that are before the pause timing and more than the above embodiment. . In addition, when the playback control unit CT determines the frame (timing) to be paused, the pause is postponed if the phase advance speed changes drastically in the interval composed of the most recent frames. May be.

In the above embodiment, the reproduction control unit CT outputs the pause signal ps when the estimation error ε is smaller than a predetermined threshold. However, instead of this, when the estimation error ε is larger than the value obtained by multiplying the power of the section in which the estimation error ε is evaluated by a predetermined constant (for example, 0.01) (see the following formula (9)), A stop signal ps may be output.

  In the above embodiment, an audio signal (accompaniment data) to be processed is stored in the storage device 14. However, the audio signal to be processed may be sequentially supplied from the external device to the reading unit RD via the external interface circuit 15. In this case, the reproduction control unit CT may output the pause signal ps and the reproduction start signal ss based on the operation state of the readout circuit RD or the external interface circuit 15. For example, the playback control unit CT may detect that the supply of the audio signal is interrupted due to a communication failure and output the pause signal ps. Then, the reproduction control unit CT may detect that the supply of the audio signal has been resumed and output the reproduction start signal ss.

  In the above-described embodiment, the amplitude X and the phase φ are calculated by performing a windowing process on a frame including a plurality of read sample values and performing Fourier transform. However, a so-called filter bank analysis may be used instead. For example, a filter bank may be configured using a plurality of biquadratic filters, and the amplitude X and phase φ may be acquired for each band corresponding to each filter. Also, constant Q filter bank analysis and wavelet analysis may be used. Further, in the above-described embodiment, the audio signal for reproduction is synthesized by performing inverse Fourier transform on the amplitude X and the phase φ output from the switching unit SW and superimposing the results. However, instead of this, an audio signal for reproduction may be synthesized by recombination of the oscillator bank.

  The present invention is not limited to an electronic musical instrument, and can be applied to any device as long as it reproduces an acoustic signal. For example, in an apparatus (streaming reproduction apparatus) that sequentially receives and reproduces music data distributed via the Internet, when the music data distribution is interrupted, when the music data reproduction is paused, the attenuation sound May be reproduced by estimating. Further, when the reproduction of music data is resumed, the music data may be corrected so that the attenuated sound reproduced during the pause period is not reproduced again. Further, the present invention may be used for a part of a function of changing the length of music data including a drum part to a predetermined time length in a digital audio workstation (DAW), a DJ device, or the like. Further, the present invention may be used for a part of the function of changing the cycle when the waveform data is repeatedly reproduced.

DESCRIPTION OF SYMBOLS 10 ... Electronic musical instrument, 11 ... Input operation element, 12 ... Computer part, 13 ... Display, 14 ... Memory | storage device, 15 ... External interface circuit, 16 ... Sound source circuit , 17 ... Sound system, a ... Attenuation coefficient, BF ... Buffer unit, BUS ... Bus, CT ... Reproduction control unit, DG ... Attenuation signal generation unit, ES ... Estimation Part, f ... frequency bin, FR ... frame, g ... gain, IT ... inverse conversion part, m ... number of frames, ps ... pause signal, RD ... reading part , S ... difference, SG ... reproduction signal generation unit, ss ... reproduction start signal, SW ... switching unit, SY ... recomposition unit, t ... frame number, TR ... Conversion unit, WF ... Windowing unit, WM ... Waveform memory, X ... Amplitude, Δφ ... Phase difference, ε ... estimation error, φ ... phase

Claims (4)

Reproduction means for reproducing music based on an audio signal representing an acoustic waveform of music, wherein the reproduction means is capable of pausing reproduction of music based on the audio signal at any timing of the audio signal;
Estimating means for estimating attenuation characteristics of each frequency component of the audio signal based on a change in frequency characteristics in a section before the music pause timing;
The audio reproduction apparatus includes an attenuation sound reproduction unit that reproduces an attenuation sound representing a state in which each frequency component is attenuated in a section after the temporary stop timing based on the attenuation characteristic. The audio playback device according to claim 1,
When the playback unit releases the pause and resumes playing the music, the playback unit corrects a section after the pause timing in the audio signal according to the attenuation characteristic and the length of the attenuated sound. An audio playback device having correcting means for The audio playback device according to claim 1 or 2,
An audio reproducing apparatus comprising: a determination unit that determines the pause timing based on the estimation error of the attenuation characteristic. Computer
Reproduction means for reproducing music based on an audio signal representing an acoustic waveform of music, wherein the reproduction means is capable of pausing reproduction of music based on the audio signal at any timing of the audio signal;
Estimating means for estimating attenuation characteristics of each frequency component of the audio signal based on a change in frequency characteristics in a section before the music pause timing;
The reproduction means comprises: an attenuation sound reproduction means for reproducing an attenuation sound representing a state in which each frequency component is attenuated in a section after the temporary stop timing based on the attenuation characteristic. As a computer program.

Images