WO2008069594A1 - A method and an apparatus for processing an audio signal - Google Patents

A method and an apparatus for processing an audio signal Download PDF

Info

Publication number
WO2008069594A1
WO2008069594A1 PCT/KR2007/006316 KR2007006316W WO2008069594A1 WO 2008069594 A1 WO2008069594 A1 WO 2008069594A1 KR 2007006316 W KR2007006316 W KR 2007006316W WO 2008069594 A1 WO2008069594 A1 WO 2008069594A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
signal
downmix
channel
processing
Prior art date
Application number
PCT/KR2007/006316
Other languages
French (fr)
Inventor
Hyen O Oh
Yang Won Jung
Original Assignee
Lg Electronics Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lg Electronics Inc. filed Critical Lg Electronics Inc.
Priority to EP07851287A priority Critical patent/EP2102856A4/en
Priority to KR1020097014213A priority patent/KR101100222B1/en
Priority to JP2009540164A priority patent/JP5450085B2/en
Priority to CN2007800453353A priority patent/CN101553865B/en
Publication of WO2008069594A1 publication Critical patent/WO2008069594A1/en

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/18—Vocoders using multiple modes
    • G10L19/20—Vocoders using multiple modes using sound class specific coding, hybrid encoders or object based coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/008—Systems employing more than two channels, e.g. quadraphonic in which the audio signals are in digital form, i.e. employing more than two discrete digital channels
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/03—Application of parametric coding in stereophonic audio systems

Definitions

  • the present invention relates to a method and an apparatus for processing an audio signal, and more particularly, to a method and an apparatus for decoding an audio signal received on a digital medium, as a broadcast signal, and so on.
  • the present invention is directed to a method and an apparatus :or processing an audio signal that substantially obviates one or more problems due to limitations and disadvantages of the related art.
  • An object of the present invention is to provide a method and an apparatus for processing an audio signal to control object gain and panning unrestrictedly.
  • Another object of the present invention is to provide a method and an apparatus for processing an audio signal to control object gain and panning based on user selection.
  • the present invention provides the following effects or advantages. First of all, the present invention is able to provide a method and an pparatus for processing an audio signal to control object gain and panning .nrestrictedly.
  • the present invention is able to provide a method and an pparatus for processing an audio signal to control object gain and panning based >n user selection.
  • FIG. 1 is an exemplary block diagram to explain to basic concept of rendering a downmix signal based on playback configuration and user control.
  • FIG. 2 is an exemplary block diagram of an apparatus for processing an audio signal according to one embodiment of the present invention corresponding to the first scheme.
  • FIG. 3 is an exemplary block diagram of an apparatus for processing an audio signal according to another embodiment of the present invention corresponding to the first scheme.
  • FIG. 4 is an exemplary block diagram of an apparatus for processing an udio signal according to one embodiment of present invention corresponding to tie second scheme.
  • FIG. 5 is an exemplary block diagram of an apparatus for processing an udio signal according to another embodiment of present invention corresponding o the second scheme.
  • FIG. 6 is an exemplary block diagram of an apparatus for processing an iudio signal according to the other embodiment of present invention orresponding to the second scheme.
  • FIG. 7 is an exemplary block diagram of an apparatus for processing an Ludio signal according to one embodiment of the present invention corresponding o the third scheme.
  • FIG. 8 is an exemplary block diagram of an apparatus for processing an iudio signal according to another embodiment of the present invention :orresponding to the third scheme.
  • FIG. 9 is an exemplary block diagram to explain to basic concept of • endering unit.
  • FIGS. 1OA to 1OC are exemplary block diagrams of a first embodiment of a lownmix processing unit illustrated in FIG. 7.
  • FIG. 11 is an exemplary block diagram of a second embodiment of a iownmix processing unit illustrated in FIG. 7.
  • FIG. 12 is an exemplary block diagram of a third embodiment of a downmix processing unit illustrated in FIG. 7.
  • FIG. 13 is an exemplary block diagram of a fourth embodiment of a iownmix processing unit illustrated in FIG. 7.
  • FIG. 14 is an exemplary block diagram of a bitstream structure of a :ompressed audio signal according to a second embodiment of present invention.
  • FIG. 15 is an exemplary block diagram of an apparatus for processing an audio signal according to a second embodiment of present invention.
  • FIG. 16 is an exemplary block diagram of a bitstream structure of a compressed audio signal according to a third embodiment of present invention.
  • FIG. 17 is an exemplary block diagram of an apparatus for processing an audio signal according to a fourth embodiment of present invention.
  • FIG. 18 is an exemplary block diagram to explain transmitting scheme for variable type of object.
  • FIG. 19 is an exemplary block diagram to an apparatus for processing an audio signal according to a fifth embodiment of present invention.
  • a method for processing an audio signal comprising: receiving a downmix signal in time iomain; if the downmix signal corresponds to a mono signal, bypassing the iownmix signal; if the number of channel of the downmix signal corresponds to at [east two, decomposing the downmix signal into a subband signal, and processing the subband signal using a downmix processing information, wherein the downmix processing information is estimated based on an object information and a mix information.
  • the number of channel of the downmix signal is equal to the number of channel of the processed downmix signal.
  • the object information is included in a side information
  • the side information includes a correlation flag information indicating whether an object is part of at least two channel object.
  • the object information includes at least one of an object level information and an object correlation information.
  • the downmix processing information corresponds to an information for controlling object panning if the number of channel the downmix signal corresponds to at least two.
  • the downmix processing tformation corresponds to an information for controlling object gain.
  • the present invention further comprising, generating a multi- hannel information using the object information and the mix information, wherein he multi-channel signal is generated based on the multi-channel information.
  • the present invention further comprising, downmixing the lownmix signal to be a mono signal if the downmix signal corresponds to a stereo ignal.
  • the mix information is generated using at least one of an object position information and a playback :onfiguration information.
  • the downmix signal is received is a broadcast signal.
  • the downmix signal is received 3n a digital medium.
  • a computer-readable medium having instructions stored thereon, which, when executed by a processor, causes the processor to perform operations, comprising: receiving a downmix signal in ime domain; if the downmix signal corresponds to a mono signal, bypassing the lownmix signal; if the number of channel of the downmix signal corresponds to at east two, decomposing the downmix signal into a subband signal, and processing he subband signal using a downmix processing information, wherein the lownmix processing information is estimated based on an object information and a nix information.
  • an apparatus for processing an iudio signal comprising: a receiving unit receiving a downmix signal in time ⁇ omain; and, a downmix processing unit bypassing the downmix signal if the lownmix signal corresponds to a mono signal, and decomposing the downmix dgnal into a subband signal and processing the subband signal using a downmix processing information if the number of channel of the downmix signal :orresponds to at least two, wherein the downmix processing information is estimated based on an object information and a mix information.
  • 'parameter' in the following description means information tcluding values, parameters of narrow sense, coefficients, elements, and so on. Iereinafter 'parameter' term will be used instead of 'information' term like an bject parameter, a mix parameter, a downmix processing parameter, and so on, ⁇ ch does not put limitation on the present invention.
  • an object parameter nd a spatial parameter can be extracted.
  • a decoder can generate output signal Lsing a downmix signal and the object parameter (or the spatial parameter).
  • the iutput signal may be rendered based on playback configuration and user control y the decoder. The rendering process shall be explained in details with reference ) the FIG. 1 as follow.
  • FIG. 1 is an exemplary diagram to explain to basic concept of rendering ownmix based on playback configuration and user control.
  • a ecoder 100 may include a rendering information generating unit 110 and a endering unit 120, and also may include a Tenderer 110a and a synthesis 120a astead of the rendering information generating unit 110 and the rendering unit 120.
  • a rendering information generating unit 110 can be configured to receive a ide information including an object parameter or a spatial parameter from an ⁇ ncoder, and also to receive a playback configuration or a user control from a levice setting or a user interface.
  • the object parameter may correspond to a jarameter extracted in downmixing at least one object signal
  • the spatial >arameter may correspond to a parameter extracted in downmixing at least one :hannel signal.
  • type information and characteristic information for ⁇ ach object may be included in the side information.
  • Type information and iharacteristic information may describe instrument name, player name, and so on.
  • the playback configuration may include speaker position and ambient information ⁇ speaker's virtual position), and the user control may correspond to a control information inputted by a user in order to control object positions and object gains, and also may correspond to a control information in order to the playback mfiguration. Meanwhile the payback configuration and user control can be ⁇ presented as a mix information, which does not put limitation on the present Lvention.
  • a rendering information generating unit 110 can be configured to generate a ⁇ ndering information using a mix information (the playback configuration and ser control) and the received side information.
  • a rendering unit 120 can jnfigured to generate a multi-channel parameter using the rendering information i case that the downmix of an audio signal (abbreviated 'downmix signal') is not ⁇ ansmitted, and generate multi-channel signals using the rendering information nd downmix in case that the downmix of an audio signal is transmitted.
  • a renderer 110a can be configured to generate multi-channel signals using a iix information (the playback configuration and the user control) and the received ide information.
  • a synthesis 120a can be configured to synthesis the multi-channel ignals using the multi-channel signals generated by the renderer 110a.
  • the decoder may render the downmix signal based on •layback configuration and user control. Meanwhile, in order to control the ndividual object signals, a decoder can receive an object parameter as a side nformation and control object panning and object gain based on the transmitted )bject parameter.
  • Controlling gain and panning of object signals may be >rovided.
  • a decoder receives an object parameter and generates the individual object signals using the object parameter, then, can control he individual object signals base on a mix information (the playback configuration, he object level, etc.)
  • the multi-channel decoder can upmix a lownmix signal received from an encoder using the multi-channel parameter.
  • the Lbove-mention second method may be classified into three types of scheme. In ⁇ articular, 1) using a conventional multi-channel decoder, 2) modifying a multi- :hannel decoder, 3) processing downmix of audio signals before being inputted to i multi-channel decoder may be provided.
  • the conventional multi-channel decoder nay correspond to a channel-oriented spatial audio coding (ex: MPEG Surround ⁇ ecoder), which does not put limitation on the present invention. Details of three ypes of scheme shall be explained as follow.
  • First scheme may use a conventional multi-channel decoder as it is without nodifying a multi-channel decoder.
  • ADG arbitrary lownmix gain
  • 5-2-5 :onfiguration for controlling object panning
  • FIG. 2 is an exemplary block diagram of an apparatus for processing an udio signal according to one embodiment of the present invention corresponding D first scheme.
  • an apparatus for processing an audio signal 200 hereinafter simply 'a decoder 200' may include an information generating unit .10 and a multi-channel decoder 230.
  • the information generating unit 210 may eceive a side information including an object parameter from an encoder and a nix information from a user interface, and may generate a multi-channel >arameter including a arbitrary downmix gain or a gain modification ;ain(hereinafter simple 'ADG').
  • the ADG may describe a ratio of a first gain :stimated based on the mix information and the obejct information over a second ⁇ ain extimated based on the object information.
  • the information generating unit 210 may generate the ADG only if the downmix signal corresponds :o a mono signal.
  • the multi-channel decoder 230 may receive a downmix of an iudio signal from an encoder and a multi-channel parameter from the information generating unit 210, and may generate a multi-channel output using the downmix signal and the multi-channel parameter.
  • the multi-channel parameter may include a channel level difference hereinafter abbreviated 'CLD'), an inter channel correlation (hereinafter abbreviated 'ICC), a channel prediction coefficient (hereinafter abbreviated 'CPC).
  • CLD Since CLD, ICC 7 and CPC describe intensity difference or correlation between two channels, and is to control object panning and correlation. It is able to :ontrol object positions and object diffuseness (sonority) using the CLD, the ICC, ⁇ tc. Meanwhile, the CLD describe the relative level difference instead of the absolute level, and energy of the splitted two channels is conserved. Therefore it is enable to control object gains by handling CLD, etc. In other words, specific object :annot be mute or volume up by using the CLD, etc.
  • the ADG describes time and frequency dependent gain for :ontrolling correction factor by a user. If this correction factor be applied, it is able to handle modification of down-mix signal prior to a multi-channel upmixing. Therefore, in case that ADG parameter is received from the information generating unit 210, the multi-channel decoder 230 can control object gains of specific time and frequency using the ADG parameter.
  • W12 and W 21 iay be a cross-talk component (in other words, cross-term).
  • the above-mentioned case corresponds to 2-2-2 configuration, which means -channel input, 2-channel transmission, and 2-channel output.
  • 5-2-5 configuration (2-channel input, 5-channel ransmission, and 2 channel output) of conventional channel-oriented spatial audio oding (ex: MPEG surround) can be used.
  • certain channel among 5 output channels of 5-2-5 configuration an be set to a disable channel (a fake channel).
  • the above-mentioned CLD and CPC • nay be adjusted.
  • gain factor g x in the formula 1 is obtained using the above mentioned ADG, and weighting factor wn ⁇ W22 in the formula 1 is obtained using CLD and CPC.
  • default mode of conventional spatial audio coding may be applied. Since characteristic of default CLD is supposed to output 2-channel, it is ible to reduce computing amount if the default CLD is applied. Particularly, since :here is no need to synthesis a fake channel, it is able to reduce computing amount argely. Therefore, applying the default mode is proper. In particular, only default 2LD of 3 CLDs (corresponding to 0, 1, and 2 in MPEG surround standard) is used or decoding.
  • CLDs corresponding 3 and 5 describe channel level iifference between left channel plus right channel and center channel ((l+r)/c) is proper to set to 15OdB (approximately infinite) in order to mute center channel, ⁇ nd, in order to implement cross-talk, energy based up-mix or prediction based ip-mix may be performed, which is invoked in case that TTT mode "bsTttModeLow' in the MPEG surround standard) corresponds to energy-based node (with subtraction, matrix compatibility enabled) (3 rd mode), or prediction node (1 st mode or 2 nd mode).
  • FIG. 3 is an exemplary block diagram of an apparatus for processing an audio signal according to another embodiment of the present invention orresponding to first scheme.
  • an apparatus for processing an iudio signal according to another embodiment of the present invention 300 hereinafter simply a decoder 300 may include a information generating unit 310, a cene rendering unit 320, a multi-channel decoder 330, and a scene remixing unit •50.
  • the information generating unit 310 can be configured to receive a side nformation including an object parameter from an encoder if the downmix signal :orresponds to mono channel signal (i.e., the number of downmix channel is 'V), nay receive a mix information from a user interface, and may generate a multi- :hannel parameter using the side information and the mix information.
  • the lumber of downmix channel can be estimated based on a flag information ncluded in the side information as well as the downmix signal itself and user ielection.
  • the information generating unit 310 may have the same configuration of he former information generating unit 210.
  • the multi-channel parameter is nputted to the multi-channel decoder 330, the multi-channel decoder 330 may lave the same configuration of the former multi-channel decoder 230.
  • the scene rendering unit 320 can be configured to receive a side information ncluding an object parameter from and encoder if the downmix signal corresponds :o non-mono channel signal (i.e., the number of downmix channel is more than '2'), nay receive a mix information from a user interface, and may generate a remixing iarameter using the side information and the mix information.
  • the remixing iarameter corresponds to a parameter in order to remix a stereo channel and ;enerate more than 2-channel outputs.
  • the remixing parameter is inputted to the cene remixing unit 350.
  • the scene remixing unit 350 can be configured to remix he downmix signal using the remixing parameter if the downmix signal is more han 2-channel signal.
  • Second scheme may modify a conventional multi-channel decoder.
  • a :ase of using virtual output for controlling object gains and a case of modifying a levice setting for controlling object panning shall be explained with reference to ?IG. 4 as follow.
  • a case of Performing TBT(2x2) functionality in a ⁇ iulti-channel decoder shall be explained with reference to FIG. 5.
  • FIG. 4 is an exemplary block diagram of an apparatus for processing an audio signal according to one embodiment of present invention corresponding to the second scheme.
  • an apparatus for processing an audio signal according to one embodiment of present invention corresponding to the second scheme 400 may include an information generating unit 410, an internal multi-channel synthesis 420, and an output lapping unit 430.
  • the internal multi-channel synthesis 420 and the output ⁇ apping unit 430 may be included in a synthesis unit.
  • the information generating unit 410 can be configured to receive a side information including an object parameter from an encoder, and a mix parameter rom a user interface. And the information generating unit 410 can be configured to generate a multi-channel parameter and a device setting information using the side nf ormation and the mix information.
  • the multi-channel parameter may have the ⁇ ame configuration of the former multi-channel parameter. So, details of the multi- :hannel parameter shall be omitted in the following description.
  • the device setting nformation may correspond to parameterized HRTF for binaural processing, /vhich shall be explained in the description of '1.2.2 Using a device setting nformation'.
  • the internal multi-channel synthesis 420 can be configured to receive a multi-channel parameter and a device setting information from the parameter generation unit 410 and downmix signal from an encoder.
  • the internal multichannel synthesis 420 can be configured to generate a temporal multi-channel output including a virtual output, which shall be explained in the description of '1.2.1 Using a virtual output'.
  • multi-channel parameter (ex: CLD) can control object panning, it is ird to control object gain as well as object panning by a conventional multi- iannel decoder.
  • the decoder 400 may map relative energy of object to a virtual channel 'x: center channel).
  • the relative energy of object corresponds to energy to be ⁇ duced.
  • the decoder 400 may map tore than 99.9% of object energy to a virtual channel.
  • the decoder 400 especially, the output mapping unit 430 does not output the virtual channel to /hich the rest energy of object is mapped. In conclusion, if more than 99.9% of bject is mapped to a virtual channel which is not outputted, the desired object can >e almost mute.
  • the decoder 400 can adjust a device setting information in order to control >bject panning and object gain.
  • the decoder can be configured to generate a parameterized HRTF for binaural processing in MPEG Surround standard.
  • the parameterized HRTF can be variable according to device setting. It is able to assume that object signals can be controlled according to the following formula 2. [formula 2]
  • Rnew bl * ⁇ bjl + bl * ⁇ bJ2 + b3 * ⁇ bJ3 + •• + bn * ⁇ bj n/
  • objk is object signals
  • L ne w and R ne w is a desired stereo signal
  • ak md bk are coefficients for object control.
  • An object information of the object signals objk may be estimated from an )bject parameter included in the transmitted side information.
  • the coefficients ak, >k which are defined according to object gain and object panning may be estimated rom the mix information.
  • the desired object gain and object panning can be idjusted using the coefficients ak, bk.
  • the coefficients ak, bk can be set to correspond to HRTF parameter for ⁇ naural processing, which shall be explained in details as follow.
  • FIG. 5 is an exemplary block diagram of an apparatus for processing an mdio signal according to another embodiment of present invention corresponding :o the second scheme.
  • FIG. 5 is an exemplary block diagram of TBT functionality in i multi-channel decoder.
  • a TBT module 510 can be configured :o receive input signals and a TBT control information, and generate output signals.
  • the TBT module 510 may be included in the decoder 200 of the FIG. 2 (or in particular, the multi-channel decoder 230).
  • the multi-channel decoder 230 may be implemented according to the MPEG Surround standard, which does not put Limitation on the present invention, [formula 9] where x is input channels, y is output channels, and w is weight.
  • the output yi may correspond to a combination input xi of the downmix multiplied by a first gain Wi 1 and input X2 multiplied by a second gain W12.
  • the TBT control information inputted in the TBT module 510 includes elements which can compose the weight w (wu, Wi 2 , Wa 1 , W22).
  • OTT(One-To-Two) module and TTT(Two-To- Three) module is not proper to remix input signal although OTT module and TTT module can upmix the input signal.
  • TBT (2x2) module 510 (hereinafter bbreviated 'TBT module 510') may be provided.
  • the TBT module 510 may can be igured to receive a stereo signal and output the remixed stereo signal.
  • the weight v may be composed using CLD (s) and ICC(s).
  • the lecoder may control object gain as well as object panning using the received veight term.
  • variable scheme may be provided. ⁇ t first, a TBT control information includes cross term like the W12 and W21. jecondly, a TBT control information does not include the cross term like the W 12 md W21. Thirdly, the number of the term as a TBT control information varies idaptively.
  • the terms which lumber is NxM may be transmitted as TBT control information.
  • the terms can be quantized based on a CLD parameter quantization table introduced in a MPEG Surround, which does not put limitation on the present invention.
  • the number of the TBT control information varies adaptively ccording to need of cross term in order to reduce the bit rate of a TBT control iformation.
  • a flag information / cross_flag' indicating whether the cross term is 'resent or not is set to be transmitted as a TBT control information. Meaning of the Lag information 'cross_flag' is shown in the following table 1. [table 1] meaning of cross_flag
  • the TBT control information does not nclude the cross term, only the non-cross term like the wn and W22 is present. Dtherwise ('cross_flag' is equal to 1), the TBT control information includes the cross erm.
  • flag information / reverse_flag / indicating whether cross term is present or non-cross term is present is set to be transmitted as a TBT control information.
  • Meaning of flag information / reverse_flag' is shown in the following :able 2. [table 2] meaning of reverse_flag
  • the TBT control information does not nclude the cross term, only the non-cross term like the W 11 and W22 is present. )therwise ('reverse_flag' is equal to 1), the TBT control information includes only he cross term.
  • Futhermore a flag information 'side_flag' indicating whether cross term is •resent and non-cross is present is set to be transmitted as a TBT control nformation. Meaning of flag information / side_flag / is shown in the following table
  • FIG. 6 is an exemplary block diagram of an apparatus for processing an udio signal according to the other embodiment of present invention orresponding to the second scheme.
  • an apparatus for processing an udio signal 630 shown in the FIG. 6 may correspond to a binaural decoder ncluded in the multi-channel decoder 230 of FIG. 2 or the synthesis unit of FIG. 4, vhich does not put limitation on the present invention.
  • An apparatus for processing an audio signal 630 may include a QMF analysis 632, a parameter conversion 634, a spatial synthesis 636, and a QMF synthesis 638.
  • Elements of the binaural decoder 330 may have the same configuration of MPEG Surround binaural decoder in VlPEG Surround standard.
  • the spatial synthesis 636 can be configured to consist of 1 2x2 (filter) matrix, according to the following formula 10: [formula 10] with yo being the QMF-domain input channels and ye, being the binaural >utput channels, k represents the hybrid QMF channel index, and i is the HRTF ilter tap index, and n is the QMF slot index.
  • the binaural decoder 630 can be :onfigured to perform the above-mentioned functionality described in subclause 1.2.2 Using a device setting information'. However, the elements hi j may be generated using a multi-channel parameter and a mix information instead of a nulti-channel parameter and HRTF parameter. In this case, the binaural decoder )00 can perform the functionality of the TBT module 510 in the FIG. 5. Details of the elements of the binaural decoder 630 shall be omitted.
  • the binaural decoder 630 can be operated according to a flag information binaural_flag'. In particular, the binaural decoder 630 can be skipped in case that a lag information binaural_flag is '0', otherwise (the binaural_flag is 'V), the binaural decoder 630 can be operated as below.
  • the first scheme of using a conventional multi-channel decoder have been xplained in subclause in '1.V
  • the second scheme of modifying a multi-channel ecoder have been explained in subclause in '1.2'.
  • the third scheme of processing ownmix of audio signals before being inputted to a multi-channel decoder shall e explained as follow.
  • FIG. 7 is an exemplary block diagram of an apparatus for processing an udio signal according to one embodiment of the present invention corresponding o the third scheme.
  • FIG. 8 is an exemplary block diagram of an apparatus for >rocessing an audio signal according to another embodiment of the present nvention corresponding to the third scheme.
  • an Lpparatus for processing an audio signal 700 (hereinafter simply 'a decoder 700') nay include an information generating unit 710, a downmix processing unit 720, md a multi-channel decoder 730.
  • an apparatus for processing m audio signal 800 may include an information generating unit 810 and a multi-channel synthesis unit 840 having a nulti-channel decoder 830.
  • the decoder 800 may be another aspect of the decoder 700.
  • the information generating unit 810 has the same configuration of the information generating unit 710
  • the multi-channel decoder 830 has the same >nfiguration of the multi-channel decoder 730
  • the multi-channel synthesis lit 840 may has the same configuration of the downmix processing unit 720 and Lulti-channel unit 730. Therefore, elements of the decoder 700 shall be explained in stails, but details of elements of the decoder 800 shall be omitted.
  • the information generating unit 710 can be configured to receive a side if ormation including an object parameter from an encoder and a mix information om an user-interface, and to generate a multi-channel parameter to be outputted ) the multi-channel decoder 730. From this point of view, the information enerating unit 710 has the same configuration of the former information enerating unit 210 of FIG. 2.
  • the downmix processing parameter may correspond > a parameter for controlling object gain and object panning. For example, it is able D change either the object position or the object gain in case that the object signal is Dcated at both left channel and right channel.
  • he downmix processing unit 720 can be a TBT module (2x2 matrix operation).
  • the information generating unit 710 can be configured to generate ADG lescribed with reference to FIG 2. in order to control object gain, the downmix )rocessing parameter may include parameter for controlling object panning but )bject gain.
  • the information generating unit 710 can be configured to ceive HRTF information from HRTF database, and to generate an extra multi- annel parameter including a HRTF parameter to be inputted to the multi-channel :coder 730.
  • the information generating unit 710 may generate multi- iannel parameter and extra multi-channel parameter in the same subband domain id transmit in syncronization with each other to the multi-channel decoder 730.
  • extra multi-channel parameter including the HRTF parameter shall be cplained in details in subclause '3. Processing Binaural Mode'.
  • the downmix processing unit 720 can be configured to receive downmix of i audio signal from an encoder and the downmix processing parameter from the [formation generating unit 710, and to decompose a subband domain signal using ibband analysis filter bank.
  • the downmix processing unit 720 can be configured > generate the processed downmix signal using the downmix signal and the ownmix processing parameter. In these processing, it is able to pre-process the ownmix signal in order to control object panning and object gain.
  • the processed ownmix signal may be inputted to the multi-channel decoder 730 to be upmixed.
  • the processed downmix signal may be outputted and laybacked via speaker as well.
  • the downmix processing unit 720 may perform synthesis filterbank using he prepossed subband domain signal and output a time-domain PCM signal. It is ble to select whether to directly output as PCM signal or input to the multi- hannel decoder by user selection.
  • the multi-channel decoder 730 can be configured to generate multi-channel utput signal using the processed downmix and the multi-channel parameter.
  • the aulti-channel decoder 730 may introduce a delay when the processed downmix ignal and the multi-channel parameter are inputted in the multi-channel decoder '30.
  • the processed downmix signal can be synthesized in frequency domain (ex: 2MF domain, hybrid QMF domain, etc), and the multi-channel parameter can be ynthesized in time domain.
  • delay and lynchronization for connecting HE-AAC is introduced. Therefore, the multichannel decoder 730 may introduce the delay according to MPEG Surround standard.
  • downmix processing unit 720 shall be explained in detail with reference to FIG. 9 ⁇ FIG. 13.
  • FIG. 9 is an exemplary block diagram to explain to basic concept of rendering unit.
  • a rendering module 900 can be configured to generate M output signals using N input signals, a playback configuration, and a user control.
  • the N input signals may correspond to either object signals or channel signals.
  • the N input signals may correspond to either object arameter or multi-channel parameter.
  • Configuration of the rendering module 900 in be implemented in one of downmix processing unit 720 of FIG. 7, the former ⁇ ndering unit 120 of FIG. 1, and the former renderer 110a of FIG. 1, which does not ut limitation on the present invention.
  • the rendering module 900 can be configured to directly generate M hannel signals using N object signals without summing individual object signals orresponding certain channel, the configuration of the rendering module 900 can e represented the following formula 11.
  • Ci is a i* channel signal
  • Oj is j* input signal
  • Rji is a matrix mapping j 1 * 1 nput signal to i* channel.
  • R matrix is separated into energy component E and de-correlation zomponent, the formula 11 may be represented as follow.
  • ⁇ j _i is gain portion mapped to j th channel
  • ⁇ k_i is gain portion mapped to k th :hannel
  • ⁇ is diffuseness level
  • D(o;) is de-correlated output.
  • weight values for all inputs mapped to certain channel are estimated cording to the above-stated method, it is able to obtain weight values for each iannel by the following method.
  • the dominant channel pair may correspond to left channel and center channel in case that certain input is positioned at point between left and center.
  • downmix processing unit includes a mixing part orresponding to 2x4 matrix
  • FIGS. 1OA to 1OC are exemplary block diagrams of a first embodiment of a lownmix processing unit illustrated in FIG. 7.
  • a first mbodiment of a downmix processing unit 720a (hereinafter simply 'a downmix >rocessing unit 720a') may be implementation of rendering module 900.
  • a downmix processing unit 720a can be configured to ypass input signal in case of mono input signal (m), and to process input signal in ise of stereo input signal (L, R).
  • the downmix processing unit 720a may include a e-correlating part 722a and a mixing part 724a.
  • the de-correlating part 722a has a e-correlator aD and de-correlator bD which can be configured to de-correlate iput signal.
  • the de-correlating part 722a may correspond to a 2x2 matrix.
  • the lixing part 724a can be configured to map input signal and the de-correlated ignal to each channel.
  • the mixing part 724a may correspond to a 2x4 matrix.
  • the downmix processing unit according to the formula 15 is illustrated FIG. LOB.
  • D2 can be configured to generate de-correlated signals Di(a*Oi+b* ⁇ 2), 3 2 (c*Oi+d*O 2 ).
  • the downmix processing unit according to the formula 15 is illustrated FIG. OC.
  • a de-correlating part 722" including two de-correlators )i, D 2 can be configured to generate de-correlated signals Di(O 1 ), D 2 ( ⁇ 2).
  • downmix processing unit includes a mixing part :orresponding to 2x3 matrix
  • the matrix R is a 2x3 matrix
  • the matrix O is a 3x1 matrix
  • the C is a 2x1 matrix.
  • FIG. 11 is an exemplary block diagram of a second embodiment of a downmix processing unit illustrated in FIG. 7.
  • a second embodiment of a downmix processing unit 720b (hereinafter simply 'a downmix processing unit 720b') may be implementation of rendering module 900 like the downmix processing unit 720a.
  • a downmix processing unit [Ob can be configured to skip input signal in case of mono input signal (m), and to rocess input signal in case of stereo input signal (L, R).
  • the downmix processing ait 720b may include a de-correlating part 722b and a mixing part 724b.
  • the de- >rrelating part 722b has a de-correlator D which can be configured to de-correlate iput signal Ch, O2 and output the de-correlated signal D(O ⁇ O 2 ).
  • the de- ⁇ rrelating part 722b may correspond to a 1x2 matrix.
  • the mixing part 724b can be Dnfigured to map input signal and the de-correlated signal to each channel.
  • the lixing part 724b may correspond to a 2x3 matrix which can be shown as a matrix [ in the formula 16.
  • the de-correlating part 722b can be configured to de-correlate a lifference signal O 1 -Ch as common signal of two input signal Oi, O2.
  • the mixing >art 724b can be configured to map input signal and the de-correlated common ignal to each channel.
  • downmix processing unit includes a mixing part with leveral matrixes
  • Certain object signal can be audible as a similar impression anywhere /vithout being positioned at a specified position, which may be called as a 'spatial sound signal'.
  • a 'spatial sound signal' For example, applause or noises of a concert hall can be an example Df the spatial sound signal.
  • the spatial sound signal needs to be playback via all speakers. If the spatial sound signal playbacks as the same signal via all speakers, it hard to feel spatialness of the signal because of high inter-correlation (IC) of the
  • FIG. 12 is an exemplary block diagram of a third embodiment of a downmix
  • FIG.12 a third embodiment of a
  • Dwnmix processing unit 720c (hereinafter simply 'a downmix processing unit
  • IQc' can be configured to generate spatial sound signal using input signal Oi
  • liich may include a de-correlating part 722c with N de-correlators and a mixing
  • the de-correlating part 722c may have N de-correlators Di, Oi, ", DN
  • Oi is i th input signal
  • R/ is a matrix mapping i th input signal Oi to j th channel
  • the md C j-i is j th output signal.
  • the ⁇ j_i value is de-correlation rate.
  • the ⁇ jj value can be estimated base on ICC included in multi-channel >arameter.
  • the mixing part 724c can generate output signals base on patialness information composing de-correlation rate ⁇ j j received from user- nterface via the information generating unit 710, which does not put limitation on >resent invention.
  • the number of de-correlators (N) can be equal to the number of output hannels.
  • the de-correlated signal can be added to output hannels selected by user. For example, it is able to position certain spatial sound ignal at left, right, and center and to output as a spatial sound signal via left hannel speaker.
  • FIG. 13 is an exemplary block diagram of a fourth embodiment of a lownmix processing unit illustrated in FIG. 7.
  • a fourth embodiment of a downmix processing unit 72Od (hereinafter simply 'a downmix processing unit 72Od') can be :onfigured to bypass if the input signal corresponds to a mono signal (m).
  • the lownmix processing unit 72Od includes a further downmixing part 722d which can )e configured to downmix the stereo signal to be mono signal if the input signal :orresponds to a stereo signal.
  • the further downmixed mono channel (m) is used is input to the multi-channel decoder 730.
  • the multi-channel decoder 730 can )ntrol object panning (especially cross-talk) by using the mono input signal.
  • the information generating unit 710 may generate a multi-channel arameter base on 5-1 -5i configuration of MPEG Surround standard.
  • the ADG may be generated by the information enerating unit 710 based on mix information.
  • FIG. 14 is an exemplary block diagram of a bitstream structure of a ompressed audio signal according to a second embodiment of present invention.
  • 3 IG. 15 is an exemplary block diagram of an apparatus for processing an audio signal according to a second embodiment of present invention.
  • downmix signal ⁇ , multi-channel parameter ⁇ , and object parameter ⁇ are ncluded in the bitstream structure.
  • the multi-channel parameter ⁇ is a parameter : or upmixing the downmix signal.
  • the object parameter ⁇ is a parameter for controlling object panning and object gain.
  • downmix signal ⁇ , a default parameter ⁇ ', and object parameter ⁇ are included in the bitstream structure.
  • the default parameter ⁇ ' may include preset information for controlling object gain and object panning.
  • the preset information may orrespond to an example suggested by a producer of an encoder side.
  • ireset information may describes that guitar signal is located at a point between ⁇ ft and center, and guitar's level is set to a certain volume, and the number of output channel in this time is set to a certain channel.
  • the default parameter for ither each frame or specified frame may be present in the bitstream.
  • Flag nformation indicating whether default parameter for this frame is different from lefault parameter of previous frame or not may be present in the bitstream. By ncluding default parameter in the bitstream, it is able to take less bitrates than side nformation with object parameter is included in the bitstream.
  • leader information of the bitstream is omitted in the FIG. 14. Sequence of the >itstream can be rearranged.
  • an apparatus for processing an audio signal according o a second embodiment of present invention 1000 may include a bitstream de-multiplexer 1005, an information generating unit 010, a downmix processing unit 1020, and a multil-channel decoder 1030.
  • the de- nultiplexer 1005 can be configured to divide the multiplexed audio signal into a lownmix ⁇ , a first multi-channel parameter ⁇ , and an object parameter ⁇ .
  • the nformation generating unit 1010 can be configured to generate a second multi- :hannel parameter using an object parameter ⁇ and a mix parameter.
  • the mix parameter comprises a mode information indicating whether the first multi- iannel information ⁇ is applied to the processed downmix.
  • the mode information iay corresponds to an information for selecting by a user. According to the mode lformation, the information generating information 1020 decides whether to •ansmit the first multi-channel parameter ⁇ or the second multi-channel parameter.
  • the downmix processing unit 1020 can be configured to determining a •rocessing scheme according to the mode information included in the mix nformation. Furthermore, the downmix processing unit 1020 can be configured to •rocess the downmix ⁇ according to the determined processing scheme. Then the lownmix processing unit 1020 transmits the processed downmix to multi-channel lecoder 1030.
  • the multi-channel decoder 1030 can be configured to receive either the first nulti-channel parameter ⁇ or the second multi-channel parameter. In case that iefault parameter ⁇ ' is included in the bitstream, the multi-channel decoder 1030 :an use the default parameter ⁇ ' instead of multi-channel parameter ⁇ .
  • the multi-channel decoder 1030 can be configured to generate multi- :hannel output using the processed downmix signal and the received multichannel parameter.
  • the multi-channel decoder 1030 may have the same configuration of the former multi-channel decoder 730, which does not put limitation on the present invention. 3. Binaural Processing
  • a multi-channel decoder can be operated in a binaural mode. This enables a ⁇ ulti-channel impression over headphones by means of Head Related Transfer 'unction (HRTF) filtering.
  • HRTF Head Related Transfer 'unction
  • the downmix signal and nulti-channel parameters are used in combination with HRTF filters supplied to he decoder.
  • FIG. 16 is an exemplary block diagram of an apparatus for processing an Ludio signal according to a third embodiment of present invention.
  • an apparatus for processing an audio signal according to a third embodiment may comprise an information generating unit 1110, a downmix processing unit 1120, and a multi-channel iecoder 1130 with a sync matching part 1130a.
  • the information generating unit 1110 may have the same configuration of :he information generating unit 710 of FIG. 7, with generating dynamic HRTF.
  • the iownmix processing unit 1120 may have the same configuration of the downmix processing unit 720 of FIG. 7.
  • multi-channel decoder 1130 except for the sync matching part 1130a is the same case of the former elements.
  • the dynamic HRTF describes the relation between object signals and virtual peaker signals corresponding to the HRTF azimuth and elevation angles, which is ime-dependent information according to real-time user control.
  • the dynamic HRTF may correspond to one of HTRF filter coefficients itself, •arameterized coefficient information, and index information in case that the aulti-channel decoder comprise all HRTF filter set.
  • tag information may >e included in ancillary field in MPEG Surround standard.
  • the tag information nay be represented as a time information, a counter information, a index nformation, etc.
  • FIG. 17 is an exemplary block diagram of an apparatus for processing an iudio signal according to a fourth embodiment of present invention.
  • the apparatus or processing an audio signal according to a fourth embodiment of present .nvention 1200 (hereinafter simply 'a processor 1200') may comprise an encoder 1210 at encoder side 1200A, and a rendering unit 1220 and a synthesis unit 1230 at ecoder side 1200B.
  • the encoder 1210 can be configured to receive multi-channel bject signal and generate a downmix of audio signal and a side information.
  • the ⁇ ndering unit 1220 can be configured to receive side information from the encoder 210, playback configuration and user control from a device setting or a user- iterface, and generate rendering information using the side information, playback onfiguration, and user control.
  • the synthesis unit 1230 can be configured to ynthesis multi-channel output signal using the rendering information and the eceived downmix signal from an encoder 1210.
  • the effect-mode is a mode for remixed or reconstructed signal.
  • ive mode club band mode, karaoke mode, etc may be present.
  • the effect-mode information may correspond to a mix parameter set generated by a producer, other iser, etc. If the effect-mode information is applied, an end user don't have to :ontrol object panning and object gain in full because user can select one of predetermined effect-mode informations.
  • effect-mode information Two methods of generating an effect-mode information can be distinguished. First of all, it is possible that an effect-mode information is generated by encoder 1200A and transmitted to the decoder 1200B. Secondly, the effect-mode information may be generated automatically at the decoder side. Details of two methods shall be described as follow. 4.1.1 Transmitting effect-mode information to decoder side
  • the effect-mode information may be generated at an encoder 1200A by a •roducer.
  • the decoder 1200B can be configured to eceive side information including the effect-mode information and output user- nterface by which a user can select one of effect-mode informations.
  • the decoder 200B can be configured to generate output channel base on the selected effect- node information.
  • the effect-mode information may be generated at a decoder 1200B.
  • the decoder 1200B can be configured to search appropriate effect-mode informations cor the downmix signal. Then the decoder 1200B can be configured to select one of the searched effect-mode by itself (automatic adjustment mode) or enable a user to select one of them (user selection mode). Then the decoder 1200B can be configured to obtain object information (number of objects, instrument names, etc) included in side information, and control object based on the selected effect-mode information and the object information. Furthermore, it is able to control similar objects in a lump. For example, ⁇ struments associated with a rhythm may be similar objects in case of 'rhythm tnpression mode'. Controlling in a lump means controlling each object imultaneously rather than controlling objects using the same parameter.
  • the input signal inputted to an encoder 1200A may be classified into three ;ypes as follow.
  • Mono object is most general type of object. It is possible to synthesis internal downmix signal by simply summing objects. It is also possible to synthesis internal downmix signal using object gain and object panning which may be one of user control and provided information. In generating internal downmix signal, it is also possible to generate rendering information using at least one of object characteristic, user input, and information provided with object. In case that external downmix signal is present, it is possible to extract and ansmit information indicating relation between external downmix and object.
  • multi-channel object it is able to perform the above mentioned nethod described with mono object and stereo object. Furthermore, it is able to .nput multi-channel object as a form of MPEG Surround. In this case, it is able to snerate object-based downmix (ex: SAOC downmix) using object downmix lannel, and use multi-channel information (ex: spatial information in MPEG urround) for generating multi-channel information and rendering information, [ence, it is possible to reduce computing amount because multi-channel object resent in form of MPEG Surround don't have to decode and encode using object- riented encoder (ex: SAOC encoder). If object downmix corresponds to stereo and bject-based downmix (ex: SAOC downmix) corresponds to mono in this case, it is ossible to apply the above-mentioned method described with stereo object.
  • SAOC downmix object-based downmix
  • SAOC encoder object-riented encoder
  • variable type of object may be transmitted from the encoder 1200A to the decoder. 1200B.
  • Transmitting scheme for variable type of object can be provided as follow:
  • a side nf ormation includes information for each object. For example, when a plural object onsists of Nth mono object (A), left channel of N+lth object (B), and right channel )f N+lth object (C), a side information includes information for 3 objects (A, B, C).
  • the side information may comprise correlation flag information indicating ⁇ rhether an object is part of a stereo or multi-channel object, for example, mono ⁇ ject, one channel (L or R) of stereo object, and so on.
  • correlation flag nformation is '0' if mono object is present
  • correlation flag information is '1' if one Lannel of stereo object is present.
  • correlation flag information for other irt of stereo object may be any value (ex: 1 O', "Y 1 or whatever).
  • >rrelation flag information for other part of stereo object may be not transmitted.
  • correlation flag information for we part of multi-channel object may be value describing number of multi-channel bject.
  • correlation flag information for ft channel of 5.1 channel may be '5'
  • correlation flag information for the other tiannel (R, Lr, Rr, C, LFE) of 5.1 channel may be either '0' or not transmitted.
  • Object may have the three kinds of attribute as follows: a) Single object
  • Single object can be configured as a source. It is able to apply one parameter o single object for controlling object panning and object gain in generating lownmix signal and reproducing.
  • the 'one parameter' may mean not only one )arameter for all time/ frequency domain but also one parameter for each ime/ frequency slot. b) Grouped object
  • m encoder 1300 includes a grouping unit 1310 and a downmix unit 1320.
  • the grouping unit 1310 can be configured to group at least two objects among inputted nulti-object input, base on a grouping information.
  • the grouping information may oe generated by producer at encoder side.
  • the downmix unit 1320 can be :onfigured to generate downmix signal using the grouped object generated by the grouping unit 1310.
  • the downmix unit 1320 can be configured to generate a side information for the grouped object.
  • Combination object is an object combined with at least one source. It is possible to control object panning and gain in a lump, but keep relation between combined objects unchanged. For example, in case of drum, it is possible to control drum, but keep relation between base drum, tam-tam, and symbol unchanged. For example, when base drum is located at center point and symbol is located at left point, it is possible to positioning base drum at right point and positioning symbol at point between center and right in case that drum is moved to right direction.
  • Relation information between combined objects may be transmitted to a decoder.
  • decoder can extract the relation information using combination object.
  • the present invention is applicable to encode and decode an Lidio signal.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Stereophonic System (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Stereo-Broadcasting Methods (AREA)

Abstract

A method for processing an audio signal, comprising: receiving a downmix signal in time domain; if the downmix signal corresponds to a mono signal, bypassing the downmix signal; if the number of channel of the downmix signal corresponds to at least two, decomposing the downmix signal into a subband signal, and processing the subband signal using a downmix processing information, wherein the downmix processing information is estimated based on an object information and a mix information is disclosed.

Description

[DESCRIPTION] [Invention Title]
A METHOD AND AN APPARATUS FOR PROCESSING AN AUDIO SIGNAL [Technical Field]
The present invention relates to a method and an apparatus for processing an audio signal, and more particularly, to a method and an apparatus for decoding an audio signal received on a digital medium, as a broadcast signal, and so on. [Background Art]
While downmixing several audio objects to be a mono or stereo signal, parameters from the individual object signals can be extracted. These parameters can be used in a decoder of an audio signal, and repositioning/ panning of the individual sources can be controlled by user' selection. [Disclosure] [Technical Problem]
However, in order to control the individual object signals, repositioning/ panning of the individual sources included in a downmix signal must be performed suitably. However, for backward compatibility with respect to the channel-oriented iecoding method (as a MPEG Surround), an object parameter must be converted lexibly to a multi-channel parameter required in upmixing process. [Technical Solution]
Accordingly, the present invention is directed to a method and an apparatus :or processing an audio signal that substantially obviates one or more problems due to limitations and disadvantages of the related art.
An object of the present invention is to provide a method and an apparatus for processing an audio signal to control object gain and panning unrestrictedly.
Another object of the present invention is to provide a method and an apparatus for processing an audio signal to control object gain and panning based on user selection.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. [Advantageous Effects]
The present invention provides the following effects or advantages. First of all, the present invention is able to provide a method and an pparatus for processing an audio signal to control object gain and panning .nrestrictedly.
Secondly, the present invention is able to provide a method and an pparatus for processing an audio signal to control object gain and panning based >n user selection. [Description of Drawings]
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this ipplication, illustrate embodiments of the invention and together with the iescription serve to explain the principle of the invention. In the drawings;
FIG. 1 is an exemplary block diagram to explain to basic concept of rendering a downmix signal based on playback configuration and user control.
FIG. 2 is an exemplary block diagram of an apparatus for processing an audio signal according to one embodiment of the present invention corresponding to the first scheme.
FIG. 3 is an exemplary block diagram of an apparatus for processing an audio signal according to another embodiment of the present invention corresponding to the first scheme. FIG. 4 is an exemplary block diagram of an apparatus for processing an udio signal according to one embodiment of present invention corresponding to tie second scheme.
FIG. 5 is an exemplary block diagram of an apparatus for processing an udio signal according to another embodiment of present invention corresponding o the second scheme.
FIG. 6 is an exemplary block diagram of an apparatus for processing an iudio signal according to the other embodiment of present invention orresponding to the second scheme.
FIG. 7 is an exemplary block diagram of an apparatus for processing an Ludio signal according to one embodiment of the present invention corresponding o the third scheme.
FIG. 8 is an exemplary block diagram of an apparatus for processing an iudio signal according to another embodiment of the present invention :orresponding to the third scheme.
FIG. 9 is an exemplary block diagram to explain to basic concept of •endering unit.
FIGS. 1OA to 1OC are exemplary block diagrams of a first embodiment of a lownmix processing unit illustrated in FIG. 7. FIG. 11 is an exemplary block diagram of a second embodiment of a iownmix processing unit illustrated in FIG. 7.
FIG. 12 is an exemplary block diagram of a third embodiment of a downmix processing unit illustrated in FIG. 7.
FIG. 13 is an exemplary block diagram of a fourth embodiment of a iownmix processing unit illustrated in FIG. 7.
FIG. 14 is an exemplary block diagram of a bitstream structure of a :ompressed audio signal according to a second embodiment of present invention.
FIG. 15 is an exemplary block diagram of an apparatus for processing an audio signal according to a second embodiment of present invention.
FIG. 16 is an exemplary block diagram of a bitstream structure of a compressed audio signal according to a third embodiment of present invention.
FIG. 17 is an exemplary block diagram of an apparatus for processing an audio signal according to a fourth embodiment of present invention.
FIG. 18 is an exemplary block diagram to explain transmitting scheme for variable type of object.
FIG. 19 is an exemplary block diagram to an apparatus for processing an audio signal according to a fifth embodiment of present invention. [Best Mode] To achieve these objects and other advantages and in accordance with the purpose of the invention, as embodied and broadly described herein, a method for processing an audio signal, comprising: receiving a downmix signal in time iomain; if the downmix signal corresponds to a mono signal, bypassing the iownmix signal; if the number of channel of the downmix signal corresponds to at [east two, decomposing the downmix signal into a subband signal, and processing the subband signal using a downmix processing information, wherein the downmix processing information is estimated based on an object information and a mix information.
According to the present invention, wherein the number of channel of the downmix signal is equal to the number of channel of the processed downmix signal.
According to the present invention, wherein the object information is included in a side information, and the side information includes a correlation flag information indicating whether an object is part of at least two channel object.
According to the present invention, wherein the object information includes at least one of an object level information and an object correlation information.
According to the present invention, wherein the downmix processing information corresponds to an information for controlling object panning if the number of channel the downmix signal corresponds to at least two. According to the present invention, wherein the downmix processing tformation corresponds to an information for controlling object gain.
According to the present invention, further comprising, generating a multi- hannel signal using the processed subband signal.
According to the present invention, further comprising, generating a multi- hannel information using the object information and the mix information, wherein he multi-channel signal is generated based on the multi-channel information.
According to the present invention, further comprising, downmixing the lownmix signal to be a mono signal if the downmix signal corresponds to a stereo ignal.
According to the present invention, wherein the mix information is generated using at least one of an object position information and a playback :onfiguration information.
According to the present invention, wherein the downmix signal is received is a broadcast signal.
According to the present invention, wherein the downmix signal is received 3n a digital medium.
In another aspect of the present invention, a computer-readable medium having instructions stored thereon, which, when executed by a processor, causes the processor to perform operations, comprising: receiving a downmix signal in ime domain; if the downmix signal corresponds to a mono signal, bypassing the lownmix signal; if the number of channel of the downmix signal corresponds to at east two, decomposing the downmix signal into a subband signal, and processing he subband signal using a downmix processing information, wherein the lownmix processing information is estimated based on an object information and a nix information.
In another aspect of the present invention, an apparatus for processing an iudio signal, comprising: a receiving unit receiving a downmix signal in time ϊomain; and, a downmix processing unit bypassing the downmix signal if the lownmix signal corresponds to a mono signal, and decomposing the downmix dgnal into a subband signal and processing the subband signal using a downmix processing information if the number of channel of the downmix signal :orresponds to at least two, wherein the downmix processing information is estimated based on an object information and a mix information.
It is to be understood that both the foregoing general description and the bllowing detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as :laimed. [Mode for Invention] Reference will now be made in detail to the preferred embodiments of the resent invention, examples of which are illustrated in the accompanying drawings, fherever possible, the same reference numbers will be used throughout the rawings to refer to the same or like parts.
Prior to describing the present invention, it should be noted that most terms isclosed in the present invention correspond to general terms well known in the rt, but some terms have been selected by the applicant as necessary and will ereinafter be disclosed in the following description of the present invention, herefore, it is preferable that the terms defined by the applicant be understood on ie basis of their meanings in the present invention.
In particular, 'parameter' in the following description means information tcluding values, parameters of narrow sense, coefficients, elements, and so on. Iereinafter 'parameter' term will be used instead of 'information' term like an bject parameter, a mix parameter, a downmix processing parameter, and so on, Λύch does not put limitation on the present invention.
In downmixing several channel signals or object signals, an object parameter nd a spatial parameter can be extracted. A decoder can generate output signal Lsing a downmix signal and the object parameter (or the spatial parameter). The iutput signal may be rendered based on playback configuration and user control y the decoder. The rendering process shall be explained in details with reference ) the FIG. 1 as follow.
FIG. 1 is an exemplary diagram to explain to basic concept of rendering ownmix based on playback configuration and user control. Referring to FIG. 1, a ecoder 100 may include a rendering information generating unit 110 and a endering unit 120, and also may include a Tenderer 110a and a synthesis 120a astead of the rendering information generating unit 110 and the rendering unit 120.
A rendering information generating unit 110 can be configured to receive a ide information including an object parameter or a spatial parameter from an ■ncoder, and also to receive a playback configuration or a user control from a levice setting or a user interface. The object parameter may correspond to a jarameter extracted in downmixing at least one object signal, and the spatial >arameter may correspond to a parameter extracted in downmixing at least one :hannel signal. Furthermore, type information and characteristic information for ϊach object may be included in the side information. Type information and iharacteristic information may describe instrument name, player name, and so on. The playback configuration may include speaker position and ambient information ^speaker's virtual position), and the user control may correspond to a control information inputted by a user in order to control object positions and object gains, and also may correspond to a control information in order to the playback mfiguration. Meanwhile the payback configuration and user control can be ^presented as a mix information, which does not put limitation on the present Lvention.
A rendering information generating unit 110 can be configured to generate a ϊndering information using a mix information (the playback configuration and ser control) and the received side information. A rendering unit 120 can jnfigured to generate a multi-channel parameter using the rendering information i case that the downmix of an audio signal (abbreviated 'downmix signal') is not ■ansmitted, and generate multi-channel signals using the rendering information nd downmix in case that the downmix of an audio signal is transmitted.
A renderer 110a can be configured to generate multi-channel signals using a iix information (the playback configuration and the user control) and the received ide information. A synthesis 120a can be configured to synthesis the multi-channel ignals using the multi-channel signals generated by the renderer 110a.
As previously stated, the decoder may render the downmix signal based on •layback configuration and user control. Meanwhile, in order to control the ndividual object signals, a decoder can receive an object parameter as a side nformation and control object panning and object gain based on the transmitted )bject parameter.
1. Controlling gain and panning of object signals Variable methods for controlling the individual object signals may be >rovided. First of all, in case that a decoder receives an object parameter and generates the individual object signals using the object parameter, then, can control he individual object signals base on a mix information (the playback configuration, he object level, etc.)
Secondly, in case that a decoder generates the multi-channel parameter to be nputted to a multi-channel decoder, the multi-channel decoder can upmix a lownmix signal received from an encoder using the multi-channel parameter. The Lbove-mention second method may be classified into three types of scheme. In ^articular, 1) using a conventional multi-channel decoder, 2) modifying a multi- :hannel decoder, 3) processing downmix of audio signals before being inputted to i multi-channel decoder may be provided. The conventional multi-channel decoder nay correspond to a channel-oriented spatial audio coding (ex: MPEG Surround ϊecoder), which does not put limitation on the present invention. Details of three ypes of scheme shall be explained as follow.
1.1 Using a multi-channel decoder
First scheme may use a conventional multi-channel decoder as it is without nodifying a multi-channel decoder. At first, a case of using the ADG (arbitrary lownmix gain) for controlling object gains and a case of using the 5-2-5 :onfiguration for controlling object panning shall be explained with reference to IG. 2 as follow. Subsequently, a case of being linked with a scene remixing unit rill be explained with reference to FIG. 3.
FIG. 2 is an exemplary block diagram of an apparatus for processing an udio signal according to one embodiment of the present invention corresponding D first scheme. Referring to FIG. 2, an apparatus for processing an audio signal 200 hereinafter simply 'a decoder 200') may include an information generating unit .10 and a multi-channel decoder 230. The information generating unit 210 may eceive a side information including an object parameter from an encoder and a nix information from a user interface, and may generate a multi-channel >arameter including a arbitrary downmix gain or a gain modification ;ain(hereinafter simple 'ADG'). The ADG may describe a ratio of a first gain :stimated based on the mix information and the obejct information over a second ^ain extimated based on the object information. In particular, the information generating unit 210 may generate the ADG only if the downmix signal corresponds :o a mono signal. The multi-channel decoder 230 may receive a downmix of an iudio signal from an encoder and a multi-channel parameter from the information generating unit 210, and may generate a multi-channel output using the downmix signal and the multi-channel parameter. The multi-channel parameter may include a channel level difference hereinafter abbreviated 'CLD'), an inter channel correlation (hereinafter abbreviated 'ICC), a channel prediction coefficient (hereinafter abbreviated 'CPC).
Since CLD, ICC7 and CPC describe intensity difference or correlation between two channels, and is to control object panning and correlation. It is able to :ontrol object positions and object diffuseness (sonority) using the CLD, the ICC, ϊtc. Meanwhile, the CLD describe the relative level difference instead of the absolute level, and energy of the splitted two channels is conserved. Therefore it is enable to control object gains by handling CLD, etc. In other words, specific object :annot be mute or volume up by using the CLD, etc.
Furthermore, the ADG describes time and frequency dependent gain for :ontrolling correction factor by a user. If this correction factor be applied, it is able to handle modification of down-mix signal prior to a multi-channel upmixing. Therefore, in case that ADG parameter is received from the information generating unit 210, the multi-channel decoder 230 can control object gains of specific time and frequency using the ADG parameter.
Meanwhile, a case that the received stereo downmix signal outputs as a stereo channel can be defined the following formula 1. [formula 1]
J>[0] = Wn • go • *[0] + W12 • g-, • x[l] y\\] = W21 • g0 ■ x[0] + W22 • g, • x[l] where x[] is input channels, y[] is output channels, gx is gains, and wxx is '•eight.
It is necessary to control cross-talk between left channel and right channel in rder to object panning. In particular, a part of left channel of downmix signal may utput as a right channel of output signal, and a part of right channel of downmix Lgnal may output as left channel of output signal. In the formula 1, W12 and W21 iay be a cross-talk component (in other words, cross-term).
The above-mentioned case corresponds to 2-2-2 configuration, which means -channel input, 2-channel transmission, and 2-channel output. In order to perform he 2-2-2 configuration, 5-2-5 configuration (2-channel input, 5-channel ransmission, and 2 channel output) of conventional channel-oriented spatial audio oding (ex: MPEG surround) can be used. At first, in order to output 2 channels for .-2-2 configuration, certain channel among 5 output channels of 5-2-5 configuration :an be set to a disable channel (a fake channel). In order to give cross-talk between '-transmitted channels and 2-output channels, the above-mentioned CLD and CPC •nay be adjusted. In brief, gain factor gx in the formula 1 is obtained using the above mentioned ADG, and weighting factor wn~W22 in the formula 1 is obtained using CLD and CPC. In implementing the 2-2-2 configuration using 5-2-5 configuration, in order :o reduce complexity, default mode of conventional spatial audio coding may be applied. Since characteristic of default CLD is supposed to output 2-channel, it is ible to reduce computing amount if the default CLD is applied. Particularly, since :here is no need to synthesis a fake channel, it is able to reduce computing amount argely. Therefore, applying the default mode is proper. In particular, only default 2LD of 3 CLDs (corresponding to 0, 1, and 2 in MPEG surround standard) is used or decoding. On the other hand, 4 CLDs among left channel, right channel, and renter channel (corresponding to 3, 4, 5, and 6 in MPEG surround standard) and 2 \DGs (corresponding to 7 and 8 in MPEG surround standard) is generated for :ontrolling object. In this case, CLDs corresponding 3 and 5 describe channel level iifference between left channel plus right channel and center channel ((l+r)/c) is proper to set to 15OdB (approximately infinite) in order to mute center channel, ^nd, in order to implement cross-talk, energy based up-mix or prediction based ip-mix may be performed, which is invoked in case that TTT mode "bsTttModeLow' in the MPEG surround standard) corresponds to energy-based node (with subtraction, matrix compatibility enabled) (3rd mode), or prediction node (1st mode or 2nd mode).
FIG. 3 is an exemplary block diagram of an apparatus for processing an audio signal according to another embodiment of the present invention orresponding to first scheme. Referring to FIG. 3, an apparatus for processing an iudio signal according to another embodiment of the present invention 300 hereinafter simply a decoder 300) may include a information generating unit 310, a cene rendering unit 320, a multi-channel decoder 330, and a scene remixing unit •50.
The information generating unit 310 can be configured to receive a side nformation including an object parameter from an encoder if the downmix signal :orresponds to mono channel signal (i.e., the number of downmix channel is 'V), nay receive a mix information from a user interface, and may generate a multi- :hannel parameter using the side information and the mix information. The lumber of downmix channel can be estimated based on a flag information ncluded in the side information as well as the downmix signal itself and user ielection. The information generating unit 310 may have the same configuration of he former information generating unit 210. The multi-channel parameter is nputted to the multi-channel decoder 330, the multi-channel decoder 330 may lave the same configuration of the former multi-channel decoder 230.
The scene rendering unit 320 can be configured to receive a side information ncluding an object parameter from and encoder if the downmix signal corresponds :o non-mono channel signal (i.e., the number of downmix channel is more than '2'), nay receive a mix information from a user interface, and may generate a remixing iarameter using the side information and the mix information. The remixing iarameter corresponds to a parameter in order to remix a stereo channel and ;enerate more than 2-channel outputs. The remixing parameter is inputted to the cene remixing unit 350. The scene remixing unit 350 can be configured to remix he downmix signal using the remixing parameter if the downmix signal is more han 2-channel signal.
In brief, two paths could be considered as separate implementations for ieparate applications in a decoder 300.
1.2 Modifying a multi-channel decoder
Second scheme may modify a conventional multi-channel decoder. At first, a :ase of using virtual output for controlling object gains and a case of modifying a levice setting for controlling object panning shall be explained with reference to ?IG. 4 as follow. Subsequently, a case of Performing TBT(2x2) functionality in a ϊiulti-channel decoder shall be explained with reference to FIG. 5.
FIG. 4 is an exemplary block diagram of an apparatus for processing an audio signal according to one embodiment of present invention corresponding to the second scheme. Referring to FIG. 4, an apparatus for processing an audio signal according to one embodiment of present invention corresponding to the second scheme 400 (hereinafter simply 'a decoder 400') may include an information generating unit 410, an internal multi-channel synthesis 420, and an output lapping unit 430. The internal multi-channel synthesis 420 and the output αapping unit 430 may be included in a synthesis unit.
The information generating unit 410 can be configured to receive a side information including an object parameter from an encoder, and a mix parameter rom a user interface. And the information generating unit 410 can be configured to generate a multi-channel parameter and a device setting information using the side nf ormation and the mix information. The multi-channel parameter may have the ■ame configuration of the former multi-channel parameter. So, details of the multi- :hannel parameter shall be omitted in the following description. The device setting nformation may correspond to parameterized HRTF for binaural processing, /vhich shall be explained in the description of '1.2.2 Using a device setting nformation'.
The internal multi-channel synthesis 420 can be configured to receive a multi-channel parameter and a device setting information from the parameter generation unit 410 and downmix signal from an encoder. The internal multichannel synthesis 420 can be configured to generate a temporal multi-channel output including a virtual output, which shall be explained in the description of '1.2.1 Using a virtual output'.
1.2.1 Using a virtual output Since multi-channel parameter (ex: CLD) can control object panning, it is ird to control object gain as well as object panning by a conventional multi- iannel decoder.
Meanwhile, in order to object gain, the decoder 400 (especially the internal Lulti-channel synthesis 420) may map relative energy of object to a virtual channel 'x: center channel). The relative energy of object corresponds to energy to be ϊduced. For example, in order to mute certain object, the decoder 400 may map tore than 99.9% of object energy to a virtual channel. Then, the decoder 400 especially, the output mapping unit 430) does not output the virtual channel to /hich the rest energy of object is mapped. In conclusion, if more than 99.9% of bject is mapped to a virtual channel which is not outputted, the desired object can >e almost mute.
1.2.2 Using a device setting information
The decoder 400 can adjust a device setting information in order to control >bject panning and object gain. For example, the decoder can be configured to generate a parameterized HRTF for binaural processing in MPEG Surround standard. The parameterized HRTF can be variable according to device setting. It is able to assume that object signals can be controlled according to the following formula 2. [formula 2]
Lnew = ai * obji + a2 * obJ2 + a3 * obJ3 + .. + an * objn,
Rnew = bl * θbjl + bl * θbJ2 + b3 * θbJ3 + •• + bn *θbjn/ where objk is object signals, Lnew and Rnew is a desired stereo signal, and ak md bk are coefficients for object control.
An object information of the object signals objk may be estimated from an )bject parameter included in the transmitted side information. The coefficients ak, >k which are defined according to object gain and object panning may be estimated rom the mix information. The desired object gain and object panning can be idjusted using the coefficients ak, bk.
The coefficients ak, bk can be set to correspond to HRTF parameter for ήnaural processing, which shall be explained in details as follow.
In MPEG Surround standard (5-l-5i configuration) (from ISO/IEC FDIS .3003-l:2006(E), Information Technology - MPEG Audio Technologies - Parti: vlPEG Surround), binaural processing is as below.
[formula 3]
Figure imgf000023_0001
where yB is output, the matrix H is conversion matrix for binaural processing. [formula 4]
Figure imgf000024_0001
The elements of matrix H is defined as follows:
[formula 5]
Figure imgf000024_0003
[formula 6]
Figure imgf000024_0002
[formula 7]
Figure imgf000024_0004
1.2.3 Performing TBT(2x2) functionality in a multi-channel decoder FIG. 5 is an exemplary block diagram of an apparatus for processing an mdio signal according to another embodiment of present invention corresponding :o the second scheme. FIG. 5 is an exemplary block diagram of TBT functionality in i multi-channel decoder. Referring to FIG. 5, a TBT module 510 can be configured :o receive input signals and a TBT control information, and generate output signals. The TBT module 510 may be included in the decoder 200 of the FIG. 2 (or in particular, the multi-channel decoder 230). The multi-channel decoder 230 may be implemented according to the MPEG Surround standard, which does not put Limitation on the present invention, [formula 9]
Figure imgf000025_0001
where x is input channels, y is output channels, and w is weight.
The output yi may correspond to a combination input xi of the downmix multiplied by a first gain Wi1 and input X2 multiplied by a second gain W12.
The TBT control information inputted in the TBT module 510 includes elements which can compose the weight w (wu, Wi2, Wa1, W22).
In MEPG Surround standard, OTT(One-To-Two) module and TTT(Two-To- Three) module is not proper to remix input signal although OTT module and TTT module can upmix the input signal. In order to remix the input signal, TBT (2x2) module 510 (hereinafter bbreviated 'TBT module 510') may be provided. The TBT module 510 may can be igured to receive a stereo signal and output the remixed stereo signal. The weight v may be composed using CLD (s) and ICC(s).
If the weight term W1I ~ W22 is transmitted as a TBT control information, the lecoder may control object gain as well as object panning using the received veight term. In transmitting the weight term w, variable scheme may be provided. \t first, a TBT control information includes cross term like the W12 and W21. jecondly, a TBT control information does not include the cross term like the W12 md W21. Thirdly, the number of the term as a TBT control information varies idaptively.
At first, there is need to receive the cross term like the W12 and W21 in order to :ontrol object panning as left signal of input channel go to right of the output :hannel. In case of N input channels and M output channels, the terms which lumber is NxM may be transmitted as TBT control information. The terms can be quantized based on a CLD parameter quantization table introduced in a MPEG Surround, which does not put limitation on the present invention.
Secondly, unless left object is shifted to right position, (i.e. when left object is moved to more left position or left position adjacent to center position, or when only level of the object is adjusted), there is no need to use the cross term. In the \se, it is proper that the term except for the cross term is transmitted. In case of N iput channels and M output channels, the terms which number is just N may be •ansmitted.
Thirdly, the number of the TBT control information varies adaptively ccording to need of cross term in order to reduce the bit rate of a TBT control iformation. A flag information /cross_flag' indicating whether the cross term is 'resent or not is set to be transmitted as a TBT control information. Meaning of the Lag information 'cross_flag' is shown in the following table 1. [table 1] meaning of cross_flag
Figure imgf000027_0001
In case that /cross_flag/ is equal to 0, the TBT control information does not nclude the cross term, only the non-cross term like the wn and W22 is present. Dtherwise ('cross_flag' is equal to 1), the TBT control information includes the cross erm.
Besides, a flag information /reverse_flag/ indicating whether cross term is present or non-cross term is present is set to be transmitted as a TBT control information. Meaning of flag information /reverse_flag' is shown in the following :able 2. [table 2] meaning of reverse_flag
Figure imgf000028_0001
In case that 'reversejflag' is equal to 0, the TBT control information does not nclude the cross term, only the non-cross term like the W11 and W22 is present. )therwise ('reverse_flag' is equal to 1), the TBT control information includes only he cross term.
Futhermore, a flag information 'side_flag' indicating whether cross term is •resent and non-cross is present is set to be transmitted as a TBT control nformation. Meaning of flag information /side_flag/ is shown in the following table
[table 3] meaning of side_config
Figure imgf000028_0002
Since the table 3 corresponds to combination of the table 1 and the table 2, itails of the table 3 shall be omitted.
1.2.4 Performing TBT(2x2) functionality in a multi-channel decoder by Lodifying a binaural decoder
The case of '1.2.2 Using a device setting information' can be performed dthout modifying the binaural decoder. Hereinafter, performing TBT icnctionality by modifying a binaural decoder employed in a MPEG Surround ecoder, with reference to FIG. 6.
FIG. 6 is an exemplary block diagram of an apparatus for processing an udio signal according to the other embodiment of present invention orresponding to the second scheme. In particular, an apparatus for processing an udio signal 630 shown in the FIG. 6 may correspond to a binaural decoder ncluded in the multi-channel decoder 230 of FIG. 2 or the synthesis unit of FIG. 4, vhich does not put limitation on the present invention.
An apparatus for processing an audio signal 630 (hereinafter 'a binaural lecoder 630') may include a QMF analysis 632, a parameter conversion 634, a spatial synthesis 636, and a QMF synthesis 638. Elements of the binaural decoder 330 may have the same configuration of MPEG Surround binaural decoder in VlPEG Surround standard. For example, the spatial synthesis 636 can be configured to consist of 1 2x2 (filter) matrix, according to the following formula 10: [formula 10]
Figure imgf000030_0001
with yo being the QMF-domain input channels and ye, being the binaural >utput channels, k represents the hybrid QMF channel index, and i is the HRTF ilter tap index, and n is the QMF slot index. The binaural decoder 630 can be :onfigured to perform the above-mentioned functionality described in subclause 1.2.2 Using a device setting information'. However, the elements hij may be generated using a multi-channel parameter and a mix information instead of a nulti-channel parameter and HRTF parameter. In this case, the binaural decoder )00 can perform the functionality of the TBT module 510 in the FIG. 5. Details of the elements of the binaural decoder 630 shall be omitted.
The binaural decoder 630 can be operated according to a flag information binaural_flag'. In particular, the binaural decoder 630 can be skipped in case that a lag information binaural_flag is '0', otherwise (the binaural_flag is 'V), the binaural decoder 630 can be operated as below.
[table 4] meaning of binaural_flag
Figure imgf000030_0002
1.3 Processing downmix of audio signals before being inputted to a multi- liannel decoder
The first scheme of using a conventional multi-channel decoder have been xplained in subclause in '1.V, the second scheme of modifying a multi-channel ecoder have been explained in subclause in '1.2'. The third scheme of processing ownmix of audio signals before being inputted to a multi-channel decoder shall e explained as follow.
FIG. 7 is an exemplary block diagram of an apparatus for processing an udio signal according to one embodiment of the present invention corresponding o the third scheme. FIG. 8 is an exemplary block diagram of an apparatus for >rocessing an audio signal according to another embodiment of the present nvention corresponding to the third scheme. At first, Referring to FIG. 7, an Lpparatus for processing an audio signal 700 (hereinafter simply 'a decoder 700') nay include an information generating unit 710, a downmix processing unit 720, md a multi-channel decoder 730. Referring to FIG. 8, an apparatus for processing m audio signal 800 (hereinafter simply 'a decoder 800') may include an information generating unit 810 and a multi-channel synthesis unit 840 having a nulti-channel decoder 830. The decoder 800 may be another aspect of the decoder 700. In other words, the information generating unit 810 has the same configuration of the information generating unit 710, the multi-channel decoder 830 has the same >nfiguration of the multi-channel decoder 730, and, the multi-channel synthesis lit 840 may has the same configuration of the downmix processing unit 720 and Lulti-channel unit 730. Therefore, elements of the decoder 700 shall be explained in stails, but details of elements of the decoder 800 shall be omitted.
The information generating unit 710 can be configured to receive a side if ormation including an object parameter from an encoder and a mix information om an user-interface, and to generate a multi-channel parameter to be outputted ) the multi-channel decoder 730. From this point of view, the information enerating unit 710 has the same configuration of the former information enerating unit 210 of FIG. 2. The downmix processing parameter may correspond > a parameter for controlling object gain and object panning. For example, it is able D change either the object position or the object gain in case that the object signal is Dcated at both left channel and right channel. It is also able to render the object ignal to be located at opposite position in case that the object signal is located at inly one of left channel and right channel. In order that these cases are performed, he downmix processing unit 720 can be a TBT module (2x2 matrix operation). In ase that the information generating unit 710 can be configured to generate ADG lescribed with reference to FIG 2. in order to control object gain, the downmix )rocessing parameter may include parameter for controlling object panning but )bject gain. Furthermore, the information generating unit 710 can be configured to ceive HRTF information from HRTF database, and to generate an extra multi- annel parameter including a HRTF parameter to be inputted to the multi-channel :coder 730. In this case, the information generating unit 710 may generate multi- iannel parameter and extra multi-channel parameter in the same subband domain id transmit in syncronization with each other to the multi-channel decoder 730. ie extra multi-channel parameter including the HRTF parameter shall be cplained in details in subclause '3. Processing Binaural Mode'.
The downmix processing unit 720 can be configured to receive downmix of i audio signal from an encoder and the downmix processing parameter from the [formation generating unit 710, and to decompose a subband domain signal using ibband analysis filter bank. The downmix processing unit 720 can be configured > generate the processed downmix signal using the downmix signal and the ownmix processing parameter. In these processing, it is able to pre-process the ownmix signal in order to control object panning and object gain. The processed ownmix signal may be inputted to the multi-channel decoder 730 to be upmixed.
Furthermore, the processed downmix signal may be outputted and laybacked via speaker as well. In order to directly output the processed signal via peakers, the downmix processing unit 720 may perform synthesis filterbank using he prepossed subband domain signal and output a time-domain PCM signal. It is ble to select whether to directly output as PCM signal or input to the multi- hannel decoder by user selection.
The multi-channel decoder 730 can be configured to generate multi-channel utput signal using the processed downmix and the multi-channel parameter. The aulti-channel decoder 730 may introduce a delay when the processed downmix ignal and the multi-channel parameter are inputted in the multi-channel decoder '30. The processed downmix signal can be synthesized in frequency domain (ex: 2MF domain, hybrid QMF domain, etc), and the multi-channel parameter can be ynthesized in time domain. In MPEG surround standard, delay and lynchronization for connecting HE-AAC is introduced. Therefore, the multichannel decoder 730 may introduce the delay according to MPEG Surround standard.
The configuration of downmix processing unit 720 shall be explained in detail with reference to FIG. 9 ~ FIG. 13.
1.3.1 A general case and special cases of downmix processing unit
FIG. 9 is an exemplary block diagram to explain to basic concept of rendering unit. Referring to FIG. 9, a rendering module 900 can be configured to generate M output signals using N input signals, a playback configuration, and a user control. The N input signals may correspond to either object signals or channel signals. Furthermore, the N input signals may correspond to either object arameter or multi-channel parameter. Configuration of the rendering module 900 in be implemented in one of downmix processing unit 720 of FIG. 7, the former ^ndering unit 120 of FIG. 1, and the former renderer 110a of FIG. 1, which does not ut limitation on the present invention.
If the rendering module 900 can be configured to directly generate M hannel signals using N object signals without summing individual object signals orresponding certain channel, the configuration of the rendering module 900 can e represented the following formula 11.
[formula 11]
C = RO
Figure imgf000035_0001
Ci is a i* channel signal, Oj is j* input signal, and Rji is a matrix mapping j1*1 nput signal to i* channel.
If R matrix is separated into energy component E and de-correlation zomponent, the formula 11 may be represented as follow.
[formula 12]
C = RO = EO +DO
Figure imgf000036_0001
It is able to control object positions using the energy component E, and it is ble to control object diffuseness using the de-correlation component D.
Assuming that only ith input signal is inputted to be outputted via jth channel nd kth channel, the formula 12 may be represented as follow.
[formula 13]
Figure imgf000036_0002
αj_i is gain portion mapped to jth channel, βk_i is gain portion mapped to kth :hannel, θ is diffuseness level, and D(o;) is de-correlated output.
Assuming that de-correlation is omitted, the formula 13 may be simplified as
:ollow.
[formula 14]
Figure imgf000036_0003
If weight values for all inputs mapped to certain channel are estimated cording to the above-stated method, it is able to obtain weight values for each iannel by the following method.
1) Summing weight values for all inputs mapped to certain channel. For example, in case that input 1 Oi and input 2 O2 is inputted and output channel corresponds to left channel L, center channel C, and right channel R, a total weight values ctL(tot), ctc(tot) , ctR(tot) may be obtained as follows: [formula 15] aL(tot) = aLl aC(tot) ~ aCl + aC2 aR(tot) = aR2 where αu is a weight value for input 1 mapped to left channel L, αci is a veight value for input 1 mapped to center channel C, αc2 is a weight value for nput2 mapped to center channel C, and oκz is a weight value for input 2 mapped o right channel R.
In this case, only input 1 is mapped to left channel, only input 2 is mapped o right channel, input 1 and input 2 is mapped to center channel together.
2) Summing weight values for all inputs mapped to certain channel, then dividing the sum into the most dominant channel pair, and mapping de- correlated signal to the other channel for surround effect. In this case, the dominant channel pair may correspond to left channel and center channel in case that certain input is positioned at point between left and center.
3) Estimating weight value of the most dominant channel, giving attenuated correlated signal to the other channel, which value is a relative value of the estimated weight value.
4) Using weight values for each channel pair, combining the de-correlated signal properly, then setting to a side information for each channel.
1.3.2 A case that downmix processing unit includes a mixing part orresponding to 2x4 matrix
FIGS. 1OA to 1OC are exemplary block diagrams of a first embodiment of a lownmix processing unit illustrated in FIG. 7. As previously stated, a first mbodiment of a downmix processing unit 720a (hereinafter simply 'a downmix >rocessing unit 720a') may be implementation of rendering module 900.
First of all, assuming that
Figure imgf000038_0001
, the ormula 12 is simplified as follow.
[formula 15] β
C1 En ODjO1
C 2.
Figure imgf000038_0002
bD O, The downmix processing unit according to the formula 15 is illustrated FIG. )A. Referring to FIG. 1OA, a downmix processing unit 720a can be configured to ypass input signal in case of mono input signal (m), and to process input signal in ise of stereo input signal (L, R). The downmix processing unit 720a may include a e-correlating part 722a and a mixing part 724a. The de-correlating part 722a has a e-correlator aD and de-correlator bD which can be configured to de-correlate iput signal. The de-correlating part 722a may correspond to a 2x2 matrix. The lixing part 724a can be configured to map input signal and the de-correlated ignal to each channel. The mixing part 724a may correspond to a 2x4 matrix.
Secondly, assuming that ^d , the formula 12 is simplified as follow.
[formula 15-2]
Figure imgf000039_0001
The downmix processing unit according to the formula 15 is illustrated FIG. LOB. Referring to FIG. 1OB, a de-correlating part 722' including two de-correlators Di7. D2 can be configured to generate de-correlated signals Di(a*Oi+b*θ2), 32(c*Oi+d*O2).
Thirdly, assuming that the formula 12 is simplified as follow. [formula 15-3]
C1 En E21 TO1
+ A O To1 " C, Eu E22 Jl O2 . o Z)2JLo2.
The downmix processing unit according to the formula 15 is illustrated FIG. OC. Referring to FIG. 1OC, a de-correlating part 722" including two de-correlators )i, D2 can be configured to generate de-correlated signals Di(O1), D2(θ2).
1.3.2 A case that downmix processing unit includes a mixing part :orresponding to 2x3 matrix
The foregoing formula 15 can be represented as follow:
[formula 16]
Figure imgf000040_0001
The matrix R is a 2x3 matrix, the matrix O is a 3x1 matrix, and the C is a 2x1 matrix.
FIG. 11 is an exemplary block diagram of a second embodiment of a downmix processing unit illustrated in FIG. 7. As previously stated, a second embodiment of a downmix processing unit 720b (hereinafter simply 'a downmix processing unit 720b') may be implementation of rendering module 900 like the downmix processing unit 720a. Referring to FIG. 11, a downmix processing unit [Ob can be configured to skip input signal in case of mono input signal (m), and to rocess input signal in case of stereo input signal (L, R). The downmix processing ait 720b may include a de-correlating part 722b and a mixing part 724b. The de- >rrelating part 722b has a de-correlator D which can be configured to de-correlate iput signal Ch, O2 and output the de-correlated signal D(O^O2). The de- }rrelating part 722b may correspond to a 1x2 matrix. The mixing part 724b can be Dnfigured to map input signal and the de-correlated signal to each channel. The lixing part 724b may correspond to a 2x3 matrix which can be shown as a matrix [ in the formula 16.
Furthermore, the de-correlating part 722b can be configured to de-correlate a lifference signal O1-Ch as common signal of two input signal Oi, O2. The mixing >art 724b can be configured to map input signal and the de-correlated common ignal to each channel.
1.3.3 A case that downmix processing unit includes a mixing part with leveral matrixes
Certain object signal can be audible as a similar impression anywhere /vithout being positioned at a specified position, which may be called as a 'spatial sound signal'. For example, applause or noises of a concert hall can be an example Df the spatial sound signal. The spatial sound signal needs to be playback via all speakers. If the spatial sound signal playbacks as the same signal via all speakers, it hard to feel spatialness of the signal because of high inter-correlation (IC) of the
gnal. Hence, there's need to add correlated signal to the signal of each channel
ϊnal.
FIG. 12 is an exemplary block diagram of a third embodiment of a downmix
rocessing unit illustrated in FIG. 7. Referring to FIG.12, a third embodiment of a
Dwnmix processing unit 720c (hereinafter simply 'a downmix processing unit
IQc') can be configured to generate spatial sound signal using input signal Oi,
liich may include a de-correlating part 722c with N de-correlators and a mixing
art 724c. The de-correlating part 722c may have N de-correlators Di, Oi, ", DN
'■hich can be configured to de-correlate the input signal Of. The mixing part 724c
iay have N matrix Rj, Rk, "", Ri which can be configured to generate output signals
]j, Ck, • ••, Ci using the input signal O,- and the de-correlated signal Dx(O1). The Rj
latrix can be represented as the following formula,
[formula 17]
Figure imgf000042_0001
Oi is ith input signal, R/ is a matrix mapping ith input signal Oi to jth channel,
md Cj-i is jth output signal. The θj_i value is de-correlation rate. The θjj value can be estimated base on ICC included in multi-channel >arameter. Furthermore, the mixing part 724c can generate output signals base on patialness information composing de-correlation rate θjj received from user- nterface via the information generating unit 710, which does not put limitation on >resent invention.
The number of de-correlators (N) can be equal to the number of output hannels. On the other hand, the de-correlated signal can be added to output hannels selected by user. For example, it is able to position certain spatial sound ignal at left, right, and center and to output as a spatial sound signal via left hannel speaker.
1.3.4 A case that downmix processing unit includes a further downmixing >art
FIG. 13 is an exemplary block diagram of a fourth embodiment of a lownmix processing unit illustrated in FIG. 7. A fourth embodiment of a downmix processing unit 72Od (hereinafter simply 'a downmix processing unit 72Od') can be :onfigured to bypass if the input signal corresponds to a mono signal (m). The lownmix processing unit 72Od includes a further downmixing part 722d which can )e configured to downmix the stereo signal to be mono signal if the input signal :orresponds to a stereo signal. The further downmixed mono channel (m) is used is input to the multi-channel decoder 730. The multi-channel decoder 730 can )ntrol object panning (especially cross-talk) by using the mono input signal. In us case, the information generating unit 710 may generate a multi-channel arameter base on 5-1 -5i configuration of MPEG Surround standard.
Furthermore, if gain for the mono downmix signal like the above-mentioned rtistic downmix gain ADG of FIG. 2 is applied, it is able to control object panning nd object gain more easily. The ADG may be generated by the information enerating unit 710 based on mix information.
2. Upmixing channel signals and controlling object signals
FIG. 14 is an exemplary block diagram of a bitstream structure of a ompressed audio signal according to a second embodiment of present invention. 3IG. 15 is an exemplary block diagram of an apparatus for processing an audio signal according to a second embodiment of present invention. Referring to (a) of 11IG. 14, downmix signal α, multi-channel parameter β, and object parameter γ are ncluded in the bitstream structure. The multi-channel parameter β is a parameter :or upmixing the downmix signal. On the other hand, the object parameter γ is a parameter for controlling object panning and object gain. Referring to (b) of FIG. 14, downmix signal α, a default parameter β', and object parameter γ are included in the bitstream structure. The default parameter β' may include preset information for controlling object gain and object panning. The preset information may orrespond to an example suggested by a producer of an encoder side. For example, ireset information may describes that guitar signal is located at a point between ϊft and center, and guitar's level is set to a certain volume, and the number of output channel in this time is set to a certain channel. The default parameter for ither each frame or specified frame may be present in the bitstream. Flag nformation indicating whether default parameter for this frame is different from lefault parameter of previous frame or not may be present in the bitstream. By ncluding default parameter in the bitstream, it is able to take less bitrates than side nformation with object parameter is included in the bitstream. Furthermore, leader information of the bitstream is omitted in the FIG. 14. Sequence of the >itstream can be rearranged.
Referring to FIG. 15, an apparatus for processing an audio signal according o a second embodiment of present invention 1000 (hereinafter simply 'a decoder 000') may include a bitstream de-multiplexer 1005, an information generating unit 010, a downmix processing unit 1020, and a multil-channel decoder 1030. The de- nultiplexer 1005 can be configured to divide the multiplexed audio signal into a lownmix α, a first multi-channel parameter β, and an object parameter γ. The nformation generating unit 1010 can be configured to generate a second multi- :hannel parameter using an object parameter γ and a mix parameter. The mix parameter comprises a mode information indicating whether the first multi- iannel information β is applied to the processed downmix. The mode information iay corresponds to an information for selecting by a user. According to the mode lformation, the information generating information 1020 decides whether to •ansmit the first multi-channel parameter β or the second multi-channel parameter.
The downmix processing unit 1020 can be configured to determining a •rocessing scheme according to the mode information included in the mix nformation. Furthermore, the downmix processing unit 1020 can be configured to •rocess the downmix α according to the determined processing scheme. Then the lownmix processing unit 1020 transmits the processed downmix to multi-channel lecoder 1030.
The multi-channel decoder 1030 can be configured to receive either the first nulti-channel parameter β or the second multi-channel parameter. In case that iefault parameter β' is included in the bitstream, the multi-channel decoder 1030 :an use the default parameter β' instead of multi-channel parameter β.
Then, the multi-channel decoder 1030 can be configured to generate multi- :hannel output using the processed downmix signal and the received multichannel parameter. The multi-channel decoder 1030 may have the same configuration of the former multi-channel decoder 730, which does not put limitation on the present invention. 3. Binaural Processing
A multi-channel decoder can be operated in a binaural mode. This enables a αulti-channel impression over headphones by means of Head Related Transfer 'unction (HRTF) filtering. For binaural decoding side, the downmix signal and nulti-channel parameters are used in combination with HRTF filters supplied to he decoder.
FIG. 16 is an exemplary block diagram of an apparatus for processing an Ludio signal according to a third embodiment of present invention. Referring to 7IG. 16, an apparatus for processing an audio signal according to a third embodiment (hereinafter simply 'a decoder 1100') may comprise an information generating unit 1110, a downmix processing unit 1120, and a multi-channel iecoder 1130 with a sync matching part 1130a.
The information generating unit 1110 may have the same configuration of :he information generating unit 710 of FIG. 7, with generating dynamic HRTF. The iownmix processing unit 1120 may have the same configuration of the downmix processing unit 720 of FIG. 7. Like the preceding elements, multi-channel decoder 1130 except for the sync matching part 1130a is the same case of the former elements. Hence, details of the information generating unit 1110, the downmix processing unit 1120, and the multi-channel decoder 1130 shall be omitted. The dynamic HRTF describes the relation between object signals and virtual peaker signals corresponding to the HRTF azimuth and elevation angles, which is ime-dependent information according to real-time user control.
The dynamic HRTF may correspond to one of HTRF filter coefficients itself, •arameterized coefficient information, and index information in case that the aulti-channel decoder comprise all HRTF filter set.
There's need to match a dynamic HRTF information with frame of downmix ignal regardless of kind of the dynamic HRTF. In order to match HRTF αformation with downmix signal, it able to provide three type of scheme as ollows:
1) Inserting a tag information into each HRTF information and bitstream lownmix signal, then matching the HRTF with bitstream downmix signal based on he inserted tag information. In this scheme, it is proper that tag information may >e included in ancillary field in MPEG Surround standard. The tag information nay be represented as a time information, a counter information, a index nformation, etc.
2) Inserting HRTF information into frame of bitstream. In this scheme, it is possible to set to mode information indicating whether current frame corresponds o a default mode or not. If the default mode which describes HRTF information of irrent frame is equal to the HRTF information of previous frame is applied, it is ήe to reduce bitrates of HRTF information.
2-1) Furthermore, it is possible to define transmission information indicating rhether HRTF information of current frame has already transmitted. If the ansmission information which describes HRTF information of current frame is qual to the transmitted HRTF information of frame is applied, it is also possible to ≥duce bitrates of HRTF information.
3) Transmitting several HRTF informations in advance, then transmitting ientifying information indicating which HRTF among the transmitted HRTF nf ormations per each frame.
Furthermore, in case that HTRF coefficient varies suddenly, distortion may •e generated. In order to reduce this distortion, it is proper to perform smoothing »f coefficient or the rendered signal.
4. Rendering
FIG. 17 is an exemplary block diagram of an apparatus for processing an iudio signal according to a fourth embodiment of present invention. The apparatus :or processing an audio signal according to a fourth embodiment of present .nvention 1200 (hereinafter simply 'a processor 1200') may comprise an encoder 1210 at encoder side 1200A, and a rendering unit 1220 and a synthesis unit 1230 at ecoder side 1200B. The encoder 1210 can be configured to receive multi-channel bject signal and generate a downmix of audio signal and a side information. The ≥ndering unit 1220 can be configured to receive side information from the encoder 210, playback configuration and user control from a device setting or a user- iterface, and generate rendering information using the side information, playback onfiguration, and user control. The synthesis unit 1230 can be configured to ynthesis multi-channel output signal using the rendering information and the eceived downmix signal from an encoder 1210.
4.1 Applying effect-mode
The effect-mode is a mode for remixed or reconstructed signal. For example, ive mode, club band mode, karaoke mode, etc may be present. The effect-mode information may correspond to a mix parameter set generated by a producer, other iser, etc. If the effect-mode information is applied, an end user don't have to :ontrol object panning and object gain in full because user can select one of predetermined effect-mode informations.
Two methods of generating an effect-mode information can be distinguished. First of all, it is possible that an effect-mode information is generated by encoder 1200A and transmitted to the decoder 1200B. Secondly, the effect-mode information may be generated automatically at the decoder side. Details of two methods shall be described as follow. 4.1.1 Transmitting effect-mode information to decoder side
The effect-mode information may be generated at an encoder 1200A by a •roducer. According to this method, the decoder 1200B can be configured to eceive side information including the effect-mode information and output user- nterface by which a user can select one of effect-mode informations. The decoder 200B can be configured to generate output channel base on the selected effect- node information.
Furthermore, it is inappropriate to hear downmix signal as it is for a listener n case that encoder 1200A downmix the signal in order to raise quality of object iignals. However, if effect-mode information is applied in the decoder 1200B, it is possible to playback the downmix signal as the maximum quality.
4.1.2 Generating effect-mode information in decoder side
The effect-mode information may be generated at a decoder 1200B. The decoder 1200B can be configured to search appropriate effect-mode informations cor the downmix signal. Then the decoder 1200B can be configured to select one of the searched effect-mode by itself (automatic adjustment mode) or enable a user to select one of them (user selection mode). Then the decoder 1200B can be configured to obtain object information (number of objects, instrument names, etc) included in side information, and control object based on the selected effect-mode information and the object information. Furthermore, it is able to control similar objects in a lump. For example, αstruments associated with a rhythm may be similar objects in case of 'rhythm tnpression mode'. Controlling in a lump means controlling each object imultaneously rather than controlling objects using the same parameter.
Furthermore, it is able to control object based on the decoder setting and levice environment (including whether headphones or speakers). For example, )bject corresponding to main melody may be emphasized in case that volume setting of device is low, object corresponding to main melody may be repressed in :ase that volume setting of device is high.
4.2 Object type of input signal at encoder side
The input signal inputted to an encoder 1200A may be classified into three ;ypes as follow.
1) Mono object (mono channel object)
Mono object is most general type of object. It is possible to synthesis internal downmix signal by simply summing objects. It is also possible to synthesis internal downmix signal using object gain and object panning which may be one of user control and provided information. In generating internal downmix signal, it is also possible to generate rendering information using at least one of object characteristic, user input, and information provided with object. In case that external downmix signal is present, it is possible to extract and ansmit information indicating relation between external downmix and object.
2) Stereo object (stereo channel object)
It is possible to synthesis internal downmix signal by simply summing ejects like the case of the former mono object. It is also possible to synthesis Lternal downmix signal using object gain and object panning which may be one of ser control and provided information. In case that downmix signal corresponds to mono signal, it is possible that encoder 1200A use object converted into mono gnal for generating downmix signal. In this case, it is able to extract and transfer if ormation associated with object (ex: panning information in each time-frequency omain) in converting into mono signal. Like the preceding mono object, in enerating internal downmix signal, it is also possible to generate rendering ^formation using at least one of object characteristic, user input, and information >rovided with object. Like the preceding mono object, in case that external lownmix signal is present, it is possible to extract and transmit information ndicating relation between external downmix and object.
3) Multi-channel object
In case of multi-channel object, it is able to perform the above mentioned nethod described with mono object and stereo object. Furthermore, it is able to .nput multi-channel object as a form of MPEG Surround. In this case, it is able to snerate object-based downmix (ex: SAOC downmix) using object downmix lannel, and use multi-channel information (ex: spatial information in MPEG urround) for generating multi-channel information and rendering information, [ence, it is possible to reduce computing amount because multi-channel object resent in form of MPEG Surround don't have to decode and encode using object- riented encoder (ex: SAOC encoder). If object downmix corresponds to stereo and bject-based downmix (ex: SAOC downmix) corresponds to mono in this case, it is ossible to apply the above-mentioned method described with stereo object.
4) Transmitting scheme for variable type of object
As stated previously, variable type of object (mono object, stereo object, and multi-channel object) may be transmitted from the encoder 1200A to the decoder. 1200B. Transmitting scheme for variable type of object can be provided as follow:
Referring to FIG. 18, when the downmix includes a plural object, a side nf ormation includes information for each object. For example, when a plural object onsists of Nth mono object (A), left channel of N+lth object (B), and right channel )f N+lth object (C), a side information includes information for 3 objects (A, B, C).
The side information may comprise correlation flag information indicating Λrhether an object is part of a stereo or multi-channel object, for example, mono Λject, one channel (L or R) of stereo object, and so on. For example, correlation flag nformation is '0' if mono object is present, correlation flag information is '1' if one Lannel of stereo object is present. When one part of stereo object and the other part stereo object is transmitted in succession, correlation flag information for other irt of stereo object may be any value (ex: 1O', "Y1 or whatever). Furthermore, >rrelation flag information for other part of stereo object may be not transmitted.
Furthermore, in case of multi-channel object, correlation flag information for we part of multi-channel object may be value describing number of multi-channel bject. For example, in case of 5.1 channel object, correlation flag information for ft channel of 5.1 channel may be '5', correlation flag information for the other tiannel (R, Lr, Rr, C, LFE) of 5.1 channel may be either '0' or not transmitted.
4.3 Object attribute
Object may have the three kinds of attribute as follows: a) Single object
Single object can be configured as a source. It is able to apply one parameter o single object for controlling object panning and object gain in generating lownmix signal and reproducing. The 'one parameter' may mean not only one )arameter for all time/ frequency domain but also one parameter for each ime/ frequency slot. b) Grouped object
Single object can be configured as more than two sources. It is able to apply Dne parameter to grouped object for controlling object panning and object gain dthough grouped object is inputted as at least two sources. Details of the grouped )bject shall be explained with reference to FIG. 19 as follows: Referring to FIG. 19, m encoder 1300 includes a grouping unit 1310 and a downmix unit 1320. The grouping unit 1310 can be configured to group at least two objects among inputted nulti-object input, base on a grouping information. The grouping information may oe generated by producer at encoder side. The downmix unit 1320 can be :onfigured to generate downmix signal using the grouped object generated by the grouping unit 1310. The downmix unit 1320 can be configured to generate a side information for the grouped object. c) Combination object
Combination object is an object combined with at least one source. It is possible to control object panning and gain in a lump, but keep relation between combined objects unchanged. For example, in case of drum, it is possible to control drum, but keep relation between base drum, tam-tam, and symbol unchanged. For example, when base drum is located at center point and symbol is located at left point, it is possible to positioning base drum at right point and positioning symbol at point between center and right in case that drum is moved to right direction.
Relation information between combined objects may be transmitted to a decoder. On the other hand, decoder can extract the relation information using combination object. 4.4 Controlling objects hierarchically
It is able to control objects hierarchically. For example, after controlling a rum, it is able to control each sub-elements of drum. In order to control objects ierarchically, three schemes is provided as follows: a) UI (user interface)
Only representative element may be displayed without displaying all objects. : the representative element is selected by a user, all objects display. b) Object grouping
After grouping objects in order to represent representative element, it is >ossible to control representative element to control all objects grouped as epresentative element. Information extracted in grouping process may be ransmitted to a decoder. Also, the grouping information may be generated in a lecoder. Applying control information in a lump can be performed based on pre- ϊetermined control information for each element. c) Object configuration
It is possible to use the above-mentioned combination object. Information :oncerning element of combination object can be generated in either an encoder or a decoder. Information concerning elements from an encoder can be transmitted as a different form from information concerning combination object. It will be apparent to those skilled in the art that various modifications and iriations can be made in the present invention without departing from the spirit : scope of the inventions. Thus, it is intended that the present invention covers the Lodifications and variations of this invention provided they come within the scope E the appended claims and their equivalents. [Industrial Applicability]
Accordingly, the present invention is applicable to encode and decode an Lidio signal.

Claims

[CLAIMS]
1. A method for processing an audio signal, comprising: receiving a downmix signal in time domain; if the downmix signal corresponds to a mono signal, bypassing the downmix signal; if the number of channel of the downmix signal corresponds to at least two, decomposing the downmix signal into a subband signal, and processing the subband signal using a downmix processing information, wherein the downmix processing information is estimated based on an object information and a mix information.
2. The method of claim 1, wherein the number of channel of the downmix signal is equal to the number of channel of the processed downmix signal.
3. The method of claim 1, wherein the object information is included in a side information, and the side information includes a correlation, flag information indicating whether an object is part of at least two channel object.
4. The method of claim 1, wherein the object information includes at least one of an object level information and an object correlation information.
5. The method of claim 1, wherein the downmix processing information corresponds to an information for controlling object panning if the number of channel the downmix signal corresponds to at least two.
6. The method of claim 1, wherein the downmix processing information corresponds to an information for controlling object gain.
7. The method of claim 1, further comprising: generating a multi-channel signal using the processed subband signal.
8. The method of claim 7, further comprising generating a multi-channel information using the object information and the mix information, wherein the multi-channel signal is generated based on the multichannel information.
9. The method of claim 1, further comprising: downmixing the downmix signal to be a mono signal if the downmix signal corresponds to a stereo signal.
10. The method of claim 1, wherein the mix information is generated using at least one of an object position information and a playback configuration information.
11. The method of claim 1, wherein the downmix signal is received as a broadcast signal.
12. The method of claim 1, wherein the downmix signal is received on a digital medium.
13. A computer-readable medium having instructions stored thereon, which, when executed by a processor, causes the processor to perform operations, comprising: receiving a downmix signal in time domain; if the downmix signal corresponds to a mono signal, bypassing the downmix signal; if the number of channel of the downmix signal corresponds to at least two, decomposing the downmix signal into a subband signal, and processing the subband signal using a downmix processing information, wherein the downmix processing information is estimated based on an object information and a mix information.
14. An apparatus for processing an audio signal, comprising: a receiving unit receiving a downmix signal in time domain; and, a downmix processing unit bypassing the downmix signal if the downmix signal corresponds to a mono signal, and decomposing the downmix signal into a subband signal and processing the subband signal using a downmix processing information if the number of channel of the downmix signal corresponds to at least two, wherein the downmix processing information is estimated based on an object information and a mix information.
PCT/KR2007/006316 2006-12-07 2007-12-06 A method and an apparatus for processing an audio signal WO2008069594A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP07851287A EP2102856A4 (en) 2006-12-07 2007-12-06 A method and an apparatus for processing an audio signal
KR1020097014213A KR101100222B1 (en) 2006-12-07 2007-12-06 A method an apparatus for processing an audio signal
JP2009540164A JP5450085B2 (en) 2006-12-07 2007-12-06 Audio processing method and apparatus
CN2007800453353A CN101553865B (en) 2006-12-07 2007-12-06 A method and an apparatus for processing an audio signal

Applications Claiming Priority (20)

Application Number Priority Date Filing Date Title
US86907706P 2006-12-07 2006-12-07
US60/869,077 2006-12-07
US87713406P 2006-12-27 2006-12-27
US60/877,134 2006-12-27
US88356907P 2007-01-05 2007-01-05
US60/883,569 2007-01-05
US88404307P 2007-01-09 2007-01-09
US60/884,043 2007-01-09
US88434707P 2007-01-10 2007-01-10
US60/884,347 2007-01-10
US88458507P 2007-01-11 2007-01-11
US60/884,585 2007-01-11
US88534707P 2007-01-17 2007-01-17
US88534307P 2007-01-17 2007-01-17
US60/885,343 2007-01-17
US60/885,347 2007-01-17
US88971507P 2007-02-13 2007-02-13
US60/889,715 2007-02-13
US95539507P 2007-08-13 2007-08-13
US60/955,395 2007-08-13

Publications (1)

Publication Number Publication Date
WO2008069594A1 true WO2008069594A1 (en) 2008-06-12

Family

ID=39492395

Family Applications (5)

Application Number Title Priority Date Filing Date
PCT/KR2007/006317 WO2008069595A1 (en) 2006-12-07 2007-12-06 A method and an apparatus for processing an audio signal
PCT/KR2007/006319 WO2008069597A1 (en) 2006-12-07 2007-12-06 A method and an apparatus for processing an audio signal
PCT/KR2007/006318 WO2008069596A1 (en) 2006-12-07 2007-12-06 A method and an apparatus for processing an audio signal
PCT/KR2007/006316 WO2008069594A1 (en) 2006-12-07 2007-12-06 A method and an apparatus for processing an audio signal
PCT/KR2007/006315 WO2008069593A1 (en) 2006-12-07 2007-12-06 A method and an apparatus for processing an audio signal

Family Applications Before (3)

Application Number Title Priority Date Filing Date
PCT/KR2007/006317 WO2008069595A1 (en) 2006-12-07 2007-12-06 A method and an apparatus for processing an audio signal
PCT/KR2007/006319 WO2008069597A1 (en) 2006-12-07 2007-12-06 A method and an apparatus for processing an audio signal
PCT/KR2007/006318 WO2008069596A1 (en) 2006-12-07 2007-12-06 A method and an apparatus for processing an audio signal

Family Applications After (1)

Application Number Title Priority Date Filing Date
PCT/KR2007/006315 WO2008069593A1 (en) 2006-12-07 2007-12-06 A method and an apparatus for processing an audio signal

Country Status (11)

Country Link
US (11) US7986788B2 (en)
EP (6) EP2102857B1 (en)
JP (5) JP5290988B2 (en)
KR (5) KR101111521B1 (en)
CN (5) CN101568958B (en)
AU (1) AU2007328614B2 (en)
BR (1) BRPI0719884B1 (en)
CA (1) CA2670864C (en)
MX (1) MX2009005969A (en)
TW (1) TWI371743B (en)
WO (5) WO2008069595A1 (en)

Families Citing this family (103)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1691348A1 (en) * 2005-02-14 2006-08-16 Ecole Polytechnique Federale De Lausanne Parametric joint-coding of audio sources
JP4988717B2 (en) 2005-05-26 2012-08-01 エルジー エレクトロニクス インコーポレイティド Audio signal decoding method and apparatus
US8577686B2 (en) 2005-05-26 2013-11-05 Lg Electronics Inc. Method and apparatus for decoding an audio signal
US8082157B2 (en) * 2005-06-30 2011-12-20 Lg Electronics Inc. Apparatus for encoding and decoding audio signal and method thereof
AU2006266655B2 (en) * 2005-06-30 2009-08-20 Lg Electronics Inc. Apparatus for encoding and decoding audio signal and method thereof
JP4651668B2 (en) * 2005-07-11 2011-03-16 パナソニック株式会社 Ultrasonic flaw detection method and ultrasonic flaw detection apparatus
JP4787331B2 (en) * 2006-01-19 2011-10-05 エルジー エレクトロニクス インコーポレイティド Media signal processing method and apparatus
JP5054034B2 (en) * 2006-02-07 2012-10-24 エルジー エレクトロニクス インコーポレイティド Encoding / decoding apparatus and method
US8611547B2 (en) * 2006-07-04 2013-12-17 Electronics And Telecommunications Research Institute Apparatus and method for restoring multi-channel audio signal using HE-AAC decoder and MPEG surround decoder
WO2008069595A1 (en) * 2006-12-07 2008-06-12 Lg Electronics Inc. A method and an apparatus for processing an audio signal
US8634577B2 (en) * 2007-01-10 2014-01-21 Koninklijke Philips N.V. Audio decoder
CN101675472B (en) 2007-03-09 2012-06-20 Lg电子株式会社 A method and an apparatus for processing an audio signal
KR20080082916A (en) * 2007-03-09 2008-09-12 엘지전자 주식회사 A method and an apparatus for processing an audio signal
CN103299363B (en) * 2007-06-08 2015-07-08 Lg电子株式会社 A method and an apparatus for processing an audio signal
EP2191462A4 (en) 2007-09-06 2010-08-18 Lg Electronics Inc A method and an apparatus of decoding an audio signal
KR101461685B1 (en) 2008-03-31 2014-11-19 한국전자통신연구원 Method and apparatus for generating side information bitstream of multi object audio signal
KR101596504B1 (en) 2008-04-23 2016-02-23 한국전자통신연구원 / method for generating and playing object-based audio contents and computer readable recordoing medium for recoding data having file format structure for object-based audio service
WO2010008198A2 (en) * 2008-07-15 2010-01-21 Lg Electronics Inc. A method and an apparatus for processing an audio signal
WO2010008200A2 (en) * 2008-07-15 2010-01-21 Lg Electronics Inc. A method and an apparatus for processing an audio signal
EP2146522A1 (en) * 2008-07-17 2010-01-20 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus and method for generating audio output signals using object based metadata
EP2175670A1 (en) * 2008-10-07 2010-04-14 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Binaural rendering of a multi-channel audio signal
WO2010041877A2 (en) * 2008-10-08 2010-04-15 Lg Electronics Inc. A method and an apparatus for processing a signal
EP2356825A4 (en) * 2008-10-20 2014-08-06 Genaudio Inc Audio spatialization and environment simulation
US8861739B2 (en) 2008-11-10 2014-10-14 Nokia Corporation Apparatus and method for generating a multichannel signal
WO2010064877A2 (en) * 2008-12-05 2010-06-10 Lg Electronics Inc. A method and an apparatus for processing an audio signal
KR20100065121A (en) * 2008-12-05 2010-06-15 엘지전자 주식회사 Method and apparatus for processing an audio signal
JP5309944B2 (en) * 2008-12-11 2013-10-09 富士通株式会社 Audio decoding apparatus, method, and program
US8620008B2 (en) 2009-01-20 2013-12-31 Lg Electronics Inc. Method and an apparatus for processing an audio signal
KR101187075B1 (en) * 2009-01-20 2012-09-27 엘지전자 주식회사 A method for processing an audio signal and an apparatus for processing an audio signal
KR101137361B1 (en) 2009-01-28 2012-04-26 엘지전자 주식회사 A method and an apparatus for processing an audio signal
US8139773B2 (en) * 2009-01-28 2012-03-20 Lg Electronics Inc. Method and an apparatus for decoding an audio signal
US8255821B2 (en) * 2009-01-28 2012-08-28 Lg Electronics Inc. Method and an apparatus for decoding an audio signal
US20100324915A1 (en) * 2009-06-23 2010-12-23 Electronic And Telecommunications Research Institute Encoding and decoding apparatuses for high quality multi-channel audio codec
AU2010305717B2 (en) * 2009-10-16 2014-06-26 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus, method and computer program for providing one or more adjusted parameters for provision of an upmix signal representation on the basis of a downmix signal representation and a parametric side information associated with the downmix signal representation, using an average value
JP5719372B2 (en) 2009-10-20 2015-05-20 フラウンホーファー−ゲゼルシャフト・ツール・フェルデルング・デル・アンゲヴァンテン・フォルシュング・アインゲトラーゲネル・フェライン Apparatus and method for generating upmix signal representation, apparatus and method for generating bitstream, and computer program
KR101106465B1 (en) * 2009-11-09 2012-01-20 네오피델리티 주식회사 Method for adjusting gain of multiband drc system and multiband drc system using the same
ES2569779T3 (en) * 2009-11-20 2016-05-12 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus for providing a representation of upstream signal based on the representation of downlink signal, apparatus for providing a bit stream representing a multichannel audio signal, methods, computer programs and bit stream representing an audio signal multichannel using a linear combination parameter
WO2011071336A2 (en) * 2009-12-11 2011-06-16 한국전자통신연구원 Audio authoring apparatus and audio playback apparatus for an object-based audio service, and audio authoring method and audio playback method using same
EP2522016A4 (en) 2010-01-06 2015-04-22 Lg Electronics Inc An apparatus for processing an audio signal and method thereof
WO2011122589A1 (en) * 2010-03-29 2011-10-06 日立金属株式会社 Initial ultrafine crystal alloy, nanocrystal soft magnetic alloy and method for producing same, and magnetic component formed from nanocrystal soft magnetic alloy
KR20120004909A (en) * 2010-07-07 2012-01-13 삼성전자주식회사 Method and apparatus for 3d sound reproducing
EP2586025A4 (en) 2010-07-20 2015-03-11 Huawei Tech Co Ltd Audio signal synthesizer
US8948403B2 (en) * 2010-08-06 2015-02-03 Samsung Electronics Co., Ltd. Method of processing signal, encoding apparatus thereof, decoding apparatus thereof, and signal processing system
JP5903758B2 (en) * 2010-09-08 2016-04-13 ソニー株式会社 Signal processing apparatus and method, program, and data recording medium
EP3893521B1 (en) 2011-07-01 2024-06-19 Dolby Laboratories Licensing Corporation System and method for adaptive audio signal generation, coding and rendering
EP2560161A1 (en) 2011-08-17 2013-02-20 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Optimal mixing matrices and usage of decorrelators in spatial audio processing
CN103050124B (en) 2011-10-13 2016-03-30 华为终端有限公司 Sound mixing method, Apparatus and system
RU2618383C2 (en) * 2011-11-01 2017-05-03 Конинклейке Филипс Н.В. Encoding and decoding of audio objects
BR112014017457A8 (en) * 2012-01-19 2017-07-04 Koninklijke Philips Nv spatial audio transmission apparatus; space audio coding apparatus; method of generating spatial audio output signals; and spatial audio coding method
US9479886B2 (en) * 2012-07-20 2016-10-25 Qualcomm Incorporated Scalable downmix design with feedback for object-based surround codec
US9761229B2 (en) 2012-07-20 2017-09-12 Qualcomm Incorporated Systems, methods, apparatus, and computer-readable media for audio object clustering
CN104541524B (en) 2012-07-31 2017-03-08 英迪股份有限公司 A kind of method and apparatus for processing audio signal
KR20140017338A (en) * 2012-07-31 2014-02-11 인텔렉추얼디스커버리 주식회사 Apparatus and method for audio signal processing
JP6141978B2 (en) * 2012-08-03 2017-06-07 フラウンホーファー−ゲゼルシャフト・ツール・フェルデルング・デル・アンゲヴァンテン・フォルシュング・アインゲトラーゲネル・フェライン Decoder and method for multi-instance spatial acoustic object coding employing parametric concept for multi-channel downmix / upmix configuration
BR122021021487B1 (en) * 2012-09-12 2022-11-22 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e. V APPARATUS AND METHOD FOR PROVIDING ENHANCED GUIDED DOWNMIX CAPABILITIES FOR 3D AUDIO
US9385674B2 (en) * 2012-10-31 2016-07-05 Maxim Integrated Products, Inc. Dynamic speaker management for multichannel audio systems
BR112015013154B1 (en) * 2012-12-04 2022-04-26 Samsung Electronics Co., Ltd Audio delivery device, and audio delivery method
BR112015016593B1 (en) * 2013-01-15 2021-10-05 Koninklijke Philips N.V. APPLIANCE FOR PROCESSING AN AUDIO SIGNAL; APPARATUS TO GENERATE A BITS FLOW; AUDIO PROCESSING METHOD; METHOD FOR GENERATING A BITS FLOW; AND BITS FLOW
RU2656717C2 (en) 2013-01-17 2018-06-06 Конинклейке Филипс Н.В. Binaural audio processing
EP2757559A1 (en) * 2013-01-22 2014-07-23 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus and method for spatial audio object coding employing hidden objects for signal mixture manipulation
US9208775B2 (en) 2013-02-21 2015-12-08 Qualcomm Incorporated Systems and methods for determining pitch pulse period signal boundaries
US9497560B2 (en) 2013-03-13 2016-11-15 Panasonic Intellectual Property Management Co., Ltd. Audio reproducing apparatus and method
CN108806704B (en) 2013-04-19 2023-06-06 韩国电子通信研究院 Multi-channel audio signal processing device and method
KR102150955B1 (en) 2013-04-19 2020-09-02 한국전자통신연구원 Processing appratus mulit-channel and method for audio signals
EP2989631A4 (en) * 2013-04-26 2016-12-21 Nokia Technologies Oy Audio signal encoder
KR20140128564A (en) * 2013-04-27 2014-11-06 인텔렉추얼디스커버리 주식회사 Audio system and method for sound localization
CN105247611B (en) 2013-05-24 2019-02-15 杜比国际公司 To the coding of audio scene
CN105229731B (en) 2013-05-24 2017-03-15 杜比国际公司 Reconstruct according to lower mixed audio scene
WO2014187987A1 (en) * 2013-05-24 2014-11-27 Dolby International Ab Methods for audio encoding and decoding, corresponding computer-readable media and corresponding audio encoder and decoder
US9763019B2 (en) * 2013-05-29 2017-09-12 Qualcomm Incorporated Analysis of decomposed representations of a sound field
KR101454342B1 (en) * 2013-05-31 2014-10-23 한국산업은행 Apparatus for creating additional channel audio signal using surround channel audio signal and method thereof
CN105378826B (en) * 2013-05-31 2019-06-11 诺基亚技术有限公司 Audio scene device
EP2830334A1 (en) 2013-07-22 2015-01-28 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Multi-channel audio decoder, multi-channel audio encoder, methods, computer program and encoded audio representation using a decorrelation of rendered audio signals
BR112016001250B1 (en) 2013-07-22 2022-07-26 Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. MULTI-CHANNEL AUDIO DECODER, MULTI-CHANNEL AUDIO ENCODER, METHODS, AND AUDIO REPRESENTATION ENCODED USING A DECORRELATION OF RENDERED AUDIO SIGNALS
EP2830045A1 (en) 2013-07-22 2015-01-28 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Concept for audio encoding and decoding for audio channels and audio objects
EP2830047A1 (en) 2013-07-22 2015-01-28 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus and method for low delay object metadata coding
EP2830050A1 (en) * 2013-07-22 2015-01-28 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus and method for enhanced spatial audio object coding
US9319819B2 (en) * 2013-07-25 2016-04-19 Etri Binaural rendering method and apparatus for decoding multi channel audio
KR102243395B1 (en) * 2013-09-05 2021-04-22 한국전자통신연구원 Apparatus for encoding audio signal, apparatus for decoding audio signal, and apparatus for replaying audio signal
TWI713018B (en) 2013-09-12 2020-12-11 瑞典商杜比國際公司 Decoding method, and decoding device in multichannel audio system, computer program product comprising a non-transitory computer-readable medium with instructions for performing decoding method, audio system comprising decoding device
KR102163266B1 (en) 2013-09-17 2020-10-08 주식회사 윌러스표준기술연구소 Method and apparatus for processing audio signals
CN105659320B (en) * 2013-10-21 2019-07-12 杜比国际公司 Audio coder and decoder
EP2866227A1 (en) * 2013-10-22 2015-04-29 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Method for decoding and encoding a downmix matrix, method for presenting audio content, encoder and decoder for a downmix matrix, audio encoder and audio decoder
EP3062534B1 (en) 2013-10-22 2021-03-03 Electronics and Telecommunications Research Institute Method for generating filter for audio signal and parameterizing device therefor
US9933989B2 (en) 2013-10-31 2018-04-03 Dolby Laboratories Licensing Corporation Binaural rendering for headphones using metadata processing
EP2879131A1 (en) 2013-11-27 2015-06-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Decoder, encoder and method for informed loudness estimation in object-based audio coding systems
CN108922552B (en) 2013-12-23 2023-08-29 韦勒斯标准与技术协会公司 Method for generating a filter for an audio signal and parameterization device therefor
WO2015104447A1 (en) 2014-01-13 2015-07-16 Nokia Technologies Oy Multi-channel audio signal classifier
CN108600935B (en) 2014-03-19 2020-11-03 韦勒斯标准与技术协会公司 Audio signal processing method and apparatus
EP3128766A4 (en) 2014-04-02 2018-01-03 Wilus Institute of Standards and Technology Inc. Audio signal processing method and device
CN110636415B (en) 2014-08-29 2021-07-23 杜比实验室特许公司 Method, system, and storage medium for processing audio
US20170289724A1 (en) * 2014-09-12 2017-10-05 Dolby Laboratories Licensing Corporation Rendering audio objects in a reproduction environment that includes surround and/or height speakers
TWI587286B (en) 2014-10-31 2017-06-11 杜比國際公司 Method and system for decoding and encoding of audio signals, computer program product, and computer-readable medium
US9609383B1 (en) * 2015-03-23 2017-03-28 Amazon Technologies, Inc. Directional audio for virtual environments
US10504528B2 (en) 2015-06-17 2019-12-10 Samsung Electronics Co., Ltd. Method and device for processing internal channels for low complexity format conversion
CN111970630B (en) 2015-08-25 2021-11-02 杜比实验室特许公司 Audio decoder and decoding method
CN109427337B (en) 2017-08-23 2021-03-30 华为技术有限公司 Method and device for reconstructing a signal during coding of a stereo signal
TWI703557B (en) * 2017-10-18 2020-09-01 宏達國際電子股份有限公司 Sound reproducing method, apparatus and non-transitory computer readable storage medium thereof
DE102018206025A1 (en) * 2018-02-19 2019-08-22 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus and method for object-based spatial audio mastering
KR102471718B1 (en) * 2019-07-25 2022-11-28 한국전자통신연구원 Broadcastiong transmitting and reproducing apparatus and method for providing the object audio
WO2021034983A2 (en) * 2019-08-19 2021-02-25 Dolby Laboratories Licensing Corporation Steering of binauralization of audio
CN111654745B (en) * 2020-06-08 2022-10-14 海信视像科技股份有限公司 Multi-channel signal processing method and display device
US20240359870A1 (en) 2023-04-25 2024-10-31 Mabuchi Motor Co., Ltd. Packaging structure

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005086139A1 (en) * 2004-03-01 2005-09-15 Dolby Laboratories Licensing Corporation Multichannel audio coding
US20060115100A1 (en) * 2004-11-30 2006-06-01 Christof Faller Parametric coding of spatial audio with cues based on transmitted channels
US20060133618A1 (en) * 2004-11-02 2006-06-22 Lars Villemoes Stereo compatible multi-channel audio coding
JP2006323408A (en) * 2006-07-07 2006-11-30 Victor Co Of Japan Ltd Audio encoding method and audio decoding method

Family Cites Families (67)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3175209D1 (en) 1981-05-29 1986-10-02 Ibm Aspirator for an ink jet printer
FR2567984B1 (en) * 1984-07-20 1986-08-14 Centre Techn Ind Mecanique PROPORTIONAL HYDRAULIC DISTRIBUTOR
WO1992012607A1 (en) 1991-01-08 1992-07-23 Dolby Laboratories Licensing Corporation Encoder/decoder for multidimensional sound fields
US6141446A (en) * 1994-09-21 2000-10-31 Ricoh Company, Ltd. Compression and decompression system with reversible wavelets and lossy reconstruction
US5838664A (en) * 1997-07-17 1998-11-17 Videoserver, Inc. Video teleconferencing system with digital transcoding
US5956674A (en) * 1995-12-01 1999-09-21 Digital Theater Systems, Inc. Multi-channel predictive subband audio coder using psychoacoustic adaptive bit allocation in frequency, time and over the multiple channels
EP0798866A2 (en) 1996-03-27 1997-10-01 Kabushiki Kaisha Toshiba Digital data processing system
US6128597A (en) 1996-05-03 2000-10-03 Lsi Logic Corporation Audio decoder with a reconfigurable downmixing/windowing pipeline and method therefor
US5912976A (en) 1996-11-07 1999-06-15 Srs Labs, Inc. Multi-channel audio enhancement system for use in recording and playback and methods for providing same
US6131084A (en) 1997-03-14 2000-10-10 Digital Voice Systems, Inc. Dual subframe quantization of spectral magnitudes
DE69817181T2 (en) 1997-06-18 2004-06-17 Clarity, L.L.C., Ann Arbor METHOD AND DEVICE FOR BLIND SEPARATING SIGNALS
US6026168A (en) * 1997-11-14 2000-02-15 Microtek Lab, Inc. Methods and apparatus for automatically synchronizing and regulating volume in audio component systems
EP1072036B1 (en) * 1998-04-15 2004-09-22 STMicroelectronics Asia Pacific Pte Ltd. Fast frame optimisation in an audio encoder
US6122619A (en) 1998-06-17 2000-09-19 Lsi Logic Corporation Audio decoder with programmable downmixing of MPEG/AC-3 and method therefor
FI114833B (en) * 1999-01-08 2004-12-31 Nokia Corp A method, a speech encoder and a mobile station for generating speech coding frames
US7103187B1 (en) 1999-03-30 2006-09-05 Lsi Logic Corporation Audio calibration system
US6539357B1 (en) 1999-04-29 2003-03-25 Agere Systems Inc. Technique for parametric coding of a signal containing information
NZ521411A (en) * 2000-03-03 2004-03-26 Cardiac M Magnetic resonance specimen analysis apparatus
JP4870896B2 (en) 2000-07-19 2012-02-08 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ Multi-channel stereo converter to obtain stereo surround and / or audio center signal
US7292901B2 (en) * 2002-06-24 2007-11-06 Agere Systems Inc. Hybrid multi-channel/cue coding/decoding of audio signals
US7583805B2 (en) 2004-02-12 2009-09-01 Agere Systems Inc. Late reverberation-based synthesis of auditory scenes
SE0202159D0 (en) * 2001-07-10 2002-07-09 Coding Technologies Sweden Ab Efficientand scalable parametric stereo coding for low bitrate applications
US7032116B2 (en) 2001-12-21 2006-04-18 Intel Corporation Thermal management for computer systems running legacy or thermal management operating systems
ATE426235T1 (en) 2002-04-22 2009-04-15 Koninkl Philips Electronics Nv DECODING DEVICE WITH DECORORATION UNIT
BR0304542A (en) 2002-04-22 2004-07-20 Koninkl Philips Electronics Nv Method and encoder for encoding a multichannel audio signal, apparatus for providing an audio signal, encoded audio signal, storage medium, and method and decoder for decoding an audio signal
JP4013822B2 (en) 2002-06-17 2007-11-28 ヤマハ株式会社 Mixer device and mixer program
BR0305555A (en) 2002-07-16 2004-09-28 Koninkl Philips Electronics Nv Method and encoder for encoding an audio signal, apparatus for providing an audio signal, encoded audio signal, storage medium, and method and decoder for decoding an encoded audio signal
KR100542129B1 (en) * 2002-10-28 2006-01-11 한국전자통신연구원 Object-based three dimensional audio system and control method
JP4084990B2 (en) 2002-11-19 2008-04-30 株式会社ケンウッド Encoding device, decoding device, encoding method and decoding method
JP4496379B2 (en) 2003-09-17 2010-07-07 財団法人北九州産業学術推進機構 Reconstruction method of target speech based on shape of amplitude frequency distribution of divided spectrum series
US6937737B2 (en) 2003-10-27 2005-08-30 Britannia Investment Corporation Multi-channel audio surround sound from front located loudspeakers
TWI233091B (en) * 2003-11-18 2005-05-21 Ali Corp Audio mixing output device and method for dynamic range control
US7394903B2 (en) * 2004-01-20 2008-07-01 Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. Apparatus and method for constructing a multi-channel output signal or for generating a downmix signal
US7805313B2 (en) * 2004-03-04 2010-09-28 Agere Systems Inc. Frequency-based coding of channels in parametric multi-channel coding systems
SE0400998D0 (en) 2004-04-16 2004-04-16 Cooding Technologies Sweden Ab Method for representing multi-channel audio signals
SE0400997D0 (en) * 2004-04-16 2004-04-16 Cooding Technologies Sweden Ab Efficient coding or multi-channel audio
US8843378B2 (en) 2004-06-30 2014-09-23 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Multi-channel synthesizer and method for generating a multi-channel output signal
CA2572805C (en) 2004-07-02 2013-08-13 Matsushita Electric Industrial Co., Ltd. Audio signal decoding device and audio signal encoding device
US7391870B2 (en) 2004-07-09 2008-06-24 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E V Apparatus and method for generating a multi-channel output signal
KR100745688B1 (en) 2004-07-09 2007-08-03 한국전자통신연구원 Apparatus for encoding and decoding multichannel audio signal and method thereof
WO2006006809A1 (en) 2004-07-09 2006-01-19 Electronics And Telecommunications Research Institute Method and apparatus for encoding and cecoding multi-channel audio signal using virtual source location information
KR100663729B1 (en) 2004-07-09 2007-01-02 한국전자통신연구원 Method and apparatus for encoding and decoding multi-channel audio signal using virtual source location information
ATE444549T1 (en) * 2004-07-14 2009-10-15 Koninkl Philips Electronics Nv SOUND CHANNEL CONVERSION
KR101147187B1 (en) 2004-07-14 2012-07-09 돌비 인터네셔널 에이비 Method, device, encoder apparatus, decoder apparatus and audio system
JP4892184B2 (en) * 2004-10-14 2012-03-07 パナソニック株式会社 Acoustic signal encoding apparatus and acoustic signal decoding apparatus
US7720230B2 (en) * 2004-10-20 2010-05-18 Agere Systems, Inc. Individual channel shaping for BCC schemes and the like
US8204261B2 (en) 2004-10-20 2012-06-19 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Diffuse sound shaping for BCC schemes and the like
SE0402652D0 (en) * 2004-11-02 2004-11-02 Coding Tech Ab Methods for improved performance of prediction based multi-channel reconstruction
KR100682904B1 (en) 2004-12-01 2007-02-15 삼성전자주식회사 Apparatus and method for processing multichannel audio signal using space information
US7903824B2 (en) 2005-01-10 2011-03-08 Agere Systems Inc. Compact side information for parametric coding of spatial audio
EP1691348A1 (en) 2005-02-14 2006-08-16 Ecole Polytechnique Federale De Lausanne Parametric joint-coding of audio sources
US8346564B2 (en) * 2005-03-30 2013-01-01 Koninklijke Philips Electronics N.V. Multi-channel audio coding
US20060262936A1 (en) 2005-05-13 2006-11-23 Pioneer Corporation Virtual surround decoder apparatus
KR20060122694A (en) * 2005-05-26 2006-11-30 엘지전자 주식회사 Method of inserting spatial bitstream in at least two channel down-mix audio signal
WO2006126856A2 (en) 2005-05-26 2006-11-30 Lg Electronics Inc. Method of encoding and decoding an audio signal
BRPI0611505A2 (en) 2005-06-03 2010-09-08 Dolby Lab Licensing Corp channel reconfiguration with secondary information
US20070055510A1 (en) * 2005-07-19 2007-03-08 Johannes Hilpert Concept for bridging the gap between parametric multi-channel audio coding and matrixed-surround multi-channel coding
RU2414741C2 (en) 2005-07-29 2011-03-20 ЭлДжи ЭЛЕКТРОНИКС ИНК. Method of generating multichannel signal
US20070083365A1 (en) 2005-10-06 2007-04-12 Dts, Inc. Neural network classifier for separating audio sources from a monophonic audio signal
EP1640972A1 (en) 2005-12-23 2006-03-29 Phonak AG System and method for separation of a users voice from ambient sound
CN101356573B (en) 2006-01-09 2012-01-25 诺基亚公司 Control for decoding of binaural audio signal
JP5134623B2 (en) 2006-07-07 2013-01-30 フラウンホッファー−ゲゼルシャフト ツァ フェルダールング デァ アンゲヴァンテン フォアシュンク エー．ファオ Concept for synthesizing multiple parametrically encoded sound sources
BRPI0716854B1 (en) 2006-09-18 2020-09-15 Koninklijke Philips N.V. ENCODER FOR ENCODING AUDIO OBJECTS, DECODER FOR DECODING AUDIO OBJECTS, TELECONFERENCE DISTRIBUTOR CENTER, AND METHOD FOR DECODING AUDIO SIGNALS
AU2007300814B2 (en) * 2006-09-29 2010-05-13 Lg Electronics Inc. Methods and apparatuses for encoding and decoding object-based audio signals
UA94117C2 (en) * 2006-10-16 2011-04-11 Долби Свиден Ав Improved coding and parameter dysplaying of mixed object multichannel coding
EP2437257B1 (en) 2006-10-16 2018-01-24 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Saoc to mpeg surround transcoding
WO2008069595A1 (en) 2006-12-07 2008-06-12 Lg Electronics Inc. A method and an apparatus for processing an audio signal

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005086139A1 (en) * 2004-03-01 2005-09-15 Dolby Laboratories Licensing Corporation Multichannel audio coding
US20060133618A1 (en) * 2004-11-02 2006-06-22 Lars Villemoes Stereo compatible multi-channel audio coding
US20060115100A1 (en) * 2004-11-30 2006-06-01 Christof Faller Parametric coding of spatial audio with cues based on transmitted channels
JP2006323408A (en) * 2006-07-07 2006-11-30 Victor Co Of Japan Ltd Audio encoding method and audio decoding method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2102856A4 *

Also Published As

Publication number Publication date
MX2009005969A (en) 2009-06-16
JP5290988B2 (en) 2013-09-18
US20100010818A1 (en) 2010-01-14
WO2008069593A1 (en) 2008-06-12
KR20090098864A (en) 2009-09-17
EP2102856A1 (en) 2009-09-23
EP2122613B1 (en) 2019-01-30
EP2187386A3 (en) 2010-07-28
US20100010821A1 (en) 2010-01-14
US7783048B2 (en) 2010-08-24
US8005229B2 (en) 2011-08-23
EP2122613A4 (en) 2010-01-13
EP2102858A1 (en) 2009-09-23
US20100014680A1 (en) 2010-01-21
US7986788B2 (en) 2011-07-26
CA2670864A1 (en) 2008-06-12
KR20090098866A (en) 2009-09-17
US20080205657A1 (en) 2008-08-28
CN101553867A (en) 2009-10-07
US8488797B2 (en) 2013-07-16
BRPI0719884A2 (en) 2014-02-11
CN101553865A (en) 2009-10-07
JP5302207B2 (en) 2013-10-02
AU2007328614A1 (en) 2008-06-12
US7783051B2 (en) 2010-08-24
CN101553866A (en) 2009-10-07
JP5270566B2 (en) 2013-08-21
WO2008069596A1 (en) 2008-06-12
KR101128815B1 (en) 2012-03-27
US8428267B2 (en) 2013-04-23
CN101553865B (en) 2012-01-25
JP2010511912A (en) 2010-04-15
CN101553866B (en) 2012-05-30
CN101553868A (en) 2009-10-07
EP2187386A2 (en) 2010-05-19
JP2010511909A (en) 2010-04-15
US20100010819A1 (en) 2010-01-14
EP2122612B1 (en) 2018-08-15
EP2122612A1 (en) 2009-11-25
JP2010511911A (en) 2010-04-15
KR101111521B1 (en) 2012-03-13
EP2122612A4 (en) 2010-01-13
US20080205670A1 (en) 2008-08-28
TWI371743B (en) 2012-09-01
CN101553868B (en) 2012-08-29
JP2010511910A (en) 2010-04-15
EP2102857A4 (en) 2010-01-20
KR101100222B1 (en) 2011-12-28
JP5450085B2 (en) 2014-03-26
AU2007328614B2 (en) 2010-08-26
EP2102858A4 (en) 2010-01-20
US20090281814A1 (en) 2009-11-12
WO2008069595A1 (en) 2008-06-12
EP2122613A1 (en) 2009-11-25
BRPI0719884B1 (en) 2020-10-27
JP5209637B2 (en) 2013-06-12
CA2670864C (en) 2015-09-29
US20080205671A1 (en) 2008-08-28
KR101100223B1 (en) 2011-12-28
KR20090098863A (en) 2009-09-17
EP2102857B1 (en) 2018-07-18
TW200834544A (en) 2008-08-16
CN101568958B (en) 2012-07-18
EP2102857A1 (en) 2009-09-23
US8311227B2 (en) 2012-11-13
US7783050B2 (en) 2010-08-24
US8340325B2 (en) 2012-12-25
EP2187386B1 (en) 2020-02-05
EP2102856A4 (en) 2010-01-13
KR20090100386A (en) 2009-09-23
CN101568958A (en) 2009-10-28
US20080192941A1 (en) 2008-08-14
US20100010820A1 (en) 2010-01-14
US7783049B2 (en) 2010-08-24
WO2008069597A1 (en) 2008-06-12
US7715569B2 (en) 2010-05-11
CN101553867B (en) 2013-04-17
JP2010511908A (en) 2010-04-15
KR20090098865A (en) 2009-09-17
KR101111520B1 (en) 2012-05-24
US20080199026A1 (en) 2008-08-21

Similar Documents

Publication Publication Date Title
EP2102856A1 (en) A method and an apparatus for processing an audio signal

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200780045335.3

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07851287

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2009540164

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: KR

Ref document number: 1020097014213

Country of ref document: KR

Ref document number: 2007851287

Country of ref document: EP