US9595266B2 - Audio encoding/decoding device using reverberation signal of object audio signal - Google Patents

Abstract

An audio coding and decoding apparatus is disclosed. The audio coding apparatus may include an audio signal encoding unit to encode an audio signal; and a bitstream transmission unit to convert the audio signal into a bitstream and transmit the bitstream, wherein the audio signal comprises a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal.

Description

TECHNICAL FIELD

The present invention relates to au audio coding and decoding apparatus using a reverberation signal of an object audio signal, and more particularly, to an audio coding and decoding apparatus which encodes and decodes audio using an audio signal including a reverberation signal of an object audio signal.

BACKGROUND ART

According to conventional methods, moving picture expert group (MPEG) spatial audio object coding (SAOC) and Dolby Atmos construct a sound scene using an input signal or an object, respectively.

MPEG SAOC considers an input audio signal as an object and receives the input audio signal. In addition, MPEG SAOC constructs the sound scene only with respect to input rendering information. In particular, MPEG SAOC is capable of transmission at a low bit rate and uses a spatial audio coding method as a high compression method.

Dolby Atmos refers to a multichannel audio format for theatres. Dolby Atmos transmits or stores a channel signal called ‘Beds’ and an object signal called ‘object’ and constructs the sound scene using metadata.

However, since the foregoing conventional methods construct the sound scene using the input audio signal or the object signal, in some cases, a sound scene not corresponding to an intention of content according to the input audio signal or the object signal may be included. This is because only base signals for constructing the sound scene are included.

Accordingly, there is a need for a method of constructing a more accurate sound scene corresponding to the intention of content according to the input audio signal or the object signal.

DISCLOSURE OF INVENTION Technical Goals

An aspect of the present invention provides an audio coding and decoding apparatus capable of reproducing an audio signal more efficiently and realistically, using a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal.

Another aspect of the present invention provides an audio coding and decoding apparatus capable of reconstructing a realistic sound scene according to a reverberation signal of an object audio signal, by rendering the object audio signal and the reverberation signal of the object audio signal.

Technical Solutions

According to an aspect of the present invention, there is provided an audio coding apparatus including an audio signal encoding unit to encode an audio signal, and a bitstream transmission unit to convert the audio signal into a bitstream and transmit the bitstream, wherein the audio signal comprises a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal.

According to an aspect of the present invention, there is provided an audio decoding apparatus including a bitstream receiving unit to receive a bitstream including an encoded audio signal, and an audio signal decoding unit to extract a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal from the bitstream by decoding the audio signal included in the bitstream.

The audio decoding apparatus may further include an audio rendering unit to render the extracted channel audio signal, object audio signal, and reverberation signal of the object audio signal based on the rendering information included in the bitstream.

According to an aspect of the present invention, there is provided an audio coding method including encoding an audio signal, and converting the audio signal into a bitstream and transmitting the bitstream, wherein the audio signal comprises a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal.

According to an aspect of the present invention, there is provided an audio decoding method including receiving a bitstream including an encoded audio signal, extracting a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal from the bitstream by decoding the audio signal included in the bitstream, and rendering the extracted channel audio signal, object audio signal, and reverberation signal of the object audio signal based on rendering information included in the bitstream.

Effects of Invention

According to an embodiment, an audio coding and decoding apparatus may be capable of reproducing an audio signal more efficiently and realistically, by using a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal, in reproducing a multichannel audio signal.

According to an embodiment, an audio coding and decoding apparatus may be to capable of reconstructing a realistic sound scene according to a reverberation signal of an object audio signal, by rendering the object audio signal and the reverberation signal of the object audio signal.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an audio coding and decoding apparatus according to an embodiment.

FIG. 2 is a diagram illustrating an audio coding apparatus according to an embodiment.

FIG. 3 is a diagram illustrating an audio decoding apparatus according to an embodiment.

FIG. 4 is a diagram illustrating the audio coding apparatus of FIG. 2 in detail.

FIG. 5 is a diagram illustrating the audio decoding apparatus of FIG. 3 in detail.

FIG. 6 is a diagram illustrating a configuration of rendering information.

FIG. 7 is a diagram illustrating an audio coding method according to an embodiment.

FIG. 8 is a diagram illustrating an audio decoding method according to an embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiments are described below in order to explain the present invention by referring to the figures.

FIG. 1 is a diagram illustrating an audio coding and decoding apparatus according to an embodiment.

Referring to FIG. 1, an audio coding apparatus 101 may receive an audio signal which includes a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal. Here, the audio coding apparatus 101 may receive the audio signal by considering the channel audio signal, the object audio signal, and the reverberation signal of the object audio signal as an object. The audio coding apparatus 101 is necessary to receive the audio signal including the foregoing three types of audio signal.

In addition, the audio coding apparatus 101 may receive rendering information. The rendering information, as additional data, may include rendering information based on a gain value and rendering information related to a time delay. In case of outputting the audio signal, the rendering information may a sound scene corresponding to the audio signal.

The audio coding apparatus 101 may encode the received audio signal, and convert the rendering information into a bit string. For example, the audio coding apparatus 101 may perform binary conversion to convert the rendering information into the bit string. In addition, the audio coding apparatus 101 may encode the audio signal and the rendering information simultaneously. Here, the audio coding apparatus 101 may include a block for converting the rendering information into the bit string.

The audio coding apparatus 101 may convert the encoded audio signal into the bitstream. The audio coding apparatus 101 may include a block capable of converting the rendering information into the bit string. The audio coding apparatus 101 may convert the rendering information and the encoded audio signal into the bitstream. The bitstream may include the rendering information and the encoded audio signal. In addition, the audio coding apparatus 101 may transmit the bitstream to an audio decoding apparatus 102.

The audio decoding apparatus 102 may receive the bitstream from the audio coding apparatus 101. The audio decoding apparatus 102 may extract the channel audio signal, the object audio signal, and the reverberation signal of the object audio signal from the bitstream by decoding the audio signal included in the received bitstream. Additionally, the audio decoding apparatus 102 may render the extracted audio signal, object audio signal, and reverberation signal of the object audio signal, based on the rendering information included in the bitstream. The audio decoding apparatus 102 may output a rendered multichannel audio signal.

FIG. 2 is a diagram illustrating an audio coding apparatus 201 according to an embodiment.

Referring to FIG. 2, the audio coding apparatus 201 may include an audio signal encoding unit 202 and a bitstream transmission unit 203.

The audio signal encoding unit 202 may encode the audio signal. The audio signal may include a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal.

The channel audio signal may be a generally used channel audio signal and allocated to a channel of a random reproduction device when reproduced. Here, the channel audio signal may be a signal not varied by the rendering information. The channel audio signal may be expressed by a vector stream with respect to an N-number of channel audio signals using Equation 1.
X _ch =[x ₁ ^ch ,x ₂ ^ch , . . . ,x _N-1 ^ch]^T[Equation 1]

The object audio signal may determine a particular audio signal among a plurality of audio signals as the object audio signal, and use the object audio signal as a subject to perform rendering. Here, the object audio signal may be a signal that may be defined in a predetermined spot through geometry analysis of the reproduction device. The object audio signal may be expressed by a matrix constituted by vector streams with respect to an M-number of object audio signals using Equation 2.
X _obj =[x ₁ ^obj ,x ₂ ^obj , . . . ,x _M-1 ^obj]^T[Equation 2]

Here, Equation 2 may be used when rendering is performed independently from location information and delay information related to the object audio signal.

Here, the object audio signal may be expressed by the matrix because each object audio signal may include a plurality of channel audio signals. For example, when a first object audio signal x₁ ^obj of the object audio signal includes stereo, the object audio signal may be expressed by Equation 3.
x ₁ ^obj =[x ₁ ^obj,1 ,x _r ^obj,1]  [Equation 3]

A reverberation signal of the object audio signal is a reverberation signal applied to the object audio signal, which expresses a sound field feeling of the object audio signal. The reverberation signal of the object audio signal may include reverberation signals of the M-number of object audio signals, corresponding to the object audio signal. The reverberation signal of the object audio signal may be expressed by Equation 4.
X _rev =[x ₁ ^rev ,x ₂ ^rev , . . . ,x _M-1 ^rev]^T[Equation 4]

In addition, in the same manner as the object audio signal, the reverberation signal of the object audio signal may include a plurality of channel audio signals. For example, the reverberation signal of the object audio signal including five 5.1 channels may be expressed by Equation 5.
x ₁ ^rev =[x ₁ ^rev,1 ,x _r ^rev,1 ,x _c ^rev,1 ,x _ls ^rev,1 ,x _rs ^rev,1]^T[Equation 5]

Here, the audio signal encoding unit 202 may encode the audio signal by including a reverberation signal having various layouts with respect to the object audio signal.

The bitstream transmission unit 203 may convert the encoded audio signal into a bitstream. The bitstream transmission unit 203 may generate the bitstream from the encoded audio signal and the rendering information for outputting the audio signal. The rendering information may be additional data with respect to the audio signal. That is, the rendering information may be information applied to the audio signal to reproduce scene information related to a sound. The rendering information may include location information of an audio object, sound pressure information of the audio object, and delay information of the audio object. The rendering information may be expressed by Equation 6.
R(t)=P(t)G _p(t)+D(t)G _d(t)  [Equation 6]

R(t) may refer to the location information of the object audio signal. G_i(t) may refer to the sound pressure of the object audio signal. D(t) may refer to the delay of the object audio signal. G₁(t) and G₂(t) may be scale matrices for controlling the sound pressure with respect to the object audio signal. In addition, t may refer to an index related to time.

When rendering is performed with respect to the location information and the delay information simultaneously, the rendering may be expressed by Equation 7.
R(t)=PD(t)G _pd(t)  [Equation 7]

The bitstream transmission 203 may transmit the bitstream to the audio decoding apparatus.

FIG. 3 is a diagram illustrating an audio decoding apparatus 301 according to an embodiment.

Referring to FIG. 3, the audio decoding apparatus 301 may include a bitstream receiving unit 302, an audio signal decoding unit 303, an audio rendering unit 304.

The bitstream receiving unit 302 may receive a bitstream including an encoded audio signal from an audio coding apparatus.

The audio signal decoding unit 303 may decode the audio signal included in the bitstream. In detail, the audio signal decoding unit 303 may extract a channel audio signal, an object audio signal, and a reverberation signal of the audio signal from the bitstream. For example, the audio signal decoding unit 303 may be expressed by Equation 8, Equation 9, and Equation 10, corresponding to the extracted channel audio signal, object audio signal, and reverberation signal of the object audio signal.
x _ch =[X ₁ ^ch ,x ₁ ^ch , . . . ,x _N-1 ^ch]^T  [Equation 8]
x _obj =[x ₁ ^obj ,x ₂ ^obj , . . . x _M-1 ^obj]^T  [Equation 9]
x _rev =[x ₁ ^rev ,x ₂ ^rev , . . . ,x _M-1 ^rev]^T  [Equation 10]

The audio rendering unit 304 may render the extracted channel audio signal, object audio signal, and reverberation signal of the object audio signal, based on the rendering information included in the bitstream. The audio rendering unit 304 may construct a sound scene based on scene information related to the sound of the rendering information.

In detail, the audio rendering unit 304 may express a principle of rendering of the audio signal by Equation 11.

$\begin{array}{cc}R\left(t\right)·X_{\mathrm{obj}}=\left[P\left(t\right)G_p\left(t\right)+D\left(t\right)G_d\left(t\right)\right]·X_{\mathrm{obj}}=\underset{\underset{1}{︸}}{P\left(t\right)\left[G_p\left(t\right)·X_{\mathrm{obj}}\right]}+\underset{\underset{2}{︸}}{D\left(t\right)·\left[G_d\left(t\right)·X_{\mathrm{obj}}\right]}& \left[\mathrm{Equation}11\right]\end{array}$

A process of applying a first term of Equation 11 will be described. The sound pressure of the object audio signal may be controlled. The process of controlling the object audio signal may be expressed by Equation 12.

$\begin{array}{cc}{G}_p\left(t\right)·X_{\mathrm{obj}}=\left[\begin{array}{ccc}{g}_{p,0}& \dots & 0\\ ⋮& \ddots & ⋮\\ 0& \dots & g_{p,M-1}\end{array}\right]\hspace{1em}\left[\begin{array}{c}{x}_0^{\mathrm{obj}}\\ ⋮\\ x_{M-1}^{\mathrm{obj}}\end{array}\right]=\left[\begin{array}{c}{g}_{p,0}·x_0^{\mathrm{obj}}\\ ⋮\\ g_{p,M-1}·x_{M-1}^{\mathrm{obj}}\end{array}\right]=X_{\mathrm{obj}}^{\prime }& \left[\mathrm{Equation}12\right]\end{array}$

x′_obj with the sound pressure controlled may be allocated to a speaker position of a reproduction device, where output is actually performed by a sound image localization matrix P(t). Elements of the sound image localization matrix P(t) may be expressed by gain values of the sound pressure. Here, the gain value may include a real number between 0 and 1. In addition, when a number of channels capable of outputting is N, x′_obj may be applied to the image localization matrix as in Equation 13.

$\begin{array}{cc}P\left(t\right)G_p\left(t\right)·X_{\mathrm{obj}}=\hspace{1em}\left[\begin{array}{ccccc}{p}_{0,0}& p_{0,1}& \dots & & p_{0,M-1}\\ p_{1,0}& \ddots & & & \\ ⋮& & p_{i,j}& & ⋮\\ & & & \ddots & \\ p_{N-1,0}& & \dots & & p_{N-1,M-1}\end{array}\right]\left[\begin{array}{c}{g}_{p,0}·x_0^{\mathrm{obj}}\\ ⋮\\ g_{p,j}{x}_j^{\mathrm{obj}}\\ ⋮\\ g_{p,M-1}·x_{M-1}^{\mathrm{obj}}\end{array}\right]& \left[\mathrm{Equation}13\right]\end{array}$

In Equation 13, when the object audio signal x_j ^obj includes a J-number of layouts, the object audio signal x_j ^obj may be expressed by Equation 14.
x _j ^obj =[x ₀ ^obj , . . . ,x _J-1 ^obj]^T[Equation 14]

As to the sound image localization matrix, calculation process of each element of the sound image localization matrix may be described through Equation 15.

$\begin{array}{cc}{p}_{i,j}·x_j^{\mathrm{obj}}=\left[\begin{array}{ccc}{p}_0^{i,j}& \dots & p_{L-1}^{i,j}\end{array}\right]\left[\begin{array}{c}{x}_0^{\mathrm{obj},j}\\ ⋮\\ x_{L-1}^{\mathrm{obj},j}\end{array}\right]=\sum _{l=0}^{L-1}{p}_l^{i,j}·x_l^{\mathrm{obj},j}& \left[\mathrm{Equation}15\right]\end{array}$

Therefore, a signal output by the sound image localization matrix P(t) may be expressed by Equation 16.

$\begin{array}{cc}P\left(t\right)G_1\left(t\right)·X_{\mathrm{obj}}=\hspace{1em}\hspace{1em}\left[\begin{array}{ccccc}{p}_{0,0}& p_{0,1}& \dots & & p_{0,M-1}\\ p_{1,0}& \ddots & & & \\ ⋮& & p_{i,j}& & ⋮\\ & & & \ddots & \\ p_{N-1,0}& & \dots & & p_{N-1,M-1}\end{array}\right]\hspace{1em}\left[\begin{array}{c}{g}_{p,0}·x_0^{\mathrm{obj}}\\ ⋮\\ g_{p,j}{x}_j^{\mathrm{obj}}\\ ⋮\\ g_{p,M-1}·x_{M-1}^{\mathrm{obj}}\end{array}\right]=\hspace{1em}\left[\begin{array}{c}{g}_{p,0}(\sum _{l=0}^{L-1}{p}_l^{0,0}·x_l^{\mathrm{obj},0}+\dots +\\ \sum _{l=0}^{L-1}{p}_l^{0,M-1}·x_l^{\mathrm{obj},M-1})\\ ⋮\\ g_{p,j}(\sum _{l=0}^{L-1}{p}_l^{i,0}·x_l^{\mathrm{obj},0}+\dots +\\ \sum _{l=0}^{L-1}{p}_l^{i,M-1}·x_l^{\mathrm{obj},M-1})\\ ⋮\\ g_{p,M-1}(\sum _{l=0}^{L-1}{p}_l^{N-1,0}·x_l^{\mathrm{obj},0}+\dots +\\ \sum _{l=0}^{L-1}{p}_l^{N-1,M-1}·x_l^{\mathrm{obj},M-1})\end{array}\right]& \left[\mathrm{Equation}16\right]\end{array}$

A second term of Equation 10 may perform matrix calculation of a same dimension. The matrix calculation of the dimension may be expressed by Equation 17.

$\begin{array}{cc}D\left(t\right)G_d\left(t\right)·X_{\mathrm{obj}}=\hspace{1em}\left[\begin{array}{ccccc}{d}_{0,0}& d_{0,1}& \dots & & d_{0,M-1}\\ d_{1,0}& \ddots & & & \\ ⋮& & d_{i,j}& & ⋮\\ & & & \ddots & \\ d_{N-1,0}& & \dots & & d_{N-1,M-1}\end{array}\right]\hspace{1em}\left[\begin{array}{c}{g}_{d,0}·x_0^{\mathrm{obj}}\\ ⋮\\ g_{d,j}{x}_j^{\mathrm{obj}}\\ ⋮\\ g_{d,M-1}·x_{M-1}^{\mathrm{obj}}\end{array}\right]& \left[\mathrm{Equation}17\right]\end{array}$

In addition, the object audio signal x_j ^obj of Equation 17 including the J-number of layouts may be expressed by Equation 18.

$\begin{array}{cc}{d}_{i,j}·x_j^{\mathrm{obj}}=\left[\begin{array}{ccc}{d}_0^{i,j}& \dots & d_{L-1}^{i,j}\end{array}\right]\left[\begin{array}{c}{x}_0^{\mathrm{obj},j}\\ ⋮\\ x_{L-1}^{\mathrm{obj},j}\end{array}\right]=\sum _{l=0}^{L-1}{p}_l^{i,j}·x_l^{\mathrm{obj},j}=\sum _{l=0}^{L-1}{x}_l^{\mathrm{obj},j}\left(t-p_l^{i,j}\right)& \left[\mathrm{Equation}18\right]\end{array}$

Here, since the delay calculation process of the object audio signal cannot be expressed through matrix multiplication, different from the sound image localization matrix application calculation, the delay calculation process may be expressed using an operator ∘. In addition, a signal output through the delay calculation matrix D(t) may be expressed by Equation 19.

$\begin{array}{cc}D\left(t\right)G_1\left(t\right)·X_{\mathrm{obj}}=\hspace{1em}\left[\begin{array}{ccccc}{d}_{0,0}& d_{0,1}& \dots & & d_{0,M-1}\\ d_{1,0}& \ddots & & & \\ ⋮& & d_{i,j}& & ⋮\\ & & & \ddots & \\ d_{N-1,0}& & \dots & & d_{N-1,M-1}\end{array}\right]·\hspace{1em}\hspace{1em}[\begin{array}{c}{g}_{d,0}·x_0^{\mathrm{obj}}\\ ⋮\\ g_{d,j}{x}_j^{\mathrm{obj}}\\ ⋮\\ g_{d,M-1}·x_{M-1}^{\mathrm{obj}}\end{array}]=[\begin{array}{c}{g}_{d,0}(\sum _{l=0}^{L-1}{x}_l^{\mathrm{obj},0}\left(t-d_l^{0,0}\right)+\dots +\\ \sum _{l=0}^{L-1}{x}_l^{\mathrm{obj},,M-1}\left(t-d_l^{0,M-1}\right))\\ ⋮\\ g_{d,j}(\sum _{l=0}^{L-1}{x}_l^{\mathrm{obj},0}\left(t-d_l^{l,0}\right)+\dots +\\ \sum _{l=0}^{L-1}{x}_l^{\mathrm{obj},M-1}\left(t-d_l^{l,M-1}\right))\\ ⋮\\ g_{d,M-1}(\sum _{l=0}^{L-1}{x}_l^{\mathrm{obj},0}\left(t-d_l^{N-1,0}\right)+\dots +\\ \sum _{l=0}^{L-1}{x}_l^{\mathrm{obj},M-1}\left(t-d_l^{N-1,M-1}\right))\end{array}]& \left[\mathrm{Equation}19\right]\end{array}$

The audio rendering unit 304 may apply the sound image localization matrix and the delay calculation matrix independently. When the audio rendering unit 304 applies the sound image localization matrix and the delay calculation matrix simultaneously, a matrix PD(t) may be expressed using Equation 20.

$\begin{array}{cc}R\left(t\right)·X_{\mathrm{obj}}=\mathrm{PD}\left(t\right)G_{\mathrm{pd}}\left(t\right)·X_{\mathrm{obj}}=\hspace{1em}[\begin{array}{ccccc}{p}_{0,0}{d}_{0,0}& p_{0,1}{d}_{0,1}& \dots & & p_{0,M-1}{d}_{0,M-1}\\ p_{1,0}{d}_{1,0}& \ddots & & & \\ ⋮& & p_{i,j}{d}_{i,j}& & ⋮\\ & & & \ddots & \\ p_{N-1,0}{d}_{N-1,0}& & \dots & & p_{N-1,M-1}{d}_{N-1,M-1}\end{array}]\left[\begin{array}{c}{g}_{\mathrm{pd},0}·x_0^{\mathrm{obj}}\\ ⋮\\ g_{\mathrm{pd},j}{x}_j^{\mathrm{obj}}\\ ⋮\\ g_{\mathrm{pd},M-1}·x_{M-1}^{\mathrm{obj}}\end{array}\right]& \left[\mathrm{Equation}20\right]\end{array}$

Through the calculation of Equation 20, the audio rendering unit 304 may extract a result as shown in Equation 21.

$\begin{array}{cc}\mathrm{PD}\left(t\right)G_{\mathrm{pd}}(t)·X_{\mathrm{obj}}=\hspace{1em}\left[\begin{array}{c}{g}_{\mathrm{pd},0}·(\sum _{l=0}^{L-1}{p}_l^{0,0}{x}_l^{\mathrm{obj},0}\left(t-d_l^{0,0}\right)+\dots +\\ \sum _{l=0}^{L-1}{p}_l^{0,M-1}{x}_l^{\mathrm{obj},M-1}\left(t-d_l^{0,M-1}\right))\\ ⋮\\ g_{\mathrm{pd},j}·(\sum _{l=0}^{L-1}{p}_l^{i,0}{x}_l^{\mathrm{obj},0}\left(t-d_l^{i,0}\right)+\dots +\\ \sum _{l=0}^{L-1}{p}_l^{i,M-1}{x}_l^{\mathrm{obj},M-1}\left(t-d_l^{i,M-1}\right))\\ ⋮\\ g_{\mathrm{pd},M-1}·(\sum _{l=0}^{L-1}{p}_l^{N-1,0}{x}_l^{\mathrm{obj},0}\left(t-d_l^{N-1,0}\right)+\dots +\\ \sum _{l=0}^{L-1}{p}_l^{N-1,M-1}{x}_l^{\mathrm{obj},M-1}\left(t-d_l^{N-1,M-1}\right))\end{array}\right]& \left[\mathrm{Equation}21\right]\end{array}$

The audio rendering unit 304 may allocate the object audio signal to a channel signal which may be output, using the foregoing equation. In addition, the audio rendering unit 304 may combine the allocated object audio signal with the decoded channel audio signal. Additionally, the audio rendering unit 304 may generate an output signal to be finally output.

The audio rendering unit 304 may render the reverberation signal of the object audio signal as shown in Equation 22 or Equation 23.
R(t)·X _rev =[P(t)G _p(t)+D(t)G _d(t)]X _rev =P(t)[G _p X·X _rev ]+D(t)∘[G _d(t)·X _rev]  [Equation 22]
R(t)·X _obj =PD(t)G _pd(t)·X _rev[Equation 23]

Rendering of the reverberation signal of the object audio signal using Equation 22 and Equation 23 may render the object audio signal. By rendering the reverberation signal of the object audio signal corresponding to the object audio signal, the sound scene with higher reality may be implemented.

In addition, when controlling the object audio signal, the audio rendering unit 304 may control the reverberation signal of the object audio signal corresponding to the object audio signal. For example, when intending to control during rendering of the object audio signal x_j ^obj, the audio rendering unit 304 may allocate a solution of the gain value of Equation 11 as in g_p,j=g_d,j=0. In addition, the audio rendering unit 304 may control the reverberation signal corresponding an index of the object audio signal in the same manner as g_pd,j=0 of Equation 11. Furthermore, the audio rendering unit 304 may allocate the solution of the gain value of Equation 22 as in g_p,j=g_d,j=0, or control the object audio signal as in g_pd,j=0 of Equation 23.

The output signal to be finally output may be an integrated signal of the rendered object audio signal, the reverberation signal of the rendered object audio signal, and the decoded channel audio signal. The output signal may be expressed by Equation 24.
y _ch =x′ _ch +R _obj(t)·X _obj −R _rev(t)·X _rev  [Equation 24]

In Equation 24, the output signal may be separated into R_obj(t) and R_rev(t). That is, the output signal may be transmitted through different methods as information on the rendered object audio signal and information on the reverberation signal of the object audio signal. Therefore, Equation 23 shows that the output signal is to be transmitted as R_obj(t) and R_rev(t) as the rendering information.

In Equation 23, the decoded channel audio signal is denoted by x′_ch since the decoded channel audio signal x′_ch is expressed in the form of a downmixed signal when the number of channels for final output does not correspond to the decoded channel audio signals. For example, when the number of the decoded channel audio signals is N and the number of output signals output through the R_obj(t) and R_rev(t) and the channels is K, x_ch may be converted into x′_ch through a downmix matrix. That is, a number of dimensions of a row matrix of the R_obj(t) and R_rev(t) may also be K.

Here, the downmix matrix may be expressed by Equation 25.
x′ _ch =DMX(t)·x _ch  [Equation 25]

Based on Equation 25, when the number of the decoded channel audio signals is N and the number of the output signals is K, the downmixing process may be expressed by Equation 26.

$\begin{array}{cc}{x}_{\mathrm{ch}}^{\prime }=\mathrm{DMX}\left(t\right)·x_{\mathrm{ch}}=\left[\begin{array}{cccc}{c}_{0,0}& \dots & & c_{0,N-1}\\ ⋮& & & ⋮\\ c_{K-1,0}& & \dots & c_{K-1,N-1}\end{array}\right]\left[\begin{array}{c}{x}_0\\ x_1\\ ⋮\\ x_{N-1}\end{array}\right]& \left[\mathrm{Equation}26\right]\end{array}$

Here, when the number of dimensions of the row matrix of the R_obj(t) and R_rev(t) is also N, the output signal may be expressed by Equation 27, by reflecting Equation 24 to Equation 23.
y _ch =DMX(t)[x _ch +R _obj(t)·X _obj +R _rev(t)·X _rev]  [Equation 27]

That is, after rendering with respect to the N-number of channel audio signals is performed, the output signal may be downmixed by using DMX(t). In addition, the time index t may be varied according to time of information of DMX(t).

The audio coding apparatus 101 and the audio decoding apparatus 102 may fully reflect a content production intention of an original sound engineer, using the reverberation signal of the object audio signal corresponding to the object audio signal. The audio coding apparatus 101 and the audio decoding apparatus 102 may control the reverberation signal of the object audio signal. Therefore, the audio coding apparatus 101 and the audio decoding apparatus 102 may include rendering information corresponding to the reverberation signal of the object audio signal, for additional control of the reverberation signal.

FIG. 4 is a diagram illustrating the audio coding apparatus of FIG. 2 in detail.

Referring to FIG. 4, the audio coding apparatus may include an audio signal encoding unit 401 and a bitstream transmission unit 402.

The audio signal encoding unit 401 may receive a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal. Here, the audio signal encoding unit 401 may implement a sound scene of a higher quality by receiving the reverberation signal of the object audio signal. Additionally, the audio signal encoding unit 401 may encode the received channel audio signal, object audio signal, reverberation signal of the object audio signal into an audio signal.

In addition, the audio coding apparatus may receive rendering information 403. The audio coding apparatus may include a block for converting the rendering information 403 into a binary form.

Here, when the audio signal encoding unit 401 includes the block for converting the rendering information 403, the audio signal encoding unit 401 may encode to the audio signal including the channel audio signal, the object audio signal, the reverberation signal of the object audio signal, and the rendering information 403.

The bitstream transmission unit 402 may convert the audio signal into a bitstream, and transmit the bitstream to the audio decoding apparatus. The bitstream may include the audio signal including the channel audio signal, the object audio signal, and the reverberation signal of the object audio signal, and the rendering information 403. The bitstream transmission unit 402 may transmit the bitstream to generate multichannel scene information. The multichannel scene information may be generated based on the rendering information 403. The rendering information 403 may be used as additional data with respect to the reverberation signal of the object audio signal.

FIG. 5 is a diagram illustrating the audio decoding apparatus of FIG. 3 in detail.

The audio decoding apparatus may include a bitstream receiving unit 501, an audio signal decoding unit 502, and an audio rendering unit 503.

The bitstream receiving unit 501 may receive a bitstream from an audio coding apparatus. The received bitstream may include the audio signal and the rendering information.

The audio signal decoding unit 502 may decode the audio signal. That is, the audio signal decoding unit 502 may extract the channel audio signal, the object audio signal, and the reverberation signal of the object audio signal included in the audio signal.

The audio rendering unit 503 may perform rendering with respect to the decoded channel audio signal, object audio signal, and reverberation signal of the object audio signal. The object audio signal may be rendered based on the rendering process of FIG. 3. When the object audio signal is rendered, the reverberation signal of the object audio signal may be rendered according to an index of the corresponding object audio signal. The reverberation signal of the object audio signal may be controlled in the same manner as the object reverberation signal being controlled, thereby providing a more realistic sound image.

The audio rendering unit 503 may generate the output signal by rendering the decoded channel audio signal, object audio signal, reverberation signal of the object audio signal. Here, the output signal may include the rendered object audio signal, the reverberation signal of the rendered object audio signal, and the decoded channel audio signal. The output signal may be output to channels of the multichannel audio signal.

FIG. 6 is a diagram illustrating a configuration of rendering information 600.

Referring to FIG. 6, the rendering information 600 may be expressed in a matrix form. Each matrix of the rendering information 600 may be expressed by a substitute value to express the rendering information. For example, location information of the object may be expressed by angles of a horizontal plane and a vertical plane. A matrix value and a gain value related to delay information may be substituted by a value indicating a distance. In addition, the rendering information 600 needs to be expressed by being converted into a matrix value to be applied to the rendered object audio signal and the reverberation signal of the rendered object audio signal corresponding to the rendering information 600 input in various types to be used as additional data of the reverberation signal of the object audio signal.

FIG. 7 is a diagram illustrating an audio coding method according to an embodiment.

In operation 701, an audio coding apparatus may include a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal. The channel audio signal may be a generally used channel audio signal allocated to a channel of a predetermined reproduction device during reproduction. The object audio signal may define a particular audio signal among a plurality of audio signals and use the particular audio signal as a subject performing rendering. The reverberation signal of the object audio signal may be applied to the object audio signal and express a sound field feeling of the object audio signal.

The audio coding apparatus may encode the received channel audio signal, the object audio signal, and the reverberation signal of the object audio signal into an audio signal.

In operation 702, the audio coding apparatus may convert the audio signal into a bitstream. The bitstream may include the audio signal including the channel audio signal, the object audio signal, and the reverberation signal of the object audio signal, and rendering information 403. The audio coding apparatus may transmit the bitstream to generate multichannel scene information.

FIG. 8 is a diagram illustrating an audio decoding method according to an embodiment.

In operation 801, an audio decoding apparatus may receive a bitstream from an audio coding apparatus. The received bitstream may include an audio signal and rendering information.

In operation 802, the audio decoding apparatus may decode the audio signal, thereby extracting a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal included in the audio signal.

In operation 803, the audio decoding apparatus may render the extracted channel audio signal, object audio signal, and reverberation signal of the object audio signal based on the rendering information included in the bitstream. When the object audio signal is rendered, the reverberation signal of the object audio signal may be rendered according to an index of the corresponding object audio signal. In addition, the reverberation signal of the object audio signal may be controlled in the same manner as the object audio signal being controlled, thereby providing a more realistic sound image. Furthermore, the audio decoding apparatus may generate an output signal by rendering the decoded channel audio signal, object audio signal, and reverberation signal of the object audio signal.

The above-described embodiments of the present invention may be recorded in non-transitory computer-readable media including program instructions to implement various operations embodied by a computer. The media may also include, alone or in combination with the program instructions, data files, data structures, and the like. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts.

A number of examples have been described above. Nevertheless, it will be understood that various modifications may be made. For example, suitable results may be achieved if the described techniques are performed in a different order and/or if components in a described system, architecture, device, or circuit are combined in a different manner and/or replaced or supplemented by other components or their equivalents.

Accordingly, other implementations are within the scope of the following claims.

Claims (16)

The invention claimed is:

1. An audio coding apparatus comprising:

an audio signal encoding unit to encode an audio signal and a rendering information; and

a bitstream transmission unit to convert the audio signal and the rendering information into a bitstream and transmit the bitstream,

wherein the audio signal comprises a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal,

wherein the rendering information indicates sound scene information with respect to the object audio signal.

2. The audio coding apparatus of claim 1, wherein the reverberation signal of the object audio signal expresses a sound field feeling of the object audio signal.

3. The audio coding apparatus of claim 1, wherein the reverberation signal of the object audio signal comprises a plurality of channel signals.

4. The audio coding apparatus of claim 1, wherein the reverberation signal of the object audio signal provides various layouts with respect to the object audio signal.

5. The audio coding apparatus of claim 1, wherein the bitstream transmission unit generates the bitstream from the encoded audio signal and the rendering information for generation of the audio signal.

6. The audio coding apparatus of claim 1, wherein the rendering information comprises at least one of location information of an audio object, sound pressure information of the audio object, and delay information of the audio object.

7. An audio decoding apparatus comprising:

a bitstream receiving unit to receive a bitstream including an encoded audio signal and a rendering information; and

an audio signal decoding unit to extract a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal from the bitstream by decoding the audio signal included in the bitstream,

wherein the rendering information indicates sound scene information with respect to the object audio signal.

8. The audio decoding apparatus of claim 7, wherein the reverberation signal of the object audio signal expresses a sound field feeling of the object audio signal.

9. The audio decoding apparatus of claim 7, wherein the reverberation signal of the object audio signal comprises a plurality of channel signals.

10. The audio decoding apparatus of claim 8, wherein the reverberation signal of the object audio signal provides various layouts with respect to the object audio signal.

11. The audio decoding apparatus of claim 7, further comprising:

an audio rendering unit to render the extracted channel audio signal, object audio signal, and reverberation signal of the object audio signal based on the rendering information included in the bitstream.

12. The audio decoding apparatus of claim 11, wherein the rendering information comprises at least one of location information of an audio object, sound pressure information of the audio object, and delay information of the audio object.

13. The audio decoding apparatus of claim 11, wherein the audio rendering unit controls the reverberation signal of the object audio signal corresponding to the object audio signal, when controlling the object audio signal.

14. The audio decoding apparatus of claim 11, wherein the audio rendering unit controls the reverberation signal of the object audio signal in consideration of an index of the object audio signal corresponding to the reverberation signal of the object audio signal.

15. An audio decoding method comprising:

receiving a bitstream comprising an encoded audio signal and a rendering information;

extracting a channel audio signal, an object audio signal, and a reverberation signal of the object audio signal from the bitstream by decoding the audio signal included in the bitstream; and

rendering the extracted channel audio signal, object audio signal, and reverberation signal of the object audio signal based on the rendering information included in the bitstream, wherein the rendering information comprises sound scene information with respect to the object audio signal.

16. The audio decoding method of claim 15, wherein the reverberation signal of the object audio signal comprises a plurality of channel signals, expresses a sound field feeling of the object audio signal, and provides various layouts with respect to the object audio signal.

Images