JP4517435B2 - Information processing apparatus and method - Google Patents

Description

Ｑ_i,jは、リカバリの対象となるＡＤＲＣブロックに含まれる欠落していないコードＱを示す。Ｑ’_i,jは、Ｑ_i,jが含まれるＡＤＲＣブロックを構成する画素を含む図１３および図１４に示す６４画素のブロックの画素で構成される他のＡＤＲＣブロックに含まれるコードＱを示す。例えば、Ｑ_i,jが図１６の番号”０”の偶数セグメントのＡＤＲＣブロックに含まれるとき、Ｑ’_i,jは、番号”３”の奇数セグメントのＡＤＲＣブロックに含まれる。Ｑ_i,jが図１６の番号”４”の偶数セグメントのＡＤＲＣブロックに含まれるとき、Ｑ’_i,jは、番号”１”の奇数セグメントのＡＤＲＣブロックに含まれる。ｑ_i,jは、Ｑ_i,jを切り出すｑを示す。ｑ’_i,jは、Ｑ’_i,jを切り出すｑを示す。Njは、ＡＤＲＣブロック内欠落していないコードＱの数を示す。ｊは、評価値が算出される３つのＡＤＲＣブロックを識別する番号である。
【０１６３】
オフセットjは、３つのＡＤＲＣブロックの間のダイナミックレンジ、ｑの違いを吸収し、より正確な評価値を算出するために用いられ、式（２）、式（３）、および式（４）で算出される。
オフセットj=(Xj×2^4-qj-Yj×2^4-q'j)/128 （２）
【数２】

【数３】

【０１６４】
セレクタ３７３は、３ブロック評価値算出回路３７２−１乃至３７２−６のそれぞれから供給された評価値から、最小値を有する評価値、および最小値の次に大きい値を有する評価値を選択し、それぞれｅｍｉｎ１およびｅｍｉｎ２として、リカバリエラー判定回路３５７に供給する。
【０１６５】
セレクタ３７３は、３ブロック評価値算出回路３７２−１乃至３７２−６のうち、最小値を有する評価値を出力したものが出力するｑおよび動きフラグＭＦを選択して、ｑビットＭＦリカバリ回路３５３の出力として、ＤＲＭＩＮリカバリ回路３５４に供給する。
【０１６６】
ＤＲＭＩＮリカバリ回路３５４は、ｑビットＭＦリカバリ回路３５３から供給されたｑの正しい値および動きフラグＭＦの正しい値とセグメント間デシャフル回路３５２からの図１２で示した対応するＡＤＲＣブロックとを基に、ＡＤＲＣブロック毎のダイナミックレンジＤＲの正しい値およびＡＤＲＣブロック毎の画素値の最小値ＭＩＮを求め、ＡＤＲＣ復号回路３５５に出力する。
【０１６７】
図３６乃至図３８を参照して、ＤＲＭＩＮリカバリ回路３５４のダイナミックレンジＤＲまたは画素値の最小値ＭＩＮのリカバリの処理を説明する。画素値Ｌは、ＡＤＲＣの方式により、コードＱから式（５）の処理により復号される。
L=DR/(2^Q)*q+MIN （５）
【０１６８】
図３６に示すように、ｑと画素値Ｌとの関係は、傾きをＤＲ／（２^Q）とし、接片を画素値の最小値ＭＩＮとする直線で表される。
【０１６９】
ＡＤＲＣブロック内の（ｑ，Ｌ）の組より、式（５）のＤＲ／（２^Q）および画素値の最小値ＭＩＮを統計的に求める。図３７に示す、（ｑ，Ｌ）の回帰直線を、式（５）の直線に等しいとする。
【０１７０】
このとき、ｂが既知であるならば、ａは、式（６）で算出される。
a=(B-nb)/A （６）
【０１７１】
また、ａが既知であるならば、ｂは、式（７）で算出される。
b=(B-Aa)/n （７）
【０１７２】
ここで、ｎは、（ｑ，Ｌ）の点の個数であり、Ａは、Σｘであり、Ｂは、Σｙである。
【０１７３】
ダイナミックレンジＤＲは、式（８）で算出される。
DR=(ΣL'-N*MIN)/(Σq)*2^Q
=(((DR'/2^Q')*Y)+N*MIN'-N*MIN)/X*2^Q
=(N*(MIN'-MIN)*2Q'+DR'*Y)*2^Q/(X*2^Q') （８）
図３８（Ａ）に示すように、ダイナミックレンジＤＲまたは画素値の最小値ＭＩＮをリカバリしようとするブロックが隣接する欠落のないのブロックの数を、ｎｉとしたとき、Ｎは、Σｎｉである。図３８において、斜線が示された四角は、ダイナミックレンジＤＲまたは画素値の最小値ＭＩＮをリカバリしようとするブロックを示す。図３８において、斜線のない四角は、欠落のないのブロックを示す。ダイナミックレンジＤＲまたは画素値の最小値ＭＩＮをリカバリしようとするブロックのｑが欠落している時、図３８（Ｂ）に示す様に、ｎｉは０である。
【０１７４】
ＤＲ’は、リカバリしようとするブロックが隣接する、欠落のないのブロックのダイナミックレンジを示す。ＭＩＮ’は、リカバリしようとするブロックが隣接する、欠落のないのブロックの画素値の最小値を示す。Ｑ’は、リカバリしようとするブロックが隣接する、欠落のないのブロックのコードＱを示す。
【０１７５】
Ｘは、式（９）で表される。
【数４】

【０１７６】
Ｙは、式（１０）で示される。
【数５】

【０１７７】
画素値の最小値ＭＩＮは、式（１１）で算出される。
MIN=(ΣL'-Σq*(DR/2^Q))
=((DR'/2^Q')*Y+N*MIN'-(DR/2^Q)*X)/N
=(N*MIN'*2^Q'+Q+DR'*Y*2^Q-DR*X*2^Q')/(N*2^Q'+Q) （１１）
【０１７８】
ＡＤＲＣ復号回路３５５は、デシャフルされたデータに欠落がないとき、セグメント間デシャフル回路３５２から供給されたデータを基に、コードＱを復号し、復号の結果得られたＹデータ、Ｕデータ、およびＶデータをブロックデシャフル回路３５６に供給する。
【０１７９】
ＡＤＲＣ復号回路３５５は、ｑまたは動きフラグＭＦに欠落があり、かつｑビットＭＦリカバリ回路３５３がｑの正しい値および動きフラグＭＦの正しい値を求めることができたとき、ＤＲＭＩＮリカバリ回路３５４を介して供給されたｑの正しい値、動きフラグＭＦの正しい値、ダイナミックレンジＤＲ、およびブロック毎の画素値の最小値ＭＩＮを基に、ＤＲＭＩＮリカバリ回路３５４から供給されたコードＱを復号し、復号の結果得られたＹデータ、Ｕデータ、およびＶデータを、Ｙデータ、Ｕデータ、およびＶデータに対応する、”０”を設定したエラーフラグと共に、ブロックデシャフル回路３５６に供給する。
【０１８０】
ＡＤＲＣ復号回路３５５は、ｑまたは動きフラグＭＦに欠落があり、かつｑビットＭＦリカバリ回路３５３がｑの正しい値および動きフラグＭＦの正しい値を求めることができないとき、コードＱに対応するＹデータ、Ｕデータ、およびＶデータのエラーフラグに”１”を設定して、ブロックデシャフル回路３５６に供給する。
【０１８１】
ＡＤＲＣ復号回路３５５は、ダイナミックレンジＤＲ、または画素値の最小値ＭＩＮに欠落があり、かつＤＲＭＩＮリカバリ回路３５４がダイナミックレンジＤＲの正しい値および画素値の最小値ＭＩＮの正しい値を求めることができたとき、ＤＲＭＩＮリカバリ回路３５４を介して供給されたｑ、動きフラグＭＦ、ダイナミックレンジＤＲの正しい値、および画素値の最小値ＭＩＮの正しい値を基に、ＤＲＭＩＮリカバリ回路３５４から供給されたコードＱを復号し、復号の結果得られたＹデータ、Ｕデータ、およびＶデータを、Ｙデータ、Ｕデータ、およびＶデータに対応する、”０”を設定したエラーフラグと共に、ブロックデシャフル回路３５６に供給する。
【０１８２】
ＡＤＲＣ復号回路３５５は、ダイナミックレンジＤＲ、または画素値の最小値ＭＩＮに欠落があり、かつかつＤＲＭＩＮリカバリ回路３５４がダイナミックレンジＤＲの正しい値および画素値の最小値ＭＩＮの正しい値を求めることができないとき、コードＱに対応するＹデータ、Ｕデータ、およびＶデータのエラーフラグに”１”を設定して、ブロックデシャフル回路３５６に供給する。
【０１８３】
リカバリエラー判定回路３５７は、ｑビットＭＦリカバリ回路３５３から供給されたｅｍｉｎ１およびｅｍｉｎ２を基に、リカバリエラー判定フラグＦｄをブロックデシャフル回路３５６に出力し、外部メモリ３０５が記憶している前のフレームの対応する画素のＹデータ、Ｕデータ、およびＶデータの更新を停止させる。
【０１８４】
ブロックデシャフル回路３５６は、ＡＤＲＣ復号回路３５５から”０”を設定したエラーフラグと共に、Ｙデータ、Ｕデータ、およびＶデータが供給されたとき、エラーフラグ、並びにＹデータ、Ｕデータ、およびＶデータを記憶する。ブロックデシャフル回路３５６は、ＡＤＲＣ復号回路３５５から”１”を設定したエラーフラグと共に、Ｙデータ、Ｕデータ、およびＶデータが供給されたとき、エラーフラグを記憶し、Ｙデータ、Ｕデータ、およびＶデータを記憶しない。また、ブロックデシャフル回路３５６は、リカバリエラー判定回路３５７またはセグメント間デシャフル回路３５２から連続エラー信号が供給された場合も、外部メモリ３０５が記憶している前のフレームの対応する画素のＹデータ、Ｕデータ、およびＶデータの更新を停止させる。
【０１８５】
ブロックデシャフル回路３５６は、Ｙデータ、Ｕデータ、およびＶデータを、図１０に示す送信装置１のブロックシャフル回路１０２と逆の動作で、元の並びに戻し、元の並びに変換されたＹデータ、Ｕデータ、およびＶデータを出力する。
【０１８６】
以上のように、復号回路３０４は、復号に必要なデータであるｑ、動きフラグＭＦ、ダイナミックレンジＤＲ、または画素値の最小値ＭＩＮが失われても、ｑビットＭＦリカバリ回路３５３およびＤＲＭＩＮリカバリ回路３５４が失われたデータをリカバリするので、復号を実行することができる。
【０１８７】
リカバリにエラーがあるときは、外部メモリ３０５に記憶されている画素値を上書きしないで、４フレーム前の画素値を出力するので、動きが無いとき、違和感のない画素値を出力できる。
【０１８８】
次に、復号回路３０４の処理のタイミングを図３９および図４０のタイミングチャートを参照して説明する。図３９は、セグメント間デシャフル回路３５２から出力されるデータに欠落がない場合の、復号回路３０４の処理のタイミングの例を示す。セグメント間デシャフル回路３５２は、デシャフルの処理を行い、所定の時間経過後（図３９の例では、入力開始から３６２クロック後）、ＡＤＲＣ復号回路３５５にデシャフルされたデータを出力する。ＡＤＲＣ復号回路３５５は、セグメント間デシャフル回路３５２から供給されたデータを復号して出力する。
【０１８９】
図４０は、セグメント間デシャフル回路３５２から出力されるデータに欠落がある場合の、復号回路３０４の処理のタイミングの例を示す。セグメント間デシャフル回路３５２は、デシャフルの処理を行い、データに欠落がない場合に、ＡＤＲＣ復号回路３５５にデータを供給するタイミングより早く（図４０の例では、入力開始から１２３クロック後）、ｑビットＭＦリカバリ回路３５３にデシャフルされたデータを出力する。ｑビットＭＦリカバリ回路３５３は、所定のタイミングで、ｑおよび動きフラグＭＦを、ＤＲＭＩＮリカバリ回路３５４に供給する。
【０１９０】
ＤＲＭＩＮリカバリ回路３５４は、データに欠落がない場合にセグメント間デシャフル回路３５２がＡＤＲＣ復号回路３５５にデータを供給するタイミングと同じタイミングで、ダイナミックレンジおよび画素の最小値を、ＡＤＲＣ復号回路３５５に供給する。
【０１９１】
従って、復号回路３０４は、ｑ、動きフラグＭＦ、ダイナミックレンジＤＲ、および画素の最小値ＭＩＮをリカバリしても、セグメント間デシャフル回路３５２から出力されるデータに欠落がない場合と同じタイミングで、データを復号し、出力することができる。
【０１９２】
以上のように、復号回路３０４は、復号に必要な所定のデータが失われても、ｑビットＭＦリカバリ回路３５３およびＤＲＭＩＮリカバリ回路３５４が失われたデータをリカバリするので、復号を実行することができる。なお、復号回路３０４にエラー処理を行わないモード（ユーザセレクタブルなモード）を選択する信号が入力されているとき、ｑビットＭＦリカバリ回路３５３およびＤＲＭＩＮリカバリ回路３５４は、リカバリを実行しない。
【０１９３】
図４１および図４２を参照して、ブロックデシャフル回路３５６のデータの出力のタイミングについて説明する。
【０１９４】
従来は、２フレームの画像を記憶するメモリを２個設けて、図４１（Ａ）に示すように、一方のメモリにデータを書き込んでいるとき、他のメモリからデータを読み出して出力し、２フレームの画像について処理が終了したとき、図４１（Ｂ）に示すように、他方のメモリにデータを書き込んで、一方のメモリからデータを読み出して、出力していた。
【０１９５】
これに対して、外部メモリ３０５に、それぞれ２フレームの画像を記憶するバンクを２つ設けて（図中のバンク１およびバンク２）、ブロックデシャフル回路３５６は、図４２（Ａ）に示すように、３個のＡＤＲＣに対応する画像データがＡＤＲＣ復号回路３５５から供給されたとき、バンク１に書き込み、図４２（Ｂ）に示すように、ＡＤＲＣ復号回路３５５から画像データが供給されていないとき（例えば、図４０に示す、入力開始から３６４クロックまでの間）、アドレスを指定することにより、バンク２からブロックデシャフルされたデータを読み出す。次の２フレームにおいては、ブロックデシャフル回路３５６は、３個のＡＤＲＣに対応する画像データがＡＤＲＣ復号回路３５５から供給されたとき、バンク２に書き込み、ＡＤＲＣ復号回路３５５から画像データが供給されていないとき、バンク１からブロックデシャフルされたデータを読み出す。
【０１９６】
このようにすることで、ブロックデシャフル回路３５６に対し、１個の外部メモリ３０５で、画像データをブロックデシャフルして、出力することができる。
【０１９７】
図４３は、リカバリエラー判定回路３５７の構成を示すブロック図である。ｑビットＭＦリカバリ回路３５３から供給される最小値を有する評価値ｅｍｉｎ１、および２番目に小さい値を有する評価値ｅｍｉｎ２は、差分算出回路４３１に供給される。差分算出回路４３１は、入力されたｅｍｉｎ１とｅｍｉｎ２との差を算出し、遅延回路４３２に出力する。遅延回路４３２は、差分算出回路４３１から供給されたデータを所定の時間遅延し、加算器４３５および遅延回路４３３に出力する。
【０１９８】
遅延回路４３３は、遅延回路４３２から供給されたデータを所定の時間遅延し、加算器４３５および遅延回路４３４に出力する。
【０１９９】
遅延回路４３４は、遅延回路４３３から供給されたデータを所定の時間遅延し、加算器４３５および判定回路４３７に出力する。
【０２００】
加算器４３５は、遅延回路４３２からから供給されたデータ、遅延回路４３３からから供給されたデータ、および遅延回路４３４からから供給されたデータを加算して、判定回路４３６に供給する。
【０２０１】
判定回路４３６は、加算器４３５から供給されたデータが予め定めた閾値Ｔｅ１以下であるか否かを判定し、加算器４３５から供給されたデータが閾値Ｔｅ１以下であると判定された場合、”１”をＡＮＤ回路４３８に出力し、加算器４３５から供給されたデータが閾値Ｔｅ１を越えると判定された場合、”０”をＡＮＤ回路４３８に出力する。
【０２０２】
判定回路４３７は、遅延回路４３４から供給されたデータが予め定めた閾値Ｔｅ２以下であるか否かを判定し、遅延回路４３４から供給されたデータが閾値Ｔｅ２以下であると判定された場合、”１”をＡＮＤ回路４３８に出力し、遅延回路４３４から供給されたデータが閾値Ｔｅ２を越えると判定された場合、”０”をＡＮＤ回路４３８に出力する。
【０２０３】
ＡＮＤ回路４３８は、判定回路４３６から供給されたデータが”１”、かつ、判定回路４３７から供給されたデータが”１”のとき、”１”をホールド回路４３９に出力する。ＡＮＤ回路４３８は、判定回路４３６から供給されたデータが”０”、または、判定回路４３７から供給されたデータが”０”のとき、”０”をホールド回路４３９に出力する。
【０２０４】
ホールド回路４３９は、入力されたデータをリセット信号が入力されるまで保持し、遅延回路４４０に出力する。遅延回路４４０は、ホールド回路４３９から出力されたデータを所定の時間遅延し、ホールド回路４３９に供給するとともに、リカバリ判定フラグＦｄとしてブロックデシャフル回路３５６に出力する。
【０２０５】
このように、リカバリエラー判定回路３５７は、３個のＡＤＲＣブロックに対する、評価値ｅｍｉｎ１と評価値ｅｍｉｎ２の差の積算値が閾値Ｔｅ１以下であり、かつ、そのＡＤＲＣブロックに対する評価値ｅｍｉｎ１と評価値ｅｍｉｎ２の差が閾値Ｔｅ２以下であるとき、すなわち、評価値ｅｍｉｎ１と評価値ｅｍｉｎ２の差が少なく、リカバリされたｑまたは動きフラグＭＦが正しくない可能性が高いとき、リカバリ判定フラグＦｄに１をセットする。
【０２０６】
次に、ＡＤＲＣ復号回路３５５で復号されたデータが一時的にブロックデシャフル回路３５６に記憶され、読み出される処理を説明する。セグメント間デシャフル回路３５２は、パケット解体回路３０２から供給されたデータに欠落があるとき、欠落するデータに対応するエラーフラグに”１”を設定し、データに欠落がないとき、データに対応するエラーフラグに”０”を設定し、ＡＤＲＣブロック単位のコードＱおよびエラーフラグをＡＤＲＣ復号回路３５５に出力する。ＡＤＲＣ復号回路３５５は、コードＱを復号し、Ｙデータ、Ｕデータ、およびＶデータ、並びにエラーフラグをブロックデシャフル回路３５６に出力する。
【０２０７】
ブロックデシャフル回路３５６は、対応するエラーフラグが”０”の場合、Ｙデータ、Ｕデータ、およびＶデータ、並びにエラーフラグを外部メモリ３０５に記憶し、対応するエラーフラグが”１”である場合、エラーフラグを外部メモリ３０５に記憶し、Ｙデータ、Ｕデータ、およびＶデータを外部メモリ３０５に記憶しない。
【０２０８】
外部メモリ３０５は、２つのバンクを有し、１つのバンクに２つのフレームのＹデータ、Ｕデータ、およびＶデータ、並びにエラーフラグを記憶する。外部メモリ３０５のバンクは、２つのフレーム毎に切り替えられる。従って、対応するエラーフラグが”１”である場合、外部メモリ３０５は、４フレーム前のＹデータ、Ｕデータ、およびＶデータを記憶している。
【０２０９】
図４４は、リカバリエラー判定回路３５７が、”１”が設定されたリカバリ判定フラグＦｄを出力した場合の、ブロックデシャフル回路３５６の処理を説明する図である。ｑビットＭＦリカバリ回路３５３から供給される最小値を有する評価値ｅｍｉｎ１、および２番目に小さい値を有する評価値ｅｍｉｎ２の差が所定の値以下であり、かつ、そのＡＤＲＣブロックを含めた連続する３つのＡＤＲＣブロックのｅｍｉｎ１、およびｅｍｉｎ２の差の和が所定の値以下であるとき、ｑビットＭＦリカバリ回路３５３は、ｑのリカバリに失敗したと判定し、このため、データの切り出しも失敗したと判定する。このとき、リカバリエラー判定回路３５７は、”１”が設定されたリカバリ判定フラグＦｄを出力する。
【０２１０】
リカバリエラー判定回路３５７が、”１”が設定されたリカバリ判定フラグＦｄを出力したとき、ブロックデシャフル回路３５６は、それ以降のバッファ（８８個のＡＤＲＣブロック）からのデータの切り出しに失敗したとみなし、それ以降に供給されるエラーフラグに”１”を設定し、そのエラーフラグを外部メモリ３０５に記憶する。ユーザセレクタブルなエスケープモードの場合には、データは外部メモリ３０５に記憶される。
【０２１１】
図４５は、セグメント間デシャフル回路３５３が、連続エラー信号を出力した場合の、ブロックデシャフル回路３５６の処理を説明する図である。セグメント間デシャフル回路３５３は、対応する偶数ＡＤＲＣブロックおよび奇数ＡＤＲＣブロックの両方の欠落を検出したとき、連続エラー信号を出力する。ｑビットＭＦリカバリ回路３５３のｑおよび動きフラグＭＦのリカバリの処理は、隣接する画素の画素値の相関を利用しているので、対応する偶数ＡＤＲＣブロックおよび奇数ＡＤＲＣブロックの両方が欠落しているとき、ｑビットＭＦリカバリ回路３５３は、リカバリの処理を実行することができない。連続エラー信号が供給されたとき、ブロックデシャフル回路３５６は、それ以降のバッファ（８８個のＡＤＲＣブロック）からのデータの切り出しに失敗したとみなし、それ以降に供給されるエラーフラグに”１”を設定し、そのエラーフラグを外部メモリ３０５に記憶し、それ以降に供給される復号されたデータを外部メモリ３０５に記憶しない。
【０２１２】
図４６は、ユーザセレクタブルなグレイモードの場合のブロックデシャフル回路３５６の処理を説明する図である。グレイモードの場合、ブロックデシャフル回路３５６は、対応するエラーフラグが”１”であるデータに、グレイ（Y,U,V全てを128にするのではなく、対応するデータのみ128にする）を設定して出力する。
【０２１３】
図４７および図４８は、ブロックデシャフル回路３５６のユーザセレクタブルなリカバリオフ入力（ユーザが図示せぬ所定のスイッチを操作したときに入力される信号）に対応した処理を説明する図である。”１”が設定されたリカバリオフ入力が供給された場合、ブロックデシャフル回路３５６は、外部メモリ３０５に、図４７に示すように、ｑビットＭＦリカバリ回路３５３を経由しないで、ＡＤＲＣ復号回路３５５でＹデータ、Ｕデータ、およびＶデータが復号されたとき、復号されたＹデータ、Ｕデータ、およびＶデータを記憶させる。図４８に示すように、ｑビットＭＦリカバリ回路３５３でｑおよび動きフラグがリカバリされたとき、ブロックデシャフル回路３５６は、外部メモリ３０５に、復号されたＹデータ、Ｕデータ、およびＶデータをメモリに記憶させず、対応するすべてのエラーフラグを”１”に設定する。リカバリオフ入力を変化させることで、ｑビットＭＦリカバリ回路３５３等のリカバリの効果が簡単に確認できる。
【０２１４】
図４９は、ブロックデシャフル回路３５６のエスケープ信号（ユーザが図示せぬ所定のスイッチを操作したときに入力される信号）に対応した処理を説明する図である。所定のエスケープ信号が供給された場合、ブロックデシャフル回路３５６は、図４９に示すように、リカバリ判定フラグＦｄが”１”に設定されても、それ以降に供給されるエラーフラグに強制的に”１”を設定しないでエラーフラグをメモリに記憶し、それ以降に供給される復号されたＹデータ、Ｕデータ、およびＶデータをメモリに記憶する。エスケープ信号を変化させることで、ブロックデシャフル回路３５６のコンシールの処理の効果が簡単に確認できる。
【０２１５】
図５０および図５１は、ブロックデシャフル回路３５６の復号したデータを出力する処理を説明するフローチャートである。ステップＳ３１において、セグメント間デシャフル回路３５２は、受信回路３０１が生成したエラーフラグを受信し、ＡＤＲＣ復号回路３５５に供給する。ＡＤＲＣ復号回路３５５は、セグメント間デシャフル回路３５２から供給されたエラーフラグをブロックデシャフル回路３５６に出力し、ブロックデシャフル回路３５６は、そのエラーフラグを受信する。
【０２１６】
ステップＳ３２において、ブロックデシャフル回路３５６は、ＡＤＲＣ復号回路３５５から供給された、バッファ（所定の８８個のＡＤＲＣブロック）に対応する、最初のエラーフラグを選択する。ステップＳ３３において、ブロックデシャフル回路３５６は、セグメント間デシャフル回路３５２から供給されている連続エラー信号が”０”であるか否かを判定し、連続エラー信号が”０”であると判定された場合、ステップＳ３４に進み、リカバリエラー判定回路３５７から供給されているリカバリ判定フラグＦｄが”０”であるか否かを判定する。ステップＳ３４において、リカバリ判定フラグＦｄが”０”であると判定された場合、ステップＳ３５に進み、ブロックデシャフル回路３５６は、外部メモリ３０５にエラーフラグを記憶させる。
【０２１７】
ステップＳ３３において、連続エラー信号が”１”であると判定された場合、およびステップＳ３４において、リカバリ判定フラグＦｄが”１”であると判定された場合、データの切り出しに失敗したので、ステップＳ４７に進み、ブロックデシャフル回路３５６は、エスケープ信号がオフであるか否かを判定し、エスケープ信号がオフであると判定された場合、ステップＳ４８に進み、ステップＳ３２で選択されたエラーフラグに”１”を設定し、ステップＳ３５に進み、ブロックデシャフル回路３５６は、外部メモリ３０５にエラーフラグを記憶させる。
【０２１８】
ステップＳ４７において、エスケープ信号がオンであると判定された場合、エラーフラグの設定を行わないので、手続きは、ステップＳ４８をスキップし、ステップＳ３５に進み、ブロックデシャフル回路３５６は、外部メモリ３０５にエラーフラグを記憶させる。
【０２１９】
ステップＳ３６において、ブロックデシャフル回路３５６は、バッファに対応する、すべてのエラーフラグを処理したか否かを判定し、すべてのエラーフラグを処理したと判定された場合、ステップＳ３７に進み、ブロックデシャフル回路３５６は、ＡＤＲＣ復号回路３５５から復号されたＹデータ、Ｕデータ、およびＶデータを受信する。
【０２２０】
ステップＳ３６において、バッファの全てのエラーフラグをまだ処理していないと判定された場合、ステップＳ４９に進み、ブロックデシャフル回路３５６は、次のエラーフラグを選択し、ステップＳ３３に戻り、エラーフラグの設定の処理を繰り返す。
【０２２１】
ステップＳ３８において、ブロックデシャフル回路３５６は、ＡＤＲＣ復号回路３５５から供給された、バッファの最初の復号されたデータを選択する。
【０２２２】
ステップＳ３９において、ブロックデシャフル回路３５６は、セグメント間デシャフル回路３５２から供給されている連続エラー信号が”０”であるか否かを判定し、連続エラー信号が”０”であると判定された場合、ステップＳ４０に進み、選択されている復号されたデータに対応するエラーフラグが”０”であるか否かを判定する。選択されている復号されたデータに対応するエラーフラグが”０”であると判定された場合、ステップＳ４１に進み、ブロックデシャフル回路３５６は、選択されている復号されたＹデータ、Ｕデータ、およびＶデータを外部メモリ３０５に記憶させる。手続きは、ステップＳ４２に進む。
【０２２３】
ステップＳ３９において、連続エラー信号が”１”であると判定された場合、および、ステップＳ４０において、選択されている復号されたデータに対応するエラーフラグが”１”であると判定された場合、データは欠落しているので、ステップＳ４１をスキップして、ステップＳ４２に進む。
【０２２４】
ステップＳ４２において、ブロックデシャフル回路３５６は、バッファの全ての復号されたデータを処理したか否かを判定し、バッファの全ての復号されたデータを処理したと判定された場合、ステップＳ４３に進む。
【０２２５】
ステップＳ４２において、バッファ（８８個のＡＤＲＣブロック）の全ての復号されたデータをまだ処理していないと判定された場合、ステップＳ５０に進み、ブロックデシャフル回路３５６は、次のデータを選択し、ステップＳ３９に戻り、復号したデータを記憶する処理を繰り返す。
【０２２６】
ステップＳ４３において、ブロックデシャフル回路３５６は、外部メモリ３０５に記憶されている最初のデータを選択する。ステップＳ４４において、ブロックデシャフル回路３５６は、エラーフラグが”１”で、かつ、グレイモードであるか否かを判定し、エラーフラグが”０”、または、グレイモードでないと判定された場合、ステップＳ４５に進み、選択されているデータをフォーマット変換回路３０６に出力する。
【０２２７】
ステップＳ４４において、エラーフラグが”１”で、かつ、グレイモードであると判定された場合、ステップＳ５１に進み、ブロックデシャフル回路３５６は、グレーデータをフォーマット変換回路３０６に出力し、ステップＳ４６に進む。
【０２２８】
ステップＳ４６において、ブロックデシャフル回路３５６は、外部メモリ３０５のバンクに記憶されている全てのデータを処理したか否かを判定し、外部メモリ３０５に記憶されている全てのデータを処理したと判定された場合、処理は終了する。
【０２２９】
ステップＳ４６において、外部メモリ３０５に記憶されている全てのデータを処理していないと判定された場合、ステップＳ５２に進み、ブロックデシャフル回路３５６は、外部メモリ３０５のバンクに記憶されている次のデータを選択し、ステップＳ４４に戻り、データを出力する処理を繰り返す。
【０２３０】
このように、ブロックデシャフル回路３５６は、コードＱが欠落していても、もっとも違和感の少ないデータを出力することができる。
【０２３１】
次に、図示せぬユーザセレクタブルなリカバリオフスイッチが”０”である場合のブロックデシャフル回路３５６のデータを出力する処理を図５２のフローチャートを参照して説明する。ステップＳ７１において、セグメント間デシャフル回路３５２は、受信回路３０１が生成したエラーフラグを受信し、ＡＤＲＣ復号回路３５５に供給する。ＡＤＲＣ復号回路３５５は、セグメント間デシャフル回路３５２から供給されたエラーフラグをブロックデシャフル回路３５６に出力し、ブロックデシャフル回路３５６は、そのエラーフラグを受信する。
【０２３２】
ステップＳ７２において、ブロックデシャフル回路３５６は、ｑおよび動きフラグ等がｑビットＭＦリカバリ回路３５３等において、リカバリされたか否かを判定し、ｑおよび動きフラグＭＦ等がリカバリされていないと判定された場合、ステップＳ７３に進み、外部メモリ３０５にエラーフラグを記憶させる。
【０２３３】
ステップＳ７２において、ｑおよび動きフラグ等がリカバリされていると判定された場合、ステップＳ７８に進み、ブロックデシャフル回路３５６は、受信したエラーフラグを全て”１”に設定し、ステップＳ７３に進む。
【０２３４】
ステップＳ７４において、ブロックデシャフル回路３５６は、ＡＤＲＣ復号回路３５５から復号されたＹデータ、Ｕデータ、およびＶデータを受信する。
【０２３５】
ステップＳ７５において、ブロックデシャフル回路３５６は、ｑおよび動きフラグ等がｑビットＭＦリカバリ回路３５３等において、リカバリされたか否かを判定し、ｑおよび動きフラグＭＦ等がリカバリされていないと判定された場合、ステップＳ７６に進み、ステップＳ７４で受信したＹデータ、Ｕデータ、およびＶデータを記憶する。
【０２３６】
ステップＳ７５において、ｑ、動きフラグＭＦ等がリカバリされていると判定された場合、ステップＳ７６をスキップし、Ｙデータ、Ｕデータ、およびＶデータを記憶させないで、ステップＳ７７に進む。
【０２３７】
ステップＳ７７において、ブロックデシャフル回路３５６は、外部メモリ３０５に記憶させたＹデータ、Ｕデータ、およびＶデータおよびエラーフラグをフォーマット変換回路３０６に出力し、処理を終了する。
【０２３８】
このように、リカバリオフ入力が”１”である場合、ブロックデシャフル回路３５６は、リカバリされたデータを出力しない。
【０２３９】
以上のように、復号回路３０４は、復号に必要な所定のデータが失われても、ｑビットＭＦリカバリ回路３５３およびＤＲＭＩＮリカバリ回路３５４が失われたデータをリカバリするので、復号を実行することができる。また、復号回路３０４は、復号ができなくても、もっとも違和感の少ないデータを出力することができる。また、リカバリオフ入力を”１”にすることで、リカバリの効果を簡単に確認することができる。
【０２４０】
次に、欠落画素創造回路３０８について説明する。欠落画素創造回路３０８は、プログレッシブ方式の画像を再生するとき、クラス分類適応処理により、図５３（Ａ）に示すように、創造の対象となる画素の同一のフレーム（フレームｔ）にある周辺の画素の画素値、並びにこれらの画素と水平方向および垂直方向に同一の位置の１つ前のフレーム（フレームｔ−１）にある画素の画素値を基に、欠落した画素の画素値などを創造する。また、欠落画素創造回路３０８は、クラス分類適応処理により、図５３（Ｂ）に示すように、創造の対象となる画素の同一のフィールド（フィールドｔ）にある周辺の画素の画素値、これらの画素と水平方向および垂直方向に同一の位置の１つ前のフィールド（フィールドｔ−１）、並びに創造の対象となる画素の周辺の画素と水平方向および垂直方向に同一の位置の２つ前のフィールド（フィールドｔ−２）にある画素の画素値を基に、欠落した画素の画素値などを創造する。
【０２４１】
図５４は、欠落画素創造回路３０８の構成を示すブロック図である。欠落画素創造回路３０８に入力されたデータである画素値および画素の欠落を示すエラーフラグは、前処理回路５０１およびタップ構築回路５０２−１に供給される。
【０２４２】
前処理回路５０１は、入力された画素値および画素の欠落を示すエラーフラグを基に、欠落している画素の値を線形補間フィルタで生成し、欠落している画素にその値を設定し、タップ構築回路５０２−２乃至５０２−５に供給する。なお、画素値は、タップデータとも称する。
【０２４３】
前処理回路５０１の処理を図５５のフローチャートを参照して説明する。ステップＳ１３１において、前処理回路５０１は、エラーフラグを基に、対象となる画素が欠落しているか否かを判定し、対象となる画素が欠落していないと判定された場合、ステップＳ１３２に進み、対象となる画素の画素値を、対象となる画素に設定し、処理は終了する。
【０２４４】
ステップＳ１３１において、対象となる画素が欠落していると判定された場合、ステップＳ１３３に進み、前処理回路５０１は、エラーフラグを基に、対象となる画素の水平に隣接している画素のいずれかが欠落しているか否かを判定する。ステップＳ１３３において、対象となる画素の水平に隣接している画素のいずれも欠落していないと判定された場合、ステップＳ１３４に進み、前処理回路５０１は、対象となる画素の水平に隣接している２つの画素の画素値の平均値を対象となる画素の画素値に設定し、処理は終了する。
【０２４５】
ステップＳ１３３において、対象となる画素の水平に隣接している画素のいずれかが欠落していると判定された場合、ステップＳ１３５に進み、前処理回路５０１は、対象となる画素の水平に隣接している画素のいずれも欠落しているか否かを判定する。ステップＳ１３５において、対象となる画素の水平に隣接している画素のいずれかが欠落していないと判定された場合、ステップＳ１３６に進み、前処理回路５０１は、対象となる画素の水平に隣接している、欠落していない画素の画素値を対象となる画素の画素値に設定し、処理は終了する。
【０２４６】
ステップＳ１３５において、対象となる画素の水平に隣接している画素のいずれも欠落していると判定された場合、ステップＳ１３７に進み、前処理回路５０１は、エラーフラグを基に、対象となる画素の垂直に隣接している画素のいずれかが欠落しているか否かを判定する。ステップＳ１３７において、対象となる画素の垂直に隣接している画素のいずれも欠落していないと判定された場合、ステップＳ１３８に進み、前処理回路５０１は、対象となる画素の垂直に隣接している２つの画素の画素値の平均値を対象となる画素の画素値に設定し、処理は終了する。
【０２４７】
ステップＳ１３７において、対象となる画素の垂直に隣接している画素のいずれかが欠落していると判定された場合、ステップＳ１３９に進み、前処理回路５０１は、エラーフラグを基に、対象となる画素に隣接している全ての画素が欠落しているか否かを判定する。ステップＳ１３９において、対象となる画素に隣接しているいずれかの画素が欠落していないと判定された場合、ステップＳ１４０に進み、前処理回路５０１は、対象となる画素に隣接している、欠落していない画素の画素値を対象となる画素の画素値に設定し、処理は終了する。
【０２４８】
ステップＳ１３９において、対象となる画素に隣接している全ての画素が欠落していると判定された場合、ステップＳ１４１に進み、前処理回路５０１は、対象となる画素と同じ位置の過去のフレームの画素の画素値を、対象となる画素の画素値に設定し、処理は終了する。
【０２４９】
以上のように、前処理回路５０１は、対象となる画素の画素値を周辺の画素の画素値から線形に補間する。前処理回路５０１による補間処理により、これに続く処理で使用できるタップの範囲を広げることができる。
【０２５０】
タップ構築回路５０２−１乃至５０２−５は、それぞれ、タップデータ等を、所定のタイミングで、動きクラス生成回路５０３、ＤＲクラス生成回路５０４、空間クラス生成回路５０５、および欠落クラス生成回路５０６に供給する。
【０２５１】
動きクラス生成回路５０３は、イニシャライズ回路３０９から供給されたパラメータ、並びにタップ構築回路５０２−１から供給されたエラーフラグおよび選択されたタップを基に、動きクラスコードおよび静動フラグを生成し、タップ構築回路５０２−２乃至２０５−５およびクラス合成回路５０７に出力する。動きクラスコードは、動きの量を示す２ビットの情報を有し、静動フラグは、動きの有り無しを１ビットで示す。
【０２５２】
図５６（Ａ）は、動きクラス生成回路５０３による時間アクティビティの算出に使用される、創造の対象となる画素を中心とした３×３の画素の例を示す図である。図５６（Ａ）において、”エラー”は、欠落した画素を示す。図５６（Ｂ）は、図５６（Ａ）に示された画素に対応する、１つ前のフレームの３×３の画素の例を示す図である。図５６（Ａ）および図５６（Ｂ）に示されるＬ１乃至Ｌ３は、それぞれラインを示し、ラインの同一の番号は、垂直方向に同一の位置であることを示す。図５６（Ａ）および図５６（Ｂ）に示されるＨ１乃至Ｈ３は、それぞれ画素の水平方向の位置を示し、その同一の番号は、水平方向に同一の位置であることを示す。
【０２５３】
図５６（Ａ）および図５６（Ｂ）に示される場合において、時間アクティビティは、式（１２）で算出される。

式（１２）において、（）は、画素の画素値を表し、｜｜は、絶対値を求める関数を表す。
【０２５４】
動きクラス生成回路５０３は、創造の対象となる画素を中心とした３×３の画素の最大値と最小値との差に１を加算した空間アクティビティを算出する。
【０２５５】
図５７は、動きクラス生成回路５０３による空間アクティビティの算出に使用される、創造の対象となる欠落した画素を中心とした３×３の画素の例を示す図である。このとき、空間アクティビティは、式（１３）で算出される。
空間アクティビティ＝Ｍａｘ（ｑｉ）−Ｍｉｎ（ｑｉ）＋１ （１３）
式（１３）において、Ｍａｘ（ｑｉ）は、ｑ１乃至ｑ９の画素値の最大値を示す。Ｍｉｎ（ｑｉ）は、ｑ１乃至ｑ９の画素値の最小値を示す。
【０２５６】
動きクラス生成回路５０３は、空間アクティビティを基に、それぞれ異なる値を有する閾値を選択する。動きクラス生成回路５０３は、動き判定の閾値、および時間アクティビティを基に、動きクラスコードを設定する。
【０２５７】
図５８は、動き判定の閾値を説明する図である。動き判定の閾値は、空間アクティビティの値によって、異なる値が使用される。空間アクティビティが大きくなれば、大きな値の閾値が使用される。これは、空間アクティビティが大きい場合、動きが少なくとも、時間アクティビティが、大きな値となることを考慮したものである。
【０２５８】
なお、スタンダードデンシティのデータに対する処理においては、空間アクティビティとは無関係に固定された閾値を使用して動きクラスコードが設定される。
【０２５９】
次に、動きクラス生成回路５０３の動きクラスコードを設定する処理を、図５９のフローチャートを参照して説明する。ステップＳ１５１において、動きクラス生成回路５０３は、時間アクティビティが、閾値１以下であるか否かを判定し、時間アクティビティが閾値１以下であると判定された場合、ステップＳ１５２に進み、動きクラスコードを０に設定し、処理は終了する。
【０２６０】
ステップＳ１５１において、時間アクティビティが閾値１を越えると判定された場合、ステップＳ１５３に進み、動きクラス生成回路５０３は、時間アクティビティが、閾値２以下であるか否かを判定し、時間アクティビティが閾値２以下であると判定された場合、ステップＳ１５４に進み、動きクラスコードを１に設定し、処理は終了する。
【０２６１】
ステップＳ１５３において、時間アクティビティが閾値２を越えると判定された場合、ステップＳ１５５に進み、動きクラス生成回路５０３は、時間アクティビティが閾値３以下であるか否かを判定し、時間アクティビティが閾値３以下であると判定された場合、ステップＳ１５６に進み、動きクラスコードを２に設定し、処理は終了する。
【０２６２】
ステップＳ１５５において、時間アクティビティが閾値３を越えると判定された場合、ステップＳ１５７に進み、動きクラス生成回路５０３は、動きクラスコードを３に設定し、処理は終了する。
【０２６３】
以上のように、動きクラス生成回路５０３は、閾値および時間アクティビティを基に、動きクラスコードを設定する。
【０２６４】
動きクラス生成回路５０３は、複数の画素の動きクラスコードを基に、再度、動きフラグを設定する。例えば、図６０に示すように、創造の対象となる画素の周辺の画素の動きクラスコードを基に、動きクラス生成回路５０３は、対象となる画素の動きクラスコードを設定する。
【０２６５】
動きクラス生成回路５０３の動きクラスコードを設定する処理を図６１のフローチャートを参照して説明する。ステップＳ１７１において、動きクラス生成回路５０３は、７つの画素の動きクラスコードのうち、３の値が設定されている動きクラスコードの数が、閾値３より大きいか否かを判定し、３の値が設定されている動きクラスコードの数が、閾値３より大きいと判定された場合、ステップＳ１７２に進み、動きクラスコードに３を設定し、処理は終了する。
【０２６６】
ステップＳ１７１において、３の値が設定されている動きクラスコードの数が、閾値３以下であると判定された場合、ステップＳ１７３に進み、動きクラス生成回路５０３は、３の値が設定されている動きクラスコードの数と２の値が設定されている動きクラスコードの数を加えた値が、閾値２より大きいか否かを判定し、３の値が設定されている動きクラスコードの数と２の値が設定されている動きクラスコードの数を加えた値が、閾値２より大きいと判定された場合、ステップＳ１７４に進み、動きクラスコードに２を設定し、処理は終了する。
【０２６７】
ステップＳ１７３において、３の値が設定されている動きクラスコードの数と２の値が設定されている動きクラスコードの数を加えた値が、閾値２以下であると判定された場合、ステップＳ１７５に進み、動きクラス生成回路５０３は、３の値が設定されている動きクラスコードの数、２の値が設定されている動きクラスコードの数、および１の値が設定されている動きクラスコードの数を加えた値が、閾値１より大きいか否かを判定し、３の値が設定されている動きクラスコードの数、２の値が設定されている動きクラスコードの数、および１の値が設定されている動きクラスコードの数を加えた値が、閾値１より大きいと判定された場合、ステップＳ１７６に進み、動きクラスコードに１を設定し、処理は終了する。
【０２６８】
ステップＳ１７５において、３の値が設定されている動きクラスコードの数、２の値が設定されている動きクラスコードの数、および１の値が設定されている動きクラスコードの数を加えた値が、閾値１以下であると判定された場合、ステップＳ１７７に進み、動きクラス生成回路５０３は、動きクラスコードに０を設定し、処理は終了する。
【０２６９】
このように、動きクラス生成回路５０３は、複数の画素の動きクラスコード、並びに予め記憶している閾値を基に、最終的な、動きクラスコードを設定する。
【０２７０】
以上のように、動きクラス生成回路５０３は、複数の画素の画素値から動きクラスコードを設定し、出力する。動きクラス生成回路５０３は、動きクラスコードを基に、静動フラグを設定し、出力する。例えば、動きクラスコードが０または１のとき、静動フラグは、０に設定され、動きクラスコードが２または３のとき、静動フラグは、１に設定される。
【０２７１】
タップ構築回路５０２−２は、動きクラス生成回路５０３から供給された動きクラスコードおよび静動フラグ、並びに欠落画素の位置を基に、全てのクラス構造を網羅した予測タップを選択して、可変タップ選択回路５０８に供給する。
【０２７２】
タップ構築回路５０２−３は、動きクラス生成回路５０３から供給された動きクラスコードおよび静動フラグを基に、タップおよびクラスタップを選択して、エラーフラグとともに、ＤＲクラス生成回路５０４に供給する。ＤＲクラス生成回路５０４は、タップ構築回路５０２−３から供給されたタップ、クラスタップ、エラーフラグを基に、ＤＲクラスコードを生成して、クラス合成回路５０７に出力する。
【０２７３】
タップ構築回路５０２−４は、動きクラス生成回路５０３から供給された動きクラスコードおよび静動フラグを基に、タップおよびクラスタップを選択して、エラーフラグとともに、空間クラス生成回路５０５に供給する。空間クラス生成回路５０５は、タップ構築回路５０２−４から供給されたタップ、クラスタップ、エラーフラグを基に、空間クラスコードを生成して、クラス合成回路５０７に出力する。
【０２７４】
タップ構築回路５０２−５は、動きクラス生成回路５０３から供給された動きクラスコードおよび静動フラグを基に、タップを選択して、エラーフラグとともに、欠落クラス生成回路５０６に供給する。欠落クラス生成回路５０６は、タップ構築回路５０２−５から供給されたタップ、エラーフラグを基に、欠落クラスコードを生成して、クラス合成回路５０７に出力する。
【０２７５】
クラス合成回路５０７は、動きクラスコード、静動フラグ、ＤＲクラスコード、空間クラスコード、および欠落クラスコードを基に、クラスコードを統合し、最終的なクラスコードを形成して、係数保持クラスコード選択回路５０９に出力する。
【０２７６】
係数保持クラスコード選択回路５０９は、イニシャライズ回路３０９から供給された係数セット、および予測構造、並びにクラス合成回路５０７から供給されたクラスコードを基に、可変タップ選択回路５０８に、可変タップを選択させるとともに、予測係数列を推定予測演算回路５１０に出力する。
【０２７７】
図６２乃至図６５を基に、可変タップ選択回路５０８が選択する予測タップデータ列を説明する。
【０２７８】
図６２は、動きクラスコードが０のとき、タップ構築回路５０２−２が可変タップ選択回路５０８に出力する予測タップ可変選択範囲と、可変タップ選択回路５０８が選択する予測タップデータ列の例を示す図である。図６２において、実線で囲んだタップは、タップ構築回路５０２−２が出力する予測タップ可変選択範囲を示し、点線で囲んだタップは、可変タップ選択回路５０８が選択する予測タップデータ列を示す。可変タップ選択回路５０８が選択する予測タップデータ列は、タップ構築回路５０２−２が出力する予測タップ可変選択範囲に含まれるタップから構成される。
【０２７９】
例えば、動きクラスコードが０のとき、タップ構築回路５０２−２は、隣接するタップを選択する。例えば、動きクラスコードが０のとき、可変タップ選択回路５０８は、隣接するタップを選択する。
【０２８０】
図６３は、動きクラスコードが１のとき、タップ構築回路５０２−２が出力する予測タップ可変選択範囲と、可変タップ選択回路５０８が選択する予測タップデータ列を示す図である。図６３において、実線で囲んだタップは、タップ構築回路５０２−２が出力する予測タップ可変選択範囲を示し、点線で囲んだタップは、可変タップ選択回路５０８が選択する予測タップデータ列を示す。可変タップ選択回路５０８が選択する予測タップデータ列は、タップ構築回路５０２−２が出力する予測タップ可変選択範囲に含まれるタップから構成される。
【０２８１】
例えば、動きクラスコードが１のとき、タップ構築回路５０２−２は、隣接するタップと、その隣接するタップとは隣接しないタップを選択する。例えば、動きクラスコードが０のとき、可変タップ選択回路５０８は、隣接する５個のタップと、その隣接する５個のタップとは離れた隣接する５個のタップを選択する。
【０２８２】
図６４は、欠落タップが存在して、動きクラスコードが０のとき、タップ構築回路５０２−２が出力する予測タップ可変選択範囲と、可変タップ選択回路５０８が選択する予測タップデータ列を示す図である。図中の黒丸は、欠落タップを示す。図６４において、実線で囲んだタップは、タップ構築回路５０２−２が出力する予測タップ可変選択範囲を示し、点線で囲んだタップは、可変タップ選択回路５０８が選択する予測タップデータ列を示す。可変タップ選択回路５０８が選択する予測タップデータ列は、タップ構築回路５０２−２が出力する予測タップ可変選択範囲に含まれるタップから構成される。
【０２８３】
例えば、欠落タップが存在して、動きクラスコードが０のとき、タップ構築回路５０２−２は、隣接するタップを選択する。例えば、欠落タップが存在して、動きクラスコードが０のとき、可変タップ選択回路５０８は、欠落タップが存在しないときと同じ数の、隣接するタップ（例えば、欠落タップの隣のタップまたは欠落タップと同じ並びのタップ）を選択する。
【０２８４】
図６５は、欠落タップが存在して、動きクラスコードが１のとき、タップ構築回路５０２−２が出力する予測タップ可変選択範囲と、可変タップ選択回路５０８が選択する予測タップデータ列を示す図である。図中の黒丸は、欠落タップを示す。図６４において、実線で囲んだタップは、タップ構築回路５０２−２が出力する予測タップ可変選択範囲を示し、点線で囲んだタップは、可変タップ選択回路５０８が選択する予測タップデータ列を示す。可変タップ選択回路５０８が選択する予測タップデータ列は、タップ構築回路５０２−２が出力する予測タップ可変選択範囲に含まれるタップから構成される。
【０２８５】
例えば、欠落タップが存在して、動きクラスコードが１のとき、タップ構築回路５０２−２は、隣接するタップと、その隣接するタップとは隣接しないタップを選択する。例えば、欠落タップが存在して、動きクラスコードが０のとき、可変タップ選択回路５０８は、欠落タップが存在しないときと同じ数の、隣接するタップ（例えば、欠落タップの隣のタップまたは欠落タップと同じ並びのタップ）と、その隣接するタップとは離れた、欠落タップが存在しないときと同じ数の、隣接するタップを選択する。
【０２８６】
推定予測演算回路５１０は、イニシャライズ回路３０９から設定された出力モードを基に、可変タップ選択回路５０８から供給された予測タップデータ列、および係数保持クラスコード選択回路５０９から供給された予測係数列を基に、線形推定式を用いて、欠落した画素の画素値を算出する。
【０２８７】
このように、欠落画素創造回路３０８は、ダイナミックレンジ、動き、欠落、および画素値の変化などによるクラス分類により、欠落した画素の周辺の画素値を基に、欠落した画素値を算出する。
【０２８８】
また、推定予測演算回路５１０は、入力された画像の画質をアップさせる（階調の増加（Ｙデータ、Ｕデータ、およびＶデータのビット数の増加）、ノイズの除去、量子化歪みの除去（時間方向の歪みの除去を含む）、４倍密度の解像度の創造など）ことができる。
【０２８９】
次に、４倍密解像度創造回路３１２について説明する。４倍密解像度創造回路３１２は、インターレース方式の画像を出力するとき、図６６（Ａ）および図６６（Ｂ）に示すように、クラス分類適応処理を利用し、２つのフィールドの所定の数の画素の画素値を基に、入力された２つのフィールドの間に２つの画素を創造する。
【０２９０】
４倍密解像度創造回路３１２は、プログレッシブ方式の画像を出力するとき、図６６（Ｃ）および図６６（Ｄ）に示すように、クラス分類適応処理を利用し、図６６に示すように、２つのフレームの所定の数の画素の画素値を基に、入力された２つのフレームの間に２つの画素を創造する。
【０２９１】
図６７は、４倍密解像度創造回路３１２の構成を示すブロック図である。フォーマット変換回路３１０から供給されるＹデータ、Ｕデータ、およびＶデータは、タップ選択回路６０１、タップ選択回路６０２、およびタップ選択回路６０３に供給される。タップ選択回路６０１は、レジスタ６０９に格納されているタップの位置情報に従って、密度を４倍にするために創造する画素の画素値を算出するのに使用する画素を選択する。タップ選択回路６０１により選択された画素は、推定予測演算回路６０７に供給される。
【０２９２】
タップ選択回路６０２は、レジスタ６１０に格納されているタップの位置情報に従って、創造する画素の近傍の画素の画素値のうち、空間クラス分類に使用する画素を選択する。タップ選択回路６０２により選択された画素は、空間クラス検出回路６０４に供給される。空間クラス検出回路６０４は、空間クラスを検出する。検出された空間クラスは、クラス合成回路６０６に供給される。
【０２９３】
タップ選択回路６０３は、レジスタ６１１に格納されているタップの位置情報に従って、創造する画素の近傍の画素に基づいて動きに対応するクラス分類に使用する画素を選択するものである。タップ選択回路６０３により選択された画素は、動きクラス検出回路６０５に供給される。動きクラス検出回路６０５は、動きクラスを検出する。検出された動きクラスは、クラス合成回路６０６およびレジスタ６１０に供給される。レジスタ６１０のタップの位置情報は、動きクラス検出回路６０５から供給された動きクラスによって切り換えられる。
【０２９４】
クラス合成回路６０６は、空間クラスおよび動きクラスを統合し、最終的なクラスコードを形成する。
【０２９５】
このクラスコードが係数メモリ６１２に対して、アドレスとして供給され、係数メモリ６１２からクラスコードに対応する係数セットが読み出される。係数セットは、推定予測演算回路６０７に供給される。推定予測演算回路６０７は、タップ選択回路６０１から供給された画素の画素値、および係数メモリ６１２から供給された係数セットを基に、線形推定式を用いて、創造する画素の画素値を算出する。推定予測演算回路６０７からの出力は、ラインデータＬ１およびラインデータＬ２として、線順次変換回路６１３に供給される。
【０２９６】
係数セットは、教師信号としての、１つのフィールド当たり、水平方向に５２８画素および垂直方向に２４０画素を含むＹデータ、それぞれ１つのフィールド当たり、水平方向に１７６画素および垂直方向に１２０画素を含むＵデータおよびＶデータを用いて、クラス毎の係数セットが予め生成されて、イニシャライズ回路３１３に記憶されている。
【０２９７】
線順次変換回路６１３は、ラインメモリを有し、推定予測演算回路６０７から出力されるラインデータＬ１およびラインデータＬ２をレジスタ６０８に格納されている出力タイミングのデータに基づき、線順次で出力する。線順次変換回路６１３から、供給された画像の４倍の密度の画像のデータが出力される。
【０２９８】
レジスタ６０８、レジスタ６０９、レジスタ６１０、レジスタ６１１、および係数メモリ６１２は、イニシャライズ回路３１３に初期化信号が供給されたとき、イニシャライズ回路３１３から供給された係数セット等を格納する。
【０２９９】
図６８は、４倍密解像度創造回路３１２がインターレース方式で４倍密解像度の画素を生成、出力する場合における、４倍密解像度創造回路３１２に入力される画素（図中、大きい円で示す）の位置、および４倍密解像度創造回路３１２が出力する画素（図中、小さい円で示す）の位置を示す図である。４倍密解像度創造回路３１２に入力されるフレームは、３０Ｈｚの周波数を有し、４倍密解像度創造回路３１２が出力するフィールドは、６０Ｈｚの周波数を有する。出力される第１のフィールドの画素は、入力された画素から、垂直方向に出力される画面のラインの１ライン分だけ上にずれて出力され、１ライン当たりの画素の数は、２倍になっている。出力される第２のフィールドの画素は、入力された画素から、垂直方向に出力される画面のラインの１ライン分だけ下にずれて出力され、１ライン当たりの画素の数は、２倍になっている。
【０３００】
図６９は、４倍密解像度創造回路３１２がプログレッシブ方式で４倍密解像度の画素を生成して、出力するときの、４倍密解像度創造回路３１２に入力される画素（図中、大きい円で示す）の位置、および４倍密解像度創造回路３１２が出力する画素（図中、小さい円で示す）の位置を示す図である。４倍密解像度創造回路３１２に入力されるフレームは、３０Ｈｚの周波数を有し、４倍密解像度創造回路３１２が出力するフレームは、６０Ｈｚの周波数を有する。出力されるフレームの画素は、入力された画素と同じライン上に出力され、１ライン当たりの画素の数は、２倍になっている。
【０３０１】
以上のように、４倍密解像度創造回路３１２は、入力された画像を基に、２倍のフレーム周波数また２倍のフィールド周波数で、１ライン当たり２倍の画素の数を有する画像を創造して出力する。
【０３０２】
次に、ＵＶ垂直２倍回路３３３について説明する。図７０は、ＵＶ垂直２倍回路３３３の構成を示すブロック図である。ＵＶ垂直２倍回路３３３には、４倍密解像度創造回路３１２により、画素の数が増やされたＵデータおよびＶデータが入力される。入力されたデータは、ラインＦＩＦＯ７０１−１乃至７０１−６により、１ライン分ずつ遅延され、順次後段に出力される。
【０３０３】
乗算回路７０２−１は、入力されたデータに係数を乗じて、加算回路７０３に出力する。
【０３０４】
乗算回路７０２−２乃至７０２−７は、それぞれラインＦＩＦＯ７０１−１乃至７０１−６より、入力されたデータに係数を乗じて、加算回路７０３に出力する。
【０３０５】
加算回路７０３は、乗算回路７０２−１乃至７０２−７から供給されたデータを加算し、ラッチ回路７０４に供給する。ラッチ回路７０４は、クロック信号が入力されたとき、入力されたデータをラッチする。
【０３０６】
ＵＶ垂直２倍回路３３３の動作を図７１を参照して説明する。ラインＦＩＦＯ７０１−１乃至７０１−６は、入力されたラインに連続する６本のライン上の、画面の水平方向に同一の位置の画素（図７１に丸印で示す画素）のデータを出力する。乗算回路７０２−１乃至７０２−７は、入力されたデータに、それぞれ係数を乗じて、加算回路７０３に出力する。加算回路７０３は、出力されたデータを加算し、ラッチ回路７０４に出力する。ラッチ回路７０４には、ラインの本数が２倍となるタイミング（図７１に黒い３角形で示すタイミング）で、クロック信号が入力される。従って、ＵＶ垂直２倍回路３３３は、入力されたＵデータおよびＶデータのラインの本数を２倍に補間して、出力する。
【０３０７】
次に、受信装置２の受信の処理を図７２のフローチャートを参照して説明する。ステップＳ２０１において、パケット解体回路３０２は、受信回路３０１が受信したパケットを解体し、復号回路３０４に供給する。ステップＳ２０２において、復号回路３０４は、パケット解体回路３０２から入力されたデータを、ＡＤＲＣ方式で復号する。伝送中にデータが欠落したとき、復号回路３０４の復号において、ｑ、動きフラグＭＦ、ダイナミックレンジＤＲ、または画素値の最小値ＭＩＮがリカバリされ、Ｙデータ、Ｕデータ、およびＶデータが復号される。
【０３０８】
ステップＳ２０３において、フォーマット変換回路３０６は、復号回路３０４から供給されたＹデータ、Ｕデータ、およびＶデータを、欠落画素創造回路３０８が処理できる所定のフォーマットに変換する。ステップＳ２０４において、欠落画素創造回路３０８は、欠落した画素のうち、復号回路３０４で復号できなかった画素を、クラス適応処理により、創造する。
【０３０９】
ステップＳ２０５において、フォーマット変換回路３１０は、欠落画素創造回路３０８から供給されたＹデータ、Ｕデータ、およびＶデータを、４倍密解像度創造回路３１２が処理できる所定のフォーマットに変換する。ステップＳ２０６において、４倍密解像度創造回路３１２は、入力されたＹデータ、Ｕデータ、およびＶデータを基に、クラス適応処理により画素を創造し、画素の密度を４倍にして、補間部３１４に出力する。
【０３１０】
ステップＳ２０７において、フォーマット変換回路３３１は、４倍密解像度創造回路３１２から供給されたＹデータ、Ｕデータ、およびＶデータを、所定の補間の処理ができるように、データ形式を変更する。ステップＳ２０８において、ＵＶ垂直２倍回路３３３は、ＵデータおよびＶデータを垂直方向に画素を補間して、ライン数が２倍になるようにする。ステップＳ２０９において、Ｄ／Ａ変換回路３３５は、所定のクロックレートでデジタルアナログ変換し、アナログＹ信号、アナログＵ信号、およびアナログＶ信号を出力し、処理は終了する。
【０３１１】
以上のように、受信装置２は、伝送路でデータが欠落しても、復号することができ、また、欠落した画素を創造して、欠落した画素の影響をほとんど受けずに伸張し、画像の信号を出力することができる。
【０３１２】
なお、画像伝送システムは、画像を送受信するとして説明したが、画像に限らず、音声などのデータを伝送するようにしてもよい。
【０３１３】
上述した一連の処理は、ハードウェアにより実行させることもできるが、ソフトウェアにより実行させることもできる。一連の処理をソフトウェアにより実行させる場合には、そのソフトウェアを構成するプログラムが、専用のハードウェアに組み込まれているコンピュータ、または、各種のプログラムをインストールすることで、各種の機能を実行することが可能な、例えば汎用のパーソナルコンピュータなどに、記録媒体からインストールされる。
【０３１４】
図７３は、記録媒体およびコンピュータの例を説明する図である。CPU（Central Processing Unit）８１１は、各種アプリケーションプログラムや、OS（Operating System)を実際に実行する。ROM（Read-only Memory）８１２は、一般的には、CPU８１１が使用するプログラムや演算用のパラメータのうちの基本的に固定のデータを格納する。RAM（Random-Access Memory）８１３は、CPU８１１の実行において使用するプログラムや、その実行において適宜変化するパラメータを格納する。これらはCPUバスなどから構成されるホストバス８１４により相互に接続されている。
【０３１５】
ホストバス８１４は、ブリッジ８１５を介して、PCI(Peripheral Component Interconnect/Interface)バスなどの外部バス８１６に接続されている。
【０３１６】
キーボード８１８は、CPU８１１に各種の指令を入力するとき、使用者により操作される。マウス８１９は、ディスプレイ８２０の画面上のポイントの指示や選択を行うとき、使用者により操作される。ディスプレイ８２０は、液晶表示装置またはCRT（Cathode Ray Tube）などから成り、各種情報をテキストやイメージで表示する。HDD（Hard Disk Drive）８２１は、ハードディスクを駆動し、それらにCPU８１１によって実行するプログラムや情報を記録または再生させる。
【０３１７】
ドライブ８２２は、装着されている磁気ディスク８４１、光ディスク８４２、光磁気ディスク８４３、または半導体メモリ８４４に記録されているデータまたはプログラムを読み出して、そのデータまたはプログラムを、インターフェース８１７、外部バス８１６、ブリッジ８１５、およびホストバス８１４を介して接続されているRAM８１３に供給する。
【０３１８】
これらのキーボード８１８乃至ドライブ８２２は、インターフェース８１７に接続されており、インターフェース８１７は、外部バス８１６、ブリッジ８１５、およびホストバス８１４を介してCPU８１１に接続されている。
【０３１９】
記録媒体は、図７３に示すように、コンピュータとは別に、ユーザにプログラムを提供するために配布される、プログラムが記録されている磁気ディスク８４１（フロッピディスクを含む）、光ディスク８４２（CD-ROM(Compact Disc-Read Only Memory)、ＤＶＤ(Digital Versatile Disc)を含む）、光磁気ディスク８４３（ＭＤ(Mini-Disc)を含む）、若しくは半導体メモリ８４４などよりなるパッケージメディアにより構成されるだけでなく、コンピュータに予め組み込まれた状態でユーザに提供される、プログラムが記録されているROM８１２や、HDD８２１などで構成される。
【０３２０】
なお、本明細書において、記録媒体に格納されるプログラムを記述するステップは、記載された順序に沿って時系列的に行われる処理はもちろん、必ずしも時系列的に処理されなくとも、並列的あるいは個別に実行される処理をも含むものである。
【０３２１】
また、本明細書において、システムとは、複数の装置により構成される装置全体を表すものである。
【０３２２】
【発明の効果】
以上の如く本発明によれば、乗算のためのハードウェアの規模を小さくし、係数テーブルを記憶するメモリ量を小さくすることができる。
【図面の簡単な説明】
【図１】従来のパケットの構造を説明する図である。
【図２】画像を送信する画像伝送システムを説明する図である。
【図３】本発明に係る送信装置１の一実施の形態の構成を示すブロック図である。
【図４】送信装置１に入力された画像信号が圧縮される過程を説明する図である。
【図５】間引きフィルタ４３の構成を示すブロック図である。
【図６】ラッチ回路８１の出力を説明する図である。
【図７】ラッチ回路８４の出力を説明する図である。
【図８】ＵＶ垂直１／２回路４５の構成を示すブロック図である。
【図９】ラッチ回路９４の出力を説明する図である。
【図１０】符号化回路１５の構成を示すブロック図である。
【図１１】処理の単位を説明する図である。
【図１２】ＡＤＲＣブロックを説明する図である。
【図１３】フレーム内の画素とＡＤＲＣブロックの関係を示す図である。
【図１４】フレーム内の画素とＡＤＲＣブロックの関係を示す図である。
【図１５】フレーム内の画素とＡＤＲＣブロックの関係を示す図である。
【図１６】ＹデータのＡＤＲＣブロックおよびセグメントを説明する図である。
【図１７】ＵデータまたはＶデータのＡＤＲＣブロックおよびセグメントを説明する図である。
【図１８】ＡＤＲＣブロック単位で行われるシャフルを説明する図である。
【図１９】セグメントを説明する図である。
【図２０】ＡＤＲＣ符号化回路１０３の構成を説明する図である。
【図２１】情報量制御回路１６４の構成を説明する図である。
【図２２】情報量制御回路１６４の構成を説明する図である。
【図２３】セグメント間シャフル回路１０４のシャフルの仕方を説明する図である。
【図２４】セグメント間シャフル回路１０４のシャフルの仕方を説明する図である。
【図２５】セグメント間シャフル回路１０４のシャフルの仕方を説明する図である。
【図２６】セグメント間シャフル回路１０４のシャフルの仕方を説明する図である。
【図２７】パケットの構成を示す図である。
【図２８】セグメント間シャフル回路１０４の構成を示すブロック図である。
【図２９】入力データカウンタ部２１２のカウンタ群について説明する図である。
【図３０】出力データカウンタ部２１６のカウンタ群について説明する図である。
【図３１】送信装置１の画像の送信の処理を説明する図である。
【図３２】受信装置２の構成を示すブロック図である。
【図３３】画像が伸張される過程を説明する図である。
【図３４】復号回路３０４のブロック図である。
【図３５】ｑビットＭＦリカバリ回路３５３の構成を示すブロック図である。
【図３６】ダイナミックレンジＤＲまたは画素値の最小値ＭＩＮのリカバリの処理を説明する図である。
【図３７】ダイナミックレンジＤＲまたは画素値の最小値ＭＩＮのリカバリの処理を説明する図である。
【図３８】ダイナミックレンジＤＲまたは画素値の最小値ＭＩＮのリカバリの処理を説明する図である。
【図３９】セグメント間デシャフル回路３５２から出力されるデータに欠落がない場合の、復号回路３０４の処理のタイミングの例を示す図である。
【図４０】セグメント間デシャフル回路３５２から出力されるデータに欠落がある場合の、復号回路３０４の処理のタイミングの例を示す図である。
【図４１】従来のデータの記憶の方法を示す図である。
【図４２】ブロックデシャフル回路３５６のデータの出力のタイミングを説明する図である。
【図４３】リカバリエラー判定回路３５７の構成を示すブロック図である。
【図４４】リカバリエラー判定回路３５７が、”１”が設定されたリカバリ判定フラグＦｄを出力するときの、ブロックデシャフル回路３５６の処理を説明する図である。
【図４５】セグメント間デシャフル回路３５３が、連続エラー信号を出力するときの、ブロックデシャフル回路３５６の処理を説明する図である。
【図４６】グレイモードにおけるブロックデシャフル回路３５６の処理を説明する図である。
【図４７】ブロックデシャフル回路３５６のリカバリオフ入力に対応した処理を説明する図である
【図４８】ブロックデシャフル回路３５６のリカバリオフ入力に対応した処理を説明する図である
【図４９】ブロックデシャフル回路３５６のエスケープ信号に対応した処理を説明する図である。
【図５０】ブロックデシャフル回路３５６の復号したデータを出力する処理を説明するフローチャートである。
【図５１】ブロックデシャフル回路３５６の復号したデータを出力する処理を説明するフローチャートである。
【図５２】リカバリオフ入力が”１”であるときのブロックデシャフル回路３５６のデータを出力する処理を説明するフローチャートである。
【図５３】欠落画素創造回路３０８の処理に使用される画素を説明する図である。
【図５４】欠落画素創造回路３０８の構成を示すブロック図である。
【図５５】前処理回路５０１の処理を説明するフローチャートである。
【図５６】時間アクティビティの算出に使用される画素の例を示す図である。
【図５７】空間アクティビティの算出に使用される画素の例を示す図である。
【図５８】動き判定の閾値を説明する図である。
【図５９】静動判定回路５２１の動きクラスコードを設定する処理を説明するフローチャートである。
【図６０】動きクラスコードの多数決判定に使用される画素の例を示す図である。
【図６１】動きクラス生成回路５０３の動きクラスコードを設定する処理を説明するフローチャートである。
【図６２】可変タップ選択回路５０８が選択する予測タップデータ列を説明する図である。
【図６３】可変タップ選択回路５０８が選択する予測タップデータ列を説明する図である。
【図６４】可変タップ選択回路５０８が選択する予測タップデータ列を説明する図である。
【図６５】可変タップ選択回路５０８が選択する予測タップデータ列を説明する図である。
【図６６】４倍密解像度創造回路３１２の処理に使用される画素の位置を説明する図である。
【図６７】４倍密解像度創造回路３１２の構成を示すブロック図である。
【図６８】４倍密解像度創造回路３１２がインターレース方式で出力する場合の４倍密解像度創造回路３１２に入力される画素の位置および４倍密解像度創造回路３１２が出力する画素の位置を示す図である。
【図６９】４倍密解像度創造回路３１２がプログレッシブ方式で出力する場合の４倍密解像度創造回路３１２に入力される画素の位置および４倍密解像度創造回路３１２が出力する画素の位置を示す図である。
【図７０】ＵＶ垂直２倍回路３３３の構成を示すブロック図である。
【図７１】ラッチ回路７０４の出力を説明する図である。
【図７２】受信装置２の受信の処理を説明するフローチャートである。
【図７３】記録媒体およびコンピュータの例を説明する図である。
【符号の説明】
１ 送信装置， ２ 受信装置， １３ 間引き部， １５ 符号化回路， １７ パケット化回路， ４２ Ａ／Ｄ変換回路， ４３ 間引きフィルタ， ４５ ＵＶ垂直１／２回路， １０２ ブロックシャフル回路， １０３ ＡＤＲＣ符号化回路， １０４ セグメント間シャフル回路， １６４ 情報量制御回路， １７１ ＡＤＲＣエンコーダ， １０４ セグメント間シャフル回路，１１１ データ選択部， ２１２ 入力データカウンタ部、 ２１３ 入出力コントローラ， ２１４ アドレス発生部， ２１５ 記憶部， ２１６ 出力データカウンタ部， ３０２ パケット解体回路， ３０４ 復号回路， ３０５ 外部メモリ， ３０８ 欠落画素創造回路， ３１２ ４倍密解像度創造回路， ３３３ ＵＶ垂直２倍回路， ３３５ Ｄ／Ａ変換回路， ３５２ セグメント間デシャフル回路， ３５３ ｑビットＭＦリカバリ回路， ３５４ ＤＲＭＩＮリカバリ回路， ３５５ ＡＤＲＣ復号回路， ３５６ ブロックデシャフル回路， ３５７ リカバリエラー判定回路， ５０１ 前処理回路， ５０２−１乃至５０２−５ タップ構築回路， ５０３ 動きクラス生成回路， ５０４ ＤＲクラス生成回路， ５０５ 空間クラス生成回路， ５０６ 欠落クラス生成回路， ５０７ クラス合成回路， ５０８ 可変タップ選択回路，５０９ 係数保持クラスコード選択回路， ５１０ 推定予測演算回路[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an information processing apparatus and method, and more particularly, to an information processing apparatus and method using a memory map in which a combination of address bits corresponds to a count counter value of input data selection, counting, and memory addressing destination on a one-to-one basis. .
[0002]
[Prior art]
In many compression methods in data transmission, variable length coding represented by entropy coding is used to improve the compression ratio. The feature of this entropy coding is that the compression is realized by changing the length of the code word in accordance with the content of the data to be compressed. For example, a short codeword is assigned to data with a high occurrence frequency, and a long codeword is assigned to data with a low occurrence frequency. As a result, the information amount of the entire data to be transmitted is reduced.
[0003]
When incorrect data is received on the receiving side due to a communication error, the length of the codeword always changes according to the content of the data, so that the decoding error continues up to the reference point of the transmitted data. This phenomenon is generally called “error propagation”.
[0004]
In addition, moving image compression methods represented by MPEG (Moving Picture Experts Group) reduce spatial redundancy by discrete cosine transform and reduce temporal redundancy by transmitting only the difference between frames. Reduce the amount of data by reducing.
[0005]
The moving image data compressed by such a compression method is decompressed at the receiving side. However, when the moving image data transmitted due to a communication error is lost, the lack of the data is expanded spatially and temporally. The higher the compression ratio, the more pronounced the effect.
[0006]
The communication path conditions in the mobile environment are inferior to those of a fixed station or wire, and burst-like data loss called “packet loss” occurs relatively frequently. Therefore, in the communication of moving images using compression in such a mobile environment, shuffling is performed between consecutive data before data is transmitted, and data is distributed, so that packets are lost. However, on the receiving side, the missing packet data can be interpolated by the received packet data.
[0007]
For example, when data encoded by the ADRC (Adaptive Dynamic Range Coding) method is shuffled and then packetized and transmitted, a packet configuration as shown in FIG. 1 has been proposed. FIG. 1 shows a packet configuration when two frames of ADRC data are divided into ATM (Asynchronous Transfer Mode) packet sizes for transmission. One packet includes a dynamic range (DR), a motion flag (MF), a minimum pixel value (MIN), a quantization table index (TI), and a code Q. Hereinafter, as appropriate, DR, MF, MIN, and TI are referred to as FL data, and the code Q is referred to as VL data.
[0008]
One DR data is composed of A bits, MIN data is composed of B bits, MF data is composed of C bits, and TI data is composed of D bits. The size of the data area of one packet is set to 47 bytes, and the remaining number of bits excluding the data amount of the four data DR, MF, MIN, and TI described above from 47 bytes is the data size of the code Q. .
[0009]
There are a total of seven types of packet configurations of type 0 to type 6, and each packet is properly used according to the packet number. The number of included data DR, MF, MIN, and TI is different for each packet type, and the number of bits of the code Q is also different accordingly. In this way, by using different packet types, even when packet loss occurs, the influence is suppressed.
[0010]
[Problems to be solved by the invention]
As described above, when data is shuffled and then transmitted, it must be temporarily stored in a storage device such as a RAM (Random Access Memory) in order to perform shuffling. RAM and the like usually handle data in byte units. Therefore, as described above, when a plurality of packet types are used, when the data in the packet is handled in units of bytes, the bit allocation of each data often shifts in units of bytes, and each data in the packet Since the position differs for each packet type, it is necessary to monitor the bit allocation state and determine the shift amount by calculation, or use a table describing the bit allocation combinations for each packet type. .
[0011]
In addition, a method using a coefficient table in which multiplication or a combination of multiplications is predicted can be considered for the addressing operation when performing shuffling. When multiplication is used for addressing, computation time is required, and the hardware configuration scale may increase. When using a coefficient that predicts a combination of multiplication, when dealing with a plurality of different data as described above, there is a problem that the combination of those values becomes enormous and the amount of memory for storing the coefficient becomes large. .
[0012]
The present invention has been made in view of such a situation. The data is handled as a byte unit, and the combination of the address bits is one-to-one corresponding to the count counter value of the input data selection, counting, and memory addressing destination. An object of the present invention is to reduce the scale of hardware for multiplication and to reduce the amount of memory for storing a coefficient table by using a memory map.
[0013]
[Means for Solving the Problems]
  An information processing apparatus according to an aspect of the present invention stores Y data, U data, and V data generated by converting a component signal in a first memory, and stores the stored Y data, U data, and U data. The data and the V data are rearranged into an ADRC block format, and a shuffle unit that shuffles within a segment range and in units of ADRC blocks, the Y data supplied from the block shuffle unit, the U data, and the Coding means for coding V data based on an ADRC (Adaptive Dynamic Range Coding) system, and storing data supplied from the coding means in a second memory, and changing the addresses of the storage and reading The inter-segment shuffling means for shuffling and the de-shuffling means supplied from the inter-segment shuffle means. Packetizing means for packetizing data into a packet including a dynamic range, a motion flag, a minimum pixel value, a quantization table index, and a code Q in one packet, and the inter-segment shuffling means includes: A data selection unit, an input data counter unit, an input / output controller unit, an address generation unit, and an output data counter unit. The data selection unit includes a dynamic range, a motion flag, a minimum pixel value, and an index of a quantization table. For each element of the first data configured, three pieces are input in parallel, and second data including the code Q is input at a timing alternated with the first data, and the input data is serialized. Convert to data, output to the input / output controller means, the input / output The controller means stores the input data in the second memory based on the address generated by the address generating means, and the input / output controller means stores the data stored in the second memory. Address generation means reads out and outputs to the packetization means based on the address generated according to the data supplied from the output data counter means.
The input data counter means includes a group of counters indicating information for identifying the type of data input to the data selection means, addressing when storing in the second memory, and a direction of shuffling. Can be.
The counter group includes addressing and shuffling, and the first data counter for addressing is a counter that counts the number of blocks of the first data in one buffer. A counter group for counting the second data for addressing, comprising a counter for counting the number of packets of the first data and a counter for counting the total number of packets of the first data in one bank Is a counter that counts the number of packets of the second data in one buffer, a counter that counts the total number of packets of the second data in one bank, and the number of buffers of the second data in one segment A counter that counts the packet, and the second data in any order. A counter that counts the remainder when dividing by the number N of bytes of the second data per hit, and the first data counter for shuffling is the counter of the first data in one bank. A counter that counts the number of segments, and a counter that determines a shuffling direction of the first data in one segment. The second data counter for shuffling is the second counter in one bank. A counter for counting the number of data segments, and a counter for determining the shuffling direction of the second data in one segment.
The output data counter means includes a group of addressing counters, and the first data addressing counter is a counter that counts an index of the quantization table in a data cell of one packet size. A counter that counts the number of motion flags in a data cell, a counter that counts the number of minimum values of the dynamic range and the pixel value in a data cell of one packet size, an index of the quantization table in bit size, and the motion flag A counter that counts the number of blanks inserted to handle as byte-size data, a counter that determines the number of the second data in the packet in one packet, and a count of the number of packets in one buffer It can be made to and a counter.
The data stored in the second memory is read based on the address generated by the address generating means according to the counter group of the output data counter means, and the reading is performed at a predetermined timing. Is input to the inter-segment shuffling means, the process of collecting the index of the quantization table and the motion flag in units of bytes is performed, and when the collection process is completed, according to the arrangement of the data in the packet , Read from the second memory, and output to the packetizing means.
[0014]
According to an information processing method of one aspect of the present invention, Y data, U data, and V data generated by converting a component signal are stored in a first memory, and the stored Y data, U data The data and the V data are rearranged into an ADRC block format, block shuffled within a segment range and in units of ADRC blocks, and the block shuffled Y data, U data, and V data are converted into ADRC ( The data is encoded based on the Adaptive Dynamic Range Coding method, the encoded data is stored in the second memory, and the address of the storage and reading is changed to shuffle between the segments. In one packet, dynamic range, motion flag, minimum pixel value, quantization A first data including a dynamic range, a motion flag, a minimum value of a pixel value, and an index of a quantization table. 3 elements are input in parallel for each element, second data including the code Q is input at a timing alternated with the first data, the input data is converted into serial data, and an address is generated. The input data is stored in the second memory based on the address generated by the means, and the data stored in the second memory is read based on the address generated by the address generating means. Done in
[0015]
In the information processing apparatus and method according to one aspect of the present invention, Y data, U data, and V data generated by converting a component signal are stored in the first memory, and the stored Y data, U data and V data are rearranged in the form of ADRC blocks, shuffled in the range of segments and in units of ADRC blocks, and shuffled Y data, U data, and V data are encoded based on the ADRC method. The encoded data is stored in the second memory, the address of the storage and reading is changed, and the segment is shuffled. In one packet, the dynamic range, the motion flag, the minimum pixel value, It is packetized into a packet containing the quantization table index and code Q. In addition, three inter-segment shuffles are input in parallel for each element of the first data composed of the dynamic range, the motion flag, the minimum pixel value, and the quantization table index. The second data including the code Q is input at an alternate timing, and the input data is converted into serial data. Based on the address generated by the address generating means, the input data is converted into the second data. The data stored in the memory and stored in the second memory is read out based on the address generated by the address generating means.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 2 is a diagram for explaining an image transmission system for transmitting and receiving an image according to the present invention. The transmission apparatus 1 compresses and encodes the input video signal by the method according to the present invention, converts it into a packet format, and transmits it to the reception apparatus 2 via the transmission path. The receiving device 2 receives a packet transmitted via a transmission path, decodes data included in the packet by the method according to the present invention, expands it, and outputs it as a video signal. In a transmission line, data included in a packet may be lost due to congestion or the like. Further, when the processing capacity of an ATM switch (not shown) constituting the transmission path is exceeded, the packet itself may be lost.
[0017]
FIG. 3 is a block diagram showing a configuration of an embodiment of the transmission apparatus 1 according to the present invention. When any one of an analog composite video signal or an analog component signal (Y / C signal) such as a luminance signal Y and a chroma signal C is input to the Y / C separation chroma decoder 11, It is converted into component signals (Y / U / V signals) such as a luminance signal Y and color signals U and V, and supplied to the selector 12. The selector 12 selects one of the Y / U / V signal supplied from the Y / C separation chroma decoder 11 or the signal input in the Y / U / V signal format from the beginning, and supplies the selected signal to the thinning unit 13. . This Y / U / V signal is, for example, a signal of standard density (SD), 4: 2: 2, a field frequency of 60 Hz, and an interlace format.
[0018]
The thinning unit 13 thins the Y signal, the U signal, and the V signal supplied from the selector 12 by a method described later, and supplies the thinned signal to the format conversion circuit 14. The thinning unit 13 includes a pre-filter 41, an A / D conversion circuit 42, a thinning filter 43, an external memory 44, a UV vertical 1/2 circuit 45, and an external memory 46.
[0019]
The prefilter 41 is a low-pass filter that outputs only predetermined frequency bands of the input Y signal, U signal, and V signal. The output of the prefilter 41 is supplied to the A / D conversion circuit 42.
[0020]
The A / D conversion circuit 42 samples the input Y signal, U signal, and V signal, and makes each of them 8-bit data, for example. The analog Y signal input to the A / D conversion circuit 42 is sampled and output to the thinning filter 43 as Y data including 528 pixels in the horizontal direction and 480 pixels in the vertical direction.
[0021]
The analog U signal input to the A / D conversion circuit 42 is sampled and output to the thinning filter 43 as U data including 176 pixels in the horizontal direction and 480 pixels in the vertical direction. The analog V signal input to the A / D conversion circuit 42 is sampled and output to the thinning filter 43 as V data including 176 pixels in the horizontal direction and 480 pixels in the vertical direction.
[0022]
Accordingly, the output of the A / D conversion circuit 42 is a signal having a 3: 1: 1 format.
[0023]
The thinning filter 43 temporarily stores the input Y data, U data, and V data in the external memory 44, and each of the temporarily stored Y data, U data, and V data is set to 1 in the horizontal direction. / 2 and ½ in the vertical direction are thinned out and supplied to the UV vertical ½ circuit 45.
[0024]
The UV vertical 1/2 circuit 45 temporarily stores the Y data, U data, and V data supplied from the thinning filter 43 in the external memory 46, and the temporarily stored U data and V data, respectively. The data is thinned down to 1/2 in the vertical direction, and Y data, U data, and V data are output to the format conversion circuit 14.
[0025]
Therefore, the signal output from the thinning unit 13 is a signal of 3: 0.5: 0.5.
[0026]
The format conversion circuit 14 rearranges the Y data, the U data, and the V data supplied from the thinning unit 13 by a method described later, and supplies the rearranged data to the encoding circuit 15 as a block structure.
[0027]
The encoding circuit 15 temporarily stores the Y data, U data, and V data supplied from the format conversion circuit 14 in the external memory 16, and stores the Y data, U data, and V data stored in the external memory 16. The data is shuffled by changing the storage and reading addresses, encoded by the ADRC (Adaptive Dynamic Range Coding) method, shuffled, and output to the packetizing circuit 17.
[0028]
The ADRC method is a variable length coding method that outputs a variable length code adapted to the dynamic range (DR) of input image data. Image data is divided into blocks (ADRC blocks described later) composed of a plurality of pixels, and a dynamic range that is a difference between the maximum value and the minimum value of pixel values included in each block is detected. Based on the dynamic range of the block, each pixel value is re-quantized with a bit number smaller than the original quantization bit number (for example, 8 bits). The smaller the dynamic range, the smaller the number of bits that can be re-quantized, and while suppressing the increase in quantization distortion, it is possible to remove only the pixel value redundancy and further reduce the amount of data. is there.
[0029]
The ADRC method selects the number of quantization bits in relation to the size of the dynamic range. In order to determine the magnitude relationship of the dynamic range, a threshold value corresponding to movement or the like is used. For example, when assigning any one of 2 bits, 3 bits, or 4 bits as the number of bits for requantization, thresholds T1 and T2 stored in the quantization table for each motion or dynamic range (however, T1 <T2) is used. The same quantization table is also used on the receiving side.
[0030]
In a block having a dynamic range equal to or less than (T1-1), 2 bits are assigned to the code for the pixel value. In a block whose dynamic range is T1 or more and (T2-1) or less, 3 bits are assigned to the code for the pixel value. In a block having a dynamic range of T2 or more, 4 bits are assigned to the code for the pixel value. Let q be the number of bits assigned to the code for the pixel value.
[0031]
Encoding is performed with 88 ADRC blocks created by dividing 2 frames into 30 units (this unit is referred to as a buffer). Here, one quantization table storing threshold values T1 and T2 is selected so that the code generated for one buffer is 16,104 bits or less. The selected quantization table is designated by a table index indicated by TI. By setting the code generated for one buffer to be 16,104 bits or less, the encoded image data information can be set to 8 Mbps.
[0032]
When the dynamic range is greater than 2 to the qth power, the code Q for the pixel value is
[(L-MIN + 0.5) × 2q / DR]
Is calculated by [] Represents truncation after the decimal point. L represents the pixel value, and MIN represents the minimum pixel value of the pixels in the block. DR represents the dynamic range of the block. When the dynamic range is 2 to the qth power or less, the code Q for the pixel value is calculated by L-MIN.
[0033]
The encoded circuit 15 includes, as encoded data, a table index TI that specifies a quantization table, a dynamic range DR, a minimum value MIN of pixel values of pixels in the block, a motion flag MF that indicates motion, and pixel values. The corresponding code Q is output. The table index TI, the dynamic range DR, the minimum value MIN of the pixel values of the pixels in the block, and the length (number of bits) of the motion flag MF indicating motion are fixed at 8 bits. On the other hand, the length of the code Q with respect to the pixel value changes.
[0034]
The packetizing circuit 17 temporarily stores the encoded data supplied from the encoding circuit 15 in the external memory 18, and divides the temporarily stored encoded data into 1.6K bits. The header is added to form a packet, which is supplied to the transmission circuit 19. The transmission circuit 19 modulates the packet supplied from the packetizing circuit 17 using a predetermined transmission method, and transmits the modulated packet via the transmission path.
[0035]
The PLL circuit 20 generates a reference signal synchronized with the image and supplies the reference signal to the control circuit 21. The control circuit 21 is based on the reference signal supplied from the PLL circuit 20 to the decimation unit 13, the format conversion circuit 14, the encoding circuit 15, the external memory 16, the packetizing circuit 17, the external memory 18, and the transmission circuit 19. A control signal is supplied to control the operation of the entire transmission apparatus 1.
[0036]
With reference to FIG. 4, a process of compressing an image signal input to the transmission apparatus 1 will be described. The interlaced Y signal having a field frequency of 60 Hz is converted into Y data of 528 pixels in the horizontal direction and 480 pixels in the vertical direction per field by the A / D conversion circuit 42. Interlaced U and V signals having a field frequency of 60 Hz are converted into U data and V data of 176 pixels in the horizontal direction and 480 pixels in the vertical direction, respectively. The image signal information input to the A / D conversion circuit 42 corresponds to 166 Mbps, and the image data information output from the A / D conversion circuit 42 is 104 Mbps, which is compressed to an information amount of 62%.
[0037]
The Y data including 528 pixels in the horizontal direction and 480 pixels in the vertical direction per frame of the interlace method having a field frequency of 60 Hz is converted into the horizontal direction per frame of the progressive method having a frame frequency of 30 Hz by the thinning filter 43. To 264 pixels and 240 pixels vertically. The U data and V data including 176 pixels in the horizontal direction and 480 pixels in the vertical direction per interlaced frame having a field frequency of 60 Hz are converted into one progressive frame having a frame frequency of 30 Hz by the thinning filter 43. The data is compressed into data including 88 pixels in the horizontal direction and 240 pixels in the vertical direction.
[0038]
The U data and V data including 88 pixels in the horizontal direction and 240 pixels in the vertical direction per frame are respectively compressed by the UV vertical 1/2 circuit 45 into data including 88 pixels in the horizontal direction and 120 pixels in the vertical direction. The The information of the image data output from the UV vertical 1/2 circuit 45 is 21 Mbps and is compressed to 13% of the information amount as compared with the information of the signal input to the transmission apparatus 1.
[0039]
Thus, by performing the thinning process in the thinning unit 13, the configuration of each circuit in the subsequent stage can be simplified, and the bit rate in the transmission path can be set to a practically small value.
[0040]
The U and V data including 88 pixels in the horizontal direction and 120 pixels in the vertical direction per frame of the progressive system having a frame frequency of 30 Hz are converted into two pieces of data of 88 pixels × 120 pixels by the format conversion circuit 14. It is converted to the combined data.
[0041]
Y data having 264 pixels in the horizontal direction and 240 pixels in the vertical direction and U data and V data of 88 pixels × 120 pixels × 2 per frame are encoded by the encoding circuit 15. The encoded image data information is 8 Mbps, and is compressed to an information amount of 5% as compared with the analog signal information input to the transmission apparatus 1.
[0042]
As described above, the transmission device 1 compresses and encodes the image signal.
[0043]
FIG. 5 is a block diagram showing a configuration of the thinning filter 43. The interlaced Y data is supplied to the horizontal ½ circuit 61 along the horizontal line. The horizontal 1/2 circuit 61 includes delay circuits (registers) 71-1 to 71-N, multiplier circuits 72-1 to 72-N, and an adder circuit 73.
[0044]
The Y data sequentially input to the horizontal 1/2 circuit 61 which is a half-band filter is delayed by one pixel in the horizontal direction by the delay circuits 71-1 to 71-N and sequentially output to the subsequent stage. The multiplier circuits 72-1 to 72-N multiply the input Y data (pixels) by 1 / N and output the result to the adder circuit 73. The adder circuit 73 adds the data supplied from the multiplier circuits 72-1 to 72-N and outputs the result to the vertical ½ circuit 62.
[0045]
The vertical 1/2 circuit 62 that is a half-band filter includes a latch circuit 81, a field FIFO (Fast In Fast Out) 82, an adder circuit 83, and a latch circuit 84. Data output from the horizontal ½ circuit 61 is supplied to the latch circuit 81. When the clock signal is input, the latch circuit 81 latches the input data and outputs the latched data.
[0046]
The operation of the horizontal 1/2 circuit 61 will be described with reference to FIG. Y data (pixels indicated by white circles in FIG. 6) sequentially input in the horizontal direction of the frame is held by delay circuits 71-1 to 71-N and coefficients are respectively multiplied by multiplication circuits 72-1 to 72-N. Is multiplied. The outputs of the multiplier circuits 72-1 to 72-N are added by the adder circuit 73 and then output to the latch circuit 81. A clock signal for instructing latching is input to the latch circuit 81 at a timing corresponding to the black square in FIG. In the example of FIG. 6, since one clock is input to the latch circuit 81 every time data of two pixels is transferred in the horizontal direction, for example, when N = 2 and the value of each coefficient is 1/2 The average value of two adjacent pixels is latched by the latch circuit 81 at the timing indicated by the black square in FIG. As described above, the number of pixels latched by the latch circuit 81 is ½ of the number of pixels of data input to the thinning filter 43.
[0047]
The value latched by the latch circuit 81 is supplied to the field FIFO 82 and the adder circuit 83. The field FIFO 82 stores the pixel data of the first field thinned by 1/2 in the horizontal direction supplied from the latch circuit 81, delays it by one field, and outputs it to the adder circuit 83. The adder circuit 83 adds the data supplied from the latch circuit 81 and the field FIFO 82 and supplies the sum to the latch circuit 84. When the enable signal is input, the latch circuit 84 latches the input data.
[0048]
The operation of the vertical 1/2 circuit 62 will be described with reference to FIG. The Y data of any pixel in the first field (black square pixel on the first field shown in FIG. 7) and the pixel in the next lower line at the same position in the horizontal direction of the screen as the pixel in the first field After the data of the pixels in the second field (black square pixels on the second field located at the lower right of the black square pixels on the first field shown in FIG. 7) are added by the adder circuit 83 Are output to the latch circuit 84. Therefore, the latch circuit 84 latches the average value of the pixels in the first field and the second field (corresponding to the pixels indicated by white squares in FIG. 7).
[0049]
In this way, the output of the latch circuit 84 is thinned out to 1/2 between the first field and the second field with respect to the pixel data output from the latch circuit 81, and then the third field and Progressive Y data having a frame frequency of 30 Hz is thinned out by a half between the fourth fields.
[0050]
Similarly, the U data and V data are also thinned in half in the horizontal direction and the vertical direction, and become progressive format data having a frame frequency of 30 Hz.
[0051]
As described above, the Y data, U data, and V data of the image output from the thinning filter 43 become progressive format data having a frame frequency of 30 Hz that has been thinned.
[0052]
Next, the UV vertical 1/2 circuit 45 will be described. FIG. 8 is a block diagram showing the configuration of the UV vertical ½ circuit 45. Progressive U data and V data having a thinned frame frequency of 30 Hz are input from the thinning filter 43 to the UV vertical 1/2 circuit 45. The input data is sequentially delayed by one line by the line FIFOs 91-1 to 91-6 and supplied to the subsequent stage. The multiplier circuit 92-1 multiplies the input data by a coefficient and outputs the result to the adder circuit 93. The multiplier circuits 92-2 to 92-7 multiply the data input from the line FIFOs 91-2 to 91-6, respectively, and output the result to the adder circuit 93.
[0053]
The adder circuit 93 adds the data supplied from the multiplier circuits 92-1 to 92-7 and supplies the sum to the latch circuit 94. When the clock signal is input, the latch circuit 94 latches the input data.
[0054]
The operation of the UV vertical 1/2 circuit 45 will be described with reference to FIG. Data of pixels (pixels indicated by white squares in FIG. 9) at the same position in the horizontal direction of the screen on seven consecutive lines are input to the multiplier circuits 92-1 to 92-7. The multiplier circuits 92-1 to 92-7 multiply the input data by a coefficient, respectively. The adder circuit 93 adds the outputs of the multiplier circuits 92-1 to 92-7 and outputs the result. A latch signal is input to the latch circuit 94 at a timing of once every two lines (a timing indicated by a black circle in FIG. 9). In this way, the UV vertical ½ circuit 45 thins out the number of lines of the input U data and V data to ½ and outputs it.
[0055]
Note that the UV vertical ½ circuit 45 passes Y data.
[0056]
Next, the encoding circuit 15 will be described. FIG. 10 is a block diagram showing a configuration of the encoding circuit 15. A control signal supplied from the control circuit 21 is input to the timing signal generation circuit 101. The timing signal generation circuit 101 generates a timing signal based on the input control signal and supplies the timing signal to the block shuffle circuit 102, the ADRC encoding circuit 103, and the inter-segment shuffle circuit 104.
[0057]
The Y data, U data, and V data converted by the format conversion circuit 14 are input to the block shuffle circuit 102. The block shuffle circuit 102 temporarily stores the supplied Y data, U data, and V data at a predetermined position in the external memory 16. The block shuffle circuit 102 rearranges the Y data, U data, and V data stored in the external memory 16 into the ADRC block format described later, shuffles within the segment range described later and in units of ADRC blocks, This is supplied to the encoding circuit 103.
[0058]
The ADRC encoding circuit 103 encodes the Y data, U data, and V data supplied from the block shuffle circuit 102 based on the ADRC method, and supplies the encoded data to the inter-segment shuffle circuit 104. The inter-segment shuffle circuit 104 temporarily stores the supplied ADRC encoded data at a predetermined position in the external memory 22.
[0059]
The inter-segment shuffle circuit 104 shuffles the ADRC data stored in the external memory 22 by changing the storage and reading addresses. The shuffle in the inter-segment shuffle circuit 104 increases the possibility that the receiving apparatus 2 can receive any one of the dynamic range DR, the minimum value MIN, and the motion flag MF even if data loss occurs in the transmission path. The purpose is to make it easier for the receiving device 2 to extract the codes Q separately even if data loss occurs in the transmission path.
[0060]
With reference to FIG. 11, a unit of processing of the encoding circuit 15 and the packetizing circuit 17 will be described. From a frame 0 and a next frame 1, pixels are selected by a method described later, and 1320 even segment ADRC blocks (consisting of 4 × 16 pixels) and 1320 odd segment ADRC blocks (4 × 16 pixels) is generated. 88 ADRC blocks are selected by the block shuffle circuit 102 from the generated ADRC blocks. The ADRC block is composed of 88 selected units by the ADRC encoding circuit 103, and is ADRC fixed-length data (FL), which is a dynamic range DR, a minimum value MIN, a motion flag MF, a table index TI, and It is encoded into code Q, which is variable length data (VL).
[0061]
Encoded data (hereinafter referred to as a buffer) corresponding to 88 ADRC blocks output from the ADRC encoding circuit 103 are grouped into 5 pieces (hereinafter referred to as segments) by the inter-segment shuffle circuit 104. ). Three segments (shown as segment 0, segment 2, and segment 4 in the figure) corresponding to the even segments are shuffled between the segments by the inter-segment shuffle circuit 104. Three segments corresponding to the odd segments (shown as segment 1, segment 3, and segment 5 in the figure) are shuffled between the segments by the inter-segment shuffle circuit 104.
[0062]
The segments shuffled between the segments are arranged in the order of segment 0, segment 1, segment 2, segment 3, segment 4, and segment 5.
[0063]
The packetizing circuit 17 corresponds to the eight ADRC blocks of the encoded data stored in the segments arranged in the order of segment 0, segment 1, segment 2, segment 3, segment 4, and segment 5. Each data (dynamic range DR, minimum value MIN, motion flag MF, table index TI, and code Q) is stored in a packet.
[0064]
The operation of the block shuffle circuit 102 will be described with reference to FIGS. FIG. 12 is a diagram illustrating the ADRC block. A block of 64 pixels adjacent to each other, horizontal 8 pixels × vertical 8 pixels, is selected from each frame, and the leftmost and uppermost pixel in the block (pixels indicated as 0-1 or 1-1 in the figure) ), And a pixel at a position horizontally moved by 2 pixels from the pixel (a pixel indicated as 0-3 or 1-3 in the drawing), and moved by 2 pixels vertically from the pixel. A pixel at a position (a pixel indicated as 0-17 or 1-17 in the figure), or a pixel at a position shifted from that pixel by one pixel horizontally and one pixel vertically (0-10 or 0 in the figure) Pixels labeled as 1-10) are referred to as even pixels. Further, the same selection is repeated using these pixels as a reference, and the selected pixel is set as an even pixel.
[0065]
The remaining pixels of each frame are odd pixels.
[0066]
As shown in FIG. 12, an even pixel (4 × 8 pixels) included in an 8 × 8 pixel block i2 of an even frame (frame 0) and an even number included in an 8 × 8 pixel block of an odd frame (frame 1). A collection of pixels (4 × 8 pixels) is an ADRC block of even segments. At the same time, odd pixels (4 × 8 pixels) included in the 8 × 8 pixel block of the even frame (frame 0) and odd pixels (4 × 8) included in the 8 × 8 pixel block of the odd frame (frame 1). A collection of (pixels) is an odd-segment ADRC block. The ADRC block is composed of every other pixel of the two frames on the original frame. Pixels located adjacent to the pixels of the even-numbered ADRC block on the original frame constitute the corresponding odd-numbered segment ADRC block.
[0067]
13 to 15 are diagrams illustrating the relationship between the pixels in the frame and the ADRC block. As shown in FIG. 13, the pixels of frame 0 are divided into blocks of 64 pixels of 8 horizontal pixels × 8 vertical pixels. Similarly, as shown in FIG. 14, the pixel of frame 1 (the frame next to frame 0) is divided into blocks of 64 pixels of 8 horizontal pixels × 8 vertical pixels. The leftmost and uppermost 64-pixel block in the frame 0 block (the 64-pixel block labeled A1 in FIG. 13) and the leftmost and uppermost block in the frame 1 The even-numbered segment ADRC block constituted by the even-numbered pixels of the 64-pixel block located (the 64-pixel block indicated as A1 in FIG. 14) is the leftmost segment 0, as shown in FIG. Placed on top.
[0068]
As shown in FIG. 15, an odd-segment ADRC block composed of a 64-pixel block labeled A1 in FIG. 13 and an odd-numbered pixel of the 64-pixel block labeled A1 in FIG. The ADRC block of the even segment of segment 0 is arranged on the right side in the figure.
[0069]
As shown in FIG. 15, an even-segment ADRC block composed of a 64-pixel block labeled A2 in FIG. 13 and an even-numbered pixel block labeled A2 in FIG. The ADRC block of segment 3 is composed of a block of 64 pixels labeled A1 in FIG. 13 and an odd number of pixels of the block of 64 pixels labeled A1 in FIG. As shown in FIG. 15, an odd-segment ADRC block composed of a 64-pixel block labeled A2 in FIG. 13 and odd-numbered pixels in the 64-pixel block labeled A2 in FIG. The ADRC block of segment 0, which is composed of a block of 64 pixels labeled A2 in FIG. 13 and an even number of pixels of a block of 64 pixels labeled A2 in FIG.
[0070]
The even-numbered ADRC block composed of the even-numbered pixels of the 64-pixel block indicated as B1 in FIG. 13 and the 64-pixel block indicated as B1 in FIG. The ADRC block of segment 0, which is composed of a block of 64 pixels indicated as A1 in FIG. 13 and an even number of pixels of a block of 64 pixels indicated as A1 in FIG. As shown in FIG. 15, an odd-segment ADRC block composed of a 64-pixel block denoted by B1 in FIG. 13 and an odd-numbered pixel of the 64-pixel block denoted by B1 in FIG. The lower part of the ADRC block of segment 3 (of the ADRC block of segment 4) composed of the 64-pixel block labeled A1 in FIG. 13 and the odd-numbered pixels of the 64-pixel block labeled A1 in FIG. (Next to the right).
[0071]
An even-numbered ADRC block composed of a 64-pixel block labeled B2 in FIG. 13 and an even-numbered pixel block labeled B2 in FIG. 14 is shown as segment 4 in FIG. The ADRC block of segment 1 is composed of a block of 64 pixels labeled 13 B1 and an odd number of pixels of the block of 64 pixels labeled B1 of FIG. An odd-segment ADRC block composed of a block of 64 pixels labeled B2 in FIG. 13 and an odd-numbered pixel of the block of 64 pixels labeled B2 in FIG. 14 is shown as segment 1 in FIG. An ADRC block of an even segment composed of a block of 64 pixels indicated by 13 B2 and an even number of pixels of the block of 64 pixels indicated by B2 in FIG. 14 is arranged on the right side in the figure.
[0072]
An even-segment ADRC block constituted by a 64-pixel block labeled C1 in FIG. 13 and an even-numbered pixel block labeled C1 in FIG. 14 is shown as segment 2 in FIG. The ADRC block of segment 4 is composed of a block of 64 pixels labeled B1 in FIG. 13 and an even number of pixels of a block of 64 pixels labeled B1 in FIG. An odd-segment ADRC block composed of a 64-pixel block labeled C1 in FIG. 13 and an odd-numbered pixel block labeled C1 in FIG. 14 is shown as segment 5 in FIG. The lower part of the ADRC block of segment 1 (the right side of the ADRC block of segment 2) composed of the block of 64 pixels labeled B1 of 13 and the odd number of pixels of the block of 64 pixels labeled B1 of FIG. Next to it).
[0073]
An even-segment ADRC block composed of a 64-pixel block labeled C2 in FIG. 13 and an even-numbered pixel block labeled C2 in FIG. 14 is shown as segment 2 in FIG. The ADRC block of segment 5 is composed of a block of 64 pixels labeled 13 C1 and an odd number of pixels of the block of 64 pixels labeled C1 in FIG. An odd-segment ADRC block composed of a 64-pixel block denoted by C2 in FIG. 13 and an odd-numbered pixel of the 64-pixel block denoted by C2 in FIG. 14 is shown as segment 5 in FIG. The ADRC block of the even segment composed of the block of 64 pixels labeled 13 C2 and the even pixels of the block of 64 pixels labeled C2 of FIG. 14 is arranged on the right side in the figure.
[0074]
By repeating the above procedure for 33 × 10 blocks (264 × 240 pixels) constituting each frame, the Y data of the two frames is used in the horizontal direction as shown in FIG. A set of 30 ADRC blocks arranged in the vertical direction is generated. As shown in FIG. 16B, the ADRC block generated from the Y data is shuffled to form a segment.
[0075]
The same processing is also performed for 88 × 120 pixel U data and V data, thereby generating a set of 22 ADRC blocks arranged in the horizontal direction and 15 in the vertical direction as shown in FIG. Is done. As in the case of Y data, as shown in FIG. 17B, ADRC blocks generated from U data and V data are block shuffled to form segments.
[0076]
Within each ADRC block, the pixel values of pixels located at the same position on the screen of the even frame and the odd frame (for example, the pixel indicated as 0-1 and the pixel indicated as 1-1 in FIG. 12). When the absolute value of the difference is taken and the maximum value of the difference is less than the threshold value Th1, the ADRC block is regarded as a still image.
[0077]
When the maximum value among the absolute values of the differences exceeds a threshold Th2 (Th1 <Th2), the ADRC block is regarded as a moving image.
[0078]
In the still image ADRC block, the average of the pixel values of the pixels at the same position on the screen of the even frame and the odd frame is averaged, and the average value of the 32 pixels is newly replaced as an ADRC block and attached to the ADRC block. The motion flag MF to be set is set to 0.
[0079]
In the moving ADRC block, 64 pixels collected from the even frame and the odd frame are directly used as the ADRC block, and the motion flag MF attached to the ADRC block is set to 1.
[0080]
Next, a description will be given of shuffle performed for each segment in units of ADRC blocks. When a serial number is assigned to an ADRC block in a segment composed of Y-data ADRC blocks numbered 0, the numbers y0 to y329 are assigned to the ADRC block as shown in FIG. The When the ADRC block is shuffled, the ADRC block of Y data is arranged as shown in FIG.
[0081]
Similarly, when a serial number is assigned to an ADRC block in a segment composed of U-data ADRC blocks numbered 0, the numbers u0 to u54 are changed to ADRC blocks as shown in FIG. It is attached to. As shown in FIG. 18D, the URC ADRC blocks are rearranged in the reverse order of the serial numbers.
[0082]
In a segment composed of ADRC blocks of V data numbered 0, if serial numbers are assigned to the ADRC blocks, numbers v0 to v54 are assigned to the ADRC blocks as shown in FIG. The As shown in FIG. 18F, the ADRC blocks of the V data are rearranged in the reverse order of the serial numbers.
[0083]
Next, as shown in FIG. 19, the AD data block of Y data, the ADRC block of U data, and the ADRC block of V data are shuffled. An ADRC block of one U data is arranged after the ADRC block of three Y data, and an ADRC block of three Y data is arranged behind it, followed by an ADRC block of one V data. Is placed. This arrangement is repeated. For example, in segment 0, an ADRC block with a serial number of y0, an ADRC block with a serial number of y221, an ADRC block with a serial number of y112, an ADRC block with a serial number of u54, a serial number of y3 An ADRC block with a serial number of y224, an ADRC block with a serial number of y115, an ADRC block with a serial number of v54, an ADRC block with a serial number of y6, etc. An ADRC block is arranged.
[0084]
Similarly, in the segments 1 to 5, the ADRC block is shuffled.
[0085]
As described above, the pixel data is shuffled for each ADRC block by the block shuffle circuit 102 and encoded by the ADRC encoding circuit 103 based on the shuffled ADRC block. By appropriately shuffling the ADRC block, pixels that are missing due to communication errors are dispersed, and it is possible to make it difficult to recognize the missing pixels in the reproduced image.
[0086]
Next, the configuration of the ADRC encoding circuit 103 will be described with reference to FIG. The image shuffled by the block shuffle circuit 102 is supplied as an ADRC block to the delay circuit 161, the motion feature amount calculation circuit 162, the DR calculation circuit 163, and the delay circuit 166.
[0087]
The delay circuit 161 delays the 2 frame block (ADRC block composed of 4 × 16 pixels) by a time corresponding to the processing time of the motion feature amount calculation circuit 162 and outputs the delayed result to the information amount control circuit 164. The motion feature amount calculation circuit 162 calculates a motion feature amount that is the maximum absolute value of the interframe difference for each ADRC block, and supplies the motion feature amount to the information amount control circuit 164. The DR calculation circuit 163 calculates, for each ADRC block, a dynamic range DR when it is regarded as a still image and a dynamic range DR when it is regarded as a moving image, and supplies the dynamic range DR to the information amount control circuit 164.
[0088]
The information amount control circuit 164 has a dynamic range when it is regarded as a two-frame block supplied from the delay circuit 161, a motion feature amount supplied from the motion feature amount calculation circuit 162, and a still image supplied from the DR calculation circuit 163. Based on the set of thresholds Th1 and Th2 and the thresholds T1 and T2 supplied from the threshold range 165 and the dynamic range DR when considered as DR and a moving image, the MF selection threshold consisting of the thresholds Th1 and Th2 is selected, It outputs to the static motion determination circuit 169.
[0089]
The information amount control circuit 164 includes two frame blocks, a motion feature amount, a dynamic range DR when regarded as a still image, a dynamic range DR when regarded as a moving image, and threshold values Th1 and Th2 supplied from the threshold value table 165. Based on the set of threshold values T1 and T2, a Q-bit selection threshold value is selected and output to the ADRC encoder 171.
[0090]
For example, when the motion feature amount is 4, the dynamic range DR when it is regarded as a still image is 14, and the dynamic range DR when it is regarded as a moving image is 15, the threshold T1 with a table index of 0 is 6. When the threshold T2 is 12, the threshold Th1 is 3, the threshold Th2 is 3, the threshold T1 of the table index 1 is 13, the threshold T2 is 40, the threshold Th1 is 5, and the threshold Th2 is 5. explain.
[0091]
The information amount control circuit 164 determines that the moving feature amount is larger than the threshold value Th2 with respect to the set of threshold values with the table index being 0, so that it is a moving image. Since the dynamic range DR when regarded as a moving image is larger than the threshold value T2, q of this ADRC block is 4, and the information amount control circuit 164 indicates that the code Q is 256 bits from (16 * 4) * 4. Ask.
[0092]
Similarly, the information amount control circuit 164 determines a still image because the motion feature amount is smaller than the threshold value Th1 for the threshold value set having the table index of 1. Since the dynamic range DR when regarded as a still image is larger than the threshold value T1 and smaller than the threshold value T2, the q of this ADRC block is 3, and the information amount control circuit 164 has the code Q from (8 * 4) * 3. Require 96 bits.
[0093]
Thus, each threshold value is set in the threshold value set so that the information amount decreases as the table index increases. The information amount control circuit 164 selects a table index having the largest information amount with a code Q generated for one buffer of 16,104 bits or less as the table index TI, and outputs the table index TI to the delay circuit 173.
[0094]
The information amount control circuit 164 may calculate the number of bits of the code Q corresponding to the table index in parallel or sequentially from the larger (or smaller) side of the table index.
[0095]
An example of the configuration of the information amount control circuit 164 will be described with reference to FIGS. 21 and 22. FIG. 21 is a diagram illustrating a configuration of a preceding stage of the information amount control circuit 164, and FIG. 22 is a diagram illustrating a configuration of a subsequent stage of the information amount control circuit 164.
[0096]
As shown in FIG. 21, the information amount control circuit 164 includes, in the preceding stage, a number of comparison circuits 181-1 and 181-2 corresponding to the number of threshold values of the dynamic range DR, and adders 182-1 and 182- 2 and registers 183-1 and 183-2. A dynamic range DR that can take a numerical value from (0 to 255) is supplied to one input terminal of each of the comparison circuits 181-1 and 181-2.
[0097]
In FIG. 21, for example, the threshold value T1 and the threshold value T2 are read from the threshold value table 165 and supplied to the other input terminals of the comparison circuits 181-1 and 181-2. Specifically, the threshold value T1 is supplied to the other input terminal of the comparison circuit 181-1, and the threshold value T2 is supplied to the other input terminal of the comparison circuit 181-2.
[0098]
Each of the comparison circuits 181-1 and 181-2 generates an output of “1” when the dynamic range DR from the input terminal 5 is equal to or greater than each threshold value. The comparison outputs of the comparison circuits 181-1 and 181-2 are supplied to one input terminal of either the adder 182-1 or 182-2 connected thereto. The other input terminal of the adder 182-1 or 182-2 is configured to be supplied with its own addition output via the registers 183-1 and 183-2.
[0099]
Therefore, the adder 182-1 calculates the integrated value of the frequency (T1 ≦ DR ≦ 255), and the obtained data is held in the register 183-1. Further, the adder 182-2 calculates the integrated value of the frequency (T2 ≦ DR ≦ 255), and the obtained data is held in the register 183-2.
[0100]
Each of the occurrence frequency data held in each of the registers 183-1 and 183-2 is taken out and supplied to the adder 191 as shown in FIG. Further, the adder 191 is supplied with frequency data stored in the information amount control circuit 164 in advance. In the adder 191, the occurrence frequency data of (T1 ≦ DR ≦ 255), the occurrence frequency data of (T2 ≦ DR ≦ 255), and the frequency data stored in advance are added, and the total amount is calculated.
[0101]
The occurrence value of (T1 ≦ DR ≦ T2-1) is (S3), and the occurrence value of (T2 ≦ DR ≦ 255) is (S4). The occurrence value of (T2 ≦ DR ≦ 255) is (S4 + S3 + S2) (corresponding to the code Q where q is 4 bits), and the occurrence value of (T1 ≦ DR ≦ 255) is (S3 + S2) (q is (Corresponding to a code Q of 3 bits). (S2) (corresponding to the code Q in which q is 2 bits) is generated corresponding to all ADRC blocks, and can be stored in the information amount control circuit 164 in advance. When these are added by the adder 191, an output of (S3 + 2 * S3 + 3 * S2) is obtained. This output is the total amount that is equal to the case where the assigned bits are multiplied and fully added.
[0102]
The total amount calculated in the adder 191 is supplied to one input terminal of the comparison circuit 193 via the register 192. The buffer maximum value is supplied to the other input terminal 12 of the comparison circuit 193, and the total amount is compared with the buffer maximum value. Based on the comparison output of the comparison circuit 193, a threshold value is determined such that the total amount is less than or equal to the buffer maximum value. That is, a plurality of sets of threshold values are stored in advance in the threshold value table 165 so that the total amount of generated data simply increases or decreases, and the threshold values are selected so that the total amount falls within the target.
[0103]
The information amount control circuit 164 selects and outputs the MF selection threshold, the Q bit selection threshold, and the table index TI based on the comparison output of the comparison circuit 193.
[0104]
Returning to FIG. 20, the delay circuit 166 delays the 2 frame block (ADRC block composed of 4 × 16 pixels) by a time corresponding to the processing time of the delay circuit 161 and the information amount control circuit 164, and generates a still block generation circuit 167. To the delay circuit 168 and the static motion determination circuit 169. The static block generation circuit 167 takes the average of the pixel values of the pixels located at the same position on the screen of the even frame and the odd frame from the two frame blocks (ADRC block composed of 4 × 16 pixels), and calculates the average of the 32 pixels. The value is newly replaced as an ADRC block and supplied to the selection circuit 170 as one frame block (ADRC block composed of 4 × 8 pixels).
[0105]
The delay circuit 168 delays the two frame blocks (ADRC block composed of 4 × 16 pixels) by a time corresponding to the processing time of the still block generation circuit 167 and supplies the delay to the selection circuit 170.
[0106]
The static motion determination circuit 169 generates and selects a motion flag MF attached to the ADRC block on the basis of a 2-frame block (ADRC block composed of 4 × 16 pixels) and an MF selection threshold composed of thresholds Th1 and Th2. The circuit 170 and the delay circuit 172 are supplied.
[0107]
When the motion flag MF is 1, that is, when the ADRC block corresponds to a moving image, the selection circuit 170 supplies the 2-frame block supplied from the delay circuit 168 to the ADRC encoder 171 and when the motion flag MF is 0, That is, when the ADRC block corresponds to a still image, the one frame block supplied from the still block generation circuit 167 is supplied to the ADRC encoder 171.
[0108]
The ADRC encoder 171 encodes the two-frame block or the one-frame block supplied from the selection circuit 170 based on the Q-bit selection threshold supplied from the information amount control circuit 164, so that the dynamic range DR, the minimum value MIN, q , And code Q are output.
[0109]
The delay circuit 172 delays and outputs the motion flag MF supplied from the static motion determination circuit 169 in accordance with the processing time of the selection circuit 170 and the processing time of the ADRC encoder 171.
[0110]
The delay circuit 173 delays and outputs the table index TI supplied from the information amount control circuit 164 in correspondence with the delay time of the delay circuit 168, the processing time of the selection circuit 170, and the processing time of the ADRC encoder 171. .
[0111]
As described above, the ADRC encoding circuit 103 generates the dynamic range DR, the minimum pixel value MIN, the motion flag MF, and the code Q for each ADRC block, and the inter-segment shuffle circuit together with the index TI of the quantization table. 104 is supplied.
[0112]
FIG. 23 is a diagram for explaining how the inter-segment shuffle circuit 104 is shuffled. As shown in FIG. 23, there are three shuffle methods, shuffle 0 to 2, and shuffle 0 is a shuffle that does not perform shuffle. In shuffle 1, data of segment 0 (segment 1) is segment 4 (segment 5), data of segment 2 (segment 3) is segment 0 (segment 1), and data of segment 4 (segment 5) is segment 2 ( Each segment 3) is shuffled (replaced).
[0113]
Similarly, for Shuffle 2, the data of segment 0 (segment 1) is sent to segment 2 (segment 3), the data of segment 2 (segment 3) is sent to segment 4 (segment 5), and the data of segment 4 (segment 5) is sent. Shuffle each segment 0 (segment 1).
[0114]
FIG. 24 is a diagram for explaining the shuffle of the dynamic range DR performed in the segment 0, the segment 3, and the segment 5. One segment includes five buffers, and one buffer includes 88 ADRC blocks. Therefore, one segment includes 440 DRs. Numbers 1 to 440 are sequentially assigned to DRs present in each segment, and in FIG. 24, for example, the DR of number 1 in segment 1 is described as DR1-1.
[0115]
Shuffle 0 is used for the DR of number 1 in each segment (that is, not replaced), shuffle 1 is used for the DR of number 2, and shuffle is used for the DR of number 3 2 is used, and for the DR of number 4, shuffle is performed in the order of shuffle 0, shuffle 1, and shuffle 2, so that shuffle 0 is used.
[0116]
Similarly, the code Q is shuffled. Further, as shown in FIG. 25, the minimum pixel value MIN is performed in the order of shuffle 1, shuffle 2, and shuffle 0 to perform shuffle. Further, the MF that is a motion flag is performed in the order of shuffle 2, shuffle 0, and shuffle 1, as shown in FIG.
[0117]
Next, the shuffling of the index TI of the quantization table will be described. The TI stored in segment 0 is also stored in segment 3, and the TI stored in segment 3 is also stored in segment 0. That is, segment 0 and segment 3 store both their own TI and the counterpart TI. Similarly, segment 1 and segment 4 and segment 3 and segment 5 store both their own TI and the counterpart TI.
[0118]
In this way, even if a communication error occurs and a packet including any data is lost due to the dynamic range DR, the minimum pixel value MIN, and the motion flag MF being shuffled, the receiving device 2 When decoding a predetermined ADRC block, it is possible to recover missing data based on other data that could be received. Further, even if a communication error occurs and a packet including the code Q is lost due to the shuffling of the code Q, the pixels corresponding to the lost code Q are dispersed, and the receiving device 2 It becomes easy to cut out, and when a predetermined ADRC block is decoded, it is easy to create a missing pixel value based on the decoded pixel value.
[0119]
In this way, by shuffling data between segments, the communication device 2 can easily recover data and create missing pixels, and can improve the quality of an image to be reproduced even if a packet is lost on the transmission path. Can be maintained.
[0120]
The data output from the inter-segment shuffle circuit 104 of the encoding circuit 15 is input to the packetizing circuit 17 and made into a packet. FIG. 27 shows a packet configuration. The packet configuration shown in FIG. 27 is composed of 201 bytes, of which DR is A × 8M bits, MIN is B × 8M bits, MF is C × 8M bits, TI is D × (8M / D) bits, The code Q (VL-Data) is composed of 8 × N bits. Data is stored using a plurality of packets of this size unit.
[0121]
In this way, the dynamic range DR, the minimum pixel value MIN, the motion flag MF, the table index TI, and the code Q are configured to be a multiple of 8, so that when performing the above-described shuffling, Regardless of the bit size of the data, it is suitable for handling in a unified architecture and in byte units.
In the above description and the description to be described later, the packet configuration shown in FIG. 27 is used.
[0122]
FIG. 28 shows the configuration of the inter-segment shuffle circuit 104 that performs shuffling as described above. The data output from the ADRC encoding circuit 103 is input to the data selection unit 211 of the inter-segment shuffle circuit 104. The data selection unit 211 receives three pieces (blocks) in parallel for each element of the FL data, and inputs VL data at a timing alternating with the FL data. The data selection unit 211 converts the input data into serial data and outputs predetermined data to the input / output controller 213. The input / output controller 213 stores the input data in the storage unit 215 based on the address generated by the address generation unit 214. The data stored in the storage unit 215 is read out based on the address generated by the address generation unit 214 according to the data supplied from the output data counter unit 216, and is output to the packetizing circuit 17 (FIG. 3). .
[0123]
The input data counter unit 212 includes a group of counters that indicate information for identifying the type of data input to the data selection unit 211, addressing and shuffling directions when stored in the storage unit 215, and the like. An example of the counter group is shown in FIG. There are counters for addressing and shuffling. As a counter for FL data for addressing, a counter (flblk) that counts the number of FL data blocks in one buffer (determines the address of FL data in a packet), and the number of fl data packets in one segment Counter (flpkt) that counts packets (determines the packet number in the segment) and a counter (flttlpkt) that counts the total number of packets of fl data in one bank (determines the address of the packet in the bank) Has been.
[0124]
The counter group for VL data for addressing includes a counter (vlpkt) that counts the number of VL data packets in one buffer (determines the packet number in the buffer), and the total number of packets of VL data in one bank Counter (vlttlpkt) that counts the address of the packet in the bank (1) counter (vlbuf) that counts the number of VL data buffers in one segment (which determines the buffer number in the segment), and any order It is composed of a counter (vlmodN) that counts the remainder when dividing VL data by the number N of bytes of VL data per packet (determines the storage location of the VL data storage unit 215 (FIG. 28) in the packet) ing.
[0125]
The counter for shuffling counts the number of FL data segments in one bank (determines the segment number of FL data) for FL data, and determines the shuffling direction of the FL data in one segment. Counter to determine (flshf), for VL data, count the number of segments of VL data in one bank (determine the segment number in a bank) counter (vlseg), the shuffling direction of VL data in one segment And a counter (vlshf) to be determined.
[0126]
Similarly, the output data counter unit 216 includes a group of addressing counters as shown in FIG. Counter for FL data addressing, counting TI data in 1 packet size data cell (for collecting TI data into data in units of bytes) (ticnt), number of MF data in 1 packet size data cell Counter (mfcnt for collecting MF data into byte data), counts the number of DR data and MIN data in one packet size data cell (number of DR data and MIN data in packet) Counter (dmcnt), a counter that counts the number of blanks inserted to handle bit-size TI data or MF data as byte-size data (tmcnt), and the VL data in the packet within one packet A counter that determines the number (vlcntN) and the number of packets in one buffer To and a (determining the packet number in the buffer) counter (Pktcnt).
[0127]
The data input to the inter-segment shuffling circuit 104 is stored in the storage unit 215 based on the address generated by the address generation unit 214 according to the counter group of the input data counter unit 212 described above. For example, when the first DR data of image data in units of two frames is input, the block number is 0 by flblk and the packet number is 0 by flttlpkt and flpkt as the counter number for the address of the DR data. It is determined and output to the address generator 214.
Further, as the counter number for shuffle, the segment number is determined to be 0 by flseg and 0 by flmod, and is output to the address generator 214.
[0128]
The address generation unit 214 determines the shuffle 0 (FIG. 23) based on the flshf counter number among the input counter numbers, and determines the segment number (flseg counter number) to be 0. Is shuffled (in this case, since it is shuffle 0, it is not shuffled and segment 0 is selected as the shuffling destination).
[0129]
Next, when the I-th VL data is input among the image data in units of two frames, the VL data number in the packet is the remainder of I / N by vlmodN as the counter number for the address of the VL data. 1 (since the number is assigned from 0), the value indicating the total number of packets by vlttlpkt is the quotient of I / N-1, and the packet number is (I / N) / (number of packets in the buffer) by vlpkt-1 , And vlbuf respectively determine the buffer number and output it to the address generator 214. In addition, as a counter number for shuffle, a segment number is determined by vlseg and a shuffling direction is determined by vlshf, and is output to the address generator 214.
[0130]
The address generation unit 214 determines the shuffling segment number by determining the shuffle direction and the segment number from the counter number of vlshf among the input counter numbers. The address is determined based on the shuffling destination data and the address counter value determined in this way.
[0131]
As described above, the data stored in the storage unit 215 according to the address generated by the address generation unit 214 is read based on the address generated by the address generation unit 214 according to the counter group of the output data counter unit 216. The reading will be described. When a pulse for reading is input to the inter-segment shuffle circuit 104 at a predetermined timing, processing for collecting TI data and MF data in units of bytes is performed. When the gathering process is completed, the data is read from the storage unit 215 according to the data output order in the packet, and in this case, according to the data arrangement in the packet shown in FIG. .
[0132]
The processing for each packet output is repeated for one buffer (11 packets) and sequentially output to the packetizing circuit 17. The packetizing circuit 17 forms a packet by temporarily storing the data input from the encoding circuit 15 in the external memory 18 as necessary and reading it out as necessary.
[0133]
Next, image transmission processing of the transmission apparatus 1 will be described with reference to the flowchart of FIG. In step S11, the A / D conversion circuit 42 of the thinning unit 13 samples the input interlaced image signal having a field frequency of 60 Hz at a predetermined sampling rate, and outputs Y data U and U at a predetermined data rate. Data and V data are output. In step S12, the thinning filter 43 thins the Y data, U data, and V data by half in the horizontal direction and the vertical direction, respectively, and converts them into progressive data having a frame frequency of 30 Hz.
[0134]
In step S13, the UV vertical ½ circuit 45 thins out U data and V data by ½ in the vertical direction. In step S <b> 14, the format conversion circuit 14 converts Y data, U data, and V data into a format that can be supplied to the encoding circuit 15.
[0135]
In step S <b> 15, the encoding circuit 15 shuffles the Y data, U data, and V data supplied from the format conversion circuit 14, encodes them, further shuffles them, and outputs them to the packetizing circuit 17. In step S <b> 16, the packetizing circuit 17 packetizes the data supplied from the encoding circuit 15 and outputs the packetized data to the transmission circuit 19. In step S19, the transmission circuit 19 transmits the packet supplied from the packetizing circuit 17 by a predetermined method, and the process ends.
[0136]
As described above, the transmission apparatus 1 compresses an input image signal and outputs the compressed signal.
[0137]
Next, the receiving device 2 will be described. FIG. 32 is a block diagram illustrating a configuration of the receiving device 2. The packet received by the receiving circuit 301 is supplied to the packet disassembling circuit 302. The reception circuit 301 sets a missing flag for each bit of data of the lost packet when the received packet is lost. The packet disassembly circuit 302 temporarily stores the supplied packet in the external memory 303, disassembles the packet temporarily stored in the external memory 303, takes out data included in the packet, and decodes the packet together with the missing flag 304 is supplied. The missing flag may be attached to the code Q in units of 8 bits, and the missing flag may be attached to each of the dynamic range DR, the minimum pixel value MIN, and the motion flag MF.
[0138]
The decoding circuit 304 temporarily stores the data supplied from the packet disassembly circuit 302 in the external memory 305, decodes the data encoded by the ADRC method temporarily stored in the external memory 305, and converts the format. Supply to circuit 306. When data such as the dynamic range DR or the minimum value MIN necessary for decoding is missing, the decoding circuit 304 recovers and decodes the missing data, and when the decoding is successful, sets the corresponding missing flag. Reset.
[0139]
The format conversion circuit 306 temporarily stores the data supplied from the decoding circuit 304 in the external memory 307, and performs processing opposite to the processing of the format conversion circuit 14 for the data temporarily stored in the external memory 307. Is supplied to the missing pixel creation circuit 308.
[0140]
Based on the data supplied from the format conversion circuit 306, the missing pixel creation circuit 308 generates pixel data missing during transmission from the pixel values of the pixels around the missing pixel by class classification adaptive processing. Data is supplied to the format conversion circuit 310. The initialization circuit 309 supplies a coefficient set or the like to the missing pixel creation circuit 308 when the receiving device 2 is activated.
[0141]
The format conversion circuit 310 temporarily stores the data supplied from the missing pixel creation circuit 308 in the external memory 311, and the quadruple resolution creation circuit 312 can process the data temporarily stored in the external memory 311. The image data is converted into progressive 3: 0.5: 0.5 image data having a frame frequency of 30 Hz, and the converted data is supplied to the quadruple-density resolution creation circuit 312.
[0142]
The quadruple-resolution creation circuit 312 creates image data having a density four times that of the supplied image based on the image data supplied from the format conversion circuit 310, and supplies the image data to the interpolation unit 314. The initialization circuit 313 supplies a coefficient set and the like to the quadruple-resolution creation circuit 312.
[0143]
The interpolation unit 314 includes a format conversion circuit 331, an external memory 332, a UV vertical double circuit 333, an external memory 334, a D / A conversion circuit 335, and a post filter 336.
[0144]
The format conversion circuit 331 receives the data supplied from the quadruple-density resolution creation circuit 312, temporarily stores it in the external memory 332, and stores the data temporarily stored in the external memory 332 as a UV vertical double circuit 333. Can be processed into progressive 3: 5: 0.5 image data having a frame frequency of 30 Hz, and the converted data is output to the UV vertical doubling circuit 333.
[0145]
The UV vertical doubling circuit 333 temporarily stores the data supplied from the format conversion circuit 331 in the external memory 334, and the U data and the V data of the data temporarily stored in the external memory 334 are each in the vertical direction. The Y data, the U data, and the V data are supplied to the D / A conversion circuit 335.
[0146]
The D / A conversion circuit 335 converts the Y data, U data, and V data supplied from the UV vertical doubling circuit 333 into analog Y signals, U signals, and V signals, and outputs them to the post filter 336. To do. The post filter 336 is a filter (so-called sin X / X filter) that allows only effective components of the analog Y signal, U signal, and V signal output from the D / A conversion circuit 335 to pass. The Y signal, U signal, and V signal that have passed through the post filter 336 are supplied to the NTSC encoder 315 or output as they are as the output of the receiving device 2.
[0147]
The NTSC encoder 315 generates and outputs a component signal (Y / C signal) and a composite video signal based on the input Y signal, U signal, and V signal.
[0148]
The PLL circuit 316 generates a reference signal serving as a reference for processing of each circuit and supplies the reference signal to the control circuit 317. The control circuit 317 generates the packet disassembly circuit 302, the decoding circuit 304, the format conversion circuit 306, the missing pixel creation circuit 308, the format conversion circuit 310, the quadruple resolution creation circuit 312, the interpolation unit 314, and the NTSC encoder 315. A control signal is supplied to control the operation of the entire receiving device 2.
[0149]
With reference to FIG. 33, a process of decompressing image data received by the receiving device 2 when a signal is output in an interlaced manner will be described. The image data decoded by the decoding circuit 304 is converted into Y data having 264 pixels in the horizontal direction and 240 pixels in the vertical direction by the format conversion circuit 306 and having a frame frequency of 30 Hz and 240 pixels in the vertical direction, and 88 pixels. It is converted into U data and V data of × 120 pixels × 2.
[0150]
The Y data, U data, and V data constituting one frame output from the format conversion circuit 306 are generated in the 60 Hz field by the quadruple resolution creation circuit 312 after the missing pixels are created by the missing pixel creation circuit 308. It is converted into an interlaced first field and a second field having a frequency. At this time, the Y data is data including 528 pixels in the horizontal direction and 240 pixels in the vertical direction per field. The U data and V data are data including 176 pixels in the horizontal direction and 120 pixels in the vertical direction for each field.
[0151]
The Y data, U data, and V data output from the quadruple-resolution creation circuit 312 include 528 pixels in the horizontal direction and 480 pixels in the vertical direction per frame by the format conversion circuit 331. The U data and the V data are converted into data including 176 pixels in the horizontal direction and 240 pixels in the vertical direction, respectively, for each frame.
[0152]
Further, the U data and V data are converted into data including 176 pixels in the horizontal direction and 480 pixels in the vertical direction by one UV vertical doubling circuit 333. The D / A conversion circuit 335 performs digital-analog conversion, converts Y data into an analog Y signal, and converts U data and V data into an analog U signal and an analog V signal, respectively.
[0153]
As described above, the interpolation unit 314 performs processing corresponding to the thinning unit 13 to reproduce the original image having sufficient image quality from the data transmitted at the bit rate reduced to a practical value. can do.
[0154]
As described above, the receiving device 2 expands the received image data and outputs the data as an interlaced Y signal, U signal, and V signal. The information of the image signal output from the receiving device 2 corresponds to 163 Mbps, and corresponds to the information amount of 98% when compared with the information of the image signal input to the transmitting device 1.
[0155]
Next, the decoding circuit 304 will be described. FIG. 34 is a block diagram showing a configuration of the decoding circuit 304. The control signal supplied from the control circuit 317 is input to the timing signal generation circuit 351. The timing signal generation circuit 351 generates a timing signal based on the control signal, and generates an inter-segment deshuffle circuit 352, a q-bit MF recovery circuit 353, a DRMIN recovery circuit 354, an ADRC decoding circuit 355, a block deshuffle circuit 356, and a recovery The error determination circuit 357 is supplied.
[0156]
Data supplied from the packet disassembly circuit 302 is input to the inter-segment deshuffle circuit 352. The inter-segment deshuffle circuit 352 performs the reverse operation of the inter-segment shuffle circuit 104 of the transmission apparatus 1 shown in FIG. 10, and returns the shuffled data to the original order. When there is no omission in the deshuffled data, the inter-segment deshuffle circuit 352 supplies the deshuffled data to the ADRC decoding circuit 355. When the deshuffled data is missing, the inter-segment deshuffle circuit 352 supplies the deshuffled data to the q-bit MF recovery circuit 353 and the DRMIN recovery circuit 354. Further, when there is an error in the data of a plurality of blocks straddling between segments, the inter-segment deshuffle circuit 352 outputs a continuous error signal to the block deshuffle circuit 356.
[0157]
The q-bit MF recovery circuit 353 uses the fact that the correlation between adjacent pixels remains in the compressed code Q, and based on the correlation between the pixel values of the pixels of three consecutive ADRC blocks, the correlation is maximized. The combination of q and the value of the motion flag MF is obtained and output to the DRMIN recovery circuit 354 as the correct value of q and the motion flag MF. The q-bit MF recovery circuit 353 transmits emin1 and emin2 (the minimum value of the adjacent pixel difference and the second smallest value) for determining whether or not the motion flag MF is correct to the recovery error determination circuit 357.
[0158]
FIG. 35 is a block diagram showing a configuration of the q-bit MF recovery circuit 353.
The deshuffled data supplied from the segment deshuffle circuit 352 is supplied to the q-bit MF extraction circuit 371.
[0159]
The q-bit MF extraction circuit 371 extracts q and the motion flag MF from the supplied data and supplies them to each of the 3-block evaluation value calculation circuits 372-1 to 372-6. When q and the motion flag MF are lost due to an error, the q-bit MF extraction circuit 371 assigns possible combinations of q and the motion flag MF to the 3-block evaluation value calculation circuits 372-1 to 372-6, respectively. Supply.
[0160]
For example, q set to “2” and motion flag MF set to “0” are supplied to the 3-block evaluation value calculation circuit 372-1, and q set to “3” and “0” are set. The motion flag MF thus set is supplied to the 3-block evaluation value calculation circuit 372-2, and the motion flag MF set to “4” and “0” is set to the 3-block evaluation value calculation circuit 372-3. To be supplied. Further, for example, q set to “2” and motion flag MF set to “1” are supplied to the 3-block evaluation value calculation circuit 372-4, and q and “1” set to “3”. Is set to the 3-block evaluation value calculation circuit 372-5, and q is set to “4” and the motion flag MF is set to “1” is the 3-block evaluation value calculation circuit 372. -6.
[0161]
Based on q and the motion flag MF, the three-block evaluation value calculation circuits 372-1 to 372-6 are based on the difference between the pixel values of pixels adjacent to each other from the pixel value decoded from the extracted code Q. An evaluation value (LE: Linear Error) is calculated, and the evaluation value is supplied to the selector 373 together with q and the motion flag MF. Hereinafter, the three-block evaluation value calculation circuits 372-1 to 372-6 are simply referred to as a three-block evaluation value calculation circuit 372 when there is no need to distinguish them individually.
[0162]
The evaluation value is calculated by Expression (1). The smaller the evaluation value, the higher the correlation between pixels.
[Expression 1]

Q_{i, j}Indicates a non-missing code Q included in the ADRC block to be recovered. Q ’_{i, j}Q_{i, j}The code | cord Q contained in the other ADRC block comprised by the pixel of the block of 64 pixels shown in FIG. 13 and FIG. 14 containing the pixel which comprises the ADRC block in which is contained. For example, Q_{i, j}Is included in the ADRC block of the even segment numbered “0” in FIG._{i, j}Are included in the ADRC block of the odd-numbered segment with the number “3”. Q_{i, j}Is included in the ADRC block of the even segment of number “4” in FIG._{i, j}Are included in the ADRC block of the odd-numbered segment with the number “1”. q_{i, j}Q_{i, j}Q to cut out. q ’_{i, j}Q ’_{i, j}Q to cut out. Nj indicates the number of codes Q that are not missing in the ADRC block. j is a number for identifying three ADRC blocks whose evaluation values are calculated.
[0163]
The offset j is used to absorb the difference in the dynamic range and q between the three ADRC blocks and calculate a more accurate evaluation value. In the equations (2), (3), and (4), Calculated.
Offset j = (Xj × 2^4-qj-Yj × 2^4-q'j) / 128 (2)
[Expression 2]

[Equation 3]

[0164]
The selector 373 selects an evaluation value having the minimum value and an evaluation value having the next largest value after the minimum value from the evaluation values supplied from the three-block evaluation value calculation circuits 372-1 to 372-6, These are supplied to the recovery error determination circuit 357 as emin1 and emin2, respectively.
[0165]
The selector 373 selects q and the motion flag MF output from the three-block evaluation value calculation circuits 372-1 to 372-6 that output the evaluation value having the minimum value, and the q-bit MF recovery circuit 353 The output is supplied to the DRMIN recovery circuit 354.
[0166]
The DRMIN recovery circuit 354 performs ADRC based on the correct value of q and the correct value of the motion flag MF supplied from the q-bit MF recovery circuit 353 and the corresponding ADRC block shown in FIG. The correct value of the dynamic range DR for each block and the minimum value MIN of the pixel value for each ADRC block are obtained and output to the ADRC decoding circuit 355.
[0167]
The recovery process of the dynamic range DR or the minimum pixel value MIN of the DRMIN recovery circuit 354 will be described with reference to FIGS. The pixel value L is decoded from the code Q by the process of Expression (5) by the ADRC method.
L = DR / (2^Q) * q + MIN (5)
[0168]
As shown in FIG. 36, the relationship between q and the pixel value L is such that the slope is DR / (2^Q), And the contact piece is represented by a straight line having the minimum pixel value MIN.
[0169]
From the set of (q, L) in the ADRC block, DR / (2^Q) And the minimum value MIN of the pixel value is obtained statistically. It is assumed that the regression line (q, L) shown in FIG. 37 is equal to the straight line of Expression (5).
[0170]
At this time, if b is known, a is calculated by equation (6).
a = (B-nb) / A (6)
[0171]
If a is known, b is calculated by equation (7).
b = (B-Aa) / n (7)
[0172]
Here, n is the number of points (q, L), A is Σx, and B is Σy.
[0173]
The dynamic range DR is calculated by Expression (8).
DR = (ΣL'-N * MIN) / (Σq) * 2^Q
= (((DR '/ 2^{Q '}) * Y) + N * MIN'-N * MIN) / X * 2^Q
= (N * (MIN'-MIN) * 2Q '+ DR' * Y) * 2^Q/ (X * 2^{Q '}(8)
As shown in FIG. 38 (A), N is Σni, where ni is the number of non-missing blocks adjacent to the block to be recovered from the dynamic range DR or the minimum pixel value MIN. In FIG. 38, the hatched squares indicate blocks that attempt to recover the dynamic range DR or the minimum pixel value MIN. In FIG. 38, squares without diagonal lines indicate blocks without omissions. When the q of the block to be recovered from the dynamic range DR or the minimum pixel value MIN is missing, ni is 0 as shown in FIG.
[0174]
DR 'indicates the dynamic range of a block without a missing block adjacent to the block to be recovered. MIN ′ indicates the minimum value of the pixel values of the non-missing block adjacent to the block to be recovered. Q 'indicates the code Q of a block without a block adjacent to the block to be recovered.
[0175]
X is represented by Formula (9).
[Expression 4]

[0176]
Y is shown by Formula (10).
[Equation 5]

[0177]
The minimum value MIN of the pixel value is calculated by Expression (11).
MIN = (ΣL'-Σq * (DR / 2^Q))
= ((DR '/ 2^{Q '}) * Y + N * MIN '-(DR / 2^Q) * X) / N
= (N * MIN '* 2^{Q '+ Q}+ DR '* Y * 2^Q-DR * X * 2^{Q '}) / (N * 2^{Q '+ Q}(11)
[0178]
The ADRC decoding circuit 355 decodes the code Q based on the data supplied from the inter-segment deshuffle circuit 352 when there is no loss in the deshuffled data, and the Y data, U data, and V obtained as a result of the decoding. Data is supplied to the block deshuffle circuit 356.
[0179]
The ADRC decoding circuit 355 passes through the DRMIN recovery circuit 354 when the q or the motion flag MF is missing and the q-bit MF recovery circuit 353 can obtain the correct value of q and the correct value of the motion flag MF. Based on the supplied correct value of q, the correct value of the motion flag MF, the dynamic range DR, and the minimum value MIN of the pixel value for each block, the code Q supplied from the DRMIN recovery circuit 354 is decoded, and the decoding result The obtained Y data, U data, and V data are supplied to the block deshuffle circuit 356 together with an error flag set to “0” corresponding to the Y data, U data, and V data.
[0180]
When the q or MF recovery circuit 355 is missing the q or motion flag MF and the q bit MF recovery circuit 353 cannot determine the correct value of q and the correct value of the motion flag MF, the ADRC decoding circuit 355 has the Y data corresponding to the code Q, The error flag of U data and V data is set to “1” and supplied to the block deshuffle circuit 356.
[0181]
The ADRC decoding circuit 355 has a missing dynamic range DR or the minimum pixel value MIN, and the DRMIN recovery circuit 354 can obtain the correct value of the dynamic range DR and the correct minimum value MIN of the pixel value. The code Q supplied from the DRMIN recovery circuit 354 is based on the correct value of q, the motion flag MF, the correct value of the dynamic range DR, and the correct value of the minimum value MIN of the pixel value supplied via the DRMIN recovery circuit 354. Decode and supply Y data, U data, and V data obtained as a result of decoding to block deshuffle circuit 356 together with an error flag set to “0” corresponding to Y data, U data, and V data To do.
[0182]
The ADRC decoding circuit 355 has a missing dynamic range DR or the minimum pixel value MIN, and the DRMIN recovery circuit 354 cannot obtain the correct value of the dynamic range DR and the correct minimum value MIN of the pixel value. , “1” is set to the error flag of Y data, U data, and V data corresponding to the code Q, and the error flag is supplied to the block deshuffle circuit 356.
[0183]
The recovery error determination circuit 357 outputs a recovery error determination flag Fd to the block deshuffle circuit 356 based on the emin1 and emin2 supplied from the q-bit MF recovery circuit 353, and the previous frame stored in the external memory 305. Update of the Y data, U data, and V data of the corresponding pixel is stopped.
[0184]
When the Y data, U data, and V data are supplied from the ADRC decoding circuit 355 together with the error flag set to “0”, the block deshuffle circuit 356 receives the error flag, Y data, U data, and V data. Remember. The block deshuffle circuit 356 stores the error flag when Y data, U data, and V data are supplied from the ADRC decoding circuit 355 together with the error flag set to “1”, and the Y data, U data, and Do not store V data. Further, the block deshuffle circuit 356 also receives the Y data of the corresponding pixel of the previous frame stored in the external memory 305 even when the continuous error signal is supplied from the recovery error determination circuit 357 or the inter-segment deshuffle circuit 352. Update of U data and V data is stopped.
[0185]
The block deshuffle circuit 356 returns the Y data, U data, and V data to the original sequence, the original Y-converted Y data by the reverse operation of the block shuffle circuit 102 of the transmission apparatus 1 shown in FIG. U data and V data are output.
[0186]
As described above, the decoding circuit 304 has the q-bit MF recovery circuit 353 and the DRMIN recovery circuit even if the data q necessary for decoding, the motion flag MF, the dynamic range DR, or the minimum pixel value MIN is lost. Since 354 recovers lost data, decryption can be performed.
[0187]
When there is an error in recovery, the pixel value stored in the external memory 305 is not overwritten, and the pixel value four frames before is output. Therefore, when there is no motion, it is possible to output a pixel value with no sense of incongruity.
[0188]
Next, the processing timing of the decoding circuit 304 will be described with reference to the timing charts of FIGS. FIG. 39 shows an example of the processing timing of the decoding circuit 304 when there is no omission in the data output from the inter-segment deshuffle circuit 352. The inter-segment deshuffle circuit 352 performs a deshuffling process, and outputs the deshuffled data to the ADRC decoding circuit 355 after a predetermined time has elapsed (in the example of FIG. 39, after 362 clocks from the start of input). The ADRC decoding circuit 355 decodes the data supplied from the inter-segment deshuffle circuit 352 and outputs it.
[0189]
FIG. 40 shows an example of processing timing of the decoding circuit 304 when data output from the inter-segment deshuffle circuit 352 is missing. The inter-segment deshuffle circuit 352 performs a deshuffling process, and when there is no missing data, the data is supplied earlier to the ADRC decoding circuit 355 (in the example of FIG. 40, 123 clocks after the start of input), q bits. The deshuffled data is output to the MF recovery circuit 353. The q-bit MF recovery circuit 353 supplies q and the motion flag MF to the DRMIN recovery circuit 354 at a predetermined timing.
[0190]
The DRMIN recovery circuit 354 supplies the dynamic range and the minimum pixel value to the ADRC decoding circuit 355 at the same timing as the inter-segment deshuffle circuit 352 supplies data to the ADRC decoding circuit 355 when there is no missing data. .
[0191]
Therefore, even if the decoding circuit 304 recovers q, the motion flag MF, the dynamic range DR, and the minimum pixel value MIN, the data is output at the same timing as when the data output from the inter-segment deshuffle circuit 352 is not missing. Can be decoded and output.
[0192]
As described above, since the decoding circuit 304 recovers the lost data even if the predetermined data necessary for decoding is lost, the decoding can be executed because the q-bit MF recovery circuit 353 and the DRMIN recovery circuit 354 recover the lost data. it can. Note that when a signal for selecting a mode in which error processing is not performed (user selectable mode) is input to the decoding circuit 304, the q-bit MF recovery circuit 353 and the DRMIN recovery circuit 354 do not perform recovery.
[0193]
The data output timing of the block deshuffle circuit 356 will be described with reference to FIG. 41 and FIG.
[0194]
Conventionally, when two memories for storing images of two frames are provided and data is written in one memory as shown in FIG. 41A, the data is read out from the other memory and output. When the processing for the frame image is completed, as shown in FIG. 41B, data is written to the other memory, data is read from one memory, and output.
[0195]
In contrast, the external memory 305 is provided with two banks each for storing an image of two frames (bank 1 and bank 2 in the figure), and the block deshuffle circuit 356 is as shown in FIG. Furthermore, when image data corresponding to three ADRCs is supplied from the ADRC decoding circuit 355, it is written in the bank 1, and as shown in FIG. 42B, when no image data is supplied from the ADRC decoding circuit 355. By specifying an address (for example, from the start of input to 364 clocks shown in FIG. 40), block deshuffled data is read from bank 2. In the next two frames, when the image data corresponding to the three ADRCs is supplied from the ADRC decoding circuit 355, the block deshuffle circuit 356 writes to the bank 2 and the image data is supplied from the ADRC decoding circuit 355. If not, the block deshuffled data is read from bank 1.
[0196]
In this way, the image data can be block deshuffled and output to the block deshuffle circuit 356 with one external memory 305.
[0197]
FIG. 43 is a block diagram showing a configuration of the recovery error determination circuit 357. The evaluation value emin1 having the minimum value and the evaluation value emin2 having the second smallest value supplied from the q-bit MF recovery circuit 353 are supplied to the difference calculation circuit 431. The difference calculation circuit 431 calculates the difference between the input emin1 and emin2 and outputs the difference to the delay circuit 432. The delay circuit 432 delays the data supplied from the difference calculation circuit 431 for a predetermined time and outputs the delayed data to the adder 435 and the delay circuit 433.
[0198]
The delay circuit 433 delays the data supplied from the delay circuit 432 for a predetermined time and outputs the delayed data to the adder 435 and the delay circuit 434.
[0199]
The delay circuit 434 delays the data supplied from the delay circuit 433 for a predetermined time and outputs the delayed data to the adder 435 and the determination circuit 437.
[0200]
The adder 435 adds the data supplied from the delay circuit 432, the data supplied from the delay circuit 433, and the data supplied from the delay circuit 434, and supplies the result to the determination circuit 436.
[0201]
The determination circuit 436 determines whether or not the data supplied from the adder 435 is equal to or less than a predetermined threshold Te1, and if it is determined that the data supplied from the adder 435 is equal to or less than the threshold Te1, 1 ”is output to the AND circuit 438, and when it is determined that the data supplied from the adder 435 exceeds the threshold Te 1,“ 0 ”is output to the AND circuit 438.
[0202]
The determination circuit 437 determines whether or not the data supplied from the delay circuit 434 is equal to or less than a predetermined threshold value Te2, and if it is determined that the data supplied from the delay circuit 434 is equal to or less than the threshold value Te2, 1 ”is output to the AND circuit 438, and if it is determined that the data supplied from the delay circuit 434 exceeds the threshold Te 2,“ 0 ”is output to the AND circuit 438.
[0203]
The AND circuit 438 outputs “1” to the hold circuit 439 when the data supplied from the determination circuit 436 is “1” and the data supplied from the determination circuit 437 is “1”. The AND circuit 438 outputs “0” to the hold circuit 439 when the data supplied from the determination circuit 436 is “0” or the data supplied from the determination circuit 437 is “0”.
[0204]
The hold circuit 439 holds the input data until a reset signal is input, and outputs the data to the delay circuit 440. The delay circuit 440 delays the data output from the hold circuit 439 for a predetermined time, supplies the data to the hold circuit 439, and outputs the data to the block deshuffle circuit 356 as the recovery determination flag Fd.
[0205]
In this way, the recovery error determination circuit 357 has an integrated value of the difference between the evaluation value emin1 and the evaluation value emin2 for the three ADRC blocks that is equal to or less than the threshold Te1, and the evaluation value emin1 and the evaluation value emin2 for the ADRC block. Is equal to or less than the threshold Te2, that is, when the difference between the evaluation value emin1 and the evaluation value emin2 is small and there is a high possibility that the recovered q or the motion flag MF is not correct, the recovery determination flag Fd is set to 1. .
[0206]
Next, a process in which data decoded by the ADRC decoding circuit 355 is temporarily stored in the block deshuffle circuit 356 and read will be described. The inter-segment deshuffle circuit 352 sets an error flag corresponding to the missing data to “1” when the data supplied from the packet disassembly circuit 302 is missing, and when there is no missing data, an error corresponding to the data. The flag is set to “0”, and the ADRC block unit code Q and error flag are output to the ADRC decoding circuit 355. The ADRC decoding circuit 355 decodes the code Q and outputs Y data, U data, V data, and an error flag to the block deshuffle circuit 356.
[0207]
When the corresponding error flag is “0”, the block deshuffle circuit 356 stores the Y data, the U data, the V data, and the error flag in the external memory 305, and the corresponding error flag is “1”. The error flag is stored in the external memory 305, and the Y data, U data, and V data are not stored in the external memory 305.
[0208]
The external memory 305 has two banks, and stores Y data, U data, V data, and an error flag of two frames in one bank. The bank of the external memory 305 is switched every two frames. Therefore, when the corresponding error flag is “1”, the external memory 305 stores Y data, U data, and V data four frames before.
[0209]
FIG. 44 is a diagram illustrating the processing of the block deshuffle circuit 356 when the recovery error determination circuit 357 outputs the recovery determination flag Fd in which “1” is set. The difference between the evaluation value emin1 having the minimum value supplied from the q-bit MF recovery circuit 353 and the evaluation value emin2 having the second smallest value is equal to or less than a predetermined value, and includes 3 consecutive ADRC blocks. When the sum of the differences between emin1 and emin2 of the two ADRC blocks is equal to or less than a predetermined value, the q-bit MF recovery circuit 353 determines that the recovery of q has failed, and thus determines that the data extraction has also failed. To do. At this time, the recovery error determination circuit 357 outputs a recovery determination flag Fd in which “1” is set.
[0210]
When the recovery error determination circuit 357 outputs the recovery determination flag Fd in which “1” is set, the block deshuffle circuit 356 has failed to cut out data from the subsequent buffers (88 ADRC blocks). Therefore, “1” is set in the error flag supplied thereafter, and the error flag is stored in the external memory 305. In the case of the user selectable escape mode, the data is stored in the external memory 305.
[0211]
FIG. 45 is a diagram for explaining the processing of the block deshuffle circuit 356 when the inter-segment deshuffle circuit 353 outputs a continuous error signal. The inter-segment deshuffle circuit 353 outputs a continuous error signal when it detects the loss of both the corresponding even-numbered ADRC block and odd-numbered ADRC block. Since the q-bit MF recovery circuit 353 recovers the q and the motion flag MF using the correlation between the pixel values of adjacent pixels, both the corresponding even-numbered ADRC block and odd-numbered ADRC block are missing. The q-bit MF recovery circuit 353 cannot execute the recovery process. When the continuous error signal is supplied, the block deshaffling circuit 356 considers that data extraction from the subsequent buffer (88 ADRC blocks) has failed, and sets “1” to the error flag supplied thereafter. And the error flag is stored in the external memory 305, and the decoded data supplied thereafter is not stored in the external memory 305.
[0212]
FIG. 46 is a diagram for explaining the processing of the block deshuffle circuit 356 in the case of the user-selectable gray mode. In the case of the gray mode, the block deshuffle circuit 356 adds gray (only corresponding data is set to 128, not all Y, U, and V are set to 128) to the data whose corresponding error flag is “1”. Set and output.
[0213]
47 and 48 are diagrams for explaining processing corresponding to a user-selectable recovery-off input (signal input when the user operates a predetermined switch (not shown)) of the block deshuffle circuit 356. FIG. When the recovery-off input in which “1” is set is supplied, the block deshuffle circuit 356 does not pass through the q-bit MF recovery circuit 353 to the external memory 305 as shown in FIG. When Y data, U data, and V data are decoded, the decoded Y data, U data, and V data are stored. As shown in FIG. 48, when q and the motion flag are recovered by the q-bit MF recovery circuit 353, the block deshuffle circuit 356 stores the decoded Y data, U data, and V data in the external memory 305. All the corresponding error flags are set to “1”. By changing the recovery off input, the recovery effect of the q-bit MF recovery circuit 353 and the like can be easily confirmed.
[0214]
FIG. 49 is a diagram for explaining processing corresponding to an escape signal (a signal input when a user operates a predetermined switch (not shown)) of the block deshuffle circuit 356. When a predetermined escape signal is supplied, the block deshuffle circuit 356 forcibly sets an error flag supplied thereafter even if the recovery determination flag Fd is set to “1”, as shown in FIG. The error flag is stored in the memory without setting “1”, and the decoded Y data, U data, and V data supplied thereafter are stored in the memory. By changing the escape signal, the effect of concealment processing of the block deshuffle circuit 356 can be easily confirmed.
[0215]
50 and 51 are flowcharts for explaining the process of outputting the decoded data by the block deshuffle circuit 356. FIG. In step S31, the inter-segment deshuffle circuit 352 receives the error flag generated by the receiving circuit 301 and supplies it to the ADRC decoding circuit 355. The ADRC decoding circuit 355 outputs the error flag supplied from the inter-segment deshuffle circuit 352 to the block deshuffle circuit 356, and the block deshuffle circuit 356 receives the error flag.
[0216]
In step S32, the block deshuffle circuit 356 selects the first error flag corresponding to the buffer (predetermined 88 ADRC blocks) supplied from the ADRC decoding circuit 355. In step S33, the block deshuffle circuit 356 determines whether or not the continuous error signal supplied from the inter-segment deshuffle circuit 352 is “0”, and it is determined that the continuous error signal is “0”. In step S34, it is determined whether the recovery determination flag Fd supplied from the recovery error determination circuit 357 is "0". If it is determined in step S34 that the recovery determination flag Fd is “0”, the process proceeds to step S35, and the block deshuffle circuit 356 stores an error flag in the external memory 305.
[0217]
If it is determined in step S33 that the continuous error signal is “1”, and if it is determined in step S34 that the recovery determination flag Fd is “1”, data extraction has failed, so step S47 The block deshuffle circuit 356 determines whether or not the escape signal is off. If it is determined that the escape signal is off, the block deshuffle circuit 356 proceeds to step S48 and sets the error flag selected in step S32 to “ 1 ″ is set, and the process proceeds to step S35, where the block deshuffle circuit 356 stores an error flag in the external memory 305.
[0218]
If it is determined in step S47 that the escape signal is on, the error flag is not set. Therefore, the procedure skips step S48, proceeds to step S35, and the block deshuffle circuit 356 stores the external memory 305 in the external memory 305. Store the error flag.
[0219]
In step S36, the block deshuffle circuit 356 determines whether or not all error flags corresponding to the buffer have been processed. If it is determined that all error flags have been processed, the process proceeds to step S37. The shuffle circuit 356 receives the Y data, U data, and V data decoded from the ADRC decoding circuit 355.
[0220]
If it is determined in step S36 that all the error flags in the buffer have not yet been processed, the process proceeds to step S49, where the block deshuffle circuit 356 selects the next error flag, returns to step S33, and returns the error flag. Repeat the setting process.
[0221]
In step S38, the block deshuffle circuit 356 selects the first decoded data in the buffer supplied from the ADRC decoding circuit 355.
[0222]
In step S39, the block deshuffle circuit 356 determines whether or not the continuous error signal supplied from the inter-segment deshuffle circuit 352 is “0”, and it is determined that the continuous error signal is “0”. In step S40, it is determined whether or not the error flag corresponding to the selected decoded data is “0”. If it is determined that the error flag corresponding to the selected decoded data is “0”, the process proceeds to step S41, and the block deshuffle circuit 356 selects the decoded Y data, U data, And V data are stored in the external memory 305. The procedure proceeds to step S42.
[0223]
If it is determined in step S39 that the continuous error signal is “1”, and if it is determined in step S40 that the error flag corresponding to the selected decoded data is “1”, Since data is missing, step S41 is skipped and the process proceeds to step S42.
[0224]
In step S42, the block deshuffle circuit 356 determines whether or not all the decoded data in the buffer has been processed. If it is determined that all the decoded data in the buffer has been processed, the process proceeds to step S43. .
[0225]
If it is determined in step S42 that all decoded data in the buffer (88 ADRC blocks) has not yet been processed, the process proceeds to step S50, where the block deshuffle circuit 356 selects the next data, Returning to step S39, the process of storing the decoded data is repeated.
[0226]
In step S43, the block deshuffle circuit 356 selects the first data stored in the external memory 305. In step S44, the block deshuffle circuit 356 determines whether or not the error flag is “1” and the gray mode, and if it is determined that the error flag is “0” or not the gray mode, In step S45, the selected data is output to the format conversion circuit 306.
[0227]
If it is determined in step S44 that the error flag is “1” and the gray mode is set, the process proceeds to step S51, where the block deshuffle circuit 356 outputs the gray data to the format conversion circuit 306, and then proceeds to step S46. move on.
[0228]
In step S46, the block deshuffle circuit 356 determines whether or not all the data stored in the bank of the external memory 305 has been processed, and determines that all the data stored in the external memory 305 has been processed. If so, the process ends.
[0229]
If it is determined in step S46 that all the data stored in the external memory 305 has not been processed, the process proceeds to step S52, and the block deshuffle circuit 356 proceeds to the next stored in the bank of the external memory 305. Data is selected, the process returns to step S44, and the process of outputting the data is repeated.
[0230]
Thus, the block deshuffle circuit 356 can output data with the least sense of incongruity even if the code Q is missing.
[0231]
Next, a process of outputting data of the block deshuffle circuit 356 when the user-selectable recovery off switch (not shown) is “0” will be described with reference to the flowchart of FIG. In step S71, the inter-segment deshuffle circuit 352 receives the error flag generated by the receiving circuit 301 and supplies it to the ADRC decoding circuit 355. The ADRC decoding circuit 355 outputs the error flag supplied from the inter-segment deshuffle circuit 352 to the block deshuffle circuit 356, and the block deshuffle circuit 356 receives the error flag.
[0232]
In step S72, the block deshuffle circuit 356 determines whether or not the q and the motion flag are recovered in the q-bit MF recovery circuit 353 and the like, and determines that the q and the motion flag MF and the like are not recovered. In this case, the process proceeds to step S73, and the error flag is stored in the external memory 305.
[0233]
If it is determined in step S72 that q and the motion flag have been recovered, the process proceeds to step S78, and the block deshuffle circuit 356 sets all the received error flags to “1”, and the process proceeds to step S73.
[0234]
In step S74, the block deshuffle circuit 356 receives the Y data, U data, and V data decoded from the ADRC decoding circuit 355.
[0235]
In step S75, the block deshuffle circuit 356 determines whether q and the motion flag etc. are recovered in the q-bit MF recovery circuit 353 etc., and determines that q and the motion flag MF etc. are not recovered. In the case, the process proceeds to step S76, and the Y data, U data, and V data received in step S74 are stored.
[0236]
If it is determined in step S75 that q, the motion flag MF, and the like have been recovered, step S76 is skipped, and the process proceeds to step S77 without storing Y data, U data, and V data.
[0237]
In step S77, the block deshuffle circuit 356 outputs the Y data, U data, V data, and error flag stored in the external memory 305 to the format conversion circuit 306, and ends the process.
[0238]
Thus, when the recovery off input is “1”, the block deshuffle circuit 356 does not output recovered data.
[0239]
As described above, since the decoding circuit 304 recovers the lost data even if the predetermined data necessary for decoding is lost, the decoding can be executed because the q-bit MF recovery circuit 353 and the DRMIN recovery circuit 354 recover the lost data. it can. Also, the decoding circuit 304 can output data with the least sense of incongruity even if it cannot be decoded. Further, by setting the recovery off input to “1”, the effect of recovery can be easily confirmed.
[0240]
Next, the missing pixel creation circuit 308 will be described. When the progressive pixel image is reproduced, the missing pixel creation circuit 308 performs peripheral classification in the same frame (frame t) of the pixel to be created as shown in FIG. Based on the pixel values of the pixels and the pixel values of the pixels in the previous frame (frame t-1) at the same position in the horizontal and vertical directions as these pixels, the pixel values of the missing pixels are created. To do. Further, the missing pixel creation circuit 308 performs, as shown in FIG. 53 (B), the pixel value of the surrounding pixels in the same field (field t) of the pixel to be created, The previous field (field t-1) at the same position in the horizontal and vertical directions as the pixel, and the two pixels before the same position in the horizontal and vertical directions as the surrounding pixels of the pixel to be created Based on the pixel values of the pixels in the field (field t-2), the pixel values of the missing pixels are created.
[0241]
FIG. 54 is a block diagram showing the configuration of the missing pixel creation circuit 308. As shown in FIG. The pixel value that is the data input to the missing pixel creation circuit 308 and the error flag indicating the missing pixel are supplied to the preprocessing circuit 501 and the tap construction circuit 502-1.
[0242]
Based on the input pixel value and the error flag indicating the missing pixel, the preprocessing circuit 501 generates a missing pixel value with a linear interpolation filter, sets the value to the missing pixel, This is supplied to the tap construction circuits 502-2 to 502-5. The pixel value is also referred to as tap data.
[0243]
The processing of the preprocessing circuit 501 will be described with reference to the flowchart of FIG. In step S131, the preprocessing circuit 501 determines whether or not the target pixel is missing based on the error flag. If it is determined that the target pixel is not missing, the process proceeds to step S132. The pixel value of the target pixel is set to the target pixel, and the process ends.
[0244]
If it is determined in step S131 that the target pixel is missing, the process proceeds to step S133, and the preprocessing circuit 501 determines which of the pixels adjacent to the target pixel is horizontally based on the error flag. Whether or not is missing is determined. If it is determined in step S133 that none of the pixels adjacent to the target pixel horizontally is missing, the process proceeds to step S134, and the preprocessing circuit 501 is adjacent to the target pixel horizontally. The average value of the two pixel values is set as the pixel value of the target pixel, and the process ends.
[0245]
When it is determined in step S133 that any of the pixels adjacent to the target pixel in the horizontal direction is missing, the process proceeds to step S135, and the preprocessing circuit 501 is adjacent to the target pixel in the horizontal direction. It is determined whether any of the pixels that are present is missing. If it is determined in step S135 that any of the pixels adjacent to the target pixel in the horizontal direction is not missing, the process proceeds to step S136, and the preprocessing circuit 501 is adjacent to the target pixel in the horizontal direction. The pixel value of the pixel that is not missing is set as the pixel value of the target pixel, and the process ends.
[0246]
If it is determined in step S135 that any of the horizontally adjacent pixels of the target pixel is missing, the process proceeds to step S137, and the preprocessing circuit 501 determines the target pixel based on the error flag. It is determined whether any of the vertically adjacent pixels is missing. When it is determined in step S137 that none of the pixels adjacent to the target pixel is missing, the process proceeds to step S138, and the preprocessing circuit 501 is adjacent to the target pixel vertically. The average value of the two pixel values is set as the pixel value of the target pixel, and the process ends.
[0247]
If it is determined in step S137 that any of the pixels adjacent to the target pixel in the vertical direction is missing, the process proceeds to step S139, and the preprocessing circuit 501 sets the target based on the error flag. It is determined whether all the pixels adjacent to the pixel are missing. When it is determined in step S139 that any pixel adjacent to the target pixel is not missing, the process proceeds to step S140, and the preprocessing circuit 501 is adjacent to the target pixel. The pixel value of the pixel that has not been set is set as the pixel value of the target pixel, and the process ends.
[0248]
If it is determined in step S139 that all the pixels adjacent to the target pixel are missing, the process proceeds to step S141, and the preprocessing circuit 501 determines the past frame at the same position as the target pixel. The pixel value of the pixel is set to the pixel value of the target pixel, and the process ends.
[0249]
As described above, the preprocessing circuit 501 linearly interpolates the pixel value of the target pixel from the pixel values of surrounding pixels. By the interpolation processing by the preprocessing circuit 501, the range of taps that can be used in subsequent processing can be expanded.
[0250]
The tap construction circuits 502-1 to 502-5 supply tap data and the like to the motion class generation circuit 503, the DR class generation circuit 504, the space class generation circuit 505, and the missing class generation circuit 506, respectively, at a predetermined timing. To do.
[0251]
The motion class generation circuit 503 generates a motion class code and a static flag based on the parameters supplied from the initialization circuit 309, the error flag supplied from the tap construction circuit 502-1, and the selected tap. The data is output to the construction circuits 502-2 to 205-5 and the class synthesis circuit 507. The motion class code has 2-bit information indicating the amount of motion, and the static flag indicates the presence / absence of motion by 1 bit.
[0252]
FIG. 56A is a diagram illustrating an example of 3 × 3 pixels centering on a pixel to be created, which is used for calculation of temporal activity by the motion class generation circuit 503. In FIG. 56A, “error” indicates a missing pixel. FIG. 56B is a diagram illustrating an example of 3 × 3 pixels of the previous frame corresponding to the pixel illustrated in FIG. L1 to L3 shown in FIGS. 56A and 56B each indicate a line, and the same number in the line indicates the same position in the vertical direction. Each of H1 to H3 shown in FIGS. 56A and 56B indicates the horizontal position of the pixel, and the same number indicates the same position in the horizontal direction.
[0253]
In the case shown in FIGS. 56 (A) and 56 (B), the time activity is calculated by Expression (12).

In Expression (12), () represents a pixel value of a pixel, and || represents a function for obtaining an absolute value.
[0254]
The motion class generation circuit 503 calculates a spatial activity obtained by adding 1 to the difference between the maximum value and the minimum value of 3 × 3 pixels centering on the pixel to be created.
[0255]
FIG. 57 is a diagram illustrating an example of a 3 × 3 pixel centered on a missing pixel to be created, which is used for calculation of a spatial activity by the motion class generation circuit 503. At this time, the spatial activity is calculated by Expression (13).
Spatial activity = Max (qi) −Min (qi) +1 (13)
In Expression (13), Max (qi) indicates the maximum value of the pixel values q1 to q9. Min (qi) represents the minimum value of the pixel values q1 to q9.
[0256]
The motion class generation circuit 503 selects thresholds having different values based on the spatial activity. The motion class generation circuit 503 sets a motion class code based on the motion determination threshold and the time activity.
[0257]
FIG. 58 is a diagram for explaining a threshold value for motion determination. A different value is used as the threshold for motion determination depending on the value of the spatial activity. If the spatial activity increases, a larger value threshold is used. This is because when the spatial activity is large, the movement is at least, and the temporal activity is a large value.
[0258]
In processing for standard density data, a motion class code is set using a fixed threshold value regardless of the spatial activity.
[0259]
Next, the process of setting the motion class code of the motion class generation circuit 503 will be described with reference to the flowchart of FIG. In step S151, the motion class generation circuit 503 determines whether or not the time activity is equal to or less than the threshold value 1. If it is determined that the time activity is equal to or less than the threshold value 1, the process proceeds to step S152, and the motion class code is determined. Set to 0 and the process ends.
[0260]
If it is determined in step S151 that the time activity exceeds the threshold value 1, the process proceeds to step S153, in which the motion class generation circuit 503 determines whether the time activity is equal to or less than the threshold value 2, and the time activity is equal to the threshold value 2. If it is determined as follows, the process proceeds to step S154, the motion class code is set to 1, and the process ends.
[0261]
If it is determined in step S153 that the time activity exceeds the threshold value 2, the process proceeds to step S155, and the motion class generation circuit 503 determines whether the time activity is less than or equal to the threshold value 3 and the time activity is less than or equal to the threshold value 3. If it is determined, the process proceeds to step S156, the motion class code is set to 2, and the process ends.
[0262]
If it is determined in step S155 that the time activity exceeds the threshold 3, the process proceeds to step S157, and the motion class generation circuit 503 sets the motion class code to 3, and the process ends.
[0263]
As described above, the motion class generation circuit 503 sets the motion class code based on the threshold and the time activity.
[0264]
The motion class generation circuit 503 sets the motion flag again based on the motion class codes of a plurality of pixels. For example, as shown in FIG. 60, the motion class generation circuit 503 sets the motion class code of the target pixel based on the motion class codes of the pixels around the pixel to be created.
[0265]
The process of setting the motion class code of the motion class generation circuit 503 will be described with reference to the flowchart of FIG. In step S171, the motion class generation circuit 503 determines whether or not the number of motion class codes for which a value of 3 is set among the motion class codes of the seven pixels is greater than the threshold 3, and the value of 3 When it is determined that the number of motion class codes for which is set is greater than the threshold 3, the process proceeds to step S172, 3 is set for the motion class code, and the process ends.
[0266]
If it is determined in step S171 that the number of motion class codes for which a value of 3 is set is equal to or less than the threshold value 3, the process proceeds to step S173, and the motion class generation circuit 503 has a value of 3 set. It is determined whether or not a value obtained by adding the number of motion class codes and the number of motion class codes for which a value of 2 is set is larger than a threshold value 2, and the number of motion class codes for which a value of 3 is set If it is determined that the value obtained by adding the number of motion class codes for which the value of 2 is set is greater than the threshold 2, the process proceeds to step S174, where 2 is set for the motion class code, and the process ends.
[0267]
If it is determined in step S173 that the value obtained by adding the number of motion class codes for which the value of 3 is set and the number of motion class codes for which the value of 2 is set is equal to or less than the threshold value 2, step S175 , The motion class generation circuit 503 determines the number of motion class codes for which a value of 3 is set, the number of motion class codes for which a value of 2 is set, and the motion class code for which a value of 1 is set. It is determined whether or not the value obtained by adding the number of is greater than the threshold 1, the number of motion class codes for which a value of 3 is set, the number of motion class codes for which a value of 2 is set, and If it is determined that the value obtained by adding the number of motion class codes for which the value is set is greater than the threshold value 1, the process proceeds to step S176, where 1 is set for the motion class code, and the process ends.
[0268]
In step S175, a value obtained by adding the number of motion class codes for which a value of 3 is set, the number of motion class codes for which a value of 2 is set, and the number of motion class codes for which a value of 1 is set Is determined to be equal to or less than the threshold value 1, the process proceeds to step S 177, and the motion class generation circuit 503 sets 0 to the motion class code, and the process ends.
[0269]
As described above, the motion class generation circuit 503 sets the final motion class code based on the motion class codes of a plurality of pixels and the threshold value stored in advance.
[0270]
As described above, the motion class generation circuit 503 sets and outputs a motion class code from pixel values of a plurality of pixels. The motion class generation circuit 503 sets and outputs a static motion flag based on the motion class code. For example, when the motion class code is 0 or 1, the static flag is set to 0, and when the motion class code is 2 or 3, the static flag is set to 1.
[0271]
The tap construction circuit 502-2 selects a prediction tap that covers all class structures based on the motion class code and static flag supplied from the motion class generation circuit 503, and the positions of the missing pixels. This is supplied to the selection circuit 508.
[0272]
The tap construction circuit 502-3 selects a tap and a class tap based on the motion class code and the static flag supplied from the motion class generation circuit 503, and supplies them to the DR class generation circuit 504 together with an error flag. The DR class generation circuit 504 generates a DR class code based on the tap, class tap, and error flag supplied from the tap construction circuit 502-3, and outputs the DR class code to the class synthesis circuit 507.
[0273]
The tap construction circuit 502-4 selects a tap and a class tap based on the motion class code and the static flag supplied from the motion class generation circuit 503, and supplies them to the space class generation circuit 505 together with an error flag. The space class generation circuit 505 generates a space class code based on the tap, class tap, and error flag supplied from the tap construction circuit 502-4 and outputs the generated space class code to the class synthesis circuit 507.
[0274]
The tap construction circuit 502-5 selects a tap based on the motion class code and the static motion flag supplied from the motion class generation circuit 503, and supplies them to the missing class generation circuit 506 together with the error flag. The missing class generation circuit 506 generates a missing class code based on the tap and error flag supplied from the tap construction circuit 502-5 and outputs the missing class code to the class synthesis circuit 507.
[0275]
The class synthesis circuit 507 integrates class codes based on the motion class code, static flag, DR class code, space class code, and missing class code, forms a final class code, and generates a coefficient holding class code. The data is output to the selection circuit 509.
[0276]
The coefficient holding class code selection circuit 509 causes the variable tap selection circuit 508 to select a variable tap based on the coefficient set supplied from the initialization circuit 309, the prediction structure, and the class code supplied from the class synthesis circuit 507. At the same time, the prediction coefficient sequence is output to the estimated prediction calculation circuit 510.
[0277]
A prediction tap data string selected by the variable tap selection circuit 508 will be described with reference to FIGS.
[0278]
FIG. 62 shows an example of the prediction tap variable selection range output by the tap construction circuit 502-2 to the variable tap selection circuit 508 and the prediction tap data string selected by the variable tap selection circuit 508 when the motion class code is 0. FIG. In FIG. 62, a tap surrounded by a solid line indicates a prediction tap variable selection range output from the tap construction circuit 502-2, and a tap surrounded by a dotted line indicates a prediction tap data string selected by the variable tap selection circuit 508. The prediction tap data string selected by the variable tap selection circuit 508 includes taps included in the prediction tap variable selection range output by the tap construction circuit 502-2.
[0279]
For example, when the motion class code is 0, the tap construction circuit 502-2 selects an adjacent tap. For example, when the motion class code is 0, the variable tap selection circuit 508 selects adjacent taps.
[0280]
FIG. 63 is a diagram illustrating a prediction tap variable selection range output by the tap construction circuit 502-2 and a prediction tap data sequence selected by the variable tap selection circuit 508 when the motion class code is 1. In FIG. 63, a tap surrounded by a solid line indicates a prediction tap variable selection range output by the tap construction circuit 502-2, and a tap surrounded by a dotted line indicates a prediction tap data string selected by the variable tap selection circuit 508. The prediction tap data string selected by the variable tap selection circuit 508 includes taps included in the prediction tap variable selection range output by the tap construction circuit 502-2.
[0281]
For example, when the motion class code is 1, the tap construction circuit 502-2 selects an adjacent tap and a tap that is not adjacent to the adjacent tap. For example, when the motion class code is 0, the variable tap selection circuit 508 selects five adjacent taps and five adjacent taps that are separated from the five adjacent taps.
[0282]
FIG. 64 is a diagram illustrating a prediction tap variable selection range output by the tap construction circuit 502-2 and a prediction tap data sequence selected by the variable tap selection circuit 508 when a missing tap exists and the motion class code is 0. It is. Black circles in the figure indicate missing taps. In FIG. 64, a tap surrounded by a solid line indicates a prediction tap variable selection range output by the tap construction circuit 502-2, and a tap surrounded by a dotted line indicates a prediction tap data string selected by the variable tap selection circuit 508. The prediction tap data string selected by the variable tap selection circuit 508 includes taps included in the prediction tap variable selection range output by the tap construction circuit 502-2.
[0283]
For example, when there is a missing tap and the motion class code is 0, the tap construction circuit 502-2 selects an adjacent tap. For example, when there is a missing tap and the motion class code is 0, the variable tap selection circuit 508 uses the same number of adjacent taps (eg, a tap next to a missing tap or a missing tap as when no missing tap exists). Taps in the same order as).
[0284]
FIG. 65 is a diagram showing a prediction tap variable selection range output by the tap construction circuit 502-2 and a prediction tap data sequence selected by the variable tap selection circuit 508 when a missing tap exists and the motion class code is 1. It is. Black circles in the figure indicate missing taps. In FIG. 64, a tap surrounded by a solid line indicates a prediction tap variable selection range output by the tap construction circuit 502-2, and a tap surrounded by a dotted line indicates a prediction tap data string selected by the variable tap selection circuit 508. The prediction tap data string selected by the variable tap selection circuit 508 includes taps included in the prediction tap variable selection range output by the tap construction circuit 502-2.
[0285]
For example, when there is a missing tap and the motion class code is 1, the tap construction circuit 502-2 selects an adjacent tap and a tap that is not adjacent to the adjacent tap. For example, when there is a missing tap and the motion class code is 0, the variable tap selection circuit 508 uses the same number of adjacent taps (eg, a tap next to a missing tap or a missing tap as when no missing tap exists). And the same number of adjacent taps as those when there are no missing taps.
[0286]
Based on the output mode set from the initialization circuit 309, the estimated prediction calculation circuit 510 receives the prediction tap data string supplied from the variable tap selection circuit 508 and the prediction coefficient string supplied from the coefficient holding class code selection circuit 509. Based on the linear estimation formula, the pixel value of the missing pixel is calculated.
[0287]
As described above, the missing pixel creation circuit 308 calculates a missing pixel value based on the pixel values around the missing pixel by class classification based on dynamic range, motion, lack, change in pixel value, and the like.
[0288]
Further, the estimated prediction calculation circuit 510 improves the image quality of the input image (increase in gradation (increase in the number of bits of Y data, U data, and V data), noise removal, quantization distortion removal ( Including the removal of distortion in the time direction) and the creation of quadruple density resolution.
[0289]
Next, the quadruple resolution creation circuit 312 will be described. When outputting an interlaced image, the quadruple-resolution creation circuit 312 uses a class classification adaptation process, as shown in FIGS. 66A and 66B, for a predetermined number of two fields. Based on the pixel value of the pixel, two pixels are created between the two input fields.
[0290]
When outputting the progressive image, the quadruple-resolution creation circuit 312 uses the class classification adaptation process as shown in FIGS. 66C and 66D, and as shown in FIG. Based on the pixel values of a predetermined number of pixels in one frame, two pixels are created between the two input frames.
[0291]
FIG. 67 is a block diagram showing a configuration of the quadruple-density resolution creation circuit 312. The Y data, U data, and V data supplied from the format conversion circuit 310 are supplied to the tap selection circuit 601, the tap selection circuit 602, and the tap selection circuit 603. The tap selection circuit 601 selects a pixel to be used for calculating a pixel value of a pixel to be created in order to quadruple the density according to the tap position information stored in the register 609. The pixels selected by the tap selection circuit 601 are supplied to the estimated prediction calculation circuit 607.
[0292]
The tap selection circuit 602 selects a pixel to be used for the spatial class classification from among the pixel values of pixels in the vicinity of the pixel to be created according to the tap position information stored in the register 610. The pixel selected by the tap selection circuit 602 is supplied to the space class detection circuit 604. The space class detection circuit 604 detects a space class. The detected space class is supplied to the class synthesis circuit 606.
[0293]
The tap selection circuit 603 selects a pixel to be used for class classification corresponding to a motion based on pixels in the vicinity of the pixel to be created according to the tap position information stored in the register 611. The pixel selected by the tap selection circuit 603 is supplied to the motion class detection circuit 605. The motion class detection circuit 605 detects a motion class. The detected motion class is supplied to the class synthesis circuit 606 and the register 610. The position information of the tap of the register 610 is switched according to the motion class supplied from the motion class detection circuit 605.
[0294]
The class synthesis circuit 606 integrates the space class and the motion class to form a final class code.
[0295]
This class code is supplied as an address to the coefficient memory 612, and a coefficient set corresponding to the class code is read from the coefficient memory 612. The coefficient set is supplied to the estimated prediction calculation circuit 607. Based on the pixel value supplied from the tap selection circuit 601 and the coefficient set supplied from the coefficient memory 612, the estimated prediction calculation circuit 607 calculates the pixel value of the pixel to be created using a linear estimation equation. . The output from the estimated prediction calculation circuit 607 is supplied to the line sequential conversion circuit 613 as line data L1 and line data L2.
[0296]
The coefficient set is Y data including 528 pixels in the horizontal direction and 240 pixels in the vertical direction as a teacher signal, and U data including 176 pixels in the horizontal direction and 120 pixels in the vertical direction, respectively. A coefficient set for each class is generated in advance using the data and the V data, and is stored in the initialization circuit 313.
[0297]
The line sequential conversion circuit 613 has a line memory, and outputs line data L1 and line data L2 output from the estimated prediction calculation circuit 607 in line sequential order based on output timing data stored in the register 608. The line sequential conversion circuit 613 outputs image data having a density four times that of the supplied image.
[0298]
The register 608, the register 609, the register 610, the register 611, and the coefficient memory 612 store the coefficient set supplied from the initialize circuit 313 when the initialization signal is supplied to the initialize circuit 313.
[0299]
FIG. 68 shows pixels input to the quadruple-resolution creation circuit 312 when the quadruple-resolution creation circuit 312 generates and outputs pixels of the quadruple-resolution using the interlace method (indicated by a large circle in the figure). And a position of a pixel (indicated by a small circle in the figure) output from the quadruple-resolution creation circuit 312. The frame input to the quadruple density creation circuit 312 has a frequency of 30 Hz, and the field output from the quadruple resolution creation circuit 312 has a frequency of 60 Hz. The output first field pixel is shifted from the input pixel by one line of the screen line output in the vertical direction, and the number of pixels per line is doubled. It has become. The output pixels of the second field are shifted from the input pixels by one line below the screen line output in the vertical direction, and the number of pixels per line is doubled. It has become.
[0300]
FIG. 69 shows pixels input to the quadruple-resolution creation circuit 312 when the quadruple-resolution creation circuit 312 generates and outputs pixels of the quadruple-resolution in a progressive manner (in the figure, a large circle). And a position of a pixel (indicated by a small circle in the figure) output from the quadruple-resolution creation circuit 312. The frame input to the quadruple-resolution creation circuit 312 has a frequency of 30 Hz, and the frame output from the quadruple-resolution creation circuit 312 has a frequency of 60 Hz. The pixels of the output frame are output on the same line as the input pixels, and the number of pixels per line is doubled.
[0301]
As described above, the quadruple-resolution creation circuit 312 creates an image having twice the number of pixels per line at the double frame frequency or double the field frequency based on the input image. Output.
[0302]
Next, the UV vertical doubling circuit 333 will be described. FIG. 70 is a block diagram showing a configuration of the UV vertical doubling circuit 333. The UV vertical doubling circuit 333 receives U data and V data in which the number of pixels is increased by the quadruple-resolution creation circuit 312. The input data is delayed by one line by line FIFOs 701-1 to 701-6 and sequentially output to the subsequent stage.
[0303]
The multiplier circuit 702-1 multiplies the input data by a coefficient and outputs the result to the adder circuit 703.
[0304]
Multipliers 702-2 to 702-7 multiply the input data by coefficients from line FIFOs 701-1 to 701-6, respectively, and output the result to adder 703.
[0305]
The adder circuit 703 adds the data supplied from the multiplier circuits 702-1 to 702-7 and supplies the sum to the latch circuit 704. When a clock signal is input, the latch circuit 704 latches the input data.
[0306]
The operation of the UV vertical doubling circuit 333 will be described with reference to FIG. The line FIFOs 701-1 to 701-6 output data of pixels (pixels indicated by circles in FIG. 71) at the same position in the horizontal direction of the screen on six lines continuous to the input line. The multiplier circuits 702-1 to 702-7 multiply the input data by a coefficient and output the result to the adder circuit 703. The adder circuit 703 adds the output data and outputs it to the latch circuit 704. A clock signal is input to the latch circuit 704 at a timing at which the number of lines is doubled (a timing indicated by a black triangle in FIG. 71). Therefore, the UV vertical doubling circuit 333 interpolates the number of lines of the input U data and V data twice, and outputs the result.
[0307]
Next, reception processing of the reception device 2 will be described with reference to the flowchart of FIG. In step S <b> 201, the packet disassembly circuit 302 disassembles the packet received by the reception circuit 301 and supplies the packet to the decoding circuit 304. In step S202, the decoding circuit 304 decodes the data input from the packet disassembly circuit 302 by the ADRC method. When data is lost during transmission, in the decoding of the decoding circuit 304, q, the motion flag MF, the dynamic range DR, or the minimum pixel value MIN is recovered, and Y data, U data, and V data are decoded. .
[0308]
In step S203, the format conversion circuit 306 converts the Y data, U data, and V data supplied from the decoding circuit 304 into a predetermined format that can be processed by the missing pixel creation circuit 308. In step S204, the missing pixel creation circuit 308 creates a pixel that cannot be decoded by the decoding circuit 304 among the missing pixels by class adaptation processing.
[0309]
In step S205, the format conversion circuit 310 converts the Y data, U data, and V data supplied from the missing pixel creation circuit 308 into a predetermined format that can be processed by the quadruple-resolution creation circuit 312. In step S206, the quadruple-resolution creation circuit 312 creates a pixel by class adaptation processing based on the input Y data, U data, and V data, quadruples the pixel density, and interpolates 314. Output to.
[0310]
In step S207, the format conversion circuit 331 changes the data format so that the Y data, U data, and V data supplied from the quadruple resolution creation circuit 312 can be subjected to predetermined interpolation processing. In step S208, the UV vertical doubling circuit 333 interpolates pixels in the vertical direction with respect to the U data and V data so that the number of lines is doubled. In step S209, the D / A conversion circuit 335 performs digital-analog conversion at a predetermined clock rate, outputs an analog Y signal, an analog U signal, and an analog V signal, and the process ends.
[0311]
As described above, the receiving device 2 can perform decoding even if data is lost in the transmission path, and creates a missing pixel and expands it almost without being affected by the missing pixel. Can be output.
[0312]
Note that the image transmission system has been described as transmitting and receiving images, but not limited to images, data such as audio may be transmitted.
[0313]
The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer.
[0314]
FIG. 73 is a diagram illustrating an example of a recording medium and a computer. A CPU (Central Processing Unit) 811 actually executes various application programs and an OS (Operating System). A ROM (Read-only Memory) 812 generally stores basically fixed data out of programs used by the CPU 811 and calculation parameters. A RAM (Random-Access Memory) 813 stores programs used in the execution of the CPU 811 and parameters that change as appropriate during the execution. These are connected to each other by a host bus 814 including a CPU bus.
[0315]
The host bus 814 is connected to an external bus 816 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 815.
[0316]
The keyboard 818 is operated by the user when inputting various commands to the CPU 811. The mouse 819 is operated by the user when specifying or selecting a point on the screen of the display 820. The display 820 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text and images. An HDD (Hard Disk Drive) 821 drives hard disks and causes them to record or reproduce programs executed by the CPU 811 and information.
[0317]
The drive 822 reads data or a program recorded in the magnetic disk 841, the optical disk 842, the magneto-optical disk 843, or the semiconductor memory 844 that is loaded, and the data or program is read out to the interface 817, the external bus 816, and the bridge. 815 and the RAM 813 connected via the host bus 814.
[0318]
These keyboard 818 to drive 822 are connected to an interface 817, and the interface 817 is connected to the CPU 811 via an external bus 816, a bridge 815, and a host bus 814.
[0319]
As shown in FIG. 73, the recording medium is distributed to provide a program to the user separately from the computer, and a magnetic disk 841 (including a floppy disk) on which the program is recorded, an optical disk 842 (CD-ROM). (Including Compact Disc-Read Only Memory), DVD (Digital Versatile Disc)), magneto-optical disk 843 (including MD (Mini-Disc)), or a package medium composed of semiconductor memory 844, etc. The program is provided with a ROM 812, HDD 821, and the like, which are provided to the user in a state of being pre-installed in the computer and in which a program is recorded.
[0320]
In the present specification, the step of describing the program stored in the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.
[0321]
Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.
[0322]
【The invention's effect】
  As aboveAccording to the present invention,The scale of hardware for multiplication can be reduced, and the amount of memory for storing the coefficient table can be reduced.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating the structure of a conventional packet.
FIG. 2 is a diagram illustrating an image transmission system that transmits an image.
FIG. 3 is a block diagram showing a configuration of an embodiment of a transmission apparatus 1 according to the present invention.
FIG. 4 is a diagram illustrating a process in which an image signal input to the transmission apparatus 1 is compressed.
5 is a block diagram showing a configuration of a thinning filter 43. FIG.
6 is a diagram for explaining an output of a latch circuit 81; FIG.
7 is a diagram for explaining an output of a latch circuit 84; FIG.
FIG. 8 is a block diagram showing a configuration of a UV vertical 1/2 circuit 45;
9 is a diagram for explaining an output of a latch circuit 94. FIG.
10 is a block diagram showing a configuration of an encoding circuit 15. FIG.
FIG. 11 is a diagram illustrating a unit of processing.
FIG. 12 is a diagram illustrating an ADRC block.
FIG. 13 is a diagram illustrating a relationship between pixels in a frame and an ADRC block.
FIG. 14 is a diagram illustrating a relationship between a pixel in a frame and an ADRC block.
FIG. 15 is a diagram illustrating a relationship between pixels in a frame and an ADRC block.
FIG. 16 is a diagram illustrating an ADRC block and a segment of Y data.
FIG. 17 is a diagram for explaining ADRC blocks and segments of U data or V data;
FIG. 18 is a diagram illustrating shuffle performed in units of ADRC blocks.
FIG. 19 is a diagram illustrating segments.
20 is a diagram illustrating the configuration of an ADRC encoding circuit 103. FIG.
21 is a diagram illustrating a configuration of an information amount control circuit 164. FIG.
22 is a diagram illustrating a configuration of an information amount control circuit 164. FIG.
23 is a diagram for explaining how to shuffle the inter-segment shuffle circuit 104. FIG.
24 is a diagram for explaining how to shuffle the inter-segment shuffle circuit 104. FIG.
25 is a diagram for explaining how to shuffle the inter-segment shuffle circuit 104. FIG.
FIG. 26 is a diagram for explaining how to shuffle the inter-segment shuffle circuit 104;
FIG. 27 is a diagram illustrating a configuration of a packet.
28 is a block diagram showing a configuration of an inter-segment shuffle circuit 104. FIG.
29 is a diagram illustrating a counter group of the input data counter unit 212. FIG.
30 is a diagram illustrating a counter group of the output data counter unit 216. FIG.
FIG. 31 is a diagram illustrating image transmission processing of the transmission apparatus 1;
32 is a block diagram illustrating a configuration of the reception device 2. FIG.
FIG. 33 is a diagram illustrating a process in which an image is expanded.
34 is a block diagram of the decoding circuit 304. FIG.
35 is a block diagram showing a configuration of a q-bit MF recovery circuit 353. FIG.
FIG. 36 is a diagram illustrating a recovery process of a dynamic range DR or a minimum pixel value MIN.
FIG. 37 is a diagram for describing recovery processing of a dynamic range DR or a minimum pixel value MIN.
FIG. 38 is a diagram illustrating a recovery process of a dynamic range DR or a minimum pixel value MIN.
FIG. 39 is a diagram illustrating an example of processing timing of the decoding circuit 304 when there is no omission in data output from the inter-segment deshuffle circuit 352;
FIG. 40 is a diagram illustrating an example of processing timing of the decoding circuit 304 when data output from the inter-segment deshuffle circuit 352 is missing.
FIG. 41 is a diagram illustrating a conventional data storage method.
42 is a diagram illustrating data output timing of the block deshuffle circuit 356. FIG.
43 is a block diagram showing a configuration of a recovery error determination circuit 357. FIG.
FIG. 44 is a diagram illustrating processing of the block deshuffle circuit 356 when the recovery error determination circuit 357 outputs a recovery determination flag Fd in which “1” is set.
FIG. 45 is a diagram illustrating processing of the block deshuffle circuit 356 when the inter-segment deshuffle circuit 353 outputs a continuous error signal.
FIG. 46 is a diagram illustrating processing of the block deshuffle circuit 356 in the gray mode.
47 is a diagram for describing processing corresponding to a recovery-off input of the block deshuffle circuit 356; FIG.
FIG. 48 is a diagram for explaining processing corresponding to a recovery-off input of the block deshuffle circuit 356;
49 is a diagram illustrating processing corresponding to an escape signal of the block deshuffle circuit 356. FIG.
FIG. 50 is a flowchart for describing processing of outputting decoded data by the block deshuffle circuit 356;
FIG. 51 is a flowchart for describing processing of outputting decoded data by the block deshuffle circuit 356;
FIG. 52 is a flowchart for describing processing for outputting data of the block deshuffle circuit 356 when the recovery-off input is “1”.
FIG. 53 is a diagram for explaining pixels used for processing by the missing pixel creation circuit 308;
54 is a block diagram showing a configuration of a missing pixel creation circuit 308. FIG.
FIG. 55 is a flowchart for explaining processing of the preprocessing circuit 501;
FIG. 56 is a diagram illustrating an example of pixels used for calculation of temporal activity.
FIG. 57 is a diagram illustrating an example of pixels used for calculation of spatial activity.
FIG. 58 is a diagram for explaining a threshold value for motion determination.
FIG. 59 is a flowchart for describing processing for setting a motion class code in a static motion determination circuit 521;
FIG. 60 is a diagram illustrating an example of pixels used for majority decision of a motion class code.
FIG. 61 is a flowchart for describing processing for setting a motion class code in the motion class generation circuit 503;
FIG. 62 is a diagram for explaining a predicted tap data string selected by the variable tap selection circuit 508;
FIG. 63 is a diagram for explaining a prediction tap data string selected by the variable tap selection circuit 508;
FIG. 64 is a diagram for explaining a prediction tap data string selected by the variable tap selection circuit 508;
FIG. 65 is a diagram for explaining a prediction tap data string selected by the variable tap selection circuit 508;
FIG. 66 is a diagram for explaining the positions of pixels used in the processing of the quadruple-resolution creation circuit 312;
67 is a block diagram showing a configuration of a quadruple-resolution creation circuit 312. FIG.
68 is a diagram showing the positions of pixels input to the quadruple density resolution creation circuit 312 and the pixel positions output by the quadruple density resolution creation circuit 312 when the quadruple resolution creation circuit 312 outputs in an interlaced manner. FIG. It is.
FIG. 69 is a diagram showing the positions of pixels input to the quadruple density creation circuit 312 and the positions of pixels output by the quadruple density resolution creation circuit 312 when the quadruple density creation circuit 312 outputs in a progressive manner; It is.
70 is a block diagram showing a configuration of a UV vertical doubling circuit 333. FIG.
71 is a diagram illustrating an output of a latch circuit 704. FIG.
72 is a flowchart for describing reception processing of the reception device 2. FIG.
Fig. 73 is a diagram for describing an example of a recording medium and a computer.
[Explanation of symbols]
1 transmitter, 2 receiver, 13 decimation unit, 15 encoding circuit, 17 packetization circuit, 42 A / D conversion circuit, 43 decimation filter, 45 UV vertical 1/2 circuit, 102 block shuffle circuit, 103 ADRC encoding Circuit, 104 inter-segment shuffle circuit, 164 information control circuit, 171 ADRC encoder, 104 inter-segment shuffle circuit, 111 data selection unit, 212 input data counter unit, 213 input / output controller, 214 address generation unit, 215 storage unit, 216 Output data counter section, 302 packet disassembly circuit, 304 decoding circuit, 305 external memory, 308 missing pixel creation circuit, 312 quadruple resolution creation circuit, 333 UV vertical double circuit, 335 D / A conversion circuit 352 inter-segment deshuffling circuit, 353 q-bit MF recovery circuit, 354 DRMIN recovery circuit, 355 ADRC decoding circuit, 356 block deshful circuit, 357 recovery error determination circuit, 501 preprocessing circuit, 502-1 to 502-5 tap construction circuit , 503 motion class generation circuit, 504 DR class generation circuit, 505 space class generation circuit, 506 missing class generation circuit, 507 class synthesis circuit, 508 variable tap selection circuit, 509 coefficient holding class code selection circuit, 510 estimation prediction calculation circuit

Claims (6)

  Y data, U data, and V data generated by converting a component signal are stored in a first memory, and the stored Y data, U data, and V data are in the form of an ADRC block. Block shuffling means for shuffling within a segment range and ADRC block unit,
  Encoding means for encoding the Y data, the U data, and the V data supplied from the block shuffle means based on an ADRC (Adaptive Dynamic Range Coding) method;
  The data supplied from the encoding means is stored in a second memory, and the inter-segment shuffling means for shuffling by changing the storage and reading addresses;
  Packetizing means for packetizing data supplied from the inter-segment shuffling means into a packet including a dynamic range, a motion flag, a minimum pixel value, an index of a quantization table, and a code Q in one packet;
  With
  The inter-segment shuffle means includes data selection means, input data counter means, input / output controller means, address generation means, and output data counter means,
  The data selection means is inputted in parallel for each element of the first data composed of the dynamic range, the motion flag, the minimum value of the pixel value, and the index of the quantization table, and the first data and Second data including the code Q is input at alternate timings, the input data is converted into serial data, and output to the input / output controller means,
  The input / output controller means stores the input data in the second memory based on the address generated by the address generating means,
  The input / output controller means reads the data stored in the second memory based on the address generated by the address generating means according to the data supplied from the output data counter means, and sends it to the packetizing means. Output
  Information processing device.   The input data counter means includes a group of counters indicating information for identifying the type of data input to the data selection means, addressing when storing in the second memory, and a direction of shuffling.
  The information processing apparatus according to claim 1.   The counter group includes addressing and shuffling.
  The counter for the first data for addressing is a counter that counts the number of blocks of the first data in one buffer, a counter that counts the number of packets of the first data in one segment, and A counter that counts the total number of packets of the first data in one bank;
  The counter group for the second data for addressing is a counter for counting the number of packets of the second data in one buffer, and a counter for counting the total number of packets of the second data in one bank A counter that counts the number of buffers of the second data in one segment, and a remainder when the second data in any order is divided by the number of bytes N of the second data per packet Consists of counters
  The first data counter for shuffling is a counter that counts the number of segments of the first data in one bank, and a counter that determines the shuffling direction of the first data in one segment. Configured,
  The second data counter for shuffling is a counter that counts the number of segments of the second data in one bank, and a counter that determines the shuffling direction of the second data in one segment; Consists of
  The information processing apparatus according to claim 2.   The output data counter means comprises a group of addressing counters,
  The counter for addressing the first data is a counter that counts an index of the quantization table in a data cell of 1 packet size, a counter that counts the number of motion flags in a data cell of 1 packet size, and a counter of 1 packet size A counter that counts the number of the minimum values of the dynamic range and the pixel value in the data cell, and counts the number of blanks that are inserted to handle the index of the quantization table of bit size and the motion flag as byte size data Counter that determines the number of the second data in the packet in one packet, and a counter that counts the number of packets in one buffer.
  The information processing apparatus according to claim 1.   The data stored in the second memory is read based on the address generated by the address generating means according to the counter group of the output data counter means,
  In the reading, when a pulse for reading is input to the inter-segment shuffling means at a predetermined timing, an index of the quantization table and the motion flag are collected in units of bytes,
  When the gathering process is completed, the data is read from the second memory according to the arrangement of data in the packet and output to the packetizing means.
  The information processing apparatus according to claim 1.   Y data, U data, and V data generated by converting a component signal are stored in a first memory, and the stored Y data, U data, and V data are in the form of an ADRC block. , Block shuffle within the segment range and ADRC block unit,
  The block shuffled Y data, U data, and V data are encoded based on an ADRC (Adaptive Dynamic Range Coding) method,
  The encoded data is stored in a second memory, and the address of the storage and reading is changed to shuffle between segments.
  The inter-segment shuffled data is packetized into one packet into a packet including a dynamic range, motion flag, minimum pixel value, quantization table index, and code Q.
  Including steps,
  The inter-segment shuffle is
  For each element of the first data composed of the dynamic range, the motion flag, the minimum value of the pixel value, and the index of the quantization table, three pieces of data are input in parallel, and the code Q is switched at the timing alternating with the first data. The input data is converted into serial data, and the input data is stored in the second memory based on the address generated by the address generation means, and the second data is stored in the second memory. The data stored in the second memory is read out based on the address generated by the address generating means.
  Information processing method.

Images