JP2004096715A - Image data interpolation apparatus, computer for image data interpolation processing, image data interpolation method and medium with image data interpolation program recorded thereon - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image data interpolation apparatus, an image data interpolation method and a medium with an image data interpolation program recorded thereon in which image data are efficiently saved while also including color images. <P>SOLUTION: In a computer system having a scanner or the like as an image input device and having a color printer or the like as an image output device, original image data are inputted to a printer driver to count the number of colors utilized in images to extract a feature amount for judging whether images of the image data are natural images or unnatural images, and determines whether the images are natural images or unnatural images based upon the feature amount to execute nearest interpolation processing in the case of unnatural images or to execute cubic interpolation processing in the case of natural images. Thus, interpolation processing corresponding to features of images is implemented to extremely easily obtain the optimal interpolated result. <P>COPYRIGHT: (C)2004,JPO

Description

また、ここで距離に応じた影響度合いを３次たたみ込み関数で表すとすると、ｆ（ｔ）　＝　｛ｓｉｎ（πｔ）｝／πｔ
となる。なお、上述した各距離ｘ１〜ｘ４，ｙ１〜ｙ４は格子点Ｐｕｖの座標値（ｕ，ｖ）について絶対値を利用して次のように算出することになる。
ｘ１　＝　１＋（ｕ−｜ｕ｜）　　　　　ｙ１　＝　１＋（ｖ−｜ｖ｜）
ｘ２　＝　　　（ｕ−｜ｕ｜）　　　　　ｙ２　＝　　　（ｖ−｜ｖ｜）
ｘ３　＝　１−（ｕ−｜ｕ｜）　　　　　ｙ３　＝　１−（ｖ−｜ｖ｜）
ｘ４　＝　２−（ｕ−｜ｕ｜）　　　　　ｙ４　＝　２−（ｖ−｜ｖ｜）
以上の前提のもとでＰについて展開すると、
【００７４】
【数２】

となる。なお、３次たたみ込み関数と呼ばれるように距離に応じた影響度合いｆ（ｔ）は次のような三次式で近似される。
【数３】

このキュービック法では一方の格子点から他方の格子点へと近づくにつれて徐々に変化していき、その変化具合がいわゆる３次関数的になるという特徴を有している。
【００７５】
図２５と図２６はキュービック法にて補間される際の具体例を示している。理解を容易にするため、垂直方向についてのデータの変化はなく、水平方向についてエッジが生じているモデルについて説明する。また、補間する画素を３点とする。
まず、図２６の具体的数値について説明する。補間前の画素の階調値を左列に「Ｏｒｉｇｉｎａｌ」として示しており、階調値「６４」の画素（Ｐ０、Ｐ１、Ｐ２、Ｐ３）が４点並び、階調値「１２８」の画素（Ｐ４）を１点挟み、階調値「１９２」の画素（Ｐ５、Ｐ６、Ｐ７、Ｐ８、Ｐ９）が５点並んでいる。この場合、エッジは階調値「１２８」の画素の部分である。
【００７６】
ここで各画素間に３点の画素（Ｐｎ１、Ｐｎ２、Ｐｎ３）を内挿することになると、内挿される画素間の距離は「０．２５」となり、上述したｘ１〜ｘ４は内挿点毎に表の中程の列の数値となる。ｘ１〜ｘ４に対応してｆ（ｘ１）〜ｆ（ｘ４）も一義的に計算されることになり、例えば、ｘ１，ｘ２，ｘ３，ｘ４が、それぞれ「１．２５」、「０．２５」、「０．７５」、「１．７５」となる場合、それに対するｆ（ｔ）については、概略「−０．１４」、「０．８９」、「０．３０」、「−０．０５」となる。また、ｘ１，ｘ２，ｘ３，ｘ４が、それぞれ「１．５０」、「０．５０」、「０．５０」、「１．５０」となる場合、それに対するｆ（ｔ）については、「−０．１２５」、「０．６２５」、「０．６２５」、「−０．１２５」となる。また、ｘ１，ｘ２，ｘ３，ｘ４が、それぞれ「１．７５」、「０．７５」、「０．２５」、「１．２５」となる場合、それに対するｆ（ｔ）については、概略「−０．０５」、「０．３０」、「０．８９」、「−０．１４」となる。以上の結果を用いて内挿点の階調値を演算した結果を表の右列に示しているとともに、図２５においてグラフで示している。なお、このグラフの意味するところについて後に詳述する。
【００７７】
垂直方向についてのデータの変化がないものとみなすと、演算は簡略化され、水平方向に並ぶ四つの格子点のデータ（Ｐ１，Ｐ２，Ｐ３，Ｐ４　）だけを参照しつつ、内挿点から各格子点までの距離に応じた影響度合いｆ（ｔ）を利用して次のように算出できる。
Ｐ＝Ｐ１・ｆ（ｘ１）＋Ｐ２１ｆ（ｘ２）＋Ｐ３・ｆ（ｘ３）＋Ｐ４・ｆ（ｘ４）
従って、内挿点Ｐ２１について算出する場合には、

となる。
【００７８】
キュービック法によれば３次関数的に表せる以上、そのカーブの形状を調整することによって補間結果の品質を左右することができる。
その調整の一例として、
０＜ｔ＜０．５　ｆ（ｔ）　＝　−（８／７）ｔ＊＊３−（４／７）ｔ＊＊２＋１
０．５＜ｔ＜１　ｆ（ｔ）　＝　（１−ｔ）（１０／７）
１＜ｔ＜１．５　ｆ（ｔ）　＝　（８／７）（ｔ−１）＊＊３＋（４／７）（ｔ−１）＊＊２−（ｔ−１）
１．５＜ｔ＜２　ｆ（ｔ）　＝　（３／７）（ｔ−２）
としたものをハイブリッドバイキュービック法と呼ぶことにする。
【００７９】
図２７はハイブリッドバイキュービック法にて補間される際の具体例を示しており、キュービック法の場合と同じ仮定のモデルについて補間した結果を示している。また、図２５にもハイブリッドバイキュービック法による補間処理結果を示しており、この例では３次関数的なカーブがわずかに急峻となり、画像全体のイメージがシャープとなる。
上述したニアリスト法やキュービック法やハイブリッドバイキュービック法の特性の理解のために他の補間手法である共１次内挿法（バイリニア補間：以下、バイリニア法と呼ぶ）について説明する。
【００８０】
バイリニア法は、図２８に示すように、一方の格子点から他方の格子点へと近づくにつれて徐々に変化していく点でキュービック法に近いが、その変化が両側の格子点のデータだけに依存する一次関数的である点で異なる。すなわち、内挿したい点Ｐｕｖを取り囲む四つの格子点Ｐｉｊ，Ｐｉ＋１ｊ，Ｐｉｊ＋１，Ｐｉ＋１ｊ＋１で区画される領域を当該内挿点Ｐｕｖで四つの区画に分割し、その面積比で対角位置のデータに重み付けする。これを式で表すと、
Ｐ＝｛（ｉ＋１）−ｕ｝｛（ｊ＋１）−ｖ｝Ｐｉｊ
＋｛（ｉ＋１）−ｕ｝｛ｖ−ｊ｝Ｐｉｊ＋１
＋｛ｕ−ｉ　　　　｝｛（ｊ＋１）−ｖ｝Ｐｉ＋１ｊ
＋｛ｕ−ｉ　　　　｝｛ｖ−ｊ｝Ｐｉ＋１ｊ＋１
となる。なお、ｉ＝［ｕ］、ｊ＝［ｖ］である。
【００８１】
二つのキュービック法とバイリニア法は一方の格子点から他方の格子点へと近づくにつれて徐々に変化していく点で共通するが、その変化状況が３次関数的であるか１次関数的であるかが異なり、画像としてみたときの差異は大きい。図２９はニアリスト法とキュービック法とハイブリッドバイキュービック法とバイリニア法における補間結果の相違を理解しやすくするために二次元的に表した図である。同図において、横軸に位置を示し、縦軸に補間関数を示している。むろん、この補間関数は上述した距離に応じた影響度合いに該当する。ｔ＝０、ｔ＝１、ｔ＝２の位置に格子点が存在し、内挿点はｔ＝０〜１の位置となる。
【００８２】
バイリニア法の場合、隣接する二点間（ｔ＝０〜１）で直線的に変化するだけであるので境界をスムージングすることになり、画面の印象はぼやけてしまう。すなわち、角部のスムージングと異なり、境界がスムージングされると、コンピュータグラフィックスでは、本来あるべき輪郭がなくなってしまうし、写真においてはピントが甘くなってしまう。
一方、キュービックにおいては、隣接する二点間（ｔ＝０〜１）においては山形の凸を描いて徐々に近接するのみならず、さらに同二点間の外側（ｔ＝１〜２）において下方に押し下げる効果をもつ。すなわち、あるエッジ部分は段差が生じない程度に大きな高低差を有するように変化され、写真においてはシャープさを増しつつ段差が生じないという好適な影響を及ぼす。また、ハイブリッドバイキュービックではよりシャープさを増す影響を及ぼす。なお、キュービック法は演算処理量が大きく、補間倍率が大きくなって補間すべき画素数が大きくなれば多大な演算処理量を要することになる。
【００８３】
画質の面を重視すれば、キュービック法のような三次関数を選びそうであるが、コンピュータの処理では速度と画質のバランスも大きい。すなわち、画質の向上程度に応じて処理速度の低下具合の許容度が大きくなるが、画質の向上が微量あるいは多少画質が向上落ちるとしても処理速度が高速である方を好むという場合もある。
一方、以上のような補間関数の比較とともに具体的な数値を示す図２５、図２６、図２７を参照するとより理解しやすい。図１５の例を参照し、もともとのエッジ部分である階調値「６４」の画素（Ｐ３）と、階調値「１２８」の画素（Ｐ４）と、階調値「１９２」の画素（Ｐ５）という三点に注目してみると、単純に直線的に連結する手法はバイリニア法に相当し、これに対してキュービック法では具体的なＳ字カーブが形成されているし、ハイブリッドバイキュービック法ではそのＳ字カーブがより急峻となっている。むろん、Ｓ字カーブの方向は画素の階調値変化を急峻とする方向であり、だからこそエッジが強調されている。また、このエッジ画素に隣接する領域（Ｐ２〜Ｐ３、Ｐ５〜Ｐ６）ではいわゆるアンダーシュートとオーバーシュートが生じており、低い側に生じるアンダーシュートと高い側に生じるオーバーシュートにより、エッジ画素を挟む両側の高低差が大きくなる。従って、これらの二つの要因によってエッジが強調されることが理解できる。
【００８４】
画像がシャープに見えるか否かはこのＳ字カーブにおける中央部分の傾斜角度が影響を与えることも容易に理解できる。また、エッジの両側のアンダーシュートとオーバーシュートによって生じる高低差も同様に影響を与えるものといえる。
各補間処理には以上のような特性の違いがあり、ステップＳＴ２０８ではステップＳＴ２０６にて求めた色数に基づいて非自然画であると判断する場合にはステップＳＴ２１０におけるニアリスト法による補間処理を実行するし、自然画であればキュービック法による補間処理を実行することになる。
【００８５】
補間処理自体は任意の倍率で実行可能であるが、プリンタドライバ１２ｃにおける処理の高速化を図るため、整数倍の補間処理を受け付けるようにする。図２９は水平方向と垂直方向に２倍に補間する処理例を示している。予め、補間後の画像データについての変数領域を確保すると、整数倍の補間処理であれば元画像の画像データは整数倍した座標値に対応する画素の画像データとなる。図に示す例で言えば、旧座標値（０，０）は新座標値（０，０）に対応し、旧座標値（１，０）は新座標値（２，０）に対応し、旧座標値（０，１）は新座標値（０，２）に対応し、旧座標値（１，１）は新座標値（２，２）に対応するということである。従って、残りの座標値についてのみ上述した補間処理に対応して画像データを生成していく。この場合、画像データの幅方向を主走査方向とし、長さ方向を副走査方向として順に走査していくことも可能であるし、画像データがある四つの格子点に囲まれた各ブロック毎に内部の座標値の補間処理をしていって埋めていくことも可能である。
【００８６】
そして、新たな座標値について全て補間処理したときにステップＳＴ２１４にて補間画像データを次段の処理へ引き渡す。ただし、補間倍率によっては補間画像データのデータ量が極めて多大になることもあるし、そもそもプリンタドライバ１２ｃが利用可能なメモリ領域がさほど多くない場合もある。このような場合には一定のデータ量ごとに分けて出力するようにしても構わない。
上述したようにステップＳＴ１１６，ＳＴ１１８は補間処理がブロック単位の領域毎に補間処理を変更する手法に対応している。これを上述したプリンタドライバ１２ｃの具体的なソフトウェア処理に対応して説明すると、図２９に示すように整数倍して格子点の画像データを新たな座標値に移行せしめておき、四つの格子点に囲まれた各ブロック毎に特徴量を取得して補間処理を選択することになる。上述した例では利用色数を集計するものであるのでかかる特徴量を利用することはできないが、他の特徴量を取得して補間処理を変更していくことが可能である。
【００８７】
例えば、四つの格子点の差を求めると、自然画であれば当然にこの差が生じていることが多い。しかし、自然画に重ねて文字が組み込まれているような場合、文字が単一色であれば四つの格子点は一致し、差が生じない。むろん、差が生じない領域に内挿する画素は四つの格子点と同じデータとすればよいのであり、最も演算量の少ないニアリスト法で補間すればよい。
また、差が生じない場合に限る必要はない。空のように余り変化のない領域ではニアリストを使用しても画質の劣化は判断できないことが多いため、四つの格子点の差が小さいときにはニアリスト法で補間するようにしても良い。なお、以上の例では領域として最小の単位でブロックを形成しているが、より大きなブロックで補間処理を変更するようにしても構わない。
【００８８】
選択可能な補間処理はニアリスト法かキュービック法かのいずれか一方だけに限る必要もない。例えば、４倍に補間するような場合に、最初にキュービック法にて２倍に補間処理し、続いてニアリスト法にて２倍に補間処理することも有意義である。補間処理によって画素数が増加する前に演算量の多い補間処理を実行しておき、その後で演算量の少ない補間処理を実行することになるからである。
【００８９】
すなわち、補間結果に関連するものとして、このような場合には補間倍率が影響しているといえる。従って、プリンタドライバ１２ｃはオペレーティングシステム１２ａにおける基準解像度とカラープリンタ１７ｂの解像度との比較において倍率が判定できたときに補間処理を選択することも可能である。
このように、画像入力デバイスとしてスキャナ１１ａなどを有するとともに画像出力デバイスとしてカラープリンタ１７ｂなどを有するコンピュータシステム１０において、プリンタドライバ１２ｃはステップＳＴ２０２にて元画像データを取得し、ステップＳＴ２０４，ＳＴ２０６にて画像の利用色数を計数することにより、当該画像データの画像が自然画であるのか非自然画であるのかを判定するための特徴量を抽出し、ステップＳＴ２０８にて同特徴量に基づいて自然画か非自然画かを決定することによって非自然画であればステップＳＴ２１０のニアリスト法の補間処理を実行するし、自然画であればステップＳＴ２１２のキュービック法の補間処理を実行するようにしたため、画像の特徴に応じた補間処理が実施され、最適な補間結果を極めて容易に得ることができる。
【００９０】
そして、このようなプログラムを実行する前提として、コンピュータ１２には、ＣＰＵ１２ｅとＲＡＭ１２ｆとＲＯＭ１２ｇとＩ／Ｏ１２ｈなどが備えられている。同ＲＡＭ１２ｆは、画像をドットマトリクス状の画素で表現した画像データを記憶する画像メモリとして作用する。また、同ＲＡＭ１２ｆは、この画像メモリに記憶された画像データを対象として補間処理に関連する特徴量を取得するとともに、同特徴量に対応して複数の補間処理の中から最適な補間結果を得ることが可能な補間処理を選択し、同補間処理を上記ＣＰＵ１２ａに実行させて上記画像メモリに書き込ませる処理プログラムを記憶する。さらに、Ｉ／Ｏ１２ｈは上記画像データを入力および出力するインターフェイスとして作用する。
【００９１】
むろん、ＣＰＵ１２ｅはＲＡＭ１２ｆを一時的なワークエリアや設定記憶領域やプログラム領域として使用しながら、ＲＯＭ１２ｇに書き込まれた基本プログラムを適宜実行し、Ｉ／Ｏ１２ｈを介して未処理の画像データを入力し、処理後の画像データを出力する。
次に、上述した画像データの変化度合いを特徴量とした実施形態について説明する。
図３１は、本画像データ補間装置を表すブロック図である。
本画像データ補間装置はこのような画素単位での拡大処理を実施するものであり、画像データ取得手段Ｄ１は、このような画像データを取得し、画素補間手段Ｄ２はこの画像データにおける構成画素数を増やす補間処理を行う。ここで、画素補間手段Ｄ２は補間処理として画素の変化度合いに応じた複数の補間処理を実行可能となっており、画素変化度合評価手段Ｄ３が上記画像データに基づいて画素ごとの変化度合いを評価する。すると、補間処理選択手段Ｄ４はそのようにして評価された画素の変化度合いに対応して最適な補間結果を得ることが可能な補間処理を選択し、上記画素補間手段Ｄ２に実行させる。
【００９２】
なお、本実施形態においては、ディスプレイドライバ１２ｂやプリンタドライバ１２ｃは上述した画素補間手段Ｄ２はもとより、以下に述べるように画素変化度合評価手段Ｄ３や補間処理選択手段Ｄ４を実行し、解像度変換において最もバランスの良い補間結果を得ることができるようにしている。
図３２は、上述したプリンタドライバ１２ｃが実行する解像度変換に関連するソフトウェアフローを示している。
【００９３】
ステップＳＴ３０２は元画像データを取得する。ステップＳＴ３０４〜ＳＴ３０８は、読み込んだ画像データにおける画素の変化度合いを評価する処理である。
上述したような簡略化した演算で輝度を求めることとした上で、図３３と図３４は、輝度勾配を算出するためのエッジ検出フィルタを示している。画像データはドットマトリクス状の画素から構成されているので、注目画素を中心とする近隣の八画素との間で画像の変化度合いを評価すべきである。そういった意味では図３４（ａ）に示すように、注目画素に８倍の重み付けを与えつつ周囲の画素を均等に評価してそれを合算することでフィルタを掛けることが好ましい。しかしながら、経験的には必ずしも周囲の八画素を評価しなくても図３３（ａ）に示すように注目画素と周囲の四画素だけから評価可能である。むろん、四画素を利用するか八画素を利用するかでは演算量の差が大きく、このようにして評価対象を少なくすると処理時間を減らすことができる。
【００９４】
また、図３３（ｂ）と図３４（ｂ）には実際の画像データ（輝度）の例を示しており図３３（ｃ）と図３４（ｃ）には（ａ）に示すフィルタを（ｂ）に示す画像データの配置に適用した場合の演算例を示している。画像データは概ね左斜め上方側に画像データ「１００」のエリアがあり、右斜め下方側に画像データ「７０」と「６０」の領域があるような場合を示している。図３３の例では、中心画素の上下左右の四画素（画像データ「１００」、「１００」、「７０」、「７０」）についてそれぞれ「−１」の重みが付加され、中心画素（画像データ「１００」）には「４」の重みが付加されている。そして、この五画素について重み付加算を行なう。この重み付加算結果は「６０」であり、しきい値（ｔｈ）の「３２」を越えている。
【００９５】
一方、図３４の例では、中心画素を取り囲む八画素についてそれぞれ「−１」の重みが付加され、中心画素には「８」の重みが付加されている。この重み付加算結果は「１００」であり、しきい値（ｔｈ）の「６４」を越えている。
図３３や図３４に示すエッジ検出フィルタを利用した結果を各画素のエッジ量Ｅと呼ぶと、その分布は図３５に示すように正規分布的となることが予想され、画像の変化度合いが大きいエッジ部分であるか否かはしきい値ｔｈと比較することによって判定できる。図３３と図３４に示すエッジ検出フィルタはそれぞれしきい値としてｔｈ＝３２およびｔｈ＝６４というエッジ量のしきい値が妥当する。従って、エッジの画素か否かは次式から評価する。
（Ｅ＜−ｔｈ）　ｏｒ　（ｔｈ＞Ｅ）
この評価をドットマトリクス状の画素の全てに実施するのがステップＳＴ３０６の処理であり、各画素単位でエッジの画素のように画像の変化度合いが大きい画素であるか否かを評価する。
【００９６】
ところで、各画素単位で画像の変化度合いが大きいか否かを判定するとしても、補間処理は一定の領域毎に画素を生成する処理であるから、その領域単位で画像の変化度合いが大きいか否かを判定する必要がある。各領域ごとにこの変化度合いを判定するのは煩雑であるから、ステップＳＴ３０８であらかじめエッジ画素であるか否かを判定してフラグを設定する。この場合、図３６に示すように、エッジ画素を取り囲む全ての画素において画像の変化度合いが大きいものと判断する。より具体的には、各画素の変化度合いが図３７（ａ）に示すようになっているとするときに、しきい値が「３２」であれば、しきい値を越える画素はｘｙ座標で示すところの（０，０）（３，０）（４，０）（１，１）（２，１）であるとしても、エッジ画素の隣接画素にはフラグを設定することになる。すると、同図（ｂ）に示すようにｙ＝０，１の全画素と、（４，２）を除くｙ＝２の画素についてはフラグが設定されることになる。この結果、後の工程で各画素単位で注目ブロックを移動させていくときにフラグだけを参照して補間処理を適宜選択できるようになる。
【００９７】
むろん、本実施形態においては、これらのステップＳＴ３０４〜ＳＴ３０８の処理が画素変化度合評価ステップに相当する。むろん、これらがＣＰＵなどのハードウェアと有機一体的に結合したものと考えると画素変化度合評価手段Ｄ３を構成することになる。
以上のように設定したフラグに基づき、ステップＳＴ３１０以下ではループ処理で補間画素を生成していく。図３８は既存の画素に対して補間して生成される画素の配置を概略的に示している。既存の画素について仮に座標を（Ｘ，Ｙ）として表示し、補間で生成される画素の座標を＜Ｘ，Ｙ＞として表示している。同図の例では、約２．５×２．５倍の補間処理を行っている。
【００９８】
既存の四つの画素で囲まれる一つの領域をブロックと呼び、各ブロックごとに補間する画素の補間処理を選択する。ステップＳＴ３０８では各画素ごとに周囲の画素の変化度合も考慮してフラグを設定しているので、各ブロックでは四つの画素（０，０）（１，０）（０，１）（１，１）についていずれについてもフラグが設定されていれば変化度合の大きい場合の補間処理を選択することになり、どれか一つでもフラグが設定されていなければ変化度合の小さい場合の補間処理を選択することになる。ステップＳＴ３１０では、この条件に基づいて当該ブロック内部に適用する補間処理を判断し、変化度合いが小さい場合にはステップＳＴ３１２にてニアリスト法による補間処理で補間を実行するし、変化度合いが大きい場合にはステップＳＴ３１４にてキュービック法による補間処理で補間を実行する。また、一つのブロックを補間処理した後、ステップＳＴ３１６とステップＳＴ３１８にて処理対象となるブロックを移動させ、全てのブロックが終了すればステップＳＴ３２０にて補間された画像データを出力する。
【００９９】
なお、図中にはステップＳＴ３１８の終了後にステップＳＴ３１０に戻るような流れを実線で示しているが、破線で示すようにブロック毎にエッジ画素を集計する処理を繰り返すようにしても良い。
むろん、このような意味でステップＳＴ３１０の処理を中心としてステップＳＴ３１６，ＳＴ３１８の処理を含めて補間処理選択ステップに相当する。むろん、これらがＣＰＵなどのハードウェアと有機一体的に結合したものと考えると補間処理選択手段Ｄ４を構成することになる。なお、プリンタドライバ１２ｃの場合、解像度変換だけで印刷データが得られるわけではなく、色変換であるとか、ハーフトーン処理が必要になる。従って、ここで画像データを出力するというのは、次の段階へのデータの受け渡しを意味することになる。
【０１００】
本実施形態の場合は、四つの画素に囲まれる領域をブロックと呼んで補間処理を選択しているが、補間処理を変更する基準は演算能力や補間処理などに応じて適宜変更可能である。例えば、図３９に示すように、注目画素を中心とする領域を基準として補間処理する場合もある。このような場合は、かかる注目画素を矢印のように走査させて移動させながら補間処理を適宜実施していけばよい。
ここで、注目画素を移動させていきながら補間処理を選択する手法について説明する。上述した例ではブロック毎に変化度合いが大きいか否かを判定するにあたり、当該領域に含まれる全てのフラグが「１」となっている場合にだけ画像の変化度合いが大きい領域と判断している。しかしながら、必ずしもこのように全てのフラグが「１」になっている必要もないともいえる。例えば、図４０（ａ）に示すように４画素で囲まれる領域に補間処理で画素を生成するものとする。この場合、図３６でエッジ画素の隣接画素にフラグを立てる関係上、本来であれば上述したように４画素全てにフラグが立っている場合だけが変化度合いの大きい領域と判定することになりえる。しかしながら、このように判断するとブロックを１画素分だ横に移動させた場合には縦の辺が共通する関係で縦の二画素について毎回判断が重複するし、縦方向に移動させれば横の辺が共通する関係で横の二画素について毎回判断が重複する。このような重複状況は演算処理において無駄である。一方、同図（ｂ）に示すように領域の隣接状況を考慮すれば、１領域ごとに左上の１画素を代表させて関連づけることが可能であるし、少なくともエッジ画素に同視しうる画素の近辺で変化度合いが大きいと判断しても対して支障はないといえる。また、隣接画素同士に囲まれる領域というのは実際には極めて微少な領域であることを鑑みても十分であるといえる。そして、このようにして１画素に１領域を対応させれば、ブロックを移動させる際に注目画素を移動させていき、その注目画素のエッジ量だけで領域の変化度合いを判定することが可能となるし、判定に要する演算処理量も低減する。
【０１０１】
また、補間する画素の側でブロックを形成するようにすることも可能である。図４１はこの例を示しており、図中、□の格子点が補間する画素を示し、○の格子点が既存の画素を示している。いま、補間する画素について５×５のブロックを一つとし、その中に含まれる既存の画素のエッジ量に基づいて当該領域が画像の変化度合いの大きいものであるか否かを判断する。この場合、一つのブロックを決めて当該ブロックに含まれる既存の画素を抽出し、そのエッジ量の積算値を求め、当該ブロック内では同一の補間処理で画素を生成すればよい。
【０１０２】
むろん、以上の場合においてより大きな領域毎にブロックを設定して補間処理を選択しても良く、例えば、１０×１０画素毎をブロックとすることも可能である。また、ブロックを設定せずに補間する画素毎にそれを取り囲む既存の画素についてのエッジ量を判断して補間処理を選択することも可能である。図４１の例で言えば内側に配列される３×３の□の格子点は、いずれも○で示す四つの既存の格子点の中に含まれ、それぞれの□の格子点を生成する際にこれを取り囲む○で示す四つの既存の格子点についてのエッジ量に基づいて補間処理を選択するということである。むろん、演算処理上、このような処理の方が都合よい場合に実現すればよい。すなわち、先に補間処理するブロックを特定して補間処理を決めてからその内部に画素を補間するという手法であっても良いし、補間する画素毎にブロックの状況を判定して補間処理を選択しても良い。
【０１０３】
さらに、上述したフローではステップＳＴ３０８にて予めエッジ画素に隣接する画素にフラグを設定しておき、ブロック毎に同フラグを参照するようにしている。しかしながら、図３２にて破線で示すようにブロックを移動させるフローとすることも可能であり、この場合には敢えてフラグを設定する必要もなく、当該ブロックの周囲の画素のエッジ量を判断して補間処理を選択するようにすれば良い。
【０１０４】
上述したように別々の補間処理を備えているステップＳＴ３１２，ＳＴ３１４の処理は画素補間ステップに相当する。むろん、これらがＣＰＵなどのハードウェアと有機一体的に結合したものと考えると画素補間手段Ｄ２を構成することになる。ここで、それぞれの補間処理について詳述する。
一方、このような関係は図２９に示す補間関数においてｔ＝０〜１の区間において傾斜が急となりつつ、ｔ＝１〜２の区間において増加した重み分を打ち消すように負の側へ引き寄せるカーブとなっている場合に生じる。
【０１０５】
従って、シャープさを調整しようとする場合には、▲１▼補間関数においてシャープさの基準となる理想的な傾斜を決定し、▲２▼ｔ＝０〜１の区間において上記傾斜を発生させるカーブを決定し、▲３▼ｔ＝１〜２の区間においてこのカーブによって増える重み付けを相殺するように負の側に引き寄せつつ、オーバーシュートとアンダーシュートが生じやすいカーブを決定することによって実現できる。むろん、この後の作業では特定されるカーブとなるように多次演算関数のパラメータを決定するが、かかるパラメータの決定方法は極めて多様であるから、実質的な意味でＳ字カーブにおける中央部分の傾斜角度とアンダーシュート及びオーバーシュートを調整することに他ならない。
【０１０６】
各補間処理には以上のような特性の違いがあり、ステップＳＴ３１０にて画像の変化度合いが小さいと判断されたブロックでは、ステップＳＴ３１２にてニアリスト法の補間処理を実行するし、逆に変化度合いが大きいと判断されたブロックでは、キュービック法やハイブリッドバイキュービック法の補間処理を実行する。キュービック法で補間処理をする場合には演算時間が多大となってしまうものの、画像の変化度合いが小さいような部分ではニアリスト法に切り替えるため、全体としての処理時間は極めて低減する。特に、コンピュータグラフィックスのように同色で一定領域を塗りつぶしてあるような場合には一律にニアリスト法を実行しても全く問題ないので、処理時間は低減する。また、自然画であっても拡大したときにジャギーが目立ちやすい部分というのは面積比でいってもそれほど大きくないのが普通であるから、このように画像の変化度合いを逐次切り替えることによって画質を劣化させることなく処理量を低減させることができる。
【０１０７】
本実施形態においては、フラグによって二種類ある補間処理のいずれかを実行するようにしているが、画素の変化度合いに対して段階的に対応する複数の補間処理を実行するようにしても良い。また、図４２に示すように、二つの補間処理を重ねて実行することとしてその拡大倍率を画像の変化度合いに対応させるようにしても良い。例えば、補間倍率が５倍であるとして画像の変化度合いが小さめであればニアリスト法で５倍に補間処理するし、画像の変化度合いが大きめであればキュービック法で５倍の補間処理する。これらの場合は上述した実施形態と同様であるが、画像の変化度合いが中間的な値である場合にはキュービック法で２倍に補間処理し、残りの２．５倍をニアリスト法で補間処理する。このようにして二つの補間処理でありながら実質的には画像の変化度合いに応じた複数の補間処理を選択できることになる。
【０１０８】
なお、前述のように、キュービック法のような補間処理の演算量が大きいものについては、補間倍率を整数倍とする。
このように、画像入力デバイスを有するとともに画像出力デバイスを有するコンピュータシステム１０において、プリンタドライバ１２ｃはステップＳＴ３０２にて元画像データを入力した後、ステップＳＴ３０４〜１０８にて画像の変化度合いを検出してフラグを設定しておき、ステップＳＴ３１０にて同フラグを参照することにより、画像の変化度合いの小さいブロックではステップＳＴ３１２にてニアリスト法による補間処理を実行するし、画像の変化度合いの大きいブロックではステップＳＴ３１４にてキュービック法による補間処理を実行するようにしたため、画質を劣化させない範囲でできる限りニアリスト法を実行するように制御され、自動的に最適な補間処理を選択しつつ演算処理量を低減させる。
【０１０９】
以上説明したように、発明においては、画像をドットマトリクス状の画素で表現した画像データを取得する画像データ取得手段と、上記画像データに基づいて画素の変化度合いを評価する画素変化度合評価手段と、上記画像データにおける構成画素数を増やす補間処理を行うに複数の補間処理の中から選択して実行可能な画素補間手段と、上記画素変化度合評価手段によって評価された画素の変化度合に基づいてその変化度合いに対応して最適な補間結果を得ることが可能な補間処理を選択して上記画素補間手段に実行させる補間処理選択手段とを具備する構成としてある。
【０１１０】
このように構成した本発明においては、画像をドットマトリクス状の画素で表現した画像データの構成画素数を増やす補間処理を行うにあたり、画素補間手段は複数の補間処理の中からいずれかを選択して実行可能となっており、画像データ取得手段が対象となる画像データを取得すると、上記画素変化度合評価手段は同画像データに基づいて画素の変化度合いを評価する。そして、上記補間処理選択手段はこの画素変化度合評価手段によって評価された画素の変化度合に基づき、その変化度合いに対応して最適な補間結果を得ることが可能な補間処理を選択して上記画素補間手段に実行させる。
【０１１１】
すなわち、画素の変化度合は補間処理の具体的手法に密接に関連するので、同画素の変化度合を評価して積極的に補間処理を変更することにより、無駄のない補間処理を実現する。
以上説明したように本発明は、画像の変化度合いに応じて補間処理を変更することにより極めて簡易に最適な補間結果を得ることが可能な画像データ補間装置を提供することができる。
画素変化度合評価手段は、画素の変化度合を評価するものであり、評価の手法や結果は特に限定されるものではない。また、補間処理選択手段での同評価結果の利用態様に応じて相対的に変更可能なものである。例えば、具体的な変化度合いを数値として要するのであれば数値を出力すればよいし、単に変化度合いが大きいか否かといったものでよければ、それに合わせて出力すればよい。
【０１１２】
また、画素の変化自体をどのようにして把握するかも適宜変更可能である。その一例として、上記画素変化度合評価手段は、各画素の明るさのパラメータを求めるとともに周囲の画素のパラメータとの比較で上記変化度合いを算出する構成とすることもできる。
このように構成した場合には、画素の評価の基準として当該画素の明るさを利用するものとし、上記画素変化度合評価手段は各画素の明るさのパラメータを求め、当該画素とその周囲の画素とで同パラメータを比較し、比較結果を上記変化度合いとして算出する。
【０１１３】
むろん、これ以外にも画素の変化度合いを把握することは可能であるが、多要素のパラメータで表される画素を一律に把握するにあたって明るさのパラメータは比較的容易である。
このようにすれば、画像の変化度合いを明るさのパラメータに基づいて判断するため、比較的容易に同変化度合いを求めることができる。
一方、補間処理の処理内容にも画素の変化度合いが影響を及ぼす範囲が異なってくると言える。例えば、補間処理を実行するにあたって必要とする画素の数が一つであるものもあれば、複数の画素に基づいて補間処理するものもある。特に、後者の例であれば、一つでも変化度合いが大きい画素があると補間処理を変えるのか、あるいは一つでも変化度合が小さい画素があると補間処理を変えるのかといったことが問題となる。
【０１１４】
このような状況に対する一例として、上記画素変化度合評価手段は、各画素ごとに求めた上記変化度合いを周囲の画素の変化度合の評価についても利用する構成とすることもできる。
補間処理で要する画素が複数であり、そのうちの一つでも変化度合いが大きいときには、二つの態様が考えられる。すなわち、補間処理で対象とする範囲の残り画素については評価する必要が無くなるし、逆に既に評価した画素について変化度合いが小さかったとしても当該評価は不要となる。
【０１１５】
このような二方向の意味で、各画素ごとに求めた上記変化度合いを周囲の画素の変化度合の評価についても利用することになる。
このようにすれば、一の画素の変化度合いを周囲の画素においても利用することにより、演算量を低減できるし、補間処理の影響を受ける適当な範囲で共有することにより、最適な補間結果を得ることができる。
画素補間手段では、画素の変化度合いに関連する複数の補間処理を実行可能であればよく、補間処理自体としては各種の処理が可能である。その一例として、上記画素補間手段は、変化度合の小さい領域で適用して好適な補間処理として、補間処理前の最近隣画素の画像データを新たな構成画素の画像データに利用する補間処理を実行可能な構成とすることもできる。
【０１１６】
このように構成した場合には、一つの補間処理として補間処理前の最近隣画素の画像データを新たな構成画素の画像データに利用するが、同じ画素のデータが増えるとしても変化度合の小さい領域であれば何ら問題なく、処理量が少ない点で好適である。
このようにすれば、変化度合の小さい領域では画質に影響することなく処理量を減らすことができる。
また、他の一例として、上記画素補間手段は、変化度合の大きい領域で適用して好適な補間処理として補間する画素の画像データがなだらかに変化するように周囲の画素の画像データから演算処理で補間画素の画像データを算出する補間処理を実行可能な構成とすることもできる。
【０１１７】
このように構成した場合には、周囲の画素の画像データを利用して演算処理することにより、補間する画素の画像データはなだらかに変化する。このように、なだらかに変化させると、変化度合いの大きい画素の並びがあったとして、この間を補間したとしても段差が目立たない。従って、変化度合いの大きい画素の並びについてこの間を補間したとしても段差が目立たたず画質の劣化を防止することができる。
【０１１８】
補間する画素の画像データがなだらかに変化する演算手法は各種のものを採用可能であるが、その変化態様は画質に影響を与える。このため、ある意味では演算手法を変えることによって画質を調整可能となるともいえる。画質を調整可能な一例として、上記画素補間手段は、変化度合いの大きい画素間で補間画素の画像データを算出するにあたり、画像データの変化態様を略Ｓ字型としつつその傾斜を調整するとともに、両端部位では低い側にアンダーシュートを発生させつつ高い側にオーバーシュートを発生させて高低差を形成してその高低差を調整することにより、画像の変化度合いを最適なものとするように調整する構成とすることもできる。
【０１１９】
このように構成した場合には、補間する画素の画像データをなだらかに変化させるにあたり、変化度合いの大きい画素間で画像データの変化態様を略Ｓ字型とする。従って、なだらかには変化するもののその変化態様は単に直線的に結ぶ勾配よりは急峻とさせることができ、その傾斜を調整して画像の変化度合いを最適なものとすることが可能となる。また、両端部位で低い側にアンダーシュートを発生させつつ高い側にオーバーシュートを発生させると高低差は大きくなり、かつ、その高低差を調整することによっても見かけ上の画像の変化度合いを最適なものとすることが可能となる。このような演算処理の一例としては、多次演算処理の３次たたみ込み内挿法などを使用可能であるし、かかる調整を可能とする演算処理はこれに限られず、他の演算手法を採用することもできる。
【０１２０】
このようにすれば、Ｓ字カーブの傾斜と、アンダーシュートとオーバーシュートによる高低差とにより、画質の調整を比較的容易に実現できる。
画素の変化度合いが画像全体にわたって一定であることはないため、補間処理選択手段は、適宜、補間処理を選択して切り換えなければならない。そして、かかる切り換えの頻度も特に限定されるものでなく、各種の手法を採用可能である。その一例として、上記補間処理選択手段は、上記画素変化度合評価手段によって評価された画素の変化度合に基づいて画素単位で上記補間処理を選択して実行させる構成とすることもできる。
【０１２１】
このように構成した場合には、上記画素変化度合評価手段によって評価された画素の変化度合に基づき、上記補間処理選択手段が画素単位で上記補間処理を選択して実行させる。すなわち、変化度合いが画素単位で評価される以上、これに対応して補間処理も変更する。
従って、画素単位で補間処理を選択するのできめ細かに補間結果を向上させることができる。
また、他の一例として、上記補間処理選択手段は、上記画素変化度合評価手段によって評価された画素の変化度合に基づいて複数画素からなる所定の小領域毎に上記補間処理を選択して実行させる構成とすることもできる。このようにすれば、小領域毎に補間処理を選択するので処理を簡易化することができる。
【０１２２】
次に、補間処理を前提としつつシャープさを併せて修正する実施形態を説明する。
図４３は、このような画像データ補間装置を表すブロック図である。
元の画像が自然画であるとすると、画像によってはシャープさに欠けることがある。例えば、ピントの甘いような写真などが該当する。また、装置間の解像度を一致させるための拡大にとどまらず、画像自体を拡大して出力したいような場合には、シャープさの欠ける画像はさらにピントが甘くなりかねない。
本画像データ補間装置は画像データについて画素単位での拡大処理を実施する際にシャープさを調整するものであり、画像データ取得手段Ｅ１は、同画像データを取得し、画素補間手段Ｅ２はこの画像データにおける構成画素数を増やす補間処理を行う。ここで、画素補間手段Ｅ２は補間処理に付随して画像のシャープさを変化させることが可能となっており、シャープ度合評価手段Ｅ３が上記画像データに基づいて画像のシャープさを評価する。すると、補間処理制御手段Ｅ４はそのようにして評価されたシャープさが低ければこれを高めるような補間処理を実行するように上記画素補間手段Ｅ２を制御する。
【０１２３】
上述したように、オペレーティングシステム１２ａで管理する解像度とカラープリンタ１７ｂの解像度とが一致しない場合にプリンタドライバ１２ｃは解像度を一致させる処理を実行する。通常、カラープリンタ１７ｂの解像度はオペレーティングシステム１２ａが管理する解像度よりも細かいので、解像度を一致させるためには画素を増やすための補間処理が行われる。このようにアプリケーション１２ｄによって拡大処理する場合と、プリンタドライバ１２ｃによって解像度を一致させる場合に補間処理が行われるが、これらの補間処理で画像のシャープさに影響を与えることができる。
【０１２４】
画像のシャープさはそれぞれの隣接画素間での変化度合いの総合評価と言える。シャープさに欠ける画像というのは本来のエッジ部分でなだらかに画素が変化していることを意味し、シャープな画像では本来のエッジ部分で隣接画素間の変化度合いが急峻である。補間処理は既存の画素と画素の間に新たな画素を生成することになるので、新たな画素をどのような値とするかで画像のシャープさが変化するからである。
【０１２５】
この意味で、本発明の画像データ補間装置では、アプリケーション１２ｄやプリンタドライバ１２ｃが上述した画素補間手段Ｅ２はもとより、以下に述べるようにシャープ度合評価手段Ｅ３や補間処理制御手段Ｅ４を構成する。
図４４は、補間処理を実行する一例としてのプリンタドライバ１２ｃが実行する解像度変換に関連するソフトウェアフローを示している。
ステップＳＴ４０２は元画像データを取得する。ステップＳＴ４０４〜ＳＴ４０８は、読み込んだ画像データにおける各画素の変化度合いから画像のシャープさを評価する処理である。
【０１２６】
図３３や図３４に示すエッジ検出フィルタを利用した結果を各画素のエッジ量Ｅと呼ぶと、その分布は図３５に示すように正規分布的となることが予想される。このようにしてドットマトリクス状の画素の全てにおいて算出するのがステップＳＴ４０６の処理であり、エッジ量の絶対値をステップＳＴ４０８にて集計する。集計は単純な平均値であっても良いが、背景部分の面積比の影響を受けやすいとも言える。例えば、図４５では被写体たる人物像が大きく映って背景部分が少ないが、図４６では被写体たる人物像が小さく映って背景部分が多くなる。背景部分では画素の変化度合いが小さくなりがちであるから、背景部分の面積割合が大きい図４６に示すものでは図４５に示すものと比較して平均値が低くなりがちである。この意味で、或る一定のしきい値を設けておき、そのしきい値以上のものだけの平均を算出するようにしても良い。
【０１２７】
一方、この集計の段階では画像のシャープさを求めるのが主目的であるが、そもそも自然画のようなシャープさを要求される画像であるのか否かをこの集計結果から判断することも可能である。自然画の場合は単なる背景のような部分であっても色の明暗であるとか背景としての実物の形状に応じて同一の画素が並んでいるわけではないことから、エッジ量の絶対値の集計結果は図４７に示すようになり、エッジ量は大きめになりがちである。これに対してビジネスグラフのような画像では同色で一定領域を塗りつぶすことが多いので、エッジ量の絶対値の集計結果は図４８に示すようになり、エッジ量が低めになる。
【０１２８】
従って、集計結果の平均値（ａｖ）があるしきい値Ｔｈより低い場合にはシャープさを増すような処理が必要でない画像と言え、シャープさに影響を与えない補間処理を実行させるようにすればよい。
以上のようにしてステップＳＴ４０４〜ＳＴ４０８において画像を構成する各画素の変化度合いを集計し、当該画像がシャープな画像と言えるか否かの評価を実施したことになるため、これらのステップＳＴ４０４〜ＳＴ４０８の処理がシャープ度合評価手段Ｅ３を構成することになる。
【０１２９】
この評価結果に基づき、ステップＳＴ４１０では画像のシャープさの高低に応じた補間処理を選択する。本実施形態においては、シャープな画像に対してキュービック法による補間処理を実行し、シャープさに欠ける画像に対してハイブリッドバイキュービック法による補間処理を実行する。従って、この意味で当該ステップＳＴ４１０は補間処理制御手段Ｅ４を構成するし、別々の補間処理を備えているステップＳＴ４１２，ＳＴ４１４の処理は画素補間手段Ｅ２を構成することになる。ここで、それぞれの補間処理について詳述する。
【０１３０】
各補間処理には以上のような特性の違いがあり、ステップＳＴ４１０にて画像がシャープであると判断されればステップＳＴ４１２にてキュービック法の補間処理を実行するし、逆にシャープではないと判断されるとハイブリッドバイキュービック法の補間処理を実行する。ハイブリッドバイキュービック法で補間処理をする場合には補間するカーブが急峻となってシャープさを増すことができ、かかる補間処理が選択されるのは対象とする画像のシャープさを評価し、その評価結果に基づくものである。このため、操作者は特段の判断をしなくてもシャープでない画像をシャープにすることができる。
【０１３１】
本実施形態においては、二種類ある補間処理のいずれかを実行するようにしているが、画素の変化度合いに対して段階的に対応する複数の補間処理を実行するようにしても良い。図４９はシャープさの評価を４段階に分けて３次内挿法のパラメータを変化させた四つのキュービック法を実施する例を示している。図中「０」は通常のシャープさの画像に適用されるキュービック法のカーブを示しており、「＋１」のカーブはわずかにシャープさに欠ける画像に適用されるキュービック法を示しており、「＋２」のカーブはかなりシャープさが欠ける画像に適用されるキュービック法を示している。また、シャープすぎる画像については「−１」のカーブのキュービック法を適用する。むろん、これらはいずれもハイブリッドバイキュービック法の場合と同様にＳ字カーブにおける中央部分の傾斜角度とアンダーシュート及びオーバーシュートを調整して実現している。
【０１３２】
次に、この場合の手続のフローを図５０に示す。
画像のシャープさに基づいてステップＳＴ５１０ではこれらのパラメータを設定し、かかるパラメータを使用したキュービック法をステップＳＴ５１４にて実行する。また、このフローでは、画像のシャープさが部分的に異なることを考慮し、画像を小領域であるブロックに分割して各ブロック毎に最適な補間処理を実行する。すなわち、各ブロック毎にシャープさを評価して補間処理を選択するため、ステップＳＴ５０８にてブロック毎のエッジ量を集計し、ステップＳＴ５１６，ＳＴ５１８にてブロックを順次移動させながら補間処理を実行するようにしている。
【０１３３】
全ての画像データについて補間処理を終了したら、ステップＳＴ４２０やステップＳＴ５２０にて補間された画像データを出力する。なお、プリンタドライバ１２ｃの場合、解像度変換だけで印刷データが得られるわけではなく、色変換であるとか、ハーフトーン処理が必要になる。従って、ここで画像データを出力するというのは、次の段階へのデータの受け渡しを意味することになる。
このように、画像入力デバイスを有するとともに画像出力デバイスを有するコンピュータシステム１０において、プリンタドライバ１２ｃはステップＳＴ４０２にて元画像データを入力した後、ステップＳＴ４０４〜１０８にて画像のシャープさを評価して集計するとともに、ステップＳＴ４１０にて同集計結果に基づいてシャープさに欠ける画像であればステップＳＴ４１４にてシャープさを増す補間処理を実行するし、シャープな画像であればステップＳＴ４１２にて通常の補間処理を実行するようにしたため、操作者が別段にシャープさを増す画像処理を選択しなくても補間処理を経るだけでシャープな画像とすることができる。
【０１３４】
以上説明したように、本実施形態においては、画像をドットマトリクス状の画素で表現した画像データを取得する画像データ取得手段と、上記画像データに基づいて画像のシャープさを評価するシャープ度合評価手段と、上記画像データにおける構成画素数を増やす補間処理を行うにあたり画像のシャープさを変化させる補間処理を実行可能な画素補間手段と、上記シャープ度合評価手段によって評価された画像のシャープさが適当でなければシャープさを変化させて適当となるように上記画素補間手段に補間処理を実行させる補間処理制御手段とを具備する構成としてある。
【０１３５】
このように構成した場合には、画像をドットマトリクス状の画素で表現した画像データの構成画素数を増やす補間処理を行うにあたり、画素補間手段は画像のシャープさを変化させる補間処理を実行可能となっており、画像データ取得手段が対象となる画像データを取得すると、上記シャープ度合評価手段は同画像データに基づいて画像のシャープさを評価する。そして、上記補間処理制御手段はこのシャープ度合評価手段によって評価されたシャープさに基づき、画像のシャープさが適当でなければシャープさを変化させて適当となるように上記画素補間手段に補間処理を実行させる。
【０１３６】
すなわち、画像のシャープさが低い場合には補間処理によってシャープさを増し、また、シャープすぎる場合にはシャープさを低減させる。
このようにすれば、画像のシャープさに応じて補間処理でシャープさを調整するようにしているため、操作を煩雑にすることなく簡易に画質を向上させることが可能な画像データ補間装置を提供することができる。
補間を実行しつつ画像のシャープさを変化させる手法として画像データの変化態様を略Ｓ字型としつつその傾斜を調整するとともに、両端部位では低い側にアンダーシュートを発生させつつ高い側にオーバーシュートを発生させて高低差を形成してその高低差を調整することにより、画像の変化度合いを最適なものとするように調整して画像のシャープさを変化させることを示した。
【０１３７】
このようなＳ字カーブをとる一例として、上記画素補間手段は、３次たたみ込み内挿法におけるパラメータを調整して画像のシャープさを変化させる構成とすることもできる。
このように構成した場合には、補間処理として利用される３次たたみ込み内挿法のパラメータを調整することにより、元の画像での隣接する画素の間に補間される画素が３次関数を採用することによってＳ字を描き、なだらかでありながら急峻さも併せ持つことになる。そして、このＳ字の曲がり具合をパラメータで調整することによって急峻さが変化し、画像のシャープさが変化する。
【０１３８】
このようにすれば、多次演算処理として３次たたみ込み内挿法を利用することにより、Ｓ字カーブを調整して比較的容易にシャープさを調整することができる。
シャープさを変化させるにあたり、必ずしも一つの演算手法だけを採用する必要はなく、シャープさに影響を与える複数の補間処理を実行することも可能である。そのような一例として、上記画素補間手段は、画像のシャープさの変化度合いの異なる複数の補間処理を実行可能であるとともに、それぞれの補間倍率の割合を変化させて画像のシャープさを調整する構成とすることもできる。
【０１３９】
このように構成した場合には、複数の補間処理のそれぞれで画像のシャープさの変化度合いが異なり、必要な補間倍率を得るために複数の補間処理を実行する。従って、その補間倍率の分担割合を互いに変化させることにより、シャープさを調整可能となる。例えば、シャープさの変化度合いの低い補間処理とシャープさの変化度合いの高い補間処理とがある場合に両者の分担割合を変化させれば二つの変化度合の中間を選択可能となる。
【０１４０】
このようにすれば、複数の補間処理で分担する補間倍率を変えるだけであるので、パラメータの設定が簡易になる。
画像のシャープさは、必ずしも高ければよいわけではない。従って、シャープさを増す必要がない場合もある。この場合、操作者が判断することも可能であるが、かかる判断を同時に実現する構成とすることもできる。その一例として、上記補間処理制御手段は、上記画像のシャープさが所定のしきい値を越えていると評価されたときに上記画素補間手段にて画像のシャープさを変化させるように制御する構成とすることもできる。
【０１４１】
このように構成した場合には、上記補間処理制御手段が上記画像のシャープさと所定のしきい値とを比較し、画像のシャープさがこれを越えていると評価されたときに上記画素補間手段にて画像のシャープさを変化させるように制御する。しかしながら、画像のシャープさがしきい値を越えていないようであればあえて画像のシャープさを変化させるようには制御しない。例えば、自然画であるときの画像のシャープさと非自然画であるときの画像のシャープさとを比較すれば、前者のものの方が一般的にはシャープさが高いと言えるからである。むろん、画像のシャープさは自然画か非自然画かといった分類だけで決まるものでもないため、他の判断要素を加えることも可能である。例えば、画像の分類を取得し、その分類にたった上で上記しきい値を変化させればより柔軟な対応が可能となる。
【０１４２】
このようにすれば、ある一定の範囲まではシャープさを変化させないようにするため、シャープさを変化させることが不適当な画像まで自動的にシャープさを調整してしまうといった不便さがなくなる。
画面のシャープさは個々の画素を基準とすると、画像全体にわたって一定であるわけではないので、必ずしも一定の補間処理に限られるものでもない。このため、画素単位で画像のシャープさを評価するとともに、評価された画像のシャープさに基づいて画素単位で画像のシャープさを変化させるように制御する構成とすることもできる。また、所定の小領域毎に画像のシャープさを評価するとともに、評価された画像のシャープさに基づいて所定の小領域毎に画像のシャープさを変化させるように制御する構成としてある。
【０１４３】
次に、画像処理を選択できるようにした実施形態を説明する。
図５１は、このような画像データ補間装置を表すブロック図である。
本画像データ補間装置は画像データについて画素単位の画像処理を実施する際に拡大処理とシャープさの変更処理を補間処理として同時に行うものであり、画像データ取得手段Ｆ１にて同画像データを取得するともに、画像処理選択手段Ｆ２で実施する画像処理を選択する。同時処理判断手段Ｆ３は実行すべき画像処理が拡大処理とシャープさの変更処理とを同時に行うものであるか否かを判断するものであり、同時に行う必要がある場合には画素補間手段Ｆ４が当該画像データにおける構成画素数を増やす補間処理を行なうのに付随して画像のシャープさを変化させる。そして、拡大処理後の画像データは画像データ出力手段Ｆ５が出力する。
【０１４４】
かかるコンピュータシステム１０では、画像入力デバイスであるスキャナ１１ａなどで画像データを取得し、アプリケーション１２ｄで所定の画像処理を実行する。この画像処理には各種のものがあり、拡大縮小、シャープさの強弱、コントラストの強弱、色合いの修正といったものがあげられる。なお、上述したように、画像出力デバイスとしてのディスプレイ１７ａやカラープリンタ１７ｂに表示出力するためには解像度の一致が必要であり、特にプリンタの解像度に合わせる際には拡大処理としても実行する補間処理が行われる。そして、この補間処理はプリンタドライバ１２ｃで実行しても良いし、アプリケーション１２ｄで実行することもできる。
【０１４５】
ここで、アプリケーション１２ｄでは、画像処理として拡大処理とシャープさの変更処理を選択することが可能であり、また、プリンタの解像度に合わせる際に拡大処理するのに伴ってシャープさを変更することも可能である。従って、これらの場合に拡大処理とシャープさの調整処理を実行することになる。
この意味で、本発明の画像データ補間装置は、上述したコンピュータシステム１０におけるアプリケーション１２ｄとして実現されることになる。そして、アプリケーション１２ｄは上述した画素補間手段Ｆ４はもとより、以下に述べるように画像処理選択手段Ｆ２や同時処理判断手段Ｆ３を構成する。また、データの入出力が伴うので、ファイル入力やファイル出力あるいは印刷データの出力という意味で画像データ取得手段Ｆ１や画像データ出力手段Ｆ５を構成する。
【０１４６】
図５２は、アプリケーション１２ｄのソフトウェアフローの概略を示している。アプリケーション１２ｄは図５３に示すようなメニュー選択によって各種の画像処理を実行可能であり、必ずしも図５２に示すようなソフトウェアフローに限定されるものではないが、理解の便宜のために簡略化して表示している。
ステップＳＴ６０２では元画像データを取得する。アプリケーション１２ｄにおけるファイルメニューなどでスキャナ１１ａから画像を読み込む処理などが該当する。本実施形態においては後述する画像処理に使用する画像データを生成すればよいので、新規ファイル作成などを選択して画像ファイルを生成するような処理であっても同様に元画像データの取得といえる。オペレーティングシステム１２ａやハードウェアの構成を除いた元画像データの取得処理が画像データ取得に相当する。むろん、これらがＣＰＵなどのハードウェアと有機一体的に結合したものと考えると画像データ取得手段Ｆ１に該当する。
【０１４７】
ステップＳＴ６０４では、画像処理を選択し、ステップＳＴ６０６，ＳＴ６０８では、選択された画像処理を判断する。画像処理を選択するには、図５３に示すように画面上にてウィンドウ枠に表示されるメニューバーの中から、マウス１５ｂによって「画像」の文字部分をクリックすると、図５４に示すように実行可能な各種の画像処理が表示される。同図に示す例では、「拡大」処理と、「シャープネス」調整処理と、「コントラスト」調整処理と、「明るさ」調整処理とが実行可能となっている。それぞれの画像処理には「ボタン」スイッチを用意してあり、複数の画像処理を同時に選択可能となっている。また、「ボタン」スイッチで選択されるまでは各処理はグレイ表示されており、非表示状態が一別できるようになっている。
【０１４８】
所望の画像処理のボタンスイッチで選択状態とするとともに、それぞれのパラメータをセットして「ＯＫ」ボタンをクリックすると、選択を入力した処理として扱われ、ステップＳＴ６０６，ＳＴ６０８にて選択された処理を判断する。この例では、ステップＳＴ６０６にて拡大処理が選択されたか否かが判断され、拡大処理が選択されていなければステップＳＴ６１０にてそれぞれに対応する画像処理を実施する。一方、拡大処理が選択されている場合には、ステップＳＴ６０８にてシャープネス調整処理が選択されているか否かを判断する。この結果、拡大処理とシャープネス調整処理が共に選択されているならば、ステップＳＴ６１２へと進み、拡大処理は選択されているもののシャープネス調整処理は選択されていないのであればステップＳＴ６１４へと進む。なお、ステップ１１２，ＳＴ６１４の処理については後述する。
【０１４９】
このソフトウェアフローでは、理解しやすくステップＳＴ６０６，ＳＴ６０８の二つの分岐処理で行っているが、実際にはケース処理で多数の分岐を選択するようにしてもよい。本実施形態においては、ステップＳＴ６０４にて画像処理を選択するので画像処理選択ステップに相当するし、ステップＳＴ６０６，ＳＴ６０８にて拡大処理とシャープネス調整処理が共に選択されているか否かを判断することになるので同時処理判断ステップに相当する。むろん、これらがＣＰＵなどのハードウェアと有機一体的に結合したものと考えると画像処理選択手段Ｆ２や同時処理判断手段Ｆ３に該当する。
【０１５０】
ここで、画像処理選択手段Ｆ２や同時処理判断手段Ｆ３の変形例について説明する。
図５５〜図５７はその一例を示しており、アプリケーション１２ｄが図５４に示すように明示的なシャープネス調整処理を備えていないものとし、拡大処理は選択可能となっているとする。この例では、ステップＳＴ７０４にて画像処理の選択として図５６に示すような拡大処理のパラメータ入力ウィンドウを表示する。ここでは拡大率を％で選択入力可能となっており、所望の倍率にセットして「ＯＫ」ボタンをクリックすると、次の段階へと進行する。通常であれば、拡大率を入力することによってその倍率に応じた補間処理を実行すればよいが、この例ではステップＳＴ７０６にて拡大処理が選択されていると判断すると、ステップＳＴ７０８にて図５７に示すようなシャープネス調整の問い合わせ用ウィンドウを表示する。このウィンドウには「ＮＯ」、「Ｌｏｗ」、「Ｈｉｇｈ」の三つの選択肢が用意され、それぞれに択一的に選択可能なボタンが割り当てられている。デフォルトは「ＮＯ」であり、操作者は必要に応じて「Ｌｏｗ」や「Ｈｉｇｈ」を選択できるようになっている。これらはシャープネス強調についての問い合わせであり、「ＮＯ」は強調せず、「Ｌｏｗ」はやや強調し、「Ｈｉｇｈ」は強調するという意味である。
【０１５１】
そして、選択された結果に基づいて、補間処理であるステップＳＴ７１２，ＳＴ７１４，ＳＴ７１６のいずれかを実行する。拡大処理の際には、後述するように補間処理を選択すればシャープネスを変化させることが可能であるので、拡大処理を選択した場合に自動的に問合せるようにしている。この例では、ステップＳＴ７０４，ＳＴ７０８が画像処理選択ステップに相当するし、ステップＳＴ７１０が同時処理判断ステップに相当する。
【０１５２】
また、図５８〜図６０には、直接には画像処理を選択しているようには見えないが、実際には内部的に拡大処理するような場合を示しており、より具体的には印刷処理の場合である。図示しないファイルメニューの中から印刷を選択すると、ステップＳＴ８０４にて図５９に示すような印刷メニューを表示する。この印刷メニューの中でも各種のパラメータを設定可能であるが、その一つとして「印刷解像度」の選択ボックスがある。アプリケーション１２ｄが内部的に扱っている解像度とは関係なく、印刷時にどの解像度で印刷実行するかによって解像度の一致作業が必要となる。カラープリンタ１７ｂ２の解像度が７２０ｄｐｉであるとして、印刷解像度を７２０ｄｐｉとして印刷する場合には画像データの１ドットが印刷時の１ドットと対応するので解像度変換は不要である。しかしながら、３００ｄｐｉで印刷する際には印刷データとの対応を一致させなければならず、この意味での解像度変換が必要となってくる。
【０１５３】
このため、ステップＳＴ８０８にて印刷解像度ボックスのパラメータとオペレーティングシステム１２ａが管理しているカラープリンタ１７ｂ２の解像度とを比較して解像度変換を実施する必要があるか否かを判断し、解像度変換が必要な場合にはステップＳＴ８１０にて図６０に示すようなシャープネスの問い合わせ用ウィンドウを表示する。この例では図５４の場合と同様にシャープネスの調整程度を％で選択して入力するようにしており、ステップＳＴ８１２では入力されたパラメータに従って処理を選択する。すなわち、調整処理が要求されない場合にはステップＳＴ８１４へと進み、調整処理が要求される場合にはステップＳＴ８１６へ進むようにしている。この後、解像度が一致した状態でステップＳＴ８１８にて印刷処理を実行する。
【０１５４】
この例では、ステップＳＴ８０４，ＳＴ８１０が画像処理選択ステップに相当するし、ステップＳＴ８１２が同時処理判断手段ステップに相当する。むろん、これらの画像処理選択手段Ｆ２と同時処理判断手段Ｆ３についてはこれら以外の手法で実現することが可能であることはいうまでもない。
以上のような判断を経て補間処理が実行されることになる。具体的には、拡大処理だけが選択されてシャープさの強調処理が選択されない場合にはニアリスト法による補間処理を実行するし（ステップＳＴ６１４，ＳＴ７１２，ＳＴ８１４）、拡大処理とシャープさの強調処理が選択されたときにはハイブリッドバイキュービック法による補間処理（ステップＳＴ６１２，ＳＴ８１６）とキュービック法による補間処理（ステップＳＴ７１４）を実行する。従って、後者のステップＳＴ６１２，ＳＴ８１６，ＳＴ７１４が画素補間手段Ｆ４を構成することになる。
【０１５５】
本実施形態のように、拡大処理の中でシャープさを調整できるようにすることは、単に拡大処理とシャープさ調整処理とを別個に行うよりも優れている点がある。例えば、シャープさを強調してから拡大する処理を行うとするとしても、拡大処理がニアリスト法のようなものであればジャギーが目立ってしまい、シャープな感じを維持できないことがある。また、ニアリスト法で拡大してからシャープさを強調するとした場合は、ジャギーが目立った状態でシャープにすることになるので、画質が向上するとも言えない。これに対して、一体の拡大処理の中でシャープさをも合わせて行うようにすれば、このような弊害は生じにくい。
【０１５６】
画像データについて補間処理や他の画像処理を終了したら、ステップＳＴ６６にて画像データを出力する。ここでいう画像データを出力するというのは広義の意味を含んでおり、カラープリンタ１７ｂ２に出力するとか、ハードディスク１３ｂに書き込むといった処理に限られず、データとしては保持しておきながらディスプレイ１７ｂ１に表示させ、次なる画像処理に備えるというものであっても構わない。むろん、本実施形態においては、このステップＳＴ６１６が画像データ出力手段Ｆ５を構成する。
【０１５７】
このように、画像入力デバイスや画像出力デバイスなどを有するコンピュータシステム１０において、アプリケーション１２ｄは各種の画像処理を実行可能となっており、ステップＳＴ６０４にて実行すべき画像処理を選択させたとき、拡大処理とシャープさの変更処理とが同時に指定された場合には、ステップＳＴ６０８の判断を経てステップＳＴ６１２にてシャープさを増す補間処理を実行し、拡大処理だけが選択された場合にはステップＳＴ６１４にてシャープさに影響を与えない通常の補間処理を実行するようにしたため、拡大処理とシャープさの変更処理とを個別に実行するための余分な時間がかからないし、両者が一体的に実行されるので、確実にシャープさを調整することができる。
【０１５８】
以上説明したように、発明においては、画像をドットマトリクス状の画素で表現した画像データを取得する画像データ取得手段と、上記画像データに対して個々の画素における画像データを変更することによって各種の画像処理を実行するべく実行可能な画像処理を表示して選択を入力する画像処理選択手段と、この画像処理選択手段によって画像の拡大処理と画像のシャープさの変更処理とが共に選択されたか否かを判断する同時処理判断手段と、この同時処理判断手段によって画像の拡大処理と画像のシャープさの変更処理とが共に選択されたと判断されたときに、上記画像データにおける構成画素数を増やして画像を拡大するにあたり、この補間する画像データの変化度合いを調整することにより選択された画像のシャープさとなるように補間処理を実行可能な画素補間手段と、生成された画像データを出力する画像データ出力手段とを具備する構成としてある。
【０１５９】
このような構成とした本発明においては、画像データ取得手段にて画像をドットマトリクス状の画素で表現した画像データを取得したら、この画像データに対して個々の画素における画像データを変更することによって各種の画像処理を実行するべく画像処理選択手段にて実行可能な画像処理を表示して選択を入力させる。ここで、同時処理判断手段はこの画像処理選択手段によって画像の拡大処理と画像のシャープさの変更処理とが共に選択されたか否かを判断し、両処理が共に選択されたと判断されれば、画素補間手段は上記画像データにおける構成画素数を増やして画像を拡大するに際し、この補間する画像データの変化度合いを調整することにより選択された画像のシャープさとなるように補間処理を実行する。そして、画像データ出力手段は生成された画像データを出力する。
【０１６０】
すなわち、画像の拡大とシャープさを変更する処理を同時に実行させる必要が生じれば、補間処理で生成する画像データを調整することによって拡大しつつシャープさを変化させる。
以上説明したように本発明は、拡大処理で必要となる補間処理によって画像のシャープさを変更するようにしたため、拡大処理とシャープさの変更処理とを個別に行う必要が無く、処理時間を短くすることが可能な画像データ補間装置を提供することができる。
【０１６１】
また、同時に行われるので、拡大処理によってはその後のシャープさを変更する際に良好な結果を得られなくなったりすることもないし、むろん、シャープさを変更してから拡大処理することによってシャープさの変更処理が無駄になってしまうこともない。
画像処理選択手段は実行可能な画像処理を表示して選択を入力するものであり、同時処理判断手段は画像の拡大処理と画像のシャープさの変更処理とが共に選択されたか否かを判断するものである。これらは、結果的に両処理を同時に実行する必要があるか否かを判断するものであればよく、選択の態様などは適宜変更可能である。
【０１６２】
その一例として、上記画像処理選択手段は、拡大処理の選択とシャープさ変更の選択とを個別に選択可能であり、上記同時処理判断手段は、この画像処理選択手段にて拡大処理の選択とシャープさ変更の選択とが同時に選択されたか否かを判断する構成とすることもできる。
このように構成した場合には、拡大処理の選択とシャープさ変更の選択とを個別に選択可能であるので、拡大処理だけであるとか、シャープさ変更だけが選択されることもある。そして、同時処理判断手段はそのような状況を前提として拡大処理の選択とシャープさ変更の選択とが同時に選択されたか否かを判断する。
【０１６３】
このようにすれば、拡大処理とシャープさの変更処理が個別に選択できる場合に好適である。
また、他の一例として、上記画像処理選択手段は、拡大処理を選択可能であるとともに、上記同時処理判断手段は、上記画像処理選択手段にて拡大処理が選択されたときにシャープさの変更度合を選択させる構成とすることもできる。
このように構成した場合には、上記画像処理選択手段で拡大処理だけが選択可能となっており、シャープさを変更する選択までは入力しない。しかしながら、この画像処理選択手段にて拡大処理が選択されたときには、上記同時処理判断手段が独自に判断してシャープさの変更度合を選択させる。むろん、シャープさを変更しないことを選択することも可能であって、その場合には拡大処理だけを実行することになるし、逆にシャープさを変化させることを選択した場合には拡大処理とシャープさを変更する処理とを同時に選択されたものと判断する。
【０１６４】
従って、表面上は拡大処理しか選択できないような場合においても合わせてシャープさを変更することが可能となる。
さらに、上述した例では拡大処理を明示的に選択するようになっているが、拡大処理自体は明示的なものに限られる必要もない。その一例として、上記同時処理判断手段は、上記画像処理選択手段が画像処理に伴って解像度の変換処理を行うときに、シャープさの変更度合を選択させる構成とすることもできる。
【０１６５】
このように構成した場合には、上記画像処理選択手段において選択された画像処理に伴って付随的に解像度の変換処理を行う必要性が生じることがあり、この場合に上記同時処理判断手段は、シャープさの変更度合を選択させる。そして、シャープさを変化させることを選択した場合には拡大処理とシャープさを変更する処理とを同時に選択されたものと判断する。
このようにすれば、付随的に拡大処理するような場合にもシャープさを合わせて変更することができる。
【０１６６】
なお、これらの場合における表示と入力の操作はＧＵＩのもとでの画面表示やマウス操作などで実現しても良いし、ハードウェア的なスイッチで実施することも可能であるなど、適宜変更可能である。
【図面の簡単な説明】
【図１】本発明の一実施形態にかかる画像データ補間装置のブロック図である。
【図２】同画像データ補間装置の具体的ハードウェアのブロック図である。
【図３】本発明の画像データ補間装置の他の適用例を示す概略図である。
【図４】本発明の画像データ補間装置の他の適用例を示す概略図である。
【図５】本発明の画像データ補間装置の他の適用例を示す概略図である。
【図６】本発明の画像データ補間装置の他の適用例を示す概略図である。
【図７】本発明の画像データ補間装置における汎用的なフローチャートである。
【図８】本発明の画像データ補間装置におけるより具体的なフローチャートである。
【図９】元画像の大きさを示す図である。
【図１０】サンプリング周期を示す図である。
【図１１】サンプリング画素数を示す図である。
【図１２】元画像とサンプリングされる画素の関係を示す図である。
【図１３】非自然画についての輝度のヒストグラムを示す図である。
【図１４】自然画についての輝度のヒストグラムを示す図である。
【図１５】画像の変化度合いを直交座標の各成分値で表す場合の説明図である。
【図１６】画像の変化度合いを縦軸方向と横軸方向の隣接画素における差分値で求める場合の説明図である。
【図１７】隣接する全画素間で画像の変化度合いを求める場合の説明図である。
【図１８】しきい値を変化させる領域を示す図である。
【図１９】プリンタドライバからオペレーティングシステムに問い合わせを行う状況を示す図である。
【図２０】ニアリスト法の概念図である。
【図２１】ニアリスト法で各格子点のデータが移行される状況を示す図である。
【図２２】ニアリスト法の補間前の状況を示す概略図である。
【図２３】ニアリスト法の補間後の状況を示す概略図である。
【図２４】キュービック法の概念図である。
【図２５】キュービック法の具体的適用時におけるデータの変化状況を示す図である。
【図２６】キュービック法の具体的適用例を示す図である。
【図２７】ハイブリッドバイキュービック法の具体的適用例を示す図である。
【図２８】バイリニア法の概念図である。
【図２９】補間関数の変化状況を示す図である。
【図３０】整数倍の補間処理を示す概略図である。
【図３１】本発明の一実施形態にかかる画像データ補間装置の概略ブロック図である。
【図３２】本発明の画像データ補間装置におけるフローチャートである。
【図３３】エッジ検出フィルタの一例を示す図である。
【図３４】エッジ検出フィルタの他の一例を示す図である。
【図３５】エッジ量の分布としきい値との関係を示す図である。
【図３６】注目画素とフラグの設定判断で対象とする画素の関係を示す図である。
【図３７】エッジ量とフラグの設定状況を示す図である。
【図３８】既存の画素で形成されるブロックと補間する画素との関係を示す図である。
【図３９】注目画素を基準としてブロックを形成する場合を示す図である。
【図４０】フラグの状況と領域の対応を示す図である。
【図４１】補間する画素でブロックを形成する場合を示す図である。
【図４２】画像の変化度合いに応じて複数の補間処理の補間倍率を分配する関係を示す図である。
【図４３】本発明の一実施形態にかかる画像データ補間装置の概略ブロック図である。
【図４４】本発明の画像データ補間装置におけるフローチャートである。
【図４５】背景部分の小さい画像を示す図である。
【図４６】背景部分の大きい画像を示す図である。
【図４７】自然画のエッジ量の集計結果を示す図である。
【図４８】ビジネスグラフのエッジ量の集計結果を示す図である。
【図４９】補間関数の変化状況を示す図である。
【図５０】補間関数を選択するフローを示す図である。
【図５１】本発明の一実施形態にかかる画像データ補間装置の概略ブロック図である。
【図５２】本発明の画像データ補間装置におけるフローチャートである。
【図５３】画像処理を実行するためのメニューを表示を示す図である。
【図５４】画像処理を選択する画面を示す図である。
【図５５】画像データ補間装置の変形例を示すフローチャートである。
【図５６】拡大処理を指示する画面を示す図である。
【図５７】シャープネスを指示する画面を示す図である。
【図５８】印刷処理のフローチャートである。
【図５９】印刷時のパラメータを指示する画面を示す図である。
【図６０】シャープネスを指示する画面を示す図である。
【符号の説明】
１０…コンピュータシステム
１１ａ…スキャナ
１１ａ２…スキャナ
１１ｂ…デジタルスチルカメラ
１１ｂ１…デジタルスチルカメラ
１１ｂ２…デジタルスチルカメラ
１１ｃ…ビデオカメラ
１２…コンピュータ本体
１２ａ…オペレーティングシステム
１２ｂ…ディスプレイドライバ
１２ｂ…ドライバ
１２ｃ…プリンタドライバ
１２ｄ…アプリケーション
１３ａ…フロッピーディスクドライブ
１３ｂ…ハードディスク
１３ｃ…ＣＤ−ＲＯＭドライブ
１４ａ…モデム
１４ａ２…モデム
１５ａ…キーボード
１５ｂ…マウス
１７ａ…ディスプレイ
１７ａ１…ディスプレイ
１７ｂ…カラープリンタ
１７ｂ１…カラープリンタ
１７ｂ２…カラープリンタ
１８ａ…カラーファクシミリ装置
１８ｂ…カラーコピー装置[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image data interpolation device for interpolating image data composed of pixels in a dot matrix, an image data interpolation method, and a medium on which an image data interpolation program is recorded.
[0002]
[Prior art]
When an image is handled by a computer or the like, the image is represented by dot matrix pixels, and each pixel is represented by a gradation value. For example, photographs and computer graphics are often displayed on a computer screen with 640 dots in the horizontal direction and 480 dots in the vertical direction.
[0003]
On the other hand, the performance of color printers has been remarkably improved, and the dot density is extremely high, such as 720 dots / inch (dpi). In this case, when an image of 640 × 480 dots is printed in correspondence with each dot, the size becomes extremely small. In this case, since the tone value is different and the meaning of the resolution itself is different, it is necessary to interpolate between dots and convert the data into printing data.
[0004]
Conventionally, as a method of interpolating dots in such a case, a nearest neighbor interpolation method (hereinafter, referred to as a nearest neighbor method) or a cubic convolution interpolation method (a cubic convolution interpolation: hereinafter, A method such as the cubic method) is known. Japanese Patent Application Laid-Open No. 6-225140 discloses a technique in which a dot pattern is prepared in advance so as to have an enlarged form in which the edge is smoothed when smoothing the edge when the dots are interpolated. Have been.
[0005]
[Problems to be solved by the invention]
The above-described conventional interpolation technique has the following problems.
Various methods such as the near list method and the cubic method have their advantages and disadvantages, but it is difficult for the user to select it, and if it is fixed to one or the other, it will be difficult to The quality of the interpolation result decreases.
In the invention disclosed in JP-A-6-225140, the interpolation magnification must be fixed because a pattern is prepared in advance, and the number of patterns becomes enormous assuming a color image. It is difficult to prepare in advance.
[0006]
The present invention has been made in view of the above problems, and has as its object to provide an image data interpolation apparatus, an image data interpolation method, and a medium on which an image data interpolation program is recorded, which can efficiently perform interpolation including a color image. I do.
[0007]
[Means for Solving the Problems]
The image data interpolating apparatus provided by the present invention includes: an image data acquiring unit that acquires image data representing an image by dot matrix pixels; and a plurality of interpolation processes for performing an interpolation process for increasing the number of constituent pixels in the image data. A pixel interpolating unit that can be selected and executed from among the processes, a feature amount obtaining unit that obtains a feature amount related to the interpolation process on the image data, and a feature amount obtained by the feature amount obtaining unit. And an interpolation processing selection means for selecting an interpolation processing capable of obtaining an optimum interpolation result and causing the pixel interpolation means to execute the selected interpolation processing.
[0008]
In the present invention configured as described above, in performing the interpolation processing for increasing the number of constituent pixels of the image data in which the image is represented by the pixels in the dot matrix, the pixel interpolation unit selects one of the plurality of interpolation processing. When the image data acquiring unit acquires the target image data, the feature amount acquiring unit acquires the feature amount related to the above-described interpolation processing on the image data, and the interpolation processing selecting unit An interpolation process capable of obtaining an optimal interpolation result corresponding to the feature amount acquired by the feature amount acquisition unit is selected and executed by the pixel interpolation unit. That is, it acquires the feature amount of the image by itself and selects an optimal interpolation process.
[0009]
As described above, the method of selecting the optimum interpolation processing according to the characteristics of the image is not necessarily limited to a substantial device, and it can be easily understood that the method also functions as the method. For this reason, the image data interpolation method provided by the present invention, when performing an interpolation process to increase the number of constituent pixels for image data representing an image by dot matrix pixels, a step of acquiring the image data, A step of acquiring a feature amount related to the above-mentioned interpolation process for data, and a step of selecting an interpolation process capable of obtaining an optimum interpolation result from a plurality of interpolation processes corresponding to the acquired feature amount And a step of executing an interpolation process selected from the plurality of interpolation processes.
[0010]
In other words, there is no difference that the present invention is not necessarily limited to a substantial device and is effective as a method.
Such an image data interpolation device may exist alone or may be used in a state of being incorporated in a certain device. The idea of the invention is not limited to this, but includes various modes. Therefore, it can be changed as appropriate, such as software or hardware.
In the case where the software of the image data interpolating apparatus is realized as an example of realizing the idea of the present invention, the present invention naturally exists on a recording medium on which such software is recorded, and it cannot be said that the present invention is used.
[0011]
As an example, a medium that stores an image data interpolation program for causing a computer to execute an interpolation process provided by the present invention performs an interpolation process to increase the number of constituent pixels for image data in which an image is expressed by dot matrix pixels. Acquiring the image data, acquiring a feature amount related to the interpolation process on the image data, and selecting an optimal interpolation from among a plurality of interpolation processes corresponding to the acquired feature amount. The computer is configured to execute a step of selecting an interpolation process capable of obtaining a result and a step of executing the interpolation process selected from the plurality of interpolation processes.
[0012]
Of course, the recording medium may be a magnetic recording medium or a magneto-optical recording medium, and any recording medium to be developed in the future can be considered in the same manner. Further, the duplication stages of the primary duplicated product, the secondary duplicated product and the like are equivalent without any question. In addition, the present invention is not limited to the case where the present invention is used even when the supply is performed using a communication line.
Further, even when a part is implemented by software and a part is implemented by hardware, the concept of the present invention is not completely different, and a part is stored on a recording medium and appropriately It may be in a form that can be read.
When the present invention is implemented by software, the present invention is naturally realized not only as a medium in which the program is recorded, but also as the program itself, and the program itself is also included in the present invention.
[0013]
Here, the image data represents the image by dot matrix pixels, and may be any data as long as each pixel is represented by data, and may be a color image or a monochrome image. Further, the gradation value may be a two-gradation value or a multi-gradation value.
The image data obtaining means is for obtaining such image data, and may be any as long as it holds the target image data when the pixel interpolating means performs the interpolation processing for increasing the number of constituent pixels. Therefore, the acquisition method is not particularly limited, and various methods can be adopted. For example, the information may be obtained from an external device via an interface, or may be an image capturing device provided with an image capturing unit. Alternatively, a computer graphic application may be executed to perform input from a mouse or a keyboard.
[0014]
The pixel interpolation means only needs to be able to select a plurality of interpolation processings by various methods, and it is sufficient that the pixel interpolation means has options having substantially different interpolation processing results. Therefore, not all options need to be independent interpolation techniques.
As an example, the pixel interpolation means may be configured to include an interpolation process in which a plurality of interpolation processes are performed in an overlapping manner as options of the interpolation process.
In the case of such a configuration, the pixel interpolation means executes a plurality of interpolation processes as one of the options of the interpolation process. That is, after first interpolating pixels in a certain interpolation process, the pixels are superimposed and interpolated in another interpolation process. As a result, even if there are two types of independent interpolation processing, it can be said that there are three types of options including the processing of overlapping them.
[0015]
Therefore, since a plurality of interpolation processes can be executed in an overlapping manner as a corresponding example of the interpolation process, variations of the interpolation processes can be increased.
Further, the interpolation process to be executed does not need to be limited to a constant one for all pixels. As an example, the feature amount obtaining unit obtains a feature amount for each partial region of the image, and the interpolation processing selecting unit performs the interpolation in the pixel interpolation unit based on the feature amount obtained for each of the regions. It is also possible to adopt a configuration in which a process is selected and executed.
[0016]
In the case of such a configuration, the feature amount acquiring unit acquires the feature amount for each partial region of the image. Depending on the merits and demerits of the respective interpolation processes, some are preferably applied uniformly, while others are not. When it is not necessary to use uniform characteristics, it is preferable to utilize each feature. For this reason, the interpolation processing selection means selects and executes the interpolation processing in the pixel interpolation means based on the characteristic amount acquired for each partial area.
[0017]
For example, even if a simple interpolation process is employed in a place where there is no change in the image, the deterioration of the image quality is difficult to understand, but in a place where the change is severe, the deterioration of the image quality is easy to understand if the interpolation method is simple. Therefore, the interpolation of the pixels increases the number of pixels between the existing pixels, so that it is possible to appropriately change the interpolation processing for the increased pixels.
Accordingly, it is not necessary to perform uniform interpolation processing on the entire image, and it is possible to obtain an optimal result in a comprehensive sense.
[0018]
The reason why the interpolation processing is selected is that the interpolation result differs depending on the interpolation processing, and there are various feature amounts serving as keys for selecting the interpolation method. As an example, the pixel interpolating unit can execute an optimal interpolation process for each of a natural image and a non-natural image, and the feature amount acquiring unit determines that the image represented by the image data is a non-natural image The interpolation processing selecting means obtains a feature value for determining whether or not there is, and when the image is determined to be a natural image based on the feature value, the pixel interpolation means performs optimal interpolation processing for the natural image. It is also possible to adopt a configuration in which the pixel interpolation means executes the most suitable interpolation processing for a non-natural image when the image is determined to be a non-natural image.
[0019]
In the case of such a configuration, assuming that the pixel interpolating means can execute optimal interpolation processing for each of a natural image and a non-natural image, it is assumed that the image represented by the image data is a natural image or a non-natural image. The feature amount obtaining unit obtains a feature amount capable of determining whether the image is a natural image, and the interpolation processing selecting unit includes the pixel interpolation unit when the image is determined to be a natural image based on the feature amount. Then, when it is determined that the image is a non-natural image, the pixel interpolating means executes the optimum interpolation process for the non-natural image.
[0020]
Therefore, it is possible to select an optimal interpolation process according to a natural image and a non-natural image that are familiar processing targets.
A natural image means something like a so-called real image, and a non-natural image means computer graphics represented by a business graph. Characteristically, it can be considered that a natural image uses multiple colors, and a non-natural image uses a small number of colors. Needless to say, it may correspond to a feature found in addition to the above, but the pixel interpolation means can execute optimal interpolation processing for each of a natural image and a non-natural image having such a characteristic difference. I have.
[0021]
More specifically, the pixel interpolating means executes an interpolation process by a nearest neighbor interpolation method on a non-natural image and executes an interpolation process by a third-order convolution interpolation method on a natural image. can do.
In this way, in the case of a non-natural image, an advantage of high-speed processing by nearest neighbor interpolation can be obtained, and in the case of a natural image, an advantage of maintaining sharpness by cubic convolution interpolation can be obtained.
There is often a slight difference in the amount of expression per pixel between non-natural images and natural images.The nearest neighbor interpolation method is simple and can be processed at high speed. This can lead to unnatural feelings. On the other hand, the third-order convolution interpolation method can maintain sharpness, but the processing is complicated, the calculation load required for the interpolation processing is large, and the processing time is long.
[0022]
Assuming such advantages and disadvantages, interpolation processing by the nearest neighbor interpolation method is performed on the non-natural image, so that the processing speed can be expected to be high, and the cubic convolution interpolation method can be expected on the natural image. , The sharpness is secured.
The feature amount acquiring means acquires a feature amount of the image data, and the feature amount itself has a property related to the interpolation processing. As described above, as an example, it can be obtained by determining whether the image is a natural image or a non-natural image, but there are various acquisition methods.
[0023]
As an example, the feature amount obtaining unit may be configured to acquire a feature amount for determining the type of an image represented by the image data by totalizing data of each pixel in the obtained image data based on a predetermined reference. it can.
In the case of such a configuration, the feature amount acquiring means totals the data of each pixel on a predetermined basis. That is, a feature value for determining the type of the content of the image is acquired from the image data itself. Of course, various statistical methods and the like can be adopted as the counting method, but a histogram and the like are one example.
[0024]
With this configuration, the type of image is determined by summing up the image data, so that even when there is no other information, it is possible to reliably select the optimal interpolation processing.
Examples of data to be totaled include the number of colors used. As described above, in computer graphics, considering that a person who draws specifies colors, not many colors can be used. In particular, such a tendency appears strongly in business graphs and the like. On the other hand, if it is a natural image, even a single color image can be multicolored by adjusting the number of light rays, and the number of colors used becomes extremely large. Therefore, if the number of colors used for image data is totaled, it can be determined whether the image is a natural image or a non-natural image.
[0025]
In this case, the number of colors used does not need to be strictly counted. For example, a histogram of the luminance of each pixel may be totaled, and the number of colors used may be determined from the frequency of use of each luminance. Although it is difficult to find the relationship between the luminance histogram and the number of colors used for a natural image, it can be easily understood by assuming a business graph or the like. Something like a business graph uses a small number of colors, and often uses only a few colors when considering a filled graph. In this case, the luminance histogram appears as a spectrum. Since the luminance may be the same for different colors, the frequency of use of the luminance does not always match the number of colors used, but it is easy to understand whether the number of colors is large or small. For example, if the image is a natural image, the brightness used surely increases because of the presence of shadows and the like, and the histogram should appear as a continuous curve rather than a spectrum.
[0026]
Also, in this case, it is apparent from the same point that the luminance itself does not have to be accurate. For example, the exact luminance of RGB image data is not usually obtained by calculation alone. However, in television broadcasting and the like, luminance is obtained from RGB by simple weighted addition. That is, such a simple method can be used.
Whether the histogram has a spectral shape or a curved shape is also derived from the aggregation result, and it can be determined as a feature amount by determining whether the image is a natural image or a non-natural image based on the result. . Further, in addition to the viewpoint of whether or not the image is a natural image, it is also possible to acquire the characteristic amount as to whether or not the image is bright. That is, if there is a suitable interpolation process for a bright image and a suitable interpolation process for a dark image, these can be selected.
[0027]
It is also possible to sum up the degree of change in the data of each pixel. When a picture is drawn with computer graphics, the degree of change is only the user's drawing.However, in the case of a natural image of a landscape, the amount of information is large at each stage, which is reflected in the magnitude of the degree of change in pixels. Come. Therefore, it is possible to determine the type of image from the degree of change between pixels.
On the other hand, since the purpose of the feature amount obtaining means is to determine the image, it can be said that the image data need only be aggregated to an extent necessary for the determination. In this sense, the feature amount acquiring unit may be configured to extract the data of some pixels constituting the image data and totalize the data.
[0028]
In the case of such a configuration, the gradation values are extracted for only some of the pixels constituting the image data and totalized.
Here, various techniques can be adopted for the extraction. It is also possible to simply select the pixels at random and sum them up, or to find a specific object and sum them up. In addition, it is also possible to predict the tendency of the tally result while tallying. If it is an example of determining whether the image is a natural image or a non-natural image, it is possible to determine that the image is a natural image at that point if it is found that the used color gently includes its surroundings. If it is found that the number of colors is extremely large, the determination may be made at that time.
[0029]
In this case, the processing is performed without summing up all the image data, so that the processing speed can be increased.
Further, assuming a computer or the like, image data is processed as a file. In many cases, the type of image can be determined from the file format. Therefore, it is also possible to adopt a configuration in which the characteristic amount is obtained based on the format of the image data.
For example, a natural image is often compressed by the JPEG method. Therefore, if the format of the image data is JPEG, it can be determined that the image is a natural image. In addition, a business graph often has an extension of a file indicating an application that outputs the business graph. Therefore, such an extension can be determined to be one type, and the type of image can be determined.
[0030]
With this configuration, the type of the image is determined based on the format of the image data, so that the processing amount can be reduced as compared with the case where the image data is analyzed.
On the other hand, the feature amount related to the interpolation processing does not necessarily have to be a kind such as an image type. Therefore, the feature amount obtaining means may be configured to obtain the interpolation magnification to be executed and use the obtained interpolation magnification as a feature amount.
The interpolation magnification also relates to the interpolation result. For example, the image may not be rough while the interpolation magnification is small, but the image may be rough when the interpolation magnification is increased. In this case, in the latter case, it can be said that the interpolation process must be complicated. For this reason, the feature amount obtaining means obtains the interpolation magnification.
[0031]
Needless to say, in this case, the threshold value of the interpolation magnification or the like changes according to the interpolation process to be selected, and it is also possible to change stepwise when shifting from one interpolation process to another interpolation process. is there.
This makes it possible to select an optimal interpolation process based on factors other than the type of image data.
[0032]
【The invention's effect】
As described above, according to the first, twelfth, and twenty-second aspects of the present invention, the feature amount of the image is acquired by itself and the most suitable interpolation processing is selected, and the interpolation according to the feature of the image is performed. Since the processing is performed, it is possible to provide an image data interpolation device, an image data interpolation method, and a medium on which an image data interpolation program is recorded, which can extremely easily obtain an optimal interpolation result.
[0033]
Furthermore, according to the inventions of claims 2, 13 and 23, it is possible to classify the image data into a natural image and a non-natural image as a simple classification, and to execute an optimal interpolation process.
According to the inventions of claims 3, 14, and 24, an interpolation process is performed on a non-natural image by the nearest neighbor interpolation method which is most suitable for its nature, and on a natural image, It is possible to execute an interpolation process by a tertiary convolution interpolation method which is most preferable in its nature.
[0034]
Further, according to the inventions of claims 4, 15 and 25, it is possible to obtain an optimum interpolation result by selecting the interpolation processing based on the sharpness which is the degree of change of the pixel.
Furthermore, according to the fifth, sixteenth, and twenty-sixth aspects of the present invention, it is possible to execute the interpolation processing with extremely simple processing in an area where the degree of change of the pixel is small.
Furthermore, according to the inventions of claims 6, 17 and 27, it is possible to improve image quality by gently changing image data in an area where the degree of change is large.
[0035]
Furthermore, according to the inventions of claims 7, 18 and 28, it is possible to increase the number of interpolation processes that can be substantially used by performing a plurality of interpolation processes in a superimposed manner and to perform finer processes. Become.
Further, according to the invention of claims 8, 19 and 29, it is possible to select and execute the optimal interpolation processing for each area of the image.
Furthermore, according to the ninth, twentieth and thirty aspects of the present invention, it is possible to use one interpolation process as a plurality of pixel interpolation processes only by adjusting the operation parameters.
[0036]
Further, according to the tenth, twenty-first, and thirty-first aspects, when both the image enlargement processing and the image sharpness change processing are selected, the processing is extremely efficiently performed.
Further, according to the eleventh aspect of the present invention, it is possible to provide a computer for image data interpolation processing having the same effect.
[0037]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a main configuration of an image data interpolation device according to the present invention.
Assuming digital processing, an image is represented by dot matrix pixels, and image data is composed of a group of data representing each pixel. In a system that performs processing on a pixel-by-pixel basis, image scaling is performed on a pixel-by-pixel basis. The present image data interpolating apparatus performs such enlargement processing in pixel units. The image data acquiring unit C1 acquires such image data, and the pixel interpolating unit C2 determines the number of constituent pixels in the image data. Is performed. Here, the pixel interpolation unit C2 can execute a plurality of interpolation processes as the interpolation process. When the feature amount acquisition unit C3 acquires a feature amount related to the interpolation process on the image data, the interpolation process selection unit C4 selects an interpolation process capable of obtaining an optimum interpolation result in accordance with the characteristic amount, and causes the pixel interpolation means C2 to execute the interpolation process.
[0038]
In the present embodiment, the computer system 10 is employed as an example of hardware for realizing such an image data interpolation device.
FIG. 2 shows the computer system 10 in a block diagram.
The computer system 10 includes a scanner 11a, a digital still camera 11b, and a video camera 11c as image input devices, and is connected to a computer main body 12. Each input device is capable of generating image data representing an image by dot matrix pixels and outputting the image data to the computer main unit 12. Here, the image data is displayed in 256 gradations in three primary colors of RGB. , About 16.7 million colors can be expressed.
[0039]
The computer main body 12 is connected to a floppy disk drive 13a, a hard disk 13b, and a CD-ROM drive 13c as external auxiliary storage devices. The hard disk 13b stores main programs related to the system. Necessary programs and the like can be appropriately read from a CD-ROM or the like.
Also, a modem 14a is connected as a communication device for connecting the computer main body 12 to an external network or the like. The modem 14a is connected to the external network via the same public communication line, and software and data can be downloaded and introduced. Has become. In this example, the modem 14a accesses the outside through a telephone line. However, a configuration in which a network is accessed through a LAN adapter is also possible. In addition, a keyboard 15a and a mouse 15b for operating the computer main body 12 are also connected.
[0040]
Further, a display 17a and a color printer 17b are provided as image output devices. The display 17a has a display area of 800 pixels in the horizontal direction and 600 pixels in the vertical direction, and can display the above-mentioned 16.7 million colors for each pixel. Of course, this resolution is merely an example, and can be changed as appropriate, such as 640 × 480 pixels or 1024 × 768 pixels.
[0041]
The color printer 17b is an ink jet printer, and is capable of printing an image with dots on printing paper as a recording medium using four color inks of CMYK. The image density can be printed at a high density of 360 × 360 DPI or 720 × 720 DPI, but the gradation table has a two-gradation expression such as whether or not to apply color ink.
On the other hand, a predetermined program is executed in the computer main body 12 in order to display or output an image output device while inputting an image using such an image input device. Of these, an operating system (OS) 12a is operating as a basic program, and the operating system 12a performs a print output to a display driver (DSP DRV) 12b for displaying on a display 17a and a color printer 17b. A printer driver (PRT DRV) 12c to be installed is incorporated. These drivers 12b and 12c depend on the model of the display 17a and the color printer 17b, and can be additionally changed to the operating system 12a according to each model. In addition, additional functions beyond standard processing can be realized depending on the model. That is, it is possible to realize various additional processes within an allowable range while maintaining a common processing system on the standard system of the operating system 12a.
[0042]
Of course, as a premise of executing such a program, the computer main body 12 includes a CPU 12e, a RAM 12f, a ROM 12g, an I / O 12h, and the like. While being used as a storage area or a program area, the basic program written in the ROM 12g is appropriately executed to control external devices and internal devices connected via the I / O 12h.
[0043]
Here, the application 12d is executed on the operating system 12a as a basic program. The processing contents of the application 12d are various, monitor the operation of the keyboard 15a and the mouse 15b as operation devices, and when operated, appropriately control various external devices to execute corresponding arithmetic processing, Further, the processing result is displayed on the display 17a or output to the color printer 17b.
[0044]
In the computer system 10, image data is acquired by a scanner 11a or the like as an image input device, and after performing predetermined image processing by an application 12d, the image data can be displayed on a display 17a or a color printer 17b as an image output device. It is possible. In this case, focusing on the correspondence between pixels, if the pixel density of the color printer 17b and the pixel density of the scanner 11a match, the size of the scanned original image matches the size of the printed image. , The size of the image will be different. In the case of the scanner 11a, there are many which approximate the pixel density of the color printer 17b. However, the pixel density of the color printer 17b whose pixel density is improved for higher image quality is higher than that of a general image input device. It is often higher than the pixel density. In particular, when compared with the display density of the display 17a, the density is higher in each stage, and if the display on the display 17a is made to correspond to each pixel and printed, an extremely small image may result.
[0045]
For this reason, resolution conversion is performed in order to eliminate the difference in the actual pixel density of each device while determining the reference pixel density in the operating system 12a. For example, assuming that the resolution of the display 17a is 72 DPI and the operating system 12a is based on 360 DPI, the display driver 12b performs resolution conversion between the two. In the same situation, if the resolution of the color printer 17b is 720 DPI, the printer driver 12c performs the resolution conversion.
[0046]
Since the resolution conversion corresponds to a process of increasing the number of constituent pixels in image data, it corresponds to an interpolation process, and the display driver 12b and the printer driver 12c perform the interpolation process as one of the functions. Here, the display driver 12b and the printer driver 12c execute not only the above-described pixel interpolating unit C2 but also the feature amount acquiring unit C3 and the interpolation processing selecting unit C4 as described below so that the image quality is not deteriorated by the resolution conversion. ing.
[0047]
The display driver 12b and the printer driver 12c are stored in the hard disk 13b, and are read and operated by the computer main body 12 at the time of startup. At the time of introduction, it is recorded on a medium such as a CD-ROM or a floppy disk and installed. Therefore, these media constitute a medium on which the image data interpolation program is recorded.
In the present embodiment, the image data interpolation device is realized as the computer system 10, but such a computer system is not necessarily required, and any system that requires interpolation processing for similar image data may be used. . For example, as shown in FIG. 3, a digital still camera 11b1 incorporates an image data interpolating device for performing interpolation processing, and uses the interpolated image data to display on a display 17a1 or print on a color printer 17b1. Is also good. As shown in FIG. 4, in a color printer 17b2 that inputs and prints image data without going through a computer system, image data input via a scanner 11a2, a digital still camera 11b2, a modem 14a2, or the like is automatically processed. It is also possible to configure so as to perform a resolution process after performing the resolution conversion.
[0048]
In addition, the present invention can be naturally applied to various devices that handle image data, such as a color facsimile device 18a as shown in FIG. 5 and a color copying device 18b as shown in FIG.
7 and 8 show a software flow relating to the resolution conversion executed by the printer driver 12c described above. Here, the former shows a general-purpose flow, and the latter shows a specific flow of the present embodiment.
[0049]
A step ST102 acquires original image data. If the application 12d reads an image from the scanner 11a, performs predetermined image processing, and then performs print processing, print data of a predetermined resolution is acquired by the printer driver 12c via the operating system 12a. . Of course, the image may be read by the scanner 11a. This processing is an image data acquisition step when viewed as software, but various steps executed by the computer including the image data acquisition step are understood as not directly including the operating system 12a itself or hardware. can do. On the other hand, if it is considered that it is organically integrated with hardware such as a CPU, it corresponds to the image data acquisition means C1.
[0050]
Step ST104 is a process of extracting a feature amount in the read image data. Details of the feature amount extraction processing will be described later. In step ST106, an interpolation process optimal for the image data is selected based on the obtained feature amount, and a flag indicating the interpolation process is set. Then, in step ST108, the respective interpolation processes 1 to N in steps ST110, ST112, and ST114 are executed with reference to the flag. Accordingly, each of the interpolation processes 1 to N shown in steps ST110, ST112, and ST114 specifically corresponds to an image interpolation step, and steps ST106 and ST108 select an interpolation process based on a feature amount, and thus correspond to an interpolation process selection step. I do. Of course, if these are considered to be organically integrated with hardware such as a CPU, they constitute the image interpolation means C2 and the interpolation processing selection means C4.
[0051]
Steps ST116 and ST118 show processing for moving blocks until the interpolation processing is completed for all blocks. The interpolation process does not necessarily have to be a uniform process over the entire image, and it is possible to change the interpolation process for each region in block units.
Therefore, when the optimal interpolation processing is to be performed for each block, when the interpolation processing in each block is completed, the processing is again executed from the processing of extracting the feature amount of the next block in step ST104. On the other hand, when processing is performed uniformly over the entire image data, the processing from step ST108 is repeated.
[0052]
Then, when all the blocks are completed, the image data interpolated in step ST120 is output. In the case of the printer driver 12c, print data cannot be obtained only by resolution conversion, but requires color conversion or halftone processing. Therefore, outputting image data here means transferring data to the next stage.
Next, more specific processing for the above-described general-purpose flow will be described. In the present embodiment, it is determined whether the original image is a computer graphic (non-natural image) or a photograph (natural image), and an interpolation process is selected based on the determination result. In step ST202, original image data is input as in step ST102.
[0053]
Several methods are available for determining the type of the original image, but in the present embodiment, the aggregation process is performed using the image data.
More specifically, the number of colors used in the image data is obtained. If the number is large, the image is determined to be a natural image. In the case of a photograph, even if an object of one color is reflected, the width is made from a bright part to a dark part due to the adjustment and shading of light rays, and the number of colors increases. Because of these characteristics, it is possible to determine whether the image is a natural image or a non-natural image by looking at the number of colors. However, it is not efficient in the processing by the program to count how many colors among the 16.7 million colors are actually used. Also, even natural pictures often use only a part of them, making it difficult to distinguish them from non-natural pictures.
[0054]
For this reason, in the present embodiment, the brightness of each pixel is obtained, and a histogram of the number of pixels is totaled in a range in which the brightness can be obtained, and the tendency of the number of colors used is determined. Naturally, there are a plurality of colors having the same luminance among the 16.7 million colors. However, if attention is paid only to the comparison with the non-natural image, it is compared whether the color or the luminance is large or small. Is possible. Furthermore, if it is considered that the non-natural image uses at most about 64 colors, it is considered that the determination can be sufficiently performed even if the range in which the luminance can be taken is 256 gradations.
[0055]
On the other hand, since the totalization of the luminances is only for determining the general tendency of the image data as described above, it is not always necessary to perform the totalization for all the pixels. That is, a thinning process is performed to select a pixel to be counted.
As shown in FIG. 9, in the case of a bitmap image, a two-dimensional dot matrix including predetermined dots in the vertical direction and predetermined dots in the horizontal direction is established. It is necessary to check the brightness for. However, it need not be accurate here. Therefore, it is possible to perform the thinning to the extent that the error falls within a certain error range. According to the statistical error, the error with respect to the number of samples N can be approximately expressed as 1 / (2 ** (1/2)). Here, ** represents a power. Therefore, N = 10000 in order to perform processing with an error of about 1%.
[0056]
Here, the bitmap screen shown in FIG. 9 has the number of pixels of (width) × (height), and the sampling cycle ratio is
ratio = min (width, height) / A + 1
And Here, min (width, height) is the smaller of width and height, and A is a constant. Further, the sampling period ratio here indicates how many pixels are sampled, and the pixels marked with a circle in FIG. 10 show the case where the sampling period ratio = 2. In other words, one pixel is sampled every two pixels in the vertical and horizontal directions, and sampling is performed every other pixel. The number of sampling pixels in one line when A = 200 is as shown in FIG.
[0057]
As can be seen from the figure, except for the case where the sampling period ratio = 1 when no sampling is performed, when the width is 200 pixels or more, at least the number of samples becomes 100 pixels or more. Therefore, when the number of pixels is 200 or more in the vertical direction and the horizontal direction, (100 pixels) × (100 pixels) = (10000 pixels) is secured, and the error can be reduced to 1% or less.
Here, the reason why min (width, height) is used as a reference is as follows. For example, assuming that width >> height as in the bitmap image shown in FIG. 12A, if the sampling period ratio is determined by the longer width, as shown in FIG. In addition, it may happen that only two lines, the upper and lower lines, are extracted in the vertical direction. However, if min (width, height) is used to determine the sampling period ratio based on the smaller one, it is possible to perform the thinning including the middle part even in the smaller vertical direction as shown in FIG. Will be able to
[0058]
Note that, in this example, thinning is performed at an accurate sampling cycle for pixels in the vertical direction and the horizontal direction. This is suitable when processing is performed while thinning out sequentially input pixels. However, when all pixels have been input, pixels may be selected by designating coordinates randomly in the vertical direction and the horizontal direction. In this way, if the minimum required number of pixels, such as 10,000 pixels, is determined, the process of random extraction until 10,000 pixels is repeated, and the extraction may be stopped when the number of pixels reaches 10,000.
[0059]
If the pixel data of the selected pixel has luminance as its component element, the distribution can be obtained using the luminance value. However, even in the case of image data whose luminance value is not a direct component value, the image data has a component value representing luminance indirectly. Therefore, a luminance value can be obtained by performing a conversion from a color space in which the luminance value is not a direct component value to a color space in which the luminance value is a direct component value.
[0060]
The color conversion between different color spaces is not uniquely determined by the conversion formula, but the mutual relationship is calculated for the color spaces having the respective component values as coordinates, and the color conversion storing the correspondence is stored. It is necessary to convert sequentially by referring to the table. Then, strictly speaking, it is necessary to have a color conversion table of 16.7 million elements. As a result of considering efficient use of storage resources, instead of preparing correspondences for all coordinate values, usually prepare correspondences for appropriate discrete grid points and use interpolation together I am trying to do it. However, since such an interpolation operation can be performed through several multiplications and additions, the amount of operation processing becomes enormous.
[0061]
That is, if a full-size color conversion table is used, the processing amount is reduced, but the table size becomes an unrealistic problem. If the table size is set to a realistic size, the calculation processing amount becomes impractical. Often.
In view of such a situation, in the present embodiment, as used in a television or the like, the following conversion formula for obtaining luminance from the three primary colors of RGB is employed. That is, for the luminance yp at the point P, from the RGB component values (Rp, Gp, Bp),
yp = 0.30Rp + 0.59Gp + 0.11Bp
And In this way, a luminance value can be obtained only by three multiplications and two additions.
[0062]
In the present embodiment, as a result of targeting the RGB color space, such a conversion formula is adopted. However, since each component value indicates the color brightness in the background, each component There is a property that when the value is viewed alone, it corresponds linearly to the luminance. So, more broadly speaking, you can simply
yp = (Rp + Gp + Bp) / 3
It is not impossible to simplify it.
[0063]
In step ST204, the luminance of the pixels subjected to the thinning processing as described above is converted into a histogram. After counting, in step ST206, the number of colors is counted. If the number of colors is small, the distribution of luminance is sparse, and a non-natural image such as a business graph appears as a line spectrum as shown in FIG. 13, and a natural image such as a photograph as shown in FIG. It is expected to be a gentle curve.
For this reason, in step ST206, the number of luminance values whose distribution number is not "0" among the 256 gradation luminances is counted, and in step ST208, it is determined that the image is not a natural image when the number is less than "64". If the number of colors is “64” or more, the image is determined to be a natural image. On the other hand, it is also possible to determine whether the distribution is in the form of a linear spectrum or not by the adjacency ratio of luminance values whose distribution number is not “0”. That is, it is determined whether or not the number of distributions is a luminance value other than “0” and the number of distributions is adjacent luminance values. If at least one of the two adjacent luminance values is adjacent, nothing is performed, and if both are not adjacent, counting is performed. As a result, the ratio of the number of non-zero luminance values to the count value is calculated. Judge it. For example, if the number of luminance values other than “0” is “80” and the number of non-adjacent luminance values is “80”, it is understood that the luminance values are distributed in a linear spectrum. Of course, the number of colors used corresponds to the feature amount.
[0064]
It should be noted that the method of acquiring the feature amount is not limited to these, and other methods can be realized.
First, in thinning out the pixels to be totaled, the present invention is not limited to the thinning-out at uniform intervals as described above. For example, it is also possible to find an original object portion in an image and to sum up feature amounts of the pixels.
Based on the empirical fact that the image of such an object is sharper than other parts, it is determined that the sharp part is a pixel of the object. If the image data is composed of pixels in a dot matrix, the difference between adjacent pixels at the edge of the image is large. This difference is a luminance gradient, which is called an edge degree, and the edge degree at each pixel is determined. When considering the XY orthogonal coordinates as shown in FIG. 15, the vector of the degree of change of the image can be calculated by obtaining the X-axis direction component and the Y-axis direction component, respectively. In a digital image composed of dot matrix pixels, pixels are adjacent to each other in the vertical and horizontal directions as shown in FIG. 16, and the difference value fx in the X direction and the difference value fy in the Y direction are
fx = f (x + 1, y) -f (x, y)
fy = f (x, y + 1) -f (x, y)
Is represented as Therefore, the magnitude | g (x, y) |
| G (x, y) | = (fx ** 2 + fy ** 2) ** (1/2)
Is represented as Of course, the edge degree is represented by | g (x, y) |. Note that the pixels are originally arranged in a grid shape vertically and horizontally as shown in FIG. 17, and there are eight adjacent pixels when focusing on the central pixel. Therefore, similarly, the difference between the image data of each of the adjacent pixels may be represented by a vector, and the sum of the vectors may be determined as the degree of change of the image.
[0065]
Since the edge degree is obtained for each pixel as described above, basically, a pixel having a larger edge degree than a certain threshold value may be determined as a pixel of an object. However, considering empirical facts, objects are often located in the center of the composition. This fact supports that the pixel of the object is more easily extracted by adopting a mechanism in which many pixels are extracted from the central part.
[0066]
Therefore, as shown in FIG. 18, the threshold values Th1, Th2, and Th3 to be compared for each portion in the image can be made different. Of course, in this example,
Th1 <Th2 <Th3
The threshold value is lower at a portion closer to the center, and the object is determined to be an object even if the edge degree is relatively low.
Obviously, the image data of the pixels determined to be the object is totaled in this way, and the feature amount corresponding to the interpolation processing is obtained.
[0067]
On the other hand, the feature amount does not necessarily need to be obtained by totaling the image data. What is necessary is just to associate whether or not the interpolation result is good depending on the interpolation process.
Whether or not the image of the image data is a natural image can also be determined from the format of the image file serving as the printing source. FIG. 19 shows a situation where the printer driver 12c uses a system function prepared in the operating system 12a. When the printer driver 12c uses a function for inquiring a file name, the operating system 12a replies with a corresponding file name. In this case, if it is “XXXX.XLS”, it is known from the extension that the business graph is present, and it can be determined that the image is a non-natural image. If the file is “XXXX.JPG”, it can be recognized from the extension as a compressed file of a photographic image, and it can be determined that the file is a natural image.
Of course, it is possible to determine whether the file structure is a draw-based file structure or a bitmap-based file structure from the information contained in the beginning of the data file, not from the extension. It will be possible to obtain an indication of whether or not. In other words, the feature amount is configured as a standard that enables the content of such an image to be inferred.
[0068]
As described above, when it is determined whether the original image data input in step ST202 is a natural image or a non-natural image, an appropriate interpolation process is executed according to each.
Here, each method of the interpolation processing executed in the present embodiment will be described.
As an interpolation process suitable for non-natural images such as computer graphics, a near-list interpolation process can be executed in step ST210. In the near-list method, as shown in FIG. 20, the distance between the four surrounding grid points Pij, Pi + 1j, Pij + 1, Pi + 1j + 1 and the point Puv to be interpolated is obtained, and the data of the closest grid point is transferred as it is. If this is represented by a general formula,
Puv = Pij
Here, i = [u + 0.5] and j = [v + 0.5]. [] Indicates that a Gaussian symbol takes an integer part.
[0069]
FIG. 21 shows a situation where the number of pixels is vertically and horizontally tripled by the near-list method. It is assumed that there are four corner pixels (□ 前 ●●) before the interpolation, and the data of the closest pixel among these pixels is directly transferred to the pixels generated by the interpolation. That is, in this example, the pixels adjacent to the four corner pixels are copied. Also, when such processing is performed, the original image in which black pixels are arranged obliquely with white pixels as the background as shown in FIG. 22 becomes oblique while the black pixels are enlarged three times vertically and horizontally as shown in FIG. Direction.
[0070]
The near-list method has a feature that an edge of an image is held as it is. Therefore, when enlarged, jaggies are noticeable but edges are retained as edges. On the other hand, in other interpolation processing, the pixel to be interpolated is changed smoothly using data of surrounding pixels. Therefore, while the jaggy becomes inconspicuous, the information of the original image is cut off, and the edge disappears, which is not suitable for non-natural images such as computer graphics.
[0071]
On the other hand, in step ST212, a cubic interpolation process is executed as an interpolation process suitable for a natural image such as a photograph. As shown in FIG. 24, the cubic method uses data of a total of 16 grid points including not only four grid points surrounding a point Puv to be interpolated but also grid points around one point.
When a total of 16 grid points surrounding the interpolation point Puv have respective values, the interpolation point Puv is determined under the influence of these. For example, if an attempt is made to interpolate using a linear expression, the weighted addition may be made in inverse proportion to the distance from the two grid points sandwiching the interpolation point. Focusing on the X-axis direction, the distance from the interpolation point Puv to the above-mentioned 16 grid points is x1, the distance to the left outer grid point is x1, the distance to the left inner grid point is x2, and the right inner The degree of influence corresponding to such a distance is represented by a function f (x) while expressing the distance x3 to the lattice point and the distance x4 to the right outer lattice point. Further, when focusing on the Y-axis direction, the distance from the interpolation point Puv to the 16 lattice points is a distance from the upper outer lattice point to y1, a distance from the upper inner lattice point to y2, and a lower inner lattice point. Similarly, the degree of influence can be expressed by a function f (y) while expressing the distance y3 to the point and the distance y4 to the lower outer grid point.
[0072]
Since the 16 grid points contribute to the interpolation point Puv with the degree of influence according to the distance as described above, the general method is to accumulate the respective degree of influence in the X-axis direction and the Y-axis direction on the data at all the grid points. The formula is as follows.
[0073]
(Equation 1)

If the degree of influence according to the distance is represented by a third-order convolution function, f (t) = {sin (πt)} / πt
It becomes. The above-described distances x1 to x4 and y1 to y4 are calculated as follows using the absolute value of the coordinate value (u, v) of the grid point Puv.
x1 = 1 + (u− | u |) y1 = 1 + (v− | v |)
x2 = (u- | u |) y2 = (v- | v |)
x3 = 1- (u- | u |) y3 = 1- (v- | v |)
x4 = 2- (u- | u |) y4 = 2- (v- | v |)
Expanding on P under the above assumptions,
[0074]
(Equation 2)

It becomes. The degree of influence f (t) according to the distance, which is called a third-order convolution function, is approximated by the following cubic expression.
[Equation 3]

The cubic method has a feature that the gradual change gradually occurs as one grid point approaches the other grid point, and the degree of the change becomes a so-called cubic function.
[0075]
FIGS. 25 and 26 show specific examples when interpolation is performed by the cubic method. For ease of understanding, a model in which there is no change in data in the vertical direction and an edge occurs in the horizontal direction will be described. The number of pixels to be interpolated is three.
First, specific numerical values in FIG. 26 will be described. The gradation value of the pixel before interpolation is shown as “Original” in the left column, and four pixels (P0, P1, P2, and P3) with the gradation value “64” are arranged, and the pixel with the gradation value “128” is displayed. (P4) is sandwiched by one point, and five pixels (P5, P6, P7, P8, P9) having a gradation value of “192” are arranged side by side. In this case, the edge is a pixel portion having a gradation value of “128”.
[0076]
Here, when three pixels (Pn1, Pn2, Pn3) are interpolated between each pixel, the distance between the interpolated pixels is “0.25”, and the above-mentioned x1 to x4 are each interpolated point. In the middle column of the table. f (x1) to f (x4) are also uniquely calculated corresponding to x1 to x4. For example, x1, x2, x3, and x4 are "1.25" and "0.25", respectively. , “0.75” and “1.75”, f (t) corresponding thereto is approximately “−0.14”, “0.89”, “0.30”, “−0.05”. ". Further, when x1, x2, x3, and x4 are “1.50”, “0.50”, “0.50”, and “1.50”, respectively, f (t) is “−”. 0.125 "," 0.625 "," 0.625 ", and" -0.125 ". Further, when x1, x2, x3, and x4 are “1.75”, “0.75”, “0.25”, and “1.25”, respectively, f (t) is roughly described as “ −0.05 ”,“ 0.30 ”,“ 0.89 ”, and“ −0.14 ”. The result of calculating the tone value of the interpolation point using the above result is shown in the right column of the table, and is shown graphically in FIG. The meaning of this graph will be described later in detail.
[0077]
Assuming that there is no change in data in the vertical direction, the operation is simplified, and each data is calculated from the interpolation points while referring to only the data (P1, P2, P3, P4) of the four grid points arranged in the horizontal direction. It can be calculated as follows using the degree of influence f (t) according to the distance to the grid point.
P = P1 · f (x1) + P21f (x2) + P3 · f (x3) + P4 · f (x4)
Therefore, when calculating for the interpolation point P21,

It becomes.
[0078]
According to the cubic method, the quality of the interpolation result can be influenced by adjusting the shape of the curve as long as it can be expressed as a cubic function.
As an example of that adjustment,
0 <t <0.5 f (t) =-(8/7) t ** 3- (4/7) t ** 2 + 1
0.5 <t <1 f (t) = (1-t) (10/7)
1 <t <1.5 f (t) = (8/7) (t-1) ** 3+ (4/7) (t-1) ** 2- (t-1)
1.5 <t <2 f (t) = (3/7) (t-2)
This is called the hybrid bicubic method.
[0079]
FIG. 27 shows a specific example when interpolation is performed by the hybrid bicubic method, and shows a result obtained by interpolating a model on the same assumption as in the case of the cubic method. FIG. 25 also shows the result of interpolation processing by the hybrid bicubic method. In this example, the cubic function curve becomes slightly steep, and the entire image becomes sharp.
In order to understand the characteristics of the above-described near-list method, cubic method, and hybrid bicubic method, a bilinear interpolation method (bilinear interpolation: hereinafter, referred to as a bilinear method) which is another interpolation method will be described.
[0080]
The bilinear method is similar to the cubic method in that it gradually changes from one grid point to the other as shown in FIG. 28, but the change depends only on the data of the grid points on both sides. In that it is linear. That is, an area partitioned by four grid points Pij, Pi + 1j, Pij + 1, and Pi + 1j + 1 surrounding a point Puv to be interpolated is divided into four sections by the interpolated point Puv, and data on diagonal positions is weighted by the area ratio. I do. Expressing this as an equation,
P = {(i + 1) -u} (j + 1) -v} Pij
+ ｛(I + 1) -u｝ ｛v-j｝ Pij + 1
+ {U-i} (j + 1) -v @ Pi + 1j
+ ｛U-i｝ ｛v-j｝ Pi + 1j + 1
It becomes. Note that i = [u] and j = [v].
[0081]
The two cubic methods and the bilinear method are common in that they gradually change from one grid point to the other grid point, but the change is either cubic or linear. The difference is large when viewed as an image. FIG. 29 is a diagram two-dimensionally showing the difference between the interpolation results in the near-list method, the cubic method, the hybrid bicubic method, and the bilinear method in order to facilitate understanding. In the figure, the horizontal axis indicates the position, and the vertical axis indicates the interpolation function. Of course, this interpolation function corresponds to the degree of influence according to the distance described above. Grid points exist at positions of t = 0, t = 1, and t = 2, and interpolation points are at positions of t = 0 to 1.
[0082]
In the case of the bilinear method, since the boundary only changes linearly between two adjacent points (t = 0 to 1), the boundary is smoothed, and the impression of the screen is blurred. That is, unlike the smoothing of the corners, when the boundary is smoothed, the contour which should be originally in computer graphics disappears, and the focus becomes loose in a photograph.
On the other hand, in the cubic, not only the two points adjacent to each other (t = 0 to 1) draw a mountain-shaped convex but gradually approach each other, and further, the lower part outside the two points (t = 1 to 2). It has the effect of pushing down. That is, a certain edge portion is changed so as to have a large difference in height such that no step is formed, and this has a favorable effect that in a photograph, no step is formed while increasing sharpness. In addition, the hybrid bicubic has an effect of increasing sharpness. Note that the cubic method requires a large amount of computational processing if the interpolation magnification is large and the number of pixels to be interpolated is large.
[0083]
If emphasis is placed on image quality, a cubic function like the cubic method is likely to be selected, but computer processing has a large balance between speed and image quality. That is, although the degree of reduction in processing speed increases with the degree of improvement in image quality, there is a case where a higher processing speed is preferred even if the image quality is slightly or slightly reduced.
On the other hand, it is easier to understand by referring to FIG. 25, FIG. 26, and FIG. 27 showing specific numerical values along with the comparison of the interpolation functions as described above. Referring to the example of FIG. 15, a pixel (P3) having a gradation value of “64”, a pixel (P4) having a gradation value of “128”, and a pixel (P5) having a gradation value of “192”, which are original edge portions. Focusing on three points, the simple linear connection method is equivalent to the bilinear method, whereas the cubic method has a specific S-shaped curve, and the hybrid bicubic method. In the figure, the S-shaped curve is steeper. Of course, the direction of the S-shaped curve is a direction in which the gradation value change of the pixel is steep, and therefore the edge is emphasized. Further, in the regions (P2 to P3, P5 to P6) adjacent to the edge pixels, so-called undershoots and overshoots occur. The height difference becomes larger. Therefore, it can be understood that the edge is emphasized by these two factors.
[0084]
It can be easily understood that whether the image looks sharp depends on the inclination angle of the central portion in the S-shaped curve. In addition, it can be said that the difference in height caused by the undershoot and the overshoot on both sides of the edge has the same effect.
Each of the interpolation processes has the above-described characteristic differences. If it is determined in step ST208 that the image is a non-natural image based on the number of colors obtained in step ST206, the interpolation process using the near-list method in step ST210 is performed. Then, if it is a natural image, interpolation processing by the cubic method is executed.
[0085]
The interpolation process itself can be executed at an arbitrary magnification. However, in order to speed up the process in the printer driver 12c, an interpolation process of an integral multiple is accepted. FIG. 29 shows an example of processing for interpolating twice in the horizontal and vertical directions. If a variable area for the interpolated image data is secured in advance, the image data of the original image will be the image data of the pixel corresponding to the coordinate value multiplied by the integer if the interpolation processing is an integral multiple. In the example shown in the figure, the old coordinate value (0,0) corresponds to the new coordinate value (0,0), the old coordinate value (1,0) corresponds to the new coordinate value (2,0), The old coordinate value (0, 1) corresponds to the new coordinate value (0, 2), and the old coordinate value (1, 1) corresponds to the new coordinate value (2, 2). Therefore, image data is generated only for the remaining coordinate values in accordance with the above-described interpolation processing. In this case, it is possible to sequentially scan the image data in the main scanning direction in the width direction and the length direction in the sub-scanning direction, or for each block surrounded by four grid points having image data. It is also possible to perform interpolation processing of the internal coordinate values and fill them.
[0086]
When all the new coordinate values have been subjected to the interpolation processing, the interpolated image data is transferred to the next-stage processing in step ST214. However, the data amount of the interpolated image data may become extremely large depending on the interpolation magnification, and the memory area available to the printer driver 12c may not be so large in the first place. In such a case, the data may be output separately for each fixed data amount.
As described above, steps ST116 and ST118 correspond to a method in which the interpolation process changes the interpolation process for each region in block units. This will be described in connection with the specific software processing of the printer driver 12c described above. As shown in FIG. 29, the image data of the grid point is shifted to a new coordinate value by multiplying the integer by four, The feature amount is acquired for each block surrounded by. And interpolation processing is selected. In the example described above, since the number of colors used is totaled, such a feature amount cannot be used. However, it is possible to acquire another feature amount and change the interpolation process.
[0087]
For example, when a difference between four grid points is obtained, the difference often occurs in a natural image. However, in a case where a character is embedded in a natural image, if the character is a single color, the four grid points match and no difference occurs. Needless to say, the pixels to be interpolated in the region where no difference occurs need only be the same data as the four grid points, and may be interpolated by the near-list method that requires the least amount of calculation.
It is not necessary to limit the case where no difference occurs. In an area such as the sky, where there is not much change, image quality deterioration cannot often be determined even by using a near list. Therefore, when the difference between the four grid points is small, interpolation may be performed using the near list method. In the above example, a block is formed in the smallest unit as a region, but the interpolation process may be changed for a larger block.
[0088]
It is not necessary that the selectable interpolation processing be limited to either the near list method or the cubic method. For example, in the case of quadruple interpolation, it is also significant to first perform double interpolation by the cubic method and then double the interpolation by the near-list method. This is because, before the number of pixels is increased by the interpolation process, the interpolation process with a large amount of calculation is performed, and then the interpolation process with a small amount of calculation is performed.
[0089]
That is, in such a case, it can be said that the interpolation magnification affects the interpolation result. Therefore, the printer driver 12c can select the interpolation processing when the magnification can be determined in the comparison between the reference resolution in the operating system 12a and the resolution of the color printer 17b.
As described above, in the computer system 10 having the scanner 11a and the like as the image input device and the color printer 17b and the like as the image output device, the printer driver 12c acquires the original image data in step ST202, and in steps ST204 and ST206. By counting the number of colors used in the image, a feature amount for determining whether the image of the image data is a natural image or a non-natural image is extracted. In step ST208, a natural amount is extracted based on the feature amount. By determining whether the image is an image or a non-natural image, if the image is a non-natural image, the near-list interpolation process of step ST210 is executed. If the image is a natural image, the cubic interpolation process of step ST212 is executed. , Interpolation processing according to the characteristics of the image is performed, It can be obtained results very easily.
[0090]
As a premise of executing such a program, the computer 12 includes a CPU 12e, a RAM 12f, a ROM 12g, an I / O 12h, and the like. The RAM 12f functions as an image memory for storing image data in which an image is represented by pixels in a dot matrix. Further, the RAM 12f acquires a feature amount related to the interpolation process for the image data stored in the image memory, and obtains an optimal interpolation result from a plurality of interpolation processes corresponding to the feature amount. A possible interpolation process is selected, and the CPU 12a executes the interpolation process and stores a processing program to be written into the image memory. Further, the I / O 12h functions as an interface for inputting and outputting the image data.
[0091]
Of course, the CPU 12e appropriately executes the basic program written in the ROM 12g while using the RAM 12f as a temporary work area, a setting storage area, and a program area, and inputs unprocessed image data via the I / O 12h. Output the processed image data.
Next, an embodiment in which the degree of change in the image data described above is used as a feature amount will be described.
FIG. 31 is a block diagram illustrating the present image data interpolation device.
The present image data interpolating apparatus performs such an enlarging process in pixel units, the image data acquiring means D1 acquires such image data, and the pixel interpolating means D2 determines the number of constituent pixels in the image data. Is performed. Here, the pixel interpolation means D2 can execute a plurality of interpolation processes according to the degree of change of pixels as the interpolation processing, and the pixel change degree evaluation means D3 evaluates the degree of change for each pixel based on the image data. I do. Then, the interpolation processing selection means D4 selects an interpolation processing capable of obtaining an optimum interpolation result in accordance with the degree of change of the pixel evaluated in this way, and causes the pixel interpolation means D2 to execute the interpolation processing.
[0092]
In the present embodiment, the display driver 12b and the printer driver 12c execute not only the above-described pixel interpolating means D2 but also the pixel change degree evaluating means D3 and the interpolation processing selecting means D4 as described below. A well-balanced interpolation result can be obtained.
FIG. 32 shows a software flow relating to the resolution conversion executed by the printer driver 12c described above.
[0093]
In step ST302, original image data is obtained. Steps ST304 to ST308 are processing for evaluating the degree of change of pixels in the read image data.
FIG. 33 and FIG. 34 show an edge detection filter for calculating a luminance gradient, after obtaining the luminance by the simplified calculation as described above. Since the image data is composed of pixels in the form of a dot matrix, the degree of change of the image should be evaluated between eight pixels adjacent to the pixel of interest. In that sense, as shown in FIG. 34 (a), it is preferable to apply a filter by equally evaluating surrounding pixels while adding eight times the weight to the pixel of interest and adding them together. However, empirically, it is possible to evaluate only the target pixel and the surrounding four pixels as shown in FIG. 33A without necessarily evaluating the eight surrounding pixels. Of course, there is a large difference in the amount of calculation between using four pixels and eight pixels. If the number of evaluation targets is reduced in this way, the processing time can be reduced.
[0094]
FIGS. 33 (b) and 34 (b) show examples of actual image data (luminance). FIGS. 33 (c) and 34 (c) show the filter shown in FIG. 2) shows an example of calculation when applied to the arrangement of image data shown in FIG. The image data generally shows a case where the area of the image data “100” is on the upper left side and the area of the image data “70” and “60” is on the lower right side. In the example of FIG. 33, a weight of “−1” is added to each of the four pixels (image data “100”, “100”, “70”, “70”) on the upper, lower, left, and right sides of the center pixel. “100”) is assigned a weight of “4”. Then, weighted addition is performed for these five pixels. The weighted addition result is “60”, which exceeds the threshold value (th) “32”.
[0095]
On the other hand, in the example of FIG. 34, a weight of “−1” is added to each of the eight pixels surrounding the center pixel, and a weight of “8” is added to the center pixel. The weighted addition result is “100”, which exceeds the threshold value (th) “64”.
If the result of using the edge detection filters shown in FIGS. 33 and 34 is called an edge amount E of each pixel, the distribution is expected to be a normal distribution as shown in FIG. 35, and the degree of change of the image is large. Whether it is an edge portion can be determined by comparing with a threshold th. In the edge detection filters shown in FIGS. 33 and 34, the threshold values of the edge amounts of th = 32 and th = 64 are appropriate as threshold values, respectively. Therefore, whether or not the pixel is an edge pixel is evaluated from the following equation.
(E <-th) or (th> E)
The process of step ST306 performs this evaluation for all the pixels in the dot matrix, and evaluates whether or not each pixel is a pixel having a large image change degree such as an edge pixel.
[0096]
By the way, even if it is determined whether or not the degree of change of the image is large in each pixel unit, since the interpolation process is a process of generating a pixel for each fixed area, it is determined whether or not the degree of change of the image is large in each area unit. Must be determined. Since it is complicated to determine the degree of change for each region, a flag is set in advance in step ST308 by determining whether or not the pixel is an edge pixel. In this case, as shown in FIG. 36, it is determined that the degree of change of the image is large in all pixels surrounding the edge pixels. More specifically, assuming that the degree of change of each pixel is as shown in FIG. 37A, if the threshold value is "32", the pixel exceeding the threshold value is represented by the xy coordinates. Even if it is (0, 0) (3, 0) (4, 0) (1, 1) (2, 1) as shown, a flag is set to a pixel adjacent to the edge pixel. Then, flags are set for all pixels at y = 0, 1 and pixels at y = 2 except (4, 2), as shown in FIG. As a result, when the target block is moved in pixel units in a later step, the interpolation process can be appropriately selected with reference to only the flag.
[0097]
Of course, in the present embodiment, the processing of steps ST304 to ST308 corresponds to a pixel change degree evaluation step. Of course, if these are considered to be organically integrated with hardware such as a CPU, the pixel change degree evaluation means D3 is configured.
Based on the flag set as described above, the interpolation pixels are generated in a loop process in step ST310 and subsequent steps. FIG. 38 schematically shows the arrangement of pixels generated by interpolating existing pixels. The coordinates of the existing pixels are temporarily displayed as (X, Y), and the coordinates of the pixels generated by interpolation are displayed as <X, Y>. In the example shown in the figure, an interpolation process of about 2.5 × 2.5 is performed.
[0098]
One area surrounded by the existing four pixels is called a block, and an interpolation process of a pixel to be interpolated is selected for each block. In step ST308, a flag is set for each pixel in consideration of the degree of change of surrounding pixels, and therefore, in each block, four pixels (0, 0) (1, 0) (0, 1) (1, 1) If the flag is set for any of the above, the interpolation process in the case where the degree of change is large is selected. If none of the flags is set, the interpolation process in the case where the degree of change is small is selected. Will be. In step ST310, an interpolation process to be applied to the inside of the block is determined based on this condition. If the degree of change is small, the interpolation is performed by the nearest-list interpolation process in step ST312. In step ST314, interpolation is performed by interpolation using the cubic method. After one block is interpolated, the block to be processed is moved in steps ST316 and ST318, and when all the blocks are completed, the interpolated image data is output in step ST320.
[0099]
Although the flow of returning to step ST310 after the end of step ST318 is shown by a solid line in the figure, the process of totalizing the edge pixels for each block may be repeated as shown by the broken line.
Needless to say, in this sense, it corresponds to an interpolation processing selection step including the processing of steps ST316 and ST318 centering on the processing of step ST310. Of course, if these are considered to be organically integrated with hardware such as a CPU, the interpolation processing selecting means D4 will be configured. In the case of the printer driver 12c, print data cannot be obtained only by resolution conversion, but requires color conversion or halftone processing. Therefore, outputting the image data here means transferring the data to the next stage.
[0100]
In the case of the present embodiment, an area surrounded by four pixels is called a block and interpolation processing is selected. However, the basis for changing the interpolation processing can be changed as appropriate in accordance with the computing capacity and the interpolation processing. For example, as shown in FIG. 39, interpolation processing may be performed with reference to a region centered on a target pixel. In such a case, the interpolation process may be appropriately performed while scanning and moving the target pixel as indicated by an arrow.
Here, a method of selecting an interpolation process while moving the pixel of interest will be described. In the above-described example, when determining whether the degree of change is large for each block, it is determined that the image has a large degree of change only when all the flags included in the area are “1”. . However, it can be said that it is not always necessary that all the flags are “1”. For example, assume that pixels are generated by interpolation processing in a region surrounded by four pixels as shown in FIG. In this case, since a flag is set to the pixel adjacent to the edge pixel in FIG. 36, only a case where the flag is set to all four pixels as described above can be determined to be a region with a large change degree. . However, when judging in this way, when the block is moved horizontally by one pixel, the judgment is duplicated each time for two vertical pixels because of the common vertical side. The judgment is duplicated each time for two horizontal pixels because of the common side. Such an overlapping situation is useless in the arithmetic processing. On the other hand, as shown in FIG. 3B, considering the adjacent state of the region, it is possible to associate one region with the upper left pixel as a representative, and at least the vicinity of a pixel that can be regarded as an edge pixel. It can be said that there is no problem even if it is determined that the degree of change is large. In addition, it can be said that the area surrounded by adjacent pixels is sufficient even in view of the fact that it is actually an extremely small area. If one area corresponds to one pixel in this way, the pixel of interest is moved when the block is moved, and the degree of change of the area can be determined only by the edge amount of the pixel of interest. In other words, the amount of arithmetic processing required for the determination is reduced.
[0101]
It is also possible to form a block on the side of the pixel to be interpolated. FIG. 41 shows this example. In the figure, the grid points indicated by □ indicate the pixels to be interpolated, and the grid points indicated by ○ indicate existing pixels. Now, it is assumed that one 5 × 5 block is used for the pixel to be interpolated, and it is determined based on the edge amount of the existing pixel included therein whether or not the area has a large degree of change in the image. In this case, one block is determined, existing pixels included in the block are extracted, the integrated value of the edge amount is obtained, and the pixel is generated by the same interpolation processing in the block.
[0102]
Of course, in the above case, a block may be set for each larger area and the interpolation processing may be selected. For example, it is possible to set a block for every 10 × 10 pixels. Further, it is also possible to determine the edge amount of the existing pixels surrounding each pixel to be interpolated without setting a block, and to select the interpolation processing. In the example of FIG. 41, the 3 × 3 □ grid points arranged inside are all included in the four existing grid points indicated by ○, and when generating each □ grid point, This means that the interpolation processing is selected based on the edge amounts of the four existing grid points indicated by を surrounding this. Of course, it may be realized when such processing is more convenient in terms of arithmetic processing. In other words, a method may be used in which a block to be interpolated is specified first and interpolation processing is determined, and then pixels are interpolated therein. Alternatively, the interpolation process is selected by determining the state of the block for each pixel to be interpolated. You may.
[0103]
Further, in the above-described flow, a flag is set in advance for a pixel adjacent to the edge pixel in step ST308, and the flag is referred to for each block. However, it is also possible to adopt a flow of moving a block as shown by a broken line in FIG. 32. In this case, it is not necessary to set a flag, and the edge amount of pixels around the block is determined. What is necessary is just to select the interpolation processing.
[0104]
As described above, the processing of steps ST312 and ST314 having separate interpolation processing corresponds to a pixel interpolation step. Of course, if these are considered to be organically integrated with hardware such as a CPU, the pixel interpolation means D2 will be configured. Here, each interpolation process will be described in detail.
On the other hand, in the interpolation function shown in FIG. 29, the slope is steep in the section from t = 0 to 1 and is drawn to the negative side so as to cancel the increased weight in the section from t = 1 to 2. Occurs when
[0105]
Therefore, when the sharpness is to be adjusted, (1) an ideal slope which is a reference of the sharpness is determined in the interpolation function, and (2) a curve for generating the above-described slope in a section of t = 0 to 1. And (3) in a section of t = 1 to 2, it is possible to realize a curve in which overshooting and undershooting are likely to occur while drawing to the negative side so as to offset the weighting increased by this curve. Of course, in the subsequent work, the parameters of the multi-order operation function are determined so as to obtain the specified curve. However, since the methods for determining such parameters are extremely diverse, the central portion of the S-shaped curve in a practical sense is substantially determined. It is nothing less than adjusting the tilt angle and the undershoot and overshoot.
[0106]
Each of the interpolation processes has a difference in characteristics as described above. In the block for which it is determined in step ST310 that the degree of change in the image is small, the interpolation process of the near-list method is executed in step ST312. For the blocks determined to have a high degree, interpolation processing of the cubic method or the hybrid bi-cubic method is executed. When the interpolation process is performed by the cubic method, the calculation time is long. However, in a portion where the degree of change of the image is small, the process is switched to the near-list method, so that the processing time as a whole is extremely reduced. In particular, in a case where a certain area is painted in the same color as in computer graphics, there is no problem even if the near-list method is executed uniformly, so that the processing time is reduced. In addition, even if it is a natural image, the portion where the jaggy is conspicuous when enlarged is usually not so large even in terms of the area ratio, and thus the image quality is changed by sequentially switching the degree of change of the image in this way. The processing amount can be reduced without deterioration.
[0107]
In the present embodiment, one of the two types of interpolation processing is executed according to the flag. However, a plurality of interpolation processing steps corresponding to the degree of change of the pixel may be executed. Also, as shown in FIG. 42, two interpolation processes may be performed in a superimposed manner, and the magnification may be made to correspond to the degree of change in the image. For example, assuming that the interpolation magnification is five times, if the degree of change of the image is small, the interpolation processing is performed five times by the near-list method, and if the degree of change of the image is large, the interpolation processing is performed five times by the cubic method. In these cases, it is the same as the above-described embodiment, but when the degree of change of the image is an intermediate value, the interpolation processing is performed twice by the cubic method, and the remaining 2.5 times is interpolated by the near-list method. To process. In this way, it is possible to select a plurality of interpolation processes according to the degree of change of the image, although the two interpolation processes are performed.
[0108]
Note that, as described above, the interpolation magnification is set to an integral multiple for a large computation amount of the interpolation processing such as the cubic method.
As described above, in the computer system 10 having both the image input device and the image output device, the printer driver 12c inputs the original image data in step ST302, and detects the degree of change of the image in steps ST304 to ST108. By setting a flag and referring to the flag in step ST310, the interpolation process by the near-list method is executed in step ST312 for a block having a small image change degree, and the block is executed for a block having a large image change degree in step ST312. Since the interpolation processing by the cubic method is executed in step ST314, it is controlled to execute the near-list method as far as possible without deteriorating the image quality, and the calculation processing amount is automatically selected while selecting the optimum interpolation processing. Reduce.
[0109]
As described above, in the present invention, an image data obtaining unit that obtains image data expressing an image by dot matrix pixels, and a pixel change degree evaluating unit that evaluates the degree of change of a pixel based on the image data In order to perform an interpolation process for increasing the number of constituent pixels in the image data, a pixel interpolation unit that can be selected and executed from a plurality of interpolation processes and a pixel change degree evaluated by the pixel change degree evaluation unit are used. An interpolation processing selecting means for selecting an interpolation processing capable of obtaining an optimum interpolation result in accordance with the degree of change and causing the pixel interpolation means to execute the selected interpolation processing.
[0110]
In the present invention configured as described above, in performing the interpolation processing for increasing the number of constituent pixels of the image data in which the image is represented by the pixels in the dot matrix, the pixel interpolation unit selects one of the plurality of interpolation processing. When the image data acquiring means acquires the target image data, the pixel change degree evaluating means evaluates the degree of change of the pixel based on the image data. The interpolation processing selecting means selects an interpolation processing capable of obtaining an optimal interpolation result corresponding to the degree of change based on the degree of change of the pixel evaluated by the pixel degree of change evaluation means, and Cause the interpolation means to execute.
[0111]
That is, since the degree of change of a pixel is closely related to a specific method of the interpolation processing, the interpolation processing without waste is realized by evaluating the degree of change of the pixel and actively changing the interpolation processing.
As described above, the present invention can provide an image data interpolation apparatus that can obtain an optimum interpolation result extremely easily by changing the interpolation processing according to the degree of change of the image.
The pixel change degree evaluation means evaluates the degree of change of pixels, and the evaluation method and result are not particularly limited. Further, it can be relatively changed in accordance with the manner of using the evaluation result in the interpolation processing selecting means. For example, if a specific degree of change is required as a numerical value, a numerical value may be output, and if it is sufficient to simply determine whether the degree of change is large, the numerical value may be output accordingly.
[0112]
Also, how to grasp the change itself of the pixel itself can be changed as appropriate. As an example, the pixel change degree evaluating means may be configured to calculate the brightness degree of each pixel and calculate the change degree by comparing the parameter with the parameters of surrounding pixels.
In the case of such a configuration, the brightness of the pixel is used as a criterion for evaluating the pixel, and the pixel change degree evaluation means obtains a parameter of the brightness of each pixel, and determines the pixel and its surrounding pixels. Are compared with each other, and the comparison result is calculated as the degree of change.
[0113]
Of course, other than this, it is possible to grasp the degree of change of the pixel, but the brightness parameter is relatively easy to uniformly grasp the pixel represented by the multi-element parameters.
According to this configuration, the degree of change in the image is determined based on the brightness parameter, so that the degree of change can be obtained relatively easily.
On the other hand, it can be said that the range in which the degree of change of the pixel affects the processing content of the interpolation processing also differs. For example, in some cases, the number of pixels required for executing the interpolation process is one, and in others, the interpolation process is performed based on a plurality of pixels. In particular, in the latter case, the problem is whether to change the interpolation processing if there is at least one pixel having a large degree of change, or whether to change the interpolation processing if at least one pixel has a small degree of change.
[0114]
As an example of such a situation, the pixel change degree evaluation means may be configured to use the change degree obtained for each pixel also for evaluating the change degree of surrounding pixels.
When a plurality of pixels are required in the interpolation processing, and one of them has a large change degree, two modes are considered. That is, there is no need to evaluate the remaining pixels in the target range in the interpolation process, and conversely, even if the degree of change is small for the already evaluated pixels, the evaluation is unnecessary.
[0115]
In such two directions, the degree of change obtained for each pixel is also used for evaluating the degree of change of surrounding pixels.
In this way, the amount of calculation can be reduced by using the degree of change of one pixel also in the surrounding pixels, and the optimal interpolation result can be obtained by sharing in an appropriate range affected by the interpolation processing. Obtainable.
The pixel interpolation means only needs to be able to execute a plurality of interpolation processes related to the degree of change of pixels, and various processes can be performed as the interpolation process itself. As an example, the pixel interpolating means executes an interpolation process using the image data of the nearest neighbor pixel before the interpolation process as image data of a new constituent pixel as a preferable interpolation process applied to an area having a small degree of change. Possible configurations are also possible.
[0116]
In the case of such a configuration, the image data of the nearest neighbor pixel before the interpolation process is used as the image data of the new constituent pixel as one interpolation process, but even if the data of the same pixel increases, the area where the degree of change is small is small. This is preferable because there is no problem and the amount of processing is small.
In this way, the processing amount can be reduced without affecting the image quality in an area where the degree of change is small.
Further, as another example, the pixel interpolating means performs arithmetic processing from image data of surrounding pixels so that the image data of the pixel to be interpolated changes smoothly as a suitable interpolation processing applied to an area having a large degree of change. It is also possible to adopt a configuration capable of executing an interpolation process for calculating image data of an interpolation pixel.
[0117]
In the case of such a configuration, the image data of the pixel to be interpolated changes smoothly by performing arithmetic processing using the image data of the surrounding pixels. As described above, if the change is made gently, it is assumed that there is a row of pixels having a large change degree, and even if interpolation is performed between the pixels, the step is not conspicuous. Therefore, even if the pixel sequence having a large degree of change is interpolated between them, the step is not conspicuous and deterioration of the image quality can be prevented.
[0118]
Various calculation methods for smoothly changing the image data of the pixel to be interpolated can be adopted, but the manner of change affects the image quality. Therefore, in a sense, it can be said that the image quality can be adjusted by changing the calculation method. As an example in which the image quality can be adjusted, the pixel interpolation unit calculates the image data of the interpolated pixel between pixels having a large degree of change, while adjusting the inclination of the image data while making the change mode of the image data substantially S-shaped, At both end portions, an undershoot is generated on the lower side and an overshoot is generated on the higher side to form a height difference, and the height difference is adjusted, so that the degree of change of the image is adjusted to be optimal. It can also be configured.
[0119]
In the case of such a configuration, when the image data of the pixel to be interpolated is smoothly changed, the change mode of the image data between the pixels having a large degree of change is substantially S-shaped. Therefore, although the change is gradual, the change can be made steeper than a slope that is linearly connected, and the slope can be adjusted to optimize the degree of change of the image. In addition, if an overshoot is generated on the high side while generating an undershoot on the low side at both ends, the height difference becomes large, and by adjusting the height difference, the degree of change of the apparent image is optimized. It becomes possible. As an example of such arithmetic processing, a cubic convolution interpolation method of multi-dimensional arithmetic processing can be used, and the arithmetic processing that enables such adjustment is not limited to this, and another arithmetic technique is adopted. You can also.
[0120]
With this configuration, the image quality can be relatively easily adjusted by the inclination of the S-shaped curve and the difference in height due to the undershoot and the overshoot.
Since the degree of pixel change is not constant over the entire image, the interpolation processing selecting means must appropriately select and switch the interpolation processing. The frequency of such switching is not particularly limited, and various methods can be adopted. As an example, the interpolation processing selection means may be configured to select and execute the interpolation processing on a pixel-by-pixel basis based on the degree of change of a pixel evaluated by the pixel change degree evaluation means.
[0121]
In the case of such a configuration, the interpolation processing selecting means selects and executes the interpolation processing on a pixel-by-pixel basis on the basis of the pixel change degree evaluated by the pixel change degree evaluation means. That is, since the degree of change is evaluated in pixel units, the interpolation processing is changed correspondingly.
Therefore, the interpolation result can be finely improved by selecting the interpolation processing for each pixel.
Further, as another example, the interpolation processing selecting means selects and executes the interpolation processing for each predetermined small area including a plurality of pixels based on the degree of change of the pixel evaluated by the pixel change degree evaluating means. It can also be configured. By doing so, the interpolation processing is selected for each small area, so that the processing can be simplified.
[0122]
Next, a description will be given of an embodiment in which sharpness is also corrected while assuming interpolation processing.
FIG. 43 is a block diagram showing such an image data interpolation device.
If the original image is a natural image, some images may lack sharpness. For example, a photograph that is out of focus corresponds to this. Further, in a case where the image itself is to be enlarged and output in addition to the enlargement for matching the resolutions between the apparatuses, the image lacking in sharpness may be further defocused.
The present image data interpolation device adjusts the sharpness when enlarging the image data in units of pixels. The image data acquisition unit E1 acquires the image data, and the pixel interpolation unit E2 acquires the image data. An interpolation process for increasing the number of constituent pixels in the data is performed. Here, the pixel interpolation means E2 can change the sharpness of the image accompanying the interpolation processing, and the sharpness evaluation means E3 evaluates the sharpness of the image based on the image data. Then, the interpolation processing control means E4 controls the pixel interpolation means E2 so as to execute an interpolation processing for increasing the sharpness evaluated in this way if the sharpness is low.
[0123]
As described above, when the resolution managed by the operating system 12a does not match the resolution of the color printer 17b, the printer driver 12c executes a process for matching the resolutions. Normally, the resolution of the color printer 17b is finer than the resolution managed by the operating system 12a. Therefore, to match the resolutions, an interpolation process for increasing the number of pixels is performed. As described above, the interpolation processing is performed when the enlargement processing is performed by the application 12d and when the resolution is matched by the printer driver 12c. These interpolation processings can affect the sharpness of the image.
[0124]
The sharpness of an image can be said to be an overall evaluation of the degree of change between adjacent pixels. An image lacking in sharpness means that pixels change gently at an original edge portion, and in a sharp image, the degree of change between adjacent pixels is sharp at the original edge portion. This is because the interpolation process generates a new pixel between existing pixels, and the sharpness of the image changes depending on the value of the new pixel.
[0125]
In this sense, in the image data interpolating apparatus of the present invention, the application 12d and the printer driver 12c constitute not only the above-described pixel interpolating means E2 but also the sharpness evaluation means E3 and the interpolation processing controlling means E4 as described below.
FIG. 44 shows a software flow relating to resolution conversion executed by the printer driver 12c as an example of executing the interpolation processing.
In step ST402, original image data is obtained. Steps ST404 to ST408 are processing for evaluating the sharpness of the image from the degree of change of each pixel in the read image data.
[0126]
If the result of using the edge detection filters shown in FIGS. 33 and 34 is referred to as an edge amount E of each pixel, the distribution is expected to be a normal distribution as shown in FIG. The processing in step ST406 is performed for all the pixels in the dot matrix in this manner, and the absolute values of the edge amounts are totaled in step ST408. The aggregation may be a simple average value, but it can be said that the aggregation is easily affected by the area ratio of the background portion. For example, in FIG. 45, the subject image is large and the background portion is small, but in FIG. 46, the subject image is small and the background portion is large. Since the degree of pixel change tends to be small in the background portion, the average value tends to be lower in the one shown in FIG. 46 where the area ratio of the background portion is large than that shown in FIG. 45. In this sense, a certain threshold value may be provided, and the average of only the threshold value or more may be calculated.
[0127]
On the other hand, the main purpose of this tally is to determine the sharpness of the image, but it is also possible to judge from the tally result whether or not the image requires sharpness like a natural image in the first place. is there. In the case of a natural image, even if it is a part just like the background, the same pixel is not arranged according to the lightness and darkness of the color or the shape of the real thing as the background, so the sum of the absolute value of the edge amount The result is as shown in FIG. 47, and the edge amount tends to be large. On the other hand, in an image such as a business graph, a certain area is often painted in the same color, and thus the totaling result of the absolute value of the edge amount is as shown in FIG. 48, and the edge amount is lower.
[0128]
Therefore, when the average value (av) of the tally result is lower than a certain threshold value Th, it can be said that the image does not require the processing to increase the sharpness, and the interpolation processing which does not affect the sharpness is executed. Just fine.
As described above, in steps ST404 to ST408, the degree of change of each pixel constituting the image is totaled, and it is evaluated whether the image can be said to be a sharp image. Will constitute the sharpness evaluation means E3.
[0129]
Based on the evaluation result, in step ST410, an interpolation process according to the level of sharpness of the image is selected. In the present embodiment, interpolation processing by a cubic method is performed on a sharp image, and interpolation processing by a hybrid bi-cubic method is performed on an image lacking sharpness. Therefore, in this sense, the step ST410 constitutes the interpolation processing control means E4, and the processing of steps ST412 and ST414 provided with separate interpolation processing constitutes the pixel interpolation means E2. Here, each interpolation process will be described in detail.
[0130]
Each of the interpolation processes has the above difference in characteristics. If it is determined in step ST410 that the image is sharp, the interpolation process of the cubic method is executed in step ST412, and it is determined that the image is not sharp. Then, an interpolation process of the hybrid bicubic method is executed. When performing the interpolation processing by the hybrid bicubic method, the curve to be interpolated becomes sharp and the sharpness can be increased. Such interpolation processing is selected because the sharpness of the target image is evaluated and the evaluation is performed. It is based on the results. For this reason, an operator can sharpen an unsharp image without making any special judgment.
[0131]
In the present embodiment, one of the two types of interpolation processing is executed. However, a plurality of interpolation processing steps corresponding to the degree of pixel change may be executed. FIG. 49 shows an example in which the evaluation of sharpness is divided into four stages and four cubic methods are implemented in which the parameters of the cubic interpolation method are changed. In the figure, "0" indicates a curve of the cubic method applied to an image with normal sharpness, and a curve of "+1" indicates a cubic method applied to an image with a slight lack of sharpness. The "+2" curve shows the cubic method applied to images that are significantly less sharp. For an image that is too sharp, a cubic method with a curve of “−1” is applied. Of course, these are realized by adjusting the inclination angle, undershoot, and overshoot of the central portion of the S-shaped curve, as in the case of the hybrid bicubic method.
[0132]
Next, FIG. 50 shows the flow of the procedure in this case.
These parameters are set in step ST510 based on the sharpness of the image, and a cubic method using these parameters is executed in step ST514. In addition, in this flow, taking into account that the sharpness of the image is partially different, the image is divided into blocks, which are small areas, and optimal interpolation processing is executed for each block. That is, in order to evaluate the sharpness of each block and select an interpolation process, the edge amounts of each block are totaled in step ST508, and the interpolation process is executed while sequentially moving the blocks in steps ST516 and ST518. I have to.
[0133]
When the interpolation processing is completed for all the image data, the image data interpolated in step ST420 or step ST520 is output. In the case of the printer driver 12c, print data cannot be obtained only by resolution conversion, but requires color conversion or halftone processing. Therefore, outputting the image data here means transferring the data to the next stage.
As described above, in the computer system 10 having both the image input device and the image output device, the printer driver 12c inputs the original image data in step ST402, and evaluates the sharpness of the image in steps ST404 to ST108. At step ST410, if the image lacks sharpness based on the result at step ST410, an interpolation process for increasing sharpness is executed at step ST414. If the image is sharp, normal interpolation is performed at step ST412. Since the processing is executed, a sharp image can be obtained only through the interpolation processing without the operator selecting the image processing for increasing sharpness.
[0134]
As described above, in the present embodiment, an image data acquisition unit that acquires image data representing an image by dot matrix pixels, and a sharpness evaluation unit that evaluates the sharpness of an image based on the image data In performing the interpolation process for increasing the number of constituent pixels in the image data, a pixel interpolation unit capable of executing an interpolation process for changing the sharpness of the image, and the sharpness of the image evaluated by the sharpness evaluation unit is appropriate. If not, the image processing apparatus is provided with interpolation processing control means for causing the pixel interpolation means to execute interpolation processing so that the sharpness is changed and becomes appropriate.
[0135]
In the case of such a configuration, in performing the interpolation process for increasing the number of constituent pixels of the image data in which the image is represented by the pixels in the dot matrix, the pixel interpolating means can execute the interpolation process for changing the sharpness of the image. When the image data acquisition unit acquires the target image data, the sharpness evaluation unit evaluates the sharpness of the image based on the image data. Then, based on the sharpness evaluated by the sharpness degree evaluation means, the interpolation processing control means performs an interpolation process on the pixel interpolation means so as to change the sharpness if the image sharpness is not appropriate so as to be appropriate. Let it run.
[0136]
That is, when the sharpness of the image is low, the sharpness is increased by the interpolation process, and when the image is too sharp, the sharpness is reduced.
In this way, since the sharpness is adjusted by the interpolation processing according to the sharpness of the image, an image data interpolation device capable of easily improving the image quality without complicating the operation is provided. can do.
As a method of changing the sharpness of the image while performing the interpolation, the inclination of the image data is adjusted while the change mode of the image data is made substantially S-shaped, and undershoot is generated at both ends at a low side and overshoot at a high side. It is shown that by generating a height difference and adjusting the difference in height, the degree of change of the image is adjusted to be optimal to change the sharpness of the image.
[0137]
As an example of taking such an S-shaped curve, the pixel interpolating means may be configured to adjust the parameters in the cubic convolution interpolation method to change the sharpness of the image.
In the case of such a configuration, by adjusting the parameter of the cubic convolution interpolation method used as the interpolation processing, the pixel interpolated between adjacent pixels in the original image becomes a cubic function. By adopting it, it draws an S-shape, and it has both gentle and steepness. By adjusting the degree of the S-shaped bend with a parameter, the steepness changes and the sharpness of the image changes.
[0138]
In this way, the sharpness can be adjusted relatively easily by adjusting the S-shaped curve by using the third-order convolution interpolation method as the multi-order operation processing.
In changing the sharpness, it is not always necessary to adopt only one calculation method, and it is also possible to execute a plurality of interpolation processes that affect the sharpness. As an example of such a configuration, the pixel interpolation means can execute a plurality of interpolation processes having different degrees of change in the sharpness of the image, and adjust the sharpness of the image by changing the ratio of each interpolation magnification. It can also be.
[0139]
In the case of such a configuration, the degree of change in the sharpness of the image differs in each of the plurality of interpolation processes, and the plurality of interpolation processes are executed to obtain a necessary interpolation magnification. Therefore, the sharpness can be adjusted by changing the sharing ratio of the interpolation magnification. For example, in the case where there is an interpolation process with a low degree of change in sharpness and an interpolation process with a high degree of change in sharpness, if the sharing ratio of the two is changed, it is possible to select an intermediate point between the two degrees of change.
[0140]
By doing so, it is only necessary to change the interpolation magnification shared by the plurality of interpolation processes, so that the parameter setting is simplified.
The sharpness of an image is not necessarily high. Therefore, it may not be necessary to increase the sharpness. In this case, it is possible for the operator to make a determination, but it is also possible to adopt a configuration in which such determination is realized simultaneously. As one example, the interpolation processing control means controls the pixel interpolation means to change the image sharpness when the sharpness of the image is evaluated as exceeding a predetermined threshold value. It can also be.
[0141]
In this case, the interpolation processing control means compares the sharpness of the image with a predetermined threshold value, and when it is determined that the sharpness of the image exceeds the predetermined threshold value, the pixel interpolation means Is controlled to change the sharpness of the image. However, if the sharpness of the image does not exceed the threshold value, no control is made to change the sharpness of the image. For example, when comparing the sharpness of an image when it is a natural image with the sharpness of an image when it is a non-natural image, it can be said that the former is generally higher in sharpness. Of course, the sharpness of an image is not determined only by the classification of a natural image or a non-natural image, and therefore, it is possible to add another judgment factor. For example, if the classification of an image is acquired, and the threshold is changed after the classification, more flexible handling is possible.
[0142]
In this way, the sharpness is not changed up to a certain range, so that the inconvenience of automatically adjusting the sharpness to an image for which changing the sharpness is inappropriate is eliminated.
Since the sharpness of the screen is not constant over the entire image on the basis of individual pixels, it is not necessarily limited to a constant interpolation process. For this reason, it is also possible to adopt a configuration in which the sharpness of an image is evaluated on a pixel-by-pixel basis, and control is performed to change the sharpness of the image on a pixel-by-pixel basis based on the sharpness of the evaluated image. In addition, the sharpness of the image is evaluated for each predetermined small area, and control is performed to change the sharpness of the image for each predetermined small area based on the sharpness of the evaluated image.
[0143]
Next, an embodiment in which image processing can be selected will be described.
FIG. 51 is a block diagram showing such an image data interpolation device.
The present image data interpolating apparatus simultaneously performs enlargement processing and sharpness change processing as interpolation processing when performing image processing of image data in pixel units, and acquires the same image data by the image data acquisition means F1. In both cases, the image processing to be performed by the image processing selecting means F2 is selected. The simultaneous processing determining means F3 determines whether or not the image processing to be performed is the processing of enlarging and the processing of changing the sharpness simultaneously. If it is necessary to perform the processing simultaneously, the pixel interpolating means F4 operates. The sharpness of the image is changed accompanying the interpolation processing for increasing the number of constituent pixels in the image data. Then, the image data after the enlargement processing is output by the image data output unit F5.
[0144]
In the computer system 10, image data is acquired by a scanner 11a or the like, which is an image input device, and predetermined image processing is executed by an application 12d. There are various types of image processing, such as enlargement / reduction, sharpness, contrast, and hue correction. As described above, in order to display and output on the display 17a or the color printer 17b as an image output device, it is necessary to match resolutions. In particular, when matching with the resolution of the printer, interpolation processing is also performed as enlargement processing. Is performed. This interpolation process may be executed by the printer driver 12c or by the application 12d.
[0145]
Here, in the application 12d, it is possible to select enlargement processing and sharpness change processing as image processing, and it is also possible to change sharpness along with enlargement processing when matching with the resolution of the printer. It is possible. Therefore, in these cases, enlargement processing and sharpness adjustment processing are executed.
In this sense, the image data interpolation device of the present invention is realized as the application 12d in the computer system 10 described above. The application 12d constitutes not only the above-described pixel interpolating means F4 but also an image processing selecting means F2 and a simultaneous processing judging means F3 as described below. Further, since data input / output is involved, the image data acquisition means F1 and the image data output means F5 are configured in the sense of file input, file output, or print data output.
[0146]
FIG. 52 shows an outline of a software flow of the application 12d. The application 12d can execute various types of image processing by menu selection as shown in FIG. 53, and is not necessarily limited to the software flow as shown in FIG. 52, but is displayed in a simplified form for easy understanding. are doing.
In step ST602, original image data is obtained. This corresponds to, for example, a process of reading an image from the scanner 11a using a file menu or the like in the application 12d. In the present embodiment, since image data to be used for image processing to be described later may be generated, the process of generating an image file by selecting creation of a new file or the like can also be said to be acquisition of original image data. . The process of acquiring the original image data except for the configuration of the operating system 12a and the hardware corresponds to the image data acquisition. Of course, if these are considered to be organically integrated with hardware such as a CPU, they correspond to the image data acquisition unit F1.
[0147]
In step ST604, image processing is selected. In steps ST606 and ST608, the selected image processing is determined. To select the image processing, click the character part of "image" with the mouse 15b from the menu bar displayed in the window frame on the screen as shown in FIG. 53, and the processing is executed as shown in FIG. Various possible image processings are displayed. In the example shown in the drawing, “enlargement” processing, “sharpness” adjustment processing, “contrast” adjustment processing, and “brightness” adjustment processing can be executed. A “button” switch is prepared for each image processing, and a plurality of image processing can be selected at the same time. Each process is displayed in gray until it is selected by the “button” switch, so that the non-display state can be distinguished.
[0148]
When the desired image processing button switch is set to the selected state, the respective parameters are set, and the “OK” button is clicked, the selection is processed as input processing, and the processing selected in steps ST606 and ST608 is determined. I do. In this example, it is determined whether or not enlargement processing has been selected in step ST606, and if enlargement processing has not been selected, corresponding image processing is performed in step ST610. On the other hand, if enlargement processing has been selected, it is determined in step ST608 whether or not sharpness adjustment processing has been selected. As a result, if the enlargement process and the sharpness adjustment process are both selected, the process proceeds to step ST612. If the enlargement process is selected but the sharpness adjustment process is not selected, the process proceeds to step ST614. Steps 112 and ST614 will be described later.
[0149]
In this software flow, the processing is performed in two branch processes of steps ST606 and ST608 for easy understanding. However, in practice, a large number of branches may be selected in case processing. In the present embodiment, image processing is selected in step ST604, which corresponds to an image processing selection step. In steps ST606 and ST608, it is determined whether or not both the enlargement processing and the sharpness adjustment processing are selected. Therefore, this corresponds to a simultaneous processing determination step. Of course, if these are considered to be organically integrated with hardware such as a CPU, they correspond to the image processing selecting means F2 and the simultaneous processing judging means F3.
[0150]
Here, a modified example of the image processing selecting means F2 and the simultaneous processing determining means F3 will be described.
FIGS. 55 to 57 show an example thereof, and it is assumed that the application 12d does not have an explicit sharpness adjustment process as shown in FIG. 54, and the enlargement process is selectable. In this example, a parameter input window for enlargement processing as shown in FIG. 56 is displayed as selection of image processing in step ST704. Here, the enlargement ratio can be selected and input in%. When the user sets the desired magnification and clicks the “OK” button, the process proceeds to the next stage. Normally, by inputting the enlargement ratio, the interpolation process according to the magnification may be executed. In this example, if it is determined in step ST706 that the enlargement process has been selected, the process proceeds to step ST708 in FIG. A window for inquiring about sharpness adjustment is displayed as shown in FIG. In this window, three choices of “NO”, “Low”, and “High” are prepared, and buttons that can be selected alternatively are assigned to each of them. The default is “NO”, and the operator can select “Low” or “High” as necessary. These are inquiries about sharpness emphasis. "NO" does not emphasize, "Low" means slight emphasis, and "High" means emphasis.
[0151]
Then, based on the selected result, one of steps ST712, ST714, and ST716, which is an interpolation process, is executed. At the time of enlargement processing, the sharpness can be changed by selecting interpolation processing as described later. Therefore, when the enlargement processing is selected, an inquiry is automatically made. In this example, steps ST704 and ST708 correspond to an image processing selection step, and step ST710 corresponds to a simultaneous processing determination step.
[0152]
Also, FIGS. 58 to 60 show a case where the image processing is not directly selected but the enlargement processing is actually performed internally. This is the case of processing. When print is selected from a file menu (not shown), a print menu as shown in FIG. 59 is displayed in step ST804. Various parameters can be set in this print menu, and one of them is a “print resolution” selection box. Regardless of the resolution handled internally by the application 12d, a resolution matching operation is required depending on which resolution is used for printing at the time of printing. Assuming that the resolution of the color printer 17b2 is 720 dpi, when printing at a printing resolution of 720 dpi, no resolution conversion is necessary because one dot of image data corresponds to one dot at the time of printing. However, when printing at 300 dpi, the correspondence with print data must be matched, and resolution conversion in this sense is required.
[0153]
Therefore, in step ST808, the parameter of the print resolution box is compared with the resolution of the color printer 17b2 managed by the operating system 12a to determine whether or not it is necessary to perform the resolution conversion. In this case, a window for inquiring about sharpness as shown in FIG. 60 is displayed in step ST810. In this example, as in the case of FIG. 54, the degree of sharpness adjustment is selected and input in%, and in step ST812, processing is selected according to the input parameters. That is, when the adjustment process is not requested, the process proceeds to step ST814, and when the adjustment process is requested, the process proceeds to step ST816. Thereafter, in a state where the resolutions match, a printing process is executed in step ST818.
[0154]
In this example, steps ST804 and ST810 correspond to the image processing selection step, and step ST812 corresponds to the simultaneous processing determination step. It goes without saying that the image processing selecting means F2 and the simultaneous processing judging means F3 can be realized by other methods.
The interpolation process is executed after the above determination. Specifically, when only the enlargement processing is selected and the sharpness enhancement processing is not selected, the interpolation processing by the near list method is executed (steps ST614, ST712, ST814), and the enlargement processing and the sharpness enhancement processing are performed. Is selected, the interpolation processing by the hybrid bicubic method (steps ST612 and ST816) and the interpolation processing by the cubic method (step ST714) are executed. Therefore, the latter steps ST612, ST816, and ST714 constitute the pixel interpolation means F4.
[0155]
The ability to adjust the sharpness during the enlargement processing as in the present embodiment is superior to simply performing the enlargement processing and the sharpness adjustment processing separately. For example, even if the enlargement process is performed after emphasizing the sharpness, if the enlargement process is of a near-list method, jaggies may be conspicuous and a sharp feeling may not be maintained. If the sharpness is emphasized after enlargement by the near-list method, the sharpness will be sharpened in a conspicuous manner, and the image quality cannot be improved. On the other hand, if the sharpening is also performed in the integrated enlargement processing, such an adverse effect hardly occurs.
[0156]
When the interpolation processing and other image processing are completed for the image data, the image data is output in step ST66. Outputting the image data here has a broad meaning, and is not limited to processing such as outputting to the color printer 17b2 or writing to the hard disk 13b, and displaying the image data on the display 17b1 while retaining the data. Alternatively, it may be prepared for the next image processing. Of course, in the present embodiment, this step ST616 constitutes the image data output means F5.
[0157]
As described above, in the computer system 10 having the image input device, the image output device, and the like, the application 12d can execute various image processing. When the user selects the image processing to be executed in step ST604, the application 12d If the processing and the change processing of the sharpness are specified at the same time, the interpolation processing for increasing the sharpness is performed in step ST612 after the determination in step ST608, and if only the enlargement processing is selected, the processing proceeds to step ST614. The normal interpolation process that does not affect sharpness is performed, so that extra time is not required to execute the enlargement process and the sharpness change process separately, and both are executed integrally. Therefore, the sharpness can be surely adjusted.
[0158]
As described above, in the present invention, various types of image data are obtained by changing image data in individual pixels with respect to the image data, and an image data obtaining unit that obtains image data expressing an image by dot matrix pixels. Image processing selecting means for displaying image processing executable to execute image processing and inputting a selection, and whether or not the image enlarging processing and the image sharpness changing processing are both selected by the image processing selecting means Means for judging whether or not the image enlargement process and the image sharpness change process are both selected by increasing the number of constituent pixels in the image data. When enlarging an image, the degree of change in the interpolated image data is adjusted to make the selected image sharper. It is constituted comprising a viable pixel interpolation means interpolating process, and an image data output means for outputting the generated image data to the.
[0159]
In the present invention having such a configuration, when image data obtained by expressing an image by dot matrix pixels is obtained by the image data obtaining means, the image data in each pixel is changed with respect to this image data. An image processing executable by the image processing selecting means to execute various image processing is displayed and a selection is input. Here, the simultaneous processing determination means determines whether the image enlargement processing and the image sharpness change processing are both selected by the image processing selection means, and if it is determined that both processings are selected, When increasing the number of constituent pixels in the image data and enlarging the image, the pixel interpolating means adjusts the degree of change of the image data to be interpolated so as to execute interpolation processing so that the selected image becomes sharp. Then, the image data output means outputs the generated image data.
[0160]
In other words, if it is necessary to simultaneously execute the process of changing the image sharpness and the enlargement of the image, the sharpness is changed while the image is enlarged by adjusting the image data generated by the interpolation process.
As described above, according to the present invention, since the sharpness of an image is changed by the interpolation process required in the enlargement process, it is not necessary to perform the enlargement process and the sharpness change process separately, and the processing time is shortened. An image data interpolating apparatus capable of performing such operations can be provided.
[0161]
Also, since they are performed at the same time, there is no possibility that good results will be obtained when changing the sharpness afterwards depending on the enlargement process, and of course, by changing the sharpness and then performing the enlargement process, the sharpness The change process is not wasted.
The image processing selection means displays executable image processing and inputs a selection, and the simultaneous processing determination means determines whether both the image enlargement processing and the image sharpness change processing have been selected. Things. As a result, it is only necessary to determine whether or not both processes need to be executed simultaneously, and the mode of selection and the like can be changed as appropriate.
[0162]
As one example, the image processing selection means can individually select the enlargement processing selection and the sharpness change selection, and the simultaneous processing determination means can select the enlargement processing and sharpen by this image processing selection means. It is also possible to adopt a configuration in which it is determined whether or not the selection of the change is simultaneously selected.
In such a configuration, since the selection of the enlargement processing and the selection of the sharpness change can be individually selected, only the enlargement processing or only the sharpness change may be selected. Then, the simultaneous processing determination means determines whether or not the selection of the enlargement processing and the selection of the sharpness change are selected at the same time on the premise of such a situation.
[0163]
This is suitable when the enlargement process and the sharpness change process can be individually selected.
Further, as another example, the image processing selection means can select an enlargement process, and the simultaneous processing determination means determines a degree of sharpness change when the enlargement process is selected by the image processing selection means. May be selected.
In such a configuration, only the enlargement processing can be selected by the image processing selection means, and no input is made until the selection for changing the sharpness is made. However, when the enlargement processing is selected by the image processing selection means, the simultaneous processing determination means independently determines the degree of sharpness change. Of course, it is also possible to choose not to change the sharpness, in which case only the enlargement processing will be executed, and conversely, if the choice is made to change the sharpness, the enlargement processing will be performed. It is determined that the processing for changing the sharpness has been selected at the same time.
[0164]
Therefore, even when only the enlargement processing can be selected on the surface, the sharpness can be changed together.
Furthermore, in the above-described example, the enlargement processing is explicitly selected, but the enlargement processing itself does not need to be limited to an explicit one. As an example, the simultaneous processing determination means may be configured to select the degree of change in sharpness when the image processing selection means performs resolution conversion processing along with image processing.
[0165]
In the case of such a configuration, it may be necessary to perform a resolution conversion process incidental to the image processing selected by the image processing selection unit. In this case, the simultaneous processing determination unit Select the degree of change in sharpness. If it is selected to change the sharpness, it is determined that the enlargement process and the process of changing the sharpness are selected at the same time.
In this way, the sharpness can be changed in the case where the enlargement process is performed incidentally.
[0166]
Note that the display and input operations in these cases may be realized by screen display or mouse operation under a GUI, or may be implemented as appropriate by using hardware switches. It is.
[Brief description of the drawings]
FIG. 1 is a block diagram of an image data interpolation device according to an embodiment of the present invention.
FIG. 2 is a block diagram of specific hardware of the image data interpolation device.
FIG. 3 is a schematic diagram showing another application example of the image data interpolation device of the present invention.
FIG. 4 is a schematic diagram showing another application example of the image data interpolation device of the present invention.
FIG. 5 is a schematic diagram showing another application example of the image data interpolation device of the present invention.
FIG. 6 is a schematic diagram showing another application example of the image data interpolation device of the present invention.
FIG. 7 is a general-purpose flowchart in the image data interpolation device of the present invention.
FIG. 8 is a more specific flowchart in the image data interpolation device of the present invention.
FIG. 9 is a diagram illustrating the size of an original image.
FIG. 10 is a diagram showing a sampling cycle.
FIG. 11 is a diagram showing the number of sampling pixels.
FIG. 12 is a diagram showing a relationship between an original image and pixels to be sampled.
FIG. 13 is a diagram illustrating a histogram of luminance for a non-natural image.
FIG. 14 is a diagram showing a histogram of luminance for a natural image.
FIG. 15 is an explanatory diagram in a case where the degree of change of an image is represented by each component value of rectangular coordinates.
FIG. 16 is an explanatory diagram in a case where the degree of change of an image is obtained by a difference value between adjacent pixels in a vertical axis direction and a horizontal axis direction.
FIG. 17 is an explanatory diagram in a case where the degree of change of an image is obtained between all adjacent pixels.
FIG. 18 is a diagram showing a region where a threshold is changed.
FIG. 19 is a diagram illustrating a situation in which a printer driver makes an inquiry to an operating system.
FIG. 20 is a conceptual diagram of a near-list method.
FIG. 21 is a diagram showing a situation in which data of each grid point is transferred by the near list method.
FIG. 22 is a schematic diagram showing a situation before interpolation in a near-list method.
FIG. 23 is a schematic diagram showing a situation after interpolation by the near-list method.
FIG. 24 is a conceptual diagram of the cubic method.
FIG. 25 is a diagram showing a data change situation when the cubic method is specifically applied.
FIG. 26 is a diagram showing a specific application example of the cubic method.
FIG. 27 is a diagram showing a specific application example of the hybrid bicubic method.
FIG. 28 is a conceptual diagram of the bilinear method.
FIG. 29 is a diagram illustrating a change state of an interpolation function.
FIG. 30 is a schematic diagram showing an interpolation process of an integral multiple.
FIG. 31 is a schematic block diagram of an image data interpolation device according to an embodiment of the present invention.
FIG. 32 is a flowchart in the image data interpolation device of the present invention.
FIG. 33 is a diagram illustrating an example of an edge detection filter.
FIG. 34 is a diagram illustrating another example of the edge detection filter.
FIG. 35 is a diagram illustrating a relationship between a distribution of edge amounts and a threshold.
FIG. 36 is a diagram illustrating a relationship between a pixel of interest and a pixel targeted for flag setting determination.
FIG. 37 is a diagram illustrating an edge amount and a setting state of a flag.
FIG. 38 is a diagram illustrating a relationship between blocks formed by existing pixels and pixels to be interpolated.
FIG. 39 is a diagram illustrating a case where a block is formed based on a target pixel.
FIG. 40 is a diagram showing a correspondence between a flag status and an area.
FIG. 41 is a diagram illustrating a case where a block is formed by pixels to be interpolated.
FIG. 42 is a diagram illustrating a relationship in which interpolation magnifications of a plurality of interpolation processes are distributed according to a degree of change of an image.
FIG. 43 is a schematic block diagram of an image data interpolation device according to an embodiment of the present invention.
FIG. 44 is a flowchart in the image data interpolation device of the present invention.
FIG. 45 is a diagram showing a small image of a background portion.
FIG. 46 is a diagram illustrating an image having a large background portion.
FIG. 47 is a diagram illustrating a tally result of edge amounts of a natural image.
FIG. 48 is a diagram showing the result of counting the edge amounts of the business graph.
FIG. 49 is a diagram illustrating a change state of an interpolation function.
FIG. 50 is a diagram showing a flow for selecting an interpolation function.
FIG. 51 is a schematic block diagram of an image data interpolation device according to an embodiment of the present invention.
FIG. 52 is a flowchart in the image data interpolation device of the present invention.
FIG. 53 is a diagram illustrating a display of a menu for executing image processing.
FIG. 54 is a diagram showing a screen for selecting image processing.
FIG. 55 is a flowchart showing a modification of the image data interpolation device.
FIG. 56 is a diagram showing a screen for instructing enlargement processing.
FIG. 57 is a diagram showing a screen for instructing sharpness.
FIG. 58 is a flowchart of a printing process.
FIG. 59 is a diagram illustrating a screen for instructing parameters at the time of printing.
FIG. 60 is a diagram showing a screen for instructing sharpness.
[Explanation of symbols]
10. Computer system
11a ... Scanner
11a2 ... Scanner
11b Digital still camera
11b1: Digital still camera
11b2 ... Digital still camera
11c ... Video camera
12 ... Computer body
12a: Operating system
12b ... display driver
12b ... driver
12c: Printer driver
12d… Application
13a: Floppy disk drive
13b ... Hard disk
13c: CD-ROM drive
14a… Modem
14a2 ... Modem
15a ... Keyboard
15b… Mouse
17a ... Display
17a1 ... Display
17b ... Color printer
17b1 ... Color printer
17b2 ... Color printer
18a ... Color facsimile machine
18b: color copying machine

Claims (31)

Image data acquisition means for acquiring image data representing an image by dot matrix pixels,
Pixel interpolation means that can be selected and executed from among a plurality of interpolation processes in performing the interpolation process for increasing the number of constituent pixels in the image data,
A feature amount obtaining unit that obtains a feature amount related to the interpolation processing on the image data,
Interpolation processing selecting means for selecting an interpolation processing capable of obtaining an optimal interpolation result corresponding to the characteristic amount obtained by the characteristic amount obtaining means and causing the pixel interpolation means to execute the selected interpolation processing. Image data interpolation device. 2. The image data interpolating apparatus according to claim 1, wherein the pixel interpolating unit is capable of executing an optimal interpolation process for each of a natural image and a non-natural image, and the feature amount obtaining unit is configured to represent the image data. The image obtains a feature value for determining whether a natural image is a non-natural image, and the interpolation processing selecting means determines whether or not the image is a natural image based on the feature amount. An image data interpolating apparatus that executes an optimal interpolation process for a non-natural image by executing an optimal interpolation process for a natural image when the image is determined to be a non-natural image. 3. The image data interpolating apparatus according to claim 2, wherein the pixel interpolating means performs an interpolation process on the non-natural image by a nearest neighbor interpolation method and performs a cubic convolution interpolation method on the natural image. An image data interpolating device for performing an interpolation process. In the image data interpolating apparatus according to claim 1, the pixel interpolating means can execute an optimal interpolation process corresponding to a change degree of a different pixel,
The feature amount obtaining means obtains a feature amount for evaluating a degree of change of a pixel based on the image data,
The interpolation processing selection means selects an interpolation processing capable of obtaining an optimum interpolation result corresponding to the degree of change based on the degree of change of the pixel acquired by the feature amount acquisition means, and An image data interpolating apparatus characterized by being executed by In the image data interpolating apparatus according to claim 4, the pixel interpolating means applies image data of a nearest neighbor pixel before the interpolation processing to a new constituent pixel as a preferable interpolation processing applied to an area having a small degree of change. An image data interpolation apparatus capable of executing an interpolation process used for image data. In the image data interpolating apparatus according to claim 4, the pixel interpolating means is applied to an area having a large degree of change, and as a suitable interpolation process, the image data of the surrounding pixels is changed so as to smoothly change. An image data interpolation device capable of executing an interpolation process of calculating image data of an interpolation pixel from image data by a calculation process. 2. The image data interpolating apparatus according to claim 1, wherein said pixel interpolating means has an interpolating process in which a plurality of interpolating processes are overlapped and executed as an option of said interpolating process. 2. The image data interpolating apparatus according to claim 1, wherein the characteristic amount acquiring unit acquires a characteristic amount for each partial region of the image, and the interpolation processing selecting unit acquires the characteristic amount acquired for each of the regions. An image data interpolating apparatus which selects and executes an interpolation process in the pixel interpolating means based on the image data. 2. The image data interpolation apparatus according to claim 1, wherein said pixel interpolation means realizes a plurality of pixel interpolation processes by adjusting operation parameters in a cubic convolution interpolation method. . The image data interpolation device according to claim 1,
Image processing selecting means for displaying executable image processing and inputting a selection;
Simultaneous processing determining means for determining whether or not the image enlargement processing and the image sharpness change processing are both selected by the image processing selection means;
When the simultaneous processing determination means determines that both the image enlargement process and the image sharpness change process have been selected, the image to be interpolated is used to enlarge the image by increasing the number of constituent pixels in the image data. An image data interpolating device comprising: a pixel interpolating unit capable of executing an interpolating process so as to sharpen a selected image by adjusting a degree of change of data. A CPU,
An image memory for storing image data representing an image by dot matrix pixels,
An interpolation process capable of obtaining a feature amount related to the interpolation process for the image data stored in the image memory and obtaining an optimal interpolation result from a plurality of interpolation processes corresponding to the feature amount. And a program memory for storing a processing program for causing the CPU to execute the interpolation process and writing the image data to the image memory;
A computer for image data interpolation processing, comprising: an interface for inputting and outputting the image data. Acquires image data that represents the image with dot matrix pixels,
Acquire a feature amount related when performing the interpolation process on the image data,
Select an interpolation process capable of obtaining an optimal interpolation result corresponding to the acquired feature amount,
An image data interpolation method, wherein the image data is processed by the selected interpolation processing. 13. The image data interpolation method according to claim 12, wherein the image represented by the image data acquires a characteristic amount for determining whether a natural image is a non-natural image, and the image is a natural image based on the characteristic amount. An image data interpolation method comprising: performing an optimal interpolation process for a natural image when it is determined that the image data is a non-natural image, and performing an optimal interpolation process for the non-natural image when it is determined that the image is a non-natural image. 14. The image data interpolation method according to claim 13, wherein the pixel interpolation means performs an interpolation process by a nearest neighbor interpolation method on a non-natural image and performs a cubic convolution interpolation method on a natural image. An image data interpolation method characterized by performing an interpolation process. The image data interpolation method according to claim 12,
Acquiring a feature amount for evaluating the degree of change of a pixel based on the image data,
An image data interpolation method characterized by selecting an interpolation process capable of obtaining an optimum interpolation result in accordance with a degree of change of an acquired pixel based on the degree of change. 16. The image data interpolation method according to claim 15, wherein the image data of the nearest neighbor pixel before the interpolation processing is used as the image data of a new constituent pixel as a suitable interpolation processing applied to an area having a small degree of change. An image data interpolation method characterized by performing processing. 16. The image data interpolation method according to claim 15, wherein the image data of the pixels to be interpolated is changed from the image data of the surrounding pixels so that the image data of the pixel to be interpolated changes smoothly as a suitable interpolation process applied in an area having a large degree of change. An image data interpolation method, comprising: performing an interpolation process of calculating image data of an interpolation pixel. 13. The image data interpolation method according to claim 12, wherein a plurality of interpolation processes are performed in an overlapping manner as an option of the interpolation process. In the image data interpolation method according to the twelfth aspect, a feature amount is obtained for each partial region of the image, and an interpolation process in the pixel interpolation unit is selected based on the feature amount obtained for each region. An image data interpolation method characterized by being executed. 13. The image data interpolation method according to claim 12, wherein a plurality of pixel interpolation processes are realized by adjusting operation parameters in the cubic convolution interpolation method. The image data interpolation method according to claim 12,
Display the available image processing and enter your selection,
When the image enlargement process and the image sharpness change process are both selected, by increasing the number of constituent pixels in the image data and enlarging the image, the degree of change of the interpolated image data is adjusted. An image data interpolation method, wherein an interpolation process is performed so that a selected image becomes sharp. A medium in which an image data interpolation program for causing a computer to execute an interpolation process so as to increase the number of constituent pixels at a predetermined interpolation magnification with respect to image data in which an image is expressed by a dot matrix pixel by a computer,
Obtaining the image data;
A feature value obtaining step of obtaining a feature value related to the interpolation processing on the image data,
And an interpolation process selecting step of selecting an interpolation process capable of obtaining an optimal interpolation result corresponding to the feature amount obtained by the feature amount obtaining step and causing the pixel interpolation step to execute the selected interpolation process. On which an image data interpolation program to be recorded is recorded. 23. A medium recording the image data interpolation program according to claim 22, wherein in the pixel interpolation step, optimal interpolation processing can be performed for each of a natural image and a non-natural image, and in the feature amount obtaining step, Acquiring an amount of feature that determines whether the image represented by the image data is a non-natural image of a natural image. In the interpolation process selecting step, when the image is determined to be a natural image based on the feature amount, An image characterized in that an optimal interpolation process for a natural image is executed in a pixel interpolation step and an optimal interpolation process for a non-natural image is executed in the pixel interpolation step when it is determined that the image is a non-natural image. Medium on which data interpolation program is recorded. 24. A medium in which the image data interpolation program according to claim 23 is recorded, wherein in the pixel interpolation step, a non-natural image is subjected to an interpolation process by nearest neighbor interpolation, and a third-order convolution is performed on the natural image. A medium on which an image data interpolation program is recorded, wherein the medium performs an interpolation process by an interpolation method. In the medium on which the image data interpolation program according to claim 22 is recorded, in the pixel interpolation step, it is possible to execute an optimal interpolation process corresponding to a different pixel change degree,
In the feature amount obtaining step, a feature amount for evaluating a degree of change of a pixel based on the image data is obtained,
In the interpolation process selection step, an interpolation process capable of obtaining an optimal interpolation result corresponding to the degree of change based on the degree of change of the pixel acquired in the feature amount acquisition step is selected, and the pixel interpolation step is performed. A medium storing an image data interpolation program characterized by being executed by a computer. 26. The medium in which the image data interpolation program according to claim 25 is recorded, wherein, in the pixel interpolation step, the image data of the nearest pixel before the interpolation processing is newly applied as a suitable interpolation processing applied to an area having a small degree of change. A medium recording an image data interpolation program capable of executing an interpolation process used for image data of various constituent pixels. 26. In the medium on which the image data interpolation program according to claim 25 is recorded, in the pixel interpolation step, the image data of the pixel to be interpolated as a suitable interpolation process applied in an area having a large degree of change is changed smoothly. A medium storing an image data interpolation program capable of executing an interpolation process of calculating image data of an interpolated pixel from image data of surrounding pixels by arithmetic processing. 23. A medium storing the image data interpolation program according to claim 22, wherein the pixel interpolation step includes an interpolation process in which a plurality of interpolation processes are performed in an overlapping manner as an option of the interpolation process. Medium on which the program is recorded. 23. The medium storing the image data interpolation program according to claim 22, wherein the feature amount obtaining step obtains a feature amount for each partial area of the image, and the interpolation processing selecting step obtains the feature amount for each of the areas. A medium in which an image data interpolation program is recorded, wherein an interpolation process in the pixel interpolation step is selected and executed based on the obtained feature amount. 23. The medium on which the image data interpolation program according to claim 22 is recorded, wherein in the pixel interpolation step, a plurality of pixel interpolation processes are realized by adjusting operation parameters in a cubic convolution interpolation method. A medium on which an image data interpolation program is recorded. A medium recording the image data interpolation program according to claim 22,
An image processing selecting step of displaying executable image processing and inputting a selection;
A simultaneous processing determination step of determining whether or not the image enlargement processing and the image sharpness change processing are both selected by the image processing selection step;
When it is determined in the simultaneous processing determination step that both the image enlargement process and the image sharpness change process have been selected, the image to be interpolated is used to increase the number of constituent pixels in the image data and enlarge the image. A pixel interpolation step capable of executing an interpolation process so as to sharpen a selected image by adjusting a degree of change in data, wherein the image data interpolation program is recorded.

Images