JP5165732B2 - Multispectral image processing method, image processing apparatus, and image processing system - Google Patents

Description

The present invention relates to a multispectral image processing method , an image processing apparatus, and an image processing system , and in particular, based on a multispectral image obtained by photographing an object (target object) having a characteristic in a wavelength direction as a subject. The present invention relates to a multispectral image processing method , an image processing apparatus, and an image processing system that perform processing such as object classification and region extraction.

  The present invention also relates to a diagnostic method using a multispectral skin image, and in particular, using a multispectral image obtained by photographing the skin as a subject (hereinafter simply referred to as a multispectral skin image), psoriasis and acne ( The present invention relates to a diagnosis method using a multispectral skin image that supports diagnosis by a dermatologist by quantifying and displaying the state of various skin diseases such as acne.

  Conventionally, as a method of classifying an object based on spectral characteristics of a specific object, for example, in the “color classification device” disclosed in Patent Document 1, reflection of an object whose class is known A method of classifying an object whose class is unknown from a spectrum using a statistical method is used. However, in such a method, when there are spatial unevenness or individual differences in components other than the spectral characteristic of interest, there arises a problem that the influence cannot be completely removed.

  In addition, as a conventional technique for visualizing a specific portion or extracting a region based on spectral characteristics of the skin, for example, a method for obtaining an image by obtaining an absorption pigment concentration from spectral reflectance according to Lambert Bear's law (Patent Document) 2), and a method of visualizing a specific portion by calculating a difference between images taken with two different wavelengths of light (see Patent Document 3).

  However, in the “diagnostic system” disclosed in Patent Document 2, in order to use Lambert-Bear's law, the absolute value of the spectral reflectance must be measured and the optical path length for each wavelength must be known. There are restrictions such as not being able to be applied, the application target is limited, and the photographing apparatus is complicated.

  In addition, in the “image diagnostic apparatus” disclosed in Patent Document 3, in the method of visualizing a specific portion by calculating the difference between images taken with two different wavelengths of light, other than the spectral features of interest If there are spatial irregularities or individual differences in the components, there arises a problem that the influence cannot be completely removed.

  Non-Patent Document 1 discloses a method of creating a color image in which only a wavelength component that characterizes a background color distribution is emphasized using a multispectral image.

Japanese Patent No. 3469619 Japanese Patent No. 3417235 JP 2002-272744 A

Co-authored by M. Mitsui, Y. Murakami, T.Obi, M. Yamaguchi, and N. Ohyama, “Color Enhancement In” Multispectral image off human skin “Color enhancement in multispectral image of human skin”, Proc. Of SPIE, Vol. 4959, p.83-88, 2003

  By the way, the multi-spectral image has more information in the wavelength direction than the three-channel color image, and can express the spectral characteristics of the subject in more detail. An analysis is often performed by extracting a specific region having a spectral feature from the multispectral image or classifying the image based on the spectral feature.

  For such purposes, there is an image using an image of a specific wavelength component, but it is easily affected by the color of the background, and there arises a problem that it is not possible to extract only the feature of the spectrum of interest.

  In order to solve this problem, that is, as a method of removing the influence of the background color, a method using a difference or ratio between images of different wavelength bands of a multispectral image is used. In short, this is done by selecting two wavelengths, the wavelength component that contains the characteristics of the spectrum of interest and the background color, and the wavelength component that contains the background color, and calculating the difference between the two wavelength components. The feature to be noticed is extracted.

  For example, when the characteristic of the spectrum of interest is v (λ), the amount of the characteristic component is a, the spectrum of the background color is u (λ), and the amount of the background color component is b, The spectrum of is represented by the following formula 1.

波長λ_０に注目する特徴が含まれており、波長λ_１に下地の色の成分のみが含まれているときに、下記数２、数３が得られる。

The following features 2 and 3 are obtained when the feature focusing on the wavelength λ ₀ is included, and only the background color component is included in the wavelength λ ₁ .

数２と数３の差分を計算すると、下記数４が得られる。

When the difference between Equation 2 and Equation 3 is calculated, the following Equation 4 is obtained.

ｕ(λ_０)−ｕ(λ_１)が一定であるとすれば、Δｇは注目する特徴成分の量に比例するので、下地の影響を取り除くことが可能である。しかし、実際には数２、数３が成立するような波長を常に選択できるとは限らない。

If u (λ ₀ ) −u (λ ₁ ) is constant, Δg is proportional to the amount of feature components of interest, so that the influence of the background can be removed. However, in practice, it is not always possible to select a wavelength that satisfies Equations 2 and 3.

  Further, in Equation 1, the background color is expressed by a single function u (λ). However, when the background color variation is expressed by a combination of a plurality of functions, the above-described method is used. Will cause a problem that cannot be canceled.

In other words, in conventional techniques such as region extraction by calculating differences and ratios between images in different wavelength bands of multispectral images, when the background color varies, the differences and ratio values vary. As a result, there arises a problem that it is difficult to stably extract a region.
The present invention has been made in view of the above described circumstances, an object of the present invention is based on multispectral image obtained by the object (target) having a feature in the wavelength direction taken as the object, multispectral image processing method for performing processing such as classification and area of extraction of the object is to provide an image processing apparatus, and an image processing system.

  The present invention has been made under the circumstances as described above, and an object of the present invention is to use various spectrum images such as psoriasis and acne using a multispectral skin image obtained by photographing the skin as a subject. By quantifying and displaying the state of skin disease, diagnosis by dermatologists is supported, and only the characteristics of lesions can be extracted even if the color of normal skin varies depending on the individual or sunburn. The object is to provide a diagnostic method based on multispectral skin images.

The present invention uses a multispectral image image obtained by photographing the object with the features in the wavelength direction as a subject, by image analysis relates multispectral image processing method of extracting the feature region of the object The above object of the present invention is to provide a first step of photographing the first portion of the object using a multi-channel image photographing device having sensitivity in different wavelength bands, and a multi-image photographed in the first step. spectrum image image, or in a part of or the entire area of the multispectral image image including a second portion of the object which is separately shot, a second step of performing principal component analysis, resulting in a second step of obtained principal component vectors, and multispectral image image reconstructed as the sum of the specified number of principal component vectors, the obtained by photographing a first portion of the object in a first step multithreading By calculating the difference between the transfected Le picture image, a third step of obtaining a difference multispectral image image of the difference multispectral image image obtained in the third step was specified one A fourth step of selecting an image of one or a plurality of channels, and performing image processing on the image selected in the fourth step, thereby extracting a feature region of the object and extracting the extracted feature effectively achieved by having a fifth step intends area of the region, the number or shape line.

Further, the present invention uses a multispectral image image obtained by photographing the object with the features in the wavelength direction as a subject, by image analysis, multispectral image processing of extracting a characteristic region of the object a method, the above object of the present invention, the step D1 to specify one or more wavelengths, the step D2 for specifying the number of dimensions M of the subspace indicating the feature of the predetermined portion of the object, specified in step D1 A channel having spectral sensitivity in a wavelength band including one or a plurality of wavelengths, and a predetermined type of different spectral sensitivity necessary for obtaining a projection value onto an M-dimensional subspace representing a predetermined portion of the object. in the imaging apparatus having a channel, a predetermined portion of the step D3 for photographing an object, the object that the multispectral image image, a separately stored obtained by photographing in step D3 Obtained by projecting the subspace spanned by the basis functions of the M-dimensional representative of the spectral characteristics, of the M-dimensional subspace formed fractionation image, the image of the wavelength component corresponding to one or more wavelengths that the specified seeking, step D4 of the specific wavelength formation fractions image on the M-dimensional subspace, from the multispectral image image obtained by photographing in step D3, one or a plurality of wavelengths the specified a step D5 to extract specific wave Nagaga image corresponding wavelength components, a specific wavelength formation fractions image on the M-dimensional subspace obtained in step D4, the specific wave Nagaga extracted in step D5 for an image, by calculating the difference for each wavelength component corresponding to the designated one or more wavelengths, the step D6 of extracting difference specific wave Nagaga image corresponding to the designated wavelength, step D6 Extracted with By using the pixel values of the differential specific wave Nagaga image of one or more channels corresponding to the specified wavelength, and the step D7 classifying each pixel or region in the image into a plurality of classes, in the step D7 A step D8 for extracting a feature region of the object based on the classified features of the class, or a projection value onto an M-dimensional subspace representing a predetermined portion of the object is obtained in step D1. M-dimensional basis functions representing spectral features of a predetermined portion of the object stored separately are calculated by including one or a plurality of wavelength band images specified by the same object. effectively achieved by a Turkey determined from an image obtained by photographing a predetermined portion of the object.
Furthermore, the present invention relates to a multispectral image processing apparatus for classifying a pixel group based on the spectral characteristics of the object, the object of the present invention is to the object having the features in the wavelength direction and the object, means for capturing multispectral image with the wavelength direction N-channel (N is an integer of 2 or more), the multispectral image image of the object obtained by photographing by the photographing means, M dimensions (M <N, M is projected in the subspace spanned by the basis functions of an integer), a first calculation means for calculating the M-dimensional subspace component image of the Target material, the obtained by the photographing means said multispectral image of the object, by calculating the difference between the M-dimensional subspace component image of the target product obtained by the first calculating means, the difference multispectral image of the object calculate Extracting a second calculation means, among the differential multispectral image before Symbol object obtained by the second calculating means, the difference specific wavelength image of one or more channels corresponding to the specified wavelength by having a means, using the pixel values of the differential specific wavelength image of one or more channels which are extracted by the extraction means, and means for determining a class belonging of each point in the image, or, said object having a characteristic in the wavelength direction as a subject, and means for taking Maruchisupe vector images having an N-channel in the wavelength direction (N is an integer of 2 or more), the obtained by photographing by said photographing means the multispectral image of the object, an M-dimensional (M <N, M is an integer) of the M-dimensional subspace component image of the object obtained by projecting the subspace spanned by the basis functions, By calculating the image of the constant has been one or more wavelength components, and the first calculation output means for calculating the specific wavelength component image on the M-dimensional subspace, the obtained photographing by said photographing means wherein the multispectral image, and means to extract a specific wavelength image wavelength component corresponding to the designated one or more wavelength components, the M-dimensional portion obtained by the first calculating means of the object a specific wavelength ingredient images in space, the for a specific wavelength image extracted by said extraction means, by calculating the difference for each wavelength component corresponding to the designated one or more wavelength components, a second calculation means for calculating a difference specific wavelength image corresponding to the specified wavelength, the difference a particular wave of one or more channels corresponding to said second wavelength that is specified obtained by calculating the hand stages of Long Using the pixel values of the image, thus effectively accomplished by having a means for determining a class which belongs each point in the image.
Furthermore, the present invention relates to multi-spectral imaging system for classifying pixel group based on the spectral characteristics of the object, the object of the present invention, first with a channel of M kinds of different spectral sensitivity by using the first imaging device, a second imaging device for taking images of the channels corresponding to the designated one or more wavelengths, means for capturing the same of the object, the first the image of the object obtained by photographing by the imaging device, of the M-dimensional subspace component image of the object obtained by projecting between the subspace spanned by the basis functions of the M-dimensional, specified in step C1 Obtained by a first calculating means for calculating an image of one or a plurality of wavelength components corresponding to the determined wavelength and calculating a specific wavelength component image on the M-dimensional subspace; and the first calculating means. specific wave in front of Symbol M-dimensional part on the space was A component image, for the one or more wavelength components image obtained by imaging by the second imaging device, and means to extract the difference for each corresponding wavelength, designated extracted by the extraction means using the pixel values of one or more channels of differential specific wavelength image corresponding to the wavelength taken, by having a means for determining a class which belongs each point in the image, or the image of the wide-band can the broadband image capturing function, as a narrow-band image of the first imaging device and the second imaging device an imaging system and a narrowband image capturing function capable shooting of having sensitivity only to a specific wavelength that note by using, Or, the imaging system, red (R) · green (G) · blue and digital imaging apparatus for capturing in three primary colors (B), emit narrowband light at a particular wavelength to the target Do By being composed of a Koto lighting apparatus, or the special lighting apparatus thus it is effectively achieved by a light emitting diode (LED).

  The diagnostic method using a multispectral skin image according to the present invention is remarkable in that a numerical value related to the state of a skin lesion is calculated from an image of one or a plurality of specific channels selected from the “difference multispectral skin image”. There is. Therefore, according to the diagnosis method based on the multispectral skin image according to the present invention, even if the color of normal skin is different, the pathological condition can be stably quantified and the skin disease state can be accurately diagnosed. There is an excellent effect of being able to.

  In other words, in the diagnostic method based on the multispectral skin image according to the present invention, by using a specific wavelength component obtained by subtracting the spectral characteristics of the normal part of the patient's skin, individual differences in the color of the normal part, the state of sunburn, etc. The variation in spectral characteristics of normal skin due to can be canceled. Quantifying blood flow in inflammatory skin diseases such as psoriasis to measure the area of the lesion, analyzing the skin rash to determine the severity, and whether the skin disease is due to an autoimmune disease Can be determined numerically.

  According to the diagnostic method using a multispectral skin image according to the present invention, dermatological diagnosis is performed based on quantitative numerical values, and additional information is provided to cases that are difficult to judge from only images in remote diagnosis. In addition, there is an effect that it is possible to quantify and determine a therapeutic effect such as medication.

It is a flowchart for demonstrating suitable 1st Embodiment of the multispectral image processing method which concerns on this invention. It is a flowchart for demonstrating suitable 2nd Embodiment of the multispectral image processing method which concerns on this invention. It is a flowchart for demonstrating suitable 3rd Embodiment of the multispectral image processing method which concerns on this invention. It is a schematic diagram which shows an example of a structure of a multispectral camera. It is a conceptual diagram of the imaging system which comprises a wideband imaging function and a narrowband imaging function suitable for the production | generation of a difference specific wavelength image. 3 is a flowchart for explaining a preferred embodiment of a diagnostic method using a multispectral skin image according to the present invention. In this invention, it is a conceptual diagram which shows the relationship between a wavelength and the component extracted. It is a figure which shows an example of the display method of the analysis result of the inflammatory skin disease obtained by applying this invention. In Example 2, it is a conceptual diagram which shows the class classification | category in three-dimensional space. In Example 2, it is a figure which shows the example of the presentation method of the acne vulgaris (acne) diagnostic result.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

  First, the multispectral image processing method according to the present invention will be described.

  The multispectral image processing method according to the present invention uses a multispectral image obtained by photographing an image of a plurality of wavelength bands using an object (target object) having a characteristic in the wavelength direction as a subject. In the following, when there are spatial unevenness or individual differences in the spectrum of the object to be imaged), the influence is removed, and a specific part (specific area) included in the object imaged based on the spectral characteristics ) Can be visualized and extracted, or the object can be classified.

  In short, the multispectral image processing method according to the present invention is one of component images obtained by projecting a multispectral image having N channels in the wavelength direction onto a subspace spanned by M-dimensional (M <N) basis functions. A difference is calculated for each corresponding wavelength component between one or a plurality of specified wavelength component images and the corresponding wavelength component image of the original multispectral image, and a difference image corresponding to the specified wavelength is obtained. Thus, the greatest feature is to determine the class to which each point in the image belongs by using the pixel values of the obtained difference image of one or a plurality of channels.

  By the way, when the variation in the background color is expressed by a linear combination of a plurality of functions, Equation 1 is as follows.

本発明では、画像の全体または一部の領域で主成分分析を行い、得られた主成分ベクトルのうちで指定した数の主成分ベクトルの和で現される成分を基準画像として、撮影された画像から基準画像を差し引くことにより、数５の第２項の成分をキャンセルする。これによって、下地の色が変化しても、その影響を受けずにスペクトルの特徴だけを抽出することが可能になる。なお、基準画像は、画像の全体または一部の領域で主成分分析を行って得られた主成分ベクトルでなくてもよく、他の画像を主成分分析して得られる主成分ベクトルや主成分分析以外の方法によって得られる基底ベクトルによって構成することもできる。

In the present invention, the principal component analysis is performed on the whole or a part of the image, and the component represented by the sum of the specified number of principal component vectors among the obtained principal component vectors is taken as a reference image. By subtracting the reference image from the image, the component of the second term of Equation 5 is canceled. As a result, even if the background color changes, it is possible to extract only the spectral features without being affected by the change. Note that the reference image does not have to be a principal component vector obtained by performing principal component analysis on the entire image or a partial region of the image, and a principal component vector or principal component obtained by performing principal component analysis on another image. It can also be constituted by basis vectors obtained by a method other than analysis.

  A preferred embodiment of the multispectral image processing method according to the present invention will be described as follows.

A first preferred embodiment of the multispectral image processing method according to the present invention is an image processing method for classifying a pixel group based on spectral characteristics of an object to be imaged, and includes the following steps. .
Step A1:
A multispectral image having an N channel (N is an integer of 2 or more) in the wavelength direction is taken with an object having characteristics in the wavelength direction as a subject.
Step A2:
The multispectral image of the object obtained by photographing in step A1 is projected onto a subspace spanned by an M-dimensional (M <N, M is an integer) basis function, and an M-dimensional subspace component of the object Get an image.
Step A3:
A difference multispectral image of the object is obtained by calculating a difference between the multispectral image of the object obtained in step A1 and the M-dimensional subspace component image of the object obtained in step A2.
Step A4:
From the differential multispectral image of the object obtained in step A3, a differential image of one or a plurality of channels corresponding to the designated wavelength (this “differential image” is also referred to as “differential specific wavelength image”) is extracted. To do.
Step A5:
Using the pixel values of the difference image of one or a plurality of channels extracted in step A4, the class to which each point (each pixel) in the image belongs is determined, that is, the pixel group is classified.

  In the multispectral image processing method of the present invention having the above steps A1 to A5, a multispectral image having N channels is projected onto a subspace spanned by an M-dimensional (M <N) basis function, and an M-dimensional subspace component is projected. Since the image is obtained, the difference between the multispectral image having N channels and the M-dimensional subspace component image when the M-dimensional basis function represents a typical spectral distribution of the object to be imaged. The differential multispectral image representing the image has only components that deviate from the typical spectral distribution of the object to be imaged.

  In the multispectral image processing method of the present invention having steps A1 to A5, the class to which each point (each pixel) in the image belongs is determined using only a specific channel of the differential multispectral image. Therefore, even if there is a spatial unevenness or individual difference in the typical spectrum of the object to be photographed, processing such as extraction of a specific area in the image or classification of pixel groups in the image is canceled. It can be performed.

  Therefore, since the multispectral image processing method of the present invention having steps A1 to A5 uses only components deviating from the typical spectral distribution of the object to be imaged, detection of abnormal parts in the object to be imaged, region extraction, etc. This is particularly effective.

  Further, in the multispectral image processing method of the present invention having steps A1 to A5, it is not necessary to use Lambert-Bear's law, so that it is not necessary to obtain an absolute spectral reflectance. Therefore, a simple imaging device such as a normal camera can be used, and photographing is easy. Even if the optical path length distribution for each wavelength is unknown, the multispectral image processing method of the present invention can be used.

  In the multispectral image processing method of the present invention, as the M-dimensional basis function, the entire multispectral image having N channels obtained by photographing at step A1 or N obtained by photographing at step A1. If the spectral data of a specified region of a part of a multispectral image with channels is subjected to principal component analysis, and M components are selected from the principal components that have a large contribution to the original data used in the principal component analysis, Good.

  In addition, from the multispectral image different from the multispectral image having N channels obtained by capturing in step A1, the spectral data of the specified region of the entire image or a part of the image is similarly analyzed. Of these, one selected from M principal components having a large contribution to the original data used in the principal component analysis may be used. At this time, a plurality of images can be used as another multispectral image. In this way, when the distribution of the typical spectrum of the object can be limited in advance, an M-dimensional basis function is created and stored from another multispectral image prepared in advance or a spectrum data group acquired by point measurement. You can also keep it.

  Further, in the multispectral image processing method of the present invention, if one of the basis functions is the spectral distribution of illumination light, the surface reflection component can be canceled, so that the influence of the surface reflection can be removed and processed. Is possible.

  Furthermore, in the multispectral image processing method of the present invention, the multispectral image having N channels in the wavelength direction may be obtained by calculating the logarithm of the value of the image taken by the camera. At this time, if one of the basis functions is the spectral distribution of the illumination light, the component of the illumination light can be canceled, so images taken under different types of illumination light can be processed in the same way. It is.

In the second preferred embodiment of the multispectral image processing method according to the present invention, a process equivalent to the first preferred embodiment of the multispectral image processing method according to the present invention is performed to obtain an M-dimensional subspace component image ( That is, the image processing method is performed more efficiently by omitting step A2), and is characterized by having the following steps.
Step B1:
A multispectral image having an N channel (N is an integer of 2 or more) in the wavelength direction is taken with an object having characteristics in the wavelength direction as a subject.
Step B2:
An M-dimensional subspace of the object obtained by projecting the multispectral image of the object obtained by imaging in step B1 onto a subspace spanned by M-dimensional (M <N, M is an integer) basis function. Among the component images, an image of one or more designated wavelength components is calculated to obtain a specific wavelength component image on the M-dimensional subspace.
Step B3
An image of a wavelength component corresponding to one or a plurality of wavelength components specified in Step B2 from the multispectral image of the object obtained by photographing in Step B1 (this “image” is also referred to as “specific wavelength image”). ).
Step B4:
By calculating the difference for each wavelength component corresponding to the specified wavelength component, for the specific wavelength component image on the M-dimensional subspace obtained in step B2 and the specific wavelength image extracted in step B3, A difference image corresponding to the designated wavelength (this “difference image” is also referred to as a “difference specific wavelength image”) is obtained.
Step B5:
Using the pixel value of the difference image corresponding to the designated wavelength obtained in step B4 (that is, the difference image of one or more channels corresponding to the designated wavelength), each point (each pixel in the image) ) Belong to, that is, classify the pixel group.

  As described above, in the multispectral image processing method of the present invention having steps A1 to A5, a captured multispectral image is projected onto a subspace spanned by M-dimensional (M <N) basis functions, and an M-dimensional portion is projected. After obtaining the spatial component image, the difference between the captured multispectral image and the M-dimensional subspace component image is calculated to obtain a differential multispectral image.

  However, in the multispectral image processing method of the present invention having steps B1 to B5, the differential image of one or a plurality of channels corresponding to the designated wavelength component is obtained from the differential multispectral image. Object obtained by projecting the multispectral image of the object obtained by photographing in step B1 onto the subspace spanned by the M-dimensional (M <N) basis function without obtaining the M-dimensional subspace component image If only one or a plurality of specified wavelength component images are calculated from the M-dimensional subspace component image of the object to obtain a specific wavelength component image on the M-dimensional subspace, steps A1 to A5 are included. Compared to the multispectral image processing method of the present invention, the same calculation can be performed more efficiently.

  In the multispectral image processing method of the present invention having the above-described steps B1 to B5, the difference is calculated after the multispectral image of the object obtained by photographing is once projected onto the subspace spanned by the M-dimensional basis function. However, the present invention is not limited to this. For example, the projection and the difference on the subspace spanned by the M-dimensional basis function can be performed by a single matrix / vector product operation.

は下記数７になる。 Specifically, if the following Equation 6 is used, the difference specific wavelength component image

Becomes the following formula 7.

ただし、

はＮ×Ｎの単位行列である。また、

は、特定波長成分数をＬとするとき、Ｌ行Ｎ列の行列で、各行の値が、特定波長に対応する列のみ１で、あとは０の値を持つ行列である。

However,

Is an N × N identity matrix. Also,

Is a matrix of L rows and N columns where the number of specific wavelength components is L, and each row has a value of 1 only for the column corresponding to the specific wavelength, and a value of 0 after that.

を定義すれば、

をかけるだけで、差分特定波長成分画像を得ることができる。 Therefore, an L × N matrix as shown in Equation 8 below.

If we define

The difference specific wavelength component image can be obtained simply by applying.

本発明に係るマルチスペクトル画像処理方法の好適な第３実施形態は、ステップＡ１〜Ａ５を有する本発明のマルチスペクトル画像処理方法と同等な結果を、改良された画像撮影装置（つまり、後述する図５に示される撮像システム）を用いることによって、さらに効率的に得る画像処理方法であり、以下のステップを有することを特徴とする。
ステップＣ１：
Ｍ種類以上の異なる分光感度のチャネルを持つ第１の撮像装置と、指定された一つまたは複数の波長に対応するチャネルの画像を撮影するための第２の撮像装置を用いて、同一の対象物を撮影する。
ステップＣ２：
第１の撮像装置で撮影して得られた対象物の画像を、Ｍ次元の基底関数の張る部分空間に投影して得られる対象物のＭ次元部分空間成分画像のうち、ステップＣ１で指定された波長に該当する一つまたは複数の波長成分の画像を計算して、Ｍ次元部分空間上での特定波長成分画像とする。
ステップＣ３：
ステップＣ２で得られたＭ次元部分空間上での特定波長成分画像と、第２の撮像装置で撮影して得られた一つまたは複数の波長成分画像とについて、それぞれ対応する波長ごとに差分を計算する。
ステップＣ４：
ステップＣ３で抽出された指定された波長に対応する一つまたは複数のチャネルの差分画像（この「差分画像」を「差分特定波長画像」とも称する）の画素値を用いて、画像内の各点（各画素）の属するクラスを判定し、つまり、画素群の分類を行う。

The third preferred embodiment of the multispectral image processing method according to the present invention provides an improved image photographing apparatus (that is, a diagram to be described later) that is equivalent to the multispectral image processing method of the present invention having steps A1 to A5. 5 is an image processing method that is obtained more efficiently by using the imaging system shown in FIG. 5, and has the following steps.
Step C1:
Using the first imaging device having M or more types of channels having different spectral sensitivities and the second imaging device for capturing images of channels corresponding to one or more designated wavelengths, the same object Shoot things.
Step C2:
Of the M-dimensional subspace component image of the object obtained by projecting the image of the object obtained by photographing with the first imaging device onto the subspace spanned by the M-dimensional basis function, it is designated in step C1. An image of one or a plurality of wavelength components corresponding to the determined wavelength is calculated to obtain a specific wavelength component image on the M-dimensional subspace.
Step C3:
The difference between the specific wavelength component image on the M-dimensional subspace obtained in step C2 and one or a plurality of wavelength component images obtained by photographing with the second imaging device is obtained for each corresponding wavelength. calculate.
Step C4:
Using the pixel values of the difference image of one or more channels corresponding to the specified wavelength extracted in step C3 (this “difference image” is also referred to as “difference specific wavelength image”), each point in the image The class to which (each pixel) belongs is determined, that is, the pixel group is classified.

  The multispectral image processing method of the present invention having steps C1 to C4 is a method for obtaining the same effect as the multispectral image processing method of the present invention having steps A1 to A5.

  That is, the multispectral image processing method of the present invention having steps C1 to C4 is based on the premise that M-dimensional basis functions are separately obtained in advance, and the multispectral image processing method of the present invention having steps A1 to A5. Image processing using an image photographed by a first imaging device having M or more types of channels having different spectral sensitivities instead of an M-dimensional subspace component image obtained by projecting onto a subspace spanned by an M-dimensional basis function Is the method.

  In addition, the first imaging device having M or more types of channels having different spectral sensitivities does not need to have narrow-band spectral sensitivities. For example, when M is 3 or less, a normal imaging sensor having RGB three primary colors is used. A color camera can be used as the first imaging device.

  On the other hand, as the second imaging device for capturing an image of a channel corresponding to one or a plurality of specified wavelengths, an imaging device used as the first imaging device can be used. In short, for example, when a color camera used as a first imaging device is used as a second imaging device and an object is photographed, an LED or laser light source having a wavelength corresponding to a designated wavelength is turned on. An image of a channel corresponding to a designated wavelength can be taken by illuminating or the like.

  For this reason, if the multispectral image processing method of this invention which has step C1-C4 is used, there exists an effect that an imaging device can be simplified further.

  Also, three preferred embodiments of the above-described multispectral image processing method according to the present invention, that is, the multispectral image processing method of the present invention having steps A1 to A5, and the multispectral image of the present invention having steps B1 to B5. The procedure of the multispectral image processing method of the present invention having the processing method and steps C1 to C4 is represented by flowcharts in FIG. 1, FIG. 2, and FIG.

  Here, the multispectral image processing method of the present invention will be described in more detail as follows based on the processing flow shown in FIG. 1, FIG. 2 and FIG.

  As an example of the multispectral camera system for capturing multispectral images used in the present invention, the configuration of a rotary filter type multispectral camera that captures the wavelength range of the visible region with 16 channels is shown in FIG. .

  This multispectral camera shown in FIG. 4 includes a dedicated optical unit 10 having a CCD with 4 million pixels, an interference filter 20 having 16 different characteristics, a motor 30, a motor controller 40, and a filter controller. 50, a camera controller 60, a computer I / F 70, a computer 80 for controlling the entire camera, an observation monitor 90, and an operation monitor 100. The captured image is stored in the computer 80 via the USB interface.

  In the present invention, the number of channels of the multispectral camera for capturing a multispectral image only needs to be sufficient to obtain the spectral component characteristics of the image of the wavelength component to be extracted and the background color. The required number of 16 channels or less may be used.

  The multispectral camera shown in FIG. 4 has a structure for acquiring an image that completely covers the filter section as the 17th channel in order to acquire the dark current component of the CCD. By subtracting the dark current level estimated from the 17th channel from the above image, the influence of dark current is removed. This dark current correction may be performed by subtracting a separately measured dark current level.

  In addition, when the illumination light used when capturing an image is constant, the effect of the present invention can be obtained without correcting the influence of the illumination light, but the image processing of the present invention can be performed without depending on the illumination environment. In order to perform the correction, the illumination environment can be corrected by the following method.

  First, a standard white plate coated with magnesium oxide or the like is photographed with a multispectral camera. The white level for each channel is acquired from the captured 16 channel pixel values, and the pixel value for each channel of the multispectral image in which the subject is captured is divided by the white level value. The spectral distribution of the illumination light may be obtained from a standard white plate taken with a spectral radiance meter or the like.

  Here, the principal component analysis used in the multispectral image processing method of the present invention will be described.

で表現したときに、その主成分分析を行うことで、

は下記数９のようにＮ個の主成分の和で表現することができる。ただし、

は主成分ベクトルで、α_ｉは係数である。 An N-channel multispectral image (that is, a multispectral image signal value corrected by the above-described “dark current correction” processing or “illumination environment correction” processing) is an N-dimensional vector.

By performing the principal component analysis when expressed in

Can be expressed as the sum of N principal components as shown in Equation 9 below. However,

Is a principal component vector, and α _i is a coefficient.

本発明において、主成分分析を行う対象としては、次の３通りが考えられる。
(１) 撮影されたマルチスペクトル画像全体
(２) 撮影されたマルチスペクトル画像のうち、指定した領域
(３) 別途撮影された一枚または複数枚のマルチスペクトル画像
(１)及び(２)の場合は、撮影されたマルチスペクトル画像から主成分ベクトルを求めるので、画像ごとに下地の色を表す基底ベクトルが異なっていても対応できる。画像ごとに基底ベクトルを変える必要が無いときは、(３)のように別途撮影された一枚または複数枚のマルチスペクトル画像を用いることができる。

In the present invention, the following three types can be considered as subjects for principal component analysis.
(1) Entire captured multispectral image
(2) Specified area of the captured multispectral image
(3) One or more multispectral images taken separately
In the case of (1) and (2), the principal component vector is obtained from the captured multispectral image, so that it can be handled even if the base vector representing the background color is different for each image. When there is no need to change the basis vector for each image, one or a plurality of multispectral images photographed separately as in (3) can be used.

  In the case of (1), it is not necessary to specify a region. However, since the base vector changes depending on the range of the captured image, a stable result may not be obtained. In the case of (2), it is necessary to manually or automatically select a reference area. It is desirable that the reference area is an area that includes only the background color as much as possible. In other words, if the region to be used as a reference is appropriately selected, the method (2) gives the best result.

  As a result of the principal component analysis, M principal component vectors are selected in descending order of eigenvalues, and the selected M principal component vectors are set as basis vectors. This basis vector can be considered as representing the background color of the object to be imaged. Accordingly, the dimension number M is selected as the minimum number necessary to represent the background color of the object to be photographed.

  As described above, in the multispectral image processing method of the present invention, basis vectors are obtained using principal component analysis. However, the present invention is not limited to this, and the basis vectors are determined using other methods. Anyway.

  Next, calculation of an M-dimensional subspace component image in the multispectral image processing method of the present invention will be described.

を、上述したように主成分分析によって求められたＭ個の基底ベクトルの張る部分空間に投影することによって、Ｍ次元部分空間成分画像を得るようにしている。これを数式で表現すると、下記数１０になる。ただし、ｔはベクトルの転置を表す。 In the present invention, multispectral image signal values after correction such as “dark current correction” and “illumination environment correction” described above.

Is projected onto a subspace spanned by M basis vectors obtained by principal component analysis as described above, thereby obtaining an M-dimensional subspace component image. When this is expressed by a mathematical formula, the following formula 10 is obtained. However, t represents transposition of a vector.

つまり、数１０で表す処理をマルチスペクトル画像の各画素に対して適用することによって、Ｍ次元部分空間成分画像を算出することができる。

That is, an M-dimensional subspace component image can be calculated by applying the processing expressed by Equation 10 to each pixel of the multispectral image.

  Next, calculation of the differential multispectral image will be described.

は、前述した「暗電流補正」や「照明環境補正」などの補正後のマルチスペクトル画像信号値

と、数１０から算出されるＭ次元部分空間成分画像

との差をとることによって、下記数１１に表されるように、算出することができる。 Differential multispectral image

Is the multispectral image signal value after correction such as “Dark current correction” and “Lighting environment correction” described above.

And an M-dimensional subspace component image calculated from Equation 10

Can be calculated as shown in Equation 11 below.

なお、この差分マルチスペクトル画像のうち、実際に本発明の画像処理に用いるのは、指定された特定波長成分のみであるので、数１０や数１１を用いて、全ての波長成分に対して計算を行わなくても、指定された特定波長成分のみについて差分を計算すれば十分である。つまり、指定された特定波長成分のみについて差分を計算する方法が、ステップＢ１〜Ｂ５を有する本発明のマルチスペクトル画像処理方法（図２参照）である。

Of these differential multispectral images, only the specified specific wavelength component is actually used for the image processing of the present invention, so calculation is performed for all wavelength components using Equations 10 and 11. Even if it is not performed, it is sufficient to calculate the difference only for the specified specific wavelength component. That is, the method of calculating the difference only for the specified specific wavelength component is the multispectral image processing method of the present invention (see FIG. 2) having steps B1 to B5.

  Next, specific wavelength component designation will be described.

  In the present invention, the specific wavelength component designation means selecting an image of one or a plurality of wavelength bands from the differential multispectral image obtained in Expression 11. The wavelength band is selected using a wavelength band in which the feature of interest appears most frequently. As an example, when extracting the reddish part of the skin, it is preferable to use a wavelength component near 550 nm, which is characteristic of hemoglobin absorption.

  Next, pixel group classification will be described.

の値に基づいて、閾値処理などの方法によって行う。複数の波長成分を指定した場合には、複数の波長成分の値で構成される多次元空間において、多変量解析の方法などを適用することによって行う。これによって、下地の色のばらつきは、差分マルチスペクトル画像を求める際の上位主成分で表されるＭ次元部分空間成分画像に吸収されるので、下地の色のばらつきに影響されず、スペクトルの特徴だけを抽出することができるようになる。 In the present invention, the classification of the pixel group is the specified specific wavelength component.

This is performed by a method such as threshold processing based on the value of. When a plurality of wavelength components are designated, it is performed by applying a multivariate analysis method or the like in a multidimensional space composed of a plurality of wavelength component values. As a result, the background color variation is absorbed by the M-dimensional subspace component image represented by the upper principal component when obtaining the differential multispectral image. Will be able to extract only.

  By applying threshold processing to an image of a specific wavelength component obtained by applying Equation 11 to an image near 550 nm, where the number of base dimensions M used to actually construct the subspace is 2. It becomes possible to extract a reddish portion where a slight color difference is visually recognized. At this time, if the difference from the M-dimensional subspace component image is not taken, the reddish part may not be extracted well due to the influence of the background color.

  By the way, as described above, in the multispectral image processing method of the present invention having steps C1 to C4, the first imaging device having M or more types of channels having different spectral sensitivities, and one or more designated wavelengths. Is used, but the first imaging device and the second imaging device do not need to be separate imaging devices. For example, An imaging device (imaging system) having a wideband image capturing function and a narrowband image capturing function may be used. An imaging apparatus (imaging system) having such a broadband imaging function and a narrowband imaging function is suitable for generating a difference specific wavelength image.

  An imaging apparatus (that is, an imaging system having a wideband imaging function and a narrowband imaging function) suitable for generating a differential specific wavelength image used in the present invention will be described.

  FIG. 5 is a conceptual diagram of an imaging system suitable for generating a differential specific wavelength image used in the present invention. As shown in FIG. 5, an imaging system suitable for generating a differential specific wavelength image is a wide-band image taken with, for example, three general primary colors of red (R), green (G), and blue (B). The imaging system includes a wide-band image capturing function for capturing images and a narrow-band image capturing function for capturing a narrow-band image having sensitivity only at a specific wavelength of interest.

  As a suitable example of such an imaging system, a digital camera that captures images with three general colors of red (R), green (G), and blue (B), and light of a narrow band at a wavelength of interest, for example, are emitted. It consists of a special lighting device such as a light emitting diode (LED). Then, using a digital camera, an image captured under normal illumination (ie, a broadband image) and an image captured under illumination of a special illumination device (eg, LED) (ie, a narrowband image) Based on this, a difference specific wavelength image can be generated.

  The flow of a method for generating a difference specific wavelength image using the imaging system as shown in FIG. 5 is shown in FIG. That is, in the multispectral image processing method of the present invention having steps C1 to C4, a difference specific wavelength image is generated based on a wideband image and a narrowband image captured using the imaging system shown in FIG. Can do.

  In short, using the imaging system shown in FIG. 5 and when the multispectral image processing method of the present invention having steps C1 to C4 is used, an object having properties similar to those of an object to be photographed. The principal component analysis is performed on the multispectral image or the spectrum such as the spectral reflectance obtained by the point measurement to acquire the principal component vector in advance. Then, an M-dimensional subspace component image or an image of a specific wavelength component of interest on the M-dimensional subspace is calculated from the wideband image captured by the imaging system shown in FIG. 5 as follows.

That is, when the spectral sensitivity of the broadband n-channel is S _n (λ) and a model such as Equation 1 is established, the captured pixel value x _n of the n-channel is expressed by the following Equation 12. .

ここで、抽出対象の特徴成分ａが小さいとき、または、Ｓ_ｎ(λ)とｖ(λ)とがスペクトル空間上でほぼ直交しているときに、数１２の第１項を無視することができる。

Here, when the feature component a to be extracted is small, or when S _n (λ) and v (λ) are substantially orthogonal on the spectrum space, the first term of Expression 12 may be ignored. it can.

Further, u _k (λ) can be considered as a principal component vector in the wavelength space. At this time, Expression 12 can be rewritten in the form of a matrix / vector product as represented by Expression 13 below.

下地の色の分布を表す基底関数の次元数Ｋが、広帯域画像のチャネル数Ｊよりも小さければ、行列

の擬似逆行列Ｔ^＋を用いて、

すなわちｂ_ｋを推定することができる。また、ｕ_ｋ(λ)が波長空間上での主成分ベクトルであれば、特定波長λ_０での部分空間成分画像は、下記数１４に基づいて求めることができる。

If the dimension number K of the basis function representing the distribution of the background color is smaller than the channel number J of the wideband image, the matrix

Using the pseudo inverse matrix T ⁺ of

That is, b _k can be estimated. If u _k (λ) is a principal component vector in the wavelength space, the subspace component image at the specific wavelength λ ₀ can be obtained based on the following _equation (14).

仮に、図５に示された撮像システムの狭帯域撮影機能によって得られる画像をｇ_ｂ(λ_０)というように単一波長での値で表したとすると、波長λ_０での差分特定波長画像Δｇ(λ_０)は、下記数１５に基づいて求めることができる。

If the image obtained by the narrow-band imaging function of the imaging system shown in FIG. 5 is expressed as a value at a single wavelength such as g _b (λ ₀ ), a differential specific wavelength image at wavelength λ ₀ is assumed. Δg (λ ₀ ) can be obtained based on the following formula 15.

実際の図５に示された撮像システムの狭帯域撮影機能では、分光感度に幅を持つので、それを考慮に入れるには、数１４を下記数１６に置き換えればよい。ここで、Ｓ_ｂ(λ)はλ_０に中心波長を持つ狭帯域チャネルの分光感度である。

In the actual narrow-band imaging function of the imaging system shown in FIG. 5, the spectral sensitivity has a width, and in order to take it into consideration, it is sufficient to replace the equation 14 with the following equation 16. Here, S _b (λ) is the spectral sensitivity of a narrow band channel having a center wavelength at λ ₀ .

次に、本発明に係るマルチスペクトル皮膚画像による診断方法について説明する。

Next, a diagnostic method using a multispectral skin image according to the present invention will be described.

  The diagnostic method using a multispectral skin image according to the present invention is a diagnosis by a dermatologist by quantifying and displaying the state of a skin disease based on a multispectral skin image obtained by photographing the skin as a subject. In more detail, a multispectral skin image obtained by imaging a skin lesion using a multispectral image capturing apparatus having sensitivity in different wavelength bands, or separately captured Multispectral skin reconstructed as the sum of a specified number of principal component vectors out of the principal component vectors obtained by performing principal component analysis on part or the entire region of a multispectral skin image including the normal part of the skin The difference between the image and the multispectral skin image obtained by imaging the skin lesion was calculated and selected according to the type of skin disease Using the differential image of one or more channels, it is possible to extract the lesion part of the skin (hereinafter also referred to as the lesion area) and calculate the area, number, or shape of the extracted lesion area It is.

  In short, in the diagnostic method using a multispectral skin image according to the present invention, a multichannel image capturing apparatus having sensitivity in different wavelength bands is used to photograph a lesioned portion of the skin, and the captured multispectral skin image, or Principal component analysis was performed on a part of or the whole area of a multispectral skin image including the normal part of the skin photographed separately, and the result was reconstructed as the sum of the specified number of principal component vectors. By calculating the difference between the multispectral skin image and the multispectral skin image obtained by photographing the skin lesion, a “differential multispectral skin image” is obtained, and the obtained “differential multispectral image” Select the image of one or more channels specified by the type of skin disease from “Skin image” and select the selected image. In this way, lesions belonging to one or a plurality of categories are extracted, and the pathological condition (skin disease state) is digitized or imaged based on the area, number, or shape of the extracted lesion areas. It is characterized by diagnosing a skin disease by performing the conversion.

  A preferred embodiment of a diagnostic method using a multispectral skin image according to the present invention will be described as follows.

A preferred embodiment of the diagnostic method using a multispectral skin image according to the present invention is a diagnostic method using a skin image for analyzing and diagnosing the state of a skin disease based on a “difference multispectral skin image”. It is characterized by having.
Step D1:
One or more wavelengths are designated according to the type of skin disease to be diagnosed (that is, the disease name of the skin disease).
Step D2:
The subspace dimension number M representing the characteristics of the normal part of the skin to be diagnosed is designated.
Step D3:
A channel having spectral sensitivity in a wavelength band including one or a plurality of wavelengths designated in step D1, and a predetermined type of different spectrum necessary for obtaining a projection value on an M-dimensional subspace representing a normal part of the skin The skin to be diagnosed is imaged with an imaging device having a sensitivity channel.

Note that the projection value onto the M-dimensional subspace representing the normal part of the skin may be calculated including an image of one or a plurality of wavelength bands specified in step D1.
Step D4:
An M-dimensional subspace obtained by projecting the multispectral skin image obtained by photographing in step D3 onto a subspace spanned by an M-dimensional basis function representing the spectral characteristics of the normal part of the skin stored separately. Among the component skin images, an image of a wavelength component corresponding to one or a plurality of wavelengths specified in step D1 is obtained and set as a specific wavelength component skin image on the M-dimensional subspace.

The M-dimensional basis function representing the spectral characteristics of the normal part of the skin stored separately may be obtained from an image obtained by photographing the skin of the normal part of the same patient.
Step D5:
A specific wavelength skin image having a wavelength component corresponding to one or a plurality of wavelengths specified in step D1 is extracted from the multispectral skin image obtained by photographing in step D3.
Step D6:
Wavelength components corresponding to one or more wavelengths specified in step D1 for the specific wavelength component skin image on the M-dimensional subspace obtained in step D4 and the specific wavelength skin image extracted in step D5 A difference specific wavelength skin image corresponding to the designated wavelength is extracted by calculating the difference for each.
Step D7:
Each pixel or region in the image is classified into a plurality of classes using the pixel values of the differential specific wavelength skin image of one or more channels corresponding to the designated wavelength extracted in step D6.
Step D8:
Based on the characteristics of the class classified in step D7, the state of the skin disease is digitized or imaged for each region in the image.

  FIG. 6 is a flowchart illustrating a preferred embodiment of the above-described diagnostic method using multispectral skin images according to the present invention, that is, the procedure of the diagnostic method using multispectral skin images according to the present invention having steps D1 to D8.

  Next, an embodiment in which the diagnosis method based on the multispectral skin image of the present invention having steps D1 to D8 described above is actually applied to a specific skin disease will be described.

  Example 1 is an example in which the diagnostic method based on multispectral skin images of the present invention is applied to “diagnosis of blood flow in inflammatory skin diseases such as psoriasis”.

  In this embodiment, it is assumed that the skin to be diagnosed is photographed with an N-channel multispectral camera, but an imaging device that photographs by other methods may be used.

  FIG. 7 is a conceptual diagram showing the relationship between the wavelength and the extracted component in the present invention. As shown in FIG. 7, the main absorbers of the skin are hemoglobin and melanin. However, since light scattering is wavelength-dependent, the characteristics near the surface layer are reflected at short wavelengths, and at longer wavelengths than the dermis. The characteristics of the absorber in the deep part are easily reflected.

  The pathology of chronic inflammatory skin disease and angiogenesis are closely related, and the state of capillaries near the surface layer may be an indicator of disease state. In the case of Example 1 in which the diagnosis method based on the multispectral skin image of the present invention is applied to “diagnosis of blood flow of inflammatory skin disease”, the central wavelength is around 550 nm, which has a high correlation with blood components of capillaries near the surface layer. Specify as.

  On the other hand, in the case of Example 2 in which the diagnosis method based on the multispectral skin image of the present invention, which will be described later, is applied to “diagnosis of acne vulgaris”, it is sensitive to melanin near the surface layer. A certain wavelength band (around 430 nm) or a wavelength band (620 nm) sensitive to blood absorption at a slightly deeper site is used.

  In the case of Example 3 where the diagnosis method based on the multispectral skin image of the present invention is applied to “diagnosis of the possibility of autoimmune skin disease”, it is deeper than the 620 nm band used in Example 2. The vicinity of 710 nm is used as a wavelength band sensitive to the blood volume of the portion.

  Of course, in the present invention, not only the visible region but also the near-infrared wavelength band can be used.

  In the diagnosis method using a multispectral skin image of the present invention, a multispectral skin image of a skin to be diagnosed (that is, a portion to be diagnosed) and a multispectral skin image of normal skin (that is, a normal portion) are photographed, and normal M principal component vectors are obtained from the partial multispectral skin image.

  In addition, this multispectral skin image of the normal part is a part of the area determined to be normal skin when a part of the area included in the multispectral skin image of the diagnosis target part is determined to be normal skin. It is also possible to use.

  The normal skin component is represented by the top M principal components, for example, two basis functions, and the multispectral skin image obtained by photographing the skin to be diagnosed is converted into a subspace spanned by the two-dimensional basis function. Only the component near the center wavelength of 550 nm is extracted by projection, and this is used as the specific wavelength component skin image on the M-dimensional subspace.

  Next, a component near the center wavelength of 550 nm is extracted from the multispectral image obtained by photographing the skin to be diagnosed, and this is used as the specific wavelength skin image.

  Then, a difference image between the specific wavelength skin image and the specific wavelength component skin image on the M-dimensional subspace (that is, the difference specific wavelength skin image) is obtained. Class classification is performed by performing threshold processing on the obtained pixel values of the difference image. As a result, it is possible to extract a portion where the blood volume of the capillary blood vessel is increasing.

  An example of a method for quantifying and displaying the state of the skin disease is shown in FIG. As shown in FIG. 8, in this example, the density of pixels exceeding the threshold value is calculated for each small region in the image, and an image assigned with a different color according to the density is created and displayed. In addition, the area of the region where the density of pixels exceeding the threshold value takes a value in a specific range is calculated and displayed as a numerical value or a graph.

  Furthermore, if the diagnostic method using multispectral skin images of the present invention is applied to a plurality of images taken at different times for the same target case, it is possible to compare changes in the quantified blood volume distribution. Thus, the degree of progression of the disease state and the therapeutic effect can be determined.

  Next, Example 2 in which the diagnosis method based on multispectral skin images of the present invention is applied to “diagnosis of acne vulgaris severity” will be described.

  When applying the diagnosis method based on the multispectral skin image of the present invention to the diagnosis of acne vulgaris, that is, in the case of Example 2, three wavelengths such as 430 nm, 580 nm, and 620 nm are set as the central wavelengths. specify.

  The normal skin component is represented by the top M principal components, for example, two basis functions, and a multispectral skin image obtained by photographing the skin to be diagnosed is projected onto the subspace spanned by the two-dimensional basis function. Then, components near the center wavelengths of 430 nm, 580 nm, and 620 nm are extracted, respectively, and set as specific wavelength component skin images on the M-dimensional subspace.

  Then, components near the center wavelength corresponding to 430 nm, 580 nm, and 620 nm are extracted from the multispectral skin image obtained by photographing the skin to be diagnosed, and a difference image is obtained for each wavelength component.

  In this way, the pixel values of the obtained three types of difference images are classified into a plurality of classes in a three-dimensional space as shown in FIG. 9, and the classification results are imaged.

  That is, for each small region in the image, the number of classified pixels is calculated for each class, and acne regions are extracted according to the number. In addition, the number of pixels classified for each acne region is calculated, and the state of the rash (inflammatory state, suppuration, pigmentation, etc.) is analyzed from the result to determine the severity of acne.

  As shown in FIG. 10, the analysis result is calculated and displayed as a numerical value or a graph by calculating the area or the number of rashes for each determined acne region, degree of progression, and severity.

  Next, Example 3 in which the diagnostic method using multispectral skin images of the present invention is applied to “diagnosis of the possibility of autoimmune skin disease” will be described.

  Skin rashes from autoimmune skin diseases such as collagen disease are often accompanied by subtle skin changes that are sometimes misdiagnosed by specialists.

  When applying the diagnostic method based on multispectral skin images of the present invention to diagnosis of the possibility of autoimmune skin diseases such as collagen disease (for example, dermatomyositis), that is, in the case of Example 3, for example, 430 nm. Three wavelengths such as 580 nm and 710 nm are designated as center wavelengths.

  The normal skin component is represented by the top M principal components, for example, two basis functions, and a multispectral skin image obtained by photographing the skin to be diagnosed is projected onto the subspace spanned by the two-dimensional basis function. Then, the components near the center wavelengths of 430 nm, 580 nm, and 710 nm are extracted, respectively, and set as specific wavelength component skin images on the M-dimensional subspace.

  Then, components near the center wavelength corresponding to 430 nm, 580 nm, and 710 nm are extracted from the multispectral skin image obtained by photographing the skin to be diagnosed, and a difference image is obtained for each wavelength component.

  In this way, the pixel value of the lesion is extracted from the three types of difference images obtained. Compare the distribution of the extracted pixel values of the lesion and the reference data stored separately, and based on the comparison result, how much features unique to autoimmune skin diseases are included, or inflammatory skin diseases The degree to which a characteristic characteristic is included is displayed as a numerical value, a graph, or an image.

  In addition, when the multispectral image processing method according to the present invention described above is applied to the diagnosis of a dermatologist, that is, the diagnosis method using the multispectral skin image of the present invention has been described as an example, the present invention is not limited thereto. For example, the subject to be photographed is a fresh fish or fruit, and can be applied to check the pain of fresh fish or fruit.

Claims (10)

Using multispectral image image obtained by photographing the object with the features in the wavelength direction as a subject, by image analysis, a multispectral image processing method of extracting the feature region of the object,
Using the image capturing apparatus for a multi-channel having sensitivity to different wavelength bands, a first step of taking a first part portion of the object,
Multispectral image image captured in the first step, or, in some or all regions of the multispectral image image including a second portion of the object which is separately shot, the second performing principal component analysis Steps,
Of the principal component vector obtained by the second step, the multispectral image image reconstructed, the first portion of the object in the first step taken as the sum of a specified number of principal component vectors by calculating a difference between the multispectral image image obtained Te, a third step of obtaining a difference multispectral image image,
Of the difference multispectral image image obtained in the third step, a fourth step of selecting an image for the specified the one or more channels,
For the selected image in the fourth step, by performing the image processing to extract the characteristic region of the object, the area of the extracted feature region, and a fifth step intends row the number or shape,
Multispectral image processing method characterized by having a. Using multispectral image image obtained by photographing the object with the features in the wavelength direction as a subject, by image analysis, a multispectral image processing method of extracting the feature region of the object,
Step D1 for designating one or more wavelengths,
Designating a dimension M of the subspace representing the characteristics of the predetermined part of the object ;
A channel having spectral sensitivity in a wavelength band including one or a plurality of wavelengths specified in step D1, and a predetermined type of a necessary type for obtaining a projection value onto an M-dimensional subspace representing a predetermined portion of the object. Step D3 of photographing the object with an imaging device having different spectral sensitivity channels;
The multispectral image image obtained by photographing in step D3, obtained by projecting the subspace spanned by the basis functions of the M-dimensional representative of the spectral signature of a predetermined portion of the object which is separately stored, M of dimensional subspace formed fractionation image, seeking an image of a wavelength component corresponding to one or more wavelengths that the designated, step D4 of the specific wavelength formation fractions image on the M-dimensional subspace,
From obtained by photographing in step D3 the multispectral image image, a step D5 to extract specific wave Nagaga image wavelength component corresponding to one or more wavelengths that the specified,
A specific wavelength formation fractions image on the M-dimensional subspace obtained in step D4, the said specific wave Nagaga image extracted in step D5, corresponding to the designated one or more wavelengths by calculating the difference for each wavelength component, and step D6 of extracting difference specific wave Nagaga image corresponding to the specified wavelength,
Using the pixel values of the differential specific wave Nagaga image of one or more channels corresponding to the wavelengths specified have been extracted in step D6, classifies each pixel or region in the image into a plurality of classes Step D7 When,
Extracting a feature region of the object based on the features of the class classified in step D7; and
Multispectral image processing method characterized by having a. The projection value of the M-dimensional subspace representing a predetermined portion of the object, multispectral image image processing according to claim 2 for calculating including an image of the specified one or more wavelength bands at step D1 Method. The basis functions of the M-dimensional representative of the spectral signature of a predetermined portion of said object which is separately stored, multispectral according to claim 2 or claim 3 obtained from the image obtained by photographing a predetermined portion of the same object Image processing method. A multispectral image processing apparatus for classifying a pixel group based on spectral characteristics of an object,
Means for taking a multispectral image having an N channel (N is an integer of 2 or more) in the wavelength direction, using the target object characterized in the wavelength direction as a subject;
The multispectral image of the object obtained by photographing by the photographing means is projected onto a subspace spanned by M-dimensional (M <N, M is an integer) basis function, and the M-dimension of the object is projected. First calculating means for calculating a subspace component image;
By calculating the difference between the multispectral image of the object obtained by the photographing means and the M-dimensional subspace component image of the object obtained by the first calculating means, the object Second calculating means for calculating the difference multispectral image of
Means for extracting a differential specific wavelength image of one or a plurality of channels corresponding to a designated wavelength from among the differential multispectral images of the object obtained by the second calculation means;
Means for determining a class to which each point in the image belongs, using pixel values of the difference specific wavelength image of one or more channels extracted by the extraction means;
A multispectral image processing apparatus comprising: A multispectral image processing apparatus for classifying a pixel group based on spectral characteristics of an object,
Means for taking a multispectral image having an N channel (N is an integer of 2 or more) in the wavelength direction, using the target object characterized in the wavelength direction as a subject;
M of the object obtained by projecting a multispectral image of the object obtained by photographing by the photographing means onto a subspace spanned by M-dimensional (M <N, M is an integer) basis function. A first calculating means for calculating a specific wavelength component image on the M-dimensional subspace by calculating an image of one or more designated wavelength components among the dimensional subspace component images;
Means for extracting a specific wavelength image of a wavelength component corresponding to the specified one or more wavelength components from the multispectral image of the object obtained by photographing by the photographing means;
For the specific wavelength component image on the M-dimensional subspace obtained by the first calculation means and the specific wavelength image extracted by the extraction means, the specified one or more wavelength components A second calculating means for calculating a difference specific wavelength image corresponding to the designated wavelength by calculating a difference for each corresponding wavelength component;
Means for determining a class to which each point in the image belongs, using pixel values of the difference specific wavelength image of one or more channels corresponding to the designated wavelength obtained by the second calculation means;
A multispectral image processing apparatus comprising: A multispectral image processing system for classifying pixels based on spectral features of an object,
Using the first imaging device having M or more types of channels having different spectral sensitivities and the second imaging device for capturing an image of a channel corresponding to one or more designated wavelengths, the same Means for photographing the object;
Of the M-dimensional subspace component image of the object obtained by projecting the image of the object obtained by photographing with the first imaging device onto the subspace spanned by the M-dimensional basis function, Step C1 Calculating a specific wavelength component image on the M-dimensional subspace by calculating an image of one or a plurality of wavelength components corresponding to the wavelength specified in
Corresponding respectively to the specific wavelength component image on the M-dimensional subspace obtained by the first calculation means and one or more wavelength component images obtained by photographing with the second imaging device Means for extracting the difference for each wavelength to be
Means for determining a class to which each point in the image belongs, using pixel values of the difference specific wavelength image of one or more channels corresponding to the designated wavelength extracted by the extraction means;
A multispectral image processing system comprising: Broadband image capture function that can capture wideband images,
The narrow-band image photographing function capable of photographing a narrow-band image having sensitivity only at a specific wavelength of interest is used as the first imaging device and the second imaging device. Multispectral image processing system.   The imaging system includes a digital imaging device that captures images in three primary colors of red (R), green (G), and blue (B), and a special illumination device that emits narrow band light at the specific wavelength of interest. The multispectral image processing system of claim 8.   The multispectral image processing system of claim 9, wherein the special illumination device is a light emitting diode (LED).

Images