JP2004340978A - System and method for color classification and color unevenness inspection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a simply-configured color classification system at a low cost which also can withstand mechanical vibration and enables superior color classification, even if its spectrum varies, without limiting light source, as well as improvement of accuracy in color classification, and to provide a color unevenness inspection device which can quantify color unevenness. <P>SOLUTION: This color classification system is provided with an imaging means for imaging the reflectance spectrum of an object, a plurality of band-pass filters set up in between the object and the imaging means, each of which has different bandpass respectively, and a classification means which computes classification spectrum for statistical method-based classification from the reflectance prism of the object imaged by the imaging means and then uses this reflectance spectrum to sort the object. Especially for the classification means, an optimum classification diagnosis is used for determining classification of the object. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention performs a colorimetric process on an object using a multispectral image obtained through a plurality of band-pass filters, and mainly uses a color to classify the object using color and a color classification apparatus. The present invention relates to a classification method and a color non-uniformity inspection device for inspecting color non-uniformity of an object.

  Conventionally, a color identification device for identifying the color of an object has been used in the management of paint color and dyeing degree at a production site of each industry, or in the color measurement of a product, or in the color measurement of a subject in medical and academic fields. It's being used.

  In this type of color discriminating apparatus and colorimetric apparatus, for example, in a conventional technique as shown in Patent Document 1, two classes are classified by performing statistical processing on a reflection spectrum of an object. ing.

Specifically, the reflection spectrum of an object whose class is known is statistically processed using the Foley-Sammon transform (FS transform) method (see Non-Patent Document 1).
The FS conversion method is a method of classifying two classes. Specifically, a spectrum di for classification that maximizes a Fisher ratio (Fisher ratio) R (di) obtained by the following equation (1) is obtained. It is to ask.

^{R (di) = (di t} S1di) / (di t S2di) ... (1)
Here, di ... Classification spectrum
di ^t ... Classification spectrum (transposition)
S1 ... inter-class covariance matrix
S2... Intra-class covariance matrix The spectrum di for this classification is hereinafter referred to as a classification spectrum.

  Since this classification spectrum di has the same number of dimensions as the spectrum of the object, it should be accurately expressed as di (λ), but is simply expressed as di for simplicity.

  Then, two types of classification spectra that increase the Fisher ratio are obtained.

  The classification spectrum di that maximizes the Fisher ratio is the classification spectrum d1, and the classification spectrum d1 that maximizes the Fisher ratio among the spectra orthogonal to the classification spectrum d1 is the classification spectrum d2.

  The two classes are classified by projecting each data into the space formed by the classification spectra d1 and d2.

  The classification spectra d1 and d2 are obtained from the following equation (2).

d1 = α1 S2 ^-1 Δ
d2 = α2 S2 ⁻¹ [I− ( ^Δt S2 ⁻² Δ) / ( ^Δt S2 ⁻³ Δ) S / 2 ⁻¹ ] Δ
… (2)
Here, α1 and α2 are normalization coefficients, Δ is X1−X2 (difference spectrum between class 1 and class 2), and I is a unit matrix.

  In order to project each data into the space constituted by the classification spectra d1 and d2 obtained in this way, an inner product of the classification spectrum and the reflection spectrum of the object is obtained. When the reflection spectrum of the object is f (λ) (λ = wavelength), the inner products t1 and t2 can be expressed by the following equations.

t1 = f (λ) · d1
t2 = f (λ) · d2 (3)
Here, represents an inner product operation.

  In the technique disclosed in Patent Document 1, a classification boundary is determined from the values of t1 and t2 as shown in FIG. 20, and a filter having the characteristics of this classification spectrum is formed by using a diffraction grating 1 and a liquid crystal filter 2 as shown in FIG. Has been realized. In FIG. 21, reference numeral 3 denotes a lamp for a light source.

  By the way, the classification spectra d1 and d2 generally have complicated shapes as shown in FIG. 22, and since they take positive and negative values, the mounting accuracy of the diffraction grating 1, the liquid crystal filter 2 and the like is also strictly required.

  Further, since the light source is limited in advance in the device disclosed in Patent Document 1, it is not suitable for classification for different light sources, and good classification cannot be performed when the spectrum of the light source changes. Diffraction gratings also have the disadvantage of high cost.

In view of this, the present applicant has previously disclosed in Japanese Patent Application Laid-Open Publication No. H10-15095 a color classification process for an object using a multispectral image obtained through a plurality of bandpass filters, thereby simplifying the device configuration. An application has been filed for an invention relating to a color classification apparatus which is low-cost, can withstand mechanical vibrations, and can satisfactorily perform color classification even when its spectrum changes without limiting the light source.
JP-A-3-267726 JP-A-8-105799 (Japanese Patent Application No. 6-241614) Q. Tian, M .; Barbaro et al., "Image Classification by the Foley-Sammon Transform", Optical Engineering, Vol. 25, no. 7, 1986

  However, in the color classification apparatus according to Patent Literature 2, since classification determination is performed using only one type of classification determination method, there is a problem to be further improved particularly in terms of classification accuracy.

  That is, in the case of only one type of classification determination method, the classification determination performance is significantly reduced due to the distribution state of a plurality of objects to be classified in a multidimensional space, and the classification performance tends to deteriorate depending on the target. is there.

  In addition, when performing color non-uniformity inspection of an object using a conventional spectrometer or color difference meter, since only spot measurement can be performed, it is impossible to two-dimensionally inspect the color non-uniformity of the object with a single measurement. there were.

  Note that, in this case, if the measurement is performed in several degrees, a variation occurs for each measurement, so that the color unevenness of the target object cannot be accurately inspected.

  In addition, in the case of inspecting for color unevenness by RGB input from a color video camera, it is difficult to detect a subtle color difference due to the characteristics of the color filter used for the inspection. Color unevenness cannot be accurately inspected.

  The present invention has been made in view of the above points, and performs color classification processing of an object using a multispectral image obtained through a plurality of band-pass filters. It is cost-effective, can withstand mechanical vibrations, and can perform good color classification even when its spectrum changes without limiting the light source. It is an object of the present invention to provide a color classification device and a color classification method that can further improve the classification accuracy by using.

  Further, the present invention has been made in view of the above points, and performs a color non-uniformity inspection process for an object using a multispectral image obtained through a plurality of band-pass filters. In addition, it is possible to perform the color non-uniformity inspection of the object well even when the spectrum is changed without limiting the light source, at a low cost, and can withstand the mechanical vibration. An object of the present invention is to provide a color nonuniformity inspection apparatus capable of improving the accuracy of a color nonuniformity inspection process of an object.

  According to the present invention, in order to solve the above-described problems, a multi-spectral image capturing unit that captures reflected light of an object as a multi-spectral image having different bands, and an image of the target captured by the multi-spectral image capturing unit. Classifying means for calculating a classification spectrum which is a vector for classifying a specific object using a statistical method from the multispectral image data, and classifying the object in a plurality of classes using the classification spectrum. A color classification apparatus is provided, wherein the classification means includes a plurality of classification determination units for performing the classification determination of the plurality of classes by a plurality of types of classification determination methods different from each other.

  Further, according to the present invention, there is provided a color classification device, wherein the plurality of classification determination units are connected in series, and classification determinations based on a plurality of different classification determination methods are performed in a superimposed manner. .

  Further, according to the present invention, the classification unit further includes a class information database in which the class information of the object is stored in advance, and the plurality of classification determination units based on the class information from the class information database. A color classification device comprising a classification determination selection function unit for selecting a classification process and a narrowing-down method is provided.

  Further, according to the present invention, there is provided a color classification apparatus, wherein the classification unit further includes a determination result determination unit that determines one or more determination results of the plurality of classification determination units. .

  Further, according to the present invention, the determination result determination unit determines whether or not to perform the classification determination in the second classification determination unit based on the determination result of the first classification determination unit in the plurality of classification determination units. And a color classification device characterized by performing the processing.

  Further, according to the present invention, the judgment result judging section may be configured such that after the classification judgments in the first and second classification judging sections in the plurality of classification judging sections are performed in a superimposed manner, the second classification judging section A color classification device that determines whether or not to perform the classification determination again in the second classification determination unit based on the determination result of the above and performs processing.

  Further, according to the present invention, the classification unit further includes a learning control unit that performs predetermined learning on the multispectral image data in advance and controls a classification process by the classification unit based on the learning data. A color classification device is provided.

  Further, according to the present invention, there is provided a color classification apparatus, wherein the classification unit further includes a learning data updating unit that updates learning data while performing the classification determination.

  According to the present invention, the classification unit further extracts a feature amount from multispectral image data of the object captured by the multispectral image capturing unit, and the extracted feature amount is larger than a predetermined value. A color classification apparatus is provided, comprising: an image selection unit that selects only multispectral image data and selects one or a plurality of spectral images to be used for determination from among all spectral images.

  Also, according to the present invention, the feature quantity extracted by the image selecting means includes at least one of a contrast and a shading difference in a multispectral image captured by the multispectral image capturing means. A classification device is provided.

  Further, according to the present invention, the multispectral image pickup means includes optical means for forming reflected light of the object as a multispectral image having different wavelength bands of light on the multispectral image pickup means. And a color classification device characterized by the following.

  Also, according to the present invention, a multispectral image providing means for providing multispectral image data of an object, a feature amount extracting means for extracting a feature amount of the multispectral image data from the multispectral image providing means, Color unevenness determination means for performing color unevenness determination based on the feature amount from the amount extraction means; and a determination result output means for outputting a color unevenness determination result based on the color unevenness determination result from the color unevenness determination means. A color unevenness inspection apparatus is provided.

  Further, according to the present invention, the color nonuniformity determining means performs color nonuniformity detection processing and color nonuniformity degree calculation based on multispectral image data from the multispectral image providing means in parallel or permutation. And an unevenness degree calculating unit.

  Further, according to the present invention, the color non-uniformity determination means stores a color non-uniformity detection result and a color non-uniformity degree calculation result from the color non-uniformity detection processing means and the color non-uniformity degree calculation means, respectively. There is provided a color non-uniformity inspection device including a memory and a color non-uniformity degree calculation result storage memory.

  Further, according to the present invention, the color nonuniformity determining means is configured to determine at least one of a colorimetric value and a color difference value of a detection area in at least one of the color nonuniformity detection processing means and the color nonuniformity degree calculating means. There is provided a color non-uniformity inspection device including a colorimetric value calculation unit for calculating a colorimetric value.

  Further, according to the present invention, the color non-uniformity determination means includes a color non-uniformity detection processing result and a color non-uniformity degree calculation result from the color non-uniformity detection processing means and color non-uniformity degree calculation means, and a colorimetric value by the colorimetric value calculation means. And a color unevenness detection result storage memory for storing a color difference value of a detection area obtained from the memory, and a color unevenness degree calculation result storage memory.

  Further, according to the present invention, the color unevenness determining means further includes normal part data creating means for creating normal part data in advance, and the normal part data by the normal part data creating means is used for the color unevenness detection processing means and the color unevenness detection processing means. There is provided a color non-uniformity inspection device, which can be referred to at the time of color non-uniformity detection processing and color non-uniformity calculation by the non-uniformity degree calculating means.

  Further, according to the present invention, the color unevenness determining means includes a determination result determining means for determining a color unevenness detection processing result and a color unevenness degree calculation result from the color unevenness detection processing means and the color unevenness degree calculating means; Class data updating means for updating the class data based on the result of the determination by the result determining means; and new class creating means for creating a new class based on the class data by the class data updating means. A color nonuniformity inspection apparatus is provided, wherein a new class by the class creating means can be fed back at the time of color nonuniformity detection processing and color nonuniformity calculation by the color nonuniformity detection processing means and color nonuniformity degree calculating means.

  According to the present invention, the color unevenness determining means further includes a detection area determining means for determining a detection area in advance, and the detection area determined by the detection area determining means is used as the color unevenness detection processing means and the color unevenness degree calculating means. A color non-uniformity inspection apparatus, which can be referred to at the time of color non-uniformity detection processing and color non-uniformity calculation.

  Further, according to the present invention, the color unevenness determining means further includes a used multispectral image determining means for determining a used multispectral image in advance based on the multispectral image data from the multispectral image providing means. A color non-uniformity inspection apparatus is provided, wherein the multi-spectral image used by the multi-spectral image determining means can be referred to at the time of the color non-uniformity detection processing and the color non-uniformity degree calculating means and the color non-uniformity degree calculating means. You.

  Further, according to the present invention, the color unevenness determination means includes a detection area determination means for determining a detection area in advance, and a use area for previously determining a used Malte spectrum image based on multispectral image data from the multispectral image providing means. A multi-spectral image determining means, and a processing order determining means for determining a processing order of the detection area determining means and the used Malte spectrum image determining means in advance, wherein the detection is performed based on the processing order by the processing order determining means. The detection area by the area determining means and the used multispectral image by the used multispectral image determining means can be referred to at the time of the unevenness detection processing and the unevenness degree calculation by the unevenness detection processing means and the unevenness degree calculating means. A color unevenness inspection apparatus is provided.

  Further, according to the present invention, there is provided a color classification method for performing a color classification determination by using a plurality of types of determination methods, wherein a luminance component of a color measurement area of an image captured using a plurality of band-pass filters is extracted and multi-dimensional. Obtaining data, performing a classification determination operation on the multidimensional data, and extracting a neighborhood class, and determining whether there is one neighborhood class. Determining whether the degree is equal to or more than a predetermined value, and when it is determined that the certainty factor of the determination class is equal to or more than a predetermined value, determining a classification; determining whether there are a plurality of the neighboring classes; Or if it is determined that the certainty factor of the determination class is not greater than or equal to a predetermined value, a step of performing a classification determination operation process different from the classification determination operation process to determine the classification, Color classification method which is characterized in that there is provided.

  Therefore, according to the present invention, the color classification processing of an object is performed using a multispectral image obtained through a plurality of band-pass filters, and the apparatus configuration is simple, low-cost, and mechanical. It can withstand vibration, and even when the spectrum changes without limiting the light source, it can satisfactorily perform color classification of the object, and is the optimal classification judgment for classifying the object. By using the method, it is possible to provide a color classification device and a color classification method capable of further improving the classification accuracy.

  Further, according to the present invention, the color unevenness inspection process for an object is performed using a multispectral image obtained through a plurality of band-pass filters, and the apparatus configuration is simple, low-cost, and Of the target object even if the spectrum changes without limiting the light source, and the accuracy of the target object color unevenness inspection process Can be provided.

  First, before describing an embodiment of the present invention, the basic principle and basic embodiment of the present invention described in Patent Document 2 described above (Japanese Patent Application Laid-Open No. 8-105799 (Japanese Patent Application No. 6-241614)). Will be described.

(Basic principle)
The present invention realizes a simple and inexpensive color classification apparatus by processing a multispectral image obtained by using a plurality of bandpass filters each transmitting only a specific wavelength as a filter for color classification. It is.

  In addition, in order to perform color classification even under different light sources, the reflection spectrum of an appropriate reference plate is measured under the same conditions as when the object is photographed, and the reflection spectrum of the object is measured. In this case, the influence of the light source (illumination light) is removed.

That is, with λ as the wavelength, the reflection spectrum of the object is f (λ), the reflection spectrum of the reference plate is s (λ), the reflection spectrum of the illumination light is L (λ), and the sensitivity spectrum of the photographing system (photographing If the transmission spectrum of the lens, the sensitivity spectrum of the image sensor, etc.) are M (λ), the imaging spectrum gi (λ) of the object and the imaging spectrum gs (λ) of the reference plate are respectively gi (λ) = f (λ). ) × L (λ) × M (λ)
gs (λ) = s (λ) × L (λ) × M (λ)
And the spectrum of the object is given by gi ′ (λ)
gi ′ (λ) = gi (λ) / gs (λ) = f (λ) / s (λ)
… (4)
Is expressed.

  By using the spectrum gi ′ (λ) of the target object from which the influence of the reflection spectrum L (λ) of the illumination light has been removed, the color classification can be performed even under different light sources.

  Further, when the luminance of the illumination light is further different, the power of the signal gi '(λ) in the division equation of equation (4) may be normalized.

  Next, a basic example of the present invention based on the above basic principle will be described with reference to the drawings.

(Basic embodiment)
A versatile basic embodiment will be described with reference to FIGS.

  The color classification device according to this basic embodiment includes a housing 101, an optical system 110 including a lens and the like as shown in FIG. 1, and a plurality of bandpass filters 112A, 112B,..., 112E as shown in FIG. Rotating color filter 112, filter position sensor 123, motor 124, motor driving circuit 124a, image sensor 114 such as CCD for capturing multi-spectral images of the object and the reference plate, amplifier 115, image sensor driving circuit 122 , An A / D converter 116, a frame memory 118, a classification operation circuit 128, an exposure value memory 129 for storing an exposure time of the image sensor 14 for obtaining an appropriate exposure of each bandpass filter, and a control circuit 126.

  As shown in FIG. 2, the rotary color filter 112 is provided with filter position detection holes 125A, 125B,... 125E and a filter initial position detection hole 126.

  , 112E are detected by detecting the initial position detection holes 126 and the filter position detection holes 125A, 125B,... 125E with a filter position sensor 123 composed of a photo interrupter or the like. You.

  In this case, the control circuit 126 controls the motor drive circuit 124 a based on a signal from the filter position sensor 123 so as to synchronize the rotation of the rotary color filter 112 with the imaging timing of the imaging element 114.

  The images that have passed through the band-pass filters 112A, 112B,..., 112E are formed on the image sensor 114 and converted into electric signals.

  This electric signal is converted into a digital signal by an A / D converter 116 via an amplifier 115, and then stored in a frame memory 118 as multispectral image data.

  From the frame memory 118, the entire multispectral image data is sent to an external monitor (not shown), and data of a predetermined area in the image is sent to the classification operation circuit 128.

  Next, the operation of the basic embodiment will be described with reference to the flowchart shown in FIG.

  First, a measurement area of an object is set (step S11).

  Next, the first band-pass filter in the rotating color filter 112 is set (band-pass filter 112A), and preliminary exposure is performed, so that the measurement data obtained thereby falls within a predetermined value range.

  Then, the exposure time of the image sensor 14 at this time is stored in the exposure value memory 129 together with the number of the bandpass filter corresponding to the exposure time (steps S12 to S17).

  Such preliminary exposure is sequentially performed for all bandpass filters (bandpass filters 112B,..., 112E) (steps S18 to S20).

  At the time of measurement after the completion of such preliminary exposure, the target object is imaged by changing the exposure time of the image sensor 114 in synchronization with each of the bandpass filters 112A, 112B,..., 112E.

  At this time, the classification operation circuit 128 converts the measurement data of the corresponding bandpass filter from the multispectral image data based on the exposure time of the image sensor 14 stored in the exposure value memory 129 for each bandpass filter. After the correction, a classification operation is performed.

  As described above, according to this basic example, the exposure value memory 129 for storing the exposure time of the image sensor 14 is provided, and the imaging is performed with the optimal exposure for each bandpass filter. SNR (signal to noise ratio) is improved, and classification accuracy is increased.

  Further, in this basic embodiment, since the pre-exposure can be performed and the measurement can be performed with the optimum exposure for each bandpass filter, the classification can be performed with high accuracy even when the measurement target changes, thereby realizing a versatile color classification apparatus. be able to.

  In the basic embodiment, since the exposure time of the image pickup device is controlled, the circuit configuration is simplified.

  Next, a specific example of the classification operation circuit 128 for performing the multi-class classification operation in the basic embodiment described above will be described with reference to (A, B, C) of FIG.

  When classifying four classes, for example, classes 1 to 4 by the above-described FS conversion method when performing multi-class classification calculation, the class 2 and the class 4 are stored in the space formed by the classification spectra calculated based on the data of class 1 and class 4. When 3 is projected, the boundaries between classes such as class 1 and class 2 or class 3 and class 4 may not be clear, and the classification accuracy may decrease.

  The classification operation circuit 128 can perform the classification effectively without lowering the classification accuracy even in the case described above.

  The classification operation circuit 128 includes a luminance component extraction unit 30, a classification operation unit 32, and a classification determination unit 34, as shown in FIG.

  First, the configuration of the classification operation unit 32 will be described with reference to FIG.

  As shown in FIG. 4B, the classification calculation unit 32 switches between the d1 memory 200 and the d2 memory 201 that store the classification spectra d1 and d2, respectively, and switches the output between the d1 memory 200 and the d2 memory 201. , A multiplier 203 for taking the product of the classification spectra d1 and d2 and the spectrum data from the unknown object, a cumulative adder 206 including an adder 204 and a memory 205, and a d1 memory 210 for storing the classification spectrum. And d2 memory 211, d1 memory 213 and d2 memory 214, d1 memory 216 and d2 memory 217, switch 212 for switching the output of d1 memory 210 and d2 memory 211, and switch for switching the output of d1 memory 213 and d2 memory 214. 215, and the output of the d1 memory 216 and the output of the d2 memory 217 are switched. 218, a classification spectrum selection circuit 207 that selects one of the three classification spectra based on the signal from the accumulator 206, and a multiplier that takes the product of the selected classification spectrum and the spectrum data from the unknown object. 223, and an accumulator 226 including an adder 224 and a memory 225.

  Next, the operation of the classification operation unit 32 will be described. Here, the number of classes to be classified is four classes of classes 1 to 4.

The four classes are assumed to be distributed in the order of numbers in a multidimensional space, and the d1 memory 200 and the d2 memory 201 store classification spectra d1 _1-4 and d2 ₁ calculated in advance from class 1 and class 4 learning data. _-4 is memorized.

The d1 memory 210 and the d2 memory 211 store the classification spectra d1 _1-2 and d21-2 calculated in advance from the class 1 and class 2 learning data, and the d1 memory 213 and the d2 memory 214 store the classification spectra d1 _1-2 and d2 _1-2 . The classification spectra d1 _2-3 and d2 _2-3 calculated in advance from the class 2 and class 3 learning data, and the d1 memory 216 and d2 memory 217 are calculated in advance from the class 3 and class 4 learning data. The classification spectra d1 _3-4 and d2 _3-4 are stored respectively.

First, for the unknown data of the object from the luminance component extraction unit 30, the product of the classification spectrum d1 _1-4 from the d1 memory 200 and each component (dimension) is obtained by the multiplier 203 in the classification calculation unit 32.

  The products of the components are added by the accumulator 206 and input to the classification spectrum selection circuit 207.

  The output of accumulator 206 is the resulting inner product of the unknown data and the classified spectrum.

  Next, the switch 202 is switched, and the inner product value is similarly calculated for the d2 memory 201 side and sent to the classification spectrum selection circuit 207.

  As shown in FIG. 4C, the classification spectrum selection circuit 207 includes a classification determination circuit 230 and a selector 231. When the inner product value from the accumulator 206 is input, the classification determination circuit 230 A classification determination is made.

Here, the unknown data is projected onto the space formed by the classification spectra d1 _1-4 and d2 _1-4 calculated from the data of class 1 and class 4, and the class is determined by the determined classification boundary.

  Here, the boundaries are determined so as to be classified into three new classes, “from class 2 to class 1”, “from class 2 to class 3”, and “from class 3 to class 4”.

  When the classification spectrum selection signal output from the classification determination circuit 230 is “from class 2 to class 1”, the selector 231 outputs the input from the switch 212 as the classification spectrum.

As a result, the classified spectrums d1 _1-2 and d2 _1-2 are selected, the inner product operation of the unknown data is performed by the multiplier 223 and the accumulator 226, and the inner product value is obtained by the classification spectrum from the classification spectrum selection circuit 207. The final class is sent to the classification judging unit 34 together with the selection signal.

  When the classification spectrum selection signal output from the classification determination circuit 230 is “between class 2 and class 3”, the selector 231 outputs the input from the switch 215 as a classification spectrum, and outputs “from class 3 to class 4”. Is output from the switch 218 as a classification spectrum.

  Thereby, the optimal classification spectrum is selected, and the inner product operation is performed.

  In this basic example, the classification spectrum of the first stage is obtained from the data of class 1 and class 4, but this does not mean that the data of the two classes at both ends of the distribution is used. A classification spectrum obtained from the data of class 2 and class 3 may be used.

  Further, a new class 1 'is defined by class 1 and class 2 together, a new class 4' is defined by class 3 and class 4 together, and the data of class 1 'and class 4' is used. The classification spectrum calculated by the calculation may be used.

  In this basic embodiment, the classification is determined in two stages, but the same effect can be obtained even in three stages or more stages depending on the number of classes to be classified. Not even.

  According to such a basic embodiment, the classification is determined in multiple stages, so that effective classification can be performed without deteriorating the classification accuracy even in multi-class classification.

  However, in the basic embodiment as described above, for example, only one type of classification determination method using the FS conversion method is used, so that the classification determination performance is remarkably large depending on the distribution state of a plurality of objects to be classified in a multidimensional space. It tends to decrease, and the classification performance deteriorates depending on the target.

  Therefore, in the present invention, in order to further advance the basic embodiment, an optimal classification determination method for classifying and determining an object is used.

  Next, some embodiments of the present invention from such a viewpoint will be described with reference to the drawings.

(1st Embodiment)
First, a first embodiment in which the classification performance of a plurality of objects is improved by making a comprehensive determination after making a determination between all two classes in a plurality of classes by one kind of classification determination method. This will be described with reference to FIGS.

  That is, in the first embodiment, one type of classification determination is performed between two classes by the number of all two combinations in a plurality of classes, and then the classification determination between these two classes is performed. In some cases, the result is comprehensively determined.

  As shown in FIG. 5, in the first embodiment, in the color classification apparatus using a plurality of band-pass filters as in the basic embodiment described above, the classification operation circuit 128 is provided with the classification determination unit 17. Thereby, the classification performance of a plurality of objects can be improved.

  The classification operation circuit 128 includes a luminance component extraction unit 30 that extracts the luminance component of the measurement area from the multispectral image data obtained as described above, And a classification determination unit 14 that makes a comprehensive determination after making a determination between all two classes.

  The classification determining unit 14 according to the first embodiment includes a classification calculating unit 16 that performs a classification determination between all two classes in a plurality of objects, and a classification that comprehensively determines the result of the classification determination between the two classes. And a decision unit 17.

  The classification operation circuit 128 outputs a classification result for the input image data.

  In the classification operation circuit 128, the learning control unit 15 stores data obtained by performing necessary learning on each image data by switching the switch 13 in advance. Will be used at the time of the classification judgment processing.

  FIG. 6 shows, as a specific example, a classification determination unit 14 including a classification calculation unit 16 and a classification determination unit 17 as an example of performing classification determination in six classes.

That is, in the classification computing section 16, a plurality of V _ij memory 21 for storing the classification vector V _ij of class i and class j, a plurality of V _Oij memory 25 for storing the offset value V _Oij of class i and class j , A plurality of computing units 22, 23a, 23b and a plurality of memories 24. A one-dimensional classification spectrum for classifying the class i and the class j on the same coordinate axis among the classification spectra is defined as a classification vector V _ij .

In this case, each of the _Vij memory 21, the _Voij memory 25, the computing units 22, 23a, 23b, and the plurality of memories 24 has fifteen classes for classifying and judging two classes out of six classes. Will be present (15 = 6 * (6-1) / 2).

The unknown data to be input is subjected to an inner product with the classification vector V _ij , added with an offset value V _oij, and sent to the classification determining unit 17 as a result a _ij .

  Note that the above calculations for inner product and addition are performed in parallel on 15 classification vectors.

That, V12 memory _{21, V} 0 12 memory 25, the class 1, the vector V for classifying the class 2, and _{V 0} is the offset is one stored respectively.

  Since FIG. 6 shows an example for classification among six classes, the number of combinations of two for all classes, that is, V12, V13, V14 to V56, in this case, fifteen memories 21, There are 25.

And their memory data, classification vector V _ij and is projected to unknown data as the offset value V _Oij, respectively calculator 22, 23a, 23b and taking the inner product operation and offset addition via the memory 24 a12, a13 , A14 to a56 are output as results a _ij .

  FIG. 7 shows a block diagram of the classification determining unit 17.

Here, the class i plus counter 26, which defines a _ij = -a _ji , checks the signs of the determination results a for all the classes i sent from the classification operation unit 16 and determines the number bi that takes a positive value. Count and output.

  The determination unit 27 checks the maximum value of the number bi having a positive value, and comprehensively determines the class i having the maximum value.

  That is, the classification determining unit 17 includes a total of six plus counters 26 from the class 1 plus counter, the class 2 plus counter, the class i plus counter to the class 6 plus counter. .

  Then, the class 1 plus counter 26 determines whether all the inputs related to class 1 from the inputs a12, a13 to a16 are positive or negative, and if the input is positive, counts the number with the counter. Things.

  Similarly, the class 2 plus counter 26 counts the plus components of the inputs for all classes 2 up to the inputs a21, a23, a24, a25, and a26.

  Then, with respect to the plus number of outputs b1 to b6 output by each of the class plus counters 26, the judging unit 27 looks at the outputs of b1, b2 to b6 and determines the class having the maximum value.

In the above, a statistical method is used to obtain the classification vector V _ij and its offset value V _oij , but FIGS. 6 and 7 show examples in which a linear discriminator is used. In this case, since the number of dimensions is not reduced, classification determination can be performed with high accuracy.

  On the other hand, FIGS. 8 and 9 show an example in which the shortest distance method is used after performing the FS conversion. In the case where the FS conversion is used, the number of dimensions is reduced. Can be.

  (A, B) of FIG. 10 shows an example of a classification result when the classification spectra d1 and d2 that maximize the extended Fisher value (Delustering Criterion) are used.

  8 and 9, since the FS conversion is used, there are a plurality of memories 31 and 32 for the classified spectra d1 and d2. 22 calculates the inner product of the input unknown data and the classification spectra d1 and d2.

  The output c12 is output via the computing unit 23 and the memory 24 when the output is c12 when the memory is the d1 memory 31 and the output d12 when the memory is the d2 memory 32.

  The determination by the FS conversion is a classification determination of the classes 1 and 2 based on the outputs c12 and d12.

  That is, here, the classified spectra d1 and d2 are stored in the d1 memory 31 and the d2 memory 32, respectively, as those which classify class 1 and class 2 most.

Here, c _ij and d _ij outputs are obtained, which indicate that there are classified spectra d 1 and d 2 that classify class i and class j most.

  Then, the results of all the projections on the classification spectra d1 and d2 that most separate any two of the two classes are sent to the classification determination unit 17.

  The classification determining unit 17 in the FS conversion shown in FIG. 9 includes a class 1-2 determining unit, a class 1-3 determining unit, a class i-j determining unit, a class 5-6 determining unit, and an arbitrary two-class determining unit 33. And a determination unit 27 are present.

  The determination unit 33 determines the distance between c12 and d12 obtained as described above and class 1 and the distance between class 2 and obtains the closer class.

  Then, a result such as class 1 or class 2 or class 1 or class 3 is output from d1 and d2 as en from the obtained output.

  In other words, n is equal to the number of combinations of 2. In this case, 5 × 6 ÷ 2 = 15, so that 15 outputs are output.

  Then, the discrimination unit 27 counts the largest number of determinations and outputs the result as a comprehensive determination result.

  In FIG. 8, instead of storing the classification spectra d1 and d2 that maximize the fish ratio in the FS conversion in the d1 memory 31 and the d2 memory 32, a vector that maximizes the extended fish ratio in these memories is used. A way of describing is also conceivable.

  This is called a declustering criterion method, and when this method is used, a classification result as shown in FIGS. 10A and 10B is obtained.

  Here, vectors such as F1 and F2 are obtained based on the Declaring Criterion method. When unknown data is projected on these vectors, vectors such as one of two classes are concentrated and the other classes are spread or scattered. I get it.

  In such a case, the determination unit 27 in FIG. 9 determines based on the output from each class determination unit 33, but determines the classification boundary and checks whether it exists inside the classification boundary.

  Then, while it is determined whether the vector belongs to one of the two classes such as the output e1 of the class 1-2 determination unit and the output e2 of the class 1-3 determination unit, the determination unit 27 Is determined comprehensively.

  In this case, the classification determination boundary is previously obtained from the learning data of class 1 and class 2 by the learning control unit 15, and the determination unit 33 determines whether the classification is inside or outside the boundary.

Note that the Delustering Criterion method is based on Non-Patent Document 2 and the like.
"A Delustrating Criterion for Feature Extraction in Pattern Recognition" by JOHN FEHLAUER AND BRUCE A. EISENSTEIN, IEEE TRANSACTIONS ON COMPUTERS. VOL. C-27, no. 3, MARCH 1978 In this way, when the delimiting criterion method is used, the configuration is the same as that of FIGS. 8 and 9, but the class ij determination unit 33 of FIG. It is determined by determining whether or not the image is projected within the classification boundary as shown.

  This method is an effective classification judgment method when the average vector in the multidimensional space to be classified is close.

  Therefore, as described above, according to the first embodiment of the present invention, two classes are selected from among a plurality of classes at a time without making a determination among a plurality of classes to be determined at once, and a determination is made therein. After that, the classification is determined comprehensively, so that the classification can be determined with higher accuracy.

(2nd Embodiment)
Next, a second embodiment of the present invention in which the classification performance of a plurality of objects is improved using two types of classification determination units will be described with reference to FIG.

  FIG. 11 shows a configuration of a color classification device according to a second embodiment of the present invention in which two types of classification determination units 41 and 42 are connected in series to perform classification determination.

  In order to classify a plurality of objects with high accuracy, there is a method of performing all two-class classification as in the above-described first embodiment and making a comprehensive determination, or a method of hierarchically performing a classification determination process to narrow down. In the method, when the number of classes to be subjected to classification determination increases, the processing time increases. Therefore, if the shortest distance method, which has the smallest processing scale among the classification determination processes, is performed first, the number of nearby classes is narrowed down, and then the classification determination process is performed with high accuracy, the classification determination process for a plurality of objects can be performed at high speed. Can be.

  Therefore, the classification operation circuit 128 according to the second embodiment includes a luminance component extraction unit 30 that extracts a luminance component for obtaining color information of an area in a multispectral image captured using a plurality of bandpass filters; It comprises a first classification determination unit 41 that first classifies the extracted data, and a second classification determination unit 42 that superimposes the result of the classification determination unit 41 and performs another classification.

  The classification determination unit 41 includes a classification calculation unit 43 and a neighborhood class determination unit 44 for narrowing down the number of classification targets.

  The classification determination unit 42 includes a classification calculation unit 45 and a classification determination unit 46 that determines the classification.

  In FIG. 11, the learning control unit 15 has the same function as that of the first embodiment.

  That is, the second embodiment is an embodiment in which the classification accuracy is improved by using two types of classification determination units.

  In FIG. 11, the classification calculation circuit 128 has two classification determination units 41 and 42 in series after extracting a luminance component, and is characterized in that a plurality of classification determination units are provided as described above. The first and second classification judgment sections 41 and 42 are characterized by using a different classification judgment method.

  In the case of the embodiment shown in FIG. 11, the first class determination unit 41 first determines a number of classes that are close to each other, and then determines a class by the second class determination unit 42 among the narrowed classes. A classification result is obtained by performing the determination.

  As described above, in the second embodiment, when there are a large number of objects to be classified, the first classification and determination unit 41 uses a simple process such as obtaining a nearby class based on the distance, and only the closest one is determined. Then, the second classification determination unit 42 performs high-precision classification determination using a piecewise linear classifier or the like, which will be described later, so that processing can be performed quickly and with high precision.

  At the time of learning, the luminance component is described as sample data of learning data in a memory for learning data via the learning control unit 15.

(Third embodiment)
A third embodiment in which the classification accuracy of a plurality of objects is improved by using two types of classification determination methods as in the above-described second embodiment will be described with reference to FIGS.

  That is, in the third embodiment, as shown in FIG. 12, in the color classification device using a plurality of bandpass filters, the classification calculation circuit 128 is provided with the determination result determination unit 51, and the classification determination unit 41 determines the determination result. The classification accuracy of a plurality of objects is improved by controlling the other classification judging section 42 based on.

  The classification operation circuit 128 according to the third embodiment includes a luminance component extraction unit 30 that extracts a luminance component for obtaining color information of an area in a multispectral image captured using a plurality of bandpass filters, and a extracted luminance component. A first classification determination unit 41 that first classifies data, a determination result determination unit 51 that determines the result of the classification determination unit 41, and classifies the data again based on the result of the determination result determination unit 51 if necessary. It is composed of a second classification judging unit for judging and 42.

  The first and second classification determination units 41 and 42 have classification calculation units 43 and 45 and classification determination units 44 and 46, respectively.

  The learning control unit 15 has the same function as that of the first embodiment.

  That is, FIG. 12 is an extension of FIG. 11 and includes a block serving as a determination result determination unit 51.

  This is because the judgment result of the first classification judgment unit 41 is checked by the judgment result judgment unit 51 to determine whether the judgment result is appropriate. If the judgment result is appropriate, the judgment result is directly passed through and output from the classification operation circuit 128 as the classification result. .

  If the result of the determination is that it is better to make another determination, it is sent to the second classification determination unit 42.

  In other words, the first classification determination unit 41 performs a classification determination process using a piecewise linear discriminator or the like. By performing the determination based on the distance such as the neighborhood after the FS conversion by the classification determination unit 42, the processing is reliably performed and the classification result is output.

  FIG. 13 shows a flow of a process of hierarchically performing the classification judgment process based on the first classification judgment result.

  A luminance component of a color measurement area of an image captured using a plurality of bandpass filters is extracted to obtain multidimensional data (step 501).

  The obtained multidimensional data is subjected to classification determination operation processing 1 (step 502), and a neighborhood class is extracted (step 503).

  The judgment result judging unit 51 checks whether the number of neighboring classes obtained by the first classification judging unit 41 is one (step 504), and checks whether the certainty factor of the judgment result is equal to or more than a predetermined value (step 505). .

  Based on the judgment result of the judgment result judging section 51, if necessary, the second classification judging section 42 performs the classification judgment calculation processing 2 (Step 506) to determine the classification (Step 507).

  By performing the classification judgment processing hierarchically with such a configuration, it is possible to classify and judge multiple classes with higher accuracy.

  For example, the first classification determination unit 41 uses a piecewise linear discriminator (Piecewise Linear Discriminant Method), which is a multi-class linear classifier, and the second classification determination unit 42 uses the shortest distance method after FS conversion. Advanced multi-class classification determination processing.

  Since the linear discriminator creates the classification discriminant line without reducing the dimension, the classification performance is good, but the classification cannot be determined depending on the type of the object and the number of the objects.

  Therefore, the judgment result judging section 51 verifies the judgment result of the first classification judging section 41, and depending on the result, the second classification judging section 42 makes a classification judgment by FS conversion.

The linear classifier is based on Non-Patent Document 3 and the like.
K. Fukunaga "Chapter4 LINEAR CLASSIFIERS" of "Introduction to Traditional Pattern Recognition" (Fourth Embodiment) Next, the fourth embodiment is extended from the third embodiment to determine a subtle color difference in an image. Will be described with reference to FIG.

  That is, in the fourth embodiment, in the color classification device using a plurality of band-pass filters, when classifying into two classes, a certain class and another class, the learning data updating unit 61 is included in the classification operation circuit 128. And the determination result determination unit 51, the two classes can be classified and determined accurately without learning both classes.

  The classification operation circuit 128 according to the fourth embodiment includes: a luminance component extraction unit 30 that extracts a luminance component for obtaining color information of an area in an image captured using a plurality of bandpass filters; , A learning control unit 15 that controls learning, a learning data updating unit 61 that updates learning data, a determination result determining unit 51 that determines a classification determination result, and a determination result determination A second classification determination unit 42 for repeatedly determining the classification of the data based on the result of the unit 51 if necessary.

  That is, in the fourth embodiment as shown in FIG. 14, after the luminance component is extracted, the processing of the first classification determination unit 41 is performed, and then the update is performed based on the content of the first classification determination unit 41. The learning data is updated by the unit 61, and the processing data is sent to the second classification determination unit 2 as it is.

  The result output from the second classification judging section 42 is sent to the judgment result judging section 51, and if necessary, the learning data is updated again by the learning data updating section 61, and is returned to the second classification judging section 42 again. It is.

  In other words, a loop from the learning data update unit 61 to the determination result determination unit 51 via the second classification determination unit 42 forms a loop, and the determination result determination unit 51 determines the determination result as the first classification result. If it is not considered as an appropriate determination result, the process is repeated in a loop so that the process is repeated.

  FIG. 15 shows a processing flow for detecting subtle color unevenness in an image according to the fourth embodiment.

  First, an outline of this processing flow will be described.

  From the images captured by the plurality of bandpass filters, the luminance component in the determination area i is extracted to obtain multidimensional data (step 603).

  If this data has a large distance from the learning data of the normal part previously learned in the memory, the determination area i is determined to be uneven in color (step 606) and registered in the learning data (step 607).

  If this data is smaller than a predetermined value, the determination area i is determined to be normal (step 608) and registered in the learning data (step 609).

  In this manner, when the determination in the entire determination area is completed, the second classification determination unit 42 performs the determination again.

  The second classification determination unit 42 checks the distance d1 between the determination area i and the normal feature vector in the learning data memory (step 614), and checks the distance d2 between the color unevenness feature vector (step 615).

  As a result, the calculated distances d1 and d2 are compared (step 616). If d2 is larger than d1, color unevenness is determined, and if d2 is smaller than d1, it is determined as a normal part (steps 617 and 619), and each learning data is updated (step 617). 618, 620).

  Then, when the determination is completed in the entire determination area, the determination result determination unit 51 re-examines the classification determination unit 42 as necessary.

  Next, a flow of color unevenness detection will be described in detail.

  At step 601, a multispectral image is input.

  Then, in step 602, the image number is set to i = 0, which is an area to be determined, and i = 0 and an initial value are input as a determination area.

  Then, in step 603, the data of the image determination area i is detected, and in step 604, the distance to the normal part learned in advance is calculated.

  Next, it is checked in step 605 whether the distance from the normal part is equal to or more than a predetermined value. If the distance is not less than the predetermined value, it is determined in step 606 that there is color unevenness. At 608, it is determined as a normal part.

  If it is determined that there is color unevenness, the image data is registered in the color unevenness learning data in step 607.

  Further, the determination result of the normal part is registered as data of the normal part in step 609.

  Then, in step 610, the determination is made in all the areas in the image. Until the determination is completed, the loop is repeated with i = i + 1 in step 611.

  Then, in all areas for detecting color unevenness, the first determination ends in step 610.

  The processing up to this point, that is, whether the image is uneven or normal based on the distance from the normal part is determined by the first classification determination unit 41. Two learning data are created.

  The second classification determination unit 42 performs a determination process as to which is closer to the created learning data.

  Also, in step 612, i = 0 and the judgment area i = 0 and an initial value are input, and in step 613, data of the judgment area i is detected.

  Then, in step 614, the distance d1 to the normal part learning unit data is obtained, and in step 615, the distance d2 to the unevenness learning data is obtained.

  Next, in step 616, it is checked whether d2 is equal to or greater than d1. If d2 is larger, color unevenness is determined in step 617. If d1 is larger, color unevenness is determined in step 619.

  Then, in steps 618 and 620, the learning data of each of the color unevenness portion and the normal portion is updated.

  Then, in the inner loop from step 621, step 622 sets i = i + 1, and first, determination is made for all areas.

  Then, in step 623, the determination result determination section 51 checks whether or not to perform re-determination. If so, the second classification determination section 42 performs another classification determination.

  Through such processing, the learning data of the color unevenness portion and the learning data portion of the normal portion are updated to appropriate ones by repeating the number of times.

  Then, according to the fourth embodiment, detection with good classification accuracy can be performed as a result.

(Fifth embodiment)
Next, a fifth embodiment in which a plurality of types of classification judgment methods are used to improve the classification judgment accuracy of a plurality of objects will be described with reference to FIG.

  That is, since the classification determination accuracy of a plurality of targets depends on the number of targets, distribution state, and the like, the classification determination accuracy of the plurality of targets is improved by selecting an optimal classification method according to the characteristics of the classification target. This is the fifth embodiment.

  As a plurality of types of classification determination methods adopted in the fifth embodiment, in addition to the above-described FS conversion method, piecewise method, declustering criterion method, KL conversion method, FK conversion method, classification method using HTC, A discriminant analysis method, a normal orthogonal discriminant analysis method, a Malina method, a method based on nonparallelization, a subspace method, an FE method, and the like can be employed.

  As shown in FIG. 16, the fifth embodiment classification operation circuit 128 extracts a luminance component for extracting a luminance component for obtaining color information of an area in a multispectral image captured using a plurality of bandpass filters. Unit 30, a class information database 71 for storing information to be classified, a classification determination method selection function unit 72 for selecting a classification process and a narrowing method based on information in the class information database, and a classifying unit for classifying the extracted data. It comprises a plurality of classification judgment sections 41 for judgment, a judgment result judgment section 51 for judging the classification judgment result, and a database updating section 73 for updating the class information database 71.

  The classification determination method selection function unit 72 includes a processing selection unit 74 and a narrowing-down method selection unit 75.

  In addition, the plurality of classification determination units 41 include a classification calculation unit 43 and a classification determination unit 44, respectively, in order to perform classification determination using different classification determination methods adopted from the plurality of types of classification determination methods described above. Have.

  That is, the fifth embodiment is an embodiment in a case where n classification determination units are prepared to perform optimal classification determination.

  The classification operation circuit 128 according to the fifth embodiment is characterized in that it has a database 71 having this class information from the extracted luminance component as a feature. It has a classification determination method selection function unit 72 for selecting a method.

  Then, the plurality of classification determination units 41 each determine and output a classification result in the same manner as in the above-described embodiment.

  In the class information database 71, the distribution status of the classes, the center coordinates in the multidimensional space, the near classes, and the like are described, and a database in which nearby classes in a certain class are described is stored.

  Then, after extracting the luminance component, the classification determination method selection function unit 72 determines which classification determination method to perform, that is, whether to use the FS conversion, the piecewise method, or the determination in the declustering criterion method. Is first selected by the processing selection unit 74.

  Then, the narrowing-down method selecting unit 75 reduces the number of classes from seven to three in the case of, for example, seven classes from among many classes, further reduces the number to two, and finally narrows down to one, or The number of narrowing-down methods and the narrowing-down method, such as trying to narrow down to five, three, two, and one, are determined.

  Then, the classification determination method selection function unit 72 determines which of the plurality of classification determination units 1 to n to use in accordance with the obtained classification method.

  The determinations by the plurality of classification determination units 41 form a loop that is returned from the determination result determination unit 51 again.

  The determination result determination unit 51 has a function of updating the database 71 via the database update unit 73 from the determination result including the information on the narrowing-down method.

  As a result, the repetitive processing is performed with higher accuracy even in the case of classification other than the two classes such as the normal part and the abnormal part as in the above-described embodiment, but the same processing is performed again by updating the data of each class. In such a case, a different output is output.

  Then, when the determination result determination unit 51 determines a certainty factor that is equal to or greater than a predetermined value, it is output as a classification result.

  As described above, the fifth embodiment uses the classification determination method selection function unit 72 to select how many times the classification determination is repeatedly performed and which classification determination is to be used. Is performed with higher accuracy.

  FIG. 17 shows a processing flow according to the fifth embodiment.

  First, a luminance component is extracted from a multispectral image to obtain multidimensional data (step 701).

  Next, it is determined whether or not the mode is the learning mode (step 702). If the mode is the learning mode, the mode switch 13 is switched to register the multidimensional data in the learning data (step 703), and the class information database is updated (step 704). ), End the process.

  In the automatic judgment mode, the classification judgment processing method is set while referring to the database (step 705).

  Here, while referring to the obtained unknown data and the distribution state in a multidimensional space of a plurality of objects to be classified and determined, it is set which classification determination process is to be used and how the determination narrowing method is to be performed (step 706). .

  Next, classification is determined according to the set classification determination method (step 707).

  Then, based on the set classification judgment method and the classification judgment result, it is judged whether or not to perform the reexamination (step 708).

  Here, when reinspection is performed, the judgment result is registered (step 709), the database is updated (step 710), and the process returns to the classification judgment selection function again.

(Sixth embodiment)
A sixth embodiment for speeding up the classification operation and improving the classification accuracy will be described with reference to FIGS.

  That is, in a color classification device using a plurality of bandpass filters, not all images captured by different bandpass filters have a feature amount effective for classification.

  Therefore, in the sixth embodiment, the calculation speed is increased and the classification performance is improved by omitting images having data that have little effect on the classification or lower the classification performance.

  The color classification device according to the sixth embodiment includes a multi-spectral image capturing unit 81 that captures an image using a plurality of band-pass filters having different pass band characteristics, an image processing unit 82, and a classification that extracts features. It comprises an image selecting means 83 for selecting an image to be used and a classifying means 84 for performing classification judgment.

  That is, in the sixth embodiment, the multi-spectral image pickup unit 81 of FIG. 18 corresponding to the color classification device using a plurality of band-pass filters as shown in the basic embodiment of FIG. The multispectral image data captured via the pass filter is output.

  This multispectral image data is subjected to image processing such as smoothing required by the image processing unit 82.

  In the present embodiment, the dimensions of the image data on which the image processing has been performed are reduced by the image selection unit 83, and the classification unit 84 performs the color classification using the reduced data.

  FIG. 19 shows a flow of processing for removing unnecessary images and performing classification determination processing as quickly as possible according to the sixth embodiment.

  Here, the operation of the image processing unit 2 and the image selection unit 83 of the present embodiment shown in FIG. 18 will be described as a color classification device having five bandpass filters.

  When data is input using five bandpass filters, five (i.e., five-dimensional) images are obtained, which are initialized and sequentially read (steps 901 and 902).

  For each read image, noise is removed by performing image processing such as smoothing if necessary (step 903).

  It is determined whether or not the processing has been completed for all the images (step 904), and the number of the image is initialized to read it again (step 905).

  Then, the images are read out one by one in order (step 906), and the feature amount is extracted (step 907).

  It is checked whether or not the feature value is larger than a predetermined value (step 908). If the feature value is larger than the predetermined value, the image number is written into the memory (step 909), and the feature value of the next image is checked. (Step 910).

  In the above description, the extracted feature amounts are the contrast and the density difference of the image.

  The classification by the classification means 84 is controlled by using only the data from the selected image (that is, the image obtained by the band-pass filter capable of obtaining a large amount of characteristic), and thereby the data amount handled for both input and classification calculation. Is reduced, so that higher-speed classification can be determined.

  In addition, since an image including noise is not used for the classification determination, the classification can be determined with higher accuracy.

  When inspecting subtle color unevenness or the like in an image, image contrast, maximum density difference, and the like are used as image feature amounts.

  An image captured using the band-pass filter corresponding to each dimension and having a small contrast or a light and shade difference cannot obtain a feature amount effective for detecting color unevenness, so that the dimension can be reduced.

  When the contrast is used, the feature amount of the entire area to be detected can be obtained more accurately. When the contrast is used, the processing speed can be further increased.

  The effects of the first to sixth embodiments described above are summarized as follows.

  (1) Classification is comprehensively performed on the basis of determinations between all two classes, so that erroneous determinations are reduced.

  (2) By using a high-speed method for rough classification and an accurate method for fine classification, high-speed determination with few errors can be performed.

  (3) By using the classification methods in series, the number of classes can be narrowed down in the first stage, so that the processing in the second stage becomes faster.

  (4) The versatility is improved because an optimal classification method is used in consideration of an error rate and an operation speed according to the number of objects to be classified and the number of classes.

  (5) If the reliability of the determination result is low, the determination is made in more detail, so that the error determination rate decreases.

  (6) If the reliability of the first determination result is low, a more detailed classification is performed, and if the reliability is high, the classification is terminated.

  (7) The learning control unit enables classification determination of a plurality of objects with minimum learning.

  (8) Since the learning data is updated while performing the classification determination, the classification determination can be performed without performing the learning in advance.

  (9) By deleting a dimension that is not necessary for classification or a dimension that hinders classification, high-speed determination with few errors can be performed.

  (10) By using the contrast and the shading difference as the feature amounts, the calculation for determining whether or not the image of that dimension is effective for classification is simplified.

  By the way, in the above-described fourth embodiment, the presence or absence of color unevenness could be detected, but the size of the color unevenness could not be quantified.

  Therefore, next, a description will be given of a basic configuration and some embodiments of a color nonuniformity inspection system capable of quantifying the size of color nonuniformity.

(Basic configuration)
FIG. 20 is a block diagram showing a basic configuration of the color shading inspection system.

  20, a multispectral image frame memory 318 corresponds to the frame memory 118 in FIG. 1, and stores multispectral image data of an object in the same manner as in FIG.

  The multispectral image data of the object stored in the multispectral image frame memory 318 is read out by the characteristic amount extracting units 329A and 329B of the classification operation circuit 328 and subjected to a predetermined process.

  In other words, the feature amount extraction units 329A and 329B function as a luminance component extraction unit similar to the luminance component extraction unit 30 in FIG. When a feature amount is required, it functions as a feature amount extraction unit.

  Among these, the feature amount such as the luminance component or the standard deviation from the feature amount extraction unit 329A is used for performing the color unevenness detection of the target in the classification evaluation unit 330 which functions as a color unevenness detection unit which will be described later. You.

  In addition, the feature value of the image such as the luminance component or the standard deviation from the feature value extraction unit 329B is calculated by the classification evaluation unit 340 which substantially functions as a color unevenness calculation unit described later. To be served.

  That is, the classification determination unit 330 outputs the classification result of the color classification of the object, whereas the classification evaluation unit 340 outputs the evaluation value such as the color unevenness determination of the object.

(Seventh embodiment)
FIG. 21 is a block diagram showing a configuration of a color non-uniformity inspection apparatus as a seventh embodiment based on the above basic configuration.

  First, a description will be given of a seventh embodiment of a color nonuniformity inspection apparatus having the color nonuniformity detection processing unit 330 and the color nonuniformity degree calculation unit 340.

  In the above-described fourth embodiment, it was possible to detect the presence / absence of color unevenness, but it was not possible to quantify the size of the color unevenness. 340 is provided to realize a color unevenness inspection system capable of quantifying the degree of color unevenness of an object.

  In FIG. 21, a multispectral image frame memory 318 is equivalent to the frame memory 118 in FIG. 1, and stores an image of an object imaged by a CCD image sensor or the like capable of capturing a multispectral image similarly to the case of FIG. It is assumed that multispectral image data is stored.

  The multispectral image data of the object stored in the multispectral image frame memory 318 is read out by the color unevenness determination unit 328A and subjected to a predetermined process.

  The determination result is output from the determination result output unit 345 based on the processing output of the color unevenness determination unit 328A.

  The color unevenness determination unit 328A includes two processing units, the same feature amount extraction units 329A and B as described above, and a color unevenness detection processing unit 330 and a color unevenness degree calculation unit 340.

  The processing results of the color unevenness detection processing unit 330 and the color unevenness degree calculation unit 340 are stored and stored in the color unevenness detection result storage memory 342 and the color unevenness degree calculation result storage memory 344.

  In FIG. 21, the uneven color detection processing and the uneven color degree calculation processing can be performed in parallel.

  Further, the color nonuniformity calculation unit 340 uses not only the shading difference as in the above-described fourth embodiment, but also the feature values such as the standard deviation of the multispectral image, the contrast, and the skewness or kurtosis in the density histogram. Thereby, the degree of color unevenness can be accurately quantified.

(Eighth embodiment)
FIG. 22 shows a color according to an eighth embodiment in which the color nonuniformity detection processing and the color nonuniformity degree calculation processing are not performed in parallel as in the above-described seventh embodiment, but are performed in permutation. FIG. 2 is a block diagram illustrating a configuration of the unevenness inspection apparatus.

  In FIG. 22, a multispectral image frame memory 318 is equivalent to the frame memory 118 of FIG. 1, and stores an image of an object captured by a CCD image sensor or the like capable of capturing a multispectral image similarly to the case of FIG. It is assumed that multispectral image data is stored.

  The multispectral image data of the object stored in the multispectral image frame memory 318 is read out by the color nonuniformity determination unit 328B and subjected to a predetermined process.

  Then, the determination result is output from the determination result output unit 345 based on the processing output of the color unevenness determination unit 328B.

  The color unevenness determination unit 328B includes two processing units, the same feature amount extraction units 329A and 329B as described above, and a color unevenness detection processing unit 330 and a color unevenness degree calculation unit 340.

  The processing results of the color unevenness detection processing unit 330 and the color unevenness degree calculation unit 340 are stored and stored in the color unevenness detection result storage memory 342 and the color unevenness degree calculation result storage memory 344.

  In this case, the multi-spectral image data of the object stored in the multi-spectral image frame memory 318 is first subjected to uneven color detection by the uneven color detection processing unit 330 via the feature amount extraction unit 329A by the route of the processing 1. Is

  Then, after the result of the color nonuniformity detection in the color nonuniformity detection processing unit 330 is stored in the color nonuniformity detection result storage memory 342, the color nonuniformity degree calculation unit is routed through the feature amount extraction unit 329 </ b> B by the route of processing 2 At 330, the degree of color unevenness is calculated.

  However, at this time, the color nonuniformity degree calculation unit 340 calculates the color nonuniformity degree while referring to the color nonuniformity detection result stored in the color nonuniformity detection result storage memory 342.

  In the above, when the result of the color nonuniformity detection stored in the color nonuniformity detection result storage memory 342 is updated, the process may be shifted to the route of processing 2 each time.

  Further, in the color nonuniformity calculation unit 340, the standard deviation of the multispectral image is set as the color nonuniformity, and the fisher ratio as described above obtained by learning the result of the color nonuniformity detection in the color nonuniformity detection processing unit 330. The value calculated by the declustering criterion method can be used as the degree of color unevenness.

  In this case, after the color unevenness is detected in two classes by the color unevenness determination method according to the above-described fourth embodiment, the FS conversion is performed on the two classes as described above to obtain the fisher ratio and the like. The degree of color unevenness can also be obtained.

  In the color nonuniformity inspection system according to the eighth embodiment, the order of the route of the process 1 and the route of the process 2 may be reversed.

  That is, the color unevenness may be detected and the color unevenness may be detected while referring to the color unevenness degree.

(Ninth embodiment)
FIG. 23 is a block diagram illustrating a configuration of a color nonuniformity inspection apparatus having a feature amount extraction unit 329 and a colorimetric value calculation unit 347 in a color nonuniformity determination unit 328C as a ninth embodiment.

  In FIG. 23, a multispectral image frame memory 318 corresponds to the frame memory 118 of FIG. 1, and stores an image of an object captured by a CCD image sensor or the like capable of capturing a multispectral image similarly to the case of FIG. It is assumed that multispectral image data is stored.

  The multispectral image data of the object stored in the multispectral image frame memory 318 is read out by the color nonuniformity determination unit 328C and subjected to a predetermined process.

  The determination result is output from the determination result output unit 345 based on the processing output of the color unevenness determination unit 328C.

  The color nonuniformity determination unit 328C includes two processing units, a color nonuniformity detection processing unit 330 and a color nonuniformity degree calculation unit 340, and includes a feature amount extraction unit 329 and a colorimetric value calculation unit 347.

  The processing results of the color unevenness detection processing unit 330 and the color unevenness degree calculation unit 340 are stored and stored in the color unevenness detection result storage memory 342 and the color unevenness degree calculation result storage memory 344.

  In this case, the multispectral image data of the object stored in the multispectral image frame memory 318 is read out by the feature extracting unit 329 of the color nonuniformity determining unit 328C and subjected to a predetermined process.

  In other words, the feature amount extraction unit 329 functions as a brightness component extraction unit similar to the brightness component extraction unit 30 in FIG. If necessary, it functions as a feature amount extraction unit.

  Then, the colorimetric value calculation unit 347 in the color unevenness determination unit 328C calculates the colorimetric value in consideration of the characteristics of the band pass filter using the rotating color filter 112 of FIG. 1 that is used to obtain multispectral image data. It is provided for reference.

  When the color nonuniformity detection processing unit 330 performs the color nonuniformity detection process, or when the color nonuniformity degree calculation unit 340 calculates the color nonuniformity degree, the colorimetric value calculation unit 347 refers to the colorimetric value processing. Let it do.

  As a result, when performing the color unevenness inspection, it is possible to determine that the color difference is not less than a certain color difference and to determine that the color difference is less than a certain color difference.

  Further, the color unevenness degree calculation unit 340 can use the maximum value of the color difference, the colorimetric value, the standard deviation of the colorimetric value itself, and the like as the color unevenness degree.

  FIG. 24 is a flowchart showing the flow of the processing in FIG.

  FIG. 24 shows an example of processing for detecting color unevenness due to color difference.

  First, when the color unevenness detection process is started, a multispectral image is input, and then an average colorimetric value of the entire detection area is obtained (steps S101 and S102).

  In this case, for example, colorimetric values such as L * a * b values and X, Y, and Zd stimulus values are obtained.

  Then, the detection area is divided, and the division area number i = 0 is input as an initial value (steps S103 and S104).

  Then, a colorimetric value of each divided area i is calculated by calculating a colorimetric value of each divided area, and a color difference value from the entire detection area is calculated (steps S105 and S106).

  Then, it is checked whether the color difference is a value equal to or more than a predetermined value. If the value is equal to or more than the predetermined value, the color difference is determined to be displayed (steps S107 and S108).

  If the color difference is not equal to or greater than the predetermined value, it is determined that the color difference is normal (step S109).

  Then, it is checked whether or not the determination has been completed for all the divided areas. If not, the divided area number is incremented by one and the processing is repeated (steps S110, S111).

  Subsequently, FIG. 25 is a flowchart illustrating an example of color unevenness detection based on color difference.

  First, when the color unevenness degree detection processing is started, after input of a multispectral image, an average colorimetric value of the entire detection area is obtained (steps S112 and S113).

  In this case, for example, colorimetric values such as L * a * b values and X, Y, and Zd stimulus values are obtained.

  Then, the detection area is divided, and the division area number i = 0 is input as an initial value (steps S114 and S115).

  Then, a colorimetric value of each divided area i is calculated by calculating a colorimetric value of each divided area, and a color difference value from the entire detection area is calculated (steps S116 and S117).

  Next, the calculated color difference is stored in the color unevenness degree calculation result storage memory 344, and it is checked whether the color difference calculation has been completed for all the divided areas. If not calculated, the divided area number is incremented by one. Then, the processing is repeatedly performed (steps S118, S119, S120).

  Then, after calculating the color difference in all the divided areas, the color unevenness degree of the entire detection area is calculated by a comprehensive filter calculation of the color difference in all the divided areas stored in the color unevenness degree calculation result storage memory 344. The color unevenness degree of the entire area is detected, and the process is terminated (step S121).

  FIG. 26 is a flowchart showing the color unevenness degree calculation processing in the processing flow of FIG.

  That is, FIG. 26 is an example in which when one divided area can be treated as primary data, the degree of color unevenness is calculated for each filter and the overall degree of color unevenness is quantified.

  In this case, when calculating the degree of color unevenness, first, which filter (arbitrary number) is used as a filter to be used for calculating the degree of color unevenness is determined based on selection criteria of a filter to be used in the input multispectral image. (Steps S122 and S123).

  Then, among the filters to be used, first, a filter number i = 0 is input as an initial value (step S124).

  Then, the color unevenness degree of the entire detection area for each filter is calculated based on a standard deviation (shading difference standard deviation) or the like, and is stored in the color unevenness degree determination result memory 344 (steps S125 and S126). .

  Then, it is checked whether or not the determination has been completed for all the filters used. If the determination has not been made, the division area number is incremented by one, and the processing is repeated, so that the color for all the filters used can be determined. The degree of unevenness is calculated (steps S127 and S128).

  If the determination has been completed for all the filters used, the degree of color non-uniformity of each filter stored in the determination result memory is comprehensively calculated, and the degree of color non-uniformity of the object is quantified (step S129).

(Tenth embodiment)
FIG. 27 is a block diagram showing a configuration of a color nonuniformity inspection apparatus according to a tenth embodiment in which a normal part data creation unit 349 is provided in the color nonuniformity determination unit 328D to perform color nonuniformity determination and color nonuniformity inspection. FIG.

  In FIG. 27, a multispectral image frame memory 318 corresponds to the frame memory 118 of FIG. 1, and similarly to the case of FIG. 1, a multi-spectral image can be captured by a CCD image sensor or the like. It is assumed that multispectral image data is stored.

  The multispectral image data of the object stored in the multispectral image frame memory 318 is read out by the color nonuniformity determination unit 328D and subjected to a predetermined process.

  The determination result is output from the determination result output unit 345 based on the processing output of the color unevenness determination unit 328D.

  The color unevenness determination unit 328D includes two processing units: a color unevenness detection processing unit 330 and a color unevenness degree calculation unit 340.

  The processing results of the color unevenness detection processing unit 330 and the color unevenness degree calculation unit 340 are stored and stored in the color unevenness detection result storage memory 342 and the color unevenness degree calculation result storage memory 344.

  In this case, first, the multispectral image data of the object stored in the multispectral image frame memory 318 is provided to the normal part data creating unit 349 of the route 1 by the switch 348 in the color unevenness determining unit 328D, and the same feature amount as described above. It is input via the extraction unit 329A.

  In the normal part data creating section 349, first, the normal part data calculating section 350 calculates the normal part data and stores it in the normal part data storage memory 351.

  Then, the process is switched to the route 2 process by the switch 348, and the color nonuniformity determination processing is performed by the color nonuniformity detection processing unit 330 and the color nonuniformity degree calculation unit 340 through the same feature amount extraction units 329B and 329C as described above. At this time, the color nonuniformity determination unit 328B performs a color nonuniformity detection process and a color nonuniformity degree calculation while referring to normal part data stored in advance in the normal part data storage memory 351 and outputs a determination result.

  As the normal part data creation unit 349, a reference colorimetric value, a reference standard deviation, or the like that is input or loaded from a file can be used.

  In this case, if the normal part data is created once, and the same normal part data may be used from the next object, the processing of route 2 may be started.

  In addition, the normal part data creating unit 349 applies a low-pass filter to the average and median values in the detection area calculated from the multispectral image data of the object stored in the multispectral image frame memory 318, and to the entire detection area. Image can be used.

  In this case, it is necessary to perform the processing of the route 1 and the route 2 in a permutation every time.

  By doing so, it is possible to inspect the color unevenness of the object without specifying the normal part learning data.

(Eleventh embodiment)
FIG. 28 shows a color unevenness inspection according to an eleventh embodiment in which a normal part data creation unit 349 and a new class creation unit 354 are provided in the color unevenness determination unit 328E to perform color unevenness determination and color unevenness inspection. FIG. 2 is a block diagram illustrating a configuration of the device.

  In FIG. 28, a multispectral image frame memory 318 corresponds to the frame memory 118 of FIG. 1, and stores an image of an object imaged by a CCD image sensor or the like capable of capturing a multispectral image similarly to the case of FIG. It is assumed that multispectral image data is stored.

  The multispectral image data of the object stored in the multispectral image frame memory 318 is read out by the color nonuniformity determining unit 328E and subjected to a predetermined process.

  The determination result is output from the determination result determination unit 352 based on the processing output of the color unevenness determination unit 328E.

  The color unevenness determination unit 328E includes two processing units: a color unevenness detection processing unit 330 and a color unevenness degree calculation unit 340.

  The processing results of the color unevenness detection processing unit 330 and the color unevenness degree calculation unit 340 are stored and stored in the color unevenness detection result storage memory 342 and the color unevenness degree calculation result storage memory 344.

  In this case, first, the multispectral image data of the object stored in the multispectral image frame memory 318 is processed by the switch 348 through the same characteristic amount extracting unit 329 in the color nonuniformity determining unit 328E as the normal route 1 by the switch 348. The data is input to the copy data creation unit 349.

  The normal part data creating unit 349 creates normal part data in the same manner as described above.

  Next, the switch to the route 2 is performed by the switch 348, and the uneven color detection processing section 330 performs the uneven color detection process in the same manner as described above, and stores the result of the uneven color detection in the uneven color detection result storage memory 342. Is done.

  Then, the new class creation unit 354 determines whether or not to update the class data while referring to the color unevenness determination result from the determination result determination unit 352 by the class data update unit 353, that is, a new class is generated based on the class data. Check if it can be created, and if a new class can be created, create a new class.

  Then, based on the new class, a method of giving feedback to the color unevenness detection processing again is executed.

  As for how a new class is created by the new class creation unit 354, a new class is created by interpolation using the Euclidean or Mahalanobis distance in the learning processing unit 355, and the new class is registered in the new class registration unit 356. can do.

  Further, the new class creation unit 354 can register a new class by clustering the inner product value of the vector of each pixel vector in the multispectral space by a threshold.

  Alternatively, the new class creation unit 354 may register a new class by learning each pixel stored in the color unevenness detection result storage memory 342 by the above-described FS conversion or multivariate analysis. .

  As described above, in the eleventh embodiment, by using the feedback process in the uneven color detection process, only the binary value indicating only the presence or absence of the uneven color can be output in the uneven color detection of the fourth embodiment. However, color unevenness can be output in multiple values.

  FIGS. 29 and 30 show flowcharts in the case of performing color unevenness multi-value determination according to the eleventh embodiment.

  When the color unevenness detection process is started, first, multispectral image data is input, a detection area average vector is created, and the average vector is used as normal part learning data (steps S130 and S131).

  Then, after dividing the detection area, the maximum value of the maximum color unevenness degree is initialized to Max = 0, and the division area number is also initialized to i = 0 (steps S132, S133, and S134).

  After obtaining the average vector for each of the divided areas i, the distance (Euclidean, Mahalanobis) D in the multidimensional space from the normal part learning data is calculated (steps S135 and S136).

  Then, it is determined whether or not this D is the maximum value. First, since Max = 0 is set, the first time always branches to YES. That is, it is assumed that Max = D is checked (steps S137 and S138).

  If the distance D is greater than Max, that is, if it is greater than the color unevenness degree Max, then input Max = D, and use the average vector of the divided area i as the re-distant learning data to obtain the largest unevenness of the learning data. (Step S139).

  Then, it is determined whether or not the determination has been completed for all the divided areas. If the determination has not been completed, the divided area number is incremented to i = i + 1, and the processing after step S135 is repeated (step S140, S141).

  Then, after performing the determination in all the divided areas, the number of classes to be determined is determined for multi-value output (in this case, the number of classes is assumed to be 5 and the number of classes is divided into five levels). Step S142).

  Then, a difference vector between the average vector of the normal part learning data and the average vector of the uneven learning part data at the farthest distance is obtained (step S144).

  By dividing this difference vector into equal classes and calculating the average vector of each class, multiple classes can be defined.

  Then, in order to perform the color unevenness determination process again, the divided area number is initialized to i = 0 again, and the average vector of the divided area is repeatedly calculated as to which class is the closest to the vector of the learning data of the average vector ( Steps S145, S146, S147).

  Then, it is determined whether or not the determination has been completed for all the divided areas. If not, the divided area number is incremented to i = i + 1, and the processing of step S146 and subsequent steps are repeated (step S148, S149).

  Then, the learning data is updated from the calculation, the determination result determination unit 352 checks whether or not to perform the reexamination, and performs a feedback process if necessary (steps S150 and S151).

  By providing the new class creating unit 356 in this manner, only binary output was possible in the color nonuniformity detection of the fourth embodiment. However, in the eleventh embodiment, color nonuniformity can be output in multiple values. become able to.

  That is, in the eleventh embodiment, as described above, the feedback process is repeatedly performed, and learning is performed based on the result output in binary in the first color nonuniformity determination. Classification vectors in multispectral space can be created.

  The second color non-uniformity determination processing unit can output the color non-uniformity in a multi-valued manner by projecting onto the obtained classification vector.

(Twelfth embodiment)
FIG. 31 shows a twelfth embodiment according to the twelfth embodiment in which a color unevenness determination unit 369, a determination result output unit 345, and a processing control unit 358 are provided to reduce processing time for color unevenness determination and color unevenness inspection. FIG. 2 is a block diagram illustrating a configuration of a color shading inspection apparatus.

  In FIG. 31, a multispectral image frame memory 318 corresponds to the frame memory 118 in FIG. 1, and similarly to the case of FIG. 1, a multi-spectral image capturing target object imaged by a CCD image sensor or the like. It is assumed that multispectral image data is stored.

  The multispectral image data of the object stored in the multispectral image frame memory 318 is read out by the color nonuniformity determining unit 340 and subjected to a predetermined process when the switch 357 switches the automatic determination mode.

  Then, the determination result is output from the determination result output unit 345 based on the processing output of the color unevenness determination unit 340.

  In FIG. 31, when the inside of the control unit 126 in FIG. 1 is divided into a hardware control unit and a processing control unit, the processing control unit 358 shows a configuration corresponding to the latter processing control unit. The accuracy of unevenness determination and color unevenness inspection can be improved, and the processing time can be shortened.

  The configuration of the processing control unit 358 can be applied to the case where the multi-class classification is performed in the above-described first to sixth embodiments.

  The processing control unit 358 includes a color unevenness inspection area determining unit 363, a used multispectral image determining unit 366, and a processing order determining unit 360.

  First, in the detection area determination unit 363, at the time of the first processing, the image processing unit 364 performs image processing, and stores the detection area itself in the detection area storage memory 365.

  When color unevenness determination is performed in the next process, color unevenness determination is performed with reference to the detection area storage memory 365.

  The detection area determination unit 363 determines not only the area surrounded by a rectangle in the image up to now, but also the color unevenness of a complicated part by binarizing, extracting an outline, or extracting a specific color. The processing can be performed, and the time for the color nonuniformity determination processing can be reduced by limiting the determination area and reducing the number of pixels.

  Further, the used multispectral image determining unit 366 calculates the feature amount of the multispectral image by the processing of the image feature amount calculating unit 367 at the time of the first processing, and uses the used image determining unit 368 based on the feature amount. The image is determined, and the determined use image number is stored in the use image number storage memory 369. .

  Then, the color unevenness determination unit 340 determines whether to use or not use the image in the filter while referring to the used image number storage memory 369, and then performs the same color unevenness determination processing as in the above-described embodiment. The determination result is output from the determination result output unit 345.

  Further, the used multispectral image determining unit 366 can use the contrast, the gradation difference, the feature amount of the density histogram, the average value, the standard deviation, and the like.

  Further, when the switch 357 switches the learning determination mode, the processing order determination unit 360 calculates the processing time via the processing time calculation unit 361 to determine the processing order, and stores the processing order in the processing order storage memory 362. To be stored.

  Here, the processing order determination is to determine which of the processes in the detection area determination unit 363 and the used multispectral image determination unit 366 is performed first, and the processing order stored in the processing order storage memory 362. The order of processing is switched by the switch 359 according to the data.

  The learning determination mode is a case where color unevenness determination is performed for each process through the detection area determination unit 363 and the used multispectral image determination unit 366 each time.

  In addition, the automatic determination mode is a case where color unevenness determination of an unknown object is sequentially performed when the measurement conditions do not change, that is, when the measurement conditions are constant.

  First, in a state where nothing is stored in the detection area storage memory 365 or the used image number storage memory 369, various determinations are made in order to store the detection area and the number of the used multispectral image in those memories. Will be.

  In the first learning determination mode, first, each detection area is determined from the multispectral image data of the target object, the multispectral image to be used is determined, and the required processing time is calculated.

  Then, the order of processing is determined from the processing time, and is stored in the processing order storage memory 362.

  When the processing order data is fed back to the switch 359, for example, if the time for determining the multispectral image is longer than the time for determining the detection area, the detection area must be determined first before the image to be used is determined. Will do.

  If the time required to determine the detection area is longer than the determination of the used multispectral image, the used multispectral image is determined first.

  In this way, by changing the processing order based on the processing time, the overall processing time can be reduced.

  When the color unevenness determination unit 340 performs the color unevenness determination in the learning determination mode or the automatic determination mode, the color unevenness determination processing is performed with reference to the detection area storage memory 365 and the used image number storage memory 369. The result can be sent to the determination result output unit 345.

  The effects of the seventh to twelfth embodiments described above are summarized as follows.

  (1) It is possible to realize a color unevenness inspection system capable of quantifying the degree of color unevenness of an object.

  (2) When performing the color unevenness inspection, it is determined that the color difference is not less than a certain color difference, and that the color difference is not more than a certain value.

  (3) In the case where the detection processing and the color unevenness calculation are performed using the colorimetric values, the color unevenness can be calculated for each filter and the overall color unevenness can be quantified.

  (4) Inspection of color unevenness of an object can be performed without specifying normal part learning data.

  (5) Color unevenness can be output in multiple values.

  (6) The accuracy of color unevenness determination and color unevenness inspection can be improved, and the processing time can be shortened.

  Therefore, according to the present invention, the color classification processing of an object is performed using a multispectral image obtained through a plurality of band-pass filters, and the apparatus configuration is simple, low-cost, and mechanical. It can withstand vibration, and even when the spectrum changes without limiting the light source, it can satisfactorily perform color classification of the object, and is the optimal classification judgment for classifying the object. By using the method, it is possible to provide a color classification device and a color classification method capable of further improving the classification accuracy.

  Further, according to the present invention, the color unevenness inspection process for an object is performed using a multispectral image obtained through a plurality of band-pass filters, and the apparatus configuration is simple, low-cost, and Of the target object even if the spectrum changes without limiting the light source, and the accuracy of the target object color unevenness inspection process Can be provided.

FIG. 1 is a block diagram showing a color classification device according to a basic embodiment of the present invention. FIG. 2 is a schematic diagram of a rotating color filter used in the basic embodiment of the present invention. 5 is a flowchart illustrating the operation of the basic embodiment of the present invention. FIG. 1 is a block diagram illustrating a classification operation circuit according to a basic embodiment of the present invention, a block diagram illustrating a classification operation unit according to a basic example, and a block diagram illustrating a classification spectrum selection circuit according to a basic example. FIG. 2 is a block diagram showing a color classification operation circuit according to the first embodiment of the present invention. FIG. 2 is a block diagram illustrating a classification determining unit according to the first embodiment of the present invention. FIG. 2 is a block diagram showing a classification determining unit according to the first embodiment of the present invention. FIG. 6 is a block diagram showing a modification of the classification determining unit according to the first embodiment of the present invention. FIG. 9 is a diagram illustrating a modification of the classification determining unit according to the first embodiment of the present invention. FIG. 7 is a diagram illustrating an example of a classification result according to the first embodiment of the present invention. FIG. 9 is a block diagram illustrating a color classification operation circuit according to a second embodiment of the present invention. FIG. 13 is a block diagram illustrating a color class arithmetic circuit according to a third embodiment of the present invention. 13 is a flowchart illustrating a flow of a process according to the third embodiment of the present invention. FIG. 14 is a block diagram illustrating a color classification calculation circuit according to a fourth embodiment of the present invention. 13 is a flowchart illustrating a flow of processing when color unevenness is detected in the fourth embodiment of the present invention. FIG. 14 is a block diagram illustrating a color classification calculation circuit according to a fifth embodiment of the present invention. 15 is a flowchart illustrating the flow of a process according to the fifth embodiment of the present invention. FIG. 14 is a block diagram illustrating a color classification calculation circuit according to a sixth embodiment of the present invention. 15 is a flowchart showing the flow of processing of main parts according to a sixth embodiment of the present invention. FIG. 1 is a block diagram showing a basic configuration of a color unevenness inspection apparatus according to the present invention. FIG. 14 is a block diagram showing a color shading inspection apparatus according to a seventh embodiment of the present invention. FIG. 16 is a block diagram showing a color shading inspection apparatus according to an eighth embodiment of the present invention. FIG. 19 is a block diagram showing a color non-uniformity inspection apparatus according to a ninth embodiment of the present invention. 18 is a flowchart showing the flow of processing of main parts of the color nonuniformity inspection device according to the ninth embodiment of the present invention. 18 is a flowchart showing the flow of processing of main parts of the color nonuniformity inspection device according to the ninth embodiment of the present invention. 18 is a flowchart showing the flow of processing of main parts of the color nonuniformity inspection device according to the ninth embodiment of the present invention. FIG. 16 is a block diagram showing a color shading inspection apparatus according to a tenth embodiment of the present invention. FIG. 21 is a block diagram showing a color shading inspection apparatus according to an eleventh embodiment of the present invention. 21 is a flowchart illustrating a flow of processing of a main part of the color nonuniformity inspection device according to the eleventh embodiment of the present invention. 21 is a flowchart illustrating a flow of processing of a main part of the color nonuniformity inspection device according to the eleventh embodiment of the present invention. FIG. 18 is a block diagram showing a color shading inspection apparatus according to a twelfth embodiment of the present invention. FIG. 4 is a diagram illustrating a classification boundary of a conventional color identification device. The figure which shows the structure of the conventional color identification device. FIG. 9 is a diagram illustrating an example of a classification spectrum of a conventional color identification device.

Explanation of reference numerals

101 ... housing,
110 ... optical system,
101 ... Aperture,
126 ... aperture control circuit,
112A to 112E: band pass filters,
112 ... rotating color filter,
123 ... Filter position sensor,
124 ... motor,
124a: motor drive circuit,
114 ... image sensor,
115 ... amplifier,
116 ... A / D converter,
118 ... frame memory,
128: Classification operation circuit,
129: Exposure value memory,
126 ... control circuit,
122 ... Imaging element drive circuit,
30: luminance component extraction unit
13 ... Switch,
14 ... Classification judgment unit,
15 learning control unit,
16 Classification calculation unit
17 ... Classification determination unit,
41, 42... Classification judgment unit,
51 ... determination result calculation unit
71 ... Class information database,
72 ... classification judgment method selection function unit
73 ... Database update unit,
81: Multi-spectral image capturing unit
82 image processing unit,
83 image selection means
84 ... Classification means,
318 ... multi-spectral image frame memory
328: Classification operation circuit,
329 (A, B, C) ... feature amount extraction unit,
330 ... classification determination unit,
340: Classification evaluation unit (color unevenness determination unit)
341: Uneven color detection processing unit
342: Color unevenness detection result storage memory
343: color unevenness degree calculation unit,
344: memory for storing the results of calculating the degree of color unevenness,
345: judgment result output unit
347: colorimetric value calculation unit
348 ... switch,
349: normal part data creation part
353: Class data update unit
354: New class creation unit,
357 ... Switch,
358: processing control unit,
360 ... processing order determination unit,
363: detection area determination unit
366: Multi-spectral image determining unit to be used.

Claims (22)

Multi-spectral image capturing means for capturing the reflected light of the object as a multi-spectral image having different bands,
A classification spectrum, which is a vector for classifying a specific object using a statistical method, is calculated from multispectral image data of the object imaged by the multispectral image imaging means, and the object is calculated using the classification spectrum. Classification means for performing classification of a plurality of classes,
The classification means,
A color classification device, comprising: a plurality of classification determination units for performing the classification determination of the plurality of classes by a plurality of different types of classification determination methods.   The color classification apparatus according to claim 1, wherein the plurality of classification determination units are connected in series, and perform classification determination by a plurality of different classification determination methods in a superimposed manner. The classification means further includes:
A class information database in which the class information of the object is stored in advance,
2. The color according to claim 1, further comprising: a classification determination selection function unit that selects a classification process and a narrowing method in the plurality of classification determination units based on the class information from the class information database. Classifier. The classification means further includes:
The color classification apparatus according to claim 1, further comprising a determination result determination unit configured to determine one or more determination results of the plurality of classification determination units. The determination result determination unit includes:
5. The method according to claim 4, wherein a determination is made as to whether or not to perform a classification determination in a second classification determination unit based on a determination result of the first classification determination unit in the plurality of classification determination units. Color classification device. The determination result determination unit includes:
After the classification judgments in the first and second classification judgment units in the plurality of classification judgment units are performed in a superimposed manner, the second classification judgment unit is re-based on the judgment result of the second classification judgment unit. 5. The color classification apparatus according to claim 4, wherein it is determined whether or not to perform the classification determination in step (a). The classification means further includes:
7. A learning control unit according to claim 1, further comprising a learning control unit that performs predetermined learning on the multispectral image data in advance and controls a classification process by the classification unit based on the learning data. The described color classification device. The classification means further includes:
The color classification device according to claim 1, further comprising a learning data updating unit that updates learning data while performing the classification determination. The classification means further includes:
The feature amount is extracted from the multispectral image data of the object imaged by the multispectral image imaging means, and only the multispectral image data in which the extracted characteristic amount is larger than a predetermined value is selected, and the extracted The color classification apparatus according to any one of claims 1 to 8, further comprising an image selection unit that selects one or a plurality of spectral images to be used for the determination from the image data.   The color classification according to claim 9, wherein the feature amount extracted by the image selection unit includes at least one of a contrast and a gray level difference in a multispectral image captured by the multispectral image capturing unit. apparatus. The multi-spectral image capturing means,
The optical system according to claim 1, further comprising an optical unit configured to form reflected light of the object on the multispectral imaging unit as a multispectral image having different light wavelength bands. Color classification device. Multi-spectral image providing means for providing multi-spectral image data of the object,
A feature amount extracting unit that extracts a feature amount of multispectral image data from the multispectral image providing unit,
Color unevenness determining means for performing color unevenness determination based on the characteristic amount from the characteristic amount extracting means;
A non-uniformity determination device that outputs a non-uniformity determination result based on the non-uniformity determination result from the non-uniformity determination unit. The color unevenness determination unit includes:
The image processing apparatus further includes a color nonuniformity detection processing unit and a color nonuniformity degree calculation unit that perform color nonuniformity detection processing and color nonuniformity degree calculation in parallel or permutation based on multispectral image data from the multispectral image providing unit. An uneven color inspection apparatus according to claim 12. The color unevenness determination unit includes:
A color unevenness detection processing result storage memory and a color unevenness degree calculation result storage memory for storing the color unevenness detection processing result and the color unevenness degree calculation result from the color unevenness detection processing means and the color unevenness degree calculation means, respectively. The color non-uniformity inspection device according to claim 13, wherein The color unevenness determination unit includes:
A colorimetric value calculation unit that calculates a colorimetric value for obtaining at least one of a colorimetric value and a color difference value of a detection area in at least one of the color nonuniformity detection processing unit and the color nonuniformity degree calculating unit. The color unevenness inspection apparatus according to claim 13, wherein: The color unevenness determination unit includes:
A color for storing a color unevenness detection processing result and a color unevenness degree calculation result from the color unevenness detection processing unit and the color unevenness degree calculation unit, and a color difference value of a detection area obtained from the colorimetric value by the colorimetric value calculation unit. 16. The color non-uniformity inspection apparatus according to claim 15, further comprising a non-uniformity detection processing result storage memory and a color non-uniformity degree calculation result storage memory. The color unevenness determination unit includes:
It further comprises normal part data creating means for creating normal part data in advance,
14. The color according to claim 13, wherein said normal part data by said normal part data creating means can be referred to at the time of color unevenness detection processing and color unevenness degree calculation by said color unevenness detection processing means and color unevenness degree calculating means. Unevenness inspection device. The color unevenness determination unit includes:
A determination result determination unit configured to determine a color unevenness detection processing result and a color unevenness degree calculation result from the color unevenness detection processing unit and the color unevenness degree calculation unit;
Class data updating means for updating class data based on the judgment of the judgment result by the judgment result judging means;
New class creating means for creating a new class based on the class data by the class data updating means,
14. The color non-uniformity inspection according to claim 13, wherein the new class created by the new class generation means can be fed back at the time of the color non-uniformity detection processing and the color non-uniformity calculation by the color non-uniformity detection processing means and the color non-uniformity degree calculation means. apparatus. The color unevenness determination unit includes:
Further comprising a detection area determining means for determining a detection area in advance,
14. The color non-uniformity inspection according to claim 13, wherein the detection area by the detection area determining means can be referred to during the color non-uniformity detection processing and the color non-uniformity calculation by the color non-uniformity detection processing means and the color non-uniformity degree calculation means. apparatus. The color unevenness determination unit includes:
Further comprising a used multispectral image determining means for determining a used multispectral image in advance based on the multispectral image data from the multispectral image providing means,
14. The color multi-spectrum image used by the color multi-spectrum image determining means and the color non-uniformity degree calculating means can be referred to at the time of color non-uniformity detection processing and color non-uniformity degree calculating means. Color unevenness inspection equipment. The color unevenness determination unit includes:
Detection area determining means for determining a detection area in advance;
A used multispectral image determining means for determining a used Malte spectrum image in advance based on the multispectral image data from the multispectral image providing means,
Further comprising a processing order determining means for determining the processing order with the detection area determining means and the use Malte spectrum image determining means in advance,
Based on the processing order by the processing order determining means, the detection area by the detection area determining means and the used multispectral image by the used multispectral image determining means are subjected to the color unevenness detection processing by the color unevenness detecting processing means and the color unevenness degree calculating means. The color non-uniformity inspection apparatus according to claim 13, wherein the color non-uniformity degree can be referred to when calculating the color non-uniformity degree. A color classification method for performing color classification determination by a plurality of types of determination methods,
Extracting multi-dimensional data by extracting the luminance component of the color measurement area of the image captured using a plurality of band-pass filters;
Performing a classification determination operation on the multidimensional data to extract a neighborhood class;
It is determined whether the neighborhood class is one, and when it is determined that there is one, it is further determined whether the certainty factor of the determination class is equal to or more than a predetermined value, and when it is determined that the certainty factor of the determination class is equal to or more than a predetermined value, Determining a classification;
It is determined whether there are a plurality of the neighboring classes, and when it is determined that there are a plurality, or when it is determined that the certainty factor of the determination class is not equal to or more than a predetermined value, a classification determination calculation process different from the classification determination calculation process is performed. Applying and determining a classification;
A color classification method comprising:

Images