JP4695571B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method.

  2. Description of the Related Art Imaging devices that capture still images and moving images using solid-state imaging devices such as CCD (Charge Coupled Devices) elements and CMOS (Complementary Metal Oxide Semiconductor) elements are widely used. As in conventional silver halide cameras, there are many imaging devices on the market that can attach and detach a lens and use a lens according to the situation in an imaging device using a CCD element or a CMOS element.

  However, in such an interchangeable lens type image pickup apparatus in which the lens can be attached and detached, there is a risk that dust and dirt (hereinafter collectively referred to as “dust”) may adhere to the image pickup element when the lens is exchanged. is there. If shooting is performed with dust attached to the image sensor, light does not enter the pixels of the image sensor to which dust has adhered, and the portion appears dark, causing a problem in the image.

  The presence of attached dust is difficult to recognize at the time of photographing, and is often noticed only when an image is displayed on a personal computer or the like after printing or printed by a printing device such as a printer. In addition, dust attached to the image sensor is often of a minute size that cannot be seen with the eyes, and it is difficult to determine which part of the image sensor is attached. Furthermore, if careless care is taken to remove the attached dust, the low-pass filter provided in front of the image sensor may be damaged.

  Therefore, a technique is known in which the photographed image is analyzed to detect the presence or absence of dust attached to the image sensor and the position of the dust so that the photographer can easily remove the dust (Patent Documents 1 to 3). 3).

Japanese Patent No. 3461482 Japanese Patent Laid-Open No. 2001-157087 Japanese Patent Laid-Open No. 2005-341181

  However, in the method disclosed in Patent Document 1, not only dust attached to the image sensor but also a so-called pixel defect that has lost the photoelectric conversion function is recognized as dust as in the case where dust is attached. Unlike the attached dust, the image defect is meaningless even if the removal operation is performed, and there is a problem that the removal operation may cause damage to the image sensor (low-pass filter).

  In the method shown in Patent Document 2, the position of the dust displayed on the captured image is different from the position of the dust actually attached on the image sensor surface, so it is difficult to determine the location of the dust. There is a problem that this method may cause damage to the image sensor (low-pass filter) by performing the removal operation on a position different from the position where dust is attached on the image sensor.

  In the method shown in Patent Document 3, when the same location in the captured image is continuously lower than a predetermined signal level in the captured image, the location is determined as a dust attachment location. However, in this method, since it is expected that there are few consecutive opportunities, there is a problem in dust detection accuracy, and as in Patent Document 1, there is a problem that it is difficult to distinguish from pixel defects. It was.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to improve the detection accuracy of dust attached to the image sensor and to clearly indicate the dust attachment position. It is an object of the present invention to provide a new and improved imaging apparatus and imaging method capable of simplifying the removing operation.

  In order to solve the above-described problems, according to an aspect of the present invention, an image is formed on an imaging plane through a lens unit including a diaphragm mechanism that adjusts the amount of incident light from a subject, and the lens unit. An imaging unit that sequentially outputs an image of a subject sequentially as image data, an aperture control unit that adjusts the amount of incident light from the subject by changing the aperture value of the aperture mechanism unit during continuous imaging by the imaging unit, and imaging A foreign matter adhesion prediction unit that predicts whether or not foreign matter is adhered on the imaging surface based on at least two or more image data having different aperture values among the image data sequentially output from the unit An imaging device is provided.

  According to such a configuration, the lens unit receives the image light from the subject, and the imaging unit continuously captures the image of the subject imaged on the imaging surface via the lens unit using the imaging element. The aperture control unit adjusts the amount of incident light from the subject by changing the aperture value of the aperture mechanism unit included in the lens unit, and the foreign matter adhesion predicting unit outputs image data sequentially output from the imaging unit. Of these, based on at least two or more pieces of image data having different aperture values, it is predicted whether or not a foreign substance is adhered on the imaging surface. As a result, it is possible to improve the detection accuracy of dust adhering to the image sensor by analyzing at least two or more image data having different aperture values taken continuously.

  The imaging apparatus includes a pixel position detection unit that detects a pixel position on the imaging surface where adhesion of a foreign material is predicted based on a prediction result of the foreign material adhesion prediction unit, and each pixel position corresponding to the pixel position on the imaging surface. A foreign substance determination unit that compares the luminance levels of the image data and determines that a foreign substance is attached to the pixel position when a difference equal to or greater than a predetermined threshold value occurs. According to this configuration, the pixel position detection unit detects the pixel position on the imaging surface where the adhesion of the foreign material is predicted based on the prediction result of the foreign material adhesion prediction unit, and the foreign material determination unit detects the luminance level of each image data. Are compared, and it is determined that foreign matter is attached to the pixel position when a difference equal to or greater than a predetermined threshold value occurs. As a result, the detection accuracy of dust attached to the image sensor can be improved by comparing the brightness levels of the image data.

  The foreign matter determination unit calculates an average value of the luminance levels of the entire image data for each constituent color of the color filter included in the image sensor, and the luminance levels of the pixel positions on the imaging plane in the image data are all If it is below the average value, it may be determined that foreign matter is attached to the pixel position. According to such a configuration, the foreign matter determination unit determines that a foreign matter is attached to the pixel position when the luminance level of the pixel position on the imaging surface in each image data is lower than the average value. As a result, continuous pixels that satisfy the predetermined condition are extracted as pixels on which a foreign object is temporarily attached, so that a pixel on which a foreign object is attached can be distinguished from a simple pixel defect.

  The foreign matter adhesion prediction unit may select image data obtained from incident light when the aperture value of the aperture mechanism unit is the largest among the image data sequentially output from the imaging unit. According to this configuration, the foreign matter adhesion prediction unit selects image data obtained from incident light when the aperture value of the aperture mechanism unit is the largest among the image data sequentially output from the imaging unit. As a result, it is possible to improve the detection accuracy of pixels to which foreign matter is attached.

  The imaging apparatus may include a display unit that displays a determination result by the foreign matter determination unit. According to such a configuration, the display unit displays the determination result by the foreign matter determination means. As a result, it is possible to simplify the dust removal operation by clearly indicating the result of the foreign matter adhesion.

  The display unit may display the adhesion position of the foreign matter corresponding to the imaging region on the imaging plane. According to such a configuration, the display unit displays the adhesion position of the foreign matter corresponding to the imaging region on the imaging plane. As a result, the displayed position corresponds to the actual position where the foreign substance is attached, so that the foreign substance removal operation can be simplified.

  In order to solve the above-described problem, according to another aspect of the present invention, a lens unit including a diaphragm mechanism that adjusts the amount of incident light from a subject is used to form an image on the image plane via the lens unit. An imaging step that sequentially outputs image of the subject imaged in succession as image data, and an aperture that adjusts the amount of incident light from the subject by changing the aperture value of the aperture mechanism during continuous imaging in the imaging step A foreign substance adhesion prediction step for predicting whether or not a foreign substance is adhered on the imaging surface based on at least two or more image data having different aperture values among the image data sequentially output in the control step and the imaging step The imaging method characterized by including these is provided.

  According to this method, the imaging step sequentially outputs the image of the subject imaged on the imaging surface sequentially as image data via the lens unit that receives the image light from the subject, and the aperture control step includes the lens control step. The amount of incident light from the subject is adjusted by changing the aperture value of the aperture mechanism included in the unit, and the foreign matter adhesion prediction step includes at least two or more different aperture values among the image data sequentially output from the imaging unit. Based on the image data, it is predicted whether or not a foreign substance is adhered on the imaging surface. As a result, it is possible to improve the detection accuracy of dust adhering to the image sensor by analyzing at least two or more image data having different aperture values taken continuously.

  The imaging method corresponds to a pixel position detection step for detecting a pixel position on the imaging surface where the adhesion of a foreign material is predicted based on a prediction result in the foreign material adhesion prediction step, and a pixel position on the imaging surface. A step of comparing the brightness levels of the image data, and determining a foreign matter adhering to the pixel position when a difference equal to or greater than a predetermined threshold occurs. According to this method, the pixel position detection step detects the pixel position on the imaging surface where the adhesion of the foreign material is predicted based on the prediction result in the foreign material adhesion prediction step, and the foreign material determination step determines the brightness of each image data. The levels are compared, and it is determined that foreign matter is attached to the pixel position when a difference equal to or greater than a predetermined threshold occurs. As a result, it is possible to improve the detection accuracy of dust adhering to the image sensor by comparing the luminance levels of the continuously captured image data.

  In the foreign matter determination step, an average value of the luminance levels of the entire image data is calculated for each of the constituent colors of the color filter included in the image sensor, and the luminance levels of the pixel positions on the image plane in each image data are averaged. When the value is less than the value, it may be determined that a foreign substance is attached to the pixel position. According to such a method, in the foreign matter determination step, it is determined that the foreign matter is attached to the pixel position when the luminance level of the pixel position on the imaging surface in each image data is lower than the average value. As a result, continuous pixels that satisfy the predetermined condition are extracted as pixels on which a foreign object is temporarily attached, so that a pixel on which a foreign object is attached can be distinguished from a simple pixel defect.

  In the foreign matter adhesion prediction step, image data obtained from incident light when the aperture value of the aperture mechanism unit is the largest may be selected from among the image data sequentially output in the imaging step. According to such a method, the image data obtained from the incident light when the aperture value of the aperture mechanism unit is the largest is selected from the image data sequentially output in the imaging step in the foreign matter adhesion prediction step. As a result, it is possible to improve the detection accuracy of pixels to which foreign matter is attached.

  The imaging method may include a display step for displaying a determination result in the foreign matter determination step. According to this method, the display step displays the determination result obtained by the foreign substance determination step. As a result, the dust removal operation can be simplified by clearly indicating the dust attachment position.

  In the display step, the adhesion position of the foreign matter may be displayed corresponding to the imaging region on the imaging plane. According to such a method, in the display step, the adhesion position of the foreign matter is displayed corresponding to the imaging region on the imaging plane. As a result, the displayed position corresponds to the actual position where the foreign substance is attached, so that the foreign substance removal operation can be simplified.

  As described above, according to the present invention, it is possible to provide a new and improved imaging apparatus and imaging method capable of simplifying the dust removal operation by clearly indicating the dust attachment position.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

(First embodiment)
First, an imaging apparatus and an imaging method according to the first embodiment of the present invention will be described.

  FIG. 1 is an explanatory diagram illustrating an imaging apparatus according to the first embodiment of the present invention. Hereinafter, an imaging apparatus according to the first embodiment of the present invention will be described with reference to FIG.

  The imaging device 100 is an imaging device according to the first embodiment of the present invention. The imaging apparatus 100 has a structure in which the lens unit 102 can be attached. Note that when the imaging apparatus 100 inspects dust using the imaging method according to the first embodiment of the present invention, the light diffusing lens cap 104 may be attached to the lens unit 102. By using the light diffusing lens cap 104, light incident from the subject diffuses, and dust can be detected more effectively.

  The image pickup apparatus according to the first embodiment of the present invention has been described with reference to FIG. Next, the configuration of the imaging apparatus according to the first embodiment of the present invention will be described.

  FIG. 2 is an explanatory diagram illustrating the configuration of the imaging apparatus according to the first embodiment of the present invention. Hereinafter, the configuration of the imaging apparatus according to the first embodiment will be described with reference to FIG.

  As shown in FIG. 2, the imaging apparatus 100 according to the first embodiment of the present invention includes a CCD element 110, an amplifier-integrated CDS (Correlated Double Sampling) circuit 112, an A / D converter 114, An image input controller 116, an image signal processing unit 118, a compression processing unit 120, a video encoder 122, an image display unit 124, a timing generator 126, a CPU (Central Processing Unit) 128, and a ROM (Read Only Memory). 130, a RAM (Random Access Memory) 132, a media controller 134, a recording medium 136, a motor driver 138, a shutter button 140, and an operation unit 142.

  In addition, the lens unit 102 to which the imaging apparatus 100 according to the first embodiment of the present invention can be attached includes a diaphragm 106 and a driving device 108.

  The diaphragm 106 adjusts the amount of light incident on the CCD element 110 when taking an image. The diaphragm 106 is controlled by the driving device 108. In the present embodiment, the aperture range can be adjusted between AV (Aperture Value) 3 to AV7 (F2.8 to F11). In the present invention, the aperture range is not limited to this.

  The CCD element 110 is an example of an imaging element of the present invention, and is an element for converting light incident from the lens unit 102 into an electrical signal. In this embodiment, the incident light is controlled by the electronic shutter to adjust the time for extracting the electric signal. However, the time for extracting the electric signal may be adjusted by controlling the incident light using the mechanical shutter. In the first embodiment of the present invention, the lens unit 102 and the CCD element 110 constitute an imaging unit.

  In the present embodiment, the CCD element 110 is used as the imaging element. However, the present invention is not limited to this example, and a CMOS (Complementary Metal Oxide Semiconductor) element may be used instead of the CCD element 110 as the imaging element. Other image sensors may be used. Since the CMOS element can convert the video light of the subject into an electrical signal at a higher speed than the CCD element, it is possible to shorten the time from when the subject is photographed until the image is displayed on the image display unit 124.

  The CDS circuit 112 is a circuit in which a CDS circuit that is a kind of sampling circuit that removes noise from the electrical signal output from the CCD element 110 and an amplifier that amplifies the electrical signal after removing noise are integrated. . In the present embodiment, the imaging apparatus 100 is configured using a circuit in which a CDS circuit and an amplifier are integrated. However, the CDS circuit and the amplifier may be configured as separate circuits.

  The A / D converter 114 converts the electrical signal generated by the CCD element 110 into a digital signal, and generates raw image data (image data).

  The image input controller 116 controls input of raw image data (image data) generated by the A / D converter 114 to the RAM 132.

  The image signal processing unit 118 performs light amount gain correction and white balance adjustment on the raw image data generated by the A / D converter 114.

  The compression processing unit 120 performs a compression process for compressing the image generated by the A / D converter 114 into image data of an appropriate format. The image compression format may be a reversible format or an irreversible format. As an example of a suitable format, you may convert into a JPEG (Joint Photographic Experts Group) format and a JPEG2000 format.

  The image display unit 124 performs live view display before performing a shooting operation, display of a captured image, and the like. In addition to the image display, various setting screens of the imaging apparatus 100 may be displayed. Display of image data and various information of the imaging apparatus 100 on the image display unit 124 is performed via the video encoder 122. As the image display unit 124, an LCD (Liquid Crystal Display) may be used.

  The timing generator 126 inputs a timing signal to the CCD element 110. The shutter speed is determined by the timing signal from the timing generator 126. In other words, the drive of the CCD element 110 is controlled by the timing signal from the timing generator 126, and the image signal from the subject is incident within the time that the CCD element 110 is driven, thereby generating an electrical signal that is the basis of the image data. The

  The CPU 128 issues a signal-related command to the CCD element 110, the CDS circuit 112, or the like, or issues an operation-related command for the operation of the operation unit 142. In the present embodiment, only one CPU is included, but a signal-related command and an operation-related command may be executed by separate CPUs.

  The ROM 130 stores a program for controlling the imaging apparatus 100.

  The RAM 132 temporarily stores captured images. Reading and writing of images to and from the RAM 132 is controlled by the image input controller 116. As the RAM 132, SRAM (Static RAM), DRAM (Dynamic RAM), SDRAM (Synchronous DRAM), FeRAM (Ferroelectric RAM), MRAM (Magnetic Resistive RAM), or the like may be used.

  The recording medium 136 records the captured image. Input / output to / from the recording medium 136 is controlled by the media controller 134. As the recording medium 136, a memory card that is a card-type storage device that records data in a flash memory can be used.

  The motor driver 138 controls the driving device 108 that operates the diaphragm 106 included in the lens unit 102. The amount of light of the subject is adjusted by operating the diaphragm 106 via the motor driver 138.

  The shutter button 140 is for performing a shooting operation, and focuses the subject in a half-pressed state (hereinafter, the half-pressed state of the shutter button 140 is also referred to as “S1 state”), and is fully pressed (hereinafter referred to as “S1 state”). , The state where the shutter button 140 is fully pressed is also referred to as “S2 state”).

  The operation unit 142 is provided with members for operating the imaging apparatus 100 and performing various settings at the time of shooting. The members arranged on the operation unit 142 are provided with a power button, a cross key for selecting a photographing mode and a photographing drive mode, a selection button, and the like.

  The configuration of the imaging apparatus according to the first embodiment of the present invention has been described above with reference to FIG. Next, the configuration of the CPU of the imaging apparatus according to the first embodiment of the present invention will be described.

  FIG. 3 is an explanatory diagram illustrating the configuration of the CPU according to the first embodiment of the present invention. Hereinafter, the configuration of the CPU of the imaging apparatus according to the first embodiment of the present invention will be described with reference to FIG.

  As shown in FIG. 3, the CPU 128 according to the first embodiment of the present invention includes a temporary dust candidate extraction unit 151, a temporary dust position extraction unit 152, a dust certification unit 154, a dust display unit 156, and an appropriate An AE calculation unit 158 is included.

  The temporary dust candidate extraction unit 151 is an example of a foreign matter adhesion prediction unit according to the present invention. The provisional dust candidate extraction unit 151 determines whether dust is attached on the CCD element 110 by analyzing a captured image, and the dust is attached. Are extracted as provisional garbage candidates.

  The temporary dust position extraction unit 152 is an example of the pixel position detection unit of the present invention, and acquires a pixel position on the CCD element 110 where dust is predicted to be attached from a photographed image. A method for extracting a part considered to be dust from the captured image will be described later.

  The dust recognition unit 154 is an example of the foreign matter determination unit according to the present invention. A part to which dust is actually attached is determined from a part where dust extracted by the temporary dust position extraction unit 152 is predicted to be attached. Authorize. A method for identifying the part to which the dust is attached will be described later.

  The dust display unit 156 causes the image display unit 124 to display a part to which the dust certified by the dust certification unit 154 is attached. When displaying the part to which the dust is attached on the image display unit 124, it may be displayed at a location corresponding to the position where the dust is actually attached to the CCD element 110.

The appropriate AE calculation unit 158 performs automatic exposure with the imaging apparatus 100 when photographing a subject, and acquires an EV (Exposure Value) value. Based on the acquired EV value, a proper combination of exposure time and shutter speed is determined. The EV value is changed by changing the aperture value and the shutter speed by setting EV = 0 as the amount of light with which an appropriate exposure can be obtained when the aperture value is F1 and the shutter speed is 1 second. The EV value can be obtained by EV = log ₂ (F ² / T) where F is the aperture value and T is the shutter speed. Therefore, the EV value increases as the shutter speed increases at the same aperture value, and the EV value increases as the aperture value increases at the same shutter speed.

  The CPU according to the first embodiment of the present invention has been described above with reference to FIG. Next, an imaging method according to the first embodiment of the present invention will be described.

  FIG. 4 is a flowchart illustrating the imaging method according to the first embodiment of the present invention. Hereinafter, the imaging method according to the first embodiment of the present invention will be described with reference to FIG.

  First, the operation unit 142 is operated to set the shooting mode to the dust shooting mode. By setting the dust shooting mode and shooting, dust attached to the CCD element 110 is detected. When the shooting mode is set to the dust shooting mode, the CPU 128 determines whether or not the shutter button 140 is pressed (step S110). If the shutter button 140 has not been pressed, the process returns to step S110, and the pressing determination process for the shutter button 140 is repeated. On the other hand, if the shutter button 140 is pressed, the appropriate AE calculation unit 158 performs photometry processing to determine the appropriate AE calculation unit 158 shutter speed and aperture, and obtains an exposure value EV0 that provides a proper exposure (step). S120). As a result of the photometric processing, it is determined whether or not the exposure value is within an appropriate exposure interlocking range in the dust shooting mode (step S130).

  In the present embodiment, the range of the shutter speed that does not affect the detection of dust is predetermined in the range of TV5 to TV10 (1/32 seconds to 1/1024 seconds). In the present embodiment, the minimum fluctuation range of the diaphragm 106 is AV2. Therefore, in the present embodiment, the exposure interlocking range in the dust shooting mode is 10 (AV3 + AV2 + TV5) to 16 (AV8−AV2 + TV10), and the exposure value EV0 that is appropriate exposure is 10 ≦ the exposure interlocking range in the dust shooting mode. EV0 ≦ 16. Therefore, in the case of EV0 <10 or EV0> 16, it is displayed on the image display unit 124 that the exposure is out of the range because dust photography is not possible (step S240), and the process ends. The range of the shutter speed and the minimum fluctuation range of the aperture are not limited to this example.

  As a result of the photometric processing, when the range of the exposure value EV0 is 10 ≦ EV0 ≦ 16, the aperture value and the shutter speed are obtained. First, in order to perform shooting when the aperture is opened (hereinafter, shooting when the aperture is opened is also referred to as “first shooting”), an aperture value and a shutter speed are obtained (step S140). The aperture value and shutter speed are calculated using a program diagram. FIG. 5 is an explanatory diagram showing an example of a program diagram used for calculating the aperture value and the shutter speed when the aperture is opened according to the present embodiment. For example, if the exposure value EV0 is 10 as a result of the photometric process, the shutter speed TV is set to 7 (1/128 seconds), and the aperture value AV is set to 3 (F2.8). Further, when the photometric processing results in an exposure value that can take a plurality of shutter speeds and aperture values, the black circles in the program diagram shown in FIG. 5 are adopted. For example, when the exposure value EV0 is 13, the shutter speed TV is set to 10 (1/1024 seconds), and the aperture value AV is set to 3 (F2.8).

  When the calculation of the shutter speed and aperture value is completed, the first shooting is performed using the calculated shutter speed and aperture value (step S150). An image shot by the first shooting (hereinafter, an image shot by the first shooting is also referred to as “first image”) is temporarily stored in the RAM 132.

  When the first shooting is completed, an aperture value and a shutter speed are obtained in order to perform shooting when the aperture is stopped (hereinafter, shooting when the aperture is stopped is also referred to as “second shooting”) ( Step S160). The aperture value and the shutter speed are calculated using a program diagram as in the first shooting. FIG. 6 is an explanatory diagram showing an example of a program diagram used for calculating the aperture value and the shutter speed when the aperture is stopped. For example, if the exposure value EV0 is 10 as a result of the photometric processing, the shutter speed TV is set to 5 (1/32 seconds), and the aperture value AV is set to 5 (F5.6). Similarly to the case of the first shooting, when the photometric processing is performed and the exposure value is capable of taking a plurality of shutter speeds and aperture values, the black circles in the program diagram shown in FIG. Is adopted. For example, when the exposure value EV0 is 12, the shutter speed TV is set to 7 (1/128 seconds), and the aperture value AV is set to 5 (F5.6).

  When the calculation of the shutter speed and aperture value is completed, the second shooting is performed using the calculated shutter speed and aperture value (step S170). An image shot by the second shooting (hereinafter, an image shot by the second shooting is also referred to as a “second image”) is also temporarily stored in the RAM 132 like the first image.

  As described above, the reason for shooting when the aperture is opened and when the aperture is stopped is to facilitate the determination of dust attached to the image sensor. FIG. 7 is an explanatory diagram illustrating an example of a photographed image when photographing is performed when dust is attached to the image sensor according to the first embodiment of the present invention. FIG. 7A shows an example of an image taken with the aperture stopped, and FIG. 7B shows an example of an image taken with the aperture opened.

  When shooting with the aperture open, subjects other than the subject that is in focus (focused) appear to be blurred, and when shooting with the aperture closed, subjects other than the subject that is in focus Is also clearly visible. The same applies when dust is attached to the image sensor. When shooting is performed with the aperture closed, dust attached to the image sensor is clearly reflected as shown in FIG. When shooting is performed, the dust adhering to the image sensor appears blurred as shown in FIG. In the present embodiment, the location where dust is attached is identified using these two types of images.

  Here, the first shooting and the second shooting are performed on the same object. Further, it is desirable that the subject to be photographed has a bright color such as white or gray in order to improve dust extraction accuracy.

  When the second shooting is completed, the temporary dust position extraction unit 152 performs a temporary dust extraction process for acquiring a pixel position where dust is predicted to be attached from the shot second image (step S180). . Hereinafter, the temporary dust extraction processing according to the first embodiment of the present invention will be described in detail.

  In the first embodiment of the present invention, the temporary dust is extracted by analyzing the second image taken in step S170. First, a process of extracting the temporary dust candidates by analyzing the second image is performed, and subsequently, a process of extracting temporary dust candidates at continuous positions as temporary dust from the extraction result of the temporary dust candidates. In the first embodiment of the present invention, provisional dust candidate extraction unit 151 performs provisional dust candidate extraction, and provisional dust position extraction unit 152 performs provisional dust extraction.

  FIG. 8 is an explanatory diagram for explaining the arrangement of the color filters of the CCD element 110 according to the first embodiment of the present invention. As shown in FIG. 8, the color filter of the CCD element 110 according to the first embodiment of the present invention has three primary colors of red (R), green (G), and blue (B), and the arrangement thereof. Is the Bayer method.

  FIG. 9 is a flowchart for explaining the temporary dust candidate extraction process according to the first embodiment of the present invention, and describes the temporary dust candidate extraction process (step S175) of FIG. 4 in detail. The temporary dust candidate extraction process according to the first embodiment of the present invention will be described below using FIG.

First, the second image taken with the aperture 106 is analyzed to determine the presence / absence of dust and the position where dust appears to be attached. First, the average signal level of each color filter of the CCD element 110 is calculated (step S210). The average values of the calculated RGB colors are R _av , G _av , and B _av , respectively.

  When the calculation of the average value of each color is completed, the signal level of each pixel and the average signal level within the pixel of each color corresponding to each calculated pixel are compared in all the pixels on the CCD element 110. If the difference between the signal level of each pixel and the average signal level is equal to or greater than a predetermined value, it is determined that dust is attached to the pixel, and is extracted as a temporary dust candidate.

  In the first embodiment of the present invention, the temporary dust position extraction unit 152 first initializes variables (step S220). In the present embodiment, the temporary dust position extraction unit 152 initializes two types of variables, n and count. The variable n indicates the number of pixels, and the variable count indicates the number of pixels regarded as temporary dust candidates. The range of the value of n is 0 to the total number of pixels-1. Next, it is determined whether or not the value of n is less than the total number of pixels of the CCD element 110 (step S230). When the value of n is less than the total number of pixels of the CCD element 110, it is determined whether or not the pixel is provisional dust (step S240). Specifically, for example, it is determined whether or not the signal level of the pixel corresponding to the value n is lower than the average signal level within the pixel of each color corresponding to the pixel by a predetermined value or more.

In the first embodiment of the present invention, whether the signal level of each pixel is lower than the average signal level in the pixel corresponding to the color of the pixel by a predetermined value or more according to the following equations 1 to 3. Judge whether. When the color of the color filter is R, it is determined by Equation 1, when the color of the color filter is B, it is determined by Equation 2, and when the color of the color filter is G, it is determined by Equation 3.

  In the first embodiment of the present invention, when the signal level of the pixel is lower than the average signal level within the pixel corresponding to the color of the pixel by a predetermined value or more, the pixel is extracted as a temporary dust candidate. . In the present embodiment, the temporary dust position extraction unit 152 records the value of the variable n at that time in the array variable DD [count] (step S250). Then, the value of the variable count is increased by 1 (step S260), and the value of the variable n is further increased by 1 (step S270). On the other hand, if the signal level of the pixel is not lower than the predetermined value by the average signal level within the pixel corresponding to the color of the pixel, the process directly proceeds to step S270.

  The temporary dust position extraction unit 152 performs the above-described series of processing for all the pixels, and when the value of n is not less than the number of all pixels of the CCD element 110 in step S230, that is, the value of n is all pixels. If it becomes equal to the number of, the process is terminated.

  The provisional dust candidate extraction process according to the first embodiment of the present invention has been described above with reference to FIG. Next, the temporary dust extraction process according to the first embodiment of the present invention will be described.

  FIG. 10 is a flowchart for explaining the temporary dust extraction process according to the first embodiment of the present invention, and describes the temporary dust extraction process (step S180) of FIG. 4 in detail. Hereinafter, the temporary dust extraction processing according to the first embodiment of the present invention will be described with reference to FIG.

  In the temporary dust extraction process according to the first embodiment of the present invention, the temporary dust position extraction unit 152 first initializes variables k, cc, count, group, and cg (step S310). Subsequently, it is determined whether or not the value of the variable k is less than the number of provisional dust candidates acquired (step S312). If the value of the variable k is less than the number of acquired temporary dust candidates, it is further determined whether or not the number of the variable cc is less than the number of acquired temporary dust candidates (step S314).

  If the number of variables cc is less than the number of acquired temporary dust candidates, the temporary dust position extraction unit 152 places the temporary dust candidate at a position adjacent to the pixel extracted as the temporary dust candidate in the temporary dust candidate extraction process. Determine whether exists. In the first embodiment of the present invention, whether or not there is a temporary dust candidate at an adjacent position is specifically determined by whether or not the array variables DD [k] and DD [cc] are adjacent. (Step S316). If the array variables DD [k] and DD [cc] are not adjacent to each other, the temporary dust position extraction unit 152 increments the value of the variable cc by 1 (step S318), and the temporary dust whose number of variables cc is acquired again. After determining whether the number is less than the number of candidates (step S314), processing is performed to determine whether the array variables DD [k] and DD [cc] are adjacent.

  When a temporary dust candidate exists at a position adjacent to a pixel extracted as a temporary dust candidate, the temporary dust position extraction unit 152 records the pixel as temporary dust. In the first embodiment of the present invention, specifically, when the array variables DD [k] and DD [cc] are adjacent to each other, the value of the variable k at that time is used as the array variable DUST [count]. (Step S320).

  When the temporary dust position extraction unit 152 records pixels as temporary dust, the temporary dust position extraction unit 152 further records the recorded pixels in a provisional dust set unit. Specifically, in the first embodiment of the present invention, the temporary dust position extraction unit 152 determines whether or not the value of the variable k is larger than the value of the variable cc (step S332). As a result of the determination, if the value of the variable k is larger than the value of the variable cc, the temporary dust position extraction unit 152 stores the value of CD [cc] in the array variable CD [count] (step S324), If the value is less than or equal to the value of the variable cc, the value of the variable group is stored in the array variable CD [count], and the value of the variable group is incremented by one (step S326). Then, the temporary dust position extraction unit 152 increments the value of the variable count by 1 (step S328), further increments the value of the variable k by 1, and initializes the value of the variable cc to 0 (step S330). When the value of the variable k is incremented by 1 and the value of the variable cc is initialized to 0, the temporary dust position extraction unit 152 returns to step S312 and determines whether the value of the variable k is smaller than the number of temporary dust candidates. To do.

  If the number of variables cc is equal to or greater than the number of provisional garbage candidates acquired in step S314, the process directly proceeds to step S330, where the value of variable k is incremented by 1 and the value of variable cc is set to 0. initialize.

  As described above, the temporary dust position extraction unit 152 repeats the processing from step S312 to step S330 until the value of the variable k becomes equal to or larger than the number of temporary dust candidates, thereby calculating the position and number of temporary dust. be able to.

  When the temporary dust position extraction unit 152 detects the position and the number of temporary dust, the temporary dust position extraction unit 152 calculates the number of pixels included in the portion regarded as the same temporary dust. In the first embodiment of the present invention, the temporary dust position extraction unit 152 first initializes variables cc, cg, and k (step S332). When the variable is initialized, the temporary dust position extraction unit 152 determines whether the value of the variable cg is less than group-1 (step S334). If the value of the variable cg is less than group-1 as a result of the determination in the temporary dust position extraction unit 152, the temporary dust position extraction unit 152 further determines whether the value of the variable k is less than count-1. Judgment is made (step S336). If the value of the variable k is less than count-1, the temporary dust position extraction unit 152 determines whether or not the value of the array variable CD [k] matches cg (step S340). If the value of CD [k] matches cg, the temporary dust position extraction unit 152 increases the value of the variable cc by 1 (step S342), and further increases the value of the variable k by 1 (step S342). S344). On the other hand, if the value of the array variable CD [k] does not match cg, the process directly proceeds to step S344, and the temporary dust position extraction unit 152 increases the value of the variable k by 1. When the value of the variable k is increased by 1, the process returns to step S336, and the temporary dust position extraction unit 152 determines whether or not the value of the variable k is less than count-1.

  In step S336, if the value of the variable k is equal to or greater than count-1, the temporary dust position extraction unit 152 stores the value of the variable cc in the array variable ND0 [cg]. The value of the variable cc at this time is the number of pixels included in one provisional garbage. Therefore, the number of pixels in one provisional dust is stored in the array variable ND0 []. Then, the temporary dust position extraction unit 152 stores the value of the variable cc in the array variable ND0 [cg], increments the value of the variable cg by 1, and initializes the value of the variable cc to 0 (step S338). .

  The temporary dust position extraction unit 152 repeats this series of processing until the value of the variable cg becomes equal to or larger than group−1. When the value of the variable cg becomes equal to or larger than group−1 in step S334, the processing ends. .

  The provisional dust extraction process according to the first embodiment of the present invention has been described above with reference to FIG. Note that the provisional dust extraction process in the present invention is not limited to such an example.

  When the temporary garbage is extracted, an image different from the image used for the temporary garbage extraction is analyzed using the extracted temporary garbage information, and a dust certification process for authorizing the extracted temporary garbage as garbage is performed. In the first embodiment of the present invention, the dust certification unit 154 performs dust certification by analyzing the first image taken in the first shooting.

  FIG. 11 is a flowchart for explaining the dust certification processing according to the first embodiment of the present invention, and describes the dust certification processing (step S190) in FIG. 4 in detail. . The dust certification process according to the first embodiment of the present invention will be described below using FIG.

  First, the dust certification unit 154 calculates the average signal level of each color of R, G, and B of the first image (step S410). The average signal level is calculated using the calculation method performed in the temporary dust candidate extraction process. When the average signal level of each color of R, G, and B is calculated, the dust certification unit 154 initializes each variable used in the dust certification processing (step S412). In the dust recognition process according to the first embodiment of the present invention, five types of variables k, c, g, n, and count are prepared, and the variables k, c, and g are initialized with 0, and the variable count. Is initialized with the number of provisional garbage extracted in the provisional garbage extraction process.

  When the average signal level of each of the R, G, and B colors of the first image is calculated, the dust recognition unit 154 applies to the pixels of the first image that are in the same position as the pixels that are regarded as temporary dust in the second image. Based on the mathematical expressions 1 to 3 in the temporary dust extraction process, it is determined whether or not the signal level of the pixel is lower than the average signal level of each color of R, G, and B of the first image by a predetermined value or more. Perform the process.

  In the first embodiment of the present invention, specifically, the dust certification unit 154 first determines whether or not the value of the variable g is less than the value of count-1 (step S414). If the value of the variable g is less than the value of count-1, then the dust certification unit 154 confirms whether or not the value of the variable g is equal to the value of the array variable CD [k] (step S416). If the value of the variable g is less than the value of the array variable CD [k], the dust recognition unit 154 stores the value of the array variable DUST [k] in the variable n (step S418). Then, it is confirmed whether or not the value of the variable k is less than count−1 (step S420). If the value of the variable k is less than count−1, the array variable P2 [n] is further changed to Equations 1 to 3. The dust certification unit 154 determines whether or not the conditions shown are satisfied (step S422). Here, the array variable P <b> 2 [] indicates a signal level in each pixel of the first image captured in the first imaging. If the array variable P2 [n] satisfies Expressions 1 to 3, the dust recognition unit 154 increases the value of the variable c by 1 (step S424), and further sets the value of the variable k to 1. (Step S426), the process returns to the process of checking whether the value of the variable g in step S416 is less than the value of the array variable CD [k].

  In step S416, when the value of variable g is different from the value of array variable CD [k], and in step S422, array variable P2 [n] does not satisfy the conditions shown in equations 1 to 3. In this case, the process moves to step S426, and the dust certification unit 154 increases the value of the variable k by one.

  If the value of variable k is greater than or equal to count-1 in step S420, the value of variable c is stored in array variable ND1 [k] (step S428). Here, the array variable ND1 [] indicates the number of pixels included in one provisional garbage. Therefore, the number of pixels in one provisional garbage is stored in the array variable ND1 []. Then, the value of the variable g is incremented by 1 (step S430), and the process returns to step S414 to determine whether or not the value of the variable g is less than the value of count-1.

  The dust recognition unit 154 repeats the series of processes until the value of the variable g becomes equal to or greater than the value of count−1, whereby the number of pixels included in one provisional dust can be obtained. When the value of the variable g becomes equal to or greater than the value of count-1 in step S414, the process for obtaining the number of pixels included in one temporary dust is ended.

  The number of pixels included in one provisional dust thus obtained is compared at the same provisional dust extraction location between the first image and the second image. If there is a difference in the number of pixels of a predetermined value or more between a pixel included in the predetermined temporary dust of the second image and a pixel included in the temporary dust of the same position of the first image, Is assumed that dust is attached to the image sensor, and the dust certification unit 154 recognizes the provisional dust as dust.

  When the trash recognition unit 154 certifies provisional trash as trash, it stores the position of the trash on the certified image. In the first embodiment of the present invention, an X coordinate and a Y coordinate of a central pixel are obtained from a set of pixels recognized as one dust and stored as Xc and Yc, respectively.

  The dust certification process according to the first embodiment of the present invention has been described above. The dust certification process in the present invention is not limited to this example.

  If it is recognized in the dust certification process that dust is attached to the image sensor, the presence of dust is notified to the user of the imaging apparatus 100, and the mode is shifted to the dust cleaning mode. The dust cleaning mode in which the dust removal operation is performed may be forcibly shifted when the dust certification process is completed, or the user may select whether to shift after the dust certification process is completed. Hereinafter, the dust cleaning mode according to the first embodiment of the present invention will be described.

  FIG. 12 is a flowchart for explaining the flow of the dust cleaning mode according to the first embodiment of the present invention. Hereinafter, the flow of the dust cleaning mode according to the first embodiment of the present invention will be described with reference to FIG.

  In the dust cleaning mode, first, the shutter on the front surface of the CCD element 110 is fully opened to facilitate dust cleaning (step S510). Subsequently, the position of the dust certified in the dust certification process is calculated (step S520). In the dust recognition process, the dust position on the image is calculated, so the dust position on the image is converted from the dust position on the image sensor surface, and there is dust at the location corresponding to the converted dust position on the image sensor surface. The adhesion information indicating that the image is to be displayed is displayed (step S530). In the first embodiment of the present invention, the dust display unit 156 performs the conversion to the dust position on the image sensor surface and the display of the dust adhesion information, and the dust display information is displayed on the image display unit 124. The dust adhesion information may be displayed, for example, as a symbol, or as an icon or a character.

In the first embodiment of the present invention, conversion to the coordinates (Xd, Yd) of the image display unit 124 corresponding to the dust position on the image sensor surface converted from the dust position on the image is performed as follows. . Here, Hd represents the number of horizontal dots displayed on the image display unit 124, Vd represents the number of vertical dots displayed on the image display unit 124, H represents the number of horizontal pixels of the image data, and V represents the number of vertical pixels of the image data.
Xd = Xc × Hd / H
Yd = Vd− (Yc × Vd / V)

  FIG. 13 is an example when a captured image is displayed on the image display unit 124 when an image is captured with dust attached to the CCD element 110, which is an example of the imaging element, according to the first embodiment of the present invention. FIG. 14 is an explanatory diagram illustrating an example of an image when the dust position after the position conversion is displayed on the image display unit 124 according to the first embodiment of the present invention. In the first embodiment of the present invention, a case where dust adheres to three locations of the CCD element 110 will be described.

  As shown in FIG. 13, dust 160 is attached to three places in the photographed image. On the imaging surface of the CCD element 110, the location where dust is actually attached is a vertically symmetrical position. Therefore, an icon or character is displayed as dust adhesion information so as to overlap the image displayed on the image display unit 124 so as to be a target for the dust position in the captured image.

  In this way, the dust recognition unit 154 obtains the dust position on the image sensor surface, and the dust display unit 156 indicates the dust position to the image display unit 124 in an easy-to-understand manner, whereby the user of the imaging device 100 is displayed. By using a blower or the like for the position, it is possible to easily remove dust. The imaging apparatus 100 prompts the user to confirm whether or not the cleaning has been completed (step S540). If the cleaning has been completed, the process ends. If not, the user has confirmed whether or not the cleaning has been completed. Continue to encourage.

  The flow of the dust cleaning mode according to the first embodiment of the present invention has been described above with reference to FIG.

  Note that the above imaging method may be executed by a computer program. The CPU 128 may execute the imaging method described above by calling a computer program recorded in the ROM 130.

  As described above, according to the imaging device and the imaging method according to the first embodiment of the present invention, the detection accuracy of dust attached to the imaging element is improved by comparing dust images and detecting the position of dust. To do. In addition, since the dust detection accuracy is improved, careless removal work is reduced, and the situation where the optical low-pass filter on the front surface of the image sensor is damaged is reduced. Furthermore, by displaying the dust attachment position, it is possible to facilitate the cleaning operation for the user of the imaging apparatus.

(Second Embodiment)
In the first embodiment of the present invention, the position of dust is detected by comparing images with different apertures, and the position of dust is displayed, thereby facilitating the cleaning operation for the user of the imaging apparatus. An imaging apparatus and an imaging method that can be realized have been described. In the second embodiment of the present invention, in a lens incorporating an image pickup device, as in the first embodiment, by comparing images with different apertures, it is inspected whether dust is attached to the image pickup device. An inspection apparatus and an imaging method that can be performed will be described.

  FIG. 15 is an explanatory diagram for explaining the outline of the second embodiment of the present invention. Hereinafter, a second embodiment of the present invention will be described with reference to FIG.

  As shown in FIG. 15, the second embodiment of the present invention is configured to include an imaging device 200 and a lens unit 202 with a built-in imaging device. The imaging device 200 is an example of the imaging device of the present invention. The imaging element may be a CCD element, a CMOS element, or another image sensor. A diaphragm is built in the lens unit 202 as in the first embodiment. The inspection device 200 and the lens unit 202 are connected by a connection cable 204.

  When performing dust inspection using the imaging method according to the second embodiment of the present invention, the light diffusing lens cap 104 may be attached to the lens unit 202 as in the first embodiment. By using the light diffusing lens cap 104, light incident from the subject diffuses, and dust can be detected more effectively.

  Since the configuration of the inspection apparatus 200 according to the second embodiment of the present invention has the same configuration as that of the imaging apparatus 100 according to the first embodiment of the present invention, detailed description thereof is omitted. However, the image pickup apparatus 100 according to the first embodiment of the present invention has a configuration including the CCD element 110 which is an example of the image pickup element. However, in the second embodiment of the present invention, the image pickup element. Is incorporated in the lens unit 202, it is not necessary to include an imaging device in the inspection apparatus 200.

  The outline of the second embodiment of the present invention has been described above with reference to FIG.

  As in the first embodiment, the imaging method according to the second embodiment of the present invention changes the aperture, captures the first image and the second image with different apertures, and analyzes the images to generate dust. Qualify the position. When it is recognized that dust is attached to the image sensor, the display unit 210 of the inspection apparatus 200 displays that dust is attached.

  Here, the first photographing and the second photographing are performed on the same object as in the first embodiment. Further, it is desirable that the subject to be photographed has a bright color such as white or gray in order to improve dust extraction accuracy.

  16 and 17 are explanatory diagrams illustrating an example of the inspection result obtained by the inspection apparatus according to the second embodiment of the present invention. FIG. 16 is an explanatory diagram illustrating an example of an inspection result in a case where dust is not attached to the image sensor provided in the lens unit 202 according to the second embodiment of the present invention. It is explanatory drawing which shows an example of the test result when the dust has adhered to the image pick-up element provided in the inside of the lens unit 202 concerning the 2nd Embodiment of this invention.

  As shown in FIG. 16, when no dust is attached to the image sensor provided in the lens unit 202, the display unit 210 displays that no dust is attached, and is shown in FIG. As described above, when dust adheres to the image sensor provided in the lens unit 202, the inspection apparatus 200 displays that the dust is attached to the display unit 210. The inspection apparatus 200 may display a location where dust is attached together with a display indicating that dust is attached when dust is attached to the image sensor. By displaying the location where dust is attached, it is possible to facilitate the operation of removing dust attached to the image sensor.

  As described above, according to the inspection apparatus and the imaging method according to the second embodiment of the present invention, as in the first embodiment, the lens with the built-in image sensor is used to capture images with different apertures. By comparing these, it is possible to inspect whether dust is attached to the image sensor.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  For example, in the first and second embodiments, two images having different aperture values are taken, and the two images are compared to determine whether dust has adhered to the image sensor. The present invention is not limited to such an example. For example, three or more images with different aperture values may be taken and compared to determine whether dust has adhered to the image sensor.

It is explanatory drawing explaining the imaging device concerning the 1st Embodiment of this invention. It is explanatory drawing explaining the structure of the imaging device concerning the 1st Embodiment of this invention. It is explanatory drawing explaining the structure of CPU concerning the 1st Embodiment of this invention. It is a flowchart explaining the imaging method concerning the 1st Embodiment of this invention. It is explanatory drawing which shows an example of the program diagram used for calculation of the aperture value and shutter speed at the time of opening an aperture concerning this embodiment. It is explanatory drawing which shows an example of the program diagram used for calculation of the aperture value at the time of aperture stop and shutter speed. It is explanatory drawing which shows an example of the picked-up image at the time of imaging | photography when dust adheres to the image pick-up element concerning the 1st Embodiment of this invention. It is explanatory drawing explaining the arrangement | sequence of the color filter concerning the 1st Embodiment of this invention. It is a flowchart explaining the temporary dust candidate extraction process concerning the 1st Embodiment of this invention. It is a flowchart explaining the extraction process of temporary garbage concerning the 1st Embodiment of this invention. It is a flowchart explaining the dust certification | authentication process concerning the 1st Embodiment of this invention. It is a flowchart explaining the flow of the dust cleaning mode concerning the 1st Embodiment of this invention. It is explanatory drawing which shows the example of a display of the picked-up image at the time of image | photographing in the state which attached dust concerning the 1st Embodiment of this invention. It is explanatory drawing which shows an example of the image at the time of displaying the dust position concerning the 1st Embodiment of this invention. It is explanatory drawing explaining the outline | summary of the 2nd Embodiment of this invention. It is explanatory drawing which shows an example of a test result when the dust is not adhering to the image pick-up element concerning the 2nd Embodiment of this invention. It is explanatory drawing which shows an example of the test result when the dust has adhered to the image pick-up element concerning the 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Imaging device 102,202 Lens unit 104 Lens cap 106 Aperture 110 CCD element 128 CPU
130 ROM
132 RAM
151 provisional dust candidate extraction unit 152 provisional dust position extraction unit 154 dust certification unit 156 dust display unit 158 appropriate AE calculation unit 200 inspection apparatus

Claims (10)

A lens unit including a diaphragm mechanism that adjusts the amount of incident light from the subject;
An imaging unit that sequentially outputs image of the subject imaged on an imaging surface via the lens unit sequentially as image data;
An aperture control unit that adjusts the amount of incident light from the subject by changing the aperture value of the aperture mechanism unit during continuous imaging by the imaging unit;
A foreign matter adhesion prediction unit that predicts whether foreign matter is adhered on the imaging surface based on at least two or more image data having different aperture values among the image data sequentially output from the imaging unit. When,
A pixel position detection unit that detects a pixel position on the imaging plane where the adhesion of the foreign matter is predicted based on a prediction result of the foreign matter adhesion prediction unit;
A foreign substance determination that compares the luminance levels of the image data corresponding to the pixel positions on the imaging plane and determines that the foreign substance is attached to the pixel positions when a difference of a predetermined threshold value or more occurs. And
An imaging apparatus comprising: The foreign matter determination unit calculates an average value of luminance levels of the entire image data for each of the constituent colors of the color filter included in the image sensor, and the luminance of the pixel position on the imaging plane in the image data. The imaging apparatus according to claim 1 , wherein when all the levels are lower than the average value, it is determined that the foreign matter is attached to the pixel position. The foreign matter adhesion prediction unit selects image data obtained from the incident light when the aperture value of the aperture mechanism unit is the largest among the image data sequentially output from the imaging unit. The imaging device according to claim 1 or 2 . Characterized in that it comprises a display unit for displaying the determination result by the foreign matter determination unit, an imaging apparatus according to claim 1. The imaging apparatus according to claim 4 , wherein the display unit displays an attachment position of the foreign matter corresponding to an imaging region on the imaging plane. Using a lens unit including a diaphragm mechanism that adjusts the amount of incident light from the subject, the subject image formed on the imaging surface via the lens unit is successively output as image data sequentially. Imaging step;
An aperture control step for adjusting the amount of incident light from the subject by changing the aperture value of the aperture mechanism unit during continuous imaging in the imaging step;
Foreign matter adhesion prediction step for predicting whether or not foreign matter is attached on the imaging surface based on at least two or more image data having different aperture values among the image data sequentially output in the imaging step. When,
A pixel position detecting step for detecting a pixel position on the imaging plane where the adhesion of the foreign matter is predicted based on a prediction result in the foreign matter adhesion prediction step;
A foreign substance determination that compares the luminance levels of the image data corresponding to the pixel positions on the imaging plane and determines that the foreign substance is attached to the pixel positions when a difference of a predetermined threshold value or more occurs. Steps,
The imaging method characterized by including. In the foreign matter determination step, an average value of the luminance levels of the entire image data is calculated for each constituent color of the color filter included in the image sensor, and the luminance level of the pixel position on the imaging plane in each image data The imaging method according to claim 6 , wherein when any of the values is lower than the average value, it is determined that a foreign substance is attached to the pixel position. In the foreign matter adhesion prediction step, image data obtained from the incident light when the aperture value of the aperture mechanism unit is the largest is selected from the image data sequentially output in the imaging step. The imaging method according to claim 6 or 7 . The imaging apparatus according to claim 6 , further comprising a display step that displays a determination result of the foreign matter determination step. The imaging method according to claim 9 , wherein in the display step, the adhesion position of the foreign matter is displayed corresponding to an imaging region on the imaging plane.

Images