JP6584122B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus applied to an imaging apparatus such as a digital video camera that generates a still image from a captured moving image, and an object thereof is to obtain still image data in a favorable shooting state.

  In recent years, an increase in the number of pixels of an imaging device capable of capturing a moving image has been rapidly progressing. Imaging devices that capture FullHD size moving images have already become widespread, and imaging devices capable of 4K2K moving image shooting are also gradually on the market.

  Due to such high definition of moving images, each frame image of moving images has a sufficient number of pixels to be used as a still image. Accordingly, it is considered that usage methods for generating a still image from each frame of a moving image will further expand in the future.

JP 11-136557 A

  A problem in generating a still image from a moving image is that it is difficult for the user to determine which frame image is the optimum image as a still image.

  For example, when viewed as a moving image, there are many cases where the blurring or out-of-focusing of an image that is not noticed because the image constantly changes is at an unacceptable level when viewed as a still image.

  It is very troublesome for the user to check it frame by frame.

  For example, Patent Document 1 evaluates such a problem based on information such as focus and exposure, and shooting state information such as image blur obtained from an angular velocity sensor.

  A technique is disclosed in which a still image data is generated by automatically selecting a frame image having a high evaluation value or satisfying a certain condition.

  However, in the above-described conventional example, only a frame with a high evaluation value is selected from the captured video, and the probability that a frame with a high evaluation value appears is determined by the user's shooting technique and shooting. The part depending on the situation of time was big.

  In addition, it is considered that the probability that a plurality of evaluation values such as focus, exposure, and image blur are all high is lowered. Therefore, there is a problem that a good still image may not be obtained depending on the skill of the photographer and the situation at the time of shooting.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an image processing apparatus that can easily generate an optimum image as a still image when generating a still image from a moving image. And

In order to achieve the above object, the image processing apparatus of the present invention evaluates the quality of an imaging parameter acquired from image data output from an imaging element that captures a subject image, and generates an evaluation value of the imaging parameter. Means for generating shake correction data using shake data output from the shake detection means, control means for controlling the image blur correction means using the shake correction data, and at the time of moving image shooting, the imaging parameter The image blur correction performance of the image blur correction unit when the evaluation value of the image pickup parameter is close to the target value of the imaging parameter, and the image of the image blur correction unit when the evaluation value of the imaging parameter is far from the target value of the imaging parameter A change means for making the image larger than the image stabilization performance, and a recording means for recording the frame image having the large image image stabilization performance as still image reproduction data. An that the image processing apparatus,
The imaging parameter is any one of an exposure parameter used for exposure control, a focus state parameter used for focus lens control, and a white balance parameter.
The changing means reduces the image blur correction performance when the evaluation value of the imaging parameter is close to the target value of the imaging parameter, and thereafter, a difference value between the evaluation value of the imaging parameter and the target value of the imaging parameter is obtained. When the value is equal to or less than a predetermined threshold value, the image blur correction performance before the image blur correction performance is reduced is restored .

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which a user can produce | generate an optimal image as a still image easily from a moving image can be provided.

1 is a block diagram illustrating a configuration example of an imaging apparatus according to an embodiment of the present invention. The block diagram which shows the structural example of the video signal processing part 14 Graph for explaining an example of a control method of WB control Graph for explaining an example of a control method of AF control The block diagram which shows the structural example of the image blurring correction control part 104 The graph for demonstrating the example of the calculation method which calculates the appropriateness of WB control The graph for demonstrating the example of the calculation method which calculates the appropriateness of AE control Graph for explaining an example of a calculation method for calculating the appropriateness of AF control Flowchart for explaining processing of imaging characteristic control unit 105 The graph for demonstrating the example which calculates the cutoff frequency of the image blur correction control part 104 from appropriateness The flowchart which concerns on Embodiment 2 of this invention The figure which shows the temporal change of AF appropriateness and image blur correction which concern on Embodiment 2 of this invention. Time chart according to Embodiment 3 of the present invention

Example 1
First, the overall configuration of the imaging apparatus according to the first embodiment of the present invention will be described. FIG. 1 is a block diagram schematically illustrating a configuration example of an imaging apparatus 1 according to the first embodiment of the present invention.

  In FIG. 1, a camera system control unit 10 is a block for controlling the overall operation of the imaging apparatus 1.

  The camera system control unit 10 that is an image processing apparatus has, for example, a CPU, a RAM, and a ROM, and controls the imaging apparatus 1 while using the RAM as a work area according to a program stored in advance in the ROM.

  In addition, each process mentioned later shall be mainly performed by the camera system control part 10 as a computer program (software).

  A camera system control unit 10 that is an image processing apparatus includes a WB control unit 101 that performs white balance adjustment processing and an AE control unit 102 that performs automatic exposure control processing.

  Further, the camera system control unit 10 that is an image processing apparatus includes an AF control unit 103 that performs an automatic focus adjustment process, and an image blur correction control unit 104 that controls image blur correction.

  By controlling the imaging lens 11, the imaging device 12, and the video signal processing unit 14, the imaging state such as exposure, white balance, focus, and image blur of the captured image is controlled to be in an appropriate state.

  Furthermore, the imaging characteristics relating to the control performed by the WB control unit 101, the AE control unit 102, the AF control unit 103, and the image blur correction control unit 104 are monitored, and the respective control characteristics are changed according to the respective imaging states. A control unit 105 is included.

  The imaging characteristic control unit 105 can generate a good still image with a higher probability when generating a still image from a moving image by controlling each shooting state to be as suitable as possible. It is a block to make.

  Details of operations performed in each block of the camera system control unit 10 will be described later.

  As the imaging lens 11, a conventional general imaging lens having functions such as zoom, focus, aperture, and image blur correction can be applied. The diaphragm 111 has a plurality of diaphragm blades, and is one of exposure control means for changing the exposure by forming a light passage opening.

  These diaphragm blades operate in response to the driving force of the diaphragm driving unit 114 and adjust the amount of light incident on the image sensor 12 by changing the aperture area (diaphragm aperture).

  The shift lens 112 is a lens that can move in a direction perpendicular to the optical axis, and can move the position of a subject image to be formed.

  The shift lens 112 moves in response to the driving force from the shift lens driving unit 115 and moves the shift lens 112 so as to cancel the shake applied to the imaging apparatus, thereby correcting the image blur of the captured image.

  The focus lens 113 receives the driving force from the focus lens driving unit 116, moves in the optical axis direction, and stops at a predetermined focusing position, thereby adjusting the focus of the subject image formed on the image sensor 12. .

  The image sensor 12 is configured by, for example, an XY address type CMOS (Complementary Metal Oxide Semiconductor) image sensor.

  The optical image formed by the imaging lens 11 is photoelectrically converted to accumulate charges, and the charges are read to supply an image signal composed of a plurality of pixels to the video signal processing unit 14.

  Further, the image sensor 12 is driven by the image sensor drive unit 13 and operates to capture a subject image at a period of 60 Hz in the case of a video signal conforming to the NTSC format, for example.

  Further, the electric charge accumulated by the control signal from the image sensor driving unit 13 can be swept out, thereby having an electronic shutter function for controlling the exposure time (accumulation time).

  Furthermore, when reading the accumulated electric charge, it has a gain means for electrically amplifying the signal, and the amplification factor can be varied from the camera system control unit 10.

  This corresponds to a change in sensitivity in imaging. The electronic shutter and gain means are one of exposure control means.

  The video signal processing unit 14 performs signal processing such as white balance adjustment and gamma correction on the image signal output from the image sensor 12, converts the image signal into a video signal, and supplies the video signal to the recording control unit 16 and the display control unit 15.

  Further, the video signal processing unit 14 generates and outputs an evaluation value for use in processing of WB control, AE control, and AF control from an image.

  The recording control unit 16 converts the video signal generated in the video signal processing unit 14 into data of a desired format, and records the data in the recording medium 18 as moving image data or still image data.

  The recording medium 18 is an information recording medium such as a semiconductor memory or a magnetic recording medium such as a hard disk.

  The recording control unit 16 performs control such as image recording start and recording stop according to an instruction from an operation unit (not shown).

  The recording control unit 16 converts the video signal output from the video signal processing unit 14 into moving image data of a predetermined format and records it on the recording medium 18 when an instruction to start recording of the moving image is given from the operation unit. .

  Then, when an instruction to stop recording is given, the generation of moving image data is terminated.

  In addition, when there is an instruction for still image shooting from the operation unit or an instruction to generate still image data from the imaging characteristic control unit 105, a desired frame of the video signal output from the video signal processing unit 14 is used. Still image data is generated from the image and recorded on the recording medium 18.

  As another method for generating still image data, metadata describing a frame used for generating still image data among a plurality of frame images constituting moving image data is recorded on a recording medium in association with the moving image data. .

  The still image data may be generated from the moving image data at a timing different from the moving image shooting.

  The display control unit 15 outputs a video signal processed according to the use such as a setting menu image and a recorded image in addition to an image (through image) based on the video signal output from the video signal processing unit 14 to display the display device 17 displays an image.

  The display device 17 is a liquid crystal display element (LCD) or the like, and displays an image generated by the display control unit 15.

  The angular velocity sensor 19 is a sensor for detecting a shake applied to the imaging apparatus 1, and is used for image blur correction control performed by the image blur correction control unit 104.

  FIG. 2 is a block diagram schematically showing a more detailed configuration of the video signal processing unit 14 in FIG.

  In FIG. 2, the image signal supplied from the image sensor 12 is divided into a luminance signal (Y) and a color signal (R, G, B) by the luminance / color signal generation unit 141.

  The WB adjustment unit 142 adjusts the gain of each color of the color signal (R, G, B) according to the control from the WB control unit 101, and the color signal (R ′, G ′, G ′) whose gain is adjusted. ) Is generated.

  In the gamma correction unit 143, the luminance signal (Y) and the color signals (R ′, G ′, B ′) are corrected according to a predetermined gamma curve.

  Further, the color difference signal generation unit 144 generates a color difference signal (R−Y, B−Y) from the gamma-corrected luminance signal and color signal and stores them in the memory 145.

  The memory 145 is an area for temporarily holding the video signal for delivery to the recording control unit 16 and the display control unit 15.

  The video signal temporarily held in the memory 145 is read from the recording control unit 16 and the display control unit 15 and converted into a signal having a format suitable for each use.

  The color evaluation value generation unit 147 calculates a color evaluation value used in the AWB control.

  The color evaluation value generation unit divides an image signal for one screen (field or frame) into a predetermined number (for example, 8 × 8) blocks, and color difference signals (RY, BY) of pixels included in each block. ) And an average color difference signal is output.

  The luminance evaluation value generation unit 148 calculates a luminance evaluation value used in AE control.

  The luminance evaluation value generation unit divides an image signal for one screen (field or frame) into a predetermined number (for example, 8 × 8) blocks, and calculates an average value of luminance signals (Y) of pixels included in each block. And output.

  The focus evaluation value generation unit 146 calculates a focus evaluation value used in AF control. The focus evaluation value generation unit calculates a signal corresponding to the contour component amount on the high frequency side among the spatial frequency components of the image.

  Specifically, a predetermined distance measurement area is set in the image signal for one screen, and a high frequency component of the spatial frequency is extracted by performing arithmetic processing such as HPF on the luminance signal (Y) in the area.

  Further, a focus evaluation value is calculated and output by performing arithmetic processing such as cumulative addition on the extracted high-frequency component.

  The focus evaluation value represents the sharpness (contrast) of the video signal, but the sharpness changes depending on the focus state of the imaging lens, and as a result, the focus evaluation value is a signal representing the focus state of the imaging lens. It becomes.

<WB control>
Next, auto white balance control (AWB control) executed by the WB control unit 101 will be described. The WB control unit 101 performs white balance control based on the color difference signals (R−Y, B−Y) generated by the color evaluation value generation unit 147.

  The WB control unit 101 extracts a signal that will be a white subject (hereinafter referred to as “a signal close to white”) from the image, and the ratio of the color signals R, G, and B of the signal is approximately 1: 1: 1. The gains of R, G, and B in the WB adjustment unit 142 are adjusted so that

  As an example of a method for extracting a signal close to white performed by the WB control unit 101, an average color evaluation value generated by the color evaluation value generation unit 147 is acquired.

  The average color evaluation value of all blocks of the entire screen is further averaged to calculate the average color evaluation value of the entire screen, and the signal is extracted as a signal close to white.

  Alternatively, as another method of extracting a signal close to white, an area that may be a white subject is extracted from the image, and the color evaluation value of the area may be a signal close to white.

  Specifically, as shown in FIG. 3, a white extraction region that is a region close to white is set on the color difference plane with RY on the vertical axis and BY on the horizontal axis.

  Then, out of the average color evaluation values of the blocks generated by the color evaluation value generation unit 147, those that fall within the white extraction region are extracted and averaged to calculate a color signal close to white.

  The white balance gain is calculated so that the signal close to white extracted in this way approaches a preset target white, and the WB adjustment unit 142 is controlled based on the calculated white balance gain.

<AE control>
Next, the automatic exposure control process executed by the AE control unit 102 will be described.

  The AE control unit 102 measures the brightness of the subject and controls the aperture, shutter (electronic shutter), and gain so that shooting is performed under suitable exposure conditions.

  In the imaging apparatus 1 according to the present invention, photometry is performed based on an image signal obtained from the imaging element 12. As an example of a photometric value calculation method, first, a luminance evaluation value output from the luminance evaluation value generation unit 148 is acquired, and a luminance value for each area is extracted.

  Then, according to the menu setting information arbitrarily set by the user or the shooting scene information determined from the subject, it is determined which area of the screen the photometric value is regarded as important, and the extracted luminance value is determined for each area. Perform weighting.

  Then, the luminance value of the entire image is calculated by performing processing such as taking an average value of all areas subjected to weighting.

  Note that the luminance value obtained here is a luminance value that represents the result of imaging the light beam that has passed through the aperture 111 of the imaging lens 11 by the imaging element 12, and therefore does not directly represent the brightness (Bv value) of the subject. Absent.

  As a photometric value used for exposure control, a Bv value is calculated from the luminance value obtained from the image, the current exposure value (Ev value), and the sensitivity of the image sensor 12.

  Then, based on the calculated photometric value, an Ev value is calculated so that the brightness of the image becomes the target brightness. The AE control unit calculates the aperture value (Av value), shutter speed (Tv value), and gain (Sv value) by comparing the Ev value with the program diagram.

  Then, the diaphragm 111 and the image sensor 12 (electronic shutter, gain) are controlled based on the calculated Av value, Tv value, and Sv value.

  The above-mentioned photometric value is calculated using the image signal obtained by the image sensor 12, but the brightness of the subject is detected by a photoelectric conversion means (photometric means) different from the image sensor, A photometric value may be obtained from the detection signal.

<AF control>
Next, AF control processing executed by the AF control unit 103 will be described. The AF control unit 103 performs AF control based on the focus evaluation value generated by the focus evaluation value generation unit 146 of FIG.

  The focus evaluation value generation unit 146 extracts a high-frequency component of the spatial frequency in a predetermined distance measurement area of the image signal obtained from the image sensor 12, and integrates the extracted high-frequency component, thereby integrating the extracted high-frequency component in the distance measurement area. A focus evaluation value indicating contrast is calculated.

  The calculated focus evaluation is supplied to the AF control unit 103 and used for the focus operation.

  The AF control unit 103 performs control so as to sample the focus evaluation value while moving the focus lens in the direction in which the focus evaluation is increased, and detect the position of the focus lens (that is, the focus) where the focus evaluation value is highest. To do.

  In FIG. 4, first, at the start of AF control, the focus evaluation value is sampled while moving the focus lens in a certain direction.

  If the focus evaluation value increases monotonously with the movement of the focus lens, it is determined that there is a maximum focus evaluation value (that is, a focus position) in the same direction, and the focus lens is continuously moved.

  On the other hand, when the focus evaluation value decreases with the movement of the focus lens, it is determined that there is a maximum focus evaluation value in the reverse direction, and the focus lens is moved in the reverse direction.

  For example, if the focus evaluation value is sampled while moving to the close side as shown in FIG. 4, the focus evaluation value monotonously increases with the movement of the focus lens, so it is determined that the maximum is on the close side and continues to the close side. Move the focus lens.

  In the case of FIG. 4, if the movement of the focus lens is continued as it is, the focus evaluation value starts to decrease after passing the peak, so that the maximum (focus position) of the focus evaluation value can be confirmed there.

  When the maximum focus evaluation value is confirmed, the focus lens is returned to the vicinity of the maximum. In this way, the focus lens can be moved to the in-focus position where the focus evaluation value is maximized. This operation is referred to as a hill-climbing drive mode.

  Further, in the vicinity of the in-focus point, the focus evaluation value is sampled while the focus lens is moved slightly in the front-rear direction, and it is confirmed that the current position of the focus lens is at the peak of the focus evaluation value. This operation is referred to as a micro drive mode.

  If it is determined that the focus evaluation value is not at the peak position, the focus lens is moved in the direction in which the evaluation value increases, and the focus lens is controlled to be maintained in the vicinity of the focal point.

<Image stabilization control>
Next, image blur correction control executed by the image blur correction control unit 104 will be described with reference to FIG.

  FIG. 5 is a block diagram for explaining the image blur correction control unit 104 and the shift lens driving unit 115 in FIG. 1 in more detail.

  The image blur correction control unit 104 detects a shake applied to the imaging device 1 based on the angular velocity information output from the angular velocity sensor 19.

  Then, the image blur generated in the captured image is corrected by driving the shift lens 112 based on the shake correction data so as to cancel it.

  The angular velocity sensor 19 has two angular velocities in two axial directions, ie, a horizontal rotation axis (Yaw) and a vertical rotation axis (Pitch) so as to form detection axes orthogonal to each other on a plane orthogonal to the optical axis. The sensor is arranged.

  The signal processing on each axis and the drive control of the shift lens 112 can be realized by the same processing on both axes, and therefore only one of the axes will be described below.

  The angular velocity sensor 19 detects an angular velocity of shake applied to the imaging device 1 and outputs a voltage corresponding to the angular velocity.

  The A / D converter 20 converts the voltage output from the angular velocity sensor 19 into digital data and captures it as angular velocity data.

  The HPF 121 removes the low frequency component of the angular velocity data and supplies it to the sensitivity correction unit 122.

  The sensitivity correction unit 122 drives the shift lens 112 with a displacement amount appropriate (preferably optimal) for image blur correction.

  Therefore, the angular velocity data is converted by multiplying the angular velocity data by the eccentric sensitivity of the shift lens 112 (that is, the coefficient obtained by the ratio of the displacement amount of the imaging position on the image plane to the displacement amount of the shift lens).

  Next, since the unit of the output of the sensitivity correction unit 122 is an angular velocity, it is converted into an angular displacement by first-order integration by the integrator 123.

  The integration calculation performed here is incomplete integration in order to prevent saturation, and is calculated by a generally known first-order LPF.

  The angular displacement data calculated by the integrator 123 is output by the saturation prevention control unit 124 with restriction so that the shift lens 112 does not hit the end of the mechanical movable range.

  Specifically, a limiter for the control range is provided inside the mechanical movable range of the shift lens 112 to limit the angular displacement data so as not to exceed the limiter.

  The centering control unit 125 determines whether an intended operation such as panning or tilting has been performed, and performs control to return the position of the shift lens to the center.

  Specifically, a predetermined threshold is provided further inside the limiter provided in the saturation prevention control unit 124, and it is determined that panning has been performed when the angular displacement data exceeds the threshold.

  The determination result is notified to the centering control unit 125, and when the panning is determined, the cut-off frequency of the HPF 121 is increased to limit the angular velocity data input to the integrator 123.

  In addition, a predetermined offset is subtracted from the angular velocity data input to the integrator 123, or the LPF calculation cut-off frequency performed by the integrator 123 is increased so that the output of the integrator 123 returns to the center. To do.

  In this way, even when panning, tilting, or the like is intended, control is performed so that the shift lens 112 is within the movable range, and the final drive target position, which is the displacement amount of the shift lens 112, is set. calculate.

  The shift lens driving unit 115 includes a position detection unit 119, an amplifier 120, a motor driving unit 117, and a motor 118.

  The position detection unit 119 detects the position of the shift lens 112 and outputs a voltage corresponding to the position, and the amplifier 120 amplifies the signal to a signal in an appropriate voltage range.

  Then, it is converted into digital data by the A / D converter 20 and taken in as position data.

  The control filter 126 receives deviation data that is a difference between the drive target position and the position data, performs various signal processing such as amplification and phase compensation, and outputs the result to the pulse width modulation unit 21.

  The pulse width modulation unit 21 modulates the output of the control filter 126 into a waveform (that is, a PWM waveform) that changes the duty ratio of the pulse wave, and supplies the modulated waveform to the motor driving unit 117.

  The motor 118 is, for example, a voice coil type motor, and the shift lens 112 moves in a direction perpendicular to the optical axis by being driven by the motor driving unit 117.

  A feedback loop is formed in which the position of the shifted shift lens 112 is detected by the position detection unit 119 and the next deviation data is calculated. The difference between the drive target position and the position data is controlled to be small. The

  In this way, the image blur can be corrected by driving the shift lens 112 in accordance with the shake data detected by the angular velocity sensor 19.

<Method for generating WB suitability>
Next, the details of the method for calculating the WB suitability will be described with reference to FIG.

  In the graph of FIG. 6A, the color difference between RY and BY is used as coordinates.

  When the coordinates of the color evaluation values output from the color evaluation value generation unit 147 described above are located near the origin O in FIG. 6A, the RGB balance is achieved, that is, the white balance is appropriate. It shows that.

  Conversely, the farther from the origin O, the more white balance is shifted. A vector of the color evaluation values on the coordinates in FIG. 6A is defined as WB_Vector.

  The horizontal axis of the graphs of FIGS. 6B and 6B is an example of a graph in which the size of WB_Vector is set and the vertical axis is the appropriateness Mdata_wb to be generated.

As described above, the horizontal axis is an image in which the white balance is shifted as the value increases. In FIG. 6B, Mdata_wb is calculated by the following calculation formula.
Mdata_wb = | WB_Vector | / WB_TH

  That is, the amount of deviation from the optimum white balance value is normalized by the predetermined threshold WB_TH. Here, WB_TH is set as an allowable value of the deviation amount of white balance.

  Since the allowable value of the color misregistration amount varies greatly among individuals and is difficult to determine uniquely, the user may arbitrarily set it.

  Also, depending on the light source, it may not be able to converge to the origin. In this case, the threshold value of WB_TH may be widened, or the origin of FIG. 6A may be shifted according to the light source.

  By this calculation, Mdata_wb becomes data indicating that the white balance is more appropriate as it is closer to 0, and that the white balance is shifted as it is larger than 1.

<Method for generating AE suitability>
Next, details of the method for generating the AE suitability will be described with reference to FIG. The horizontal axis of the graphs in FIGS. 7A and 7B is expressed in a unit system using an APEX (Additive System of Photographic Exposure) system. The definition of each symbol is as follows.
Ev_now ... current exposure value Ev_target ... appropriate exposure value determined by the AE control unit 102

  FIG. 7B shows an example of a graph in which the horizontal axis is Ev_target−Ev_now, that is, the difference between the appropriate exposure value and the current exposure value, and the vertical axis is the appropriateness degree Mdata_exposure to be generated.

  The abscissa indicates overexposure when it increases in the plus direction, underexposure when it decreases in the minus direction, and indicates a state where the exposure is accurately matched when 0.

At this time, Mdata_exposure = 0. In FIG. 7A, Mdata_exposure is calculated by the following calculation formula.
Mdata_exposure = | Ev_target−Ev_now | / (1/3)

  That is, the amount of deviation of the current exposure from the appropriate exposure is normalized by a predetermined Ev value (here, 1/3 Ev). Here, normalization by 1/3 Ev is merely an example.

  This numerical value may be arbitrarily set by the user, or may be variable depending on the luminance distribution of the subject.

  For example, when a deviation of 1/3 Ev occurs, overexposure or blackout occurs, a method such as normalization by 1/5 Ev may be employed.

  By this calculation, Mdata_exposure is data indicating that the exposure is more appropriate as it is closer to 0, and that the exposure is underexposed or overexposed as it is greater than 1.

FIG. 7B is a graph showing still another example of the method of calculating Mdata_exposure, in which the horizontal axis and the vertical axis are the same as those in FIG. 7A. In FIG. 7B, Mdata_exposure is calculated by the following calculation formula.
Mdata_exposure = (2 ^ (| Ev_target-Ev_now |) -1)
/ (2 ^ (1/3) -1)

  The Ev value is a unit system in which the amount of light incident on the image sensor is expressed as a logarithm with a base of 2. That is, when the Ev value changes by 1, the light amount is doubled or halved.

  In the above equation, normalization is performed after the unit system of the APEX system is converted into the unit system of the actual light quantity, and the amount of exposure deviation can be expressed more accurately by Mdata_exposure.

<Method for generating AF suitability>
Details of the AF appropriateness generation method will be described with reference to FIG. For the sake of explanation, symbols are defined as follows.
Dl: Focus evaluation value (peak value) held during focusing
Dt: Current focus evaluation value Df: Depth of field (infinity)
Dn: Depth of field (closest side)

  FIG. 8A shows an example of a graph in which the horizontal axis is D1−Dt, the difference between the focus evaluation value held at the time of focusing and the current focus evaluation value, and the vertical axis is the appropriateness Mdata_focus generated. Yes.

When D1−Dt is 0, it indicates a state where the focus is completely achieved. At this time, Mdata_focus = 0. In FIG. 8A, Mdata_focus is calculated by the following calculation formula.
Mdata_focus = | (Dl−Dt) / (Df−Dt) |
However, D1-Dt ≧ 0
Mdata_focus = | (Dl−Dt) / (Dn−Dt) |
However, D1-Dt <0

  That is, the amount of deviation from the target value of the shooting subject distance is normalized by the depth of field. As a result, Mdata_focus is data indicating that the closer to 0, the more focused, and the greater the value, the more out-of-focus.

<Method for generating image blur correction appropriateness>
The sensitivity correction unit 127, the integrator 128, and the tracking degree calculation unit 129 in FIG. 5 are blocks for calculating the tracking degree of the image blur correction control (that is, the appropriateness level for the image blur correction control).

  Similar to the sensitivity correction unit 122, the sensitivity correction unit 127 multiplies the angular velocity data by the eccentricity sensitivity of the shift lens 112, and an appropriate (preferably optimum) amplitude for performing image blur correction with the shift lens. Convert to

  The integrator 128 converts the angular velocity into angular displacement by first-order integration. However, unlike the integrator 123, the calculation is based on the complete integration, not the incomplete integration realized by the LPF.

  Since the output of the integrator 128 is calculated without filtering the angular velocity data by HPF, LPF, or the like, the shake applied to the imaging device can be obtained with higher accuracy when compared with the output of the integrator 123. .

The follow-up degree calculation unit 129 calculates the image blur correction appropriateness Mdata_shake by the following calculation formula.
Shake_det ... Output of integrator 128 (detected angular displacement)
Shake_cor: Output of A / D converter 22 (corrected angular displacement)
f ... Current focal length Mdata_shake = | Shake_det-Shake_cor | / f

  In other words, by calculating the difference between the angular displacement amount calculated from the shake information detected by the angular velocity sensor and the angular displacement amount actually corrected by the shift lens 112, the image blur amount which is the shake correction data remaining in the image data is obtained. Further, it is normalized by the focal length f.

<Operation of imaging characteristic control unit>
Next, the operation of the imaging characteristic control unit 105 in FIGS. 1 and 5 will be described. The imaging characteristic control unit 105 is the most characteristic part in the present invention.

  The purpose is to increase the probability that the shooting state for each of the imaging parameters is in a suitable state, and to easily obtain a high-quality still image.

  Here, as an example of processing performed by the imaging characteristic control unit 105, the appropriateness of AE, AF, and WB is referred to, and control is performed so as to change the control characteristic of image blur correction according to these appropriatenesses.

  Here, a method for changing the control characteristics of the image blur correction control unit 104 will be described.

  The image blur correction control unit 104 can change the characteristic by manipulating the cutoff frequency of the HPF 121 or the cutoff frequency of the LPF calculated by the integrator 123.

  When the cutoff frequency of the HPF or LPF is increased, the low frequency component is removed from the detected blur information, and the effect of image blur correction is reduced.

  In addition, since the phase near the cutoff frequency changes in the traveling direction, the followability of the image blur correction with respect to the detected shake decreases (the effect of the image blur correction decreases).

  On the other hand, if the cut-off frequency is lowered, low-frequency blur can also be corrected, and phase advancement becomes smaller even when focusing on the phase characteristics. Therefore, the followability of image blur correction to detected blur is high (image blur correction). Is highly effective).

  The imaging characteristic control unit 105 can control the image blur correction characteristics by changing the cutoff frequency by controlling the centering control unit 125.

  By the way, by making the followability of image blur correction high (the effect of image blur correction is strong), it is possible to obtain a high quality image with little image blur, but always keep the followability high when shooting movies. I can't.

  Because there is a mechanical (or optical) movable end in the movable range of the shift lens, when image blur correction is performed to follow the detected blur, the position of the shift lens gradually approaches the movable end, Eventually it will hit the movable end.

  When it comes into contact with the movable end, no more blur correction can be performed, and it appears as image blur in the captured image.

  Therefore, the position of the shift lens 112 as image blur correction means is controlled so as to be as central as possible with respect to the movable range. Therefore, a large margin is secured in the image blur correction range, and a large blur may occur in the future. It is possible to make corrections possible.

  The imaging characteristic control unit 105, which is an evaluation unit, evaluates the quality of imaging parameters acquired from image data output from an imaging element that captures a subject image, and generates an evaluation value of the imaging parameters.

  The imaging parameter is one of an exposure parameter used for exposure control, a focus state parameter used for focus lens control, and a white balance parameter.

  The image blur correction control unit 104, which is a changing unit, performs image blur correction performance when the imaging parameter evaluation value is close to the target value during moving image shooting, from image blur correction performance when the imaging parameter evaluation value is far from the target value. It is also bigger.

  That is, in the imaging apparatus according to the present invention, the appropriateness of AE control, AF control, and WB control is referred to.

  When each appropriate value is low, the followability of image blur correction is lowered, and the shift lens 112 is positioned near the center to secure a margin so that correction can be made even for large blurs. To do.

  Then, when the appropriateness of AE control, AF control, and WB control is high, control is performed so that image blur correction can be followed by increasing image blur correction as much as possible. By controlling in this way, a high-quality still image can be obtained while shooting a moving image.

  Hereinafter, processing executed by the imaging characteristic control unit 105 will be described in detail with reference to a flowchart of FIG.

  Note that the processing shown in FIG. 9 is repeatedly executed at a predetermined cycle, for example, 60 Hz, which is an image capturing cycle of one frame of the imaging apparatus. In step S101, the AF appropriateness generated by the AF control unit 103 is acquired, and it is determined whether the AF appropriateness is equal to or greater than a predetermined threshold.

  If it is determined that the threshold value is equal to or greater than the predetermined threshold value, the process proceeds to step S105. If it is determined that the threshold value is less than the predetermined threshold value, the process proceeds to step S102. In step S102, the AF appropriateness generated by the AE control unit 102 is acquired, and it is determined whether the AE appropriateness is equal to or greater than a predetermined threshold.

  If it is determined that the threshold value is equal to or greater than the predetermined threshold value, the process proceeds to step S105. If it is determined that the threshold value is less than the predetermined threshold value, the process proceeds to step S103. In step S103, the WB appropriateness level generated by the WB control unit 101 is acquired, and it is determined whether the WB appropriateness level is equal to or greater than a predetermined threshold.

  If it is determined that the threshold value is equal to or greater than the predetermined threshold value, the process proceeds to step S105. If it is determined that the threshold value is less than the predetermined threshold value, the process proceeds to step S104.

  Step S105 is a process performed when any of the determinations in steps S101 to S103 is determined to be greater than or equal to a predetermined value, and changes the characteristics of image blur correction performed by the image blur correction control unit 104 to reduce the correction effect. To be.

  In step S104, the characteristics of the image blur correction performed by the image blur correction control unit 104 are changed so as to increase the correction effect, and the process proceeds to step S106.

  As a specific method for the processing performed in steps S104 and S105, the cutoff frequency of the HPF 121 is lowered, and the cutoff frequency of the LPF used for the calculation of the integrator 123 is set low.

  FIG. 10 is a diagram illustrating a method for calculating the cut-off frequency of HPF and LPF according to each appropriateness.

  In FIG. 10, the horizontal axis indicates the appropriateness, and the vertical axis indicates the cutoff frequency of HPF and LPF.

  In step S105, the cutoff frequency is calculated for each appropriateness with reference to the appropriateness of AE control, AF control, and WB control.

  Then, the highest cut-off frequency among the calculated cut-off frequencies is set as the LPF used for the calculation of the HPF 121 and the integrator 123.

  In step S106, the image blur correction follow-up degree generated by the image blur correction control unit 104 is acquired, and it is determined whether the image blur correction follow-up degree is equal to or greater than a predetermined threshold.

  If it is determined that the threshold is equal to or greater than the predetermined threshold value, the process proceeds to step S107. If it is determined that it is less than the predetermined threshold value, the process is terminated.

  Step S107 is processing when it is determined that all the imaging parameters are close to appropriate values, and notifies the recording control unit 16 in FIG. 1 of an instruction for generating still image data from the current frame of the moving image data.

  A recording control unit 16 as a recording unit records a frame image having a high image blur correction performance as still image reproduction data.

  The recording control unit 16 generates still image data from a desired frame image among the video signals output from the video signal processing unit 14 when there is an instruction to generate still image data from the imaging characteristic control unit 105. To the recording medium 18.

  As another method for generating still image data, metadata describing a frame used for generating still image data among a plurality of frame images constituting moving image data is recorded on a recording medium in association with the moving image data. .

  The still image data may be generated from the moving image data at a timing different from the moving image shooting.

  As described above, in the present invention, each imaging parameter is controlled based on the four types of appropriatenesses of AF appropriateness, AE appropriateness, WB appropriateness deviation, and image blur correction appropriateness.

  Therefore, it is possible to control each photographing state so that it is as suitable as possible.

  When a still image is generated from a moving image, a good still image can be generated with a higher probability.

  In addition, when a still image is generated from a captured moving image, it is possible to provide an imaging apparatus that can easily generate an optimal image as a still image.

  Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to these specific embodiments.

  Various forms within the scope of the present invention are also included in the present invention. For example, it is not necessary to generate all the four types of appropriatenesses, and a system using at least two types of appropriatenesses may be used.

(Example 2)
Next, Embodiment 2 of the present invention will be described. Since the present embodiment can be realized with the same configuration as the imaging device 1 described with reference to FIG. 1, the description thereof is omitted.

  In the present embodiment, another method of control performed by the imaging characteristic changing unit 105 will be described.

  Here, as an example of processing performed by the imaging characteristic control unit 105, a method for referring to the appropriateness of AF and changing the control characteristic of image blur correction according to the temporal change will be described.

  In the imaging characteristic changing unit 105 of the present embodiment, when the focus is changed from the out-of-focus state, in other words, during the focusing operation by AF, the image blur correction is performed. Set the correction effect low. Then, the correction effect of the image blur correction is increased in a state where focus is achieved.

  The purpose is to increase the probability that the shooting state for each imaging parameter is in a suitable state as in the first embodiment, and to easily obtain a high-quality still image. It makes it easier to achieve the objective by monitoring changes.

  Specifically, when the AF operation is finished, the shift lens 112 is controlled so as to be positioned near the center of the control range, thereby maximizing the effect of image blur correction in a focused state. It becomes possible.

  In order to clarify the gist of the invention, a method of performing control with reference to only the appropriateness of AF will be described, but the same method is applied even when a plurality of appropriateness of AE and AWB are referred to. be able to.

  Hereinafter, processing executed by the imaging characteristic changing unit 105 will be described in detail with reference to a flowchart of FIG.

  Note that the processing shown in FIG. 11 is repeatedly executed at a predetermined cycle, for example, 60 Hz, which is an image capturing cycle of one frame of the imaging apparatus.

  In step S201, Af_Flag which is a variable of the internal memory is referred to and it is determined whether or not it is zero. If it is 0, the process proceeds to step S202, and if it is not 0, the process proceeds to S205. Af_Flag is a flag for indicating a processing state of the imaging characteristic changing unit 105, and is a flag that is set to 1 when it is determined that the focus is once out of focus.

  In step S202, the characteristics of the image blur correction performed by the image blur correction control unit 104 are changed, and the cutoff frequency is set low so that the correction effect is enhanced. The value at this time is set to Fc_base, which is a normal cutoff frequency.

  In step S203, the AF appropriateness generated by the AF control unit 103 is acquired, and it is determined whether the AF appropriateness is greater than a predetermined threshold. If it is determined that the AF appropriateness is greater than the predetermined threshold (out of focus), the process proceeds to step S204, and Af_flag is set to 1. If it is determined that the value is equal to or less than the predetermined threshold, the process is terminated.

  Step S205 is a process performed when it is determined in step S201 that Af_flag is not 0, that is, a process performed once in an out-of-focus state.

  In step S205, the AF appropriateness generated by the AF control unit 103 is acquired, and it is determined whether the AF appropriateness is monotonously decreasing.

  In this determination, the AF appropriateness acquired in the past is stored in the memory, and it is determined whether or not the AF appropriateness is continuously decreased for a predetermined number of times. If it is determined in step S205 that there is a monotonic decrease, the process proceeds to step S206. If it is determined that it is not a monotonic decrease, the process proceeds to step S207.

  In step S206, the characteristics of the image blur correction performed by the image blur correction control unit 104 are changed, the cutoff frequency is set high so that the correction effect is lowered, and the process is terminated.

  In step S207, the image blur correction characteristic is set to Fc_base, which is a normal cutoff frequency.

  In step S208, it is determined whether or not the AF appropriateness is equal to or less than a predetermined threshold. If it is determined that the AF appropriateness is equal to or less than the predetermined threshold, the process proceeds to step S209.

  In step S209, Af_flag is cleared to 0, and the process proceeds to step S210. In step S210, an instruction for generating still image data from the current frame of the moving image data is notified to the recording control unit 16 in FIG.

  The recording control unit 16 indicates a case where there is an instruction to generate still image data from the imaging characteristic control unit 105.

  In that case, among the video signals output from the video signal processing unit 14, still image data is generated from a desired frame image and recorded on the recording medium 18.

  As another method for generating still image data, metadata describing a frame used for generating still image data among a plurality of frame images constituting moving image data is recorded on a recording medium in association with the moving image data. .

  The still image data may be generated from the moving image data at a timing different from the moving image shooting.

  Next, FIG. 12 is a diagram for explaining temporal changes in AF appropriateness and image blur correction by control performed by the imaging characteristic changing unit 105. FIG. 12A is a graph showing a temporal change in AF appropriateness when the horizontal axis is time and the vertical axis is AF appropriateness.

  FIG. 12 is a graph when the AF operation is started from a state where the focus is not achieved (that is, the AF appropriateness is large) and the state is changed to a state where the focus is focused (that is, the AF appropriateness is small). It is. Here, the time T1 indicates the time when the AF operation is started from the out-of-focus state and the driving of the focus lens is started, and the time T2 indicates the time when the subject is focused and the AF operation is stopped. Show.

  Accordingly, as shown in FIG. 12A, the AF appropriateness changes from time T1 to time T2 so as to decrease.

  FIG. 12B shows the amount of angular displacement of the shake applied to the imaging apparatus 1 and is information obtained from the output of the angular velocity sensor 19 in this embodiment.

  FIGS. 12C and 12D are graphs for explaining conventional image blur correction control. FIGS. 11E and 11F are image blur correction control in the image pickup apparatus of the present embodiment. It is a graph for demonstrating.

  12C and 12E show changes in the cutoff frequency of the LPF used for the calculation of the integrator 123, and the vertical axis represents the cutoff frequency. The cutoff frequency is related to the correction effect of the image blur correction. The lower the cutoff frequency, the higher the blur correction effect. The higher the cutoff frequency, the lower the image blur correction effect.

  FIGS. 12D and 12F show solid amounts of shake angular displacement actually corrected with respect to the detected shake angular displacement amount (dotted line). Further, a control range limiter is provided inside the movable range of the shift lens 112 (processing performed by the saturation control unit 125).

  First, conventional image blur correction control will be described with reference to FIGS.

  In conventional image blur correction control, the cutoff frequency of the LPF used for the calculation of the integrator 123 is manipulated so that the shift lens 112 does not hit the end of the mechanical movable range.

  That is, when the output of the integrator 123 approaches the limiter, the cutoff frequency is increased and control is performed so that the output returns to the center (processing performed by the centering control unit 125).

  In FIG. 12C and FIG. 12D, the cutoff frequency is set to the normal Fc_base before time T3, and the correction effect is high.

  Therefore, there is almost no difference between the detected angular displacement amount and the corrected angular displacement amount. In the vicinity of time T3, when the image blur correction amount (the output of the integrator 123) approaches the end of the control range, the cutoff frequency is controlled to gradually increase so that the correction amount returns to the center.

  By controlling in this way, the shift lens 112 is prevented from hitting the end of the mechanical movable range.

  In other words, when the AF evaluation value is close to the target value of the imaging parameter, the image blur correction control unit 104 serving as a changing unit temporarily reduces the image blur correction performance, and then the image blur before the image blur correction performance is once reduced. It is larger than the correction performance.

  However, the conventional image blur correction is controlled independently of the AF control and is not related to each other.

  Therefore, as is apparent from FIGS. 12A and 12C, the appropriateness of AF is high (appropriateness is close to 0), and the effect of image blur correction is high (cutoff frequency is low). There are many cases.

  Next, image blur correction (processing performed by the imaging characteristic control unit 105) in this embodiment will be described with reference to FIGS. 12 (e) and 12 (f). The imaging characteristic control unit 105 in this embodiment refers to the AF appropriateness level, and operates the cutoff frequency of the integrator 123 according to the temporal change.

  FIG. 12A shows a state in which the subject is out of focus until time T1, and the AF appropriateness is large.

  In this period, as shown in the period up to time T4 in FIG. 12E, the cutoff frequency of the integrator 123 is set to the normal Fc_base. Next, the period from time T1 to time T2 in FIG. 12A is a period in which the AF appropriateness monotonously decreases.

  This is because the AF operation starts at time T1 and changes to a focused state at time T2. When the imaging characteristic changing unit 105 determines that the AF appropriateness monotonously decreases as shown from time T4 to time T5 in FIG. 12E, the imaging characteristic changing unit 105 sets the cutoff frequency of the integrator 123 high.

  As a result, as shown in FIG. 12 (f) from time T4 to time T5, the cutoff frequency of the integrator 123 increases, so that the image blur correction amount (the output of the integrator 123) becomes the center of the control range. It is controlled to approach.

  At time T5, when the AF appropriateness is not monotonically decreasing and is equal to or lower than a predetermined threshold, the cutoff frequency of the integrator 123 is set to return to the normal Fc_base, and the image blur correction effect is high. To.

  Then, still image data is generated at time T6. At this time, at time T6, the camera is in focus and the image blur correction effect is high.

  Further, as shown at time T5 in FIG. 12F, the position of the shift lens 112 is located near the center of the control range, and it is possible to perform image blur correction that hits the end of the control range at time T6. It has become.

  As described above, in the present invention, the control characteristic of the image blur correction is changed according to the temporal change of the AF appropriateness. As a result, it is possible to perform control so that each photographing state is as suitable as possible. When a still image is generated from a moving image, a good still image can be generated with a higher probability.

Example 3
Next, a third embodiment of the present invention will be described.

  In the second embodiment, referring to the AF appropriateness level, the control characteristic of the image blur correction can be changed according to the temporal change.

  In the third embodiment, a method for changing the control characteristic of AF control and the control characteristic of image blur correction according to the temporal change with reference to the appropriateness of AE control will be described.

  The processing is the same as that in the flowchart of FIG. 11 except that the AE suitability is used instead of the AF suitability and that the AF control characteristics are changed in addition to the image blur correction control cutoff control.

  FIG. 13A is a graph illustrating an appropriate exposure value (Ev_target) calculated from a photometric value of AE control and a current exposure value (Ev_now).

  The graph of FIG. 13 shows a case where the composition is changed so that a brighter subject is reflected by performing panning during shooting, for example.

  FIG. 13B shows the degree of AE suitability, and the closer to 0, the closer to the appropriate exposure, and the greater the value, the greater the difference from the proper exposure. Normally, AE control does not immediately change the exposure in accordance with the change in the photometric value.

  This is to prevent an operation in which a change in the brightness of the screen periodically repeats due to too sensitive control for changing the exposure according to the photometric value. Therefore, the exposure parameter is kept unchanged until the AE suitability exceeds a predetermined threshold (from time T11 to time T12).

  Then, after the predetermined threshold value is exceeded, the exposure parameter is changed, and control is performed so as to approach Ev_target (from time T12 to time T18).

  FIG. 13C shows the AF appropriateness. The closer the AF appropriateness is to 0, the more focused, and the greater the AF appropriateness, the more out of focus.

  From time T14 to time T15 in FIG. 13, by performing panning during shooting, the distance between the camera and the subject changes, and the AF appropriateness increases (changes in the direction of defocusing). Show.

  Similar to the second embodiment, the imaging characteristic control unit 105 monitors temporal changes in the appropriateness. In the third embodiment, the AE suitability is monitored.

  If it is determined that the AE suitability is monotonously decreasing, the control characteristic of the image blur correction unit and the driving speed of the focus lens, which is the control characteristic of the AF control, are calculated and set.

  First, if it is determined at time T16 in FIG. 13B that the AE suitability is monotonically decreasing, the slope of the AE suitability is calculated (the slope from time T13 to time T16 in FIG. 13B).

  A time Ae_time until the AE suitability falls within a predetermined range is calculated from the calculated inclination and the current AE suitability. That is, Ae_time is calculated from the difference between the current AE appropriateness and Ae_thresh and the slope of the AE appropriateness.

Next, the control characteristic of the AF control is calculated from the calculated Ae_time. In the AF control, the driving speed of the focus lens 113 corresponding to Ae_time is calculated and set in the AF control unit 103. The driving speed Af_speed of the focus lens is calculated by the following calculation formula.
Af_speed = K_af × Mdate_focus / Ae_time

  Here, K_af is an arbitrary coefficient. The driving speed Af_spped is larger as Ae_time is shorter, and is smaller as Ae_time is longer.

  In addition, Af_speed increases as the current AF appropriateness increases, and Af_speed decreases as the AF appropriateness decreases.

  In other words, when the evaluation value of the imaging parameter changes so as to approach the target value of the imaging parameter, the image blur correction control unit 104 that is a changing unit evaluates the evaluation based on the magnitude of the evaluation value and the gradient of the evaluation value change. Calculate the predicted time when the value is appropriate. Then, the image blur correction performance of the image blur correction unit is changed according to the predicted time.

Next, the cutoff frequency of the LPF used for the calculation of the integrator 123, which is the control characteristic of the image blur correction control, is calculated. The cut-off frequency Is_fc is calculated by the following calculation formula.
Is_fc = (K_is / Ae_time) + Fc_base

  Here, K_is is an arbitrary coefficient. Fc_base is a normal cutoff frequency. Thus, Is_fc is higher as Ae_time is shorter, and Is_fc is lower as Ae_time is shorter.

  The focus lens drive speed Af_speed and the image blur correction control cutoff frequency Is_fc calculated as described above are set in the AF control unit 103 and the image blur correction control unit 104.

  At time T17 in FIG. 13, the driving speed of the focus lens is returned to the original value. Similarly, at time T17, the cutoff frequency of the image blur correction control is changed so as to gradually return to the original value.

  By controlling in this way, it is possible to bring the AF appropriateness close to 0 before the time AE appropriateness is reached at time T18.

  Similarly, in the image blur correction control, the cutoff frequency can be returned to the normal Fc_base before the AE suitability reaches the proper exposure.

  After the AE suitability becomes appropriate, still image data is generated at time T19. At this time, the AF appropriateness can be close to 0, and the cutoff frequency for image blur correction is low, and a good still image can be generated with a higher probability.

  Further, as described above, the configuration is such that the AF control characteristics are changed or the image blur correction control characteristics are changed in accordance with the AE suitability.

  The reason for controlling in this way is the control period in each control.

  In normal AE control, exposure parameters such as aperture, shutter, and gain are changed based on a photometric value obtained from an image. Therefore, several frames from photometry to exposure change, for example, several tens ms to several hundred ms for NTSC. Time is required.

  On the other hand, in image blur correction, when camera shake of 1 to 20 Hz is targeted for correction, it is necessary to perform control at a frequency sufficiently higher than the frequency of camera shake. It becomes a cycle.

  Thus, it is desirable to refer to an appropriate value (for example, AE control) for a process having a long control cycle, and to change the characteristics of control (for example, image blur correction control) having a shorter control cycle.

  As described above, in the present invention, the control characteristics of the image blur correction and the AF control are changed according to the temporal change of the AF appropriateness.

  As a result, it is possible to perform control so that each photographing state is as suitable as possible. When a still image is generated from a moving image, a good still image can be generated with a higher probability.

(Other embodiments)
The object of the present invention can also be achieved as follows. That is, a storage medium in which a program code of software in which a procedure for realizing the functions of the above-described embodiments is described is recorded is supplied to the system or apparatus.

  Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the novel function of the present invention, and the storage medium and program storing the program code constitute the present invention.

  Examples of the storage medium for supplying the program code include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk.

  Further, a CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-R, magnetic tape, nonvolatile memory card, ROM, or the like can also be used.

  Further, by making the program code read by the computer executable, the functions of the above-described embodiments are realized.

  Furthermore, when the OS (operating system) running on the computer performs part or all of the actual processing based on the instruction of the program code, the functions of the above-described embodiments are realized by the processing. Is also included.

  Furthermore, the following cases are also included. First, the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer.

  Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing.

  In addition, the present invention is not limited to devices such as digital cameras, but includes built-in or external connection of imaging devices such as mobile phones, personal computers (laptop type, desktop type, tablet type, etc.), game machines, etc. It can be applied to any device.

  Therefore, the “imaging device” in this specification is intended to include any electronic device having an imaging function.

DESCRIPTION OF SYMBOLS 1 Imaging device 10 Camera system control part 11 Imaging lens 12 Imaging element 14 Video signal processing part 16 Recording control part 101 WB control part 102 AE control part 103 AF control part 104 Image blur correction control part 105 Imaging characteristic control part

Claims (7)

An evaluation unit that evaluates the quality of an imaging parameter acquired from image data output from an image sensor that captures a subject image and generates an evaluation value of the imaging parameter, and shake data output from the shake detection unit A generation unit that generates shake correction data, a control unit that controls the image blur correction unit using the shake correction data, and the evaluation value of the imaging parameter when moving image shooting is close to a target value of the imaging parameter. Changing means for making the image blur correction performance of the image blur correction means larger than the image blur correction performance of the image blur correction means when the evaluation value of the imaging parameter is far from the target value of the imaging parameter; and the image blur correction A recording means for recording a frame image having high performance as still image reproduction data,
The imaging parameter is any one of an exposure parameter used for exposure control, a focus state parameter used for focus lens control, and a white balance parameter.
The changing means reduces the image blur correction performance when the evaluation value of the imaging parameter is close to the target value of the imaging parameter, and thereafter, a difference value between the evaluation value of the imaging parameter and the target value of the imaging parameter is obtained. An image processing apparatus characterized by returning to the image blur correction performance before decreasing the image blur correction performance when a predetermined threshold value or less is reached.   When the evaluation value of the imaging parameter is close to the target value of the imaging parameter, the changing unit increases a cutoff frequency of a low-pass filter that determines the image blur correction performance, and then the evaluation value of the imaging parameter and the imaging 2. The image processing according to claim 1, wherein when the difference value of the target value of the parameter becomes equal to or less than a predetermined threshold value, the cutoff frequency is returned to the cutoff frequency before the cutoff frequency of the low-pass filter that determines the image blur correction performance is increased. apparatus.   When the evaluation value of the imaging parameter changes so as to approach the target value of the imaging parameter, the changing means determines that the evaluation value is appropriate from the magnitude of the evaluation value and the slope of the change in the evaluation value. The image processing apparatus according to claim 1, wherein an image blur correction performance of the image blur correction unit is changed according to the predicted time. An evaluation process for evaluating the quality of imaging parameters acquired from image data output from an imaging device that captures a subject image and generating an evaluation value of the imaging parameters, and using shake data output from shake detection means A generation step of generating shake correction data, a control step of controlling the image blur correction means using the shake correction data, and the evaluation value of the imaging parameter when moving image shooting is close to the target value of the imaging parameter A step of changing the image blur correction performance of the image blur correction means to be larger than the image blur correction performance of the image blur correction means when the evaluation value of the imaging parameter is far from the target value of the imaging parameter; and the image blur correction performance A recording step of storing a large frame image as still image reproduction data,
The imaging parameter is any one of an exposure parameter used for exposure control, a focus state parameter used for focus lens control, and a white balance parameter.
In the changing step , when the evaluation value of the imaging parameter approaches the target value of the imaging parameter, the image blur correction performance is reduced, and thereafter, a difference value between the evaluation value of the imaging parameter and the target value of the imaging parameter is calculated. An image processing method characterized by returning to the image blur correction performance before decreasing the image blur correction performance when the threshold value is below a predetermined threshold.   A computer-executable program in which the procedure of the image processing method according to claim 4 is described.   A computer-readable storage medium storing a program for causing a computer to execute each step of the image processing method according to claim 4.   An imaging apparatus comprising: the image processing apparatus according to any one of claims 1 to 3; and the imaging element.

Images