JPH0275284A - Image pickup device - Google Patents

Abstract

PURPOSE:To improve a blur correcting function by extracting movement information of an external sensor and a video camera respectively comparing a moving vector of a picture with movement information of the video camera so as to obtain a blur correction based on the arithmetic result. CONSTITUTION:An image pickup element 101 and a moving vector extraction section 102 extracting a moving vector of a picture from an output video signal detect a movement vector of a camera itself. On the other hand, an external sensor 106 and a camera movement information detection section 107 detecting the movement information of the camera by an output of the external sensor 108 detect the movement vector of the camera itself. Then a vector comparison section 103 comparing and evaluating the moving vectors calculate the moving vector and a blur correction signal generating section 104 outputs a blur correction signal 104 to cancel the moving vector and a blur correction section 105 moves a lens system in a direction to correct the blur. Thus, an excellent blur correction function is obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an imaging device having a function of correcting image blur caused by vibration of the device.

[Conventional technology]

BACKGROUND ART Conventionally, when taking pictures with a camera, camera shake has been a major problem as a cause of failure in taking pictures. Camera shake is relatively preventable when shooting still images because the shooting is instantaneous, but with video cameras and other devices that have become rapidly popular in recent years with the development of video equipment, Moreover, with miniaturization, hand-held shooting is becoming more common, and camera shake can be said to be an extremely important problem that degrades monitor screens and playback screens.

On the other hand, a method has been proposed in which this camera shake is detected and the lens system is automatically moved in a direction to offset the camera shake. These methods include detecting camera shake using a rotating gyro or acceleration sensor attached to the camera, and correcting the shake by moving the lens system etc. with an actuator according to the amount of shake; One possibility is to analyze the output video signal to detect image blur, control an actuator that moves the lens system, and correct the blur.

[Problem to be solved by the invention]

However, with the former method, it is difficult to obtain fully satisfactory characteristics such as response, accuracy, and sensitivity using only an external sensor such as a rotating gyro or an acceleration sensor, and with the latter method, an external sensor is not required. However, since the detected motion information is relative motion information between the subject and the camera, depending on the image pattern, it may be possible to determine whether the movement of the image is due to camera shake or the like, or whether it is due to the movement of the subject. There was a problem in that it was difficult to judge whether the image was a real object, and errors were likely to occur depending on the subject.

[Failure to solve the problem]

The present invention has been made with the aim of solving all of the above-mentioned problems, and its feature is that the present invention detects the amount of shake of the device based on mutually different information.
and a second motion detection means, and a third motion detection means for calculating the first and second motion information outputted from the first and second motion detection means and outputting third motion information. and,
and a correction means for controlling the position of an optical system to correct the amount of blur based on the third motion information outputted from the third motion detection means, and the imaging apparatus includes a plurality of different detection targets. For example, by detecting camera movement information using an external sensor (a sensor that detects acceleration, velocity, displacement, etc.), and performing image processing on the video signal, it is compared with the motion vector of the obtained image and calculated. An object of the present invention is to provide an imaging device equipped with a blur correction function that optimizes a correction signal, prevents variations in detection accuracy and malfunctions due to image patterns, and has excellent properties such as responsiveness and sensitivity.

〔Example〕

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an imaging apparatus according to the present invention will be described in detail below with reference to the respective figures.

FIG. 1 is a block diagram showing the basic configuration of a shake correction system for an imaging device according to the present invention.

In the figure, a motion vector of the camera itself is detected by an image sensor 101 such as a CCD, and an image motion vector extractor 02 that extracts a motion vector of an image from the output video signal of the image sensor 101. part sensor 106
A camera movement information detection unit 107 detects movement information of the camera based on the output of the external sensor 1.06, and a motion vector of the camera itself is detected.A vector comparison unit 103 compares and evaluates these motion vectors. A motion vector representing a blur component is calculated, a blur correction signal generation unit 104 outputs a blur correction signal 104 for canceling the motion vector representing image blur, and a blur correction unit 105 corrects the blur in the lens system. direction.

FIG. 2 is a block diagram showing an embodiment of the image stabilization system in the imaging device of the present invention, which is composed of a camera section A, a lens state detection section, a pre-correction signal generation section, and a correction section. There is.

In the camera section A, 1 includes a photographing lens and forms a subject image on the imaging surface of an image sensor (to be described later); An image sensor such as a CCD, which is converted into a signal, is a camera signal processing circuit 3 that reads out the electrical signal output from the image sensor, performs predetermined signal processing, and outputs a luminance signal Y11 signal C and the like.

In the lens state detection unit B, 11 is an angular velocity sensor attached to the camera (the attached state is not shown);
12 is a sensor signal processing circuit that performs predetermined signal processing on the output of the angular velocity sensor 11 and outputs angular velocity information according to the amount of camera shake detected; This is a zoom encoder that detects focal length information due to zooming.

In the blur correction signal generation section C, 4 converts the video signal output from the camera signal processing circuit 3 into one field (or one field).
5 is an A/D converter that converts the A/D converter into a digital signal in units of frames), divides the screen into n blocks, compares the digital information in each block in the screen of the previous field, and compares the digital information in both screens. From the displacement of the representative point above, the block motion vector ilj + (x=1.2.
... + n), and 13 is an A/D that converts the angular velocity information of the sensor signal processing circuit I2 in the lens state detection system B into a digital signal.
A converter 14 is a camera movement device that calculates a camera motion vector υcam due to shake etc. using the following equation based on the digital angular velocity information output from the A/D converter 13 and the lens focal length information obtained from the zoom encoder 16. It is a vector calculation circuit.

v cam = f-tan ((J) ・±) where f: focal length (mm) ω: angular velocity of camera (rad/s) 6 is the motion vector υ2 of the image output from the block motion vector detection circuit 5 ( i = 1, 2... + n) and the camera motion vector υcam output from the camera motion vector calculation circuit 14 are compared and evaluated, and a representative motion vector υ, which is integrated as screen movement and will be described later. Panning/delting control signal that controls panning and delting judgment signals.
This is a representative motion vector calculation circuit that calculates and outputs the tilt control signal PTCONT. This algorithm will be described later. Reference numeral 15 compares the camera motion vector υcam output from the camera motion vector calculation circuit 14 for each field, and performs panning according to the displacement of the vector.

This is a pan/tilt determination circuit that determines whether tilting or the like is occurring and outputs a pan/tilt mode signal PTMODE representing the determination result. This pan/tilt judgment circuit uses the pan/tilt mode signal PTMO during normal shooting.
DE=0 is output, and if the direction of the camera motion vector does not change after a certain number of fields, it is determined that the pan/tilt mode has entered, and the pan/tilt mode signal PTM is output.
It outputs ODE=1 and continues to output "1" until the direction of the camera motion vector is reversed.

7 is focal length information f1 from the zoom encoder 16; pan/tilt mode signal P from the pan/tilt determination circuit 15;
TMODE, representative motion vector υ from representative motion vector calculation circuit 6 and pan/tilt control signal PTC
ONT and calculates the overall amount of blur, and generates a correction signal that outputs a blur correction signal υdriv corresponding to the amount of blur and a trigger signal R that resets the optical axis rotation means of the correction system to the initial position, which will be described later. It is a circuit. Reference numeral 8 denotes a D/A converter 8 that converts the blur correction signal υdr1v and trigger signal R outputted from the correction signal generation circuit 7 into analog signals that can control a correction system to be described later.

Reference numeral 17 denotes a system control circuit comprising a microcomputer that controls the operation of the entire apparatus including each circuit in the shake correction signal generating section C at a predetermined timing using control signals 01 to 08 and the like.

In the correction system, 9 is a blur correction signal υdriv output from the correction signal generation circuit 7 in the blur correction signal generation system C.
10 includes a lens position control mechanism such as an optical axis rotation means (not shown) disposed in the lens system. This is a correction system that moves the lens system in the direction that corrects the movement of the screen.

The imaging device of the present application has the above configuration, and next, the main blocks in the above block diagram will be explained in more detail.

FIG. 3(a) is a flowchart for explaining an algorithm for calculating the representative motion vector υ of an image and the pan/tilt control signal PTCONT in the representative motion vector calculation circuit 6 in the blur correction signal generation system C.

In the same figure, n set on the imaging screen at 5 tepl.
Each block motion vector υ+ (i=1.2...,n) and camera motion vector υ in each of the blocks bI. 3m is input. However, Fig. 3(b)
As shown in , numbers are assigned in a spiral pattern from the upper left corner, b2°, . There is. Camera motion vector υ. 3m is 5tep
2 evaluates the magnitude of the vector, 1υcaml
If ε is smaller than the predetermined value ε, the process proceeds to step 3, where the representative motion vector υ is set to O, and the blur correction operation is stopped. In other words, it is determined that the movement of the camera is small and there is no need for correction.

In addition, in 5tep3, if lυcam l≧ε, it is determined that a blur has occurred, and the process moves to 5tep4, where each block motion vector υ+ (r = 1, 2, .
...+n) is compared with the camera motion vector υcam in all blocks of the screen 20 as shown in FIG. The number of vectors υi that match am and their average vector V COR+ are calculated.

Next, proceed to 5tep 5, 5tep 6, and υca
The number N of υ1 that matches m is compared with predetermined values N, , N2.

However, there is a relationship between N, and N2 as follows: 0≦N1≦N2≦n (n is the number of blocks on the screen).

Then, the number N matching υcam is very small and 0
In the case of screen pattern 1 where ≦N≦N1, 5te
Proceed to p7 and output the pan/tilt control signal PTCO.
NT is set to "1", and then in step 5, the camera motion vector υ is set as the representative motion vector υ. am and outputs the pan/tilt control signal PTCONT (
=1) is output.

Also υ. The number N that matches am is very large, N≧N2
In the case of screen pattern 2, pan/pan in step 9.
Set the tilt control signal PTCONT to “1” and then set the representative motion vector υ at 5teplO for 5tep.
It outputs the average vector VCORI obtained from 4, and also outputs the pan/tilt control signal PTCONT(-
]) is output.

Note that specific states of vectors on the imaging screen in the above-mentioned screen pattern 1 and screen pattern 2 will be described later.

When the number N of motion vectors υ1 of each block that matches the camera motion vector υcam takes an intermediate value between N1 and N2 (N I≦N≦N2), proceed to step 5 and proceed to Fig. 3 (d). ), the vector υ is centered at the center of the imaged screen. υi (i=m...
The number M of n-1) and its average vector V C0R2 are calculated and determined. In 5tep12, 5tepl l
The determined number M is compared with a predetermined value M1, and in the case of screen pattern 3 where M<M, the process advances to step 515 and the pan/tilt control signal PTCONT is set to "]"
Then, in 5tep16, the average vector VCORI of the vector υ1 that matches the camera movement vector υCam described above is output as the representative motion vector υ, and the pan/tilt control signal PTCONT (=1.) is output.
Output. In addition, in the case of screen pattern 4 where M≧M1 in 5tep 12, proceed to 5tep 13,
The pan/tilt control signal PTCONT is set to "0", and then, at Step ], 4, υ in the center area of the screen is set. The average vector V C0 of υ that matches
It outputs R2 as a representative motion vector V and also outputs a pan/tilt control signal PTCONT (20). However, the above-mentioned predetermined value M1 is a constant determined within the range of 0≦M,≦n-m (m is the number of the first block in the screen center area in FIG. 3(d)).

Next, the four cases (1) to (4) in Figure 4(a)
For each, the state of the vector of each block on the screen at that time will be specifically explained using FIGS. 4(a) to 4(d).

■) For screen pattern 1, N<N.

In this case, the motion vector υ1 of each block on the screen
(i=1.2...+n) Camera motion vector υ. a
This is a state in which the number of objects that match m is very small, which means that the movement of the subject is greater than the movement of the camera.

In this case, as shown in FIG. 4(a), the motion vector of each block is υ. am is different from υ, or the motion vector υ1 of each block in FIG.

A case may be considered in which it coincides with a direction different from am. It can be assumed that in the case of FIG. 3(a), a small object is moving in a scattered manner, and in FIG. 2(b), the object occupying most of the screen is moving in a certain direction. In addition, there may be an intermediate state between the state shown in FIG. 4A and the state shown in FIG.

■) For the case of screen pattern 2, N>N2 camera motion vector υ. When the number of motion vectors υ1 of each block that match am is very large, that is, when the movement of the subject is small (the movement of the screen is due to the movement of the camera).

As shown in FIG. 3C, the motion vector υ1 of each block on the screen is aligned in approximately the same direction as the camera motion vector υcam.

Next, the cases of screen patterns 3 and 4 will be explained.

In the case of screen patterns 3 and 4, values are intermediate between the above-mentioned screen patterns 1 and 2, and in these cases, the effects on the screen due to camera movement and subject movement are the same. As shown in the control flow from step 12 onwards, the cases can be further divided depending on the state of the subject being photographed.

1. TI) Regarding the case of screen pattern 3: M<M.

In this case, since screen patterns 1 and 2 do not apply, it is assumed that the subject mainly exists in the center of the screen, and as explained in FIG. 4(d), the center of the imaged screen is is the motion vector υ at the center of the screen. The motion vector υ1 (1= m , m +1 , −
, n-1) and the average vector V C0R2 at that time to determine whether there is an object to be consciously captured in the center of the screen. That is, in case (3) when M is less than the predetermined value M1,
), it is determined that there is no consciously captured subject in the center of the screen, and the average vector V C0R2 is output as the representative motion vector υ.

■■) Regarding the case of screen pattern 4 2M≧M1 In this case, there is a subject that is consciously captured in the center of the screen, and as shown in Figure 4 (e), the camera is located in the center of the screen. There is a constant motion vector I of the subject that is different from the motion vector υcam. The representative motion vector υ is the average vector V C0R2 of the motion vector υ+(+-m,...,n-1) of each block that matches the motion vector υm of block bn in the center of the screen in the center area of the screen. Output.

In the above four image patterns, image pattern 1.2
．． 3, the camera motion vector υ. Since a representative motion vector based on a□ is output, conscious operation of the camera, that is, judgment of panning, tilting, etc. is required. However, in the image pattern, since the motion vector based on the movement of the subject is used as the representative motion vector, determination of panning and tilting is not necessary. Therefore, the pan and tilt control signal PTCONT is the image pattern].

For 2.3, the value is set to 1'', and a signal indicating that it is necessary to take panning and tilting into consideration is output.

Regarding image pattern 4, since there is no need to consider panning and tilting, the panning and tilting control signal PR
CONT is set to "0" to output a signal indicating that pan/tilt judgment is not performed.

Next, in the block diagram of FIG. 2, the configuration and operation of the blur correction signal generation circuit 7 will be explained based on the representative motion vector υ output from the representative motion vector I/L calculation circuit 6 and the pan/tilt control signal PTCONT. do.

FIG. 5(a) is a block diagram showing the internal configuration of the correction signal generation circuit 7. As shown in FIG.

The sign of the representative motion vector υ(x, y) outputted from the representative motion vector calculation circuit 6 is inverted by the sign inversion circuit 201, and the sign is inverted by the correction angle calculation circuit 202, which is necessary for blur correction, based on the focal length information f of the lens system. The lens correction angles θyaw and θpitch are calculated (θyaw
is the correction angle in the y a w i n g direction, θpitc
h is the correction angle in the pitching direction). These lens correction angles are determined by the following formula,
It is output as a blur correction signal υdriv.

where f, focal length Signal PT
Based on MODE, the multiplier 204 performs the logical product (A
ND) is output.

Figure 5(b) shows these signals PTCONT and PTMOD.
FIG. 6 is a diagram showing AND output logical values in each of cases (1) to (4) according to the above-mentioned screen situation of E. The AND output of the multiplier 204 is used as a switching control signal for the switch 203, and when the AND output of the switch 203 is "0", the contact is switched to the output of the correction angle calculation circuit 202, that is, the above correction angle θyaw,
The blur correction signal υdriv is output based on θpitch, and when the AND output of the multiplier 204 is "1", it is switched to the open contact a, and the blur correction signal υdriv is output as "
Output as 0''.

The rising edge detection edge trigger circuit 205 also includes a multiplier 20
The trigger signal R is output in synchronization with the timing when the output of 4 is inverted from "0" to "1". This trigger signal indicates that the camera has entered the pan/tilt mode, but by the time pan/tilt mode is determined, the lens system has already been corrected in one direction for several fields by the correction means. If the correction means is stopped at this position, when the correction is started after panning and tilting, the correctable area of the correction means will be biased in one direction, making correction difficult.

Therefore, when this trigger signal is output, the drive circuit 9 returns the optical axis rotation means of the correction system 10 to the initial position at a speed that is not visually noticeable on the screen, and the panning and tilting are completed. When blur correction is started, it is configured to always perform correction from the initial position, so that the correction range can be maximized.

To summarize the operations of the representative motion vector calculation circuit 6 and the correction signal generation circuit 7 described above with reference to FIG. Since it is acting on the motion vector υ1 of the block, it is necessary to consider panning and tilting. Therefore, in these cases, the pan/tilt judgment signal PTMODE output from the pan/tilt judgment circuit I5
In order to take into account the pan/tilt control signal PTCON
Set T to “1” and output AND output according to PTMODE.
obtain.

Regarding image pattern 4, since the influence of the camera motion vector is small and the motion vector of the subject is used as the representative motion vector, the pan/tilt mode determination signal PT
There is no need to consider MODE; the pan/tilt control signal PTCONT is set to "O" and the AND output is set to 0.

As described above, according to the imaging device of the present invention, the movement of the camera itself is detected by the angular velocity sensor 11 attached to the camera, and the sensor signal processing circuit 12, A/D converter 13, and camera motion vector calculation circuit 14 detects the camera motion vector υcam and whether the camera movement continues in a certain direction for a predetermined period or longer, determines whether or not it is in the pan/tilt I state, and outputs a pan/tilt mode signal PTMODE representing the result. do.

In addition, a video signal corresponding to the subject image obtained by the image sensor 2 and the camera signal processing circuit 3 is converted into a digital signal by the A/D converter 4, and each block motion vector detection circuit 5 sets the video signal on the imaging surface. The motion vector distribution state υ1 (i=1.2...
, n).

Then, the camera motion vector υcam is compared with the motion vector υ1 of each block on the imaging surface, and the relationship between them is evaluated using the above-mentioned algorithm, and the motion vector υ that represents the screen is obtained, and at the same time panning is performed.

The necessity of considering tilt is evaluated, and a pan/tilt control signal PTCONT is obtained for determining whether or not it is necessary to consider pan and tilt.

Next, the correction signal generation circuit 7 generates the representative motion vector υ
A lens correction amount υdriv for correcting blur is calculated using the pan/tilt control signal PTCONT and the pan/tilt mode signal PTMODE, and is supplied to the drive circuit 9 via the D/A converter 8 to correct the amount of blur. The correction system 10 is driven to adjust the lens system.

Through the above operations, it is possible to realize a blur correction system that takes into account both camera movement and subject movement.

〔Effect of the invention〕

As described above, according to the imaging device of the present invention, a method of extracting a motion vector of an image by image processing of an imaging signal from an imaging means, means for extracting motion information of an external sensor and a video camera, respectively; comprising a means for comparing the motion vector of the image and the motion information of the video camera;
By determining the amount of blur correction based on the calculation results, it is possible to improve accuracy, sensitivity, and responsiveness compared to devices that use only external sensors, and to reduce the amount of image blur caused by relative movement between the subject and the video camera. It becomes possible to separate whether the movement is caused by the movement of the subject or the movement of the video camera. Therefore, by recognizing the movement of the image that should be corrected and the movement of the image that should not be corrected, it is effective to greatly reduce errors in the blur correction system. Furthermore, by determining the magnitude of the camera motion vector, the blur correction system operation can be stopped when the camera is not moving, thereby preventing malfunctions due to subject movement.

[Brief explanation of the drawing]

Fig. 1 is a block diagram showing the basic configuration of the shake correction system for an imaging device according to the present invention, Fig. 2 is a block diagram embodying the block diagram of Fig. 1, and Fig. 3(a) is a block diagram of Fig. 2. Flowchart showing the calculation algorithm of the representative motion vector calculation circuit in the figure, Figure 3(b) is a diagram for explaining the setting state of the motion vector detection block on the imaging screen,
d) is a diagram showing the image capture screen pattern used to explain the calculation flowchart in FIG. Figure 5 is a block diagram showing the internal configuration of the correction signal generation circuit in the block diagram of Figure 2, Figure 6 is Figure 4(a) showing the distribution pattern.
FIG. 4 is a logic diagram for explaining the operation of the correction signal generation circuit in each image pattern of FIG. 4(e).

Claims (1)

[Scope of Claims] First and second motion detecting means for detecting the amount of shake of the device based on mutually different information; and first and second motion detecting means output from the first and second motion detecting means. a third motion detecting means for calculating motion information and outputting third motion information; and correcting the amount of blur based on the third motion information output from the third motion detecting means. An imaging device comprising: a correction means for controlling the position of an optical system.