JP2003141506A - Image processing apparatus and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To arbitrarily vary a focal range after imaging. SOLUTION: A personal computer 20, which creates a composite image for which the focal range of a subject is arbitrarily adjusted, by performing image processing on image data Dg such that images of the subject are picked up at N focal positions for the same subject, includes a display part 23 for displaying an editing operation screen, a keyboard 24 for inputting the contents of processes, and a CPU 27 causing the display part 23 to display the editing operation screen and creating a composite image by performing image processing on the image data Dg according to the contents of the processes inputted. Of the images corresponding to the image data Dg, an image conforming to inputted focal position information is displayed by the CPU 27 on the editing operation screen as a reference image, and in accordance with the inputted focal range information, image information corresponding to the image range is extracted from the image data Dg and composed into the reference image so as to adjust the focal range.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and an image processing program which generate a composite image based on digital image data obtained by capturing object images of the same object at different focal positions.

[0002]

2. Description of the Related Art Today, the spread of personal computers,
With the generalization of Internet connection using personal computers, digital still cameras for recording digital image data have begun to spread widely.
This digital still camera receives a subject image in response to a release operation by a photographer, and converts the optical information (subject image) into an electric signal to generate digital image data. In addition, the generated digital image data is recorded in a removable memory such as a memory card. In this case, a general digital still camera is equipped with an automatic focusing mechanism, and in order to facilitate focusing while reducing the size and weight of the camera body, a lens having a large F number (depth of field) is used. But deep optics) is adopted. Therefore, by simply pointing the digital still camera toward the subject and operating the release button, an image that is in focus over the entire image from the subject that is focused on by the autofocus mechanism to the subject on the farther side can be displayed. You can shoot. In addition, digital image data of the captured image is recorded in the removable memory.

On the other hand, with respect to the digital image data recorded in the removable memory, various image processing, display on a display unit, and printing by a printer can be performed by, for example, a personal computer in which an image processing program is installed. In this case, since the conventional digital still camera employs an optical system with a deep depth of field, it can be applied to a subject (for example, a background such as a flower field) located far away from a focused subject (for example, a person). It will be in focus. Therefore, it becomes difficult to capture an image with a blurred background such as a portrait image. Therefore,
For example, when editing an image in which a flower field exists around a person as a background and performing image processing in a portrait-like manner by blurring only the flower field, first, start an image processing program and set the digital image data to be processed. Read. Next, the area other than the person, that is, the background flower field is selected by designating the outline of the person by the range selection operation. At this time, the pointer on the screen is moved along the boundary of the selection range (in this case, the outline of the person) by operating the mouse, for example. Next, by performing a filtering process on the selection range, for example, the color information between pixels adjacent to the boundary portion where the color information greatly differs in the selection range is averaged. As a result, the image of the selected range is blurred. Then, the blurred background is superimposed on the image at the beginning of editing to combine the images into one image.
As a result, a portrait-like image in which the flower field existing in the background of the person is fluffed is completed.

[0004]

However, the conventional image processing program has the following problems. That is,
In a conventional image processing program, when a portrait-like image with a blurred background is edited, filter processing is performed on the background portion selected by the operator to average color information and blur the image. There is. In this case, the operation of specifying the selection range (moving the pointer) is extremely difficult for a person who is unfamiliar with the operation of the personal computer, and the background of a portion close to the contour of the person is out of the selection range or Some are included in the selection. When the filtering process is executed in such a state, a part of the background that should be blurred is not blurred, or a part of the person who is not the target of blurring is blurred. As described above, the conventional image processing program has a problem in that it is very difficult to generate an image with a desired blur amount due to the difficulty in the range selection operation.

In this case, by adopting an optical system having a shallow depth of field for the digital still camera, it is possible to generate an image with a blurred background while eliminating the need for an image processing program. However, when using a digital still camera incorporating this optical system to capture an image that is in focus over the entire image, it is necessary to change the aperture value of the optical system to increase the depth of field. Will be needed. In this case, since this operation is very complicated for an unfamiliar person, it becomes difficult to capture an image that is in focus throughout. Also,
In a conventional image processing program, it is difficult to generate an in-focus image over an entire image because it is difficult to edit an out-of-focus image into an in-focus image. There is a point.

On the other hand, there is an all-focus digital microscope capable of photographing a plurality of digital images having different focal positions for the same subject. This all-focus digital microscope shoots a plurality of images while changing the focus position within the depth of field range of the optical system at the time of shooting. Therefore, it is possible to include in-focus image information for each captured image. Further, after the image capturing is completed, only the image information of the in-focus portion (focus range) is extracted from each image by the image processing program incorporated in the all-focus digital microscope, and is combined as one image data. . Therefore, it is possible to display an image focused on the entire screen. However, the image processing program installed in the omnifocal digital microscope can execute various image processing for images captured by the microscope, but executes image processing for digital image data captured by a digital still camera, for example. There is a problem that you cannot do it. Also, an all-focus digital microscope can shoot very close subjects, but it cannot shoot, for example, distant mountains or trees, so use this microscope like a digital still camera. There is also a problem that it is not possible to photograph people etc.

The present invention has been made in view of the above problems, and a main object of the present invention is to provide an image processing apparatus and an image processing program capable of arbitrarily changing a focusing range after photographing.

[0008]

In order to achieve the above object, the image processing apparatus according to the present invention takes an image of a subject at different focal points of N points (N is a natural number of 2 or more) for the same subject. An image processing apparatus configured to generate a composite image in which a focusing range of the subject is arbitrarily adjusted by executing predetermined image processing on each digital image data, and displaying an edit work screen. A display unit for performing processing contents, an operation unit for inputting processing contents, the editing work screen is displayed on the display unit, and the digital image data corresponding to the digital image data is input according to the processing contents input via the operation unit. A control unit for generating the composite image by executing predetermined image processing, wherein the control unit controls the front of the images corresponding to the digital image data. An image matching the focus position information input via the operation unit is displayed on the editing work screen as a reference image, and predetermined from the digital image data according to the focus range information input via the operation unit. The in-focus range is adjusted by extracting image information corresponding to the image range and combining with the reference image.

In this image processing apparatus, the control section causes the image corresponding to the focal position information among the images corresponding to the respective digital image data to be displayed as the reference image on the editing work screen, and according to the focus range information. By adjusting the focus range by extracting image information corresponding to a predetermined image range from each digital image data and synthesizing it with the reference image, for example, an image with a desired blur amount after completion of shooting by the digital still camera Can be easily created.

Further, in the image processing apparatus according to the present invention, in the image processing apparatus, the control unit includes an image matching the focus position information among the images corresponding to the digital image data. When the focus position is not set, the focus position is set to a position 2 between the focus positions corresponding to the focus position information.
The two reference images are generated by specifying two images and performing a complementary process on the two images.

In this image processing apparatus, when there is no image matching the focus position information among the images corresponding to each digital image data, the control unit controls the focus position whose focus position corresponds to the focus position information. By identifying two images at positions sandwiching the two and performing a complementary process on the two images to generate a reference image, based on the digital image data of various focus positions, the focusing range of the operator's desired focus range can be obtained. An image can be generated reliably and easily.

Further, in the image processing device according to the present invention, in the image processing device, the control unit determines the focus range in the reference image as a part of the composite image, and then sets the focus range information. Correspondingly, the predetermined image range in the image is extracted sequentially from the image whose focal position is a long distance next to the reference image, is combined with the reference image, and the combined range is determined as a part of the combined image. .

In this image processing device, after the control unit determines the focus range in the reference image as a part of the composite image, the control unit determines the focus position from the image at the long distance next to the reference image according to the focus range information. It is possible to accurately determine a desired focus range by sequentially extracting a predetermined image range in the image, combining the extracted image range with the reference image, and determining the combined range as a part of the combined image.

Further, in the image processing apparatus according to the present invention, in the image processing apparatus, the control section includes an edit image display area for displaying the reference image being edited, and the focus position via the operation section. Display on the display unit the editing work screen configured to include a focus position input region for inputting information and a focus range input region for inputting the focus range information via the operation unit. With it, when the predetermined focus position information is input to the focus position input area, the digital image data that matches the focus position information is selected and displayed in the edited image display area as the reference image, When predetermined focus range information is input to the focus range input area, image information corresponding to the predetermined image range from each of the digital image data according to the focus range information is input. Extracts combined into the reference image.

In this image processing apparatus, an edit work screen including an edit image display area, a focus position input area and a focus range input area is displayed on the display unit, and input values of the focus position input area and the focus range input area are displayed. By the control unit executing various image processings accordingly, even an operator unfamiliar with the operation can easily generate an image with a desired blur amount by designating a desired focusing range.

Further, in the image processing apparatus according to the present invention, in the image processing apparatus, the focus range input area is configured to be able to input the focus position information by inputting subject distance information, and the control unit. Specifies an image to be displayed as the reference image based on the focus position information corresponding to the input subject distance information.

In this image processing apparatus, the focus range input area is configured so that the focus position information can be input by inputting the subject distance information, so that a desired focus range can be designated and an image with a desired blur amount can be specified. Can be easily generated.

The image processing apparatus according to the present invention is the image processing apparatus as described above, wherein the focus range input area specifies an aperture value that simulates the specification of an aperture value in a photographing apparatus or an image blur amount. The focusing range information can be input by any one of the blurring amount designations.

In this image processing apparatus, the focus range information can be input by either the aperture value designation imitating the aperture value designation in the photographing device or the blur amount designation that designates the blur amount of the image. By configuring the input area, especially for an operator who is accustomed to operating the optical system photographing device (still camera), it is possible to easily generate an image with a desired blur amount as if the operator is operating the photographing device. It will be possible.

Further, in the image processing device according to the present invention, in the image processing device, the control unit determines each image corresponding to each digital image data based on a focus position corresponding to each digital image data. Virtually superposed and developed, three-dimensional image data including the superposition position information of each image in the developed state and each digital image data is generated, and the predetermined three-dimensional image data is generated using the generated three-dimensional image data. Image processing is executed.

In this image processing apparatus, the control unit virtually superimposes and develops the images corresponding to the respective digital image data based on the focal position corresponding to the respective digital image data, and develops the respective images. By generating 3D image data including the overlay position information and each digital image data, and performing the predetermined image processing using the generated 3D image data, the time required for the image processing can be significantly reduced. Is possible.

An image processing apparatus according to the present invention is the image processing apparatus described above, wherein each of the digital image data is
The digital image data corresponding to the L-th focal point (L is a natural number of 2 or more and N or less) is (L-1) th digital image data and the Lth digital image data. It is composed of difference data obtained by extracting the difference between and.

In this image processing apparatus, the L-th point is formed by the difference data obtained by extracting the difference between the (L-1) -th shot digital image data and the L-th shot digital image data. Since the digital image data corresponding to the focus position can be image-processed, it is possible to easily and arbitrarily generate an image with a desired blur amount based on the digital image data that cannot be processed by the existing image processing program. It will be possible.

Further, in the image processing device according to the present invention, in the image processing device, each digital image data is recorded in a single data file format in which each digital image data is made into one file.

In this image processing apparatus, since the image processing can be performed on the digital image data recorded in a single data file format in which each digital image data is made into one file, the image processing can be performed by the existing image processing program. An image with a desired blur amount can be arbitrarily and easily generated based on unprocessable digital image data.

An image processing apparatus according to the present invention is the image processing apparatus as described above, wherein each of the digital image data is
A data structure including at least one of generation order data for specifying the generation order, focus position data for specifying the focus position, and object distance data for specifying a distance to a focused object. The control section specifies the focus position corresponding to each of the digital image data based on any one of the generation order data, the focus position data, and the subject distance data.

In this image processing apparatus, generation order data,
Since the image processing can be performed on the digital image data having the data structure including at least one of the focus position data and the subject distance data, it is possible to perform various image processing by using the information.

Further, in the image processing device according to the present invention, in the image processing device, each of the digital image data includes angle-of-view data for specifying an angle of view of the photographing device when the digital image data is generated. It consists of a data structure.

In this image processing apparatus, since the image processing can be performed on the digital image data having the data structure including the angle-of-view data for specifying the angle of view of the photographing apparatus at the time of generation, the non-telecentric optical system is particularly preferable. It is possible to surely and easily perform image correction (position correction at the time of composition processing) on digital image data taken by a digital still camera adopting the device.

The image processing apparatus according to the present invention is the image processing apparatus described above, further comprising a printing device for printing the composite image.

This image generating apparatus is provided with a printing apparatus for printing a composite image, so that an image adjusted to a desired blur amount can be printed and attached to, for example, an album or photo stand or sent along with a letter. It becomes possible.

Further, the image processing program according to the present invention causes the above image processing apparatus to execute the image processing, and an image in which the synthesized image in which the focus range is arbitrarily adjusted can be generated is described. A processing program, wherein the image processing program causes a computer as the control unit to identify an image that matches the focus position information input via the operation unit, and The image information corresponding to the predetermined image range from each digital image data is displayed on the editing work screen in the display unit as the reference image, and in accordance with the focusing range information input via the operation unit. Is extracted and combined with the reference image to adjust the focusing range.

In this image processing program, the computer as the control unit is caused to specify the image that matches the focus position information input via the operation unit, and the specified image is used as the reference image in the display unit. The focus range is displayed on the editing work screen, and image information corresponding to a predetermined image range is extracted from each digital image data according to the focus range information input via the operation unit, and is combined with the reference image to focus range. By adjusting the, it becomes possible to reliably and easily create an image with a desired blur amount using a computer.

[0034]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an image processing apparatus and an image processing program according to the present invention will be described below with reference to the accompanying drawings.

First, the structure of the digital still camera 1 according to the present invention, which is configured to be capable of photographing "each digital image data having different focal positions", will be described with reference to the drawings.

As shown in FIG. 1, the digital still camera 1 includes an optical system device 2 and a CCD (Charge Coupled Device).
) 3, shutter mechanism 4, linear motor 5, position detector 6, release button 7, CPU 8, memory 9, LCD panel 10, RAM 11 and ROM 12.
The optical system device 2 is a device for forming an image of a subject to be photographed on the CCD 3, and an F number of 2.
It is composed of about eight telecentric optical system lens groups. The optical system device 2 includes a plurality of lenses such as a lens 2a (focusing lens; see FIG. 2) for changing the focal position. In this case, the lens 2a
Is moved by the linear motor 5 in the directions of arrows A and B shown in the figure, thereby changing the focal position of the image formed on the CCD 3. Specifically, when the lens 2a is moved in the direction of arrow A from the position indicated by the broken line in FIG. 4, the focal position is changed from the short distance side to the long distance side, and the lens 2a is changed from the position indicated by the dashed line to the arrow. When moved in the direction of B, the focus position is changed from the long distance side to the short distance side. As shown in FIG. 2, a plurality of slits are formed in the lens 2a at equal intervals, and a gauge 2b for detecting the position of the lens 2a by the position detection unit 6 is attached. The detailed configuration of the optical system device 2 including the lenses other than the lens 2a is a general variable focus lens system, and therefore the description and illustration thereof are omitted.

The CCD 3 is an electronic component that converts (photoelectrically converts) light information (subject image) incident through the optical system device 2 into a plurality of light receiving elements (pixels) that convert light into electricity. ) Are arranged in a matrix to output an electric signal according to the light receiving level at each arrangement position. The shutter mechanism 4 regulates light incident on the CCD 3 in a normal state, and a mechanical mechanism that allows light incident on the CCD 3 for a predetermined time (a time corresponding to the shutter speed) under the control of the CPU 8 or a predetermined time. It is an electronic shutter mechanism that captures optical information. As described above, the linear motor 5 moves the lens 2a of the optical system device 2 under the control of the CPU 8. In this case, as shown in FIG. 3, the rate of change of the focal position per moving distance of the lens 2a is in accordance with the optical characteristics of the optical system device 2 (characteristics indicated by the characteristic line X). The closer it is to the position), the more gradual the change, and the longer the distance (lens 2a).
Changes closer to CCD3).
Therefore, when the lens 2a is moved from the position of the broken line shown in FIG. 4 by the linear motor 5 in the direction of the arrow A at a constant speed, the focal position is first gently changed,
After that, it is changed so that it becomes gradually steeper.

The position detector 6 detects the position of the lens 2a moved by the linear motor 5 and outputs the position data to the CPU 8. As shown in FIG. 2, the position detection unit 6 includes a light emitting element 6a and a light receiving element 6b that are opposed to each other with the gauge 2b interposed therebetween.
The light emitting element 6a is driven when the linear motor 5 is driven (that is,
The light is emitted by the CPU 8 when the lens 2a moves. The light receiving element 6b receives the light that has passed through the slit of the gauge 2b among the light generated by the light emitting element 6a. In this case, the light emitting element 6a is moved along with the movement of the gauge 2b.
By passing a plurality of slits in front of, the light-receiving element 6b is irradiated with blinking light. At this time, the light receiving element 6b
Outputs an electric signal to the CPU 8 each time it receives light. Therefore, the CPU 8 specifies the position of the lens 2a based on the number of input electric signals output from the light receiving element 6b. The release button 7 functions as a switch for starting shooting. In this case, when the photographer operates the release button 7, the CPU 8 controls the operation of each unit to generate digital image data.

The CPU 8 generates image data Dg1 to Dg8 corresponding to each digital image data in the present invention,
Extraction of difference data Dd1 to Dd7, focus position data Dl
Generation of 1 to D18 and generation of the image data file Fg are executed. The memory 9 is composed of a memory card (removable memory) that can be attached to and detached from the digital still camera 1 and a personal computer 20 described later. The LCD panel 10 displays a subject image converted into an electric signal by the CCD 3 under the control of the CPU 8, and also displays an image based on digital image data of which photographing is completed. The RAM 11 temporarily stores the calculation result of the CPU 8, and the ROM 12 stores the operation program of the CPU 8.

Next, a method of using the digital still camera 1 will be specifically described with reference to the drawings.

First, the digital still camera 1 is held toward a subject. At this time, the CPU 8 causes the shutter mechanism 4 to
The subject image is made to enter the CCD 3 by intermittently opening and closing at a predetermined interval. As a result, an image based on the electric signal photoelectrically converted by the CCD 3 is displayed on the LCD panel 1.
Displayed at 0. In this case, as shown in FIG. 4, a cup 31, a house 32, a standing tree 33, and a mountain 34 are present in this order from the front side at the tip of the digital still camera 1. Next, when the release button 7 is operated by the photographer, the CPU 8 calculates the angle of view based on the driving state of the optical system device 2 and its characteristics, and the angle of view data D
a is stored in the RAM 11. At the same time, the CPU 8 drives the linear motor 5 to move the lens 2a of the optical system device 2 in the direction of arrow A, so that the focus position on the shortest distance side (the focus position where the lens 2a exists at the position indicated by the broken line). The focus position is changed from to the focus position on the infinity side (the focus position where the lens 2a exists at the position indicated by the alternate long and short dash line). Along with this, an electric signal from the light receiving element 6b is sent to the CPU 8
Are sequentially output to.

At this time, an arbitrary one point can be designated as the near side of the focusing range and an arbitrary one point can be designated as the far side. For example, when photographing indoors, the focusing range is designated as a relatively narrow range. On the other hand, when shooting outdoors, the focus range is specified as a relatively wide range. In this case, as an example, the designated position on the short distance side is designated as the focal position on the shortest distance side of the optical system device 2, and the designated position on the far distance side is designated as the focal position on the infinity side of the optical system device 2. . Then C
The PU 8 receives the predetermined number of electric signals from the light receiving element 6b (the lens 2 moves to the focal position where the position P1 is in focus).
The shutter mechanism 4 is driven at a shutter speed of, for example, 1/125 second, and the subject image is incident on the CCD 3. At the same time, the CPU 8 generates image data Dg1 based on the electric signal output from the CCD 3 and stores it in the RAM 11, and also generates focus position data Dl1 for specifying the position of the lens 2a and stores it in the RAM 11.

Next, the CPU 8 receives a predetermined number of electric signals from the light receiving element 6b (when the lens 2 is positioned at the focal position where the positions P2 to P8 are in focus).
As a predetermined timing, the shutter mechanism 4 is driven, image data Dg2 to Dg8 are generated based on an electric signal output from the CCD at that time, and each image data Dg2 to Dg is generated.
Focus position data D12 to D18 corresponding to g8
Are sequentially generated and stored in the RAM 11. As a result, the RAM 11 stores eight image data D
g1 to Dg8 (hereinafter, also referred to as “image data Dg” when not distinguished) and corresponding focus position data D11 to D18 (hereinafter, also referred to as “focus position data Dl” when not distinguished) are stored. in this case,
Lens 2a at the time of generating image data Dg1 to Dg8
For each position (focus position) of, the focus position of the image data Dg to be captured next is the image data Dg generated immediately before.
Is defined to be included within the depth of field of the optical system device 2 at the focus position at the time of shooting. Therefore, when it becomes necessary to change the shutter speed according to the amount of light at the time of shooting, the number of image data Dg to be generated is appropriately changed based on the depth of field of the optical system device 2 according to the shutter speed. In addition, this digital still camera 1
Then, the generation of one image data Dg is completed in about 1/10 second. Therefore, for example, since the shooting of the eight image data Dg is completed within one second, it is possible to generate the image data Dg1 to Dg8 even for a moving subject without significantly changing the shooting position.

In this case, a general optical system device including the optical system device 2 has a characteristic that the depth of field is shallower on the near side and deeper on the far side. Also,
When the lens 2a is moved by the linear motor 5, since the lens 2a and the gauge 2b have a certain weight, the moving speed of the lens 2a gradually increases from the start of the movement.
Therefore, in this digital still camera 1, by changing the focus position from the short distance side to the long distance side,
In the initial stage where the moving speed of the lens 2a is slow, an image in which the focus position is focused on the short distance side where the depth of field is shallow is taken, and the far side where the depth of field is deep after the latter half when the moving speed of the lens 2a increases. Take an image with the focus position on. Therefore, each image corresponding to the image data Dg1 to Dg8 is
It will be configured with focus on at least one place. Actually, when the subject does not exist at the position corresponding to the focal position, the focused image information may not be included.

Specifically, as shown in FIG. 5, the position P
In the image data Dg1 and Dg2 captured at the focal positions where the focal points 1 and P2 are in focus, since the subject does not exist at the positions P1 and P2, the in-focus image information does not exist.
In FIGS. 5 to 10, 14, and 16, an in-focus object is shown by a solid line, and an out-of-focus object is shown by a broken line. Further, the greater the degree of blurring, the wider the broken line interval. In this case, as shown in FIG.
The image data Dg1 and Dg2 are composed of blurred image information of the subject (the cup 31, the house 32, the standing tree 33, and the mountain 34) existing behind the position P2. On the other hand, FIG.
As shown in, the image data Dg3 captured at the focus position where the position P3 is focused includes the image information where the front part of the cup 31 is focused because the front part of the cup 31 is present at the position P3. Consists of in this case,
The image data Dg3 is composed of blurred image information of the subject (the back side portion of the cup 31, the house 32, the standing tree 33, and the mountain 34) existing behind the position P3.

Further, as shown in FIG. 7, image data Dg4 photographed at a focus position where the position P4 is in focus.
Because the back side portion of the cup 31 exists at the position P4,
The cup 31 is configured to include the focused image information on the back side portion. In this case, the image data Dg3 has the position P
Objects existing before 4 (the front part of cup 31)
And the subject (the house 32, the standing tree 33, and the mountain 34) existing behind the position P4 are blurred. Further, as shown in FIG. 8, the image data Dg6 captured at the focal position where the position P6 is in focus is the house 3
Since 2 exists at the position P6, the image information in which the house 32 is focused is included. In this case, the image data D
g6 is the subject (cup 3
1) and a subject existing behind the position P6 (Tachiki 3
3 and mountains 34) are composed of blurred image information. Further, as shown in FIG. 9, the image data Dg7 captured at the focal position where the position P7 is in focus includes the image information in which the standing tree 33 is in focus because the standing tree 33 exists at the position P7. To be done. In this case, the image data Dg7
Is composed of blurred image information of the subject (the cup 31 and the house 32) existing before the position P7 and the subject (mountain 34) existing behind the position P7. further,
As shown in FIG. 10, in the image data Dg8 captured at the focus position where the position P8 is in focus, the peak 34 is at the position P.
8, the image information in which the mountain 34 is focused is included. In this case, the image data Dg8 is composed of blurred image information of the subject (the cup 31, the house 32, and the standing tree 33) existing before the position P8.

Next, the CPU 8 causes the image data Dg1 to Dg
An image data file Fg (see FIG. 12) is generated by performing data processing on g8. At this time, as shown in FIG. 11, the CPU 8 generates the difference data Dd by comparing the image data Dg and Dg whose focal positions are adjacent to each other. Specifically, the image data D
Difference data D by extracting the difference between g1 and Dg2
d1 is generated and stored in the RAM 11, and the image data Dg
The difference data Dd by extracting the difference between 2 and Dg3.
2 is generated and stored in the RAM 11. By sequentially performing these data processes, the differential data Dd1 to Dd7 are stored in the RAM 11. In this case, since the image data Dg and Dg whose focal positions are adjacent to each other have a strong correlation between the corresponding pixels, the difference data Dd obtained by extracting the difference between the two image data Dg and Dg is compared with the image data Dg. The amount of data is sufficiently reduced.

Next, the CPU 8 determines the number of captured images (the number of image data Dg1 to Dg8. In this case, "8").
The image number data Dc is generated based on the above, and the title data Dt is generated as the title of the image data file Fg based on the date and time data absorbed from, for example, an internal clock (not shown). Next, as shown in FIG.
The CPU 8 combines the generated image number data Dc and title data Dt with the image data Dg1, the difference data Dd1 to Dd7, the angle of view data Da, and the focus position data D11 to D18 stored in the RAM 11 to form one file. By doing so, the image data file Fg is generated. In this case, the image data file Fg includes the header information Dh, the image data Dg1 and the difference data Dd1 to Dd.
It is composed of d7. Further, the header information Dh includes title data Dt, image number data Dc, and view angle data Da.
Also, focus position data D11 to D18 and the like are recorded. After that, the CPU 8 records the generated image data file Fg in the memory 9. This completes the photographing of the subject by the digital still camera 1.

Next, the image processing program Pg for performing the image processing on the image data file Fg recorded by the digital still camera 1 will be specifically described with reference to the drawings.

The memory 9 is stored by the digital still camera 1.
As described above, the image data file Fg recorded in the image data file 8 includes eight image data (image data D
g1 and difference data Dd1 to Dd7). In this case, since the existing image processing program does not support a file format such as the image data file Fg, image processing cannot be performed. Therefore, when generating one image (composite image in the present invention) with the amount of blur arbitrarily adjusted from this image data file Fg, a dedicated image processing program Pg is used. The image processing program Pg is written so that it can be started on a personal computer by installing it in various existing personal computers. Therefore, when using the image processing program Pg, first,
The image processing program Pg is installed in the personal computer. The personal computer in which the image processing program Pg is installed is not particularly limited, but the configuration of the personal computer 20 will be described below as a specific example.

The personal computer 20 corresponds to the image processing apparatus of the present invention, and as shown in FIG.
D-ROM drive 21, HDD (hard disk device) 22, display unit 23, keyboard 24, mouse 25,
Printer 26, CPU (computer) 27, RAM2
8 and a ROM 29, the memory 9 can be read and written. CD-RO
Under the control of the CPU 27, the M drive 21 is configured to be capable of reading various data from a CD-ROM (not shown) in which the image processing program Pg is recorded. The HDD 22 stores (installs) the image processing program Pg recorded in the CD-ROM, various programs other than the image processing program Pg (for example, an operation system for using the image processing program Pg), and the image processing program. The data file Fg1 edited based on the image data file Fg by Pg is stored. The display unit 23 displays an editing screen 41 (see FIG. 14) and the like under the control of the CPU 27.
The keyboard 24 and the mouse 25 correspond to the operation unit in the present invention, and are used to operate the blur amount adjusting scroll bar 43, the subject distance adjusting scroll bar 44, and the like, as described later.

The printer 26 is, for example, 1200 dp.
Various color images can be color printed with a printing accuracy of about i, and under the control of the CPU 27, for example, an image corresponding to the data file Fg1 generated by the image processing program Pg is printed. The CPU 27 corresponds to the control unit in the present invention, various image processes based on the image processing program Pg, display control of the display unit 23, and the printer 26.
The print control of is executed. RAM 28 is CPU 27
The calculation result of is temporarily stored in the ROM 29.
The operation program of No. 7 is stored.

When the image data file Fg is image-processed by the personal computer 20, the image processing program Pg is first installed as described above. At this time, the CD-ROM in which the image processing program Pg is recorded is loaded into the CD-ROM drive 21, and then the setup program (installation program) in the CD-ROM is executed. In response, CP
U27, CD according to the description of the setup program
While copying the image processing program Pg in the ROM to the HDD 22, generate environment setting information that enables the image processing program Pg in the personal computer 20 to be recorded in the HDD 22. This completes the installation of the image processing program Pg. In this case, the image processing program Pg restores the image data Dg2 to Dg8 from the image data Dg1 and the difference data Dd1 to Dd7 in the image data file Fg, and the image data Dg.
It is described that the personal computer 20 and the like execute the image generation processing based on the image information included in each of 1 to Dg8.

Next, the memory 9 removed from the digital still camera 1 is attached to the personal computer 20, and the image processing program Pg is activated. At this time, as shown in FIG. 14, the editing screen 4 is displayed on the display unit 23.
1 is displayed. On the editing screen 41, an image display section 42 corresponding to an edited image display area in the present invention for confirming an image being edited, and an image display section 42 corresponding to a focusing range input area in the present invention are displayed. Blur amount adjusting scroll bar 43 for adjusting the blur amount of the displayed image
And a subject distance adjusting scroll bar 44 for selecting a distance to a focused subject corresponding to the focus position input area in the present invention, and an image data file corresponding to the image displayed on the image display unit 42. It is composed of a decision button 45 for saving and ending editing. In this case, as for the operation of the blur amount adjusting scroll bar 43, the subject distance adjusting scroll bar 44, and the enter button 45, both the operation using the keyboard 24 and the pointing device such as the mouse 25 are performed. Can be appropriately selected. Further, the adjustment of the blur amount and the subject distance is not limited to the scroll operation of the blur amount adjusting scroll bar 43 and the subject distance adjusting scroll bar 44, and a numerical value can be input through the keyboard 24. Further, regarding adjustment of the subject distance (that is, the position where the focus is achieved), the position where the focus is achieved by clicking the mouse 25 with the pointer (not shown) moved to an arbitrary position on the image display unit 42. Can also be determined.

Next, the keyboard 24 or the mouse 25
The image data file Fg is read from the memory 9 by operating. At this time, the CPU 27 causes the image data file F to follow the description of the image processing program Pg.
image data Dg1 in g and difference data Dd1 to Dd
The image data Dg2 to Dg8 are restored from No. 7 and stored in the RAM 28 together with the image data Dg1. Also, CPU2
Reference numeral 7 indicates the RA of the angle-of-view data Da, the number-of-images data Dc, the focus position data Dl, etc. in the image data file Fg.
Store in M28. Further, the CPU 27 has the RAM 2
8, the images corresponding to the image data Dg1 to Dg8 are virtually overlapped on the basis of the focus position data Dl1 to Dl8 stored in the RAM 28 to be expanded in a predetermined area in the RAM 28, and the overlapping positions of the respective images in the expanded state 3D image data including is generated. The three-dimensional image data is used when the CPU 27 specifies an image corresponding to the operation position when the blur amount adjusting scroll bar 43 or the subject distance adjusting scroll bar 44 described below is operated, for example. It

Immediately after reading the image data file Fg by the image processing program Pg, for example, an image based on the image data Dg1 (the image shown in FIG. 5) is displayed on the image display section 42. Next, for example, when generating a portrait image in which the entire cup 31 is in focus, the operator displays an image in which the front part of the cup 31 is in focus (the image shown in FIG. 6) on the image display unit 42. As described above, the subject distance adjusting scroll bar 44 is scrolled. At this time, the CPU 27 selects the image data Dg (in this case, the image data Dg3) matching the operation position of the subject distance adjusting scroll bar 44, and the image corresponding to the image data Dg (hereinafter referred to as “reference image”).
(Also called) is displayed on the image display unit 42. in this case,
In this image processing program Pg, in addition to the slide operation of the subject distance adjusting scroll bar 44, the mouse 25
By operating to move the pointer to an arbitrary position in the image display unit 42 and clicking, it is possible to specify that the position where the pointer was displayed is focused. At this time, the CPU 27 identifies an image in which the portion where the pointer is displayed when the mouse 25 is clicked is in focus, and displays the image on the image display unit 42 as a reference image.

Further, in this image processing program Pg,
If neither the image data Dg that matches the focal position corresponding to the specified subject distance nor the image data Dg that is in focus at the position specified by the operation of the mouse 25 exist, the specified subject distance The image of the focus position corresponding to is generated. Specifically, CPU2
When the desired image data Dg does not exist, 7 is two image data D that are closest to each other on the short distance side and the long distance side sandwiching the object distance designated by the image data Dg.
By specifying g and Dg and complementing the specified image data Dg and Dg, an image at the focal position corresponding to the specified subject distance is generated and displayed as a reference image.
In this case, as an example of the interpolation processing, an image matching the corresponding focus position is generated by performing spline interpolation on the pixel data corresponding to the pixel at the same position of both image data Dg, Dg.

Next, in order to focus on the back side of the cup 31 in addition to the front side, the blur amount adjusting scroll bar 43 is slid rightward. In this case, the blur amount adjusting scroll bar 43 is provided with an aperture value that is generally used in optical cameras. The larger the numerical value of this aperture value, the deeper the depth of field and the wider the focusing range. Therefore, by sliding the blur amount adjusting scroll bar 43 using this aperture value as a guide, the blur amount of the image on the image display unit 42 can be adjusted as if the aperture value of the optical camera was changed. On the other hand, when the predetermined blur amount is input by the sliding operation of the blur amount adjusting scroll bar 43, the CPU 27
First, image information in a focused range is extracted from the image data Dg3 of the image displayed as the reference image.

In this case, as shown in FIG. 15, for example, the color information for any one pixel in the front portion of the cup 31 is calculated in each of the image data Dg1 to Dg8.
The values are different from each other. Specifically, for example, "R
The total value of the respective values of "GB (Red, Green, Blue)" indicates the largest value or the smallest value in the focused image data Dg3. Therefore, the CPU 27 determines that RA
The image data Dg in which the focus positions are adjacent to all the pixels of the image (image data Dg3 in this case) corresponding to the reference image in the three-dimensional image data developed in M28.
2, by comparing the color information with the corresponding pixels of Dg4, the pixel portion having the largest or smallest total value of “RGB” in the pixels of the image data Dg3 is determined as the focused portion. Specify the focal range.

Next, for example, when the house 32, the tree 33 and the mountain 34 are blurred with the entire cup 31 in focus, the blur amount adjusting scroll bar 43 is set to "2.8". Adjust to the part. At this time, in the image processing program Pg, when the designated blur amount is larger than the blur amount of the image displayed on the image display section 42, a plurality of image data Dg, Dg ... An image with a blur amount is generated. For example, in the image data Dg3, although the front part of the cup 31 is in focus,
The back part is out of focus. On the other hand, in the image data Dg4, the back portion of the cup 31 is in focus, but the front portion thereof is out of focus. Therefore,
The image data Dg that corresponds to the specified blur amount
Image data Dg3 extracted from No. 4 and image data Dg3 that has already been determined (in-focus range) and image data Dg3.
The image range extracted from 4 is combined. Similarly, an image range that matches the blur amount corresponding to the operation position of the blur amount adjusting scroll bar 43 is extracted from the image data Dg5 and is combined with the already combined image range. In this way, by extracting the combination range from each of the image data Dg4 to Dg8 and sequentially executing the combination process on the already combined image range, as shown in the image display unit 42 of FIG. An image focused on is generated.

After that, the image of the desired blur amount is displayed on the image display unit 4.
2 is displayed, the operator selects the enter button 45.
To operate. At this time, the CPU 27 causes the image display unit 42 to
Data file Fg1 corresponding to the image displayed in
And a screen for inputting a recording medium (drive name or folder name) for saving the data file Fg1 and a file name at the time of saving is displayed on the display unit 23. Next, by inputting a file name or the like on this screen and operating the execute button, the generation of the data file Fg1 having the desired blur amount based on the image data file Fg recorded by the digital still camera 1 is completed, and for example, the HDD 22 Is recorded with a predetermined file name. Further, when printing an image based on the generated data file Fg1, the image data of the data file Fg1 is output to the printer 26 by the print function of the image processing program Pg or the existing image processing program. As a result, the image corresponding to the data file Fg1 is color printed by the printer 26.

As described above, according to the personal computer 20 and the image processing program Pg, the image processing program Pg is installed in the personal computer 20 and executed, whereby the CPU 27 corresponds to each image data Dg, Dg. Among the images to be displayed, the image that matches the focus position information (the focus position corresponding to the operation position of the subject distance adjustment scroll bar 44) input via the keyboard 24 or the mouse 25 is used as the reference image on the editing screen 41. The image is displayed on the image display unit 42, and the keyboard 2
4 or corresponding to a predetermined image range from each image data Dg, Dg ... In accordance with the focus range information (focus range corresponding to the operation position of the blur amount adjusting scroll bar 43) input through the mouse 25. The image data file F generated by the digital still camera 1 is extracted by extracting the image information to be combined with the reference image and adjusting the focus range.
Even if there is no image data Dg for the image in the desired focus range (blur amount) in g, it is possible to generate an image in which the focus range is arbitrarily adjusted.

According to the personal computer 20 and the image processing program Pg, the image data D
When the image at the desired focus position does not exist in the images corresponding to g, Dg, ..., The CPU 27 specifies the two images at the positions sandwiching the desired focus position, and then complements the image. By executing the above step to generate the reference image, it is possible to reliably generate an image in the focusing range desired by the operator. Further, after the CPU 27 determines the focus range in the reference image as a part of the image to be generated, the focus position is next to the reference image in accordance with the focus range corresponding to the operation position of the blur amount adjusting scroll bar 43. By extracting a predetermined image range from a long-distance image in order, combining it with the reference image, and confirming the combined range as a part of the image to be generated sequentially, the image with the focal position is from the far-distance side Unlike the method of determining the image range, the desired focusing range can be accurately determined. Further, the image display unit 42, the blur amount adjusting scroll bar 43, and the subject distance adjusting scroll bar 44 are displayed on the editing screen 41, and the blur amount adjusting scroll bar 43 and the subject distance adjusting unit are referred to while referring to the image display unit 42. By configuring the personal computer 20 and the image processing program Pg so that the processing content can be input by operating the operation scroll bar 44, even an operator unfamiliar with the operation can specify a desired focus range and obtain a desired blur amount. The image of can be easily generated.

Furthermore, by describing the image processing program Pg so that the focus position information can be input by inputting the subject distance and the focus range information can be input by specifying the aperture value, the image pickup apparatus (still camera etc.) For those who are accustomed to the operation of, it is possible to generate an image with a desired blur amount at intervals at which the imaging device is operated. Further, according to the personal computer 20 and the image processing program Pg, the images corresponding to the image data Dg1 to Dg8 are virtually overlapped and developed based on the focus position data D11 to D18, and the developed images are displayed. Position information and image data Dg, Dg
By generating three-dimensional image data including .and using it for image processing, each image data Dg, Dg ..
Image data Dg1 and difference data D
The processing time can be shortened as compared with the method of restoring the image data Dg2 to Dg8 based on d. Further, according to the image processing program Pg, an installation program for installing the image processing program Pg is included in the CD-ROM and recorded in the CD-ROM, so that the image processing program Pg can be installed in various personal computers including the personal computer 20 and have a favorite blur. An amount of images can be generated.

The present invention is not limited to the above-mentioned embodiments of the present invention. For example, in the embodiment of the present invention, an example in which the image data file Fg photographed by the digital still camera 1 is set as an editing target has been described, but each digital image data in the present invention is not limited to this, and for example, conventional It is possible to edit a plurality of image data obtained by changing the focus position of a stationary subject using the digital still camera described above, and image data drawn using image creating software. In this case, by providing the digital still camera 1 with a function capable of executing the image processing program Pg shown in the embodiment of the present invention, the digital image data image-processed in the digital still camera 1 is recorded in the memory 9. Good.

[0066]

As described above, according to the image processing apparatus of the present invention, the control unit performs the editing operation with the image matching the focus position information among the images corresponding to each digital image data as the reference image. By adjusting the focus range by displaying it on the screen and extracting the image information corresponding to the predetermined image range from each digital image data according to the focus range information and combining it with the reference image, for example, a digital still camera It is possible to easily create an image with a desired blur amount after the completion of shooting by.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a digital still camera 1 according to an embodiment of the present invention.

FIG. 2 is a configuration diagram showing configurations of an optical system device 2 and a position detection unit 6.

FIG. 3 is a characteristic diagram of the optical system device 2 showing the relationship between the position of the lens 2a and the focal position.

FIG. 4 is an explanatory diagram for describing a positional relationship between the digital still camera 1 and a subject and a focal position.

FIG. 5 is an image diagram showing an image based on image data Dg1 and Dg2.

FIG. 6 is an image diagram showing an image based on image data Dg3.

FIG. 7 is an image diagram showing an image based on image data Dg4.

FIG. 8 is an image diagram showing an image based on image data Dg6.

FIG. 9 is an image diagram showing an image based on image data Dg7.

FIG. 10 is an image diagram showing an image based on image data Dg8.

FIG. 11 is an explanatory diagram illustrating a process of generating difference data Dd1 to Dd7 based on image data Dg1 to Dg8.

FIG. 12 is a data structure diagram showing an example of a data structure of an image data file Fg.

FIG. 13 is a block diagram showing a configuration of a personal computer 20 according to the embodiment of the present invention.

FIG. 14 is a display screen diagram showing an example of an editing screen 41 displayed according to an image processing program Pg.

FIG. 15 is an explanatory diagram for explaining a value of color information regarding each pixel corresponding to the same position in image data Dg1 to Dg8.

FIG. 16 is another display screen diagram showing an example of the editing screen 41.

[Explanation of symbols]

1 Digital still camera 2 Optical system device 9 memory 20 personal computer 21 CD-ROM drive 22 HDD 23 Display 24 keyboard 25 mice 26 Printer 27 CPU 28 RAM 29 ROM 31 cups 32 houses 33 Tachiki 34 mountains 41 Editing screen 42 Image display section 43 Scroll bar for blur amount adjustment 44 Scroll bar for adjusting subject distance 45 decision button Da angle of view data Dc image number data Dd difference data Dd1 to Dd7 difference data Dg1 to Dg8 image data D11 to D18 focus position data Fg image data file Fg1 data file Pg image processing program

Front page continuation (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 5/232 H04N 5/232 A 5C076 5/76 5/76 EZ 5/91 5/91 J HF term (reference) ) 5B050 AA09 BA04 BA10 BA15 CA07 EA03 EA16 EA17 EA19 FA02 5B057 BA02 BA23 CA12 CB12 CB13 CE04 CE08 CE09 CH01 CH08 CH12 DC02 5C022 AA13 AB21 AB68 AC69 5C052 AA12 CC01 CC10 CC20 DD04 FA03 FC06 AFD12 FA01 GA013C02 FA01 GA013 CA10

Claims (13)

[Claims] 1. Different Ns for the same subject
(N is a natural number of 2 or more) By executing a predetermined image processing on each digital image data obtained by capturing the subject image at the focal position of a point, a combined image in which the focusing range of the subject is arbitrarily adjusted is obtained. An image processing apparatus configured to be able to generate, a display unit for displaying an editing work screen, an operation unit for inputting processing contents, and a display unit for displaying the editing work screen on the display unit. And a control unit that generates the composite image by performing the predetermined image processing on each of the digital image data according to the processing content input via the control unit. Of the corresponding images, an image that matches the focus position information input through the operation unit is displayed on the editing work screen as a reference image, and the image is displayed through the operation unit. An image processing apparatus that adjusts the focusing range by extracting image information corresponding to a predetermined image range from each of the digital image data according to the focusing range information input as described above and combining the image information with the reference image. 2. The control unit, when there is no image matching the focus position information among the images corresponding to the digital image data, the focus position corresponds to the focus position information. The image processing apparatus according to claim 1, wherein the reference image is generated by identifying two images at positions sandwiching the focal position and performing a complementing process on the two images. 3. The control unit determines an in-focus range in the reference image as a part of the composite image, and then, in accordance with the in-focus range information, an image in which a focus position is a long distance next to the reference image. 3. The image processing apparatus according to claim 1, wherein the predetermined image range in the image is sequentially extracted from, and is combined with the reference image, and the combined range is determined as a part of the combined image. 4. The control unit, an edit image display area for displaying the reference image being edited, a focus position input area for inputting the focus position information via the operation unit, and the operation unit. While displaying the editing work screen including the focusing range input area for inputting the focusing range information via the display unit, predetermined focus position information is displayed in the focus position input area. When input, the digital image data that matches the focus position information is selected and displayed as the reference image in the edited image display area, and predetermined focus range information is input in the focus range input area. 4. When any of the above-mentioned is performed, the image information corresponding to the predetermined image range is extracted from each of the digital image data according to the focus range information and is combined with the reference image. The image processing device described. 5. The focus range input area is configured to be able to input the focus position information by inputting the subject distance information, and the control unit is configured to set the focus position corresponding to the input subject distance information. The image processing apparatus according to claim 4, wherein an image to be displayed as the reference image is specified based on information. 6. The focus range information is input to the focus range input area by either an aperture value designation simulating the aperture value designation in a photographing device or a blur amount designation that designates a blur amount of an image. The image processing apparatus according to claim 4, which is configured to be capable. 7. The control unit virtually superimposes and develops the respective images corresponding to the respective digital image data on the basis of a focus position corresponding to the respective digital image data, and develops the images in the developed state. 7. The three-dimensional image data including the overlay position information of each image and each digital image data is generated, and the predetermined image processing is executed using the generated three-dimensional image data. The image processing device described. 8. The digital image data corresponding to the L-th focal point (L is a natural number of 2 or more and N or less) is (L-1) th digital image data. The image processing apparatus according to claim 1, wherein the image processing apparatus includes differential data obtained by extracting a difference between the image data and the L-th captured digital image data. 9. The image processing apparatus according to claim 1, wherein each of the digital image data is recorded in a single data file format in which each of the digital image data is made into one file. 10. The digital image data includes generation order data for specifying a generation order thereof, focus position data for specifying the focus position, and a subject for specifying a distance to a focused subject. The control unit is configured with a data structure including at least one of the distance data, and the control unit includes the focus corresponding to each of the digital image data based on any one of the generation order data, the focus position data, and the subject distance data. The image processing device according to claim 1, wherein the position is specified. 11. The digital image data according to claim 1, wherein each of the digital image data has a data structure including field angle data for specifying a field angle of a photographing device when the digital image data is generated. The image processing device according to claim 1. 12. The image processing apparatus according to claim 1, further comprising a printing device that prints the composite image. 13. An image in which the image processing apparatus according to any one of claims 1 to 12 is caused to execute the image processing to generate the composite image in which the focus range is arbitrarily adjusted. A processing program, wherein the image processing program causes a computer as the control unit to identify an image that matches the focus position information input via the operation unit, and The image information displayed on the editing work screen in the display unit as the reference image, and the image information corresponding to the predetermined image range from each digital image data according to the focus range information input via the operation unit. An image processing program that adjusts the focusing range by extracting the image and combining the extracted image with the reference image.