KR100781552B1 - Apparatus and method for recovering high pixel image processing - Google Patents

Abstract

An apparatus and a method for high resolution image recovery are provided to use four of 1 million pixel images of an image sensor to restore an image having a resolution equal to an image of 4 million pixel resolution. A camera module(301) includes plural lenses and sub image sensors corresponding to the plural lenses. A color filter is included in the sub image sensors and formed in signal color which is not redundant. Original images by each color are obtained in each sub image sensor. An original image generating module(302a) receives plural original images obtained by colors. An intermediate image generation module(302b) rearranges pixel information of pixels at the same positions at the plural inputted original images and generates an intermediate image having a higher resolution than the original image obtained by colors. A final image generating module(302c) performs demosaicing of the generated intermediate image, performs deblurring of the demosaiced intermediate image, and generates a final image.

Description

Apparatus and method for recovering high pixel image processing

1 is a diagram schematically illustrating the principle of a digital camera embedded in a conventional small digital device.

Figure 2a is a diagram showing the basic structure of a conventional digital camera.

FIG. 2B is a cross-sectional view of a unit pixel constituting the image sensor illustrated in FIG. 2A.

3 is a block diagram illustrating a structure of a high resolution image reconstruction apparatus 300 according to an embodiment of the present invention.

4A to 4C illustrate the structure of a digital camera according to an embodiment of the present invention.

4C is a cross-sectional view of a unit pixel constituting the image sensor illustrated in FIG. 4A.

5A to 5B are color filter coating methods according to an embodiment of the present invention.

FIG. 6 is a diagram illustrating a method of original image position correction and sensitivity non-uniformity correction of the original image generation module 302a.

7A to 7B illustrate an intermediate image generating method according to an exemplary embodiment of the present invention.

8 is a diagram illustrating a high resolution image reconstruction method through the image generation module 302 according to another embodiment of the present invention.

9 is a diagram illustrating the structure of a camera module according to a second embodiment of the present invention.

FIG. 10 is a flowchart illustrating a high resolution image reconstruction method using the camera module having the structure illustrated in FIG. 4A and the first image generating module described with reference to FIG. 7B.

FIG. 11 is a flowchart illustrating a high resolution image reconstruction method using the camera module having the structure illustrated in FIG. 4A and the second image generating module described with reference to FIG. 8.

<Explanation of symbols on main parts of the drawings>

401a to 401d: first lens to fourth lens

402a to 402d: sub-image sensors matched with the first to fourth lenses

402: an image sensor in which four sub image sensors are arranged

402d-1 to 402d-4: Unit pixel constituting the sub image sensor

The present invention relates to a method and method for restoring a high resolution image, and more particularly, to an apparatus for restoring an image obtained through a miniaturized digital camera module mounted on a small digital device such as a mobile phone, a PDA, or an MP3 player, to a high resolution image; It is about a method.

Like most digital devices, digital cameras have shown a new world for many people.

Unlike film cameras, digital cameras can take pictures just as easily as professionals, and can instantly check photos taken in the field without going through development and printing. .

In addition, as digital cameras become smaller and more portable, and built into small digital devices such as mobile phones, PDAs, and MP3 players, taking pictures and enjoying them in everyday life is a natural thing. In selecting and purchasing digital cameras, built-in digital cameras have become an important factor.

In recent years, digital devices are becoming smaller and smaller, and consumers are demanding smaller and thinner digital device products than they are today.

As a result, it is obvious that in order for a small digital device with a built-in digital camera to be more compact and slimmer than the present, the built-in digital camera should be miniaturized and slimmed.

1 is a diagram schematically illustrating the principle of a digital camera embedded in a conventional small digital device.

An image of the object 101 taken by the user is formed in the image sensors 102a and 102b of the corresponding digital camera through the lenses 101a and 101b having different diameters Da and Db. .

In this case, when the diameter Db of the lens 101b is large, the resolution is excellent, but the focal length fb for forming the image of the subject (image B) becomes long, so that the height of the digital camera module to be embedded in the small digital device becomes longer. do.

Therefore, there is a difficulty in miniaturization and slimming due to the large lens and longer focal length.

On the contrary, if the diameter Da of the lens 101a is relatively small, the focal length fa for forming the image (image A) of the subject 101 is also reduced, which may be suitable for miniaturization and slimming, but it is most important for the digital camera. As the resolution, which is a factor, is lowered in proportion to the diameter of the lens, it is far from the needs of consumers who want clear high resolution pictures.

In this regard, various inventions (eg, Korean Laid-Open Patent Publication No. 2003-0084343, 'Method for manufacturing a CMOS image sensor for securing a focal length') have been proposed to obtain a high resolution image at the same time as the focal length is reduced. It is not solved.

An object of the present invention is to reduce the size and height of a digital camera embedded in a small digital device, thereby contributing to miniaturization and slimming of the small digital device in which the digital camera is embedded.

Another object of the present invention is to provide a high resolution image while reducing the size and height of a digital camera embedded in a small digital device.

Still another object of the present invention is to easily correct optical inconsistencies and sensitivity unevenness in representing high resolution images.

Still another object of the present invention is to simultaneously realize a high sensitivity sensing region and a low sensitivity sensing region without changing the structure of the image sensor by using color filters having different transmittances.

Still another object of the present invention is to increase the degree of freedom in designing a small digital device in which a digital camera is embedded.

The objects of the present invention are not limited to the above-mentioned objects, and other objects that are not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, the high resolution image reconstruction device according to an embodiment of the present invention includes a plurality of lenses and sub-image sensors corresponding to the plurality of lenses, respectively, and color filters included in the sub-image sensors overlap each other. A camera module which is formed of one single color which does not have a single color, and acquires a source image for each color from each sub-image sensor, a raw image generation module for receiving a plurality of original images acquired for each color, and the received plurality An intermediate image generation module for rearranging pixel information of pixels of the same position in the original image of the intermediate image to generate an intermediate image having a higher resolution than the original image acquired for each color, and demosaicing the generated intermediate image Final deblur of the kingized image to produce the final image Includes an image generation module.

In order to achieve the above object, a high resolution image reconstruction device according to another embodiment of the present invention includes a plurality of lenses and a sub image sensor corresponding to each of the plurality of lenses, the color filters included in the sub image sensor A camera module which is formed of one single color that does not overlap and obtains an original image for each color in each sub-image sensor, and an original image for dividing the original image obtained for each color into a plurality of pixel groups. An intermediate image generation module and an intermediate image interpolating the intermediate image to generate an intermediate image by mapping pixel information of pixels having the same position in the original image divided into the plurality of pixel groups to predetermined pixels of the pixel group corresponding to the position A final image generation model that restores and deblurs based on the algorithm It includes.

In order to achieve the above object, a high resolution image reconstructing apparatus according to another embodiment of the present invention includes a plurality of color lenses and a sub image sensor corresponding to each of the plurality of color lenses, wherein the color lenses do not overlap each other. A camera module which is formed of one single color and obtains an original image for each color from each of the sub-image sensors, an original image generation module that receives a plurality of original images obtained for each color, and the plurality of input circles An intermediate image generation module for rearranging pixel information of pixels of the same position in an image to generate an intermediate image having a higher resolution than the original image acquired for each color, and demosaicing the generated intermediate image and performing the demosaicing Create final image by deblurring the image to produce the final image It includes module.

In order to achieve the above object, a high resolution image reconstructing apparatus according to another embodiment of the present invention includes a plurality of color lenses and a sub image sensor corresponding to each of the plurality of color lenses, wherein the color lenses do not overlap each other. A camera module which is formed of one single color and obtains an original image for each color in each sub-image sensor, and an original image generation module for dividing the original image obtained for each color into a plurality of pixel groups, Based on the intermediate image generation module and an intermediate image interpolation algorithm that generates an intermediate image by mapping the pixel information of the pixels of the same position in the original image divided into a plurality of pixel groups to a predetermined pixel of the pixel group corresponding to the position It includes a final image generation module to restore and deblur.

In addition, in order to achieve the above object, the high-resolution image reconstruction method according to an embodiment of the present invention includes a plurality of lenses and a sub image sensor corresponding to each of the plurality of lenses, the color filter included in the sub image sensor Obtaining a source image for each color from each of the sub-image sensors by forming a single color that does not overlap each other, generating an original image for receiving a plurality of original images acquired for each color, and inputting the input image An intermediate image generation step of rearranging pixel information of pixels having the same position in a plurality of original images received to generate an intermediate image having a higher resolution than the original image acquired for each color, and demosaicing the generated intermediate image The final deblur of the demosaiced image to produce the final image It includes image generating step.

In order to achieve the above object, a high resolution image reconstruction method according to another embodiment of the present invention includes a plurality of lenses and a sub image sensor corresponding to each of the plurality of lenses, the color filters included in the sub image sensor Obtaining an original image for each color from each of the sub-image sensors by forming one single color that does not overlap, and generating an original image for dividing the original image obtained for each color into a plurality of pixel groups An intermediate image generation step of generating an intermediate image by mapping pixel information of pixels of the same position in the original image divided into the plurality of pixel groups to a predetermined pixel of the pixel group corresponding to the position and an interpolation algorithm of the intermediate image The final image creation step of restoring The.

In order to achieve the above object, a high resolution image reconstruction method according to another embodiment of the present invention includes a plurality of color lenses and a sub image sensor corresponding to each of the plurality of color lenses, wherein the color lenses do not overlap each other. Forming a single color to obtain a source image for each color in each of the sub-image sensors, generating a circle image for receiving a plurality of circle images obtained for each color, and receiving the plurality of circle images An intermediate image generation step of rearranging pixel information of pixels having the same position in to generate an intermediate image having a higher resolution than the original image acquired for each color, and demosaicing the generated intermediate image and performing the demosaicing image To create the final image by deblurring It should.

In order to achieve the above object, a high resolution image reconstruction method according to another embodiment of the present invention includes a plurality of color lenses and a sub image sensor corresponding to each of the plurality of color lenses, wherein the color lenses do not overlap each other. Obtaining a source image for each color from each sub-image sensor by forming a single color, and generating an original image for dividing the original image obtained for each color into a plurality of pixel groups, respectively, An intermediate image generation step of generating an intermediate image by mapping pixel information of pixels of the same position in the original image divided into pixel groups of to a predetermined pixel of the pixel group corresponding to the position, and based on the interpolation algorithm Reconstructing and deblurring the final image generation step.

Specific details of other embodiments are included in the detailed description and the drawings.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be embodied in various different forms, and the present embodiments merely make the disclosure of the present invention complete, and those of ordinary skill in the art to which the present invention belongs. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In general, the digital camera can estimate the performance of the digital camera through the pixel (Pixel). The higher the number of pixels, the sharper the image.

In addition to the number of pixels, the brightness of a lens can be used to determine the performance of a digital camera. The brightness of a lens is called an F number or an aperture value.

The F number represents the amount of light per unit area that reaches the image sensor of the digital camera.

f(초점거리)/D(렌즈의 직경) F number =

f (focal length) / D (lens diameter)

That is, it can be expressed as the ratio of the focal length of the lens and the lens diameter.

The larger the F number, the lower the amount of light per unit area that reaches the image sensor of the digital camera. The lower the F number, the greater the amount of light per unit area that reaches the image sensor of the digital camera, resulting in a brighter and higher resolution image.

Therefore, the size of the F number is closely related to the amount of light reaching the image sensor of the digital camera and the resolution of the image obtained through the digital camera.

If the F numbers are the same in two digital cameras with different lens size, focal length, and number of pixels, the same brightness can be obtained because the lens size, focal length, and number of pixels are different, but the amount of light is the same.

The present invention intends to propose a high resolution image reconstruction method applicable to a digital camera module including a plurality of lenses capable of expressing a high resolution image while reducing the diameter and the focal length of the lens.

Figure 2a is a diagram showing the basic structure of a conventional digital camera.

The basic structure of a conventional digital camera is one lens 201 having a diameter D2 for collecting light reflected from a subject and a pixel unit for generating an electric image signal in response to the light collected by the lens 201. Level image sensor 202.

The image sensor 202 includes a Bayer Patten color filter that implements the light received by the lens 201 in its original natural color, and the image sensor 202 shown in FIG. 2A is viewed from above. .

Bayer pattern is the most important basic principle of digital image since the 1970's presentation. The beginning of the theory starts from the fact that the actual image existing in the natural world is not composed of dots, but digital images can only be realized as dots.

In order to create an image composed of dots by collecting the brightness and color of the object, the points receiving the brightness of each of red (R), green (G) and blue (B) are placed on a two-dimensional plane.

When placing the dots, 50% of the green (G), which is most sensitive to the human eye, and 25% of the red (R) and blue (B) are allocated to form a grid, which is called a Bayer pattern color filter.

In the Bayer pattern color filter, each pixel forming the grid does not recognize the total natural color but only the assigned color among red (R), green (G), and blue (B), and later infers the natural color by interpolating it. Serve

FIG. 2B is a cross-sectional view of a unit pixel constituting the image sensor illustrated in FIG. 2A.

Looking at the cross section of the portions 202a to 202d of the unit pixels constituting the image sensor, it can be seen that the Bayer pattern color filter 203 is included in the image sensor.

3 is a block diagram illustrating a structure of a high resolution image reconstruction apparatus 300 according to an embodiment of the present invention.

The high resolution image reconstructing apparatus 300 according to an embodiment of the present invention is based on a plurality of images provided by the camera module 301 and the camera module 301 to collect incident light to generate a plurality of images separated by colors. An image generating module 302 for generating a final image and a display module 303 for displaying a final image provided by the image generating module.

The camera module 301 includes a plurality of lenses 401 and sub image sensors 402 respectively corresponding to the plurality of lenses, and the color filters included in the sub image sensors are in one single color that do not overlap each other. It is formed to obtain a source image for each color in each of the sub-image sensor.

First, a structure of a digital camera including a plurality of lenses to which a high resolution image reconstruction method according to an embodiment of the present invention is applicable will be described with reference to FIGS. 4A to 4C.

The structure of a digital camera including a plurality of lenses to which a high resolution image reconstruction method according to an embodiment of the present invention is applicable may include a plurality of lenses 401 having the same size of diameter condensing light reflected from a subject and light reflected from the subject. The sub image sensor 402 includes a plurality of sub image sensors 402 which generate an electrical image signal in response to the color filter, and the color filter divided into a plurality of color areas that implements the light condensed by the lens in the original natural color. It is included in.

For reference, FIG. 4B is a side view of the structure of the digital camera module illustrated in FIG. 4A. The focal length f3 from the plurality of lenses having the same diameter D3 to the image sensor in which the image of the subject is formed is illustrated in FIG. Assume all are the same.

Therefore, it is preferable that a plurality of lenses 401 having the same diameter are located on the same plane.

In addition, the embodiment of the present invention in a plurality of lens arrangement method is arranged in a symmetrical form up, down, left and right, but can also be arranged in a straight line of the horizontal or vertical shape, and when the number of lenses is odd radiating around one lens It is also possible to form the arrangement and in addition to the lens arrangement of the various forms not mentioned in the present invention is possible.

For reference, the plurality of lenses 401 may condense the light reflected from the subject by fixing the position of each lens or shifting the other lens by a predetermined pixel based on one lens. The movement of the lens will be described later in detail.

Hereinafter, for convenience of description, a case in which four lenses are arranged in the form of horizontal X vertical 2 X 2 will be described as an example.

As mentioned above, when two digital cameras having different lens sizes, focal lengths, and pixel numbers are the same, the F number is the same. Using the principle, in the embodiment of the present invention, the lenses 401a to 401d having a smaller size than the lens 201 shown in FIG. 2 and the image sensor 402a having fewer pixels than the image sensor 202 shown in FIG. 402d), the focal length f3 is shorter than the focal length f2 shown in FIG.

It is assumed that the number of pixels of the image sensor 202 shown in FIG. 2 is 4 million pixels, and the number of pixels of the sub image sensors 402a to 402d shown in FIG. 4A is 1 million each.

Therefore, the image sensor 402 illustrated in FIG. 4A is an image sensor in which four sub-image sensors 402a to 402d of one million pixels are arranged, and each of four lenses 401a to 401d having the same diameter D3. Generates an image of a subject in the sub image sensors 402a to 402d matched to each lens.

In addition, the color filters included in the sub image sensors 402a to 402d are divided into four parts according to the sizes of the sub image sensors 402a to 402d and coated with one single color that does not overlap each other.

The cross sections of the unit pixels 402d-1 to 4 constituting one sub image sensor 402d including the above-described color filter are shown in FIG. 4C and are all the same because they exist in the same sub image sensor 402d. The color filter 403 is included.

At this time, the color filter 403 is divided into a first filter region and a second filter region according to the transmittance of the color to be coated.

High sensitivity and low sensitivity image sensing may be realized by generating a difference in the amount of light by varying the transmittance of colors coated on the first filter area and the second filter area.

In the classification of the filter region, it is preferable to divide the color region having the highest transmittance among the color regions constituting the color filter into the second filter region, and divide the remaining portion of the color filter except the second color region into the first filter region.

For convenience of description, an embodiment will be described by using a region of a color filter divided into a plurality and a drawing symbol of each sub image sensor.

For example, each of the four divided color regions constituting the color filters included in the sub image sensors 402a to 402d shown in FIG. 4A, that is, the green (G) 402a, the red (R) 402b, When the gray (Gr) 402d region, which is the color having the highest transmittance among the blue (B) 402c and gray (Gr) 402d regions, is divided into the second filter region, the gray (Gr) 402d portion is used in the color filter. The remaining color areas other than the area, that is, the green (G) 402a, the red (R) 402b, and the blue (B) 402c areas are included in the first filter area.

In addition, colors other than yellow may be formed in the second filter area among the color areas constituting the color filter.

For example, a color region having a color of any one of white (W), yellow (Y), cyan or magenta may be formed.

However, the color formed in the second filter region is not limited to the above-described example, and any color having a higher transmittance than the color formed in the first filter region may be regarded as belonging to the scope of the present invention.

As a result, the transmittance of the color filter shown in FIG. 4A is increased in the order of blue (B), green (G), and red (R), and gray (Gr) has a higher transmittance than red (R).

As described above, when the transmittance of the color corresponding to the second filter region 402d forms a color filter having a transmittance higher than the transmittance of the color corresponding to the first filter regions 402a to 402c, it passes through each color filter region. A difference occurs in the amount of light to be made.

This means that there is a difference in the amount of light reaching the sub image sensors 402a to 402d matching the color filter area, and high sensitivity sensing and low sensitivity sensing are simultaneously implemented in each of the sub image sensors 402a to 402d. Can be.

Examples of the above-described color filter coating method include a photo-lithography method and an inkjet method, and are illustrated in FIGS. 5A and 5B.

For example, if you coat four single colors green (G), red (R), blue (B), and gray (Gr), the photolithography technique first coats the green (G) color across the image sensor ( 502) After leaving 1/4 green color coating, the remaining 3/4 green color coating is patterned 503.

Coat the red (R) color on the 3/4 image sensor stripped of the green (G) color coating (504) and leave 1/4 the red (R) color coating, then the remaining 2/4 red (R) Color coating is removed (505).

The image sensor is coated with green (G) and red (R) colors 1/4 each, and the remaining 2/4 coatings are blue (B) and gray like the green (G) and red (R) color coating methods. (Gr) can be coated (506 ~ 508).

Photolithography has an easy advantage over Bayer pattern color filter processes

5B illustrates an ink saving coating method, which is an embodiment of a second coating method.

The ink saving coating method is a method of forming a color filter by forming four dividing walls equal to the number of lenses in an image sensor (510) and coating ink of desired color in each divided quarter of space (511 to 514). ).

Ink saving coating is a very simple process and has the advantage of reducing the manufacturing cost of the sensor with the effect of saving ink.

The color filters coated in the manner described above are matched to each lens and each sub image sensor.

For example, when the green (G) color filter, the red (R) color filter, the blue (B) color filter, and the gray (Gr) color filter are respectively included in the sub image sensors 402a to 402d in FIG. The light of the subject entering through the lens 401a is formed by the green (G) color filter in the sub-image sensor 402a matched with the first lens 401a, and an image of the green (G) color is formed. The image entered through the 401b is formed by the red (R) color filter in the sub image sensor 402b matched with the second lens 401b.

In this manner, the light of the subject entering through the third lens 401c and the fourth lens 401d is an image of blue (B) color by color filters in the sub-image sensors 402c and 402d matched with each lens. And gray (Gr) color images are formed, respectively.

That is, light of a subject incident through each of the four lenses 401a to 401d has the color of the corresponding color filter through the color filters in the sub-image sensors 402a to 402d that match each of the lenses 301a to 301d. As the image is formed, four images of the same size having different colors exist in each of the sub image sensors 402a to 402d.

Meanwhile, the image generating module 302 receives a plurality of images separated for each color from the camera module 301 and generates a final image.

To this end, the image generation module 302 includes an original image generation module 302a, an intermediate image generation module 302b, and a final image generation module 302c.

In the image generation module 302 according to an embodiment of the present invention, the original image generation module 302a receives a plurality of images separated for each color provided by the camera module 301.

That is, the original image generation module 302a is obtained by the green color image obtained by the sub image sensor 402a including the green color filter shown in FIG. 4A and the sub image sensor 402b including the red color filter. The red color image, the blue color image obtained by the sub image sensor 402c including the blue color filter, and the gray color image obtained by the sub image sensor 402d including the gray color filter.

In this case, the green color image, the red color image, and the blue color image serve to provide color information necessary to generate the final image through the final image generation module 302c which will be described later.

In contrast, the gray color image serves to provide luminance information required to generate the final image.

In addition, the original image generation module 302a corrects the position of the original image when the original image acquired for each color described above is out of a predetermined position in each of the sub-image sensors 402a to 402d, and acquires the original image for each color. If the sensitivity of the nonuniformity is corrected, the sensitivity of the other original image is corrected based on the original image having the lowest sensitivity among the original images acquired for each color.

FIG. 6 is a diagram illustrating a method of original image position correction and sensitivity non-uniformity correction of the original image generation module 302a.

The light of a subject incident through each of the four fixed positions of the lenses 401a to 401d passes through the color filters in the sub-image sensors 402a to 402d that match the lenses 401a to 401d. When the image having the image is formed, the original image generating module 302a checks the position of each image in each of the sub image sensors 402a to 402d.

6 shows positions 601 to 604 of respective images formed in the respective sub image sensors 402a to 402d.

601 illustrates an image formed on the sub-image sensor 402a. In the exemplary embodiment of the present invention, the sub-image sensor 402a like 601 does not move four lenses 401a to 401d having a fixed position by a predetermined pixel. The position of the image at the center of the image is defined as the normal position, that is, the predetermined position of the image and is indicated by the dotted rectangle 605.

602 illustrates an image formed in the sub image sensor 402b and is moved by one pixel in a left direction from a predetermined position, and 603 illustrates an image formed in the sub image sensor 403b and is moved by one pixel in a downward direction from a predetermined position. have.

604 illustrates an image formed in the sub image sensor 402d and is moved by one pixel in a diagonal direction, that is, in a right direction and a downward direction, at a predetermined position.

As shown in FIG. 6, the reason that the image deviates from a predetermined position in each of the sub image sensors 402a to 402d may be caused by an optical misalignment. In general, when an optical mismatch occurs, an image formed in the sub image sensor Since the position of is out of a fixed position, it is difficult to obtain a clear image.

Thus, the original image generation module 302a of the present invention may correct the image deviated from a predetermined position due to optical mismatch in software.

For example, in the case of 602, the image is shifted by one pixel to the right since the pixel is deviated by one pixel to the left. In the case of 603, one pixel is moved upwards, and in the case of 604, it moves leftward. The position of the image due to optical inconsistency may be corrected by moving one pixel and moving one pixel upward again or one pixel upward and one pixel moved leftward.

In addition, the original image generation module 302a may correct the nonuniformity of sensitivity.

This will be described with reference to FIG. 6 again.

Assuming that the image sensitivity is 10, 602 at 601, the image sensitivity is 9, 603, the image sensitivity is 8, 604, and the image sensitivity is 7, the original image generation module 302a at 604 has the lowest sensitivity. Based on the sensitivity 7, the sensitivity of the image can be adjusted so that the sensitivity of the remaining image is 7 to correct the nonuniformity of the sensitivity.

Through the above-described process, the image for each color is provided to the intermediate image generation module 302b which will be described later.

The intermediate image generation module 302b rearranges pixel information of pixels having the same position in each color-colored image provided from the original image generation module 302a to generate an intermediate image having a higher resolution than the color-specific image.

In this case, the intermediate image is a term used to distinguish each image formed in each sub-image sensor, that is, the original image. The intermediate image rearranges pixel information of pixels having the same position in the original image, and has a higher resolution than the original image. Means image.

However, the intermediate image does not mean the final image with high resolution, and the intermediate image is made into the final image through a process such as demosaicing or deblur.

Meanwhile, the generated intermediate image preferably has the same resolution as the image sensor 402 in which the plurality of sub image sensors 402a to 402d are arranged.

For example, when each sub image sensor 402a to 402d has a resolution of 4x4, it is preferable that the intermediate image has the same resolution of 8x8 as the image sensor.

The intermediate image is provided to the final image generation module 302c.

The final image generation module 302c serves to demosaize the intermediate image provided from the intermediate image generation module 302b and to deblur the demosaiced image to generate a final image.

The display module 303 serves to display the final image provided from the final image generation module 302c.

The display module 303 may be implemented in the form of, for example, a flat panel display or a touch screen.

7A to 7B illustrate an intermediate image generating method according to an exemplary embodiment of the present invention.

Before describing the high resolution image reconstruction method with reference to the drawings, four lenses of the camera module 301 of the device 300 shown in FIG. 3 have the other lenses determined based on one lens having a fixed position. Assume that it is moved by a pixel of

Here, the movement by a predetermined pixel includes both the movement length and the movement direction, and may vary according to embodiments.

When photographing the subject 701 having four squares of the same size as shown in FIG. 7A using the camera module 301, one lens 401a is configured to photograph the square 701a engraved as A among the subjects. The position is fixed, and the other lenses 401b to 401c have predetermined positions relative to one lens 401a so as to photograph the rectangles engraved with B 701b, C 701c, and D 701d, respectively. Assume that it is moved by.

Here, the movement by the predetermined position, the lens for photographing the rectangle B (701b) will have to move to the right by a predetermined position relative to the lens for photographing the rectangle A (701a), and photograph the rectangle C (701c) The lens must move by a predetermined position in the downward direction with respect to the rectangle A 701a.

In addition, the lens photographing the quadrangle D 701d means that the lens has to move by a predetermined position in the diagonal direction or the right direction and the downward direction with respect to the rectangle A 701a.

The images formed in each of the sub image sensors 402a to 402d through the respective lenses become squares A, B, C, and D 702a to 702d, respectively.

In this case, if the pixels of the sub-image sensor matched to each lens are 1 million pixels, the images of the rectangles A, B, C, and D (702a to 702d) are arranged in the shape of the subject to form a single image. (Indicated by a + symbol in the drawing), a camera having 4 million pixels can be used to generate an image 703 having the same pixels as the image 704 of the entire subject.

7B is a diagram illustrating a method of generating an intermediate image according to an exemplary embodiment of the present invention.

Using the same principle as described with reference to FIG. 7A, one of the four lenses 401a to 401d of the camera module 301 for photographing the subject 705 fixes the position and the other lenses. Reference numerals 401b to 401d move the lenses so that the pixels are photographed from pixels shifted by one pixel based on one lens 401a.

That is, the position of the lens is shifted so that the image is taken from a pixel moved by 1 pixel to the right, 1 pixel to the bottom, 1 pixel to the diagonal direction, or 1 pixel to the right direction and the downward direction based on one lens 401a. .

Through the camera module 301 having the structure of such a lens, the intermediate image generation module 302b rearranges pixel information of pixels of the same position of the original images 706 acquired for each color to obtain higher resolution than the original image. Produces an intermediate image 708 having the same.

Final image generation module 302c Demosaices the intermediate image 708 generated in the intermediate image generation module, and deblur the demosaiced image to restore the original image to the final image with high resolution.

For convenience, the image generation module 302 described with reference to FIG. 7B is called a first image generation module.

8 is a diagram illustrating a high resolution image reconstruction method through the image generation module 302 according to another embodiment of the present invention.

Description of the camera module 301 is the same as the description of the camera module described above will be omitted.

Meanwhile, the original image generation module 302a of the image generation module 302 divides the original images acquired for each color from the camera module 301 into a plurality of pixel groups, and the intermediate image generation module 302b is illustrated in FIG. Generate a first intermediate image 802 having the same resolution as the sub image sensors 402a-402d shown in FIG.

Here, the first intermediate image 802 may be divided into a plurality of pixel groups 803 ˜ 805 each having 2 × 2 virtual pixels as a horizontal × vertical pixel shown in FIG. 8.

Here, each pixel group 803 to 805 is positioned around the main pixels 803a, 804a and 805a to which the color information and the luminance information are mapped, and the main pixels 803a, 804a and 805a, and does not have information. It may be divided into subpixels 803b, 804b, and 805b.

The positions of the main pixels 803a, 804a, and 805a may be designated at various positions in the pixel groups 803 to 805.

For example, as shown in FIG. 8, in each pixel group 803 to 805 of 2 × 2, positions 803a, 804a, and 805a corresponding to the first column of the first row may be designated as the positions of the main pixels. In each pixel group, positions 803b, 804b, and 805b corresponding to the second column of the first row may be designated as the position of the main pixel.

As such, when the first intermediate image 802 is generated, the intermediate image generation module 302b maps pixel information of pixels having the same position in each color-specific image to main pixels of the pixel group corresponding to the position.

For example, the intermediate image generation module 302b may convert pixel information of pixels located in the first row and the first column of each color-specific image to the pixel group 803 located in the first row and the first column of the first intermediate image 802. Is mapped to the main pixel 803a.

Similarly, the intermediate image generation module 802b may obtain pixel information of pixels located in the second column of the first row of each color-specific image of the pixel group 804 located in the first row and the second column of the first intermediate image 802. Map to main pixel 804a.

In addition, the intermediate image generation module 302b calculates luminance information based on color information among pixel information of pixels having the same position in each color image, and calculates the luminance information from each pixel group 803 to 805. Are mapped to the main pixels 803a to 803b.

For example, the intermediate image generation module 302b calculates luminance information based on the color information provided in the pixels located in the first row and the first column of the green, red and blue color images.

In the first intermediate image 802, luminance information calculated is mapped to main pixels of a pixel group located in a first row and a first column.

Referring to FIG. 8, the main pixels 803a to 803b of each pixel group 803 to 605 have green, red, and blue color information G, R, and B provided from the three sub-image sensors 802a to 802c, respectively. ), Gray color information Gr representing the luminance information provided from the remaining sub-image sensor 402d is mapped. Although not shown in FIG. 8, the luminance information Y detected from the three-color information is not included. May be further mapped.

For example, in the main pixel 803a of the first pixel group 803, the green color information of the pixel located in the first column of the first row of the green color image, and the pixel located in the first column of the first row of the red color image This has red color information, blue color information having pixels located in the first column of the first row in the blue color image, gray color information representing luminance information possessed by the pixels located in the first column of the first row in the gray color image, and 3 The detected luminance information is mapped based on two color information.

Similarly, the main pixel 804a of the second pixel group 804 includes the green color information of the pixel located in the second column of the first row in the green color image, and the pixel located in the second column of the first row in the red color image. Has red color information, blue color information of the pixel located in the second column of the first row in the blue color image, gray color, which is luminance information of the pixel located in the second column of the first row of the gray color image, and three provided The detected luminance information is mapped based on the color information.

Thus, the intermediate image generation module 302b generates a second intermediate image 806 in which three color information and two luminance information are mapped to the main pixels 803a to 805a of each pixel group 803 to 805.

The intermediate image generation module 302b then interpolates the second intermediate image 806 using interpolation.

That is, the intermediate image generation module 302b writes the pixels recorded in the subpixels 803b to 805b based on the information held by the main pixels 803a to 805a of each pixel group 803 to 805 shown in FIG. 8. Calculate the information.

In this case, the intermediate image generation module 302b may interpolate the second intermediate image 806 according to various algorithms.

For example, pixel information recorded in each subpixel in the second intermediate image 806 may be calculated from information of a main pixel adjacent to the corresponding subpixel.

More specifically, in FIG. 8, the pixel information recorded in the subpixel 803b positioned between the main pixel 803a of the first pixel group 803 and the main pixel 804a of the second pixel group 804 is The average value 807 of the pixel information of the two main pixels 803a and 804a may be specified.

Similarly, the pixel information recorded in the subpixel 804b located between the main pixel 804a of the second pixel group 804 and the main pixel 805a of the third pixel group 805 is two main pixels 804a and 805a. ) May be designated as an average value 808 of pixel information.

When interpolation of the second intermediate image 806 is performed by this method, the final image generation module 302c de-blurring the interpolated second intermediate image 806.

As a result, a final image 809 of high resolution (i.e., resolution x 4 of the sub-image sensor) is generated from the original image for each color of the low resolution (i.e., resolution of the sub-sensing area) obtained through each sub-sensing area.

For convenience, the image generation module 302 described with reference to FIG. 8 is called a second image generation module.

9 is a diagram showing the structure of a camera module according to a second embodiment of the present invention.

The camera module according to the second embodiment of the present invention has the same components except for the following compared to the camera module according to the first embodiment of FIG. 4A.

That is, in the camera module according to the second embodiment, the plurality of lenses 901a to 901d have different colors.

Here, the plurality of lenses 901a to 901d may be divided into a first group and a second group according to the transmittance.

In this case, it is preferable that the lens included in the second group has a color with higher transmittance than the lens included in the first group.

More specifically, the first lens 901a, the second lens 901b, and the third lens 901c corresponding to the first group of four lenses have green, red, and blue colors, respectively. The fourth lens 901d included preferably has a color having a high transmittance, for example, a gray color, as compared with the green, red, and blue colors.

As described above, when the plurality of lenses 901a to 901d have different colors, no separate color filter layer is formed on each of the sub image sensors 902a to 902d.

The image sensor 902 may be divided into sub-image sensors 902a to 902d corresponding to the respective lenses 901a to 901d, and may obtain an image separated by colors by the lenses 901a to 901d.

The image separated by color is provided with a plurality of images separated by color from the camera module 301 through the image generating module 302 illustrated in FIG. 3 to generate a final image and display the image through the display module 303. do.

The description of the image generating module 302 for generating the final image is the same as the content described with reference to FIGS. 3 to 8.

Next, a high resolution image restoration method according to an embodiment of the present invention will be described with reference to FIGS. 10 to 11.

10 is a flowchart illustrating a high resolution image reconstruction method using the camera module having the structure illustrated in FIG. 4A and the first image generation module described with reference to FIG. 7B.

First, for convenience of description, green, red, blue, and gray color filters are included in the sub image sensors 402a to 402d constituting the image sensor 402 in FIG. 7B, and the image sensor 402 is a horizontal × It is assumed that the length is composed of 8x8 pixels, and each of the sub-image sensors 402a to 402d constituting the image sensor 402 is 4x4 pixels in width x length.

Light reflected from the subject 705 is condensed through the four lenses 401a to 401d, respectively (S1001).

Light condensed through each of the lenses 401a to 401d is transmitted through the color filter included in each of the sub-image sensors 402a to 402d matching the lenses 401a to 401d (S1002).

As a result, a plurality of images separated for each color are obtained in each of the sub image sensors 402a to 402d (S1003).

In this case, the images acquired by the sub image sensors 402a to 402d have a resolution corresponding to 1/4 of the resolution of the image sensor 402.

That is, since the resolution of the image sensor 402 is 8x8, the image acquired by each sub image sensor 402a-402d has a resolution of 4x4.

After S1003, the image generation module 302 checks whether the plurality of original images 706 obtained in S1003 exist at a predetermined position in each of the sub image sensors 401a to 401d (S1004).

As a result of the check, if the deviation from the predetermined position corrects the position so that the original image is located at the predetermined position (S1005).

If the original image exists at the predetermined position, the image generation module 302 checks the sensitivity of the plurality of original images 706 (S1006).

As a result of the check, if the sensitivity is uneven, the image generating module corrects the sensitivity of the original images based on the lowest sensitivity among the original images (S1007).

If the sensitivity is constant, the image generation module 302 receives a plurality of color-specific images to generate an original image, and rearranges pixel information of pixels of the same position in the plurality of original images to achieve higher resolution than the original image for each color. An intermediate image 708 is generated (S1008).

After S1008, demosaicing the intermediate image (S1009), and debluring the demosaiced image to generate a final image (S1010).

After S1010, the final image generated by the image generating module 302 is displayed through the display module 303 (S1011).

FIG. 11 is a flowchart illustrating a high resolution image reconstruction method using the camera module having the structure illustrated in FIG. 4A and the second image generation module described with reference to FIG. 8.

First, a process of providing a plurality of original images for each color from the camera module 301 of the apparatus illustrated in FIG. 4A is the same as that of S1001 to S1007 illustrated in FIG. 10, and the original image generation module 302a illustrates the original image. Horizontal x vertical shown in 8, it is divided into a plurality of pixel groups (303 ~ 305) each of 2 x 2 virtual pixels (S1101).

After S1101, the intermediate image generation module 302b generates the first intermediate image 802 having the same resolution as the image sensor 402 shown in FIG. 8 (S1102).

When the first intermediate image 802 is generated, the intermediate image generation module 302b maps pixel information included in pixels of the same position in each color-specific image to main pixels of the pixel group corresponding to the position (S1103).

After S1103, the intermediate image generation module 302b generates a second intermediate image 806 in which three color information and two luminance information are mapped to the main pixels 803a to 805a of each pixel group 803 to 805. (S1104).

Thereafter, the intermediate image generation module 302b interpolates the second intermediate image 806 by using interpolation (S1105).

When interpolation of the second intermediate image 806 is performed by the above method, the final image generation module 302c de-blurring the interpolated second intermediate image 806 (S1106).

As a result, a final image 809 of a high resolution (i.e., resolution × 4 of the sub-image sensor) is generated from the original image for each color of the low resolution (i.e., the resolution of the sub-sensing area) obtained through each sub-sensing area, and the display module It is displayed through the 303 (S1107).

Meanwhile, the high resolution image reconstruction method using the camera module having the structure shown in FIG. 9 and the first image generating module described with reference to FIG. 7B uses light through a color lens instead of a color filter in the contents of S1001 to S1007 shown in FIG. 10. All the same except for the difference of condensing and acquiring the original image separated by color in the sub-image sensor, and the method of reconstructing the high resolution image through the second image generating module described with reference to FIG. 8 is also described in S1001 to S1007 shown in FIG. In this case, the content of condensing light through the color lens instead of the color filter and acquiring the original image separated for each color is applied to the sub image sensor, which is the same as the image restoration method illustrated in FIG. 11.

So far, the above-mentioned contents are summarized in terms of four lenses relatively smaller than the lens shown in FIG. 2, a relatively small focal length, and a relatively smaller number of pixels than the 4 million pixel image sensor shown in FIG. 2. The digital camera structure including four image sensors having 1 million pixels has the same brightness as the 4 million pixels image shown in FIG. 2 because the F numbers are the same, and is shown in FIG. 4 through a predetermined process. By using four 1 million pixel images of the image sensor, an image having a resolution equivalent to that of the 4 million pixel image illustrated in FIG. 2 may be reconstructed.

Since the structure has a relatively small lens and a small focal length, it is possible to obtain an image having the same resolution while achieving miniaturization and slimming of the digital camera.

Although embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. You will understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

According to the digital camera module of the present invention as described above has one or more of the following effects.

By reducing the size and height of the digital camera embedded in the small digital device, there is an advantage that contributes to the miniaturization and slimming of the small digital device in which the digital camera is embedded.

It also has the advantage of providing high resolution images while reducing the size and height of digital cameras embedded in small digital devices.

It also has the advantage of easily correcting optical inconsistencies and sensitivity unevenness when displaying high resolution images.

By using color filters having different transmittances, a high sensitivity sensing region and a low sensitivity sensing region can be simultaneously realized without changing the structure of the image sensor.

It also has the advantage of increasing design freedom of small digital devices with built-in digital cameras.

Claims (44)

A plurality of lenses and a sub image sensor corresponding to each of the plurality of lenses, each of the color filters included in the sub image sensor is formed of a single color that does not overlap each other to be circled for each color in each sub image sensor A camera module for acquiring an image; An original image generation module configured to receive a plurality of original images acquired for each color; An intermediate image generation module for rearranging pixel information of pixels having the same position in the plurality of input original images to generate an intermediate image having a higher resolution than the original image acquired for each color; And And a final image generating module for demosaicing the generated intermediate image and debluring the demosaiced image to generate a final image. The method of claim 1, The plurality of lenses is a high-resolution image reconstructing device for condensing the light reflected from the subject by fixing the position of each lens or the other lens relative to one lens by a predetermined pixel. The method of claim 1, And the original image generating module corrects the position of the original image when the original image acquired for each color is out of a predetermined position in each of the sub-image sensors. The method of claim 1, The original image generating module, when the sensitivity of the original image acquired for each color is non-uniform, a high-resolution image for correcting the sensitivity of the other original image based on the lowest sensitivity of the original image obtained for each color Restore device. The method of claim 1, The color filter is divided into a first filter region and a second filter region according to a color transmittance, and a high-resolution image restoration apparatus having a transmittance of a color coated on the second filter region is higher than a transmittance of a color coated on the first filter region. . A plurality of lenses and a sub image sensor corresponding to each of the plurality of lenses, each of the color filters included in the sub image sensor is formed of a single color that does not overlap each other to be circled for each color in each sub image sensor A camera module for acquiring an image; An original image generation module for dividing the original image acquired for each color into a plurality of pixel groups; An intermediate image generation module configured to generate an intermediate image by mapping pixel information of pixels having the same position in the original image divided into the plurality of pixel groups to predetermined pixels of the pixel group corresponding to the position; And And a final image generation module for reconstructing and debluring the intermediate image based on an interpolation algorithm. The method of claim 6, And the original image generating module corrects the position of the original image when the original image acquired for each color is out of a predetermined position in each of the sub-image sensors. The method of claim 6, The original image generating module, when the sensitivity of the original image acquired for each color is non-uniform, a high-resolution image for correcting the sensitivity of the other original image based on the lowest sensitivity of the original image obtained for each color Restore device. The method of claim 6, The color filter is divided into a first filter region and a second filter region according to color transmittance, and a high-resolution image reconstruction having a transmittance of a color coated on the second filter region is higher than a transmittance of a color coated on the first filter region. Device. The method of claim 6, And the pixel group comprises a plurality of pixels corresponding to the array pattern of the color filter. The method of claim 6, And the pixel information includes luminance information provided by a sub image sensor matched to the first filter area and color information provided by a sub image sensor matched to the second filter area. A plurality of color lenses and sub-image sensors respectively corresponding to the plurality of color lenses, wherein the color lenses are formed of one single color that does not overlap each other, and thus the original image for each color in each of the sub-image sensors. A camera module to obtain a; An original image generation module configured to receive a plurality of original images acquired for each color; An intermediate image generation module for rearranging pixel information of pixels having the same position in the plurality of input original images to generate an intermediate image having a higher resolution than the original image acquired for each color; And And a final image generating module for demosaicing the generated intermediate image and debluring the demosaiced image to generate a final image. The method of claim 12, The plurality of lenses is a high-resolution image reconstructing device for condensing the light reflected from the subject by fixing the position of each lens or the other lens relative to one lens by a predetermined pixel. The method of claim 12, And the original image generating module corrects the position of the original image when the original image acquired for each color is out of a predetermined position in each of the sub-image sensors. The method of claim 12, The original image generating module, when the sensitivity of the original image acquired for each color is non-uniform, a high-resolution image for correcting the sensitivity of the other original image based on the lowest sensitivity of the original image obtained for each color Restore device. The method of claim 12, The color lens is divided into a first lens region and a second lens region according to the transmittance of color, and the high-resolution image restoration apparatus having a transmittance of the color coated on the second lens region is higher than the transmittance of the color coated on the first lens region. . A plurality of color lenses and sub-image sensors respectively corresponding to the plurality of color lenses, wherein the color lenses are formed of one single color that does not overlap each other, and thus the original image for each color in each of the sub-image sensors. A camera module to obtain a; An original image generation module for dividing the original image acquired for each color into a plurality of pixel groups; An intermediate image generation module configured to generate an intermediate image by mapping pixel information of pixels having the same position in the original image divided into the plurality of pixel groups to predetermined pixels of the pixel group corresponding to the position; And And a final image generation module for reconstructing and debluring the intermediate image based on an interpolation algorithm. The method of claim 17, And the original image generating module corrects the position of the original image when the original image acquired for each color is out of a predetermined position in each of the sub-image sensors. The method of claim 17, The original image generating module, when the sensitivity of the original image acquired for each color is non-uniform, a high-resolution image for correcting the sensitivity of the other original image based on the lowest sensitivity of the original image obtained for each color Restore device. The method of claim 17, The color lens is divided into a first lens region and a second lens region according to the transmittance of color, and the high-resolution image restoration apparatus having a transmittance of the color coated on the second lens region is higher than the transmittance of the color coated on the first lens region. . The method of claim 17, The pixel group includes a plurality of pixels corresponding to the array pattern of the color lens. The method of claim 17, The pixel information may include luminance information provided by a sub image sensor matched to the first lens area and color information provided by a sub image sensor matched to the second lens area. A plurality of lenses and a sub image sensor corresponding to each of the plurality of lenses, each of the color filters included in the sub image sensor is formed of a single color that does not overlap each other to be circled for each color in each sub image sensor (源) obtaining an image; An original image generation step of receiving a plurality of original images acquired for each color; Generating an intermediate image having a higher resolution than the original image acquired for each color by rearranging pixel information of pixels of the same position in the plurality of input original images; And And a final image generation step of demosaicing the generated intermediate image and debluring the demosaiced image to generate a final image. The method of claim 23, wherein The plurality of lenses is a high-resolution image reconstructing method for condensing the light reflected from the subject by fixing the position of each lens or the other lens relative to one lens by a predetermined pixel. The method of claim 23, wherein And generating the original image by correcting the position of the original image when the original image acquired for each color is out of a predetermined position in each of the sub-image sensors. The method of claim 23, wherein In the generating of the original image, when the sensitivity of the original image acquired for each color is non-uniform, a high resolution image for correcting the sensitivity of the other original image based on the original image having the lowest sensitivity among the original images acquired for each color Restore method. The method of claim 23, wherein The color filter may be divided into a first filter region and a second filter region according to color transmittance, and the transmittance of the color coated on the second filter region is higher than the transmittance of the color coated on the first filter region. . A plurality of lenses and a sub image sensor corresponding to each of the plurality of lenses, each of the color filters included in the sub image sensor is formed of one single color that does not overlap each other to be circled for each color in each sub image sensor (源) obtaining an image; An original image generation step of dividing the original image acquired for each color into a plurality of pixel groups; Generating an intermediate image by mapping pixel information of pixels having the same position in the original image divided into the plurality of pixel groups to predetermined pixels of the pixel group corresponding to the position; And And reconstructing the intermediate image based on an interpolation algorithm to deblur the final image. The method of claim 28, And generating the original image by correcting the position of the original image when the original image acquired for each color is out of a predetermined position in each of the sub-image sensors. The method of claim 28, In the generating of the original image, when the sensitivity of the original image acquired for each color is non-uniform, a high resolution image for correcting the sensitivity of the other original image based on the original image having the lowest sensitivity among the original images acquired for each color Restore method. The method of claim 28, The color filter is divided into a first filter region and a second filter region according to color transmittance, and a high-resolution image reconstruction having a transmittance of a color coated on the second filter region is higher than a transmittance of a color coated on the first filter region. Way. The method of claim 28, And the pixel group includes a plurality of pixels corresponding to the array pattern of the color filter. The method of claim 28, The pixel information may include luminance information provided by a sub image sensor matched to the first filter area and color information provided by a sub image sensor matched to the second filter area. A plurality of color lenses and sub-image sensors respectively corresponding to the plurality of color lenses, wherein the color lenses are formed of one single color that does not overlap each other, and thus the original image for each color in each sub-image sensor Obtaining a; An original image generation step of receiving a plurality of original images acquired for each color; Generating an intermediate image having a higher resolution than the original image acquired for each color by rearranging pixel information of pixels of the same position in the plurality of input original images; And And a final image generation step of demosaicing the generated intermediate image and debluring the demosaiced image to generate a final image. The method of claim 34, The plurality of lenses is a high-resolution image reconstructing method for condensing the light reflected from the subject by fixing the position of each lens or the other lens relative to one lens by a predetermined pixel. The method of claim 34, And generating the original image by correcting the position of the original image when the original image acquired for each color is out of a predetermined position in each of the sub-image sensors. The method of claim 34, In the generating of the original image, when the sensitivity of the original image acquired for each color is non-uniform, a high resolution image for correcting the sensitivity of the other original image based on the original image having the lowest sensitivity among the original images acquired for each color Restore method. The method of claim 34, The color lens is divided into a first lens region and a second lens region according to the color transmittance, and the transmittance of the color coated on the second lens region is higher than the transmittance of the color coated on the first lens region. . A plurality of color lenses and sub-image sensors respectively corresponding to the plurality of color lenses, wherein the color lenses are formed of one single color that does not overlap each other, and thus the original image for each color in each of the sub-image sensors. Obtaining a; An original image generation step of dividing the original image acquired for each color into a plurality of pixel groups; Generating an intermediate image by mapping pixel information of pixels having the same position in the original image divided into the plurality of pixel groups to predetermined pixels of the pixel group corresponding to the position; And And reconstructing the intermediate image based on an interpolation algorithm to deblur the final image. The method of claim 39, And generating the original image by correcting the position of the original image when the original image acquired for each color is out of a predetermined position in each of the sub-image sensors. The method of claim 39, In the generating of the original image, when the sensitivity of the original image acquired for each color is non-uniform, a high resolution image for correcting the sensitivity of the other original image based on the original image having the lowest sensitivity among the original images acquired for each color Restore method. The method of claim 39, The color lens is divided into a first lens region and a second lens region according to the color transmittance, and the transmittance of the color coated on the second lens region is higher than the transmittance of the color coated on the first lens region. . The method of claim 39, And the pixel group includes a plurality of pixels corresponding to an array pattern of the color lens. The method of claim 39, The pixel information includes luminance information provided by a sub image sensor matched to the first lens area and color information provided by a sub image sensor matched to the second lens area.

Images