JP5180052B2 - Image evaluation apparatus and image evaluation program - Google Patents

Description

  The present invention relates to an image evaluation apparatus and an image evaluation program.

  When taking photos (images) with a camera, people often take multiple shutters in their favorite scenes, especially with the widespread use of digital cameras. It tends to increase. At the stage of organizing the photographs (images) thus taken, the user faces the fact that there are many similar photographs (hereinafter referred to as similar images). Work such as selecting the best image from multiple similar images and organizing it for easy viewing, or creating a digest to select and print the best image It is a very time consuming and laborious task.

As one means for solving such a situation, the similarity of a plurality of images is analyzed and divided into groups, and thumbnails are displayed based on the results, thereby shortening the user's image search time. The method is known. In addition, when creating an album by laying out a plurality of images on one page in order to make it easy to view the images, groups of similar images are grouped to form a group of keywords specified by the user. A method is known in which images similar to images are extracted by the number designated by the user, and the images are laid out on one page to constitute one page (scene data) of the album (for example, Patent Document 1). And 2). According to such a method, when a user creates an album or selects an image for printing, an image group is configured by grouping similar images. Or by picking up images with low similarity (unlike images) and presenting them to the user, image extraction for creating a digest of the image group can be assisted.
JP 2005-235546 A JP 2008-65670 A

  However, in the conventional method, it is not possible to automatically pick up an image with high importance from the entire image group and create a digest. For this reason, when performing a layout for browsing, there is a problem that it is still necessary for the user to select a highly important image and specify the number of narrowed down images.

  The present invention has been made in view of such a problem, and an image evaluation that configures album data scene data as a digest image by automatically picking up a highly important image from a plurality of stored images. An object is to provide an apparatus and an image evaluation program.

  In order to solve the above-described problem, an image evaluation apparatus according to the present invention calculates a feature amount of an image, a storage unit that stores a plurality of images and a plurality of templates, and arrangement information related to the template, and includes the feature in the image. A feature amount calculation unit that associates quantities and stores them in the storage unit, performs image classification based on the feature amounts stored in the storage unit, and executes a clustering process to generate a plurality of clusters composed of similar images; A clustering execution unit that associates the cluster with the storage unit and stores it in the storage unit, a digest image determination unit that determines a digest image that is a representative image from images constituting the cluster, and a template arrangement information from among the plurality of clusters Importance determining unit that extracts clusters and determines the order of importance of the extracted clusters, and digest of extracted clusters And placed in the template according to the image in order of importance to create a scene data constituting the composite image has an album creating section for storing the album data composed of scene data in the storage unit.

  In such an image evaluation apparatus, the importance level determination unit is configured to determine the importance level of the cluster based on the information on the number of images constituting the cluster.

  In addition, such an image evaluation apparatus includes a failure image processing unit that detects a failure image from an image and stores the failure image in association with the image in a storage unit, and the digest image determination unit excludes the failure image. The digest image is determined.

  Further, in such an image evaluation apparatus, the digest image determination unit, when determining the digest image that is the representative image according to the number of images that form the cluster, is an image that excludes the failed images of the images that form the cluster. If the number of images is three or more, the similarity between the images is calculated, and the image having the highest similarity among the three or more images is used as the digest image that is the representative image, and the failed images are excluded. If the number of images is two, the similarity between the digest image of the cluster adjacent to the cluster composed of the two images and the two images is calculated, and the two images If the image with the lower similarity to the digest image is the digest image that is the representative image and the number of images excluding the failed image is one, the image is the digest image that is the representative image. As, it is a configuration determined.

  In order to solve the above problems, an image evaluation program according to the present invention includes a control unit and a storage unit in which at least a plurality of images and a plurality of templates are stored, and a composite image in which the plurality of images are arranged in a template. An image evaluation program that allows a computer to execute a process of configuring a composite image from a plurality of images and displaying it on an image display unit when the image is displayed on a displayable image display unit To read out the image from the storage unit, calculate the feature amount of the image, associate the feature amount with the image and store the feature amount in the storage unit, and image classification based on the feature amount stored in the storage unit Clustering execution processing for generating a plurality of clusters composed of similar images, associating the clusters with the images and storing them in the storage unit, and images constituting the clusters. In the above, a digest image determination process for calculating the similarity between images based on the feature amount and determining the image having the highest similarity as a digest image that is a representative image, and template arrangement information from a plurality of clusters In accordance with importance determination processing for extracting clusters in accordance with the importance of the extracted clusters and the extracted digest images of the clusters in a template according to the order of importance, scene data that constitutes a composite image And an album creation process for storing album data composed of scene data in the storage unit.

  In such an image evaluation program, the importance level determination process is a process for determining the importance level of a cluster based on information on the number of images constituting the cluster.

  Further, in such an image evaluation program, a failure image is detected from the image, and a failure image processing step is performed in which the failure image is associated with the image and stored in the storage unit, and the digest image determination process excludes the failure image. This process determines the digest image.

  Further, in such an image evaluation program, the digest image determination process is an image excluding the failed images of the images constituting the cluster when determining the digest image that is the representative image according to the number of images constituting the cluster. If the number of images is three or more, the similarity between the images is calculated, and the image having the highest similarity among the three or more images is used as the digest image that is the representative image, and the failed images are excluded. If the number of images is two, the similarity between the digest image of the cluster adjacent to the cluster composed of the two images and the two images is calculated, and the two images If the image with the lower similarity to the digest image is the digest image that is the representative image and the number of images excluding the failed image is one, the image is the digest image that is the representative image. As strike image, a process of determining.

  The image evaluation program according to the present invention is stored in a storage medium or storage device such as a flexible disk, a CD-ROM, a magneto-optical disk, a semiconductor memory, or a hard disk. Moreover, it may be distributed as a digital signal via a network or the like. At this time, intermediate processing results are temporarily stored in a storage device such as a main memory (RAM).

  When the image evaluation apparatus and the image evaluation program according to the present invention are configured as described above, similar images can be automatically clustered from a plurality of image data, and the degree of importance is high among clusters of similar images (similar image groups). Image data can be automatically picked up.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. First, the configuration of the image evaluation apparatus 100 according to the present embodiment will be described with reference to FIGS. 1 and 2. In this embodiment, the image evaluation apparatus 100 will be described. However, the image evaluation apparatus 100 can be applied to a digital camera having an imaging unit, and can also be applied to a digital photo frame or the like. As shown in FIG. 1, the image evaluation apparatus 100 includes an HDD 11, a memory 12, a CPU (control unit) 13, an external I / F 14, a video memory 15, and an input device 16, which are connected via a data bus 17. The image data connected and stored in the HDD 11 or the like can be displayed and viewed on the image display unit (monitor) 18 via the video memory 15. The input device 16 receives an instruction from the user, transmits the signal to a CPU (control unit) 13, and each unit operates under the control of the CPU (control unit) 13 based on the signal. The external I / F 14 includes a controller for reading data stored in an external device such as a digital camera (not shown), a PC (Personal Computer), or a removable memory, and an image captured by the external device is displayed. The data is stored in a storage unit such as the HDD 11 under the control of the CPU (control unit) 13 via F14. Further, the HDD 11 stores a program and the like for functioning as the image evaluation apparatus 100 and data such as an image. The memory 12 has a function of storing a startup program for the image evaluation apparatus 100 and temporarily storing programs and data stored in the HDD 11. The CPU (control unit) 13 executes programs stored in the HDD 11, the memory 12, and the like, and controls each unit of the image evaluation apparatus 100.

  As shown in FIG. 2, the image evaluation apparatus 100 includes a CPU (control unit) 13, a digest image determination unit 102, a database (storage unit) 103, a feature amount calculation unit 104, a clustering execution unit 105, a work An area (work RAM) 106, an importance level determination unit 107, a failed image processing unit 108, an album creation unit 109, and an image display unit 18 are included. The CPU (control unit) 13 controls the entire apparatus, and the feature amount calculation unit 104 reads out an image stored in the database (storage unit) 103 based on an instruction from the CPU (control unit) 13 and performs a work area (Work). RAM) 106 calculates the feature amount of the image. The clustering execution unit 105 classifies the images based on the feature amounts, and generates a plurality of clusters composed of similar images (executes clustering processing). The digest image determination unit 102 determines an image (digest image) with high importance from images classified into a plurality of clusters, and the necessity determination unit 107 determines the importance (priority) of each digest image. Further, the failed image processing unit 108 determines whether the image is well-exposed or poorly exposed, camera shake, and the like, and removes the failed image from the images classified into clusters. The album creating unit 109 is stored in the database (storage unit) 103. An image is arranged on a template to determine the composition of scene data to be a composite image based on the digest image and its importance, and album data composed of a plurality of scene data is created. The image display unit 18 displays a composite image based on the scene data included in the album data. The work area (Work RAM) 106 has a configuration for temporarily storing the results of clustering processing, intermediate calculations, and the like.

  As shown in FIGS. 3 and 4, the database (storage unit) 103 includes an image information storage file 113 in which the storage destination of the image data, the cluster number to which the image data belongs, and the template number and attributes (number of images, image layout). A template information file 115 for storing information and a album data file 114 for storing templates, image file data, scene numbers, and the like. The image information storage file 113 has a data structure as shown in FIG. 3, a file name column 113a in which the file name of the image data is stored, and an address of the storage destination of the image file composed of the image data and attribute information. Are stored, an image file storage address column 113b, a feature amount column 113c for storing an image feature amount as a result of feature amount calculation described later, a cluster number column 113d for storing a clustering result described later, A failure flag data column 113e for determining whether the image is a failure image or not.

  On the other hand, the template information file 115 has a data structure as shown in FIG. 4B. The template number column 115a stores the number of the template data, and the number data stores the number of sheets that can be arranged in the template. It includes a column 115b and a position size information column 115c in which information on the size and position of the image to be arranged on the template is stored.

  Also, the album data file 114 has a data structure as shown in FIG. 4A, a scene number data column 114a in which the order of scene data is stored, and a template that forms a composite image of the scene data. A template number column 114b in which the number of template data for identifying data is stored, and a position information data column in which image file names corresponding to the number of images that can be arranged in the template with the number specified in the template number column 114b are stored in the arrangement order. 114c. The album data stored in the album data file 114 is configured to have a plurality of one scene (scene data) constituting a composite image in which a plurality of images are arranged on a template.

  The template data used in the scene data constituting the album data can be designated by the template number, and the attribute information of the designated template is determined by the template information file 115 in terms of the number of sheets and the arrangement position and size of the image. . The importance of the images when arranged in the template is 1, 2, 3, 4,... In the position information data column 114c of the album data file 114, and this number of the position information data column 114c is the template information. This corresponds to the number in the position size information column 115c of the file 115. The attribute information included in the image file also includes shooting time data, and the file format is, for example, Exif (Exchangeable image file format) standardized by the Japan Electronics Industry Promotion Association (JEIDA). The data structure is as follows.

  In one scene data of album data stored in the album data file 114, image data constituting the one scene is specified by a file name, and is specified by an address in the image file storage address column 113b corresponding to the file name. it can. Therefore, the CPU (control unit) 13 refers to the file name of one scene in the album data file 114, the corresponding file name from the file name column 113a of the image information storage file 113 in the database (storage unit) 103, and An address corresponding to the file name is specified with reference to the image file storage address column 113b, and image data is read from the database (storage unit) 103 with reference to the address. Further, the CPU (control unit) 13 extracts the template number of one scene in the album data file 114, reads out the template information of the corresponding number from the template information file 115 in the database (storage unit) 103, and the template and A composite image is constructed based on the image data, and this composite image is displayed on the image display unit 18.

(Create album data)
A process of creating album data using the image evaluation apparatus 100 using the image evaluation and the evaluation result will be described with reference to the flowchart of FIG. When the user instructs to create album data from the input device 16, the signal is transmitted to the CPU (control unit) 13, and the CPU (control unit) 13 creates an album from the database (storage unit) 103. The image file is read out based on the address associated with the file name (for example, 0001.jpg) stored in the file name column 113a of the image information storage file 113, and the image data included in the image file is read out. Based on this, the feature quantity calculation unit 104 calculates the feature quantity of the image, and returns the feature quantity as a calculation result to the CPU (control unit) 13, and the result is stored in the feature quantity column 113 c of the image information storage file 113. Stored (step ST100). Here, as the image feature amount, various image statistics defined by mpge7 visual description are used as the feature amount, and each image is characterized thereby. Specifically, the feature amount of the image is an image statistic called an edge histogram associated with the texture of the image, a color histogram or a color layout which is a statistic regarding color distribution. For example, the case of a color histogram which is a statistic regarding color distribution will be described as an example. Here, it is assumed that the image data input to the feature amount calculation unit 104 is expressed in the YCbCr (hereinafter simply referred to as “YCC”) color system as the first color system. In the following description, this image data is simply referred to as “YCC data”. The YCC data input to the feature quantity calculation unit 104 is plotted on the YCC color space, and the feature quantity calculation unit 104 counts the number plotted for each block. Then, a three-dimensional YCC histogram is generated by counting. Accordingly, the feature amount calculation unit 104 calculates the feature amount using the feature amount, and the CPU (control unit) 13 associates the feature amount as a calculation result with the data of the image and stores it in the database (storage unit) 103. And stored in the feature amount column 113c of the image information storage file 113.

  Next, when the CPU (control unit) 13 passes the image data to the failed image processing unit 108 and gives an instruction to detect the failed image, the failed image processing unit 108 exposes the received image data. Alternatively, based on information such as exposure failure and blurring (camera shake), it is determined whether or not the image corresponds to a failed image, and the result is returned to the CPU (control unit) 13, and the CPU (control unit) 13 Based on the result, a flag for identifying the failure image is stored in the failure flag data column 113e of the image information storage file 113 (step ST110). Here, a case will be described in which “1” is stored if the image is a failed image, and “0” is stored otherwise. The determination of the failed image by the failed image processing unit 108 is, for example, based on the saturation and brightness of the image, when the image is within a certain threshold range, the exposure is good. Or based on the edge information of the image, it is determined whether or not the image is a failed failure image. The result of such determination is returned to the CPU (control unit) 13 and, as described above, a flag for identifying the failure image is stored in the failure flag data column 113e.

  The above-described feature amount calculation processing (step ST100) and failure image detection processing (step ST110) are repeated until all the images stored in the database (storage unit) 103 are executed.

  As described above, at the stage where the image data has been read for all the images stored in the database (storage unit) 103 and the feature amount calculation and the storage in the database (storage unit) 103 have been completed, the image evaluation apparatus 100 continues the image processing. Clustering processing is performed (step ST120). In the image clustering process, the CPU (control unit) 13 reads out each image file stored in the database (storage unit) 103, and based on the image data included in the image file, all the feature values associated with the image data are extracted. The feature amounts for all the images (hereinafter referred to as feature amount sets) are transferred to the clustering execution unit 105, and a clustering execution command is issued to the clustering execution unit 105. The clustering execution unit 105 executes clustering to classify images and generate a plurality of clusters based on the given feature quantity set, and as shown in FIG. 7, which image is associated with each feature quantity. A calculation result indicating whether it belongs to a cluster (hereinafter referred to as a clustering result) is returned to the CPU (control unit) 13. This cluster is a group of similar images (similar image group) detected by detecting the similarity of images.

That is, in step ST120, based on the feature amount set, a feature vector having each feature amount as a component is defined for each image, and a linear space defined in the feature amount vector is defined as a feature space. Grouping is performed according to the “distance” of the feature vector corresponding to the image. More specifically, as described above, an image statistic called an edge histogram associated with an image texture, a color histogram or a color layout that is a statistic regarding color distribution is used as a feature quantity, and When similarity is determined according to the Euclidean distance at, and grouping is performed by hierarchical clustering, a system diagram (FIG. 6) showing which images belong to the same group is generated according to the similarity (distance). Is done. The generated system diagram by cutting at a distance T _D previously determined, can be similar image group is determined, clustering. Then, the CPU (control unit) 13 assigns cluster numbers to all the generated clusters in ascending order based on the crystallizing result received from the clustering execution unit 105, and applies to each image in the database (storage unit) 103. The cluster number belonging to each image is associated and stored in the cluster number column 113d of the image information storage file 113. For example, in this embodiment, as shown in FIG. 7, the images are clustered into four clusters (similar image groups), and each cluster is given a cluster number of 1 to 4, and as shown in FIG. The cluster number to which the image belongs is stored in the cluster number column 113d of the image. Here, various types of data such as the image data, feature set, and clustering result as described above are temporarily stored in the work area 106 and are used by each unit so that data between the units can be obtained. Is delivered. At this time, if the CPU (control unit) 13 has only one image belonging to the cluster and one of the images is a failed image in the clustering result, the album creating unit 109 displays the album data. Since the cluster of only one failed image is excluded from the cluster candidates selected when creating the scene data, the cluster number is not stored in the cluster number column 113d. Specifically, the image file name “0011.jpg” in FIG. 6 stores a flag “1” indicating that the image is a failed image in the failure flag data column 113e in FIG. It is empty data. Therefore, such data is excluded when creating the following scene data.

  When the CPU (control unit) 13 completes the above image clustering execution process (ST120), it reads all the image data in the database (storage unit) 103, the cluster number to which it belongs, the template information file 115 and the album data file 114. Then, when it is passed to the album creation unit 109 and an instruction to start scene data creation is issued, the album creation unit 109 refers to the scene number data column 114a of the album data file 114 and selects the first scene (scene selection). (ST130). Next, the album creating unit 109 refers to the template number column 114b and, when the template number is confirmed, reads the template number information confirmed from the template number column 115a of the template information file 115, and arranges the template according to the scene. The number of sheets is determined (ST140), and the determination is returned to the CPU (control unit) 13. For example, referring to FIG. 4A, in the case of the scene number “1” in the scene number data column 114a of the album data file 114, referring to the template number column 114b, the template No. “1” is stored. 4B, the template number in the template number column 115a of the template information file 115 is the template number. In the case of “1”, “4” is stored in the number data column 115b. The number of images required for the “1” template is “4”. Since the number of images required for the scene number “1” is “4”, the file names of the images can be stored in columns 1 to 4 of the position information data column 114c. At this time, the order of 1 to 4 in the position information data column 114 c of the album data file 114 is a number corresponding to the numbers 1 to 4 in the position size information column 115 c of the template information file 115. The order is also the importance of the arranged image.

  Then, album creating section 109 extracts a cluster (similar image group) corresponding to the required number from the image data and the cluster number to which it belongs (ST150). At this time, for example, the cluster numbers are arranged in the order of the photographing date and time, and therefore, in ascending order of the cluster numbers, in this case, the cluster numbers 1 to 4 are selected when the scene data of the scene number 1 is created. Hereinafter, the next cluster number is selected in ascending order according to the number of templates required.

  Next, when the CPU (control unit) 13 receives the extraction result of the cluster (similar image group) corresponding to the required number from the album creating unit 109, the digest image determining unit 102 has the selected cluster number. Pass the image data file. Then, digest image determination unit 102 executes digest image determination processing for each cluster (for each similar image group) of the selected cluster number, and passes the result to CPU (control unit) 13 (ST160). Upon receiving the digest image (A (n)) determination result from the digest image determining unit 102, the CPU (control unit) 13 receives information about the digest image (A (n)) and the digest image (A (n)). The file of the image data having the cluster number to which n)) belongs is transferred to the importance determining unit 107. Then, the importance level determination unit 107 counts the number of image data files having the cluster number to which the digest image (A (n)) belongs, that is, the number of images belonging to the cluster, and determines the importance level in descending order. (C (n)) is determined (ST170), and the result is passed to the CPU (control unit) 13. Specifically, when the album creation unit 109 selects a cluster (similar image group (n) 1 to 4), the digest image determination unit 102 selects a digest image (A (n)) from the group of images belonging to this cluster. Is determined (see FIG. 9).

  Here, the determination of the digest image (A (n)) will be described in detail later, but the most similar image is selected by comparing the similarity of the images of each image group. In FIG. 9, from the image group “1” in the similar image group (n), the second image from the left (image file name “0002.jpg”) is the digest image (A (n)). Three images belong to “1” of the similar image group (n) (image file names “0001.jpg”, “002.jpg”, “0003.jpg”), and other similar images. Compared to the number of images belonging to the image group (n), it is the second largest. Therefore, in this case, the importance (C (n)) is 2. Also, from the image group “3” in the similar image group (n), the fourth image from the left (image file name “0008.jpg”) is the digest image (A (n)), and this similar image In group (n) “3”, four images belong (image file names “0005.jpg”, “006.jpg”, “0007.jpg”, “0008.jpg”), and other Compared to the number of images belonging to the similar image group (n), this is the first largest number. Therefore, in this case, the importance (C (n)) is 1. Thus, the importance is set higher in descending order of the number of sheets (the number of image data files having the same cluster).

  Next, when the CPU (control unit) 13 passes the information on the digest image (A (n)) and the information on the importance (C (n)) to the album creating unit 109, the album creating unit 109 C (n)) is referred to, the file names of the image data to be stored in the position information data column 114c of the album data file 114 are determined in the order of 1 to 4, and the information is sent to the CPU (control unit) 13. When returned, the CPU (control unit) 13 determines the arrangement on the template by storing it in the position information data column 114c of the album data file 114 of the database (storage unit) 103 (ST step 180). At this time, the order of 1 to 4 in the position information data column 114c corresponds to the importance (C (n)) information. Accordingly, as shown in FIG. 4B, the file name “0008.jpg” of the image data is stored in “1” of the position information data column 114c, and the file name “0008” of the image data is stored in “2”. “0002.jpg” is stored, “3” stores the file name “0009.jpg” of the image data, and “4” stores the data file name “0003.jpg” of the image.

  Proceeding to step ST190, the CPU (control unit) 13 determines whether or not the above-described processing has been completed for all the groups (clusters). If not, the process returns to step ST130 and all similar images are determined. The process is executed until the process ends for the group (cluster). If it is determined that all similar image groups (clusters) have been completed, this process is terminated.

(Determination of digest image)
Next, details of the process (step ST160) in which the digest image determination unit 102 determines the digest image (A (n)) for each cluster (similar image group) will be described based on the flowchart of FIG. For each selected cluster (similar image group), digest image determination unit 102 extracts the number of image files (number of images m) included in the cluster (hereinafter referred to as similar image group) (step ST1601). If it is determined that m is two or more, whether or not a failure image is included is detected (step ST1602). On the other hand, when determining that the number m of images is not two or more (one), the digest image determination unit 102 sets one image belonging to the similar image group as a digest image (step ST1611). For example, since the similar image group “2” has only the file name “0004.jpg” of the image data to which it belongs, the image data with this file name “0004.jpg” is the digest image (A (n)) ( (See FIGS. 7 and 9).

  The digest image determination unit 102 determines whether “1” data is stored in the failure flag data column 113e from the image data belonging to the similar image group (data for each image file name in the image information storage file 113). Then, the number of failed images is determined (step ST1602), and then the number of unfailed images (k = m−number of failed images) is calculated (step ST1603).

  If the digest image determination unit 102 determines in step ST1603 that the number k of images that have not failed is two, then the digest image determination unit 102 determines whether there is no failure, that is, the number of image files (images belonging to a similar image group). It is determined whether or not the number m) is 2 (step ST1607), and if “no failure” is determined, the previous and subsequent similar groups are compared with the digest images (A (n)) of the previous and subsequent similar groups. Is determined as a digest image (A (n)) (step ST1608). That is, when there are only two similar images and neither of them is a failed photo, the digest image determining unit 102 compares the digest image with the previous and subsequent digest images (A (n)) and has a similarity degree. A low one, that is, an image that is not similar to the images before and after the digest is adopted. On the other hand, if the digest image determination unit 102 determines that m = 2 (“no failure”), that is, if one of the two images is a failure image, the digest image (not the failure image) is selected. A (n)) (step ST1609). For example, in the case of the similar image group “4”, the number of images included in this similar image group is two, but one of them (the file name “0010.jpg” of the image is stored in the failure flag data column 113c. Since the data of “1” is stored and is a failed image, the number of images excluding the failed image is one here, so that the image (image file name “0009.jpg”) is the digest image (A (N)) (see FIGS. 3 and 9).

  On the other hand, when the digest image determining unit 102 determines in step ST1603 that the number k of images that have not failed is one, the failed image is excluded from the similar image group (step ST1610), and the process proceeds to step ST1611. An image that is not one failed image is referred to as a digest image (A (n)).

  If it is determined in step ST1603 that the number k of images that have not failed is three or more, the digest image determination unit 102 next excludes the failed images from the image group of similar image groups. (Step ST1604). Then, the similarity between the images in the image group belonging to the similar image group is calculated (step ST1605), and the image with the highest similarity is set as the digest image (A (n)) (step ST1606). As a method of calculating the similarity between the images when determining the digest image (A (n)), a known method can be used. For example, the similarity between the images can be determined by the color of each image. The distance between the images is obtained in the i-dimensional space based on the feature amount (i-dimensional) of the space, and the average value of the distances is obtained for each image, and the image having the smallest average value is the most similar image. The image can be determined as a digest image (A (n)). Here, the result of which the digest image (A (n)) for each similar image group is, the image data of the similar image group, and the like are temporarily stored in the work area (Work Ram) 106. It has become so. Thus, when the digest image determination unit 102 determines the digest image (A (n)) for each selected similar image group, the digest image determination unit 102 determines that the digest image (A (n)) has been completed and the digest image (A ( n)) is notified to the CPU (control unit) 13 and the process ends.

  As described above, an album composed of digest data (A (n)) for each similar image group and scene data whose arrangement position on the template is determined based on the result of the importance (C (n)). Based on the data file 114, the CPU (control unit) 13 can arrange image data on a template and display a composite image 200 as shown in FIG. 10 on the image display unit 18. Here, the template with the template number “1” is used, the image 200a is an image with the data file name “0008.jpg” of the image “1” in the position information data column 114c, and the image 200b is The data file name “0002.jpg” of the image “2” in the position information data column 114c, and the image 200c has the data file name “0009.jpg” of the image “3” in the position information data column 114c. The image 200d is an image having the data file name “0003.jpg” of the image “4” in the position information data column 114c. Each template has a predetermined number of images, positions, sizes, and the like, and the information is stored in the template data file 115. The position and size arranged on the template are determined in order of importance (C (n)), and the image that is automatically considered the most favorite (the number of similar images is large). Image) becomes a digest image, and a composite image 200 is configured by arranging a plurality of the digest images. In addition, the scene data of the album data is configured in the order of popularity (the order in which the number of similar images is large), and in the composite image 200 of one scene data, the image with the highest importance is arranged the largest. It has come to be. Therefore, the importance can be automatically generated and the favorite image can be selected and the layout can be determined in descending order of importance, and the album layout can be automatically performed without bothering the user. In the present embodiment, since the similarity between the images is calculated by excluding images that are blurred or poorly exposed in the image group of the similar image group, the digest is automatically performed. Users can be satisfied with the result of the creation. However, the process of detecting and removing failed photos can be omitted depending on the system configuration and the user's will.

It is a hardware block diagram showing the structure of the image evaluation apparatus of this embodiment. It is a block diagram showing the structure of the image evaluation apparatus of this embodiment. It is a data structure figure of the image data file of this embodiment. It is a data structure figure of this embodiment, (a) is a data structure figure of an album data file, (b) is a data structure figure of a template data file. It is a flowchart showing the process sequence of album data creation. It is a schematic diagram showing the result of having cut _| disconnected the systematic diagram obtained as a result of hierarchical clustering by _Td . It is a schematic diagram showing the example of the clustering result finally obtained. It is a flowchart which shows the procedure which discriminate | determines the digest image which is an image with high attention degree in a similar image group. It is a schematic diagram which shows the example of the image which determines the digest of the image group of a similar image group, and its importance. It is a schematic diagram which shows the example of the screen displayed on the image display part, applying the result of the importance of an image to the screen layout for a display.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image evaluation apparatus 13 CPU (control part) 18 Image display part 102 Digest image determination part 103 Database (memory | storage part)
104 feature amount calculation unit 105 clustering execution unit 107 importance level determination unit 109 album creation unit

Claims (2)

A storage unit for storing a plurality of images and a plurality of templates and arrangement information related to the templates;
A feature amount calculation unit that calculates a feature amount of the image, associates the feature amount with the image, and stores the feature amount in the storage unit;
Based on the feature quantity stored in the storage unit, image classification is performed, clustering processing for generating a plurality of clusters composed of similar images is executed, and the cluster is associated with the image and stored in the storage unit A clustering execution unit;
A failure image processing unit that detects a failure image from the image and stores the failure image in association with the image in the storage unit;
A digest image determining unit that determines a digest image that is a representative image from the images constituting the cluster;
An importance determining unit that extracts the clusters from a plurality of the clusters according to the arrangement information of the template, and determines the order of importance of the extracted clusters;
An album in which the digest images of the extracted clusters are arranged in the template according to the order of importance to create scene data constituting a composite image, and album data composed of the scene data is stored in the storage unit possess a creation unit, the,
The importance determining unit determines the importance of the cluster based on information on the number of images constituting the cluster;
The digest image determining unit, when determining the digest image that is the representative image according to the number of the images constituting the cluster, the image of the image excluding the failed image of the image constituting the cluster When the number is three or more, the similarity between the images is calculated, and the image having the highest similarity among the three or more images is set as the digest image as the representative image, When the number of the images excluding the failed image is two, the similarity between the digest image of the cluster adjacent to the cluster composed of the two images and the two images is calculated. The image having the lower similarity to the digest image of the two images is the digest image that is the representative image, and the number of the images excluding the failed image is one. In this case, the image evaluation apparatus, wherein the said image as the digest view is the representative image, a configuration determined. A control unit, and a storage unit storing at least a plurality of images and a plurality of templates, and when displaying a composite image in which the plurality of images are arranged in the template on a displayable image display unit, An image evaluation program that allows a computer to execute a process of accessing the storage unit and configuring the composite image from a plurality of the images and displaying the composite image on the image display unit,
By the control unit,
A feature amount calculation process for reading the image from the storage unit, calculating a feature amount of the image, and associating the feature amount with the image and storing the feature amount in the storage unit;
Clustering execution processing for classifying images based on the feature values stored in the storage unit, generating a plurality of clusters composed of similar images, and associating the clusters with the images and storing them in the storage unit;
Failed image processing for detecting a failed image from the image and storing the failed image in association with the image in the storage unit;
Digest image determination processing for calculating the similarity between the images based on the feature amount in the images constituting the cluster, and determining the image with the highest similarity as a digest image that is a representative image;
Importance determination processing for extracting the cluster from a plurality of the clusters according to the arrangement information of the template, and determining the order of importance of the extracted clusters,
An album in which the digest images of the extracted clusters are arranged in the template according to the order of importance to create scene data constituting a composite image, and album data composed of the scene data is stored in the storage unit Execute the creation process ,
The importance determination process is a process of determining the importance of the cluster based on information on the number of images constituting the cluster,
The digest image determination process determines the number of the images excluding the failed images of the images constituting the cluster when determining the digest image that is the representative image according to the number of the images constituting the cluster. If the number of images is three or more, the similarity between the images is calculated, the image having the highest similarity among the three or more images is set as the digest image that is the representative image, and the failure When the number of images excluding images is two, the similarity between the digest image of a cluster adjacent to the cluster composed of the two images and the two images is calculated, Of the two images, the image having the lower similarity to the digest image is the digest image that is the representative image, and the number of the images excluding the failed image is one. Expediently, an image evaluation program, characterized in that the image as the digest view is the representative image, a process of determining.

Images