JP2018077712A - Information processing device and information processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To output an operation for image data according to characteristics of a user.SOLUTION: An information processing device 10 according to one embodiment includes: a generation unit 15 that generates association information in which an evaluation model and an operation correlated with an evaluation value output by the evaluation model, which is a user operation for the image data, are associated; an evaluation unit 17 that calculates an evaluation value of one image data using the evaluation model; and an output unit 18 that outputs information indicating an operation associated with the evaluation model in the association information as information indicating an operation recommended for one image data in accordance with the evaluation value calculated by the evaluation unit 17.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus and an information processing program.

  Conventionally, a technique is known in which a predetermined feature amount is extracted from image data, the quality of the image data is determined based on the extracted feature amount, and an evaluation value is assigned to the image data. For example, in Patent Document 1 below, by assigning a feature amount extracted from image data to a predetermined evaluation formula (evaluation model), an evaluation value of the image data is calculated, and image data with a high evaluation value is obtained. Automatic selection as image data to be assigned to an album is disclosed.

JP 2011-186714 A

  According to the above method, an operation according to the evaluation value of the image data (in the above example, an operation of assigning image data having a high evaluation value to the album) can be output. However, the evaluation formula used in the above method is based on general image quality criteria, and the characteristics (preference and behavior characteristics, etc.) of the user who evaluates the image data (for example, the user who owns the image data). It is not a reflection. For this reason, depending on the user, an appropriate operation (in the above example, an operation for assigning an image appropriate for the user to an album) may not be output.

  Accordingly, an object of one aspect of the present invention is to provide an information processing apparatus and an information processing program that can output an operation on image data in accordance with user characteristics.

  An information processing apparatus according to an aspect of the present invention includes an evaluation model that inputs a predetermined feature amount of image data and outputs an evaluation value of the image data, and a user operation on the image data, which is output by the evaluation model A generation unit that generates association information that associates an evaluation value with a correlated operation, an evaluation unit that calculates an evaluation value of one image data using an evaluation model, and an evaluation value calculated by the evaluation unit And an output unit that outputs information indicating an operation associated with the evaluation model in the association information as information indicating an operation recommended for one image data.

  According to the information processing apparatus according to one aspect of the present invention, a user operation (for example, the evaluation model) correlated with the evaluation value of the evaluation model according to the evaluation value output by the evaluation model for one image data. The operation that tends to be executed by the user when the evaluation value is high) can be output as a recommended operation for the one image data. Therefore, according to the information processing apparatus, it is possible to output an operation on the image data according to the user characteristics.

  According to one aspect of the present invention, it is possible to provide an information processing apparatus and an information processing program that can output an operation on image data in accordance with user characteristics.

It is a block diagram which shows the function structure of the information processing apparatus which concerns on one Embodiment. It is a figure which shows an example of the operation history stored in the operation history storage part. It is a figure which shows an example of the evaluation history stored in an evaluation history storage part. It is a flowchart which shows an example of operation | movement of information processing apparatus. It is a flowchart which shows the procedure of the 1st process for producing | generating correlation information. It is a flowchart which shows the procedure of the 2nd process for producing | generating correlation information. It is a block diagram which shows the module structure of the information processing program which concerns on one Embodiment. It is a block diagram which shows the hardware constitutions of information processing apparatus.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or corresponding elements are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is a diagram showing a functional configuration of an information processing apparatus 10 according to an embodiment of the present invention. The information processing apparatus 10 is an apparatus that outputs an operation associated with the evaluation model as a recommended operation for the image data in accordance with the evaluation value of the image data output by the evaluation model. In this embodiment, as an example, the information processing apparatus 10 is mounted on a mobile terminal owned by a user such as a smartphone and a tablet terminal. However, the information processing apparatus 10 is not limited thereto, and may be a stationary computer apparatus such as a personal computer and a server apparatus.

  An evaluation model is a method for inputting a predetermined feature amount of image data and outputting an evaluation value of the image data. For example, by inputting a numerical value indicating one or more feature amounts as an explanatory variable, It can be expressed as a function for which an evaluation value is obtained. The evaluation model is, for example, predetermined machine learning using, as learning data, a plurality of image data to be positive examples (evaluation value is “1”) and a plurality of image data to be negative examples (evaluation value is “0”). Is obtained by executing A large evaluation value obtained for one image data by such an evaluation model (close to 1) indicates that the degree of similarity between the one image data and the image data input as a positive example is large. On the other hand, a small evaluation value (close to 0) indicates that the degree to which one image data is similar to the image data input as a positive example is small. Therefore, the characteristics of the generated evaluation model differ depending on the characteristics of the image data used as the positive example or the negative example. For example, when an evaluation model is generated using a photograph (image data) listed as a good sample in a textbook of photographs as a positive example, the tendency is to output a higher evaluation value for a photograph taken in accordance with a textbook theory It is thought that a certain evaluation model can be obtained. In addition, when an evaluation model is generated using a photograph that a specific person likes as a positive example, an evaluation model that tends to output a higher evaluation value is obtained for a photo that has a higher degree of matching with the specific person's preference. it is conceivable that.

  The operation associated with the evaluation model is a user operation on the image data and an operation correlated with the evaluation value output by the evaluation model. The operation having a correlation with the evaluation value indicates an operation having a predetermined tendency (for example, a high probability of being executed) corresponding to a predetermined tendency (for example, having a high evaluation value). For example, let us consider a case where image data having a high evaluation value according to a specific evaluation model (for example, a predetermined threshold value or more) tends to be posted on a predetermined SNS (Social Networking Service) site by a user. In this case, the user's operation of posting image data to a predetermined SNS site can be associated with the specific evaluation model as an operation correlated with the evaluation value output by the evaluation model. The information processing apparatus 10 can generate association information indicating such association, and output information indicating an operation recommended for arbitrary image data by referring to such association information. For example, the information processing apparatus 10 can present to a user by outputting information indicating a recommended operation on a display or the like of the information processing apparatus 10. Further, the information processing apparatus 10 can automatically execute the operation by the application by outputting information indicating the recommended operation to, for example, a predetermined application that executes the operation.

  As illustrated in FIG. 1, the information processing apparatus 10 includes, as functional components, a data storage unit 11, an evaluation model storage unit 12, an operation history storage unit 13, an evaluation history storage unit 14, and a generation unit 15. And an association information storage unit 16, an evaluation unit 17, and an output unit 18. The generation unit 15 includes, as functional components for executing the first process for generating association information, an operation history acquisition unit 151, an extraction unit 152, an evaluation model generation unit 153, and a first association Part 154. In addition, the generation unit 15 includes the above-described operation history acquisition unit 151, evaluation history acquisition unit 155, and operation extraction unit 156 as functional components for executing the second process for generating association information. And a second associating unit 157.

  The data storage unit 11 stores a plurality of image data. The image data is, for example, information of an image captured at a preset resolution (for example, 4000 × 3000 pixels), and the pixel value of each pixel (for example, a value expressing luminance, brightness, color, and the like as numerical values). Includes information to show. In addition, auxiliary information such as Exif (Exchangeable image file format) information may be embedded in the image data. Exif information includes information related to shooting conditions and camera settings at the time of shooting, such as shooting date and time, shutter speed, and focal length.

  The evaluation model storage unit 12 stores a plurality of evaluation models. Each evaluation model stored in the evaluation model storage unit 12 is a model having a unique characteristic (that is, an evaluation model generated using a set of positive examples or negative examples different from each other). In the present embodiment, as an example, the evaluation model storage unit 12 is an evaluation model (model A) generated by using a photograph that Mr. A, a professional photographer, is convinced of as a result of photographing as a positive example, and a professional photographer. An evaluation model (model B) generated using a photograph that Mr. B is satisfied with the photographing result as a positive example is stored.

  The operation history storage unit 13 stores a history of operations (operation history) performed on image data stored in the data storage unit 11 by a user who uses the information processing apparatus 10 (hereinafter simply “user”). . For example, the operation history is information in which the operation date and time when the operation is executed, the operation content, and the like are associated with each other. However, the data format of the operation history is not limited to a specific mode. For example, the operation history may be information indicating whether each operation has been executed without including the operation date and time.

  FIG. 2 is a diagram illustrating an example of an operation history stored in the operation history storage unit 13. As shown in FIG. 2, each piece of information included in the operation history (information for one record in the table shown in FIG. 2) is information that associates an image ID with the operation content. The image ID is identification information for uniquely specifying the image data stored in the data storage unit 11. The operation content is information indicating the content of the operation performed by the user on the image data specified by the image ID. In the example of FIG. 2, an operation “Save to Album” is performed on two image data with image IDs “1000” and “1001”, and two image data with image IDs “1002” and “1003” are executed. It is shown that the operation “Post to SNS” is executed.

  Such operation history generation (that is, operation content recording) is performed as follows, for example. That is, the operation history storage unit 13 monitors the operation executed on the image data and detects that any of the operation contents (operation pattern etc.) registered in advance is executed on the image data. Information that associates the image ID of the image data to be operated with the detected operation content (information for one record in the table shown in FIG. 2) is added. As an example of the operation contents registered in advance, as shown in the example shown in FIG. 2, a specific area such as “save to album” (here, an album area in which image data that the user likes as an example is registered) For example, storing image data and posting image data to a predetermined SNS site such as “post to SNS”. However, the operation contents registered in advance are not limited to these examples. The operation content registered in advance may be, for example, an operation of sending image data attached to a mail addressed to a specific destination (for example, a mail address set as a contact address of the user's mother). The operation may be a relatively simple data operation (for example, an operation for deleting image data).

  The evaluation history storage unit 14 outputs (calculates) each image data stored in the data storage unit 11 by each evaluation model stored in the evaluation model storage unit 12 (model A and model B as an example in the present embodiment). Stored evaluation value is stored.

  FIG. 3 is a diagram illustrating an example of the evaluation history stored in the evaluation history storage unit 14. As shown in FIG. 3, each piece of information included in the evaluation history (information for one record in the table shown in FIG. 3) is information that associates an image ID, an evaluation model ID, and an evaluation value. The evaluation model ID is identification information for uniquely specifying the evaluation model stored in the evaluation model storage unit 12. The evaluation value is an evaluation value output for the image data specified by the image ID by the evaluation model specified by the evaluation model ID. In the example of FIG. 3, for the image data with the image ID “1000”, the evaluation value “0.68” is output by the model A, the evaluation value “0.78” is output by the model B, and the like.

  Such an evaluation history is stored in the evaluation history storage unit 14 every time the evaluation unit 17 described later calculates an evaluation value for each image data using each evaluation model.

  The production | generation part 15 produces | generates the correlation information which linked | related the evaluation model and the user's operation with respect to image data, Comprising: The evaluation value output by an evaluation model and the correlation. The generation unit 15 generates association information by executing, for example, a first process and a second process described below. The generation unit 15 may execute both the first process and the second process, or may execute only one of them.

(First process)
In the first process, the generation unit 15 newly generates an evaluation model associated with the operation executed by the user on the image data, and generates association information by associating the generated evaluation model with the operation. To do. Specifically, the generation unit 15 performs the above-described process using the operation history acquisition unit 151, the extraction unit 152, the evaluation model generation unit 153, and the first association unit 154.

  The operation history acquisition unit 151 acquires an operation history in which image data is associated with an operation performed on the image data by a user. In the present embodiment, the operation history acquisition unit 151 can acquire the operation history stored in the operation history storage unit 13 by referring to the operation history storage unit 13. Here, as an example, it is assumed that the operation history acquisition unit 151 acquires the operation history shown in FIG.

  The extraction unit 152 refers to the operation history acquired by the operation history acquisition unit 151, extracts one or a plurality of image data on which one operation has been executed as a positive example, and one or more of which one operation has not been executed A plurality of image data is extracted as a negative example. For example, when one operation is the above-described “save to album” operation, the extraction unit 152 performs image data (image data and image ID having an image ID of “1000”) on which the “save to album” operation is performed. Is extracted as a positive example. On the other hand, the extraction unit 152 extracts, as a negative example, image data that has not been subjected to the “save to album” operation (image data with an image ID “1002” and image data with an image ID “1003”).

  The evaluation model generation unit 153 generates an evaluation model by executing machine learning using positive examples and negative examples extracted by the extraction unit 152. The evaluation model generation unit 153 stores the evaluation model generated in this way in the evaluation model storage unit 12.

  As machine learning executed by the evaluation model generation unit 153, conventionally known methods such as multiple regression analysis, convolutional neural network, and SVM (support vector machine) can be used. The evaluation model generated by the evaluation model generation unit 153 is not limited to a specific mode. For example, a predetermined feature amount of image data (for example, the pixel value of each pixel, various information embedded in Exif information ( (E.g., shutter speed, focal length, etc.) are input as explanatory variables, and an evaluation value of the image data is output as an objective variable.

  The first association unit 154 generates association information that associates the evaluation model generated by the evaluation model generation unit 153 with the above-described one operation (here, “Save to Album” operation). The first association unit 154 stores the association information generated in this way in the association information storage unit 16.

  According to the first process described above, a new evaluation model is generated using image data (positive example) in which one operation is executed and image data (negative example) in which the one operation is not executed. Thus, the association information that associates the evaluation model with one operation can be generated. For example, when there is no evaluation model suitable for associating with one operation ("save to album" in the above example) among existing evaluation models (evaluation model stored in the evaluation model storage unit 12) The first process is suitable.

(Second process)
In the second process, the generation unit 15 outputs the image data based on the existing evaluation model based on the evaluation value for the image data output by the existing evaluation model and the user operation performed on the image data. An operation having a correlation with the evaluated value is extracted. Then, the generation unit 15 generates association information by associating the existing evaluation model with the extracted operation. Specifically, the generation unit 15 performs the above-described process using the operation history acquisition unit 151, the evaluation history acquisition unit 155, the operation extraction unit 156, and the second association unit 157 described above.

  The operation history acquisition unit 151 acquires the operation history stored in the operation history storage unit 13 by referring to the operation history storage unit 13 as described above. Here, as an example, it is assumed that the operation history acquisition unit 151 acquires the operation history shown in FIG.

  The evaluation history acquisition unit 155 acquires an evaluation history in which image data, an evaluation model, and an evaluation value output from the evaluation model for the image data are associated with each other. In the present embodiment, the evaluation history acquisition unit 155 can acquire the evaluation history stored in the evaluation history storage unit 14 by referring to the evaluation history storage unit 14. Here, as an example, the evaluation history acquisition unit 155 acquires the evaluation history shown in FIG.

  The operation extraction unit 156 has an operation correlated with the evaluation value output by one or a plurality of evaluation models based on the operation history acquired by the operation history acquisition unit 151 and the evaluation history acquired by the evaluation history acquisition unit 155. To extract. The process of extracting the operation is not limited to a specific mode, but the operation extraction unit 156 can extract the operation by executing the processes of the first to third extraction examples described below, for example.

(First extraction example)
The operation extraction unit 156 extracts image data whose evaluation value output by one evaluation model is a predetermined threshold value α (for example, α = 0.7) or more. Then, the operation extraction unit 156 performs the operation when a specific operation is performed on image data having a ratio equal to or greater than a predetermined threshold β (for example, β = 50%) among the extracted one or more image data. A specific operation may be extracted as an operation having a correlation with the evaluation value output by the one evaluation model. Note that the threshold values α and β in this example may be appropriate values obtained by a prior experiment and verification, or may be values arbitrarily set by a user or the like.

  In the example of FIGS. 2 and 3, the image data (image IDs “1001”, “1002”, and “1003”) of which the evaluation value output by the model A (one evaluation model) is 0.7 or more are used. The operation “Post to SNS” is executed at a rate of 2/3 (≈66.7%). Therefore, the operation extraction unit 156 extracts the “post to SNS” operation as an operation having a correlation with the evaluation value output by the model A. Also, 2/2 (= 100) for image data (two image data with image IDs “1000” and “1001”) output by model B (one evaluation model) having an evaluation value of 0.7 or more. %), “Save to Album” is being executed. Therefore, the operation extraction unit 156 extracts the “save to album” operation as an operation having a correlation with the evaluation value output by the model B.

(Second extraction example)
The operation extraction unit 156 specifies an evaluation model that outputs the maximum evaluation value for each image data, and extracts image data in which one evaluation model is specified as an evaluation model that outputs the maximum evaluation value. Then, the operation extraction unit 156 performs the operation when a specific operation is performed on image data having a ratio equal to or greater than a predetermined threshold γ (for example, γ = 50%) among the extracted one or more image data A specific operation may be extracted as an operation having a correlation with the evaluation value output by the one evaluation model. Note that the threshold value γ in this example may be an appropriate value obtained by a prior experiment and verification, or may be a value arbitrarily set by a user or the like.

  In the example of FIGS. 2 and 3, for image data (two image data with image IDs “1002” and “1003”) whose evaluation model that outputs the maximum evaluation value is model A (one evaluation model), The operation “Post to SNS” is executed at a rate of 2/2 (= 100%). Therefore, the operation extraction unit 156 extracts the “post to SNS” operation as an operation having a correlation with the evaluation value output by the model A. Also, 2/2 (= 100%) of image data (two image data with image IDs “1000” and “1001”) whose evaluation model that outputs the maximum evaluation value is model B (one evaluation model). ) At the rate of “Save to Album”. Therefore, the operation extraction unit 156 extracts the “save to album” operation as an operation having a correlation with the evaluation value output by the model B.

(Third extraction example)
There is a case where a correlated operation cannot be uniquely extracted only by an evaluation value based on one evaluation model. For example, in the method of the first or second extraction example described above, “Post to SNS” and “To Album” are operations correlated with the case where the evaluation value output by one model (hereinafter “model C”) is high. Two operations “save” are extracted, and two operations “delete” and “save to album” are extracted as operations correlated with the case where the evaluation value output by the model C is low. Two operations of “Save to Album” and “Delete” are extracted as operations correlated with the case where the evaluation value output by the model D ”is high, and the correlation between the case where the evaluation value output by the model D is low There are cases where two operations of “post to SNS” and “save to album” are extracted as certain operations. In such a case, a correlated operation cannot be uniquely specified only by considering only the evaluation value of a single model (model C or model D). Therefore, in the third extraction example, the operation extraction unit 156 extracts one correlated operation based on a combination of evaluation values output by a plurality of (here, two) evaluation models.

  For example, the operation extraction unit 156 extracts image data for each combination pattern of evaluation values output by a plurality of evaluation models (here, models C and D). Here, as an example, the operation extraction unit 156 has an evaluation value of the model C in which the evaluation value of the model C is equal to or greater than the predetermined threshold t1 and the evaluation value of the model D is equal to or greater than the predetermined threshold t2, and the evaluation value of the model C And the evaluation value of the pattern C, the evaluation value of the model D being less than the threshold value t2, the evaluation value of the model C being less than the threshold value t1, and the evaluation value of the model D being the threshold value t2 or more. Image data corresponding to each pattern D that is less than the threshold value t1 and whose model D evaluation value is less than the threshold value t2 is extracted.

  Subsequently, the operation extraction unit 156 performs specific processing for image data having a ratio equal to or greater than a predetermined threshold γ (for example, γ = 50%) among the one or more image data extracted for one pattern (for example, pattern A). When an operation is being executed, the specific operation is extracted as an operation correlated with the one pattern. Thereby, the operation extraction unit 156 can extract one operation (for example, an operation “save to album”) as an operation correlated with the pattern A, for example. Similarly, the operation extraction unit 156 performs an operation “post to SNS” as an operation correlated with the pattern B, an operation “delete” as an operation correlated with the pattern C, and an operation “correlate with the pattern D” “to album”. Each “save” operation can be extracted. In this way, based on a combination of evaluation values of multiple evaluation models, even if there is no way to uniquely identify a correlated operation using only one evaluation model, the correlated operation can be uniquely identified. Can do. Thereby, the output unit 18 described later can output information indicating one operation as information indicating a recommended operation. As a result, it is possible to recommend one appropriate operation (an operation that is likely to be performed by the user) to the user.

  The second association unit 157 generates association information in which one or a plurality of evaluation models and the operation extracted by the operation extraction unit 156 are associated. When the operation extraction unit 156 extracts an operation according to the first and second extraction examples described above, the second association unit 157 associates one evaluation model with the operation extracted by the operation extraction unit 156. Generate information. In the above example, the operation extraction unit 156 generates association information that associates the model A with the “post to SNS” operation, and association information that associates the model B with the “store to album” operation. On the other hand, when the operation extraction unit 156 extracts an operation according to the above-described third extraction example, the second association unit 157 includes a plurality (here, two) of evaluation models and the operations extracted by the operation extraction unit 156. Is generated. Specifically, the second associating unit 157 includes each pattern of the combination of the evaluation values of the model C and the model D (the four patterns A to D described above as an example in the present embodiment) and the operation extracted for each pattern. Generate association information that associates. In the above example, the second associating unit 157 associates the pattern A with the “save to album” operation, the association information with the pattern B associated with the “post to SNS” operation, the pattern C, and the “delete” And the association information that associates the pattern D with the “save to album” operation. The second association unit 157 stores the association information generated in this way in the association information storage unit 16.

  According to the second process described above, by extracting an operation having a correlation with an evaluation value output by one or more existing evaluation models, one or more existing ones can be generated without generating a new evaluation model. The association information that associates the evaluation model with the extracted operation can be generated.

  In the above-described third extraction example, the operation extraction unit 156 may extract a correlated operation for each pattern of combinations of evaluation values of three or more evaluation models. In this case, the second associating unit 157 generates association information that associates a combination pattern of three or more evaluation models with an operation extracted for the pattern.

  Moreover, the generation unit 15 may execute the following processing instead of executing the processing of the operation extraction unit 156 and the second association unit 157. That is, the generation unit 15 determines whether each evaluation value of the plurality of evaluation models and the executed operation are based on the operation history acquired by the operation history acquisition unit 151 and the evaluation history acquired by the evaluation history acquisition unit 155. Acquired image data is acquired as learning data (correct answer data). Then, the generation unit 15 inputs the evaluation values of a plurality of evaluation models as explanatory variables by executing machine learning (for example, decision tree analysis) using the learning data acquired in this way, A classification model (for example, a decision tree) that outputs an operation as an objective variable is generated. The classification model generated in this way corresponds to association information in which evaluation values of a plurality of evaluation models input as explanatory variables are associated with one operation output as an objective variable.

  In the first and second processes (first and second extraction examples) described above, an example in which the evaluation model is associated with a specific operation when the evaluation value output by the evaluation model is high has been described. There may be an operation having a tendency to be easily executed when the evaluation value output by the evaluation model is low. Therefore, for example, in the first process described above, the first association unit 154 extracts image data whose evaluation value output by the evaluation model is equal to or less than a predetermined threshold, and the same as described above based on the extracted image data. The process may be executed. The second associating unit 157 may execute the same process as described above, focusing on the evaluation model that outputs the lowest evaluation value for each image data in the second process described above.

  The association information storage unit 16 stores the association information generated by the generation unit 15 (the first association unit 154 or the second association unit 157). Specifically, the association information storage unit 16 stores, for example, one or a plurality of set information in which the evaluation model ID and the operation content are associated with each other. However, the data structure of the association information stored in the association information storage unit 16 is not limited to the above example. For example, the association information generated by the third extraction example of the second process described above associates information indicating a combination pattern of evaluation values of a plurality of evaluation models with the operation content corresponding to the pattern. Can be stored in the association information storage unit 16 as information. The association information storage unit 16 can also store information indicating a classification model (for example, a decision tree) generated by the generation unit 15 as association information.

  The evaluation unit 17 calculates the evaluation value of one image data using the evaluation model stored in the evaluation model storage unit 12. Here, one image data is, for example, image data selected as an evaluation target by the user. In this embodiment, as an example, the evaluation unit 17 calculates an evaluation value for each evaluation model for one image data based on each evaluation model stored in the evaluation model storage unit 12, and outputs the calculation result to the output unit 18. Output.

  The output unit 18 acquires the evaluation value for each evaluation model calculated by the evaluation unit 17 and refers to the association information stored in the association information storage unit 16. Then, the output unit 18 recommends, for one image data, information indicating an operation associated with the evaluation model in the association information stored in the association information storage unit 16 according to the evaluation value calculated by the evaluation unit 17. Is output as information indicating the operation to be performed.

  For example, consider the case where the association information storing the association between the model A and the “post to SNS” operation is stored in the association information storage unit 16. In this case, for example, the output unit 18 determines whether or not the evaluation value of one image data output by the model A is greater than or equal to a predetermined threshold (for example, 0.7). When the output unit 18 determines that the value is equal to or greater than a predetermined threshold value, the information indicating the recommended operation for one image data is the information indicating the “post to SNS” operation associated with the model A in the association information. May be output as Alternatively, when the evaluation model that outputs the maximum evaluation value for one image data is the model A, the output unit 18 displays information indicating the “post to SNS” operation associated with the model A for the one image data. Information indicating recommended operations may be output. However, the method in which the output unit 18 determines whether to output information indicating the operation associated with the evaluation model in the association information according to the evaluation value calculated by the evaluation unit 17 is limited to the above example. Absent. For example, the output unit 18 may execute processing that combines the above two determinations. That is, in the output unit 18, the evaluation model that outputs the maximum evaluation value for one image data is the model A, and the evaluation value of the one image data output by the model A is a predetermined threshold value (for example, 0. 7) When the above is true, information indicating the “post to SNS” operation associated with the model A may be output as information indicating a recommended operation for one image data. Further, the output unit 18 refers to the association information generated by the third extraction example of the second process described above (information that associates a combination pattern of evaluation values of a plurality of evaluation models with one operation). Thus, information indicating an operation associated with a pattern of combinations of evaluation values of a plurality of evaluation models for one image data may be output as information indicating a recommended operation.

  The output destination to which the output unit 18 outputs information indicating the recommended operation is not particularly limited. For example, the output unit 18 may display and output information indicating a recommended operation on a display or the like of the information processing apparatus 10. More specifically, the output unit 18 may display and output a menu screen or the like for allowing the user to select whether or not to perform the operation, together with information indicating the recommended operation. In this case, the user can select whether or not to perform a recommended operation. Thereby, it becomes possible to perform the recommended operation based on the user's intention.

  On the other hand, for example, when a setting for permitting automatic execution is set in advance by the user, the output unit 18 may directly output information indicating a recommended operation to an application or the like that executes the operation. Good. In this case, a recommended operation can be automatically executed by the application or the like. This saves the user from having to confirm whether or not to perform the recommended operation.

  An example of the operation of the information processing apparatus 10 will be described with reference to the flowcharts shown in FIGS. FIG. 4 is a flowchart illustrating an example of the operation of the information processing apparatus 10. FIG. 5 is a flowchart showing a procedure of a first process (first example of the process of step S1 in FIG. 4) for generating association information. FIG. 6 is a flowchart showing the procedure of the second process (second example of the process of step S1 in FIG. 4) for generating association information.

  As illustrated in FIG. 4, first, the generation unit 15 generates association information that associates an evaluation model and a user operation on image data and an operation correlated with an evaluation value output by the evaluation model. (Step S1). With reference to FIG.5 and FIG.6, the example (1st process and 2nd process) of the process which the production | generation part 15 performs in order to produce | generate association information in step S1 is demonstrated.

  An example of the procedure of the first process will be described with reference to FIG. First, the operation history acquisition unit 151 acquires an operation history (see FIG. 2) from the operation history storage unit 13 (step S101). Subsequently, the extraction unit 152 selects an operation (one operation) that is a target for generating association information (step S102), and extracts image data on which the selected operation has been executed as a positive example (step S103). The image data for which the selected operation has not been executed is extracted as a negative example (step S104). For example, when the operation history shown in FIG. 2 is acquired and the operation “save to album” is selected as a target for generating association information, image data with image IDs “1000” and “1001” are extracted as positive examples. Then, image data with image IDs “1002” and “1003” are extracted as negative examples.

  Subsequently, the evaluation model generation unit 153 generates a new evaluation model by executing machine learning using the positive example and the negative example extracted by the extraction unit 152. (Step S105). The evaluation model generated by the evaluation model generation unit 153 is stored in the evaluation model storage unit 12.

  Subsequently, the first association unit 154 generates association information that associates the evaluation model generated by the evaluation model generation unit 153 with the selected operation (here, “store to album” operation) ( Step S106). The association information generated by the first association unit 154 is stored in the association information storage unit 16.

  With reference to FIG. 6, an example of the procedure of the second process will be described. First, the operation history acquisition unit 151 acquires an operation history (see FIG. 2) from the operation history storage unit 13 (step S201). Further, the evaluation history acquisition unit 155 acquires the evaluation history (see FIG. 3) from the evaluation history storage unit 14 (step S202).

  Subsequently, the operation extraction unit 156 correlates with the evaluation value output by one or a plurality of evaluation models based on the operation history acquired by the operation history acquisition unit 151 and the evaluation history acquired by the evaluation history acquisition unit 155. A certain operation is extracted (step S203). For example, the operation extraction unit 156 extracts an operation by executing any one of the processes of the first to third extraction examples described above. For example, when the processing of the first or second extraction example described above is executed, in the example of the operation history and the evaluation history in FIGS. 2 and 3, the operation extraction unit 156 performs an operation “post to SNS” as a model A. Is extracted as an operation having a correlation with the evaluation value output by, and the operation “save to album” is extracted as an operation having a correlation with the evaluation value output by the model B.

  Subsequently, the second association unit 157 generates association information in which one or a plurality of evaluation models are associated with the operation extracted by the operation extraction unit 156 (step S204). For example, when the processing of the first or second extraction example described above is executed, the second association unit 157 associates the model A with the “post to SNS” operation, the model B, and “to album”. Correlation information associated with the operation of “save” is generated. The association information generated by the second association unit 157 is stored in the association information storage unit 16. The association information stored in the association information storage unit 16 is referred to by the output unit 18 described later.

  Returning to FIG. 4, in step S <b> 2, the evaluation unit 17 calculates an evaluation value of one image data selected as an evaluation target. For example, the evaluation unit 17 calculates an evaluation value for each evaluation model using each evaluation model stored in the evaluation model storage unit 12 for one image data.

  Subsequently, the output unit 18 uses the association information stored in the association information storage unit 16 according to the evaluation value calculated by the evaluation unit 17 to display information indicating an operation associated with the evaluation model for one image data. Output as recommended operation information (step S3). For example, for the image data whose evaluation value by the model A is equal to or greater than a predetermined threshold, the output unit 18 performs a recommended operation with information indicating the “post to SNS” operation associated with the model A in the association information. It is output as information to show. For example, the output unit 18 may display and output a menu screen for allowing the user to select whether or not to perform a recommended operation on a display or the like, or output information indicating the recommended operation to an application or the like. The recommended operation may be executed by the application.

  An information processing program P for causing a computer to function as the information processing apparatus 10 will be described with reference to FIG.

  The information processing program P includes a main module P10, a data storage module P11, an evaluation model storage module P12, an operation history storage module P13, an evaluation history storage module P14, a generation module P15, an association information storage module P16, an evaluation module P17, and an output module. P18 is provided. The main module P10 is a part that comprehensively controls processing as the information processing apparatus 10. Functions realized by executing the data storage module P11, the evaluation model storage module P12, the operation history storage module P13, the evaluation history storage module P14, the generation module P15, the association information storage module P16, the evaluation module P17, and the output module P18. Are the data storage unit 11, the evaluation model storage unit 12, the operation history storage unit 13, the evaluation history storage unit 14, the generation unit 15, the association information storage unit 16, the evaluation unit 17, and the output unit 18 of the information processing apparatus 10, respectively. This is the same as the function.

  The information processing program P is provided by a recording medium such as a CD-ROM, a DVD, or a ROM, or a semiconductor memory, for example. The information processing program P may be provided via a network as a computer data signal superimposed on a carrier wave.

  According to the information processing apparatus 10 described above, according to the evaluation value output by the evaluation model for one image data, a user operation correlated with the evaluation value of the evaluation model (for example, the evaluation value of the evaluation model) Operation that tends to be performed by the user when the image data is high) can be output as a recommended operation for the one image data. Therefore, according to the information processing apparatus 10, it is possible to output an operation on the image data according to the user characteristics (preference, behavior characteristics, etc.).

  In addition, the block diagram (FIG. 1) used for description of the said embodiment has shown the block of the functional unit. These functional blocks (components) are realized by any combination of hardware and / or software. Further, the means for realizing each functional block is not particularly limited. That is, each functional block may be realized by one device physically and / or logically coupled, and two or more devices physically and / or logically separated may be directly and / or indirectly. (For example, wired and / or wirelessly) and may be realized by these plural devices.

  For example, the information processing apparatus 10 in the above embodiment may function as a computer that performs processing of the information processing apparatus 10 in the above embodiment. FIG. 8 is a diagram illustrating an example of a hardware configuration of the information processing apparatus 10 according to the present embodiment. The information processing apparatus 10 described above may be physically configured as a computer apparatus including a processor 1001, a memory 1002, a storage 1003, a communication apparatus 1004, an input apparatus 1005, an output apparatus 1006, a bus 1007, and the like.

  In the following description, the term “apparatus” can be read as a circuit, a device, a unit, or the like. The hardware configuration of the information processing apparatus 10 may be configured to include one or a plurality of each device illustrated in FIG. 8, or may be configured not to include some devices.

  Each function in the information processing apparatus 10 reads predetermined software (program) on hardware such as the processor 1001 and the memory 1002, so that the processor 1001 performs an operation and performs communication by the communication apparatus 1004, memory 1002, and storage 1003. This is realized by controlling the reading and / or writing of data.

  For example, the processor 1001 controls the entire computer by operating an operating system. The processor 1001 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic device, a register, and the like.

  Further, the processor 1001 reads a program (program code), a software module, and / or data from the storage 1003 and / or the communication device 1004 to the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least a part of the operations described in the above embodiments is used. For example, the generation unit 15 of the information processing apparatus 10 may be realized by a control program stored in the memory 1002 and operated by the processor 1001, and may be similarly realized for the other functional blocks illustrated in FIG. 1. . Although the above-described various processes have been described as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. Note that the program may be transmitted from a network via a telecommunication line.

  The memory 1002 is a computer-readable recording medium, and includes, for example, at least one of ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), and the like. May be. The memory 1002 may be referred to as a register, a cache, a main memory (main storage device), or the like. The memory 1002 can store a program (program code), a software module, and the like that can be executed to perform the information processing method according to the above-described embodiment (the procedure illustrated in the flowcharts of FIGS. 4 to 6).

  The storage 1003 is a computer-readable recording medium such as an optical disc such as a CD-ROM (Compact Disc ROM), a hard disc drive, a flexible disc, a magneto-optical disc (eg, a compact disc, a digital versatile disc, a Blu-ray). (Registered trademark) disk, smart card, flash memory (for example, card, stick, key drive), floppy (registered trademark) disk, magnetic strip, and the like. The storage 1003 may be referred to as an auxiliary storage device. The storage medium described above may be, for example, a database, a server, or other suitable medium including the memory 1002 and / or the storage 1003.

  The communication device 1004 is hardware (transmission / reception device) for performing communication between computers via a wired and / or wireless network, and is also referred to as a network device, a network controller, a network card, a communication module, or the like.

  The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that accepts an input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that performs output to the outside. The input device 1005 and the output device 1006 may have an integrated configuration (for example, a touch panel).

  The devices such as the processor 1001 and the memory 1002 are connected by a bus 1007 for communicating information. The bus 1007 may be configured with a single bus or may be configured with different buses between apparatuses.

  The information processing apparatus 10 includes hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), and a field programmable gate array (FPGA). A part or all of each functional block may be realized by the hardware. For example, the processor 1001 may be implemented by at least one of these hardware.

  Although the present invention has been described in detail above, it will be apparent to those skilled in the art that the present invention is not limited to the embodiments described herein. The present invention can be implemented as a modified and changed mode without departing from the spirit and scope of the present invention defined by the description of the scope of claims. Therefore, the description of the present specification is for illustrative purposes and does not have any limiting meaning to the present invention.

  The processing procedures, sequences, flowcharts, and the like of each aspect / embodiment described in this specification may be switched in order as long as there is no contradiction. For example, the methods described herein present the elements of the various steps in an exemplary order and are not limited to the specific order presented.

  Input / output information or the like may be stored in a specific location (for example, a memory) or may be managed by a management table. Input / output information and the like can be overwritten, updated, or additionally written. The output information or the like may be deleted. The input information or the like may be transmitted to another device.

  The determination may be performed by a value represented by 1 bit (0 or 1), may be performed by a true / false value (Boolean: true or false), or may be compared with a numerical value (for example, a predetermined value) Comparison with the value).

  Each aspect / embodiment described in the present specification may be used alone, may be used in combination, or may be used by switching according to execution.

  Software, whether it is called software, firmware, middleware, microcode, hardware description language, or other names, instructions, instruction sets, codes, code segments, program codes, programs, subprograms, software modules , Application, software application, software package, routine, subroutine, object, executable file, execution thread, procedure, function, etc. should be interpreted broadly.

  Also, software, instructions, etc. may be transmitted / received via a transmission medium. For example, software may use websites, servers, or other devices using wired technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and / or wireless technology such as infrared, wireless and microwave. When transmitted from a remote source, these wired and / or wireless technologies are included within the definition of transmission media.

  Information, signals, etc. described herein may be represented using any of a variety of different technologies. For example, data, commands, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description are voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these May be represented by a combination of

  Note that the terms described in this specification and / or terms necessary for understanding this specification may be replaced with terms having the same or similar meaning.

  In addition, information, parameters, and the like described in this specification may be represented by absolute values, may be represented by relative values from predetermined values, or may be represented by other corresponding information. .

  The names used for the above parameters are not limiting in any way. Further, mathematical formulas and the like that use these parameters may differ from those explicitly disclosed herein.

  As used herein, the phrase “based on” does not mean “based only on,” unless expressly specified otherwise. In other words, the phrase “based on” means both “based only on” and “based at least on.”

  Any reference to elements using the designations “first”, “second”, etc. as used herein does not generally limit the amount or order of those elements. These designations can be used herein as a convenient way to distinguish between two or more elements. Thus, a reference to the first and second elements does not mean that only two elements can be employed there, or that in some way the first element must precede the second element.

  As long as “including”, “including”, and variations thereof, are used in the specification or claims, these terms are inclusive of the term “comprising”. Intended to be Further, the terms “or” and “or” as used herein or in the claims are not intended to be exclusive ORs.

  In this specification, a device is intended to include a plurality of devices unless the context or technology clearly indicates that there is only one device.

  Throughout this disclosure, the plural is included unless the context clearly indicates one.

  DESCRIPTION OF SYMBOLS 10 ... Information processing apparatus, 11 ... Data storage part, 12 ... Evaluation model storage part, 13 ... Operation history storage part, 14 ... Evaluation history storage part, 15 ... Generation part, 16 ... Association information storage part, 17 ... Evaluation part, DESCRIPTION OF SYMBOLS 18 ... Output part, 151 ... Operation history acquisition part, 152 ... Extraction part, 153 ... Evaluation model production | generation part, 154 ... 1st correlation part, 155 ... Evaluation history acquisition part, 156 ... Operation extraction part, 157 ... 2nd correlation part , 1001 ... Processor, 1002 ... Memory, 1003 ... Storage, 1004 ... Communication device, 1005 ... Input device, 1006 ... Output device, 1007 ... Bus, P ... Information processing program, P11 ... Data storage module, P12 ... Evaluation model storage module , P13 ... operation history storage module, P14 ... evaluation history storage module, P15 ... generation module, P16 ... association information Storage module, P17 ... evaluation module, P18 ... output module.

Claims (4)

An evaluation model that inputs a predetermined feature amount of image data and outputs an evaluation value of the image data, and a user operation on the image data, and an operation correlated with the evaluation value output by the evaluation model A generation unit for generating associated association information;
An evaluation unit that calculates an evaluation value of one image data using the evaluation model;
An output unit that outputs information indicating an operation associated with the evaluation model in the association information as information indicating a recommended operation for the one image data according to the evaluation value calculated by the evaluation unit;
An information processing apparatus comprising: The generator is
An operation history acquisition unit that acquires an operation history in which image data and an operation executed by the user on the image data are associated;
An extraction unit that refers to the operation history, extracts image data in which one operation is performed as a positive example, and extracts image data in which the one operation is not performed as a negative example;
An evaluation model generation unit that generates an evaluation model by executing machine learning using positive examples and negative examples extracted by the extraction unit;
A first associating unit that generates the association information associating the evaluation model generated by the evaluation model generating unit with the one operation;
The information processing apparatus according to claim 1, comprising: The generator is
An operation history acquisition unit that acquires an operation history in which image data and an operation executed by the user on the image data are associated;
An evaluation history acquisition unit for acquiring an evaluation history in which image data, an evaluation model, and an evaluation value output by the evaluation model for the image data are associated;
Based on the operation history and the evaluation history, an operation extraction unit that extracts an operation correlated with an evaluation value output by one or more evaluation models;
A second association unit that generates the association information that associates the one or more evaluation models with the operation extracted by the operation extraction unit;
The information processing apparatus according to claim 1, further comprising: Computer
An evaluation model that inputs a predetermined feature amount of image data and outputs an evaluation value of the image data, and a user operation on the image data, and an operation correlated with the evaluation value output by the evaluation model A generation unit for generating associated association information;
An evaluation unit that calculates an evaluation value of one image data using the evaluation model;
An output unit that outputs information indicating an operation associated with the evaluation model in the association information as information indicating a recommended operation for the one image data according to the evaluation value calculated by the evaluation unit;
Information processing program to function as.

Images