JP7581153B2 - DISPLAY CONTROL DEVICE, DISPLAY CONTROL METHOD, AND DISPLAY CONTROL PROGRAM - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act Announced at the Unicharm New Product Launch held at Shinagawa Prince Hotel on November 26, 2019

Article 30, paragraph 2 of the Patent Act applies November 26, 2019 https://www.webarphoto.com/unicharm/5918313810/index.html Digital technology related to this application as of November 26, 2019 is published.

The present invention relates to a display control device, a display control method, and a display control program.

Conventionally, with the advancement of information processing, technologies have been proposed for providing information to users in various ways. For example, a technology has been proposed that promotes product sales by displaying a virtual image of a product superimposed on an image taken by the user.

JP 2012-181688 A

However, the above-mentioned conventional technology does not necessarily allow a user to effectively recognize whether a disposable wearable article is suitable for the user before purchasing it. For example, the above-mentioned conventional technology recognizes facial features in images sequentially acquired from an imaging unit, and based on the positions of the facial features included in the recognition results, displays a virtual makeup image that tracks the face in the image and is superimposed on the image.

As such, the above-mentioned conventional technologies are specialized for makeup assistance, and do not display disposable wearable items such as masks by superimposing them on an image, so they are not necessarily able to effectively allow users to recognize whether a disposable wearable item is suitable for them before purchasing it.

The present application has been made in consideration of the above, and aims to allow users to effectively determine whether a disposable wearable article is suitable for them before they purchase it.

The display control device according to the present application is characterized in that it has an acquisition unit that acquires article information indicating a shape of a wearable article corresponding to a content selected by a user from among specific wearable articles related to wearable articles identified by identification information input by the user, and a display control unit that displays an image of the wearable article having the shape indicated by the article information superimposed on a captured image of the user in a state in which the user is virtually wearing the wearable article.

According to one aspect of the embodiment, it is possible to effectively allow a user to recognize whether a disposable wearable article is suitable for them before purchasing it.

FIG. 1 is an explanatory diagram conceptually explaining a display corner. FIG. 2 is a diagram illustrating an example (1) of a display control process according to the embodiment. FIG. 3 is a diagram illustrating an example (2) of the display control process according to the embodiment. FIG. 4 is a diagram illustrating an example (3) of the display control process according to the embodiment. FIG. 5 is a diagram illustrating an example of image comparison according to the embodiment. FIG. 6 is a diagram illustrating an example of the configuration of an information processing apparatus according to an embodiment. FIG. 7 is a flowchart showing the first half of the display control process according to the embodiment. FIG. 8 is a flowchart showing the latter half of the display control process according to the embodiment. FIG. 9 is a diagram illustrating an example of a hardware configuration.

The present specification and accompanying drawings make clear at least the following:

A display control device comprising: an acquisition unit that acquires article information indicating the shape of a disposable wearable article specified in response to information input by a user; and a display control unit that displays an image of the disposable wearable article superimposed on an image captured of the user in a state in which the user is virtually wearing the disposable wearable article having the shape indicated by the article information.

This type of display control device acquires product information indicating the shape of a disposable wearable item specified in response to information input by a user, and displays an image of the disposable wearable item superimposed on an image captured by the user in a state in which the user is virtually wearing the disposable wearable item having the shape indicated by the acquired product information, thereby allowing the user to effectively recognize whether the disposable wearable item is suitable for the user before purchasing it.

The display control device also acquires the item information indicating the shape of a disposable mask made of nonwoven fabric as the disposable wearable item.

This type of display control device obtains product information indicating the shape of a disposable mask made of nonwoven fabric as a disposable wearable item, allowing the user to effectively recognize whether or not the disposable mask is a suitable mask product before purchasing it.

The display control device also acquires, as the input information, the item information indicating the shape of the disposable wearing article identified according to identification information attached to the display shelf on which the disposable wearing article is displayed as a commodity.

This type of display control device obtains, as input information, item information indicating the shape of the disposable wearable item identified according to identification information attached to the display shelf on which the disposable wearable item is displayed as a commodity, thereby effectively increasing the opportunities to experience wearing the disposable wearable item.

The display control device also acquires the item information indicating the shape of a disposable wearable item identified by the identification information and currently displayed on the display shelf as the disposable wearable item specified according to the identification information.

According to such a display control device, the disposable wearable article specified according to the identification information is a disposable wearable article identified by the identification information, and article information indicating the shape of the disposable wearable article currently displayed on the display shelf is obtained, so that an image of the wearable article that the user is currently paying attention to can be displayed.

The display control device also acquires the item information indicating the shape of the disposable wearable item corresponding to the category selected by the user from among the wearable item identified by the identification information and the disposable wearable items that are display candidates related to the wearable item identified by the identification information as the disposable wearable item specified according to the identification information.

According to such a display control device, the disposable wearable item identified according to the identification information is the wearable item identified by the identification information, and among the disposable wearable items that are display candidates related to the wearable item identified by the identification information, item information is obtained that indicates the shape of the disposable wearable item corresponding to the category selected by the user, thereby effectively supporting the user to more easily find the disposable wearable item that the user wishes to try on.

The display control device also acquires the item information indicating the shape of a wearing article that belongs to a second category further selected by the user from among the disposable wearing articles that are candidates for display and belong to a first category selected by the user, the second category being a lower category than the first category.

This display control device obtains item information indicating the shape of a wearable item belonging to a second category further selected by the user from among the disposable wearable items that are display candidates belonging to a first category selected by the user, the second category being a lower category than the first category, and can effectively help the user more easily find a disposable wearable item that he or she wishes to try on.

The display control device also acquires the item information indicating the shape of the disposable wearable item selected by the user from among the display candidates for disposable wearable items.

This type of display control device obtains article information indicating the shape of a disposable wearable article selected by the user from among the displayed candidates, allowing the user to easily try on a disposable wearable article that meets their needs.

The display control device also acquires the item information indicating the shape of a disposable wearable item selected by the user from among the display candidates for disposable wearable items identified according to the attribute information or preferences of the user.

This display control device obtains article information indicating the shape of a disposable wearable article selected by the user from among the display candidates for disposable wearable articles identified according to the user's attribute information or preferences, allowing the user to easily experience wearing a disposable wearable article that matches his or her preferences.

The display control device also acquires, as the input information, the item information indicating the shape of a disposable wearable item identified according to the captured image of the user.

With such a display control device, the input information obtained is product information indicating the shape of a disposable wearable item identified according to an image captured by the user, which can increase the user's interest and effectively guide the user to a store selling the disposable wearable item.

The display control device also displays an image of the disposable wearable article superimposed on the captured image of the user based on the body parts of the user detected from the captured image of the user, so that the disposable wearable article having the shape indicated by the article information is positioned at an appropriate body position according to the intended use of the disposable wearable article.

This type of display control device displays an image of the disposable wearable article superimposed on the captured image so that the disposable wearable article having the shape indicated by the article information is positioned at an appropriate body position according to the intended use of the disposable wearable article based on the body parts of the user detected from the captured image of the user, thereby allowing the user to experience wearing the disposable wearable article without feeling unnatural.

The display control device controls the shape indicated by the item information according to the orientation of the user's face shown in the captured image relative to the image capturing device capturing the user's image, and displays an image of the disposable wearable item superimposed on the captured image in a manner such that the user is virtually wearing the disposable wearable item having the controlled shape.

With this type of display control device, the shape indicated by the item information is controlled according to the orientation of the user's face shown in the captured image relative to the imaging means that captures the user, and an image of the disposable wearable item with the controlled shape is displayed superimposed on the captured image in a state in which the user is virtually wearing the disposable wearable item, allowing the user to accurately visualize how the appearance of the disposable wearable item changes depending on the orientation of the face.

The display control device also controls the size of the image of the disposable wearable article according to the size of the user's face, and superimposes the image of the controlled size on the captured image.

This type of display control device controls the size of the image of the disposable wearable item according to the size of the user's face, and superimposes the image of the controlled size on the captured image, allowing the user to experience wearing the disposable wearable item without feeling uncomfortable.

In addition, when the color or pattern of the disposable wearable item is specified by the user, the display control device superimposes an image of the disposable wearable item of the specified color or pattern on the captured image as an image of the disposable wearable item.

When a user specifies a color or pattern of a disposable wearable item, such a display control device superimposes an image of the disposable wearable item with the specified color or pattern on the captured image as an image of the disposable wearable item, thereby allowing the user to effectively recognize whether or not a disposable wearable item with a certain color or pattern is suitable for the user before purchasing the item.

In addition, when the size of the disposable wearable item is specified by the user, the display control device superimposes an image of the disposable wearable item of a size corresponding to the specified size on the captured image.

When a user specifies the size of a disposable wearable item, such a display control device superimposes an image of the disposable wearable item of a size corresponding to the specified size on the captured image, thereby allowing the user to effectively recognize what size disposable wearable item is suitable for the user before purchasing the product.

The display control device also displays guidance information on a terminal device of a user who is identified based on the user information about the user and who does not have an application installed that displays an image of a disposable wearable article superimposed on the captured image, guiding the user to install the application.

This type of display control device displays guidance information on a terminal device of a user who is identified based on user information about the user and who does not have an application installed that displays an image of a disposable wearable article superimposed on a captured image, guiding the user to install the application, thereby effectively lowering the barrier to experiencing wearing disposable wearable articles in a store.

In addition, after the input information is input, if it is determined that a body part of the user that is to be virtually worn by the disposable wearable article is not included in the captured image of the user, the display control device displays guidance information to take an image including that body part.

According to such a display control device, after input information is input, if it is determined that a part of the user's body that is to be virtually worn by the disposable wearable article is not included in the image captured of the user, guidance information is displayed to take an image that includes that part, so that the image of the disposable wearable article can be controlled to be displayed in the correct manner according to the intended use of the disposable wearable article.

The display control device also displays images of the user that have been taken so far, among images on which the image of the disposable wearable article is superimposed, in a manner that allows comparison.

This type of display control device allows images taken of the user, in which images of disposable wearable items are superimposed, to be displayed in a comparative manner, allowing the user to easily compare which disposable wearable item suits them best.

The display control device also displays images of the user that have been captured in the past and have the image of the disposable wearable superimposed thereon, together with a currently captured image that has the image of the disposable wearable superimposed thereon.

This display control device displays images of the user that have been taken in the past and have images of disposable wearables superimposed on them together with images of disposable wearables that are currently being taken and have images of disposable wearables superimposed on them, allowing the user to easily compare which disposable wearables look best on them.

The display control device also displays predetermined information corresponding to the posting to a predetermined information sharing service on a captured image of the user on which an image of the disposable wearable article is superimposed.

This type of display control device allows a user to display specific information corresponding to a post to a specific information sharing service on an image captured by the user on which an image of a disposable wearable item is superimposed, thereby enabling effective advertising of disposable wearable items and businesses that sell disposable wearable items.

The display control device also displays information about the disposable wearable article as the predetermined information.

With such a display control device, information about disposable wearables can be displayed as the specified information, making it possible to effectively advertise disposable wearables and businesses that sell disposable wearables.

[Embodiment]
An example of a form for implementing a display control device, a display control method, and a display control program (hereinafter, referred to as an "embodiment") will be described in detail below with reference to the drawings. Note that the display control device, the display control method, and the display control program are not limited to this embodiment. Also, the same components in the following embodiments are given the same reference numerals, and duplicated descriptions will be omitted.

1. Overview of display control process according to embodiment
First, an overview of the display control process according to the embodiment will be described along the premise. One example of a disposable wearable article is a mask, and the demand for masks has been increasing in recent years. In addition, since masks are often used throughout the year, many people are concerned about the appearance when wearing them, and wear them as fashionably as possible. For this reason, it would be convenient if masks could be tried on before purchasing them, but since masks are hygiene products and many of them are disposable products, it is difficult to try them on before purchasing them.

The display control process according to the embodiment is an idea for solving such problems, and allows the user to simulate the appearance of wearing a mask by having the user wear a virtual image of a mask via a terminal device owned by the user (e.g., a smartphone, etc.) or a terminal device placed in a store (e.g., a tablet terminal or digital signage, etc.).

Specifically, the display control process according to the embodiment acquires article information indicating the shape of a disposable wearable article specified in response to information input by a user, and displays an image of the disposable wearable article superimposed on an image captured by the user in a state in which the user is virtually wearing the disposable wearable article having the shape indicated by the acquired article information.

This type of display control process allows users to effectively determine whether a disposable wearable item (e.g., a mask) is suitable for them before they purchase it.

As described above, the display control process according to the embodiment allows the user to virtually wear an image of a mask via a terminal device, so that the user can easily try on a mask in a store, etc. Using this as a "selling point," for example, by displaying masks of the company's own product (e.g., business T) together with a store pop-up such as "Easily try on the masks with your smartphone!", the appeal of the company's product can be increased. An image of a display corner where masks are displayed with a store pop-up attached will be shown later using FIG. 1.

2. Programs Corresponding to Display Control Processing
Next, a program corresponding to the display control process according to the embodiment will be described. The program is a program for causing a computer to execute an acquisition procedure for acquiring article information indicating a shape of a disposable wearing article specified in response to information input by a user, and a display control procedure for displaying an image of a disposable wearing article having a shape indicated by the article information acquired by the acquisition procedure, superimposed on an image captured by the user in a state in which the user virtually wears the disposable wearing article.

The application according to the embodiment, which is such a program, is introduced (installed) into so-called end-user terminal devices, such as a terminal device owned by a user (e.g., a smartphone) or a terminal device placed in a store (e.g., a tablet terminal or digital signage), or into a server device that communicates with such terminal devices.

In the following embodiment, a terminal device 10 is illustrated as an example of such a terminal device, and a server device 100 is illustrated as an example of such a server device. In addition, both the terminal device 10 and the server device 100 are devices that can be considered as examples of a display control device.

The application according to the embodiment may be a dedicated application (hereinafter referred to as "app AP11") installed in the terminal device 10 to link the terminal device 10 and the server device 100. On the other hand, the application according to the embodiment may be a general-purpose application ("app AP12") installed in the terminal device 10. For example, the app AP12 may be a web app (an application for displaying web content) that runs on a browser of the terminal device 10.

The apps AP11 and AP12 can be considered as applications for causing at least one of the terminal device 10 and the server device 100 to execute the display control process according to the embodiment. In this embodiment, the terminal device 10 and the server device 100 cooperate to perform the display control process according to the embodiment in accordance with the control of the app AP11 or the app AP12. However, for example, the terminal device 10 may independently perform the display control process according to the embodiment in accordance with the control of the app AP11 or the app AP12.

In addition, the display control process according to the embodiment performed between the terminal device 10 and the server device 100 when the app AP11 is installed, and the display control process according to the embodiment performed between the terminal device 10 and the server device 100 when the app AP12 is installed will be described separately. The former will be mainly described with reference to FIG. 3, and the latter will be mainly described with reference to FIG. 4. On the other hand, the display control process when the app AP11 is installed may be performed under the control of the app AP12. In addition, the display control process when the app AP12 is installed may be performed under the control of the app AP11.

[3. Disposable clothing items]
Next, a disposable wearing article will be described. In this embodiment, the disposable wearing article according to the embodiment is a mask. Specifically, the disposable wearing article according to the embodiment is a disposable mask made of a nonwoven fabric. On the other hand, the disposable wearing article does not necessarily have to be a mask, and may be, for example, an absorbent article for adults (adult diapers).

[4. Memory and store display]
Next, a storage unit included in the server device 100 and a display corner in which masks are displayed with in-store POPs attached will be described with reference to Fig. 1. Fig. 1 is an explanatory diagram conceptually explaining the display corner.

The left side of FIG. 1 shows an example of a product information storage unit 121 that stores various information related to masks as a product, among the storage units of the server device 100, which is an example of a display control device. The product information storage unit 121 shown in FIG. 1 has items such as "series information," "first type," "second type," "product ID," "AR information," and "stores selling the product."

"Series information" indicates series information about mask products by business operator T, which is a business operator (mask sales manufacturer) corresponding to server device 100. Products (products identified by "product ID") associated with "series information" correspond to the product series indicated by "series information". In the example of Figure 1, a mask product (mask PD11) identified by product ID "PD11" is associated with series information "New products released this spring". In this example, mask PD11 is one of the new product series (lineup) released by business operator T this spring.

The "first type" and "second type" correspond to classification information for classifying masks by purpose (e.g., for women, for children, etc.), shape (pleated, three-dimensional, etc.), various specifications (comfort when worn, breath released through the mask, etc.), etc., i.e., mask categories, and the "first type" and "second type" each cover a different range (granularity). Specifically, the "second type" is a category with a narrower granularity than the "first type." In other words, the "second type" corresponds to a lower category than the "first type." Also, for this reason, the "first type" and the "second type" have a hierarchical relationship.

In the example of the product information storage unit 121 shown in FIG. 1, "TY11", "TY12", "TY13", and "TY14" are shown as the first type. Also, in the example of the product information storage unit 121 shown in FIG. 1, the first type "TY11" is associated with "TY11-1" and "TY11-2" as the second type. Also, in the example of the product information storage unit 121 shown in FIG. 1, the first type "TY12" is associated with "TY12-1" and "TY12-2" as the second type. Also, in the example of the product information storage unit 121 shown in FIG. 1, the first type "TY13" is associated with "TY13-1" and "TY13-2" as the second type. In addition, in the example of the product information storage unit 121 shown in FIG. 1, the first type "TY14" is associated with "TY14-1" and "TY14-2" as the second type.

"Product ID" indicates identification information that identifies a mask product. "Product ID" may be any information that can identify a mask product, and examples of "product ID" include product numbers and product names (brand names). Furthermore, each mask product identified by a "product ID" can be said to be a display candidate mask product.

"AR information" is information that is the basis for generating an AR image, which is a virtual image of a mask (for example, a virtual stereoscopic image showing a mask in three dimensions), and includes at least information indicating the shape of the mask. Furthermore, "AR information" is an example of product information that indicates the shape of a mask. Furthermore, "AR information" may also include information indicating the default color or pattern of the mask. In the example of the product information storage unit 121 shown in FIG. 1, AR information "3D11" is associated with mask PD11, which is a mask product identified by product ID "PD11". In this example, AR information "3D11" includes information indicating the characteristic shape of mask PD11 as product information. For example, if mask PD11 is a pleated type mask product that has a shape specific to business operator T, AR information "3D11" includes information indicating this characteristic shape.

"Stores that sell the mask product identified by the product ID" is information that indicates a store that sells the mask product identified by the product ID. For example, "stores that sell the mask product" is a store ID, store name, store location, etc. that identifies a store that sells the mask product identified by the product ID.

Next, the in-store POP will be described. The right side of FIG. 1 shows an example of a display corner where masks are displayed with an in-store POP attached. In the example of FIG. 1, a mask PD21, which is one of the mask products managed in the product information storage unit 121, is shown being sold in a display corner CN11 installed in the store SH1. In the example of FIG. 1, an in-store POP P21 stating "Try wearing it easily with your smartphone!" is attached to the front of the display corner CN11. In addition, the in-store POP P21 shows code information QR21 as a QR code (registered trademark) that triggers the display control according to the embodiment. In addition, the code information QR21 includes, for example, a product ID that identifies the mask PD21. Therefore, when a user wants to simulate how he or she will look when wearing a mask via his or her own terminal device 10, first, the user inputs the code information QR21 into the terminal device 10 using the imaging means (imaging unit 13 shown in FIG. 6) of the terminal device 10, thereby causing the terminal device 10 to read the code information QR21.

For example, if store SH1 sells mask PD22 in addition to mask PD21, mask PD22 is displayed in display corner CN12 (see attached figure) next to display corner CN11, and display corner CN12 is provided with an in-store POP P22 (see attached figure) corresponding to mask PD22. Here, the QR code shown on the in-store POP P22 may be the same code information QR21 as the in-store POP P21, or may be another QR code including a product ID that identifies mask PD22.

Furthermore, when one wishes to perform a simulation of each different mask product, it may be troublesome to input the QR code for each mask product each time. Therefore, even if a single store displays multiple different mask products from business T, a QR code may be associated with only one of these mask products. Even if a single store displays multiple different mask products from business T but only has one QR code, it is possible to display a mask image corresponding to the mask product that the user wishes to simulate by having the user select the mask product that they wish to display from among the mask products that are candidates for display. This point will be explained in detail in FIG. 2.

5. Display control process
From here, an example of the display control process according to the embodiment will be described with reference to Fig. 2 to Fig. 4. Fig. 2 describes the display control process according to the embodiment, in which the server device 100 acquires AR information corresponding to a mask product according to input information in response to an input by a user. Fig. 3 describes an example of the display control process when the terminal device 10 operates according to the application AP11. Fig. 4 describes an example of the display control process when the terminal device 10 operates according to the application AP12.

In addition, in Figures 2 to 4, it is assumed that user U1 is using his/her terminal device 10 to simulate how he/she will look when wearing a mask. More specifically, it is assumed that user U1 enters code information QR21 shown on an in-store POP P21 to simulate how he/she will look when wearing mask PD21 placed in display corner CN11 of store SH1.

5-1. Display control process (1)
First, a process in which the server device 100 acquires AR information corresponding to a mask product according to input information by a user will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example (1) of a display control process according to an embodiment.

First, assume that user U1 inputs code information QR21, which is a QR code, into terminal device 10 using the camera of the terminal device (imaging unit 13 shown in FIG. 6). In this example, code information QR21 can be considered as information input by user U1. Furthermore, code information can also be considered as identification information attached to the display shelf on which mask products are displayed.

Then, when the terminal device 10 accepts the input of the QR code, it reads the QR code to detect the product ID indicated by the QR code (step S11). Specifically, when the terminal device 10 accepts the input of the code information QR21, it reads the code information QR21 to detect the product ID indicated by the code information QR21. In the example of FIG. 1, it is assumed that the terminal device 10 detects the product ID "PD21" indicating the mask PD21 as the product ID.

Next, the terminal device 10 transmits the detected product ID to the server device 100. For example, the terminal device 10 transmits the detected product ID to the server device 100 via a predetermined application. For example, if the application AP11 is installed in the terminal device 10, the terminal device 10 transmits the detected product ID to the server device 100 under the control of the application AP11. Also, if the application AP11 is not installed in the terminal device 10 but the application AP12 is installed (i.e., if a browser, which is a web application corresponding to the application AP12, is installed), the terminal device 10 transmits the detected product ID to the server device 100 via the application AP12. By such transmission, the server device 100 acquires the product ID from the terminal device 10 (step S12). Specifically, the server device 100 acquires the product ID "PD21" from the terminal device 10 as a result of the terminal device 10 transmitting the product ID "PD21".

If the code information QR21 includes a store ID that identifies store SH1, the terminal device 10 may transmit this store ID to the server device 100 along with the product ID. This enables the server device 100 to identify mask products from operator T among the mask products sold at store SH1 by matching the "store" in the product information storage unit 121 with the "store ID" acquired from the terminal device 10.

Next, the server device 100 identifies mask products that are candidates for display based on the product ID, and generates selection content that allows the user to select a mask product that the user wants to display from the identified mask products that are candidates for display (step S13). Specifically, the server device 100 identifies mask products that are candidates for display based on the product ID "PD21", and generates selection content that allows the user to select a mask product that the user wants to display from the identified mask products that are candidates for display.

For example, the server device 100 identifies the mask PD21 identified by the product ID "PD21" and mask products related to the mask PD21 as mask products to be displayed. According to the example of the product information storage unit 121 shown in FIG. 1, masks PD11 and PD12 are products in the same series as mask PD21, and are also related from the perspective of brand. Therefore, in the example of FIG. 2, the server device 100 extracts masks PD11 and PD12, which are mask products related to mask PD21, and as a result, masks PD11, PD12, and PD21 are identified as mask products to be displayed.

In this example, mask PD11, mask PD12, and mask PD21 are examples of masks identified according to the identification information. The process of step S13 can be conceptually expressed as the server device 100 identifying, as disposable wearable articles identified according to the identification information, wearable articles identified by the identification information, and wearable articles related to the wearable article identified by the identification information, as disposable wearable articles to be displayed. Note that the criteria for identifying related products do not need to be limited to series or brand.

Here, an example of the generation of selection content will be described using the example of the product information storage unit 121 shown in FIG. 1. For example, the server device 100 generates selection content C111 that allows the user to select either a first type "TY12" corresponding to the first category, which is a broader category (higher category) among the categories to which the mask PD21 belongs, or a first type "TY11" corresponding to the first category, which is a broader category (higher category) among the categories to which the masks PD11 and PD12 belong.

The server device 100 also generates selection content C111-1 that allows the user to select one of the second types "TY12-1" and "TY12-2" that correspond to the first type of mask PD21 and are a second type corresponding to a second category that is narrower in granularity than the first category (i.e., lower in granularity than the first category). The server device 100 also generates selection content C111-2 that allows the user to select one of the second types "TY11-1" and "TY11-2" that correspond to the first types of mask PD11 and mask PD12 and are a second type corresponding to a second category that is narrower in granularity than the first category (i.e., lower in granularity than the first category).

Furthermore, the server device 100 generates the selected content C11 so that when the first type "TY11" is selected in the selected content C111, the selected content C111 transitions to the selected content C111-2, and when the first type "TY12" is selected in the selected content C111, the selected content C111 transitions to the selected content C111-1. Specifically, the server device 100 generates the selected content C11 by combining the selected content C111, the selected content C111-1, and the selected content C111-2 so that the transition described above is realized. This allows the user U1 to narrow down the mask products that he or she wishes to display by selecting a first category and then further selecting a second category that corresponds to the selected first category.

In addition, instead of having the user select categories in a hierarchical structure in order to narrow down the mask products that the user wants to display, as in the example shown in FIG. 2, the server device 100 may simply generate list content that lists product information (e.g., product names) indicating each of the mask products that are candidates for display, and allow the user to select the mask products that the user wants to display from the list.

The server device 100 also identifies a display candidate mask product corresponding to the user U1 from among the display candidate mask products based on the user information on the user U1. The server device 100 may generate a list content in which product information (e.g., product name) indicating each of the identified display candidate mask products is displayed in a list, thereby allowing the user to select a mask product to be displayed from the list. For example, the server device 100 identifies a display candidate mask product corresponding to the user U1 from among the display candidate mask products based on attribute information (e.g., age, gender, etc.) of the user U1 or the preferences (e.g., preferred shape, color, specifications) of the user U1. For example, the server device 100 can estimate the preferences of the user U1 based on behavioral information (e.g., purchase history, browsing history) indicating the behavior of the user U1.

The server device 100 may also identify mask products as display candidates for user U1 from among the products handled at the store currently used by user U1 (store SH1 in the example of FIG. 2). For example, the server device 100 may identify the store currently used by user U1 from the location information of user U1, and identify mask products as display candidates for user U1 from among the products handled at the identified store. Also, for example, if the server device 100 has acquired a store ID along with the product ID in step S12, it may identify mask products as display candidates for user U1 from among the products handled at the store identified by the acquired store ID.

Returning to the explanation of FIG. 2, the server device 100 transmits the selected content generated in step S13 to the user U1 (step S14). Specifically, the server device 100 transmits the selected content C11 to the terminal device 10 of the user U1.

The user U1 then selects a first type that is believed to correspond to the mask product desired to be displayed from among the first types (first category) displayed in the selected content C11, and also selects a second type that is believed to correspond to the mask product desired to be displayed from among the second types (second category) displayed in the selected content C11 to which the transition has been made in response to the selection. In response to such selection, the terminal device 10 transmits category information indicating the selected category to the server device 100 (step S15). In the example of FIG. 2, if the user U1 selects the second type "TY12-1" in the selected content C111-2, the terminal device 10 transmits category information indicating the second type "TY12-1" to the server device 100.

When the server device 100 receives the category information, it acquires AR information (an example of product information) indicating the shape of a mask product belonging to the selected category based on the received category information (step S16). For example, if the server device 100 receives category information indicating the second type "TY12-1", it identifies a mask product belonging to the second type "TY12-1" and acquires AR information indicating the shape of the identified mask product from the product information storage unit 121. In the example of the product information storage unit 121 shown in FIG. 1, the server device 100 identifies a mask PD21 as a mask product belonging to the second type "TY12-1" and acquires AR information 3D21 from the product information storage unit 121 as AR information indicating the shape of the mask PD21.

If there are multiple mask products belonging to the second type "TY12-1", the server device 100 may present product information indicating these mask products to the user U1, allowing the user U1 to select the mask product he or she wishes to display.

In the above examples, the server device 100 has narrowed down the mask products desired to be displayed from among the mask products that are candidates for display, or selected the mask product desired to be displayed itself from among the mask products that are candidates for display. However, if the QR code acquired from the terminal device 10 uniquely determines a mask product from among the mask products that are candidates for display, the server device 100 may acquire AR information for this mask product. For example, if the product ID indicated by the code information QR21 is product ID "PD21", the mask product is determined to be one mask PD21 by product ID "PD21". Furthermore, mask PD21 is a mask product currently displayed in the display corner CN11 that caught the eye of user U1.

Therefore, in the example of FIG. 2 where the server device 100 acquires the product ID "PD21" in step S12, the server device 100 may acquire AR information 3D21 as the AR information for the mask PD21 without performing steps S13 to S16. Furthermore, this type of processing can be conceptually expressed as the server device 100 acquiring product information indicating the shape of a disposable wearable product that is identified by the identification information and is currently displayed on a display shelf.

Returning to the explanation of FIG. 2, the server device 100 transmits the AR information acquired in step S16 to the user U1 (step S17). Specifically, the server device 100 transmits the AR information 3D21 acquired in step S16 to the user's terminal device 10.

5-2. Display control process (2)
Up to this point, the process in which the server device 100 acquires AR information corresponding to a mask product according to the input information by a user has been described with reference to Fig. 2. From here, an example of a display control process in which the terminal device 10 operates according to the application AP11 will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example (2) of a display control process according to an embodiment. The process described below is a process following step S17 in Fig. 2.

When the terminal device 10 acquires AR information from the server device 100, the terminal device 10 generates a mask image from the acquired AR information (step S21). Specifically, when the terminal device 10 acquires AR information 3D21 from the server device 100, the terminal device 10 generates a stereoscopic image that virtually reproduces the shape indicated by the AR information 3D21 based on the shape of the mask PD21 as a mask image of the mask PD21. In the example of FIG. 3, the terminal device 10 generates a mask image GP21 as such a mask image.

In addition, in response to acquiring the AR information, the terminal device 10 dynamically activates the camera and displays guidance information on the display screen (output unit 14 shown in FIG. 6) that guides the user to capture a picture of the face so that the entire face is included within the capture range of the activated camera (step S22). For example, if the app AP11 also has a function as a camera app, the terminal device 10 dynamically activates the camera (imaging unit 13 shown in FIG. 6) in the app AP11 to display the current image captured by the camera on the display screen. On the other hand, if the app AP11 does not have a function as a camera app, the terminal device 10 may dynamically activate a predetermined camera app that has been installed in advance to display the current image captured by the camera on the display screen. The terminal device 10 activates the in-camera so that the user U1 can capture his or her own face.

In addition, the terminal device 10 displays on the display screen frame information corresponding to the range of photography by the camera and guidance information consisting of text information such as "Please take the photo so that your face is within this frame."

The camera continues to take pictures continuously as long as it is activated. Therefore, the terminal device 10 detects the facial features of user U1 based on each captured image by sequentially acquiring images captured by the camera (step S23). For example, the terminal device 10 detects the facial features of user U1 that are the parts on which a mask is virtually worn based on the captured images. For example, the terminal device 10 detects the eyes, nose, mouth, and ears of user U1. The terminal device 10 also detects the shape of user U1's jaw line and the orientation of user U1's face relative to the camera capturing the image of user U1. The terminal device 10 may also detect (estimate) the size of user U1's face.

Furthermore, the terminal device 10 may determine whether or not the facial parts of the user U1 that are the parts to virtually wear a mask are included in the current captured image based on the detection result detected as described above, and if it determines that they are not included, may display guidance information on the display screen to capture the image so that the facial parts are included. For example, if the terminal device 10 cannot detect all/part of the facial parts (e.g., the part of the face below the nose) according to the use of the mask based on the detection result, it determines that the facial parts of the user U1 are not included in the current captured image. Then, the terminal device 10 displays guidance information on the display screen, which consists of frame information corresponding to the capture range and text information such as, for example, "Your face is not captured. Please capture the image so that your face is within this frame."

Next, the terminal device 10 displays a mask image of the mask superimposed on the current captured image of the user U1, with the user U1 virtually wearing a mask having a shape indicated by the AR information based on the detected facial features (step S24). Specifically, the terminal device 10 displays a mask image GP21 of the mask PD21 superimposed on the current captured image of the user U1, with the user U1 virtually wearing a mask PD21 having a shape indicated by the AR information 3D21 based on the detected facial features. For example, the terminal device 10 displays the mask image GP21 superimposed on the current captured image so that the mask PD21 having a shape indicated by the AR information 3D21 is positioned at an appropriate face position according to the use of the mask, based on the detected facial features.

When the terminal device 10 detects the shape of the jaw line of user U1, it controls the shape of the mask image GP21 to match the detected shape. When the terminal device 10 detects the size of the face of user U1, it controls the shape of the mask image GP21 to match the detected size. When the terminal device 10 detects the orientation of the face of user U1 relative to the camera capturing the image of user U1, it controls the shape of the mask image GP21 to match the detected orientation. For example, when the user is facing to the right relative to the camera, it controls the shape of the mask image GP21 by rotating the AR information 3D21 to match the orientation.

Then, the terminal device 10 displays a mask image GP21 of the mask PD21 superimposed on the current captured image of the user U1, with the user U1 virtually wearing the mask PD21 in the controlled shape. Specifically, the terminal device 10 displays the mask image GP21 superimposed on the current captured image so that the mask PD21 in the controlled shape is positioned at an appropriate face position according to the intended use of the mask.

The terminal device 10 also displays the mask image GP21 so that it follows changes in the movement of the user U1's face. The terminal device 10 also controls the shape of the mask image GP21 in response to changes in the orientation of the user U1's face.

5-3. Display control process (2)
Next, an example of a display control process when the terminal device 10 operates according to the application AP12 will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example (3) of the display control process according to the embodiment. The process described below is a process following step S16 in Fig. 2.

As described in step S16 of FIG. 1, when the server device 100 acquires AR information from the product information storage unit 121, it generates a mask image from the acquired AR information (step S31). Specifically, when the server device 100 acquires AR information 3D21 from the product information storage unit 121 as AR information indicating the shape of the mask PD21, it generates a stereoscopic image that virtually reproduces the shape indicated by the AR information 3D21 based on the mask PD21 as a mask image of the mask PD21. In the example of FIG. 4, the server device 100 generates mask image GP21 as such a mask image, similar to the example of FIG. 3.

The server device 100 also transmits guidance information to the terminal device 10 of the user U1 to guide the user U1 to capture a picture of the face so that the entire face is included within the shooting range of the camera (step S32). For example, the server device 100 transmits guidance information to the terminal device 10 to guide the user U1 to capture a picture of the face so that the entire face is included within the shooting range of the camera.

When the terminal device 10 receives the guidance information, it dynamically activates the camera in response to the reception of the guidance information and displays the received guidance information on the display screen (output unit 14 shown in FIG. 6) (step S33). For example, if the app AP12 also has a function as a camera app, the terminal device 10 dynamically activates the camera (imaging unit 13 shown in FIG. 6) within the app AP12 to display the current image captured by the camera on the display screen. On the other hand, if the app AP12 does not have a function as a camera app, the terminal device 10 may dynamically activate a predetermined camera app that has been installed in advance to display the current image captured by the camera on the display screen. The terminal device 10 activates the in-camera so that the user U1 can take a picture of his or her own face.

In addition, the terminal device 10 displays on the display screen frame information corresponding to the range of photography by the camera and guidance information consisting of text information such as "Please take the photo so that your face is within this frame."

The camera continues to capture images continuously as long as it is activated. Therefore, the terminal device 10 sequentially transmits images captured by the camera to the server device via the app AP12. As a result, while the terminal device 10 camera is capturing images, the server device 100 sequentially acquires captured images from the terminal device 10 via the app AP (step S34).

Next, the server device 100 detects the facial features of user U1 based on each captured image (step S35). For example, the server device 100 detects the facial features of user U1 that are the parts on which a mask is virtually worn based on the captured images. For example, the server device 100 detects the eyes, nose, mouth, and ears of user U1. The server device 100 also detects the shape of user U1's jaw line and the direction of user U1's face relative to the camera capturing the image of user U1. The server device 100 may also detect (estimate) the size of user U1's face.

Based on the detection result detected as described above, the server device 100 determines whether the facial parts of the user U1, which are the parts to virtually wear the mask, are included in the current captured image. If the server device 100 determines that the facial parts are not included, the server device 100 may transmit guidance information to the terminal device 10 to take a picture so that the facial parts are included, thereby displaying the guidance information on the display screen. For example, if the server device 100 cannot detect all/part of the facial parts (e.g., the part of the face below the nose) according to the use of the mask based on the detection result, the server device 100 determines that the facial parts of the user U1 are not included in the current captured image. The server device 100 transmits guidance information consisting of frame information corresponding to the capture range and text information such as, for example, "Your face is not captured. Please capture the picture so that your face is within this frame" to the terminal device 10, thereby displaying the guidance information.

Returning to the explanation of FIG. 4, the server device 100 causes the terminal device 10 to display the mask image generated in step S31 based on the detected facial parts (step S36). Specifically, the server device 100 causes the display screen of the terminal device 10 to display the mask image GP21 based on the detected facial parts. For example, the server device 100 continuously acquires captured images from the terminal device 10, and detects facial parts in the acquired captured images each time a captured image is acquired. Therefore, each time a facial part is detected, the server device 10 transmits the detection result to the terminal device 10, thereby controlling the terminal device 10 to display the mask image GP21 based on the detection result and the current captured image.

By such control in step S36, the terminal device 10 displays a mask image of the mask indicated by the AR information superimposed on the current captured image of the user U1, with the user U1 virtually wearing the mask having the shape indicated by the AR information. Specifically, based on the detection result and the current captured image, the terminal device 10 displays a mask image GP21 of the mask PD21 superimposed on the current captured image of the user U1, with the user U1 virtually wearing the mask PD21 having the shape indicated by the AR information 3D21. For example, based on the detection result and the current captured image, the terminal device 10 displays the mask image GP21 superimposed on the current captured image so that the mask PD21 having the shape indicated by the AR information 3D21 is positioned at an appropriate face position according to the intended use of the mask.

In the example of FIG. 4, the server device 100 transmits a mask image to the terminal device 10, and the terminal device 10 acquires the mask image from the server device 100. Here, since the mask image is generated from AR information, the terminal device 10 acquiring the mask image from the server device 100 is essentially equivalent to the terminal device 10 acquiring the AR information from the server device 100.

As described above, the display control device (e.g., terminal device 10) according to the embodiment acquires article information indicating the shape of a disposable wearable article specified in response to information input by a user, and displays an image of the disposable wearable article superimposed on an image captured by the user in a state in which the user is virtually wearing the disposable wearable article having the shape indicated by the acquired article information.

The display control process performed by the display control device allows the user to simulate how they would look wearing a disposable wearable item (e.g., a mask) via the terminal device 10, allowing the user to effectively recognize whether the disposable wearable item is suitable for them before they purchase it.

In addition, such display control processing by the display control device can increase the appeal of disposable wearable articles as products, thereby contributing to sales at stores that sell disposable wearable articles.

[6. Variations]
The display control device according to the above embodiment may be implemented in various different forms other than the above embodiment, so other embodiments of the display control device will be described below.

[6-1. Terminal Device]
Up to this point, the terminal device 10 has been described as a terminal device owned by a user, but the terminal device 10 may be, for example, a terminal device (for example, a tablet terminal or a digital signage) placed in a store. Therefore, here, the display control process in the case where the terminal device 10 is a terminal device placed in a store will be described. When the terminal device 10 is a terminal device placed in a store, for example, when a user passing in front of the store enters the shooting range of the camera of the terminal device 10, a situation is considered in which the result of the display control according to the embodiment is displayed on the display screen of the terminal device 10. Specifically, when a user passing in front of the store enters the shooting range of the camera of the terminal device 10, the user is displayed on the display screen of the terminal device 10 in a state where he or she is virtually wearing a mask.

When the terminal device 10 is a terminal device placed in a store (particularly when it is a digital signage), it is difficult for the user to input a QR code into the terminal device 10. Therefore, in this example, the input information is not the identification information (QR code) attached to the display shelf on which the mask products are displayed, but an image captured dynamically by the user. Specifically, in this example, when a user passes in front of the terminal device 10 and enters the camera's shooting range, the captured image that is captured dynamically as a result becomes the input information by the user. For this reason, the terminal device 10 obtains AR information indicating the shape of the mask identified according to the captured image, using the captured image captured by the user as input information by the user.

In this way, when the terminal device 10 is a terminal device placed in a store, the terminal device 10 performs the display control process according to the embodiment, thereby increasing the interest of users and effectively guiding the users to the store.

6-2. Controlling the state of the worn article in response to user operation (1)
In the above embodiment, when the terminal device 10 detects (estimates) the size of the user's face from the captured image, the terminal device 10 controls the shape of the mask image to match the detected size, and displays the mask image after the control in a state where it is superimposed on the currently captured image. However, the terminal device 10 may control the state of the mask in response to an operation by the user.

For example, when a mask color or pattern is specified by the user, the terminal device 10 superimposes a mask image of the specified color or pattern onto the captured image as a mask image. This will be explained using the example of FIG. 3. For example, in the example of FIG. 3, the color variations of the mask PD21 are white, black, and pink. In this case, the terminal device 10 initially displays a mask image GP showing the mask PD21 in a default color (e.g., white), i.e., a white mask image GP21.

In this state, the terminal device 10 displays color selection content on the display screen, which allows the user to select any color from white, black, and pink. For example, if user U1 selects "pink" via the color selection content, the color of the currently displayed mask image GP21 is controlled (changed) from white to pink. Also, although color selection has been described as an example here, the terminal device 10 can also perform similar processing with patterns.

This display control process allows the user to simulate, via the terminal device 10, how the color or pattern of a disposable wearable item (e.g., a mask) will affect his or her own image, allowing the user to effectively recognize whether or not a disposable wearable item of a certain color or pattern is suitable for the user before purchasing it.

6-3. Controlling the state of the worn article in response to user operation (2)
Furthermore, when a mask size is specified by the user, the terminal device 10 superimposes a mask image of a size corresponding to the specified size on the captured image as a mask image. This point will be described using the example of Fig. 3. For example, the terminal device 10 displays size selection content on the display screen that allows a user to select any size from small, normal, and large. Then, for example, if the user U1 specifies "large size" via the size selection content, the size of the currently displayed mask image GP21 is controlled (changed) to a size corresponding to the "large size".

This display control process allows the user to simulate, via the terminal device 10, how their image changes depending on the size of the disposable wearable item (e.g., a mask), allowing the user to effectively recognize whether a certain size of disposable wearable item is suitable for the user before purchasing the product.

[6-4. Installation guide display]
In addition, the terminal device 10 may display guidance information to guide a user who has been identified based on user information about the user and who has not installed the app AP11, which is an application that displays an image of a disposable wearable item superimposed on a captured image, to install the app AP11.

For example, the server device 100 identifies users who have registered with the business operator T but have not yet installed the app AP11 by matching information indicating users who have registered with the business operator T with information indicating users who have installed the app AP11. The server device 100 then transmits guidance information to the terminal device 10 of the identified user to guide the user to install the app AP11. As a result, the terminal device 10 displays the guidance information to guide the user to install the app AP11 on the display screen.

The server device 100 may identify users who have accessed content related to the provider T but have not installed the app AP11 by matching cookie information indicating users who have accessed content related to the provider T with information indicating users who have installed the app AP11.

The server device 100 also identifies users who are located in the vicinity of a store indicated by "Stores that carry the product" in the product information storage unit 121, based on the user's location information. The server device 100 may then identify users who have not installed the app AP11 among the users who are located in the vicinity of a store indicated by "Stores that carry the product", by matching information about the identified users (e.g., the terminal ID of the identified users) with information indicating users who have installed the app AP11.

In this way, by displaying guidance information on a terminal device of a user who is identified based on user information about the user and who does not have an application installed that displays an image of a disposable wearable article superimposed on a captured image, guiding the user to install the application, it is possible to effectively lower the barrier to experiencing wearing disposable wearable articles in a store.

[6-5. Posting to information sharing services]
The terminal device 10 may also display predetermined information corresponding to posting to a predetermined information sharing service (SNS) on a captured image on which a mask image is superimposed, which is a captured image taken by a user. For example, when the shutter button is pressed in a state in which the mask image GP21 is superimposed and displayed on the captured image, the terminal device 10 acquires a still image from the captured image on which the mask image GP21 is superimposed, and displays predetermined information corresponding to posting to a predetermined information sharing service (SNS) on the acquired still image. For example, the terminal device 10 displays information on the mask PD21 (e.g., the company logo of the business operator T, the product of the mask PD21, the brand name of the mask PD21, etc.) as the predetermined information. The terminal device 10 may also display guidance information for guiding posting to a predetermined information sharing service as the predetermined information. The guidance information is, for example, guidance information consisting of text information such as "Take a photo and post it!"

In this way, by displaying predetermined information corresponding to a post to a predetermined information sharing service on an image taken by a user on which an image of a disposable wearable item is superimposed, it becomes possible to effectively advertise disposable wearable items and businesses that sell disposable wearable items via the predetermined information sharing service.

[6-6. Comparison]
Furthermore, the terminal device 10 may display images of the user on which a mask image is superimposed that have been taken so far so that they can be compared. For example, the terminal device 10 displays images of the user on which a mask image is superimposed that have been taken in the past together with an image currently being taken on which a mask image is superimposed. This point will be described with reference to FIG. 5. FIG. 5 is a diagram showing an example of image comparison according to the embodiment.

For example, the terminal device 10 displays a mask image GP31 of the mask PD31 superimposed on the captured image in response to the first selection made by the user U1 in the selected content C11, and when the shutter is pressed in this state, a still image SM3 is acquired in which the user U1 is virtually wearing the mask PD31.

In this state, it is assumed that the display control process has now progressed up to step S23 in Figures 2 and 3. Then, as explained in step S24 of Figure 3, the terminal device 10 displays a mask image GP21 of the mask PD21 superimposed on the current captured image of the user U1, with the user U1 virtually wearing a mask PD21 having the shape indicated by the AR information 3D21, based on the detected facial features. At this time, the terminal device 10 displays in the form shown in Figure 5.

For example, the terminal device 10 divides the display area on the display screen where the captured image is displayed into two display areas, area AR11 and area AR12. The terminal device 10 then displays a still image SM3 acquired in advance in area AR11 as a comparison image. In addition to this display, the terminal device 10 also displays in area AR12 a mask image GP21 of the mask PD21 superimposed on the current captured image captured of the user U1, with the user U1 virtually wearing a mask PD21 having a shape indicated by the AR information 3D21.

In this way, the terminal device 10 displays a previously captured image together with a currently captured image on which an image of a disposable wearable item is superimposed, thereby allowing the user to easily compare which disposable wearable item suits them best, thereby avoiding a situation in which the user finds out after purchasing a disposable wearable item that it does not suit them.

In the example of FIG. 5, the terminal device 10 divides the display area in which the captured image is displayed into two parts, an upper part and a lower part, but it may also be divided into two parts, for example, a left part and a right part. Furthermore, the terminal device 10 does not necessarily need to divide the display area in which the captured image is displayed into two parts, and it can be divided into any number of parts depending on the number of comparison images.

7. Configuration of the display control device
Next, a display control device according to an embodiment will be described with reference to Fig. 6. Fig. 6 is a diagram showing a configuration example of an information processing device according to an embodiment. Specifically, Fig. 6 is a diagram showing a configuration example of a terminal device 10 and a server device 100, which are examples of the information processing device according to an embodiment. In the following, first, a configuration example of the terminal device 10 will be described, and then a configuration example of the server device 100 will be described.

7-1. Configuration of terminal device
(Regarding communication unit 11)
The communication unit 11 is realized by, for example, a network interface card (NIC) etc. The communication unit 11 is connected to the network N by wire or wirelessly, and transmits and receives information to and from the server device 100, for example.

(Regarding the input unit 12, the imaging unit 13, and the output unit 14)
The input unit 12 is an input device that accepts various operations from a user. For example, the input unit 12 is realized by a keyboard, a mouse, operation keys, etc. The output unit 14 is a display device for displaying various information. For example, the output unit 14 is realized by a liquid crystal display, etc. Note that, when a touch panel is adopted in the terminal device 10, the input unit 12 and the output unit 14 are integrated. Also, the output unit 14 corresponds to, for example, a display screen of the terminal device 10. Also, the imaging unit 13 corresponds to a camera function that captures an image of a target. The imaging unit 13 is controlled, for example, by an application that performs a display control process according to the embodiment.

(Regarding the control unit 15)
The control unit 15 is realized by a central processing unit (CPU), a micro processing unit (MPU), etc., executing various programs (e.g., the application AP11 or the application AP12) stored in a storage device inside the terminal device 10 using a RAM as a working area. The control unit 15 is also realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

(Regarding the control unit 15)
As shown in Fig. 6, the control unit 15 has an acquisition unit 15a, a generation unit 15b, a detection unit 15c, and a display control unit 15d, and realizes or executes the functions and actions of the information processing described below. Note that the internal configuration of the control unit 15 is not limited to the configuration shown in Fig. 6, and may be other configurations as long as they perform the information processing described below. Also, the connection relationship between the processing units of the control unit 15 is not limited to the connection relationship shown in Fig. 6, and may be other connection relationships.

(Regarding the acquisition unit 15a)
The acquiring unit 15a acquires article information indicating the shape of a disposable wearing article specified in response to information input by a user. For example, the acquiring unit 15a acquires the article information from the server device 100. Also, for example, the acquiring unit 15a acquires article information (AR information) indicating the shape of a disposable mask made of nonwoven fabric as the disposable wearing article.

In addition, the acquisition unit 15a acquires, as input information, the article information indicating the shape of the disposable wearing article identified according to identification information (QR code) attached to the display shelf on which the disposable wearing article is displayed as a commodity.

For example, the acquisition unit 15a acquires item information indicating the shape of a disposable wearable item identified by the identification information and currently displayed on a display shelf as a disposable wearable item specified according to the identification information.

For example, the acquisition unit 15a may acquire, as the disposable wearing article specified according to the identification information, the wearing article identified by the identification information, and the article information indicating the shape of the disposable wearing article corresponding to the category selected by the user from among the display candidate disposable wearing articles that are related to the wearing article identified by the identification information. For example, the acquisition unit 15a acquires, from among the display candidate disposable wearing articles that belong to the first category selected by the user, the article information indicating the shape of the wearing article that belongs to a second category that is further selected by the user and is a lower category than the first category. On the other hand, the acquisition unit 15a may acquire, from among the display candidate disposable wearing articles, the article information indicating the shape of the disposable wearing article selected by the user, or may acquire, from among the display candidate disposable wearing articles specified according to the attribute information or preference of the user, the article information indicating the shape of the disposable wearing article selected by the user.

The acquisition unit 15a may also acquire, as input information, item information indicating the shape of a disposable wearable item identified according to an image captured of the user.

(Regarding the generation unit 15b)
When the generating unit 15b acquires the article information, the generating unit 15b generates an image of the disposable wearing article from the acquired article information. For example, when the generating unit 15b acquires the article information from the server device 100, the generating unit 15b generates a three-dimensional image that virtually reproduces the shape indicated by the acquired article information based on the shape of the disposable wearing article.

(Regarding the detection unit 15c)
The detection unit 15c detects body parts of the user based on captured images of the user. For example, the detection unit 15c sequentially acquires captured images by the camera (imaging unit 13) and detects body parts of the user based on each acquired captured image. For example, the detection unit 15c detects body parts of the user that are virtually worn by the disposable wearable article based on the captured images.

For example, the detection unit 15c detects the facial features of the user based on a captured image of the user. For example, the detection unit 15c detects the user's eyes, nose, mouth, and ears. The detection unit 15c also detects the shape of the user's jaw line and the direction of the user's face relative to the camera capturing the user's image. The detection unit 15c also detects the size of the user's face.

(Regarding the display control unit 15d)
The display control unit 15d displays an image of a disposable wearable article having a shape indicated by the article information superimposed on a captured image of the user in a state in which the user virtually wears the disposable wearable article. For example, the display control unit 15d displays an image of the disposable wearable article superimposed on a captured image such that the disposable wearable article having a shape indicated by the article information is positioned at an appropriate body position according to the usage purpose of the disposable wearable article based on a body part of the user detected from the captured image of the user.

For example, the display control unit 15d controls the shape indicated by the item information according to the orientation of the user's face shown in the captured image relative to the imaging means (imaging unit 13) that images the user, and displays an image of the disposable wearable item superimposed on the captured image in a manner in which the user is virtually wearing the disposable wearable item of the controlled shape.For example, the display control unit 15d controls the size of the image of the disposable wearable item according to the size of the user's face, and superimposes an image of the controlled size on the captured image.

When the color or pattern of the disposable wearing article is specified by the user, the display control unit 15d superimposes an image of the disposable wearing article of the specified color or pattern on the captured image as an image of the disposable wearing article. For example, the display control unit 15d displays color selection content that allows an arbitrary color to be selected on the display screen. Then, the display control unit 15d controls (changes) the color of the currently displayed image of the disposable wearing article to the color specified in the color selection content. Note that the color selection content is generated by, for example, the generation unit 15b. For example, the generation unit 15b generates the color selection content based on the color variation of the disposable wearing article identified according to the identification information.

When the size of the disposable wearing article is specified by the user, the display control unit 15d superimposes an image of the disposable wearing article of a size corresponding to the specified size on the captured image. For example, the display control unit 15d displays size selection content on the display screen, which allows the user to select any size from small, normal, and large. The display control unit 15d then controls (changes) the size of the currently displayed image of the disposable wearing article to the size specified in the size selection content. The size selection content is generated by, for example, the generation unit 15b. For example, the generation unit 15b generates the size selection content based on the size variations of the disposable wearing article identified according to the identification information.

The display control unit 15d may also display guidance information on a terminal device of a user who is identified based on the user information about the user and who does not have an application installed that displays an image of a disposable wearable article superimposed on a captured image, guiding the user to install the application.

In addition, after the input information is input, if it is determined that a part of the user's body that is to be virtually worn by the disposable wearable article is not included in the captured image of the user, the display control unit 15d may display guidance information to take a photograph that includes the part.

The display control unit 15d may also display images taken in the past among images taken of the user on which an image of a disposable wearable item is superimposed, so that the images can be compared. For example, the display control unit 15d displays images taken in the past among images taken of the user on which an image of a disposable wearable item is superimposed, together with an image currently being taken on which an image of a disposable wearable item is superimposed. For this reason, the terminal device 10 may have a storage unit that stores the captured images. For example, the terminal device 10 may have a storage unit that stores a still image obtained from the captured image by pressing the shutter.

The display control unit 15d may display predetermined information corresponding to a posting to a predetermined information sharing service on an image captured by a user on which an image of a disposable wearable article is superimposed. For example, the display control unit 15d displays information related to the disposable wearable article as the predetermined information. For example, when the shutter button is pressed while an image of the disposable wearable article is superimposed and displayed, the display control unit 15d acquires a still image from the captured image on which the image of the disposable wearable article is superimposed. Then, the display control unit 15d displays predetermined information corresponding to a posting to a predetermined information sharing service (SNS) on the acquired still image. For example, the display control unit 15d displays information related to the disposable wearable article (for example, a company logo, a product name of the disposable wearable article, a brand name of the disposable wearable article, etc.) as the predetermined information. The display control unit 15d may display guidance information for guiding posting to a predetermined information sharing service as the predetermined information.

7-2. Server Device Configuration
(Regarding communication unit 110)
The communication unit 110 is realized by, for example, a NIC etc. The communication unit 110 is connected to the network N by wire or wirelessly, and transmits and receives information to and from the terminal device 10, for example.

(Regarding the storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk, an optical disk, etc. The storage unit 120 has the product information storage unit 121 described in FIG.

(Regarding the control unit 130)
The control unit 130 is realized by a CPU, an MPU, or the like executing various programs (e.g., the application AP11 or the application AP12) stored in a storage device inside the server device 100 using the RAM as a working area. The control unit 130 is also realized by an integrated circuit such as an ASIC or an FPGA.

As shown in FIG. 6, the control unit 130 has an acquisition unit 131, an identification unit 132, a generation unit 133, and a transmission unit 134, and realizes or executes the functions and actions of the information processing described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 6, and may be other configurations as long as they perform the information processing described below. Also, the connection relationship between each processing unit of the control unit 130 is not limited to the connection relationship shown in FIG. 6, and may be other connection relationships.

(Regarding the Acquisition Unit 131)
The acquiring unit 131 acquires article information indicating the shape of a disposable wearing article specified in response to information input by a user. For example, the acquiring unit 131 acquires the article information from the product information storage unit 121. Also, for example, the acquiring unit 131 acquires article information (AR information) indicating the shape of a disposable mask made of nonwoven fabric as the disposable wearing article.

In addition, the acquisition unit 131 acquires, as input information, the article information indicating the shape of the disposable wearing article identified according to identification information (QR code) attached to the display shelf on which the disposable wearing article is displayed as a commodity.

For example, the acquisition unit 131 acquires item information indicating the shape of a disposable wearable item identified by the identification information and currently displayed on a display shelf as a disposable wearable item specified according to the identification information.

For example, the acquisition unit 131 may acquire, as the disposable wearing article specified according to the identification information, the wearing article identified by the identification information, and the article information indicating the shape of the disposable wearing article corresponding to the category selected by the user from among the display candidate disposable wearing articles that are wearing articles related to the wearing article identified by the identification information. For example, the acquisition unit 131 acquires, from among the display candidate disposable wearing articles that belong to the first category selected by the user, the article information indicating the shape of the wearing article that belongs to a second category that is further selected by the user and is a lower category than the first category. On the other hand, the acquisition unit 131 may acquire, from among the display candidate disposable wearing articles, the article information indicating the shape of the disposable wearing article selected by the user, or may acquire, from among the display candidate disposable wearing articles specified according to the attribute information or preference of the user, the article information indicating the shape of the disposable wearing article selected by the user.

The acquisition unit 131 also acquires captured images from the terminal device 10.

(Regarding the identification unit 132)
The identification unit 132 identifies the display candidate disposable wearing articles based on the input information (QR code) by the user. For example, the identification unit 132 identifies the display candidate disposable wearing articles based on the product ID included in the input information. For example, the identification unit 132 identifies the disposable wearing article uniquely identified by the product ID included in the input information and the disposable wearing article related to this wearing article as the display candidate disposable wearing articles.

The identification unit 132 may also identify disposable wearable articles to be displayed based on the user's attribute information or the user's preferences.

(Regarding the generation unit 133)
The generating unit 133 generates a selection content for allowing the user to select a wearing article desired to be displayed from among the display candidate disposable wearing articles identified by the identifying unit 132. For example, the generating unit 133 generates a selection content for allowing the user to select any one of first categories that are higher than the categories to which the display candidate disposable wearing articles belong, based on the product information storage unit 121, and for allowing the user to further select any one of second categories that correspond to the selected first category and are lower than the first category. In the example of Fig. 2, the generating unit 133 generates a selection content C11.

The generating unit 133 may have the same function as the generating unit 15b. For example, the generating unit 133 may have a function of generating an image of a disposable wearable article from the article information.

(Regarding the transmission unit 134)
The transmission unit 134 transmits various information. For example, the transmission unit 134 transmits various information to the terminal device 10. For example, the transmission unit 134 transmits to the terminal device 10 the selected content, the item information, the guidance information, the information indicating the body parts detected from the user (corresponding to the detection result by the detection unit 15c) generated by the generation unit 133, and an image of the disposable wearing item generated from the item information.

(Other processing units)
In addition, the server device 100 may further have a processing unit corresponding to the detection unit 15c and a processing unit corresponding to the display control unit 15d, so as to perform the display control processing described in FIG. 4 as being performed by the server device 100.

8. Processing Procedure
Next, the procedure of the display control process according to the embodiment will be described with reference to Fig. 7 and Fig. 8. Fig. 7 describes a display control process procedure up to acquiring article information (AR information) corresponding to a disposable wearing article specified according to information input by a user, among the display control processes according to the embodiment. Fig. 8 describes a display control process procedure for superimposing an image of a disposable wearing article having a shape indicated by the acquired article information on a captured image, among the display control processes according to the embodiment. In Fig. 7 and Fig. 8, the disposable wearing article will be described as a disposable mask made of nonwoven fabric.

8-1. Processing Procedure (1)
First, a display control process procedure up to acquisition of article information (AR information) corresponding to a disposable wearing article specified in response to input information by a user will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the display control process procedure for the first half of the display control process according to the embodiment.

First, the acquisition unit 131 determines whether or not product information (product ID) associated with the input of code information has been acquired (step S101). For example, when a user inputs code information into the terminal device 10, and the terminal device 10 transmits product information detected from the code information to the server device 100, the acquisition unit 131 determines that product information associated with the input of code information has been acquired.

Then, while the acquisition unit 131 has not been able to acquire product information (product ID) accompanying the input of code information (step S101; No), it waits until it can acquire the product information. On the other hand, when the acquisition unit 131 has acquired product information (step S101; Yes), the identification unit 132 identifies a display candidate mask based on the acquired product information (step S102). For example, the identification unit 132 further narrows down the display candidate masks stored in the product information storage unit 121 based on the product information to a display candidate mask corresponding to the product information.

Next, the generating unit 133 generates selection content that allows the user to select a desired mask from the display candidate masks identified by the identifying unit (step S103). Then, the transmitting unit 134 transmits the selection content generated by the generating unit 133 to the terminal device 10 (step S104). The terminal device 10 is a terminal device owned by the user who input the code information.

In addition, in this state, the acquisition unit 131 determines whether or not a selection of any of the categories (first category, second category) indicated in the selected content has been accepted (step S105). That is, the acquisition unit 131 determines whether or not category information indicating a selected category of the categories (first category, second category) indicated in the selected content has been acquired from the terminal device 10. While the acquisition unit 131 determines that a category selection has not been accepted because it is unable to acquire category information (step S105; No), it waits until it can determine that a category selection has been accepted by acquiring category information.

On the other hand, when the acquisition unit 131 determines that a category selection has been accepted by acquiring the category information (step S105; Yes), the acquisition unit 131 acquires AR information indicating the shape of a mask belonging to the selected category from the product information storage unit 121 (step S106). Then, the transmission unit 134 transmits the AR information acquired by the acquisition unit to the terminal device 10.

8-2. Processing Procedure (2)
First, a display control process procedure for superimposing an image of a disposable wearing article having a shape indicated by acquired article information on a captured image will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the latter half of the display control process procedure of the display control process according to the embodiment.

First, the acquisition unit 15a determines whether or not the AR information has been acquired from the server device 100 (step S201). If the acquisition unit 15a has not been able to acquire the AR information from the server device 100 (step S201; No), the acquisition unit 15a waits until the AR information can be acquired.

On the other hand, if AR information is acquired by the acquisition unit 15a (step S201; Yes), the generation unit 15b generates a mask image from the AR information (step S202).

When the mask image is generated, the display control unit 15d dynamically activates the in-camera and displays guidance information to guide the user to capture a face image with the in-camera (step S203).

Next, the detection unit 15c detects the user's facial features based on the captured image taken in response to the guidance (step S204).

Then, the display control unit 15d displays a mask image of the mask superimposed on the captured image of the user, in a state in which the user is virtually wearing a mask having the shape indicated by the AR information, based on the facial features detected by the detection unit 15c (step S205).

[9. Other]
Among the above-mentioned processes, all or part of the processes described as being performed automatically may be performed manually. Also, all or part of the processes described as being performed manually may be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings may be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

The components of each device shown in the figure are conceptual and functional, and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure. Furthermore, each component may be functionally or physically distributed and integrated in any unit, in whole or in part, depending on various loads and usage conditions. Furthermore, each process described above may be executed in an appropriate combination as long as there are no contradictions.

10. Hardware Configuration
The display control device according to the embodiment described above is realized by a computer 1000 having a configuration as shown in Fig. 9, for example. Fig. 9 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and a calculation device 1030, a cache 1040, a memory 1050, an output IF (Interface) 1060, an input IF 1070, and a network IF 1080 are connected by a bus 1090.

The arithmetic device 1030 operates based on programs stored in the cache 1040 or memory 1050, or programs read from the input device 1020, and executes various processes. The cache 1040 is a cache such as a RAM that temporarily stores data used by the arithmetic device 1030 for various calculations. The memory 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, and is a memory realized by a ROM (Read Only Memory), HDD (Hard Disk Drive), flash memory, etc.

The output IF 1060 is an interface for transmitting information to be output to an output device 1010 such as a monitor or printer, which outputs various types of information, and may be realized, for example, by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (registered trademark) (High Definition Multimedia Interface). On the other hand, the input IF 1070 is an interface for receiving information from various input devices 1020 such as a mouse, keyboard, scanner, etc., and may be realized, for example, by a USB.

For example, the input device 1020 may be realized by a device that reads information from an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. The input device 1020 may also be realized by an external storage medium such as a USB memory.

The network IF 1080 has the function of receiving data from other devices via the network N and sending it to the computing device 1030, and also transmitting data generated by the computing device 1030 to other devices via the network N.

Here, the arithmetic device 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic device 1030 loads a program from the input device 1020 or the memory 1050 onto the cache 1040 and executes the loaded program. For example, when the computer 1000 functions as a display control device, the arithmetic device 1030 of the computer 1000 realizes the functions of the control unit 130 by executing the program loaded onto the cache 1040.

REFERENCE SIGNS LIST 10 Terminal device 15 Control unit 15a Acquisition unit 15b Generation unit 15c Detection unit 15d Display control unit 100 Server device 121 Product information storage unit 130 Control unit 131 Acquisition unit 132 Identification unit 133 Generation unit 134 Transmission unit

Claims (19)

an identification unit that refers to a storage device that stores, for a plurality of masks, item information indicating the shape of the mask, identification information of the mask, a series to which the mask belongs, and a category to which the mask belongs, in association with each other, and identifies a plurality of masks that belong to a different category from the mask corresponding to the identification information input by a user, among the masks that belong to the same series as the mask corresponding to the identification information ;
an acquisition unit that acquires product information indicating a shape of a mask corresponding to the content selected by the user from among the masks identified by the identification unit;
and a display control unit that displays an image of the mask superimposed on an image captured of the user in a state in which the user is virtually wearing the mask having the shape indicated by the item information. The display control device according to claim 1 , wherein the acquisition unit acquires the item information indicating a shape of a mask corresponding to a category selected by the user from among the masks identified by the identification unit. The display control device according to claim 2, characterized in that the acquisition unit acquires, as the item information, item information indicating the shape of a mask belonging to a second category further selected by the user among masks belonging to a first category selected by the user, the second category being a lower category relative to the first category. The display control device according to any one of claims 1 to 3 , characterized in that the acquisition unit acquires the item information indicating a shape of a mask selected by the user from among the masks identified by the identification unit. The display control device according to any one of claims 1 to 4, characterized in that the acquisition unit acquires the item information indicating the shape of a mask selected by the user from among the masks identified by the identification unit according to the attribute information or preferences of the user. The display control device according to any one of claims 1 to 5 , characterized in that the acquisition unit acquires, as the item information, item information that indicates a shape corresponding to the user and is included in a captured image. The display control device described in any one of claims 1 to 6, characterized in that the display control unit displays an image of the mask superimposed on the captured image so that a mask of a shape indicated by the item information is positioned at an appropriate body position depending on the intended use of the mask, based on body parts of the user detected from the captured image of the user. The display control device described in any one of claims 1 to 7, characterized in that the display control unit controls the shape indicated by the item information in accordance with the orientation of the user's face shown in the captured image relative to the imaging means that images the user, and displays an image of a mask of the controlled shape superimposed on the captured image, in a state in which the user is virtually wearing the mask. The display control device according to any one of claims 1 to 8, characterized in that the display control unit controls the size of the image of the mask in accordance with the size of the user's face, and superimposes an image of the controlled size on the captured image . The display control device according to any one of claims 1 to 9, characterized in that, when a mask color or pattern is specified by the user, the display control unit superimposes an image of a mask of the specified color or pattern onto the captured image as the image of the mask. The display control device according to any one of claims 1 to 10, characterized in that, when a size of a mask is specified by the user, the display control unit superimposes an image of a size corresponding to the specified size onto the captured image as the image of the mask . The display control device described in any one of claims 1 to 11, characterized in that the display control unit displays guidance information on a terminal device of a user identified based on user information regarding the user, the user not having installed an application that displays a mask image superimposed on the captured image , guiding the user to install the application. The display control device according to any one of claims 1 to 12, characterized in that, when it is determined that a part of the user's body that is virtually worn by the mask is not included in the captured image of the user, the display control unit displays guidance information to take a photograph that includes the part. The display control device according to any one of claims 1 to 13, characterized in that the display control unit displays images of the user that have been taken so far, among the images on which the mask image is superimposed , in a manner that allows comparison. The display control device according to claim 14, characterized in that the display control unit displays a previously taken image among images of the user on which the mask image is superimposed, together with a currently taken image on which the mask image is superimposed. The display control device according to any one of claims 1 to 15, characterized in that the display control unit displays specified information corresponding to a post made to a specified information sharing service on a captured image of the user on which the mask image is superimposed. The display control device according to claim 16 , wherein the display control unit causes information about the mask to be displayed as the predetermined information. A display control method executed by a display control device, comprising:
an identification process for identifying a plurality of masks that belong to a different category from the mask corresponding to the identification information input by the user among the masks that belong to the same series as the mask corresponding to the identification information input by the user, by referring to a storage device that stores therein item information indicating the shape of the mask, the identification information of the mask, the series to which the mask belongs, and the category to which the mask belongs, in association with each other;
An acquisition step of acquiring product information indicating a shape of a mask corresponding to the content selected by the user from among the masks identified by the identification step;
and a display control step of displaying an image of the mask superimposed on an image captured of the user in a state in which the user is virtually wearing a mask having a shape indicated by the item information. an identification step of referring to a storage device that stores, for a plurality of masks, item information indicating the shape of the mask, the identification information of the mask, the series to which the mask belongs, and the category to which the mask belongs , in association with each other, and identifying a plurality of masks that belong to a different category from the mask corresponding to the identification information input by the user, among the masks that belong to the same series as the mask corresponding to the identification information ;
An acquisition step of acquiring product information indicating a shape of a mask corresponding to the content selected by the user from among the masks identified by the identification step;
A display control program for causing a computer to execute a display control procedure for displaying an image of the mask superimposed on an image captured of the user, with the user virtually wearing the mask having the shape indicated by the item information.

Images