JP2013054598A - Image display device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display device and a program which can easily receive presentation of augmented reality.SOLUTION: An image display device comprises imaging means, identification means, generation means, and display control means. The imaging means captures a moving image. The identification means identifies an object included in the captured moving image. The generation means generates a composite image composed of an image corresponding to the identified object and the captured moving image. The display control means displays the generated composite image on the display unit.

Description

  Embodiments described herein relate generally to a video display device and a program.

  In recent years, a technique called augmented reality that uses a computer or the like to synthesize a virtual object as electronic information and present it with real information such as an image captured by an imaging device has been used in various situations. .

  By the way, in the prior art, when receiving presentation by augmented reality, the user has to select a virtual object to be combined as additional information with an image captured as actual information. Therefore, there is a need for a technology that can more easily receive presentations in augmented reality.

  The video display apparatus according to the embodiment includes an imaging unit, an identification unit, a generation unit, and a display control unit. The imaging means captures a moving image. The identification means identifies an object included in the captured moving image. The generation unit generates a composite image obtained by combining an image corresponding to the identified object and the captured moving image. The display control means displays the generated composite image on a display unit.

FIG. 1 is a block diagram illustrating a configuration of an augmented reality system according to the first embodiment. FIG. 2 is a perspective view showing a fitting room in which a fitting room terminal is installed. FIG. 3 is a block diagram showing a hardware configuration of the fitting room terminal. FIG. 4 is a diagram showing the data structure of the product file. FIG. 5 is a diagram showing the data structure of the product classification file. FIG. 6 is a block diagram illustrating a hardware configuration of the client terminal. FIG. 7 is a block diagram showing a functional configuration of the fitting room terminal according to the present embodiment. FIG. 8 is a flowchart showing a procedure of augmented reality presentation processing performed by the fitting room terminal. FIG. 9 is a diagram for explaining processing for generating a composite image in which a plurality of images and frame images captured by a camera are superimposed. FIG. 10 is a block diagram illustrating a functional configuration of the client terminal according to the present embodiment. FIG. 11 is a flowchart illustrating a procedure of augmented reality presentation processing performed by the client terminal. FIG. 12 is a diagram for explaining processing for generating a composite image in which a background image and a frame image captured by a camera are superimposed.

(First embodiment)
FIG. 1 is a block diagram illustrating a configuration of an augmented reality system according to the first embodiment. In the present embodiment, an example in which the augmented reality system 1 is applied to a store that sells merchandise such as clothing will be described.

  The augmented reality system 1 according to the present embodiment includes a computer 301, a POS (Point Of Sale) terminal 501, and a try-on that are electrically connected to each other via a wired or wireless communication line (for example, the Internet) 300 or the like. A room terminal 101, a client terminal 401, a handy terminal 200, and the like are provided.

  In the augmented reality system 1 according to the present embodiment, one or a plurality of computers 301 can function as a server that provides a predetermined service. In the augmented reality system 1, a service can be provided, for example, in the form of SaaS (Software as a Service).

  The POS terminal 501 is installed in a cashier counter (not shown) provided near the store exit. The POS terminal 501 includes a scanner 502 that reads and outputs a unique product ID for each product. The product ID is printed in the form of a code symbol such as a barcode or a two-dimensional code on an RFID (Radio Frequency Identification) tag 111 (see FIG. 2) attached to the product. The POS terminal 501 executes merchandise sales data processing based on the read information.

  A computer 301, a client terminal 401, a fitting room terminal 101, and a handy terminal 200 are connected to such a POS terminal 501 through a communication line 300.

  The computer 301 includes a storage device such as an HDD (Hard Disk Drive) as a storage device, and various files are stored in the storage device.

  The client terminal 401 is a so-called smartphone owned by a customer who uses a store where the augmented reality system 1 is introduced, and is a mobile terminal with a mobile phone and a mobile information terminal function.

  The handy terminal 200 is a portable terminal that is lent to a customer who visits the store and is operated by the customer. In the present embodiment, the handy terminal 200 is connected to the communication line 300 via the access point 201, and by accessing various files stored in the computer 301, the customer can register the product himself.

  The fitting room terminal 101 is installed in a fitting room 100 used when a customer brings in a clothing item 110 (see FIG. 2) as a product and tries on the clothing item 110 brought in. The fitting room terminal 101 captures a moving image in the fitting room 100 entered by the customer as actual information, and the captured moving image and products as additional information (clothes 110 brought into the fitting room 100 by the customer). Augmented reality presentation processing for displaying a synthesized image obtained by synthesizing these images is executed.

  In a store that is a clothing store, a plurality of fitting rooms 100 that are used by customers to try on clothing 110 (see FIG. 2) are installed. The customer considers the purchase of the clothing item 110 by bringing the clothing item 110 to be tried on into the fitting room 100 and viewing the composite image displayed by the fitting room terminal 101 or actually trying on the clothing item 110. .

  FIG. 2 is a perspective view showing a fitting room in which a fitting room terminal is installed. As shown in FIG. 2, the fitting room 100 has a rectangular parallelepiped shape with an upper surface and a part of the front surface opened. An opening portion in front of the fitting room 100 serves as an entrance 102 for a customer to enter and exit. A cloth curtain 103 is attached to the doorway 102 so as to be freely opened and closed by a curtain rail (not shown). In addition, instead of the curtain 103, for example, a door that can be freely opened and closed may be provided at the entrance 102 of the fitting room 100. In FIG. 2, since the fitting room 100 with the curtain 103 opened is shown, the inside of the fitting room 100 can be visually recognized. A full-length mirror 104 is attached to the back wall of the fitting room 100 for projecting customers during fitting. An illumination 105 that is a fluorescent lamp is attached to a position above the full-length mirror 104.

  Further, on the upper part of the full-length mirror 104, a moving image in the fitting room 100 where the customer has entered (that is, a moving image including an image of the customer who has entered the fitting room 100 and the clothing 110 brought into the fitting room 100 by the customer) A camera 109 that is an image pickup unit for picking up an image) is attached.

  A fitting room terminal 101 is attached to one side wall in the fitting room 100. The fitting room terminal 101 displays information for the customers in the fitting room 100 and receives operation inputs from the customers.

  The fitting room terminal 101 includes a flat plate-like housing 106 having a thickness. An LCD (Liquid Crystal Display) 107 as a display unit for displaying information is attached to the housing 106. The LCD 107 includes a touch panel 108 as an operation input unit that is stacked on the display surface and can input information by touch designation.

  Further, the fitting room terminal 101 incorporates a communication interface 123 (see FIG. 3) for executing data communication with the computer 301 and the like via the communication line 300.

  In addition, an RFID reader / writer 114 for performing wireless communication with the RFID tag 111 is attached below the fitting room terminal 101. The RFID reader / writer 114 is connected by wire to the fitting room terminal 101 located above the RFID reader / writer so that data communication is possible.

  The RFID tag 111 with which the RFID reader / writer 114 performs wireless communication is attached to the clothing 110 as a price tag. When the clothing 110 is brought into the fitting room 100 by the customer, the RFID reader / writer 114 performs wireless communication with the RFID tag 111 attached to the clothing 110. Therefore, the RFID reader / writer 114 makes the fitting room 100 within the communicable range.

  The RFID tag 111 is a passive tag that does not incorporate a battery, and an IC chip 112 and a coiled tag antenna 113 are embedded therein. The IC chip 112 stores product ID data that is identification information for identifying a product.

  The RFID reader / writer 114 and the RFID tag 111 communicate with each other by an electromagnetic induction method using, for example, a 13.56 MHz frequency band. That is, the RFID reader / writer 114 and the RFID tag 111 communicate with each other by using an induced voltage caused by an induced magnetic flux between the antenna coil (not shown) of the RFID reader / writer 114 and the tag antenna 113. Using such a communication state, the RFID reader / writer 114 reads the product ID stored in the IC chip 112. Then, the RFID reader / writer 114 outputs the read product ID to the CPU 119 (see FIG. 3) of the fitting room terminal 101.

  An elongated cylindrical pole 115 is erected on the back of the fitting room 100. A light emitting unit 116 as a notification unit capable of selectively emitting blue, red, and green is provided at the tip of the pole 115. The light emitting unit 116 is connected to the fitting room terminal 101 by a cable (not shown) passing through the pole 115, and is subjected to operation control by the CPU 119 (see FIG. 3) of the fitting room terminal 101. Note that the light emitting unit 116 does not perform the light emitting operation in a state where the operation control by the CPU 119 is not received.

  An infrared sensor 117 is attached to one side of the entrance / exit 102 of the fitting room 100 as a human sensor that detects a customer passing through the entrance / exit 102. This infrared sensor 117 is a so-called pyroelectric infrared sensor. That is, the infrared sensor 117 receives infrared rays emitted from the customer's human body passing through the doorway 102, converts the received infrared rays into heat, and converts the heat into electric charges by the pyroelectric effect of the element and outputs (detected output). The infrared sensor 117 is connected to the fitting room terminal 101 via a cable (not shown), and the detection output of the infrared sensor 117 is input to the CPU 119 of the fitting room terminal 101 (see FIG. 3).

  Although not shown in FIG. 2, a pressure-sensitive sensor 118 (see FIG. 3) is disposed on the floor of the fitting room 100. Although the floor of the fitting room 100 is not shown in detail, two floor boards are overlapped, and a plurality of pressure sensors 118 are arranged so as to be sandwiched between the two floor boards. Therefore, when a customer enters the fitting room 100 through the doorway 102 and steps on the floor with his / her foot, pressure is applied to one of the pressure-sensitive sensors 118. The pressure sensor 118 converts such a pressure change into an electrical signal by an internal pressure sensor and outputs (detects) the signal. The pressure sensor 118 is connected to the fitting room terminal 101 via a cable (not shown), and the detection output of the pressure sensor 118 is input to the CPU 119 of the fitting room terminal 101 (see FIG. 3).

  FIG. 3 is a block diagram showing a hardware configuration of the fitting room terminal. The fitting room terminal 101 includes a CPU (Central Processing Unit) 119 as an information processing unit that executes various arithmetic processes and controls each unit. The CPU 119 includes a ROM (Read Only Memory) 120 that fixedly stores fixed data such as a computer program, a RAM (Random Access Memory) 121 that stores variable data in a rewritable manner and is used as a work area, The flash memory 122, which is a nonvolatile memory that retains data contents even when the fitting room terminal 101 is turned off, is connected by a bus.

  The CPU 119 includes an LCD 107, a touch panel 108, a camera 109, an RFID reader / writer 114, a light emitting unit 116, an infrared sensor 117, a pressure sensor 118, and a communication interface 123, all of which are various input / output circuits (all illustrated). (Not shown) and the operation is controlled by the CPU 119. Although a plurality of pressure sensors 118 are actually provided as described above, only one pressure sensor 118 is shown in FIG.

  The flash memory 122 stores various computer programs and various files. Computer programs and files stored in the flash memory 122 are used by being copied entirely or partially into the RAM 121 when the fitting room terminal 101 is activated.

  Then, the CPU 119 of the fitting room terminal 101 accesses data to various files during processing executed according to the computer program. The accessed file is a file stored in an HDD (Hard Disk Drive) (not shown) of the computer 301. The computer 301 is connected to the fitting room terminal 101 via the communication line 300. Therefore, these files can be accessed by the CPU 119 of the fitting room terminal 101.

  4 and 5, various files (product file F1, product classification file F2, etc.) stored in the HDD (not shown) of the computer 301 and accessible by the CPU 119 of the fitting room terminal 101 are used. ).

  FIG. 4 is a diagram showing the data structure of the product file. The product file F1 is associated with a product ID that is specification information for specifying a product, the product name of the product specified by the product ID, the product classification to which the product specified by the product ID belongs, and the product ID This is a file for storing the unit price of the product specified by the product and an image (product image in this embodiment) corresponding to the product specified by the product ID.

  FIG. 5 is a diagram showing the data structure of the product classification file. The product classification file F2 stores a predetermined position and order in association with the product classification stored in the product file F1. Here, the predetermined position is a position set in advance according to the product classification. For example, the predetermined position of the product classification: “jacket” is the position of the tops of the customer (person) included in the moving image captured by the camera 109. The predetermined position of the product classification: “trousers” is the position of the bottoms of the customer (person) included in the moving image captured by the camera 109.

  The order is an order set in advance according to the product classification to which each product belongs when a plurality of images corresponding to a plurality of products are superimposed on the moving image captured by the camera 109. For example, the order of the product category: “underwear” is the order before the product category: “outerwear”.

  FIG. 6 is a block diagram illustrating a hardware configuration of the client terminal. The client terminal 401 includes a CPU 402 as an information processing unit that executes various arithmetic processes and controls each unit. The CPU 402 has a ROM 403 for fixedly storing and saving fixed data such as computer programs, a RAM 404 for storing variable data in a rewritable manner and used as a work area, and data contents even when the client terminal 401 is turned off. A flash memory 405 that is a nonvolatile memory to be held is connected by a bus.

  The CPU 402 is configured to perform data communication with the computer 301 or the like via the touch panel 408 serving as an operation input unit that can be input by touching the LCD 406 that displays information, and the display surface of the LCD 406 that is stacked. A communication interface 407 for execution, an RFID reader / writer 409 for performing wireless communication with an RFID tag 111 attached to a product such as clothing 110, and a camera 410 that is an imaging unit for capturing a moving image are all of various types. They are connected via an input / output circuit (both not shown), and their operation is controlled by the CPU 402. Note that the RFID reader / writer 409 has the same configuration as the RFID reader / writer 114 included in the fitting room terminal 101, and thus the description thereof is omitted here.

  The flash memory 405 stores various computer programs and various files. The computer programs and files stored in the flash memory 405 are used by copying all or part of them to the RAM 404 when the client terminal 401 is activated.

  Then, the CPU 402 of the client terminal 401 performs data access to various files (for example, the product file F1, the product classification file F2, etc.) during processing executed according to the computer program. The accessed file is a file stored in an HDD (not shown) of the computer 301. The computer 301 is also connected to the client terminal 101 via the communication line 300. Therefore, these files can also be accessed by the CPU 402 of the client terminal 401.

  Next, functions of the fitting room terminal 101 according to the present embodiment will be described. FIG. 7 is a block diagram showing a functional configuration of the fitting room terminal according to the present embodiment. The CPU 119 operates as a discrimination unit 701, a generation unit 702, and a display control unit 703 as illustrated in FIG. 7 by operating according to a program stored in the ROM 120 expanded in the RAM 121.

  When the customer is detected by the infrared sensor 117 and the pressure detection is output by the pressure-sensitive sensor 118, the identification unit 701 determines that the customer has entered the fitting room 100 and outputs an imaging ON signal to the camera 109. This causes the camera 109 to start an imaging operation. Then, the identification unit 701 captures a frame image (moving image) of the imaging area captured by the camera 109 and stores it in the RAM 121.

  The identification unit 701 identifies an object included in the frame image stored in the RAM 121. In the present embodiment, the identification unit 701 controls the RFID reader / writer 114 to perform wireless communication with the RFID tag 111 attached to the clothing 110, and reads the product ID data stored in the RFID tag 111. . Then, the identification unit 701 identifies the clothing item 110 specified by the read product ID as an object included in the frame image stored in the RAM 121.

  In this embodiment, the identification unit 701 identifies the clothing item 110 specified by the product ID read by the RFID reader / writer 114 as an object included in the frame image stored in the RAM 121. It is not limited. For example, the identification unit 701 refers to the product image stored in the product file F1 and reads a specific state by reading a surface state such as a hue or a surface roughness as a feature amount from a frame image stored in the RAM 121. A product is recognized (identified) as an object. Note that the identification unit 701 does not consider the outline or size of the product in order to shorten the processing time.

Recognizing an object contained in an image in this way is called generic object recognition. Regarding such general object recognition, various recognition techniques are described in the following documents.
Keiji Yanai, “Current Status and Future of General Object Recognition”, IPSJ Journal, Vol. 48, no. SIG16 [Search August 10, 2010], Internet <URL: https://mm.cs.uec.ac.jp/IPSJ-TCVIM-Yanai.pdf>

Further, techniques for performing general object recognition by dividing an image into regions for each object are described in the following documents.
Jamie Shotton et al., “Semantic Texton Forests for Image Categorization and Segmentation”, [August 10, 2010 search], Internet <URL: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1. 1.145.3036 & rep = rep1 & type = pdf>

  Alternatively, the identification unit 701 extracts characters, pictures, symbols, and other features from the frame image stored in the RAM 121 by pattern recognition or OCR (Optical Character Reader) character recognition. Then, the identification unit 701 may identify an object included in the frame image from the extracted feature.

  The generation unit 702 generates a composite image obtained by combining the frame image stored in the RAM 121 and the image corresponding to the object identified by the identification unit 701. In the present embodiment, the generation unit 702 reads, from the product file F1, an image associated with the product ID read by the RFID reader / writer 114 as an image corresponding to the object identified by the identification unit 701. Further, the generation unit 702 reads the product classification associated with the product ID read by the RFID reader / writer 114 from the product file F1.

  Next, the generation unit 702 reads a predetermined position associated with the read product classification from the product classification file F2. Then, the generation unit 702 performs transmission processing different from the display at the predetermined position on the read predetermined position in the frame image stored in the RAM 121. Further, the generation unit 702 generates a composite image by superimposing the image read from the product file F1 on the portion subjected to the transparent process.

  In addition, when a plurality of product IDs are read by the RFID reader / writer 114 (that is, when a plurality of objects are identified by the identification unit 701), the generation unit 702 is read by the RFID reader / writer 114 from the product file F1. The product classification and images associated with the plurality of product IDs are read out. Furthermore, the generation unit 702 reads the order associated with the read product classification from the product classification file F2. Then, the generation unit 702 generates a composite image by superimposing the read images on the frame images stored in the RAM 121 according to the read order.

  The display control unit 703 displays the composite image generated by the generation unit 702 on the LCD 107. Thereby, the customer who entered the fitting room 100 can confirm the image of the clothing 110 that he / she tried on without actually trying on the clothing 110 brought into the fitting room 100. Further, when the composite image is displayed on the LCD 107 (that is, when receiving the augmented reality presentation), the customer needs to select additional information (product image) to be superimposed on the actual information (the frame image stored in the RAM 121). Because it is not, you can receive the augmented reality presentation more easily.

  Next, the augmented reality presentation process performed by the fitting room terminal 101 will be described with reference to FIG. FIG. 8 is a flowchart showing a procedure of augmented reality presentation processing performed by the fitting room terminal.

  The identification unit 701 waits for a customer to enter the fitting room 100 (step S801: No). When the customer is detected by the infrared sensor 117 and the detection of the pressure is output by the pressure sensor 118, the identification unit 701 determines that the customer has entered the fitting room 100 (step S801: Yes), and enters the camera 109. An imaging on signal is output to cause the camera 109 to start an imaging operation (step S802). Then, the identification unit 701 captures a frame image constituting a moving image captured by the camera 109 and stores it in the RAM 121.

  Next, the identification unit 701 controls the RFID reader / writer 114 to perform wireless communication with the RFID tag 111 attached to the clothing 110 brought into the fitting room 100 by the customer, and is stored in the RFID tag 111. The product ID is read (step S803).

  Next, the generation unit 702 reads an image stored in association with the read product ID from the product file F1. Then, the generation unit 702 generates a composite image obtained by combining the read image and the frame image stored in the RAM 121 (step S804).

  Here, referring to FIG. 9, when a plurality of product IDs are read from RFID tags 111 attached to a plurality of clothing items 110 (for example, jackets, shirts, slacks) brought into the fitting room 100 by the customer. A composite image generation process will be described. FIG. 9 is a diagram for explaining processing for generating a composite image in which a plurality of images and frame images captured by a camera are superimposed.

  The generation unit 702 stores, from the product file F1, images (jacket image, shirt image, slack image) and product classification (jacket product classification: jacket image) stored in association with a plurality of product IDs read by the identification unit 701. “Outerwear”, shirt product category: “underwear”, slacks product category: “pants”). Next, the generation unit 702 is associated with the read product category (jacket product category: “overwear”, shirt product category: “underwear”, slacks product category: “pants”) from the product category file F2. The predetermined position (product classification: “outerwear”, predetermined position associated with “underwear”: “tops”, product classification: predetermined position associated with “pants”: “bottoms”) is read out. .

  Further, the generation unit 702 (from the product classification file F2), the order (product) corresponding to (product category of jacket: “upper”, product category of shirt: “underwear”, product category of slacks: “pants”). Classification: Order associated with “outerwear”: “3”, Product classification: Order associated with “pants”: “2”, Product classification: Order associated with “underwear”: “1” ).

  Next, the generation unit 702 reads out the predetermined position 903 read from the product classification file F2 in the frame image 901 stored in the RAM 121: “the position of the customer's tops included in the frame image 901” and the predetermined position 904: Transparent processing is performed on “the position of the customer's bottoms included in the frame image 901”. More specifically, the generation unit 702 performs object recognition, pattern recognition, or the like on the frame image 901 to thereby obtain a predetermined position 903 in the frame image 901: “the position of the customer's tops included in the frame image 901” ”And a predetermined position 904:“ the position of the customer's bottoms included in the frame image 901 ”is extracted. And the production | generation part 702 shall perform the permeation | transmission process with respect to the extracted predetermined position 903,904. Next, the generation unit 702 adds an image (a jacket image, a shirt image, and a slack image) read from the product file F1 to a portion (predetermined positions 903 and 904) subjected to the transmission processing in the frame image 901. Order read from the product category file F2 (product category: order associated with “outerwear”: “3”, product category: order associated with “pants”: “2”, product category: “underwear” Are combined in the order of “1”). In other words, the generation unit 702 superimposes a shirt image 906, a slack image 907 image, and a jacket image 908 in this order on a portion (predetermined positions 903 and 904) on which transmission processing has been performed in the frame image 901. Is generated.

  Returning to FIG. 8, when a composite image is generated by the generation unit 702, the display control unit 703 displays the generated composite image on the LCD 107 (step S805).

  As described above, according to the fitting room terminal 101 according to the present embodiment, the camera 109 that captures the moving image in the fitting room 100 and the object included in the frame image that forms the moving image captured by the camera 109 are identified. A generating unit 702 that generates a combined image obtained by combining the identifying unit 701, a frame image that forms a moving image captured by the camera 109, and an image that corresponds to the object identified by the identifying unit 701, and a generating unit 702 And a display control unit 703 that displays the composite image generated by the LCD 107 on the LCD 107, so that when the composite image is displayed on the LCD 107, the image to be combined with the frame image that constitutes the moving image captured by the camera 109 is displayed. Therefore, it is possible to receive augmented reality (composite image) more easily.

(Second Embodiment)
The present embodiment is an example in which the augmented reality presentation process in the first embodiment is executed in a client terminal. Note that the hardware configuration of the client terminal is the same as that of the first embodiment, and a description thereof will be omitted here.

  FIG. 10 is a block diagram illustrating a functional configuration of the client terminal according to the present embodiment. The CPU 402 operates according to a program stored in the ROM 403 expanded in the RAM 404, thereby functioning as an identification unit 1001, a generation unit 1002, and a display control unit 1003 as shown in FIG.

  When the customer who operates the client terminal 401 operates the touch panel 408 and requests the presentation of augmented reality, the identification unit 1001 outputs an imaging on signal to the camera 410 to start the imaging operation of the camera 410. Then, the identification unit 1001 captures a frame image (moving image) of the imaging area captured by the camera 410 and stores it in the RAM 404.

  The identification unit 1001 identifies an object included in the frame image stored in the RAM 404. In the present embodiment, the identification unit 1001 controls the RFID reader / writer 409 to perform wireless communication with the RFID tag 111 attached to a product (for example, a car displayed in a showroom or the like), and the RFID tag 111 The product ID stored in is read. Then, the identification unit 1001 identifies the product specified by the read product ID as an object included in the frame image stored in the RAM 404. Note that the identification unit 1001 can recognize an object included in the frame image using general object recognition, pattern recognition, OCR character recognition, or the like, as with the identification unit 701 according to the first embodiment.

  The generation unit 1002 generates a composite image obtained by combining the frame image stored in the RAM 404 and the image corresponding to the object identified by the identification unit 1001. In the present embodiment, the generation unit 1001 reads the product classification and the image associated with the product ID read by the RFID reader / writer 409 from the product file F1. Next, the generation unit 1002 reads a predetermined position (for example, a background portion in a frame image stored in the RAM 403) associated with the read product classification from the product classification file F2. Then, the generation unit 1002 performs transmission processing different from the display at the predetermined position on the read predetermined position in the frame image stored in the RAM 404. Further, the generation unit 1002 combines a transparent image with an image corresponding to the object identified by the identification unit 1001 (an image read from the product file F1, such as a background moving image). Is generated.

  The display control unit 1003 displays the composite image generated by the generation unit 1002 on the LCD 406. Thus, a customer who visits a car showroom or the like can confirm an image of a car running on a public road without seeing the car actually running on a public road. In addition, when the composite image is displayed on the LCD 406, it is not necessary for the customer to select additional information (background image) to be combined with the actual information (frame image stored in the RAM 404), so it is easier to present augmented reality. Can receive.

  Next, an augmented reality presentation process performed by the client terminal 101 will be described with reference to FIG. FIG. 11 is a flowchart illustrating a procedure of augmented reality presentation processing performed by the client terminal.

  The identification unit 1001 waits for the customer operating the client terminal 401 to operate the touch panel 408 to request the presentation of augmented reality (step S1101: No). When the touch panel 408 is operated and an augmented reality presentation is requested (step S1101: Yes), the identification unit 1001 outputs an imaging on signal to the camera 410 and causes the camera 410 to start an imaging operation (step S1102). Then, the identification unit 1001 captures a frame image constituting a moving image captured by the camera 410 and stores it in the RAM 404.

  Next, the identification unit 1001 controls the RFID reader / writer 409 to perform wireless communication with the RFID tag 111 attached to a product such as a car displayed in a store such as a showroom, and is stored in the RFID tag 111. The product ID is read (step S1103).

  Next, the generation unit 1002 reads an image stored in association with the read product ID from the product file F1. Then, the generation unit 1002 generates a composite image obtained by combining the read image with the frame image stored in the RAM 404 (step S1104).

  Here, a composite image generation process in the case where a product ID is read from an RFID tag 111 attached to a product (for example, a car) displayed in a showroom will be described with reference to FIG. FIG. 12 is a diagram for explaining processing for generating a composite image in which a background image and a frame image captured by a camera are superimposed.

  The generation unit 1002 reads an image (background image) and product classification (car product classification: “on-road”) stored in association with the product ID read by the RFID reader / writer 409 from the product file F1. Next, the generation unit 1002 reads a predetermined position (predetermined position: background) associated with the read commodity classification (car commodity classification: “on-road”) from the commodity classification file F2.

  Next, the generation unit 1002 performs a transparency process on the background image 1203 excluding the car image 1202 included in the frame image 1201 stored in the RAM 404. Next, the generation unit 1002 generates a composite image 1204 in which the background image 1205 read from the product file F1 is superimposed on the portion of the frame image 1201 that has been subjected to the transparent process.

  Returning to FIG. 11, when a composite image is generated by the generation unit 1002, the display control unit 1003 displays the generated composite image on the LCD 406 (step S1105).

  As described above, according to the client terminal 401 according to the present embodiment, the camera 410 that captures the moving image in the showroom and the identification unit 1001 that identifies the object included in the frame image that forms the moving image captured by the camera 410. And a generation unit 1002 that generates a composite image obtained by combining the frame image constituting the moving image captured by the camera 401 and the image corresponding to the object identified by the identification unit 1001, and the generation unit 1002. A display control unit 1003 for displaying the synthesized image on the LCD 406, so that when the synthesized image is displayed on the LCD 406, the customer selects an image to be synthesized with the frame image constituting the moving image captured by the camera 410. So you can receive the augmented reality (synthesized image) more easily.

  As described above, according to the first and second embodiments, it is possible to receive an augmented reality presentation more easily.

  Note that the program executed in the fitting room terminal 101 and the client terminal 401 of the present embodiment is provided by being incorporated in advance in a ROM or the like, but is not limited thereto. For example, the program executed in the fitting room terminal 101 and the client terminal 401 according to the present embodiment is a file in an installable format or an executable format, and is a CD-ROM, flexible disk (FD), CD-R, DVD (Digital It may be configured to be recorded on a computer-readable recording medium such as Versatile Disk).

  Furthermore, the program executed in the fitting room terminal 101 and the client terminal 401 according to the present embodiment may be configured to be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. good. Further, the program executed in the fitting room terminal 101 and the client terminal 401 of the present embodiment may be provided or distributed via a network such as the Internet.

  The program executed in the fitting room terminal 101 and the client terminal 401 according to the present embodiment has a module configuration including the above-described units (identification units 701 and 1001, generation units 702 and 1002, display control units 703 and 1003). As actual hardware, a CPU (processor) reads out and executes a program from the ROM, and the above-described units are loaded onto the main storage device. The identification units 701 and 1001, the generation units 702 and 1002, and the display control unit 703. , 1003 are generated on the main storage device.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

1 Augmented Reality System 101 Fitting Room Terminal 107,406 LCD
109,410 Camera 110 Clothing 111 RFID tag 112 IC chip 113 Tag antenna 114,409 RFID reader / writer 119,402 CPU
120,403 ROM
121,404 RAM
123,407 Communication interface 300 Communication line 301 Computer 401 Client terminal 701, 1001 Identification unit 702, 1002 Generation unit 703, 1003 Display control unit F1 Product file F2 Product classification file

JP 2007-37078 A

Claims (6)

An imaging means for capturing a moving image;
Identification means for identifying an object included in the captured moving image;
Generating means for generating a composite image obtained by combining the image corresponding to the identified object and the captured moving image;
Display control means for displaying the generated composite image on a display unit;
A video display device comprising:   The identification unit reads the specific information by wireless communication from a tag in which specific information for specifying an object included in the captured moving image is stored and the specific information is read by wireless communication, and the read specific information The video display device according to claim 1, wherein an object specified by the method is identified as an object included in the captured moving image.   The video display device according to claim 1, wherein the generation unit generates the composite image obtained by combining an image corresponding to the object identified by the identification unit at a predetermined position in the captured moving image.   The generating means combines the synthesized image obtained by synthesizing the image corresponding to the identified object at the predetermined position set in advance according to the classification to which the identified object belongs in the captured moving image. The video display device according to claim 3 to be generated.   In the case where a plurality of objects are identified by the identifying means, the generating means displays a plurality of images corresponding to the identified objects according to a predetermined order according to a classification to which the identified objects belong. The video display device according to claim 1, wherein the composite image superimposed on the captured moving image is generated. Computer
Identification means for identifying an object included in the moving image captured by the imaging unit;
Generating means for generating a composite image obtained by combining the image corresponding to the identified object and the captured moving image;
Display control means for displaying the generated composite image on a display unit;
Program to function as.

Images