JP6707342B2 - Transmission method, transmission device, and program - Google Patents

Description

The present invention relates to a visible light signal transmission method, a transmission device, and a program.

In recent home networks, in addition to the cooperation of AV home appliances by IP (Internet Protocol) connection by Ethernet (registered trademark) or wireless LAN (Local Area Network), management of power consumption corresponding to environmental problems and from outside the house The home energy management system (HEMS), which has functions such as power ON/OFF, is promoting the introduction of home appliance cooperation functions in which various home appliances are connected to a network. However, there are home appliances that do not have sufficient computing power to have a communication function, and home appliances for which it is difficult to install a communication function in terms of cost.

In order to solve such a problem, in Patent Document 1, in an optical space transmission device that transmits information to free space by using light, communication using a plurality of monochromatic light sources of illumination light is performed, so that limited transmission is performed. Among the devices, a technique for efficiently realizing communication between devices is described.

JP, 2002-290335, A

However, the above conventional method is limited to the case where the device to be applied has a three-color light source such as an illumination.

The present invention solves such a problem and provides a transmission method that enables communication between various devices including devices other than lighting.

A transmission method according to an aspect of the present invention is a transmission method for transmitting a visible light signal according to a change in luminance, and a step of determining a luminance change pattern for each symbol period by modulating the visible light signal, and a display. A common switch control step of switching a common switch, which is included in the light source group included in the light source group, for commonly lighting a plurality of light sources for representing pixels in an image, according to the brightness change pattern; Turning on the first pixel switch for turning on the first light source of the plurality of light sources included in the common switch is on and the first pixel switch is on. only in the period, to turn on the first light source, in synchronization with the symbol period, by switching to Rukoto the pixel switch, and a first pixel switch control step of transmitting the visible light signal.

Note that these comprehensive or specific aspects may be realized by a recording medium such as a system, a method, an integrated circuit, a computer program or a computer-readable CD-ROM, and the system, the method, the integrated circuit, the computer program. It may be realized by any combination of the recording medium and the recording medium.

According to the present invention, it is possible to realize a transmission method that enables communication between devices other than a lighting device.

FIG. 1 is a diagram showing an example of a method of observing the brightness of the light emitting unit in the first embodiment. FIG. 2 is a diagram showing an example of a method of observing the brightness of the light emitting unit in the first embodiment. FIG. 3 is a diagram showing an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 4 is a diagram showing an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 5A is a diagram showing an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 5B is a diagram showing an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 5C is a diagram showing an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 5D is a diagram showing an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 5E is a diagram showing an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 5F is a diagram showing an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 5G is a diagram showing an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 5H is a diagram showing an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 6A is a flowchart of the information communication method according to the first embodiment. FIG. 6B is a block diagram of the information communication device according to the first embodiment. FIG. 7 is a diagram showing an example of each mode of the receiver in the second embodiment. FIG. 8 is a diagram showing an example of a photographing operation of the receiver according to the second embodiment. FIG. 9 is a diagram showing another example of the photographing operation of the receiver in the second embodiment. FIG. 10A is a diagram showing another example of the shooting operation of the receiver in the second embodiment. FIG. 10B is a diagram illustrating another example of the imaging operation of the receiver in Embodiment 2. FIG. 10C is a diagram illustrating another example of the imaging operation of the receiver in Embodiment 2. FIG. 11A is a diagram illustrating an example of camera arrangement of a receiver in Embodiment 2. FIG. 11B is a diagram showing another example of camera arrangement of the receiver in the second embodiment. FIG. 12 is a diagram illustrating an example of a display operation of the receiver in Embodiment 2. FIG. 13 is a diagram illustrating an example of a display operation of the receiver in Embodiment 2. FIG. 14 is a diagram illustrating an example of operation of a receiver in Embodiment 2. FIG. 15 is a diagram illustrating another example of operation of the receiver in Embodiment 2. FIG. 16 is a diagram illustrating another example of operation of the receiver in Embodiment 2. FIG. 17 is a diagram showing another example of the operation of the receiver in the second embodiment. FIG. 18 is a diagram showing another example of the operation of the receiver in the second embodiment. FIG. 19 is a diagram illustrating another example of operation of the receiver in Embodiment 2. FIG. 20 is a diagram showing another example of the operation of the receiver in the second embodiment. FIG. 21 is a diagram illustrating an example of operations of the receiver, the transmitter, and the server according to the second embodiment. FIG. 22 is a diagram showing another example of the operation of the receiver in the second embodiment. FIG. 23 is a diagram illustrating another example of the operation of the receiver in Embodiment 2. FIG. 24 is a diagram showing an example of initial setting of the receiver in the second embodiment. FIG. 25 is a diagram showing another example of the operation of the receiver in the second embodiment. FIG. 26 is a diagram showing another example of the operation of the receiver in the second embodiment. FIG. 27 is a diagram showing another example of operation of the receiver in Embodiment 2. FIG. 28 is a diagram illustrating another example of operation of the receiver in Embodiment 2. FIG. 29 is a diagram illustrating another example of operation of the receiver in Embodiment 2. FIG. 30 is a diagram illustrating another example of operation of the receiver in Embodiment 2. FIG. 31A is a diagram showing a pen used for operating the receiver in the second embodiment. FIG. 31B is a diagram showing an operation of the receiver using the pen in the second embodiment. FIG. 32 is a diagram illustrating an example of an external appearance of the receiver in Embodiment 2. FIG. 33 is a diagram showing another example of the external appearance of the receiver in the second embodiment. FIG. 34 is a diagram showing another example of the operation of the receiver in the second embodiment. FIG. 35A is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 35B is a diagram illustrating an application example using the receiver in Embodiment 2. FIG. 36A is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 36B is a diagram illustrating an application example using the receiver in Embodiment 2. FIG. 37A is a diagram illustrating an example of operation of a transmitter in Embodiment 2. FIG. 37B is a diagram showing another example of operation of the transmitter in Embodiment 2. FIG. 38 is a diagram illustrating another example of operation of the transmitter in Embodiment 2. FIG. 39 is a diagram illustrating another example of operation of the transmitter in Embodiment 2. FIG. 40 is a diagram showing an example of a communication form between a plurality of transmitters and a receiver in the second embodiment. 41 is a diagram illustrating an example of operation of a plurality of transmitters in Embodiment 2. [FIG. FIG. 42 is a diagram showing another example of a communication form between a plurality of transmitters and a receiver in the second embodiment. FIG. 43 is a diagram illustrating another example of operation of the receiver in Embodiment 2. FIG. 44 is a diagram illustrating an application example of the receiver in Embodiment 2. FIG. 45 is a diagram illustrating an application example of the receiver in Embodiment 2. FIG. 46 is a diagram illustrating an application example of the receiver in Embodiment 2. FIG. 47 is a diagram illustrating an application example of the transmitter in Embodiment 2. FIG. 48 is a diagram showing an application example of the transmitter in the second embodiment. FIG. 49 is a diagram showing an application example of the receiving method in the second embodiment. FIG. 50 is a diagram illustrating an application example of the transmitter in Embodiment 2. FIG. 51 is a diagram illustrating an application example of the transmitter in Embodiment 2. FIG. 52 is a diagram showing an application example of the transmitter in the second embodiment. FIG. 53 is a diagram showing another example of operation of the receiver in Embodiment 2. FIG. 54 is a flowchart showing an example of operation of the receiver in Embodiment 3. FIG. 55 is a flowchart showing another example of the operation of the receiver in the third embodiment. FIG. 56A is a block diagram showing an example of a transmitter in Embodiment 3. FIG. 56B is a block diagram showing another example of the transmitter according to the third embodiment. FIG. 57 is a diagram showing a configuration example of a system including a plurality of transmitters according to the third embodiment. FIG. 58 is a block diagram showing another example of the transmitter according to the third embodiment. FIG. 59A is a diagram illustrating an example of a transmitter in Embodiment 3. FIG. 59B is a diagram illustrating an example of a transmitter in Embodiment 3. FIG. 59C is a diagram illustrating an example of a transmitter in Embodiment 3. FIG. 60A is a diagram illustrating an example of a transmitter in Embodiment 3. FIG. 60B is a diagram illustrating an example of a transmitter in Embodiment 3. FIG. 61 is a diagram showing an example of processing operations of the receiver, the transmitter, and the server in the third embodiment. FIG. 62 is a diagram showing an example of processing operations of the receiver, the transmitter, and the server according to the third embodiment. FIG. 63 is a diagram showing an example of processing operations of the receiver, the transmitter, and the server in the third embodiment. FIG. 64A is an explanatory diagram for explaining synchronization of a plurality of transmitters according to the third embodiment. FIG. 64B is an explanatory diagram for explaining synchronization of a plurality of transmitters in the third embodiment. FIG. 65 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 66 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 67 is a diagram showing an example of operations of a transmitter, a receiver, and a server in Embodiment 3. FIG. 68 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 69 is a diagram illustrating an example of an outer appearance of the receiver in Embodiment 3. FIG. 70 is a diagram showing an example of operations of the transmitter, the receiver, and the server in the third embodiment. 71 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 72 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. [FIG. FIG. 73 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 74 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 75A is a diagram showing an example of a configuration of information transmitted by the transmitter according to the third embodiment. FIG. 75B is a diagram showing another example of the structure of information transmitted by the transmitter in Embodiment 3. FIG. 76 is a diagram showing an example of a four-value PPM modulation method by the transmitter in the third embodiment. FIG. 77 is a diagram illustrating an example of a PPM modulation method performed by the transmitter in Embodiment 3. FIG. 78 is a diagram illustrating an example of a PPM modulation method in the transmitter in Embodiment 3. FIG. 79A is a diagram showing an example of a luminance change pattern corresponding to the header (preamble portion) in the third embodiment. FIG. 79B is a diagram showing an example of a luminance change pattern in the third embodiment. FIG. 80A is a diagram showing an example of a luminance change pattern in the third embodiment. FIG. 80B is a diagram showing an example of a luminance change pattern in the third embodiment. FIG. 81 is a diagram illustrating an example of operation of a receiver in a situation in front of a store in Embodiment 4. FIG. 82 is a diagram showing another example of the operation of the receiver in the situation in front of the store in the fourth embodiment. FIG. 83 is a diagram showing an example of next operation of the receiver in the situation in front of the store in the fourth embodiment. FIG. 84 is a diagram illustrating an example of next operation of the receiver in the situation in front of the store in Embodiment 4. FIG. 85 is a diagram showing an example of next operation of the receiver in the situation in front of the store in the fourth embodiment. FIG. 86 is a diagram showing an example of the operation of the display device in the in-store situation in the fourth embodiment. FIG. 87 is a diagram showing an example of the next operation of the display device in the in-store situation in the fourth embodiment. FIG. 88 is a diagram showing an example of the next operation of the display device in the in-store situation in the fourth embodiment. FIG. 89 is a diagram illustrating an example of next operation of the receiver in the in-store situation in Embodiment 4. FIG. 90 is a diagram illustrating an example of next operation of the receiver in an in-store situation in Embodiment 4. FIG. 91 is a diagram illustrating an example of next operation of the receiver in an in-store situation in Embodiment 4. FIG. 92 is a diagram illustrating an example of next operation of the receiver in an in-store situation in Embodiment 4. FIG. 93 is a diagram illustrating an example of next operation of the receiver in an in-store situation in Embodiment 4. FIG. 94 is a diagram illustrating an example of next operation of the receiver in the in-store situation in Embodiment 4. FIG. 95 is a diagram illustrating an example of operation of a receiver in a store search situation in Embodiment 4. FIG. 96 is a diagram illustrating an example of next operation of the receiver in the store search situation in Embodiment 4. FIG. 97 is a diagram illustrating an example of next operation of the receiver in the store search situation in Embodiment 4. FIG. 98 is a diagram illustrating an example of operation of a receiver in a movie advertisement situation in Embodiment 4. FIG. 99 is a diagram illustrating an example of next operation of the receiver in the movie advertisement situation in Embodiment 4. FIG. 100 is a diagram illustrating an example of next operation of a receiver in a movie advertisement situation in Embodiment 4. FIG. 101 is a diagram illustrating an example of next operation of a receiver in a movie advertisement situation in Embodiment 4. FIG. 102 is a diagram illustrating an example of operation of a receiver in a museum situation in Embodiment 4. FIG. 103 is a diagram illustrating an example of next operation of the receiver in a museum situation in Embodiment 4. FIG. 104 is a diagram illustrating an example of next operation of a receiver in a museum situation in Embodiment 4. FIG. 105 is a diagram illustrating an example of next operation of a receiver in a museum situation in Embodiment 4. FIG. 106 is a diagram illustrating an example of next operation of the receiver in a museum situation in Embodiment 4. FIG. 107 is a diagram illustrating an example of next operation of the receiver in a museum situation in Embodiment 4. FIG. 108 is a diagram illustrating an example of operation of a receiver in a bus stop situation in Embodiment 4. FIG. 109 is a diagram illustrating an example of next operation of a receiver in a bus stop situation in Embodiment 4. FIG. 110 is a diagram for explaining imaging in the fourth embodiment. FIG. 111 is a diagram for explaining transmission and imaging in the fourth embodiment. FIG. 112 is a diagram for explaining transmission in the fourth embodiment. FIG. 113 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. 114 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. 115 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. 116 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 117 is a diagram illustrating an example of operation of a receiver in Embodiment 5. FIG. 118 is a diagram illustrating an example of operation of a receiver in Embodiment 5. 119 is a diagram illustrating an example of operation of a system including a transmitter, a receiver, and a server in Embodiment 5. FIG. 120 is a block diagram showing the configuration of the transmitter according to the fifth embodiment. FIG. 121 is a block diagram showing a structure of the receiver in Embodiment 5. FIG. 122 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. 123 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. 124 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. 125 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. 126 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. 127 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. FIG. 128 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. 129 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. 130 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. [FIG. 131 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. [FIG. FIG. 132 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 133 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 134 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 135 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 136 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 137 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 138 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 139 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 140 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 141 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 142 is a diagram showing an encoding method in Embodiment 5. FIG. 143 is a diagram showing an encoding method in which light can be received even when an image is taken from an oblique direction in the fifth embodiment. FIG. 144 is a diagram illustrating an encoding method in which the amount of information differs depending on the distance in Embodiment 5. FIG. 145 is a diagram illustrating an encoding method in which the amount of information differs depending on the distance in Embodiment 5. FIG. 146 is a diagram illustrating an encoding method in which data is divided in Embodiment 5. FIG. 147 is a diagram showing an effect of inserting a reverse phase image in the fifth embodiment. FIG. 148 is a diagram showing an effect of inserting a reverse phase image in the fifth embodiment. FIG. 149 is a diagram illustrating super-resolution processing in Embodiment 5. FIG. 150 is a diagram showing a display indicating that visible light communication is supported in the fifth embodiment. FIG. 151 is a diagram showing information acquisition using a visible light communication signal in Embodiment 5. FIG. 152 is a diagram showing a data format in the fifth embodiment. FIG. 153 is a diagram showing reception by estimating a three-dimensional shape in the fifth embodiment. FIG. 154 is a diagram showing reception by estimating a three-dimensional shape in the fifth embodiment. FIG. 155 is a diagram showing stereoscopic projection in the fifth embodiment. FIG. 156 is a diagram showing stereoscopic projection in the fifth embodiment. FIG. 157 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 158 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 159 is a diagram illustrating an example of a transmission signal of Embodiment 6. FIG. 160 is a diagram showing an example of a transmission signal of Embodiment 6. FIG. 161A is a diagram illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6. FIG. 161B is a diagram illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6. FIG. 161C is a diagram illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6. FIG. 162A is a diagram illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6. FIG. 162B is a diagram illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6. FIG. 163A is a diagram illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6. FIG. 163B is a diagram illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6. FIG. 163C is a diagram illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6. FIG. 164 is a diagram illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6. FIG. 165 is a diagram illustrating an example of a transmission signal in Embodiment 6. FIG. 166 is a diagram illustrating an example of operation of a receiver in Embodiment 6. FIG. 167 is a diagram illustrating an example of user instructions displayed on the screen of the receiver in Embodiment 6. FIG. 168 is a diagram showing an example of an instruction to the user which is displayed on the screen of the receiver in Embodiment 6. FIG. 169 is a diagram illustrating an example of a signal transmission method in Embodiment 6. FIG. 170 is a diagram illustrating an example of a signal transmission method in Embodiment 6. FIG. 171 is a diagram illustrating an example of a signal transmission method in Embodiment 6. FIG. 172 is a diagram illustrating an example of a signal transmission method in Embodiment 6. FIG. 173 is a diagram for explaining a use case in the sixth embodiment. FIG. 174 is a diagram showing an information table transmitted by the smartphone to the server in the sixth embodiment. FIG. 175 is a block diagram of the server according to the sixth embodiment. FIG. 176 is a flowchart showing the overall processing of the system according to the sixth embodiment. FIG. 177 is a diagram illustrating an information table transmitted to the smartphone by the server in Embodiment 6. FIG. 178 is a diagram showing a screen flow displayed on the wearable device from when the user receives the information from the server in front of the store to when the user actually purchases the product in the sixth embodiment. FIG. 179 is a diagram for explaining another use case in the sixth embodiment. FIG. 180 is a diagram showing a service providing system using the receiving method described in each embodiment. FIG. 181 is a flowchart showing the flow of service provision. FIG. 182 is a flowchart showing service provision in another example. FIG. 183 is a flowchart showing service provision in another example. FIG. 184A is a diagram for describing a modulation scheme that is easy to receive in Embodiment 8. FIG. 184B is a diagram for describing a modulation scheme that is easy to receive in Embodiment 8. FIG. 185 is a diagram for explaining a modulation scheme which is easy to receive in Embodiment 8. FIG. 186 is a diagram for explaining a combination of communication using bright lines and image recognition in the eighth embodiment. FIG. 187A is a diagram for describing a method of using an image sensor suitable for receiving a visible light signal in Embodiment 8. FIG. 187B is a diagram for describing a method of using an imaging element suitable for receiving a visible light signal in Embodiment 8. FIG. 187C is a diagram for describing a method of using an imaging element suitable for receiving a visible light signal in Embodiment 8. FIG. 187D is a diagram for describing a method of using an imaging element suitable for receiving a visible light signal in Embodiment 8. FIG. 187E is a flowchart for explaining a method of using the image sensor suitable for receiving the visible light signal in the eighth embodiment. FIG. 188 is a diagram showing a captured image size suitable for receiving a visible light signal in Embodiment 8. FIG. 189A is a diagram illustrating a captured image size suitable for receiving a visible light signal in Embodiment 8. FIG. 189B is a flowchart showing an operation for switching to a captured image size suitable for receiving a visible light signal in Embodiment 8. FIG. 189C is a flowchart showing an operation for switching to a captured image size suitable for receiving a visible light signal in Embodiment 8. FIG. 190 is a diagram for explaining reception of a visible light signal using zoom in Embodiment 8. FIG. 191 is a diagram for describing an image data size compression method suitable for receiving a visible light signal in Embodiment 8. FIG. 192 is a diagram for explaining a modulation method with high reception error detection accuracy in Embodiment 8. FIG. 193 is a diagram for describing a change in operation of a receiver depending on a difference in circumstances in Embodiment 8. FIG. 194 is a diagram for describing notification of visible light communication to humans in Embodiment 8. FIG. 195 is a diagram for explaining expansion of the reception range by the light diffusing plate in the eighth embodiment. FIG. 196 is a diagram for describing a method of synchronizing signal transmission from a plurality of projectors in Embodiment 8. FIG. 197 is a diagram for describing a method of synchronizing signal transmission from a plurality of displays in Embodiment 8. FIG. 198 is a diagram for explaining reception of a visible light signal by the illuminance sensor and the image sensor in Embodiment 8. FIG. 199 is a diagram for explaining a reception start trigger in the eighth embodiment. FIG. 200 is a diagram for explaining a gesture for starting reception in the eighth embodiment. FIG. 201 is a diagram for explaining an application example to a car navigation system in the eighth embodiment. FIG. 202 is a diagram for explaining an application example to a car navigation system in the eighth embodiment. FIG. 203 is a diagram for explaining an application example to content protection according to the eighth embodiment. FIG. 204A is a diagram for describing an application example as an electronic lock according to the eighth embodiment. FIG. 204B is a flowchart of the information communication method in the eighth embodiment. FIG. 204C is a block diagram of an information communication device in the eighth embodiment. FIG. 205 is a diagram for explaining an application example as the store visit information transmission in the eighth embodiment. FIG. 206 is a diagram for explaining an application example of order control according to location in the eighth embodiment. FIG. 207 is a diagram for explaining an application example to the route guidance in the eighth embodiment. FIG. 208 is a diagram for explaining an application example to the location contact in the eighth embodiment. FIG. 209 is a diagram for explaining an application example to the usage log storage and analysis in the eighth embodiment. FIG. 210 is a diagram for explaining an application example to screen sharing in the eighth embodiment. FIG. 211 is a diagram for explaining an application example to screen sharing in the eighth embodiment. FIG. 212 is a diagram for explaining an application example of position estimation using the wireless access point according to the eighth embodiment. FIG. 213 is a diagram illustrating a configuration for performing position estimation by visible light communication and wireless communication in Embodiment 8. FIG. 214 is a diagram showing an application example of the information communication method in the eighth embodiment. FIG. 215 is a flowchart showing an application example of the information communication method in the eighth embodiment. FIG. 216 is a flowchart showing an application example of the information communication method in the eighth embodiment. FIG. 217 is a diagram illustrating an example of application of the transmitter and the receiver in Embodiment 9. FIG. 218 is a diagram illustrating an example of application of the transmitter in Embodiment 9. FIG. 219 is a flowchart of the information communication method in the ninth embodiment. FIG. 220 is a block diagram of an information communication device in the ninth embodiment. FIG. 221A is a diagram illustrating an example of application of the transmitter and the receiver in Embodiment 9. FIG. 221B is a flowchart showing operation of the receiver in Embodiment 9. FIG. 222 is a diagram illustrating an example of application of the transmitter and the receiver in Embodiment 9. FIG. 223 is a diagram illustrating an example of application of the transmitter in Embodiment 9. FIG. 224A is a diagram illustrating an example of application of the transmitter and the receiver in Embodiment 9. FIG. 224B is a flowchart showing operation of the receiver in Embodiment 9. FIG. 225 is a diagram illustrating an operation of the receiver in Embodiment 9. FIG. 226 is a diagram illustrating an example of application of the transmitter in Embodiment 9. 227 is a diagram illustrating an example of application of the receiver in Embodiment 9. FIG. 228A is a flowchart illustrating an example of operation of a transmitter in Embodiment 9. FIG. 228B is a flowchart illustrating an example of operation of a transmitter in Embodiment 9. FIG. 229 is a flowchart illustrating an example of operation of a transmitter in Embodiment 9. FIG. 230 is a flowchart illustrating an example of operation of the imaging device in Embodiment 9. FIG. 231 is a flowchart illustrating an example of operation of the imaging device in Embodiment 9. FIG. 232 is a diagram illustrating an example of a signal transmitted by the transmitter in Embodiment 9. FIG. 233 is a diagram illustrating an example of a signal transmitted by the transmitter in Embodiment 9. FIG. 234 is a diagram illustrating an example of a signal transmitted by the transmitter in Embodiment 9. FIG. 235 is a diagram illustrating an example of a signal transmitted by the transmitter in Embodiment 9. FIG. 236 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9. FIG. 237 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9. FIG. 238 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9. FIG. 239 is a diagram illustrating an example of operation of a transmitter in Embodiment 9. FIG. 240 is a diagram illustrating an example of operation of a transmitter in Embodiment 9. FIG. 241 is a diagram illustrating an example of a structure and operation of a transmitter in Embodiment 9. 242 is a diagram illustrating an example of a structure and operation of a transmitter in Embodiment 9. [FIG. FIG. 243 is a diagram showing a timepiece equipped with the optical sensor according to the tenth embodiment. FIG. 244 is a diagram illustrating an example of a receiver in Embodiment 10. FIG. 245 is a diagram illustrating an example of a receiver in Embodiment 10. FIG. 246A is a flowchart of an information communication method according to an aspect of the present invention. FIG. 246B is a block diagram of a mobile terminal according to one embodiment of the present invention. FIG. 247 is a diagram illustrating an example of a reception system in Embodiment 10. FIG. 248 is a diagram illustrating an example of a reception system in Embodiment 10. FIG. 249A is a diagram illustrating an example of a modulation scheme in Embodiment 10. FIG. 249B is a diagram illustrating an example of a modulation scheme in Embodiment 10. FIG. 249C is a diagram illustrating an example of a modulation scheme in Embodiment 10. FIG. 249D is a diagram illustrating an example of separation of mixed signals in Embodiment 10. FIG. 249E is a diagram illustrating an example of separation of mixed signals in Embodiment 10. FIG. 249F is a flowchart illustrating processing of the information processing program in Embodiment 10. FIG. 249G is a block diagram of an information processing device in Embodiment 10. FIG. 250A is a diagram showing an example of a visible light communication system in Embodiment 10. FIG. 250B is a diagram for explaining a use case in the tenth embodiment. FIG. 250C is a diagram showing an example of a signal transmission/reception system in Embodiment 10. FIG. 251 is a flowchart showing a reception method with interference removed according to the tenth embodiment. FIG. 252 is a flowchart showing a method for estimating the azimuth of a transmitter in the tenth embodiment. FIG. 253 is a flowchart showing a reception start method in Embodiment 10. FIG. 254 is a flowchart showing an ID generation method using information of another medium together in the tenth embodiment. FIG. 255 is a flowchart showing a method of selecting a reception system by frequency separation according to the tenth embodiment. FIG. 256 is a flowchart showing a signal receiving method when the exposure time is long in the tenth embodiment. FIG. 257 is a diagram illustrating an example of a light adjustment (adjusting brightness) method of a transmitter in Embodiment 10. FIG. 258 is a diagram illustrating an example of a method for configuring the dimming function of the transmitter in Embodiment 10. 259A is a flowchart illustrating an example of operation of a receiver in Embodiment 11. [FIG. 259B is a flowchart illustrating an example of operation of a receiver in Embodiment 11. [FIG. 259C is a flowchart illustrating an example of operation of a receiver in Embodiment 11. [FIG. FIG. 259D is a flowchart illustrating an example of operation of a receiver in Embodiment 11. FIG. 260 is a diagram for explaining the EX zoom. FIG. 261A is a flowchart showing processing of the reception program in the tenth embodiment. FIG. 261B is a block diagram of a receiver in Embodiment 10. FIG. 262 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 263 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 264 is a diagram showing an example of a signal reception method in Embodiment 12. FIG. 265 is a diagram illustrating an example of a screen display method of a receiver in Embodiment 12. FIG. 266 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 267 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 268 is a flowchart showing an example of a signal reception method in Embodiment 12. FIG. 269 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 270A is a flowchart showing processing of the reception program according to the twelfth embodiment. FIG. 270B is a block diagram of a receiver in Embodiment 12. FIG. 271 is a diagram showing an example of display on the receiver when a visible light signal is received. FIG. 272 is a diagram illustrating an example of display on the receiver when a visible light signal is received. FIG. 273 is a diagram showing an example of display of the acquired data image. FIG. 274 is a diagram showing an example of an operation for saving or discarding acquired data. FIG. 275 is a diagram illustrating a display example when browsing acquired data. FIG. 276 is a diagram illustrating an example of a transmitter in Embodiment 12. FIG. 277 is a diagram illustrating an example of a reception method in Embodiment 12. FIG. 278 is a diagram showing an example of the header pattern in the thirteenth embodiment. FIG. 279 is a diagram for describing an example of a packet structure of a communication protocol in Embodiment 13. FIG. 280 is a flowchart showing an example of a reception method in Embodiment 13. FIG. 281 is a flowchart illustrating an example of a reception method in Embodiment 13. 282 is a flowchart illustrating an example of a reception method in Embodiment 13. FIG. 283 is a diagram for explaining a receiving method in which the receiver in Embodiment 13 uses an exposure time longer than the cycle of the modulation frequency (modulation cycle). FIG. 284 is a diagram for explaining a receiving method in which the receiver in Embodiment 13 uses an exposure time longer than the cycle of the modulation frequency (modulation cycle). FIG. 285 is a diagram showing an efficient division number with respect to the size of transmission data in the thirteenth embodiment. FIG. 286A is a diagram illustrating an example of a setting method in Embodiment 13. FIG. 286B is a diagram showing another example of the setting method in the thirteenth embodiment. FIG. 287A is a flowchart showing processing of the information processing program in the thirteenth embodiment. FIG. 287B is a block diagram of an information processing device in Embodiment 13. FIG. 288 is a diagram for describing an application example of the transmitting and receiving system in Embodiment 13. FIG. 289 is a flowchart showing the processing operation of the transmitting and receiving system in Embodiment 13. FIG. 290 is a diagram for describing an application example of the transmitting and receiving system in Embodiment 13. FIG. 291 is a flowchart showing the processing operation of the transmission/reception system in Embodiment 13. FIG. 292 is a diagram for describing an application example of the transmitting and receiving system in Embodiment 13. FIG. 293 is a flowchart showing the processing operation of the transmitting and receiving system in Embodiment 13. FIG. 294 is a diagram for describing an application example of the transmitter in Embodiment 13. FIG. 295 is a diagram for describing an application example of the transmitting and receiving system in Embodiment 14. FIG. 296 is a diagram for describing an application example of the transmitting and receiving system in Embodiment 14. FIG. 297 is a diagram for describing an application example of the transmitting and receiving system in Embodiment 14. FIG. 298 is a diagram for describing an application example of the transmitting and receiving system in Embodiment 14. FIG. 299 is a diagram for describing an application example of the transmission/reception system in Embodiment 14. FIG. 300 is a diagram for explaining an application example of the transmission/reception system in Embodiment 14. FIG. 301 is a diagram for describing an application example of the transmitting and receiving system in Embodiment 14. FIG. 302 is a diagram for describing an application example of the transmission/reception system in Embodiment 14. FIG. 303 is a diagram for explaining an application example of the transmitting/receiving system in the fourteenth embodiment. FIG. 304 is a diagram for describing an application example of the transmission/reception system in Embodiment 14. FIG. 305 is a diagram for describing an application example of the transmitting and receiving system in Embodiment 14. FIG. 306 is a diagram for describing an application example of the transmitting and receiving system in Embodiment 14. FIG. 307 is a diagram for describing an application example of the transmitting and receiving system in Embodiment 14. FIG. 308 is a diagram for describing an application example of the transmitting and receiving system in Embodiment 14. FIG. 309 is a diagram for explaining an operation of the receiver in Embodiment 15. FIG. 310A is a diagram for explaining another operation of the receiver in Embodiment 15. FIG. 310B is a diagram showing an example of an indicator displayed by the output unit 1215 in the fifteenth embodiment. FIG. 310C is a diagram showing a display example of an AR in the fifteenth embodiment. FIG. 311A is a diagram for describing an example of a transmitter in Embodiment 15. FIG. 311B is a diagram for describing another example of the transmitter in Embodiment 15. FIG. 312A is a diagram for explaining an example of synchronous transmission by a plurality of transmitters in Embodiment 15. FIG. 312B is a diagram for explaining another example of synchronous transmission by a plurality of transmitters in Embodiment 15. FIG. 313 is a diagram for describing another example of synchronous transmission by a plurality of transmitters in Embodiment 15. FIG. 314 is a diagram for describing signal processing of a transmitter in Embodiment 15. FIG. 315 is a flowchart illustrating an example of a reception method in Embodiment 15. FIG. 316 is an explanatory diagram for describing an example of a reception method in Embodiment 15. FIG. 317 is a flowchart illustrating another example of the reception method in Embodiment 15. FIG. 318 is a diagram illustrating an example in which the exposure time is three times the transmission cycle and the transmission signal is a binary value of 0 or 1 in Embodiment 15. FIG. 319 is a diagram showing a state transition path in the fifteenth embodiment. FIG. 320 is a diagram showing an image of an object which blinks at high speed in Embodiment 16. FIG. 321 is a diagram illustrating a reception period and a blind period when LSS is used in Embodiment 16. FIG. 322 is a diagram illustrating a cutout scan which enables continuous reception in Embodiment 16. FIG. 323 is a diagram illustrating an example of symbols by frequency modulation in Embodiment 16. FIG. 324 is a diagram showing a frequency response by LSS in the sixteenth embodiment. FIG. 325 is a diagram illustrating an example of 4PPM symbols and V4PPM symbols in Embodiment 16. FIG. 326 is a diagram illustrating an example of Manchester code symbols and VPPM symbols in the sixteenth embodiment. FIG. 327 is a diagram for comparing and explaining the efficiencies of V4PPM and VPPM in the sixteenth embodiment. FIG. 328 is a diagram illustrating the magnitudes of signals and noises in the frequency domain in the sixteenth embodiment. FIG. 329A is a diagram illustrating a difference between the transmission frequency and the reception frequency (the maximum frequency component of the reception signal) in Embodiment 16. FIG. 329B is a diagram illustrating an example of an error rate with respect to a frequency margin in Embodiment 16. FIG. 329C is a diagram illustrating another example of the error rate with respect to the frequency margin in Embodiment 16. FIG. 329D is a diagram illustrating another example of the error rate with respect to the frequency margin in Embodiment 16. FIG. 329E is a diagram showing another example of the error rate with respect to the frequency margin in the sixteenth embodiment. FIG. 329F is a diagram showing another example of the error rate with respect to the frequency margin in the sixteenth embodiment. FIG. 330 is a diagram showing a V4PPM symbol packet reception error rate in the sixteenth embodiment. FIG. 331 is a block diagram illustrating a structure of the display system in Embodiment 17. FIG. 332 is a diagram illustrating a transmission/reception mode between the video standard signal transmission unit and the video standard signal reception unit in Embodiment 17. FIG. 333 is a diagram illustrating an example of a specific transmission and reception mode between the video standard signal transmission unit and the video standard signal reception unit in Embodiment 17. FIG. 334 is a diagram showing another example of a specific transmission/reception mode between the video standard signal transmission unit and the video standard signal reception unit in the seventeenth embodiment. FIG. 335 is a diagram illustrating another example of a specific transmission/reception mode between the video standard signal transmission unit and the video standard signal reception unit in Embodiment 17. FIG. 336A is a diagram showing an example of electric power transmitted through the power transmission transmission path in the seventeenth embodiment. FIG. 336B is a diagram showing another example of power transmitted through the power transmission transmission path in the seventeenth embodiment. FIG. 337 is a diagram illustrating another example of a specific transmission and reception mode between the video standard signal transmission unit and the video standard signal reception unit in Embodiment 17. 338] FIG. 338 is a diagram illustrating another example of a specific transmission and reception mode between the video standard signal transmission unit and the video standard signal reception unit in Embodiment 17. [FIG. FIG. 339 is a schematic diagram showing an example of a visible light communication system according to the eighteenth embodiment. FIG. 340 is a block diagram showing an example of a schematic configuration of a display device according to the eighteenth embodiment. 341A is a diagram illustrating a B.I. It is a figure which shows an example of the state before superimposing a visible light communication signal on an L control signal. FIG. 341B is a diagram illustrating an example of B.I. It is a figure which shows an example of the state which superimposed the visible light communication signal on the L control signal. FIG. 342 is a timing chart for explaining the first method in the second example of the eighteenth embodiment. FIG. 343 is a timing chart for explaining the first method in the second example of the eighteenth embodiment. FIG. 344A is a timing chart for explaining the second method in the second example of the eighteenth embodiment. FIG. 344B is a timing chart for explaining the second method in the second example of the eighteenth embodiment. FIG. 344C is a timing chart for explaining the second method in the second example of the eighteenth embodiment. FIG. 344D is a timing chart for explaining the second method in the second example of the eighteenth embodiment. FIG. 345A is a timing chart for explaining the second method in the second example of the eighteenth embodiment. FIG. 345B is a timing chart for explaining the second method in the second example of the eighteenth embodiment. FIG. 345C is a timing chart for explaining the second method in the second example of the eighteenth embodiment. FIG. 345D is a timing chart for explaining the second method in the second example of the eighteenth embodiment. FIG. 346 is a diagram illustrating the B.V. 6 is a timing chart for explaining a method of superimposing a visible light communication signal on an L control signal. FIG. 347 is a flowchart for explaining the operation of the second processing unit in the nineteenth embodiment. FIG. 348A is a block diagram of B.I. It is a figure for demonstrating the concrete method of superimposing a coded signal on an L control signal. FIG. 348B is a block diagram of B.I. It is a figure for demonstrating the concrete method of superimposing a coded signal on an L control signal. FIG. 348C is a schematic diagram of B.I. It is a figure for demonstrating the concrete method of superimposing a coded signal on an L control signal. FIG. 348D shows B. It is a figure for demonstrating the concrete method of superimposing a coded signal on an L control signal. FIG. 349 illustrates the B. It is a figure for demonstrating another concrete method which superimposes a coding signal on a L control signal. FIG. 350 is a flowchart for explaining the operation of the second processing unit according to the twentieth embodiment. FIG. 351 is a timing chart showing an example of group division of regions in Embodiment 20. FIG. 352 is a timing chart showing another example of region division in Embodiment 20. FIG. 353 is a timing chart showing another example of area division in Embodiment 20. FIG. 354 is a flowchart for explaining the operation of the second processing unit in the twenty-first embodiment. FIG. 355A is a block diagram of B.I. It is a figure for demonstrating the relationship of the phase of a L control signal and a visible light communication signal. FIG. 355B is a block diagram of B.I. It is a figure for demonstrating the relationship of the phase of a L control signal and a visible light communication signal. FIG. 356A is a timing chart for explaining the operation of the second processing unit in the twenty-first embodiment. FIG. 356B is a timing chart for explaining the operation of the second processing unit in the twenty-first embodiment. FIG. 356C is a timing chart for explaining the operation of the second processing portion in Embodiment 21. FIG. 357A is a timing chart for explaining the operation of the second processing unit according to the twenty-second embodiment. FIG. 357B is a timing chart for explaining the operation of the second processing unit in the twenty-second embodiment. FIG. 358 is a timing chart showing backlight control during local dimming adaptation in the twenty-third embodiment. FIG. 359 is a flowchart for explaining an example of operation of the second processing portion in Embodiment 23. FIG. 360 is a timing chart for describing an example of operation of the second processing portion in Embodiment 23. FIG. 361 is a flowchart for explaining an example of operation of the second processing portion in Embodiment 23. FIG. 362 is a timing chart for describing an example of operation of the second processing portion in Embodiment 23. FIG. 363 is a timing chart for describing an example of operation of the second processing portion in Embodiment 23. FIG. 364 is a schematic diagram of the visible light communication system according to the twenty-fourth embodiment. FIG. 365 is a block diagram of a display device according to the twenty-fourth embodiment. FIG. 366 is a diagram illustrating an example of generation of a visible light communication signal according to the twenty-fourth embodiment. FIG. 367 is a block diagram of a reception device according to the twenty-fourth embodiment. FIG. 368 is a diagram illustrating a captured image of the reception device with respect to turning on/off of the backlight of the display device according to the twenty-fourth embodiment. FIG. 369 is a diagram illustrating a captured image of a reception device with respect to a transmission frame of the display device according to the twenty-fourth embodiment. FIG. 370 is a diagram illustrating the relationship between the frequency of the transmission clock of the display device and the frame rate of the imaging unit of the reception device according to the twenty-fourth embodiment. FIG. 371 is a diagram for explaining a first generation example of a transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372A is a diagram for explaining the second generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372B is a diagram for explaining the third generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372C is a diagram for explaining the fourth generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372D is a diagram for explaining the fifth generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372E is a diagram for explaining the sixth generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 373 is a flowchart illustrating the operation of the visible light communication signal processing unit of the display device according to the twenty-fourth embodiment. FIG. 374 is a flowchart illustrating the operation of the visible light communication signal processing unit of the display device according to the twenty-fifth embodiment. FIG. 375 is a diagram illustrating an example of a method for determining the number of times of transmission of an arbitrary block of a transmission frame for one signal unit according to the twenty-fifth embodiment. FIG. 376 is a diagram for explaining an example of generation of a transmission frame for one signal unit according to the twenty-fifth embodiment. FIG. 377 is a flowchart illustrating the operation of the visible light communication signal processing unit of the display device according to the twenty-sixth embodiment. FIG. 378 is a diagram illustrating an example of a method for determining the number of times of transmission of an arbitrary block of a transmission frame for one signal unit according to the twenty-sixth embodiment. FIG. 379 is a diagram illustrating an example of generation of a transmission frame for one signal unit output from the display device in Embodiment 26. FIG. 380 is a diagram for explaining another example of generation of a transmission frame for one signal unit output from the display device according to the twenty-sixth embodiment. FIG. 381 is a diagram for explaining a first generation example of a transmission frame for one signal unit according to the twenty-seventh embodiment. FIG. 382A is a diagram for explaining a second generation example of a transmission frame for one signal unit according to the twenty-seventh embodiment. FIG. 382B is a diagram illustrating a third generation example of a transmission frame for one signal unit according to the twenty-seventh embodiment. FIG. 382C is a diagram illustrating a fourth generation example of a transmission frame for one signal unit according to the twenty-seventh embodiment. FIG. 383 is a flowchart for explaining operation of the visible light communication signal processing portion of the display device in Embodiment 27. FIG. 384 is a diagram for describing switching control of visible light communication in the case where the transmission device according to Embodiment 28 is a moving image display device such as a television. FIG. 385 is a diagram showing a procedure for transmitting logical data by visible light communication according to the twenty-ninth embodiment. FIG. 386 is a diagram showing a procedure for transmitting logical data by visible light communication according to the twenty-ninth embodiment. FIG. 387 is a diagram for explaining a division process by the logical data division unit according to the twenty-ninth embodiment. FIG. 388 is a diagram for explaining the division processing by the logical data division unit according to the twenty-ninth embodiment. FIG. 389 is a diagram illustrating an example of a transmission signal in Embodiment 29. FIG. 390 is a diagram illustrating another example of the transmission signal in Embodiment 29. FIG. 391 is a diagram illustrating another example of the transmission signal in Embodiment 29. FIG. 392A is a diagram for describing a transmitter in Embodiment 30. FIG. 392B is a diagram illustrating a change in luminance of each of RGB in Embodiment 30. FIG. 393 is a diagram showing afterglow characteristics of the green fluorescent component and the red fluorescent component in the thirtieth embodiment. FIG. 394 is a diagram for explaining a problem newly generated in order to suppress the occurrence of a barcode reading error in Embodiment 30. FIG. 395 is a diagram for explaining downsampling performed by the receiver in the thirtieth embodiment. FIG. 396 is a flowchart showing processing operations of the receiver in Embodiment 30. FIG. 397 is a diagram illustrating a processing operation of a reception device (imaging device) in Embodiment 31. 398 is a diagram illustrating a processing operation of a reception device (imaging device) in Embodiment 31. FIG. 399 is a diagram illustrating a processing operation of the reception device (imaging device) in Embodiment 31. FIG. 400 is a diagram illustrating processing operations of the reception device (imaging device) in Embodiment 31. FIG. 401 is a diagram showing an example of an application in the thirty-second embodiment. FIG. 402 is a diagram showing an example of the application in the thirty-second embodiment. FIG. 403 is a diagram illustrating an example of a transmission signal and an example of a voice synchronization method in Embodiment 32. FIG. 404 is a diagram illustrating an example of a transmission signal in Embodiment 32. FIG. 405 is a diagram illustrating an example of a process flow of the receiver in Embodiment 32. FIG. 406 is a diagram illustrating an example of a user interface of a receiver in Embodiment 32. FIG. 407 is a diagram illustrating an example of a process flow of the receiver in Embodiment 32. FIG. 408 is a diagram illustrating another example of the process flow of the receiver in Embodiment 32. FIG. 409A is a diagram for explaining a specific method for synchronous reproduction in Embodiment 32. FIG. 409B is a block diagram illustrating a structure of a reproduction device (receiver) that performs synchronous reproduction in Embodiment 32. FIG. 409C is a flowchart showing process operations of the playback device (receiver) that performs synchronous playback in the thirty-second embodiment. FIG. 410 is a diagram for explaining advance preparation for synchronous playback in the thirty-second embodiment. FIG. 411 is a diagram illustrating an example of application of the receiver in Embodiment 32. FIG. 412A is a front view of the receiver held by the holder according to the thirty-second embodiment. FIG. 412B is a rear view of the receiver held by the holder according to the thirty-second embodiment. FIG. 413 is a diagram for describing a use case of a receiver held by a holder in Embodiment 32. FIG. 414 is a flowchart showing processing operations of the receiver held in the holder in Embodiment 32. FIG. 415 is a diagram illustrating an example of an image displayed by the receiver in Embodiment 32. FIG. 416 is a diagram showing another example of the holder according to the thirty-second embodiment. FIG. 417A is a diagram illustrating an example of a visible light signal in Embodiment 33. FIG. 417B is a diagram illustrating an example of a visible light signal in Embodiment 33. FIG. 417C is a diagram illustrating an example of a visible light signal in Embodiment 33. FIG. 417D is a diagram illustrating an example of a visible light signal in Embodiment 33. FIG. 418 is a diagram showing the structure of the visible light signal in the thirty-third embodiment. FIG. 419 is a diagram illustrating an example of a bright line image obtained by imaging with a receiver in Embodiment 33. FIG. 420 is a diagram showing another example of the bright line image obtained by the image pickup by the receiver in the thirty-third embodiment. FIG. 421 is a diagram illustrating another example of the bright line image obtained by imaging with the receiver in Embodiment 33. FIG. 422 is a diagram for describing adaptation of the receiver in Embodiment 33 to a camera system which performs HDR combination. FIG. 423 is a diagram for explaining a processing operation of the visible light communication system in Embodiment 33. FIG. 424A is a diagram illustrating an example of inter-vehicle communication using visible light in Embodiment 33. FIG. 424B is a diagram illustrating another example of inter-vehicle communication using visible light in Embodiment 33. FIG. 425 is a diagram showing an example of a method for determining the positions of a plurality of LEDs in the thirty-third embodiment. FIG. 426 is a diagram showing an example of a bright line image obtained by imaging a vehicle in the thirty-third embodiment. FIG. 427 is a diagram illustrating an example of application of the receiver and the transmitter in Embodiment 33. Note that FIG. 427 is a view of the automobile as seen from the rear. FIG. 428 is a flowchart illustrating an example of process operations of the receiver and the transmitter in Embodiment 33. FIG. 429 is a diagram illustrating an example of application of the receiver and the transmitter in Embodiment 33. FIG. 430 is a flowchart illustrating an example of process operations of the receiver 7007a and the transmitter 7007b in Embodiment 33. 431] FIG. 431 is a diagram illustrating a structure of a visible light communication system applied to the inside of a train in Embodiment 33. [FIG. FIG. 432 is a diagram showing the configuration of the visible light communication system applied to a facility such as an amusement park in the thirty-third embodiment. FIG. 433 is a diagram illustrating an example of a visible light communication system including a playground equipment and a smartphone in Embodiment 33. FIG. 434 is a diagram illustrating an example of a transmission signal in Embodiment 34. FIG. 435 is a diagram illustrating an example of a transmission signal in Embodiment 34. FIG. 436 is a diagram illustrating an example of a transmission signal in Embodiment 34. FIG. 437 is a diagram illustrating an example of a transmission signal in Embodiment 34. FIG. 438 is a diagram illustrating an example of a transmission signal in Embodiment 34. FIG. 439 is a diagram illustrating an example of the transmission signal in Embodiment 34. FIG. 440 is a diagram illustrating an example of the transmission signal in Embodiment 34. FIG. 441 is a diagram illustrating an example of the reception algorithm in Embodiment 34. FIG. 442 is a diagram illustrating an example of the reception algorithm in Embodiment 34. FIG. 443 is a diagram illustrating an example of the reception algorithm in Embodiment 34. FIG. 444 is a diagram illustrating an example of the reception algorithm in Embodiment 34. FIG. 445 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 446 is a diagram showing an example of a transmission signal in Embodiment 35. FIG. 447 is a diagram showing an example of a transmission signal in Embodiment 35. FIG. 448 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 449 is a diagram showing an example of a transmission signal in Embodiment 35. FIG. 450 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 451 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 452 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 453 is a diagram showing an example of a transmission/reception system in Embodiment 35. FIG. 454 is a flowchart showing an example of processing of the transmitting and receiving system in Embodiment 35. FIG. 455 is a flowchart showing operation of the server in the thirty-fifth embodiment. 456 is a flowchart illustrating an example of operation of a receiver in Embodiment 35. FIG. 457 is a flowchart showing a method of calculating the progress situation in the simple mode in the thirty-fifth embodiment. FIG. 458 is a flowchart showing a method of calculating the progress situation in the maximum likelihood estimation mode in the thirty-fifth embodiment. FIG. 459 is a flowchart showing a display method in which the progress status is not reduced in Embodiment 35. FIG. 460 is a flowchart showing a method of displaying the progress status when there are a plurality of packet lengths in the thirty-fifth embodiment. FIG. 461 is a diagram illustrating an example of operating states of a receiver in Embodiment 35. FIG. 462 is a diagram showing an example of a transmission signal in Embodiment 35. 463 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 464 is a diagram showing an example of a transmission signal in Embodiment 35. FIG. 465 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 466 is a block diagram showing an example of a transmitter in Embodiment 35. FIG. 467 is a diagram showing a timing chart when the LED display in the thirty-fifth embodiment is driven by the optical ID modulation signal of the present invention. FIG. 468 is a diagram showing a timing chart in the case of driving the LED display in the thirty-fifth embodiment with the optical ID modulation signal of the present invention. FIG. 469 is a diagram showing a timing chart when the LED display in the thirty-fifth embodiment is driven by the optical ID modulation signal of the present invention. FIG. 470A is a flowchart illustrating a transmission method according to an aspect of the present invention. FIG. 470B is a block diagram illustrating a functional configuration of the transmission device according to one embodiment of the present invention. FIG. 471 is a diagram illustrating an example of a transmission signal in Embodiment 35. 472 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 473 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 474 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 475 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 476 is a diagram illustrating an example of a transmission signal in Embodiment 35.

A transmission method according to an aspect of the present invention is a transmission method for transmitting a visible light signal according to a change in brightness, and a determination step of determining a brightness change pattern by modulating the visible light signal, and a display method are provided. A common switch control step of switching a common switch, which is included in the light source group, for commonly lighting a plurality of light sources for expressing pixels in an image according to the brightness change pattern; and a plurality of light source groups included in the light source group. Turning on a first pixel switch for turning on the first light source of the light sources, the common switch is on, and only during a period when the first pixel switch is on, A first pixel switch control step of transmitting the visible light signal by turning on the first light source.

Thereby, as shown in FIGS. 463 to 470B, for example, a visible light signal can be appropriately transmitted from a display including a plurality of LEDs as a light source. Therefore, it is possible to perform communication between devices other than the lighting device. When the display is a display for displaying an image by controlling the common switch and the first pixel switch, the common switch and the first pixel switch may be used to transmit a visible light signal. it can. Therefore, it is possible to easily transmit the visible light signal without significantly changing the configuration for displaying an image on the display.

In the determining step, the brightness change pattern may be determined for each symbol cycle, and in the first pixel switch control step, the first pixel switch may be switched in synchronization with the symbol cycle.

Thereby, as shown in FIG. 463, for example, even when the symbol period is 1/2400 seconds, the visible light signal can be appropriately transmitted in accordance with the symbol period.

Further, in the first pixel switch control step, when displaying a video on the display, the visible light is included in a lighting period for expressing a pixel value of a pixel in the video corresponding to the first light source. The first pixel switch may be switched so as to supplement the lighting period only during a period in which the first light source is turned off for transmitting an optical signal. For example, the lighting period may be supplemented by changing the pixel value of the pixel in the image.

As a result, as shown in FIGS. 463 and 465, for example, even when the first light source is turned off for the transmission of the visible light signal, the lighting period is supplemented, and the original image is appropriately displayed without being destroyed. can do.

Further, the pixel value may be changed at a cycle of 1/2 of the symbol cycle.

Thereby, for example, as shown in FIG. 465, it is possible to appropriately display an image and transmit a visible light signal.

Further, in the transmission method, the common switch is further activated by turning on a second pixel switch, which is included in the light source group and is for turning on a second light source around the first light source. A second pixel switch control step of transmitting the visible light signal by turning on the second light source only during a period in which the second pixel switch is on and the second pixel switch is on. In the first and second pixel switch control steps, when the same symbols included in the visible light signal are simultaneously transmitted from the first and second light sources, respectively, the first and second pixel switch control steps are performed. Among a plurality of timings to turn on or off for transmitting the same symbol, the timing at which the rising of the luminance unique to the same symbol is obtained is the same for each of the first and second pixel switches. And other timings are made different for each of the first and second pixel switches, and the average brightness of the entire first and second light sources in the period in which the same symbol is transmitted is predetermined. The brightness may be matched with the given brightness.

As a result, for example, as shown in FIG. 464, in the spatially averaged luminance, the rising can be made steep only at the timing when the rising of the luminance peculiar to the symbol is obtained, and the occurrence of the reception error can be prevented. Can be suppressed.

Further, in the first pixel switch control step, when the same symbol included in the visible light signal is transmitted in a first period and a second period following the first period, the first pixel switch control step is performed. In each of the second period and the second period, among the plurality of timings at which the first pixel switch is turned on or off to transmit the same symbol, the timing at which the rising of the brightness unique to the same symbol is obtained is set. The average brightness of the first light source in the entire first and second periods may be set to be the same and different from each other at different timings to match a predetermined brightness.

As a result, for example, as shown in FIG. 464, in the temporally averaged luminance, the rising can be made steep only at the timing at which the rising of the luminance peculiar to the symbol is obtained, and the occurrence of the reception error can be prevented. Can be suppressed.

In the first pixel switch control step, the first pixel switch is turned on during a signal transmission period in which the common switch is switching in accordance with the brightness change pattern, and the transmission method further includes: The common switch is turned on by turning on the common switch during a video display period different from the transmission period, and turning on the first pixel switch according to the video to be displayed during the video display period, and An image display step of displaying pixels in the image by turning on the first light source may be included only during a period in which the first pixel switch is on.

Thus, the display of the image and the transmission of the visible light signal are performed in different periods, so that the display and the transmission can be easily performed.

Note that these comprehensive or specific aspects may be realized by a recording medium such as an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM. It may be realized by any combination of a circuit, a computer program, or a recording medium.

Hereinafter, embodiments will be specifically described with reference to the drawings.

It should be noted that each of the embodiments described below shows a comprehensive or specific example. Numerical values, shapes, materials, constituent elements, arrangement positions and connection forms of constituent elements, steps, order of steps, and the like shown in the following embodiments are examples, and are not intended to limit the present invention. Further, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims showing the highest concept are described as arbitrary constituent elements.

(Embodiment 1)
Hereinafter, the first embodiment will be described.

(Observation of the brightness of the light emitting part)
We propose an imaging method that starts and ends exposure at different times for each image sensor, instead of exposing all the image sensors at the same timing when capturing one image. FIG. 1 shows an example in which the image pickup devices arranged in one row are exposed at the same time, and the image pickup is performed by shifting the exposure start time in the order of the closer rows. Here, it is called an exposure line of an image pickup device that is exposed at the same time, and a line of pixels on an image corresponding to the image pickup device is called a bright line.

When this blinking light source is used to image the entire surface of the image sensor using this image capturing method, a bright line (bright and dark line of pixel values) along the exposure line is generated on the captured image as shown in FIG. .. By recognizing this bright line pattern, it is possible to estimate the change in the light source luminance at a speed higher than the image capturing frame rate. Accordingly, by transmitting the signal as a change in the light source luminance, it is possible to perform communication at a speed equal to or higher than the image capturing frame rate. When a light source expresses a signal by taking two types of luminance values, the lower luminance value is called low (LO) and the higher luminance value is called high (HI). Low may be in a state where the light source is not lit or may be weaker than high.

By this method, information is transmitted at a speed exceeding the image capture frame rate.

When there are 20 exposure lines in which the exposure times do not overlap each other in a single captured image and the image capturing frame rate is 30 fps, it is possible to recognize a luminance change of a 1.67 millisecond cycle. When there are 1000 exposure lines that do not overlap with each other in exposure time, it is possible to recognize a luminance change in a period of 1/30,000 second (about 33 microseconds). The exposure time is set shorter than 10 milliseconds, for example.

FIG. 2 shows a case where the exposure of one exposure line is completed and then the exposure of the next exposure line is started.

In this case, when the number of frames per second (frame rate) is f, and the number of exposure lines forming one image is l, the maximum information is transmitted depending on whether or not each exposure line receives light above a certain level. Can transmit information at a rate of fl bits per second.

When exposure is performed with a time difference for each pixel instead of for each line, communication can be performed at a higher speed.

At this time, when the number of pixels per exposure line is m and information is transmitted depending on whether or not each pixel receives light above a certain level, the maximum transmission rate is flm bits per second.

As shown in FIG. 3, if the exposure state of each exposure line due to the light emission of the light emitting unit can be recognized at a plurality of levels, the light emission time of the light emitting unit can be controlled by a unit of time shorter than the exposure time of each exposure line. , More information can be transmitted.

When the exposure state can be recognized at the Elv stage, information can be transmitted at a maximum rate of flElv bits per second.

Further, the basic cycle of transmission can be recognized by causing the light emitting unit to emit light at a timing slightly shifted from the exposure timing of each exposure line.

FIG. 4 shows a case where the exposure of the next exposure line is started before the exposure of one exposure line is completed. That is, the exposure times of adjacent exposure lines partially overlap with each other in terms of time. With such a configuration, it is possible to increase the number of samples within a predetermined time, compared with (1) waiting for the end of the exposure time of one exposure line to start the exposure of the next exposure line. By increasing the number of samples within the predetermined time, it becomes possible to more appropriately detect the optical signal generated by the optical transmitter as the subject. That is, it is possible to reduce the error rate when detecting the optical signal. Further, (2) the exposure time of each exposure line can be lengthened compared to the case where the exposure of the next exposure line is started after waiting for the end of the exposure time of one exposure line. However, it is possible to obtain a brighter image. That is, it becomes possible to improve the S/N ratio. It should be noted that in all the exposure lines, the exposure times of the adjacent exposure lines do not have to be partially overlapped in time, and some of the exposure lines are not partially overlapped in time. It is also possible to By configuring some exposure lines so that they do not partially overlap with each other in time, it is possible to suppress the generation of intermediate colors due to the overlap of exposure times on the imaging screen, and it is possible to more appropriately detect bright lines. ..

In this case, the exposure time is calculated from the brightness of each exposure line, and the light emission state of the light emitting unit is recognized.

When the brightness of each exposure line is determined by a binary value indicating whether or not the brightness is equal to or higher than a threshold value, the light emitting unit determines that the light emitting unit is not emitting light in order to recognize the light emitting state. Must be continued for more than the exposure time.

FIG. 5A shows the influence of the difference in exposure time when the exposure start times of the exposure lines are the same. 7500a is the case where the exposure end time of the previous exposure line and the exposure start time of the next exposure line are the same, and 7500b is the case where the exposure time is longer than that. By configuring the exposure times of adjacent exposure lines to partially overlap with each other as in 7500b, the exposure time can be extended. That is, the amount of light incident on the image sensor increases, and a bright image can be obtained. Further, since the image pickup sensitivity for picking up an image having the same brightness can be suppressed to a low level, an image with less noise can be obtained, so that a communication error is suppressed.

FIG. 5B shows the influence of the difference in the exposure start time of each exposure line when the exposure times are the same. 7501a is the case where the exposure end time of the previous exposure line is the same as the exposure start time of the next exposure line, and 7501b is the case where the exposure of the next exposure line is started earlier than the exposure end of the previous exposure line. By configuring the exposure times of the adjacent exposure lines to partially overlap with each other as in 7501b, it is possible to increase the number of lines that can be exposed per hour. As a result, the resolution becomes higher and a large amount of information can be obtained. By making the sample interval (=difference in exposure start time) close, it is possible to more accurately estimate the change in the light source brightness, reduce the error rate, and recognize the change in the light source brightness in a shorter time. be able to. By making the exposure times overlap, it is possible to recognize the blinking of the light source that is shorter than the exposure time by utilizing the difference in the exposure amount between the adjacent exposure lines.

As described with reference to FIGS. 5A and 5B, in the configuration in which each exposure line is sequentially exposed so that the exposure times of the adjacent exposure lines partially overlap with each other, the exposure time is longer than that in the normal shooting mode. It is possible to dramatically improve the communication speed by using the bright line pattern generated by setting it short for signal transmission. Here, it is possible to generate an appropriate bright line pattern by setting the exposure time during visible light communication to 1/480 seconds or less. Here, it is necessary to set the exposure time as exposure time<1/8×f, where frame frequency=f. The maximum blanking that occurs during shooting is half the size of one frame. That is, since the blanking time is half the shooting time or less, the actual shooting time is 1/2f in the shortest time. Further, since it is necessary to receive four-valued information within the time of 1/2f, it is necessary to make at least the exposure time shorter than 1/(2f×4). Since the normal frame rate is 60 frames/second or less, by setting the exposure time to 1/480 seconds or less, it is possible to generate an appropriate bright line pattern in image data and perform high-speed signal transmission. Become.

FIG. 5C shows advantages when the exposure times of the exposure lines do not overlap and when the exposure times are short. When the exposure time is long, even if the light source has a binary luminance change like 7502a, an intermediate color portion like 7502e is formed in the captured image, and it is difficult to recognize the luminance change of the light source. There is. However, as in the case of 7502d, when the exposure time of one exposure line is finished and the exposure time of the next exposure line is started, a predetermined non-exposure time (predetermined waiting time) t _D2 is provided to change the brightness of the light source. Can be easily recognized. That is, it is possible to detect a more appropriate bright line pattern such as 7502f. A configuration in which a predetermined free time such as 7502d is provided can be realized by making the exposure time t _E smaller than the time difference t _D between the exposure start times of the exposure lines. When the normal shooting mode is configured so that the exposure times of adjacent exposure lines partially overlap with each other in time, the exposure time is set shorter than in the normal shooting mode until a certain unused time occurs. By doing so, it can be realized. Further, in the normal shooting mode, even when the exposure end time of the previous exposure line and the exposure start time of the next exposure line are equal, by setting the exposure time short until a predetermined non-exposure time occurs, Can be realized. Also, by increasing the interval t _D of the exposure start time of each exposure line as in 7502g, a predetermined non-exposure time (predetermined time) from the end of the exposure of one exposure line to the start of the exposure of the next exposure line. (Waiting time) t _D2 can be _adopted . With this configuration, the exposure time can be lengthened, a bright image can be captured, and noise is reduced, resulting in high error resistance. On the other hand, in this configuration, the number of exposure lines that can be exposed within a certain period of time decreases, so there is a drawback that the number of samples decreases as in 7502h. For example, when the imaging target is bright, the former configuration is used, and when the imaging target is dark, the latter configuration is used, whereby the estimation error of the light source luminance change can be reduced.

It should be noted that in all the exposure lines, the exposure times of the adjacent exposure lines do not have to be partially overlapped in time, and some of the exposure lines are not partially overlapped in time. It is also possible to Further, in all the exposure lines, it is not necessary to provide a certain idle time (predetermined waiting time) during which exposure is not performed after the exposure of one exposure line is completed until the exposure of the next exposure line is started. It is also possible to adopt a configuration in which the lines partially overlap in time. With such a configuration, it is possible to take advantage of the advantages of each configuration. Further, the same reading method is used in the normal shooting mode for shooting at a normal frame rate (30 fps, 60 fps) and the visible light communication mode for shooting with an exposure time of 1/480 seconds or less for visible light communication. Alternatively, the circuit may read the signal. By reading out signals by the same reading method or circuit, it is not necessary to use separate circuits for the normal photographing mode and the visible light communication mode, and the circuit scale can be reduced.

FIG. 5D shows the relationship between the minimum change time t _S of the light source luminance, the exposure time t _E , the time difference t _D between the exposure start times of the exposure lines, and the captured image. When t _E +t _D <t _S , one or more exposure lines are always imaged in a state where the light source does not change from the start to the end of exposure, so an image with clear brightness like 7503d is obtained, It is easy to recognize the brightness change of the light source. When 2t _E >t _S , a bright line having a pattern different from the brightness change of the light source may be obtained, and it becomes difficult to recognize the brightness change of the light source from the captured image.

FIG. 5E shows the relationship between the transition time t _T of the light source luminance and the time difference t _D between the exposure start times of the exposure lines. As t _D is larger than t _T , the number of exposure lines having intermediate colors decreases, and the light source luminance can be easily estimated. When t _D >t _T , the exposure line of the intermediate color is continuously 2 lines or less, which is desirable. Since t _T is 1 microsecond or less when the light source is an LED and about 5 microseconds when the light source is an organic EL, setting t _D to 5 microseconds or more facilitates estimation of the light source brightness. be able to.

FIG. 5F shows the relationship between the high frequency noise t _HT of the light source luminance and the exposure time t _E. As t _E is larger than t _HT , the captured image is less affected by high-frequency noise, and the light source brightness can be easily estimated. When t _E is an integer multiple of t _HT , the influence of high frequency noise is eliminated, and the light source luminance is most easily estimated. To estimate the light source brightness, it is desirable that t _E >t _HT . The main cause of high-frequency noise comes from the switching power supply circuit. Since t _HT is 20 microseconds or less in many switching power supplies for electric lights, t _{E is set} to 20 microseconds or more to estimate the light source brightness. It can be done easily.

FIG. 5G is a graph showing the relationship between the exposure time t _E and the magnitude of high frequency noise when t _HT is 20 microseconds. Considering that t _HT varies depending on the light source, from the graph, t _E is a value equal to the value when the noise amount has a maximum value, that is, 15 microseconds or more, or 35 microseconds or more, or It can be confirmed that the efficiency is good if it is set to 54 microseconds or longer, or 74 microseconds or longer. From the viewpoint of high-frequency noise reduction, it is desirable that t _E is large, but as described above, the smaller t _E is, the more difficult it is for intermediate color portions to occur, which makes it easier to estimate the light source luminance. Therefore, t _E is 15 microseconds or more when the light source brightness change cycle is 15 to 35 microseconds, and t _E is 35 microseconds or more when the light source brightness change cycle is 35 to 54 microseconds. t _E is 54 microseconds or more when the period of the change is 54 to 74 microseconds, t _E when the period of the change in light source luminance is 74 microseconds or more may be set as 74 microseconds or more.

FIG. 5H shows the relationship between the exposure time t _E and the recognition success rate. Since the exposure time t _E has a relative meaning with respect to the time when the brightness of the light source is constant, the value (relative exposure time) obtained by dividing the cycle t _{S in} which the brightness of the light source changes by the exposure time t _E is the horizontal axis. There is. From the graph, it can be seen that the relative exposure time may be set to 1.2 or less when the recognition success rate is desired to be almost 100%. For example, when the transmission signal is set to 1 kHz, the exposure time may be set to about 0.83 milliseconds or less. Similarly, if the recognition success rate is 95% or more, the relative exposure time is 1.25 or less, and if the recognition success rate is 80% or more, the relative exposure time is 1.4 or less. Recognize. Further, the recognition success rate sharply decreases when the relative exposure time is around 1.5, and becomes almost 0% at 1.6, so it can be seen that the relative exposure time should be set so as not to exceed 1.5. .. Further, it can be seen that after the recognition rate reaches 0 at 7507c, it rises again at 7507d, 7507e, and 7507f. Therefore, when you want to take a bright image with a long exposure time, use the exposure time that makes the relative exposure time 1.9 to 2.2, 2.4 to 2.6, 2.8 to 3.0. Just do it. For example, these exposure times may be used as the intermediate mode in FIG.

FIG. 6A is a flowchart of the information communication method in this embodiment.

The information communication method according to the present embodiment is an information communication method for acquiring information from a subject, and includes steps SK91 to SK93.

That is, this information communication method is such that, in an image obtained by photographing the subject with an image sensor, a plurality of bright lines corresponding to a plurality of exposure lines included in the image sensor are generated according to a change in the luminance of the subject. A first exposure time setting step SK91 for setting a first exposure time of the image sensor, and the image sensor photographing the subject whose brightness changes with the set first exposure time, First image acquisition step SK92 for acquiring a bright line image including a plurality of bright lines, and information acquisition for acquiring information by demodulating data specified by the plurality of bright line patterns included in the acquired bright line image. In the first image acquisition step SK92, the exposure of each of the plurality of exposure lines starts sequentially at different times, and the exposure of the adjacent exposure line adjacent to the exposure line ends. After a predetermined vacant time has elapsed, exposure is started.

FIG. 6B is a block diagram of the information communication device according to the present embodiment.

The information communication device K90 in the present embodiment is an information communication device that acquires information from a subject, and includes components K91 to K93.

That is, the information communication device K90 is configured so that, in an image obtained by photographing the subject with the image sensor, a plurality of bright lines corresponding to the plurality of exposure lines included in the image sensor are generated according to the change in the luminance of the subject. An exposure time setting unit K91 that sets the exposure time of the image sensor, and the image sensor that acquires a bright line image including the plurality of bright lines by photographing the subject that changes in brightness with the set exposure time. And an information acquisition unit K93 that acquires information by demodulating the data specified by the patterns of the plurality of bright lines included in the acquired bright line image. Each of which starts exposure at different times in sequence, and starts exposure after a predetermined vacant time has elapsed after the exposure of the adjacent exposure line adjacent to the exposure line ends.

In the information communication method and the information communication device K90 shown in FIGS. 6A and 6B as described above, for example, as shown in FIG. 5C, each of the plurality of exposure lines is exposed to the adjacent exposure line adjacent to the exposure line. Since the exposure is started after the lapse of a predetermined free time after the end, it is possible to easily recognize the change in the brightness of the subject. As a result, information can be properly acquired from the subject.

In addition, in the above-described embodiment, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory. For example, the program causes a computer to execute the information communication method shown by the flowchart of FIG. 6A.

(Embodiment 2)
In the present embodiment, each application example using a receiver such as a smartphone, which is the information communication device K90 in the first embodiment, and a transmitter that transmits information as a blinking pattern of a light source such as an LED or an organic EL will be described. explain.

FIG. 7 is a diagram illustrating an example of each mode of the receiver in this embodiment.

In the normal shooting mode, the receiver 8000 acquires a normal shooting image by shooting at a shutter speed of, for example, 1/100 second, and displays the normal shooting image on the display. In this case, the normal photographed image clearly shows a subject such as a streetlight or signage configured as a signboard of a store and its surroundings.

In the visible light communication mode, the receiver 8000 acquires a visible light communication image by shooting at a shutter speed of 1/10000 second, for example. For example, in the case where the streetlight or the signage described above is transmitting the signal by the brightness change as the light source shown in the first embodiment, that is, the transmitter, the location where the signal is transmitted in the visible light communication image. One or a plurality of bright lines (hereinafter, referred to as a bright line pattern) is projected on the screen, and nothing is projected on other parts than that part. In other words, in this visible light communication image, only the bright line pattern is displayed, and the portion where the brightness of the subject is not changed and the periphery of the subject are not displayed.

In the intermediate mode, the receiver 8000 acquires an intermediate image by shooting at a shutter speed of 1/3000 second, for example. In this intermediate image, not only the bright line pattern is projected, but also the above-mentioned portion of the subject where the luminance has not changed and the periphery of the subject are also projected. Therefore, by displaying the intermediate image on the display by the receiver 8000, the user can know from where or from which position the signal is transmitted. It should be noted that the bright line pattern, the subject, and their surroundings displayed by this intermediate image are not clearer than the bright line pattern of the visible light communication image and the subject and its surroundings in the normal captured image, respectively, but have a sharpness recognized by the user. ..

In the following description, normal shooting mode or shooting in the normal shooting mode is called normal shooting, and shooting in the visible light communication mode or visible light communication mode is called visible light shooting (visible light communication). Further, instead of the normal shooting and the visible light shooting, shooting in the intermediate mode may be used, and an intermediate image may be used instead of the composite image described below.

FIG. 8 is a diagram showing an example of a photographing operation of the receiver in the present embodiment.

The receiver 8000 switches the shooting mode to normal shooting, visible light communication, normal shooting,... Then, the receiver 8000 synthesizes the normal photographed image and the visible light communication image to generate a synthesized image in which the bright line pattern, the subject, and its surroundings are clearly projected, and displays the synthesized image on the display. .. This composite image is an image generated by superimposing the bright line pattern of the visible light communication image on the portion of the normal captured image where the signal is transmitted. In addition, the bright line pattern, the subject, and their surroundings displayed by the composite image are clear, and have a sharpness that is sufficiently recognized by the user. By displaying such a composite image, the user can more clearly know from where or from which position the signal is transmitted.

FIG. 9 is a diagram showing another example of the shooting operation of the receiver in the present embodiment.

The receiver 8000 includes a camera Ca1 and a camera Ca2. In such a receiver 8000, the camera Ca1 performs normal shooting and the camera Ca2 performs visible light shooting. As a result, the camera Ca1 acquires the normal captured image as described above, and the camera Ca2 acquires the visible light communication image as described above. Then, the receiver 8000 synthesizes the normal captured image and the visible light communication image to generate the above-described synthesized image and displays it on the display.

FIG. 10A is a diagram showing another example of the photographing operation of the receiver in the present embodiment.

In the receiver 8000 having two cameras, the camera Ca1 switches the shooting mode to normal shooting, visible light communication, normal shooting,... On the other hand, the camera Ca2 continues to perform normal shooting. Then, when the camera Ca1 and the camera Ca2 are simultaneously performing normal shooting, the receiver 8000 receives stereoscopic images (principle of triangulation) from the normal shooting images acquired by those cameras. The distance from the machine 8000 to the subject (hereinafter referred to as the subject distance) is estimated. By using the subject distance estimated in this way, the receiver 8000 can superimpose the bright line pattern of the visible light communication image on an appropriate position of the normal captured image. That is, an appropriate composite image can be generated.

FIG. 10B is a diagram showing another example of the shooting operation of the receiver in the present embodiment.

The receiver 8000 includes, for example, three cameras (camera Ca1, camera Ca2, and camera Ca3). In such a receiver 8000, the two cameras (camera Ca2 and camera Ca3) continuously perform normal shooting, and the remaining one camera (camera Ca1) continuously performs visible light communication. Thereby, at any timing, the subject distance can be estimated based on the normally photographed images obtained by the two cameras that are normally photographing.

FIG. 10C is a diagram showing another example of the shooting operation of the receiver in the present embodiment.

The receiver 8000 includes, for example, three cameras (camera Ca1, camera Ca2, and camera Ca3). In such a receiver 8000, each camera switches the shooting mode to normal shooting, visible light communication, normal shooting,... Here, the shooting mode of each camera is switched for each period so that any two of these cameras perform normal shooting in one period and the remaining one camera performs visible light communication. Be done. That is, the combination of cameras that are normally shooting changes periodically. Accordingly, in any period, the subject distance can be estimated based on the normally shot images obtained by the two cameras that are normally shooting.

FIG. 11A is a diagram showing an example of a camera arrangement of the receiver in the present embodiment.

When the receiver 8000 includes two cameras Ca1 and Ca2, as shown in FIG. 11A, the two cameras Ca1 and Ca2 are arranged apart from each other. Thereby, the subject distance can be accurately estimated. That is, the longer the distance between the two cameras, the more accurately the subject distance can be estimated.

FIG. 11B is a diagram showing another example of camera arrangement of the receiver in the present embodiment.

When the receiver 8000 includes the three cameras Ca1, the camera Ca2, and the camera Ca3, the two cameras Ca1 and Ca2 for normal shooting are arranged at positions separated from each other, as shown in FIG. 11B. The visible light communication camera Ca3 is disposed, for example, between the camera Ca1 and the camera Ca2. Thereby, the subject distance can be accurately estimated. That is, the object distance can be accurately estimated by using the two cameras that are farthest apart from each other for normal shooting.

FIG. 12 is a diagram showing an example of a display operation of the receiver in this embodiment.

As described above, the receiver 8000 switches the shooting mode to visible light communication, normal shooting, visible light communication,... Here, the receiver 8000 activates an application program when first performing visible light communication. Then, the receiver 8000 estimates its own position based on the signal received by visible light communication. Next, when performing the normal shooting, the receiver 8000 displays AR (Augmented Reality) information on the normal shooting image acquired by the normal shooting. This AR information is acquired based on the position estimated as described above. Further, the receiver 8000 estimates the movement and the change in the direction of the receiver 8000 based on the detection result by the 9-axis sensor, the motion detection of the normally captured image, and the like.
The display position of the AR information is moved according to the estimated movement and change in direction. As a result, the AR information can be made to follow the subject image of the normal captured image.

Further, when the shooting mode is switched from the normal shooting to the visible light communication, the receiver 8000 superimposes the AR information on the latest normal shot image acquired during the last normal shooting during the visible light communication. Then, the receiver 8000 displays the normally shot image on which the AR information is superimposed. Further, the receiver 8000 estimates the movement and the change in the direction of the receiver 8000 based on the detection result by the 9-axis sensor, as in the case of the normal photographing, and AR according to the estimated movement and the change in the direction. Move information and normally captured images. Thus, even during visible light communication, AR information can be made to follow the subject image of the normal captured image in accordance with the movement of the receiver 8000, as in the normal capturing. Also, the normal image can be enlarged or reduced in accordance with the movement of the receiver 8000.

FIG. 13 is a diagram showing an example of a display operation of the receiver in this embodiment.

For example, the receiver 8000 may display the above-mentioned composite image on which a bright line pattern is projected, as shown in FIG. Also, as shown in FIG. 13B, the receiver 8000 normally shoots a signal explicit object that is an image having a predetermined color for notifying that a signal is being transmitted instead of the bright line pattern. A composite image may be generated by superimposing it on the image and the composite image may be displayed.

Further, in the receiver 8000, as shown in (c) of FIG. 13, the place where the signal is transmitted is normally indicated by a dotted frame and an identifier (for example, ID: 101, ID: 102, etc.). The captured image may be displayed as a composite image. Further, as shown in (d) of FIG. 13, the receiver 8000 has a signal identification which is an image having a predetermined color for notifying that a specific type of signal is transmitted instead of the bright line pattern. A composite image may be generated by superimposing an object on a normal captured image, and the composite image may be displayed. In this case, the color of the signal identification object differs depending on the type of signal output from the transmitter. For example, when the signal output from the transmitter is position information, the red signal identification object is superimposed, and when the signal output from the transmitter is a coupon, the green signal identification object is Are overlaid.

FIG. 14 is a diagram showing an example of operation of the receiver in this embodiment.

For example, when the receiver 8000 receives a signal by visible light communication, the receiver 8000 may display a normal captured image and output a sound for notifying the user that the transmitter has been found. In this case, the receiver 8000 determines the type of sound to be output, the number of times of output, or the output time depending on the number of found transmitters, the type of received signal, or the type of information specified by the signal. You may let me.

FIG. 15 is a diagram illustrating another example of operation of the receiver in this embodiment.

For example, when the user touches the bright line pattern displayed in the composite image, the receiver 8000 generates an information notification image based on the signal transmitted from the subject corresponding to the touched bright line pattern, and notifies the information notification. Display the image. This information notification image shows, for example, a coupon of a store or a place. The bright line pattern may be the signal explicit object, the signal identification object, or the dotted frame shown in FIG. The same applies to the bright line patterns described below.

FIG. 16 is a diagram showing another example of operation of the receiver in this embodiment.

For example, when the user touches the bright line pattern displayed in the composite image, the receiver 8000 generates an information notification image based on the signal transmitted from the subject corresponding to the touched bright line pattern, and notifies the information notification. Display the image. This information notification image shows, for example, the current location of the receiver 8000 by a map or the like.

FIG. 17 is a diagram showing another example of the operation of the receiver in this embodiment.

For example, the receiver 8000 receives signals from two street lamps, which are objects configured as transmitters. Then, the receiver 8000 estimates its own current location based on these signals, as described above. Then, the receiver 8000 displays the normal captured image, and displays the information notification image (image indicating the latitude and longitude) indicating the estimation result in a superimposed manner on the normal captured image. Note that the receiver 8000 may display the auxiliary information notification image by superimposing it on the normal captured image. This auxiliary information notification image recommends to the user, for example, an operation for calibrating the 9-axis sensor (in particular, a geomagnetic sensor), that is, an operation for canceling the drift. By performing such an operation, the above-mentioned present location is estimated with high accuracy.

Further, when the displayed information notification image is touched by the user, the receiver 8000 may display a map indicating the estimated position thereof instead of the normal captured image.

FIG. 18 is a diagram illustrating another example of the operation of the receiver in this embodiment.

For example, when the user swipes the receiver 8000 on which the composite image is displayed, the receiver 8000 performs the normal shooting including the dotted frame and the identifier similar to the normal shooting image shown in (c) of FIG. The image is displayed and a list of information is displayed so as to follow the swipe operation. This list shows the information specified by the signal transmitted from the location (transmitter) indicated by each identifier. Further, the swipe may be an operation of moving a finger from the outside to the inside of the right side of the display of the receiver 8000, for example. The swipe may be an operation of moving a finger inward from the upper side, the lower side, or the left side of the display.

Further, when the user taps the information included in the list, the receiver 8000 may display an information notification image (for example, an image showing a coupon) showing the information in more detail.

FIG. 19 is a diagram showing another example of operation of the receiver in this embodiment.

For example, when the user swipes the receiver 8000 displaying the composite image, the receiver 8000 displays the information notification image by superimposing the information notification image on the composite image so as to follow the swipe operation. This information notification image shows the subject distance together with the arrow so that the user can easily understand. The swipe may be, for example, an operation of moving a finger from outside to inside of the lower side of the display of the receiver 8000. The swipe may be an operation of moving a finger inward from the left side, the upper side, or the right side of the display.

FIG. 20 is a diagram showing another example of the operation of the receiver in this embodiment.

For example, the receiver 8000 photographs a transmitter, which is a signage indicating a plurality of stores, as a subject, and displays a normal photographed image acquired by the photographing. Here, when the user taps the image of the signage of one store included in the subject displayed in the normal captured image, the receiver 8000 generates the information notification image based on the signal transmitted from the signage of the store. Then, the information notification image 8001 is displayed. The information notification image 8001 is an image showing, for example, the vacant seat status of the store.

FIG. 21 is a diagram showing an example of operations of the receiver, the transmitter, and the server according to the present embodiment.

First, the transmitter 8012, which is configured as a television, transmits a signal to the receiver 8011 by changing the luminance. This signal includes, for example, information for prompting the user to purchase the content related to the program being watched. Upon receiving the signal by visible light communication, the receiver 8011 displays an information notification image prompting the user to purchase the content based on the signal. When the user performs an operation to purchase the content, the receiver 8011 causes the receiver 8011 to include information, user ID, terminal ID, credit card information, and billing information included in a SIM (Subscriber Identity Module) card. Information, a password, and a transmitter ID to the server 8013. The server 8013 manages the user ID and the payment information in association with each other for each user. Then, the server 8013 specifies the user ID based on the information transmitted from the receiver 8011 and confirms the payment information associated with the user ID. Based on this confirmation, the server 8013 determines whether to allow the user to purchase the content. Then, when the server 8013 determines to permit, it transmits permission information to the receiver 8011. Upon receiving the permission information, the receiver 8011 transmits the permission information to the transmitter 8012. Upon receiving the permission information, the transmitter 8012 acquires the content via the network, for example, and reproduces the content.

Further, the transmitter 8012 may transmit information including the ID of the transmitter 8012 to the receiver 8011 by changing the brightness. In this case, the receiver 8011 sends the information to the server 8013. When the server 8013 acquires the information, the server 8013 can determine that, for example, a television program is being viewed by the transmitter 8012, and can perform a television program audience rating survey.

Also, the receiver 8011 includes the content (voting etc.) operated by the user in the above information and transmits the content to the server 8013, so that the server 8013 can reflect the content in the television program. That is, a viewer participation type program can be realized. Further, when the receiver 8011 accepts the writing by the user, the contents of the writing are included in the above information and transmitted to the server 8013, so that the server 8013 can write the writing in the TV program or the bulletin board on the network. Etc. can be reflected.

Further, by the transmitter 8012 transmitting the above-mentioned information, the server 8013 can charge for viewing the television program by the pay broadcast or the on-demand program. Further, the server 8013 may display an advertisement on the receiver 8011, display detailed information of a television program displayed on the transmitter 8012, and display a URL of a site indicating the detailed information. You can Further, the server 8013 obtains the number of times the advertisement is displayed by the receiver 8011 or the amount of the product purchased by the advertisement, and charges the advertiser according to the number of times or the amount. be able to. Charging with such an amount of money can be performed even if the user who saw the advertisement does not immediately purchase the product. Further, when the server 8013 acquires information indicating the manufacturer of the transmitter 8012 from the transmitter 8012 via the receiver 8011, the server 8013 provides a service to the manufacturer indicated by the information (for example, a reward for selling the above-described product). Payment).

22 is a diagram showing another example of operation of the receiver in this embodiment.

For example, the user points the camera of receiver 8021 at a plurality of transmitters 8020a-8020d configured as lights. At this time, the receiver 8021 is moved so that each of the transmitters 8020a to 8020d is sequentially photographed as a subject. Then, the receiver 8021 receives a signal from each of the transmitters 8020a to 8020d by performing visible light communication while being moved. These signals contain information indicating the position of the transmitter. The receiver 8021 is based on the position indicated by the received signals from the transmitters 8020a to 8020d, the detection result of the 9-axis sensor included in the receiver 8021, and the movement of the image obtained by shooting. , The position of the receiver 8021 is estimated using the principle of triangulation. In this case, the receiver 8021 is moved to eliminate the drift of the 9-axis sensor (particularly, the geomagnetic sensor), so that the position can be estimated with higher accuracy.

FIG. 23 is a diagram showing another example of operation of the receiver in this embodiment.

The receiver 8030 is configured as, for example, a head mounted display including a camera. The receiver 8030 starts shooting in the visible light communication mode, that is, visible light communication when the start button is pressed. Then, when a signal is received by visible light communication, the receiver 8030 notifies the user of information according to the received signal. This notification is performed by, for example, outputting sound from a speaker included in the receiver 8030 or displaying an image. In addition, in the visible light communication, when the receiver 8030 receives an input of a voice instructing start, or when a signal instructing start is received by the receiver 8030 by wireless communication, other than when the start button is pressed. It may be started immediately. In addition, visible light communication is started when the range of change in the value obtained by the 9-axis sensor included in the receiver 8030 exceeds a predetermined range or when a bright line pattern appears in a normal captured image even slightly. You may.

FIG. 24 is a diagram showing an example of initial setting of the receiver in the present embodiment.

The receiver 8030 displays the alignment image 8031 at the time of initial setting. The alignment image 8031 is for aligning the position indicated by the user in the image obtained by the camera of the receiver 8030 and the image displayed by the receiver 8030. When the user aligns the fingertip with the position of the circle indicated by the alignment image 8031, the receiver 8030 associates the position of the fingertip with the position of the circle to perform the alignment. That is, calibration of the position pointed by the user is performed.

FIG. 25 is a diagram showing another example of the operation of the receiver in the present embodiment.

When the receiver 8030 identifies the portion to which the signal is transmitted by the visible light communication, the receiver 8030 displays the composite image 8034 in which the bright line pattern is projected on the portion. The user performs an operation such as tapping or double tapping on the bright line pattern. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays the information notification image 8032 based on the signal transmitted from the location corresponding to the bright line pattern.

FIG. 26 is a diagram showing another example of the operation of the receiver in this embodiment.

The receiver 8030 displays the composite image 8034 as described above. Here, the user performs an operation of moving the fingertip so as to surround the bright line pattern in the composite image 8034. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays the information notification image 8032 based on the signal transmitted from the location corresponding to the bright line pattern.

FIG. 27 is a diagram showing another example of operation of the receiver in this embodiment.

The receiver 8030 displays the composite image 8034 as described above. Here, the user performs an operation of applying the fingertip to the bright line pattern in the composite image 8034 for a predetermined time or longer. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays the information notification image 8032 based on the signal transmitted from the location corresponding to the bright line pattern.

FIG. 28 is a diagram showing another example of operation of the receiver in this embodiment.

The receiver 8030 displays the composite image 8034 as described above. Here, the user performs an operation of moving the fingertip toward the bright line pattern of the composite image 8034 by swiping. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays the information notification image 8032 based on the signal transmitted from the location corresponding to the bright line pattern.

FIG. 29 is a diagram showing another example of operation of the receiver in this embodiment.

The receiver 8030 displays the composite image 8034 as described above. Here, the user performs an operation of continuing the state in which the line of sight is directed to the bright line pattern of the composite image 8034 for a predetermined time or more. Alternatively, the user performs an operation of blinking a predetermined number of times with the line of sight directed toward the bright line pattern. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays the information notification image 8032 based on the signal transmitted from the location corresponding to the bright line pattern.

FIG. 30 is a diagram showing another example of the operation of the receiver in this embodiment.

Similarly to the above, the receiver 8030 displays the composite image 8034 and also displays the arrow associated with each bright line pattern in the composite image 8034. These arrows point in different directions for each bright line pattern. Here, the user performs an operation of moving the head along any of the arrows. When the receiver 8030 receives this operation based on the detection result by the 9-axis sensor, the receiver 8030 identifies the bright line pattern associated with the arrow corresponding to the operation, that is, the arrow pointing in the direction in which the head is moved. Then, the receiver 8030 displays the information notification image 8032 based on the signal transmitted from the location corresponding to the bright line pattern.

FIG. 31A is a diagram showing a pen used for operating the receiver in the present embodiment.

The pen 8033 includes a transmitter 8033a that transmits a signal according to a change in luminance, a button 8033b, and a button 8033c. When the button 8033b is pressed, the transmitter 8033a transmits a predetermined first signal, and when the button 8033c is pressed, the transmitter 8033a transmits a predetermined first signal different from the first signal. 2 signal is transmitted.

FIG. 31B is a diagram showing an operation of the receiver using the pen in this embodiment.

The pen 8033 is used in place of the above-mentioned user's fingertip, and is used like a stylus pen. Further, by selectively using the button 8033b and the button 8033c, the pen 8033 can be used as a normal pen or can be used like an eraser.

FIG. 32 is a diagram showing an example of an external appearance of the receiver in this embodiment.

The receiver 8030 includes a first touch sensor 8030a and a second touch sensor 8030b. These touch sensors are attached to the frame of the receiver 8030. For example, when the user puts a fingertip on the first touch sensor 8030a and moves it, the receiver 8030 moves the pointer on the image displayed to the user according to the movement of the fingertip. When the user touches the second touch sensor 8030b, the receiver 8030 performs a process of selecting an object pointed by the pointer on the image displayed for the user.

FIG. 33 is a diagram showing another example of the external appearance of the receiver in this embodiment.

The receiver 8030 includes a touch sensor 8030c. The touch sensor 8030c is attached to the frame of the receiver 8030. For example, when the user puts a fingertip on the touch sensor 8030c and moves it, the receiver 8030 moves the pointer according to the movement of the fingertip on the image displayed to the user. When the user presses the touch sensor 8030c, the receiver 8030 performs a process of selecting an object pointed by the pointer on the image displayed to the user. That is, the touch sensor 8030c is configured as a so-called clickable touch sensor.

FIG. 34 is a diagram showing another example of operation of the receiver in this embodiment.

The receiver 8030 displays the combined image 8034 and the pointer 8035 described above in the combined image 8034, as described above. When the receiver 8030 includes the first touch sensor 8030a and the second touch sensor 8030b, the user moves the pointer by touching and moving the first touch sensor 8030a with a fingertip, and a bright line pattern is displayed. Point that object to the object that is. Then, the user causes the receiver 8030 to select the bright line pattern by touching the second touch sensor 8030b. When the bright line pattern is selected, the receiver 8030 displays the information notification image 8032 based on the signal transmitted from the position of the bright line pattern.

In the case where the receiver 8030 includes the touch sensor 8030c, the user moves the pointer by touching the finger on the touch sensor 8030c to move the pointer, and then applies the pointer to the object having the bright line pattern. Then, the user causes the receiver 8030 to select the bright line pattern by pressing the touch sensor 8030c. When the bright line pattern is selected, the receiver 8030 displays the information notification image 8032 based on the signal transmitted from the position of the bright line pattern.

FIG. 35A is a diagram showing another example of operation of the receiver in this embodiment.

The receiver 8030 displays the gesture confirmation image 8036 based on the signal obtained by performing visible light communication. The gesture confirmation image 8036 prompts the user to make a predetermined gesture, for example, in order to provide the service to the user.

FIG. 35B is a diagram illustrating an application example using the receiver in this embodiment.

A user 8038 wears the receiver 8030 and is at a store, for example. Here, the receiver 8030 displays the gesture confirmation image 8036 described above to the user 8038. The user 8038 makes a predetermined gesture according to the gesture confirmation image 8036. Here, the clerk 8039 in the store wears the receiver 8037. The receiver 8037 is configured as a head mounted display equipped with a camera, and may have the same configuration as the receiver 8030. The receiver 8037 also displays the gesture confirmation image 8036 based on the signal obtained by performing visible light communication. The store clerk 8039 determines whether or not the predetermined gesture shown by the displayed gesture confirmation image 8036 and the gesture made by the user 8038 match. When the store clerk 8039 determines that they match, the clerk 8039 provides the service associated with the gesture confirmation image 8036 to the user 8038.

FIG. 36A is a diagram showing another example of operation of the receiver in this embodiment.

The receiver 8030 displays the gesture confirmation image 8040 based on the signal obtained by performing visible light communication. This gesture confirmation image 8040 prompts the user to make a predetermined gesture, for example, to permit wireless communication.

FIG. 36B is a diagram showing an application example using the receiver according to the present embodiment.

User 8038 wears receiver 8030. Here, the receiver 8030 displays the gesture confirmation image 8040 described above to the user 8038. The user 8038 makes a predetermined gesture according to the gesture confirmation image 8040. Here, a person 8041 around the user 8038 wears the receiver 8037. The receiver 8037 is configured as a head mounted display equipped with a camera, and may have the same configuration as the receiver 8030. The receiver 8037 captures a predetermined gesture performed by the user 8038 to acquire authentication information such as a password included in the gesture. Then, when the receiver 8037 determines that the authentication information matches the predetermined information, the receiver 8037 establishes a wireless connection with the receiver 8030. After this establishment is performed, the receiver 8030 and the receiver 8037 can perform wireless communication with each other.

FIG. 37A is a diagram showing an example of operation of the transmitter in this embodiment.

The transmitter alternately transmits the signal 1 and the signal 2 in a predetermined cycle, for example. The transmission of the signal 1 and the transmission of the signal 2 are each performed by a change in luminance such as blinking of visible light. Further, the luminance change pattern for transmitting the signal 1 and the luminance change pattern for transmitting the signal 2 are different from each other.

FIG. 37B is a diagram showing another example of operation of the transmitter in this embodiment.

The transmitter may intermittently transmit the signal 1 and the signal 2 without a buffer time, instead of continuously transmitting the signal 1 and the signal 2 as described above. Here, the transmitter does not change the brightness during the buffer time. Alternatively, the transmitter may transmit a signal indicating that it is the buffering time by changing the luminance, or perform a brightness changing different from the brightness changing for transmitting each of the signal 1 and the signal 2 in the buffering time. .. As a result, the receiver can properly receive the signal 1 and the signal 2 without interference.

FIG. 38 is a diagram showing another example of the operation of the transmitter in this embodiment.

The transmitter repeatedly transmits a signal sequence of a structural unit including a preamble, a block 1, a block 2, a block 3 and a check signal by changing the luminance. Here, the block 1 has a preamble, an address 1, a data 1 and a check signal. The blocks 2 and 3 are also constructed similarly to the block 1. In addition, specific information can be obtained by using the data included in each of block 1, block 2, and block 3.

That is, in the signal train as described above, one piece of data or information is divided into three blocks and stored. Therefore, a receiver that needs a blanking period for capturing cannot receive all the data of block 1, block 2, and block 3 from one signal sequence, but can receive the remaining data from other signal sequences. Can be received. As a result, even a receiver that requires a blanking period can appropriately acquire specific information from at least one signal train.

Further, in the signal sequence as described above, the preamble and the check signal are arranged for the set of three blocks. Therefore, a receiver capable of receiving light without requiring a blanking period, for example, a receiver including an illuminance sensor or the like, uses a preamble and a check signal arranged for the set to generate one signal sequence. It can be received at once, and specific information can be obtained in a short period of time.

FIG. 39 is a diagram showing another example of the operation of the transmitter in this embodiment.

As described above, the transmitter changes the arrangement of blocks included in the signal sequence for each signal sequence when repeatedly transmitting the signal sequence of the structural unit including the block 1, the block 2, and the block 3. Good. For example, each block is arranged in the order of block 1, block 2, and block 3 in the first signal sequence, and each block is arranged in the order of block 3, block 1, and block 2 in the next signal sequence. This avoids that only the same block is acquired by a receiver that requires a periodic blanking period.

FIG. 40 is a diagram showing an example of a communication form between a plurality of transmitters and a receiver in the present embodiment.

The receiver 8050 may receive signals (visible light) transmitted from a transmitter 8051a and a transmitter 8051b configured as illumination and reflected by a reflecting surface. This allows signals from many transmitters to be received together. Further, in this case, the transmitter 8051a and the transmitter 8051b respectively transmit signals of different frequencies or protocols. This allows receiver 8050 to receive signals from those transmitters without interference.

FIG. 41 is a diagram illustrating an example of operation of a plurality of transmitters in this embodiment.

One of the transmitter 8051a and the transmitter 8051b may monitor the transmission status of the signal from the other and transmit the signal so as to prevent interference with the signal of the other. For example, one transmitter receives a signal transmitted from the other transmitter and transmits a signal of a protocol different from that signal. Alternatively, one transmitter detects a period in which no signal is transmitted from the other transmitter and transmits a signal in that period.

FIG. 42 is a diagram showing another example of a communication form between a plurality of transmitters and a receiver in the present embodiment.

The transmitter 8051a and the transmitter 8051b may each transmit signals of the same frequency or protocol. In this case, the receiver 8050 specifies the intensity of signals transmitted from those transmitters, that is, the edge intensity of the bright line included in the image obtained by photographing. This strength becomes weaker as the distance between the receiver 8050 and the transmitter increases. When the distance between the receiver 8050 and each of the transmitter 8051a and the transmitter 8051b is different, such difference in distance can be used. That is, the receiver 8050 can appropriately separate and receive the signals transmitted from the transmitter 8051a and the transmitter 8051b, respectively, according to the specified strength.

FIG. 43 is a diagram showing another example of operation of the receiver in this embodiment.

The receiver 8050 receives the signal transmitted from the transmitter 8051a and reflected by the reflecting surface. At this time, the receiver 8050 may estimate the position of the transmitter 8051a based on the intensity distribution of brightness (difference in brightness at a plurality of positions) in the image obtained by shooting.

FIG. 44 is a diagram illustrating an application example of the receiver in this embodiment.

For example, the receiver 7510a configured as a smartphone captures an image of the light source 7510b with a back camera (out-camera) 7510c, receives a signal transmitted from the light source 7510b, and acquires the position and orientation of the light source 7510b from the received signal. The receiver 7510a estimates the position and orientation of the receiver 7510a itself based on how the light source 7510b appears in the captured image and the sensor values of the 9-axis sensor included in the receiver 7510a. The receiver 7510a captures an image of the user 7510e with a front camera (face camera, in-camera) 7510f, and estimates the position and orientation of the head of the 7510e and the gaze direction (position and orientation of the eyeball) by image processing. .. The receiver 7510a transmits the estimation result to the server. The receiver 7510a changes the behavior (display content of the display and reproduced sound) according to the line-of-sight direction of the user 7510e. The image pickup by the back camera 7510c and the image pickup by the front camera 7510f may be performed simultaneously or alternately.

FIG. 45 is a diagram illustrating an application example of the receiver in this embodiment.

For example, the receivers 7511d and 7511i configured as smartphones receive signals from the light sources 7511b and 7511g, estimate their own positions and orientations, and estimate the line-of-sight directions of the users 7511e and 7511j, as described above. In addition, the receivers 7511d and 7511i acquire information of surrounding objects 7511a to 7511c and 7511f to 7511h from the server based on the received data. The receivers 7511d and 7511i change the display on the display as if the user sees the object on the other side through the receivers 7511d and 7511i. The receivers 7511d and 7511i display an AR (augmented reality) object such as 7511k according to the content displayed on the display. When the line of sight of the user 7511j is outside the image capturing range of the camera, the receiver 7511i displays that it is out of the range, as in 7511l. Alternatively, the AR object and other information are displayed in the area outside the range. Alternatively, images obtained by previously imaging the out-of-range area are stitched together and displayed.

FIG. 46 is a diagram showing an application example of the receiver in this embodiment.

For example, the receiver 7512c configured as a smartphone receives the signal from the light source 7512a, estimates its own position and orientation, and estimates the line-of-sight direction of the user 7512d, as described above. The receiver 7512c performs processing on the object 7512b in the line-of-sight direction of the user 7512d. For example, information about the object 7512b is displayed on the screen. When the gaze direction of the user 7512h moves from the object 7512f to the receiver 7512g, the receiver 7512g determines that the user 7512h is interested in the object 7512f, and continues the process related to the object 7512f. For example, the information of the object 7512f is kept displayed on the screen.

FIG. 47 is a diagram showing an application example of the transmitter according to the present embodiment.

For example, the transmitter 7513a configured as illumination has high brightness, and when the brightness of the transmission signal is high (high) or low (low), the brightness exceeds the upper limit of brightness when imaged by the receiver. , 7513b, no bright line appears. Therefore, as shown by 7513c, by providing a portion 7513d for diffusing or weakening light such as a diffuser plate or a prism to reduce the brightness, the receiver can image a bright line like 7513e. ..

FIG. 48 is a diagram illustrating an application example of the transmitter in this embodiment.

For example, in the transmitter 7514a configured as illumination, since the light source is not uniform, the captured image has uneven brightness as in 7514b, which causes a reception error. Therefore, as shown by 7514c, a reception error can be suppressed by providing a light diffusing portion 7514d such as a diffuser plate or a prism so that the luminance becomes uniform.

FIG. 49 is a diagram showing an application example of the receiving method in the present embodiment.

In the transmitters 7515a and 7515b, the brightness of the central portion is high and no bright line appears in the image captured by the receiver, and the bright line appears in the peripheral portion. The receiver cannot receive the signal from the portion 7515d because the bright line is broken, but can receive the signal from the portion 7515c. The receiver can receive the signal from more bright lines than the portion of 7515c by reading the bright lines on the path 7515e.

FIG. 50 is a diagram illustrating an application example of the transmitter in this embodiment.

For example, the transmitters 7516a, 7516b, 7516c, and 7516d configured as illumination have high brightness as well as 7513a, and bright lines are unlikely to occur when the image is captured by the receiver. Therefore, by providing a diffuser plate/prism 7516e, a reflector plate 7516f, a reflector plate/half mirror 7516g, a reflector plate 7516h, and a diffuser plate/prism 7516j, it is possible to diffuse light and widen a portion where a bright line is generated. You can In these transmitters, the captured image is captured in the form of bright lines around 7515a. Since the receiver estimates the distance between the receiver and the transmitter using the size of the transmitter on the captured image, the part where the light is diffused is set as the size of the light source, and the server is associated with the transmission ID. The receiver can accurately estimate the distance to the transmitter.

FIG. 51 is a diagram illustrating an application example of the transmitter in this embodiment.

For example, the transmitter 7517a configured as illumination has high luminance as well as the 7513a, and bright lines are unlikely to occur when an image is captured by the receiver. Therefore, by providing the reflection plate 7517b, it is possible to diffuse the light and widen the portion where the bright line occurs.

FIG. 52 is a diagram showing an application example of the transmitter according to the present embodiment.

The transmitter 7518a reflects the light from the light source at the 7518c, so that the receiver can image the bright line in a wide range. The transmitter 7518d directs the light source to the diffuser plate or the prism 7518e, so that the receiver can image the bright line in a wide range.

FIG. 53 is a diagram showing another example of operation of the receiver in this embodiment.

The receiver displays the bright line pattern by the above-mentioned composite image or intermediate image. At this time, the receiver may not be able to receive the signal from the transmitter corresponding to the bright line pattern. Here, when the user performs an operation (for example, tap) on the bright line pattern to select the bright line pattern, the receiver performs an optical zoom to enlarge the synthetic line image or the intermediate image in which the bright line pattern is enlarged. Display the image. By performing such optical zoom, the receiver can properly receive the signal from the transmitter corresponding to the bright line pattern. That is, even if the image obtained by imaging is too small to acquire the signal, the signal can be properly received by performing the optical zoom. Further, even when an image having a size capable of acquiring a signal is displayed, it is possible to perform fast reception by performing optical zoom.

(Summary of this Embodiment)
The information communication method according to the present embodiment is an information communication method for acquiring information from a subject, in which an image obtained by photographing the subject with an image sensor has a bright line corresponding to an exposure line included in the image sensor. A first exposure time setting step of setting an exposure time of the image sensor so as to occur according to a change in the brightness of the object; and the image sensor, photographing the object having the brightness change with the set exposure time. Accordingly, a bright line image acquisition step of acquiring a bright line image that is an image including the bright line, and a spatial position of the site where the bright line appears can be identified based on the bright line image, and the subject and the subject An image display step of displaying a display image showing the surroundings of the subject, and an information acquisition step of acquiring transmission information by demodulating data specified by the pattern of the bright lines included in the acquired bright line image. Including and

For example, a composite image or an intermediate image as shown in FIGS. 7 to 9 and 13 is displayed as a display image. In addition, in the display image showing the subject and the surroundings of the subject, the spatial position of the portion where the bright line appears is identified by the bright line pattern, the signal indicating object, the signal identifying object, the dotted frame, or the like. Therefore, the user can easily find the subject that is transmitting the signal due to the change in luminance by looking at such a display image.

Further, the information communication method further includes a second exposure time setting step of setting an exposure time longer than the exposure time, and the image sensor photographing the subject and the surroundings of the subject with the long exposure time. By performing a normal image acquisition step of acquiring a normal captured image, the region where the bright line appears in the normal captured image is specified based on the bright line image, and the signal object that is an image indicating the region is usually the normal object. A composite step of generating a composite image by superimposing the captured image may be included, and the composite image may be displayed as the display image in the image display step.

For example, the signal object is a bright line pattern, a signal explicit object, a signal identification object, a dotted frame, or the like, and a composite image is displayed as a display image as shown in FIGS. 8, 9, and 13. This allows the user to more easily find the subject that is transmitting the signal due to the brightness change.

Further, in the first exposure time setting step, the exposure time is set to 1/3000 seconds, in the bright line image acquisition step, the bright line image showing the surroundings of the subject is acquired, and in the image display step. The bright line image may be displayed as the display image.

For example, as shown in FIG. 7, the bright line image is acquired and displayed as an intermediate image. Therefore, it is not necessary to perform processing such as obtaining and combining the normal captured image and the visible light communication image, and the processing can be simplified.

Further, the image sensor includes a first image sensor and a second image sensor, and in the normal image acquisition step, the normal image is acquired by the first image sensor capturing, and the bright line is acquired. In the image acquisition step, the bright line image may be acquired by the second image sensor taking an image at the same time as the first image sensor takes an image.

For example, as shown in FIG. 9, a normal captured image and a visible light communication image which is a bright line image are acquired by the respective cameras. Therefore, as compared with the case where the normal captured image and the visible light communication image are acquired by one camera, those images can be acquired earlier, and the processing speed can be increased.

Further, the information communication method, further, in the case where the region in which the bright line appears in the display image is designated by an operation by the user, the transmission information acquired from the pattern of the bright line of the designated region is used. An information presenting step of presenting the presented information based on the above may be included. For example, the operation performed by the user includes tapping, swiping, an operation of continuously applying a fingertip to the part for a predetermined time or more, an operation of keeping a state of directing sight to the part for a predetermined time or more, and shown in association with the part. An operation of moving a part of the user's body along the arrow indicated, an operation of applying a pen tip that changes in brightness to the site, or a touch sensor to apply the pointer displayed in the display image to the site. It is an operation.

For example, as shown in FIGS. 15 to 20 and 25 to 34, the presentation information is displayed as an information notification image. Thereby, desired information can be presented to the user.

The image sensor may be included in a head mounted display, and in the image display step, a projector mounted on the head mounted display may display the display image.

Thereby, for example, as shown in FIGS. 23 to 30, information can be easily presented to the user.

Further, in the information communication method for acquiring information from a subject, a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject with an image sensor according to a change in luminance of the subject. As described above, a first exposure time setting step of setting an exposure time of the image sensor, and an image including the bright line by the image sensor capturing the subject whose brightness changes with the set exposure time. A bright line image acquisition step of acquiring a bright line image that is, and an information acquisition step of acquiring information by demodulating the data specified by the pattern of the bright line included in the acquired bright line image, the bright line image In the acquisition step, during the period in which the image sensor is being moved, by capturing a plurality of the subjects, the bright line image including a plurality of regions where the bright lines appear is acquired, and in the information acquisition step, each of the regions is acquired. In, by demodulating the data identified by the pattern of the bright line of the site, to obtain the position of each of the plurality of subjects, the information communication method further, for each of the plurality of obtained subjects. A position estimation step of estimating the position of the image sensor based on the position and the moving state of the image sensor may be included.

Thereby, for example, as shown in FIG. 22, the position of the receiver including the image sensor can be accurately estimated by the change in the brightness due to the subject such as a plurality of lights.

Further, in the information communication method for acquiring information from a subject, a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject with an image sensor according to a change in luminance of the subject. As described above, a first exposure time setting step of setting an exposure time of the image sensor, and an image including the bright line by the image sensor capturing the subject whose brightness changes with the set exposure time. And a bright line image acquisition step of acquiring a bright line image, an information acquisition step of acquiring information by demodulating data specified by the pattern of the bright line included in the acquired bright line image, and the acquired information In the information presenting step, an image prompting a predetermined gesture may be presented to the user of the image sensor as the information.

Accordingly, for example, as shown in FIGS. 35A to 36B, depending on whether or not the user makes the gesture as prompted, the user can be authenticated and the convenience can be improved.

Further, in the information communication method for acquiring information from a subject, a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject with an image sensor according to a change in luminance of the subject. As described above, an exposure time setting step of setting the exposure time of the image sensor, and the image sensor capturing the subject whose brightness changes with the exposure time set, thereby obtaining a bright line image including the bright line. And an information acquisition step of acquiring information by demodulating the data specified by the pattern of the bright lines included in the acquired bright line image. In the image acquisition step, the information is reflected on a reflecting surface. The bright line image is acquired by photographing a plurality of the subjects, and in the information acquisition step, the bright line is changed to a bright line corresponding to each of the plurality of subjects in accordance with the intensity of the bright line included in the bright line image. Information may be acquired by separating and demodulating the data specified by the pattern of the bright line corresponding to the subject for each of the subjects.

Thereby, for example, as shown in FIG. 42, even when the brightness of each of the subjects such as a plurality of lights changes, it is possible to obtain appropriate information from each of the subjects.

Further, in the information communication method for acquiring information from a subject, a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject with an image sensor according to a change in luminance of the subject. As described above, an exposure time setting step of setting the exposure time of the image sensor, and the image sensor capturing the subject whose brightness changes with the exposure time set, thereby obtaining a bright line image including the bright line. And an information acquisition step of acquiring information by demodulating the data specified by the pattern of the bright lines included in the acquired bright line image. In the image acquisition step, the information is reflected on a reflecting surface. The bright line image may be acquired by photographing the subject, and the information communication method may further include a position estimation step of estimating the position of the subject based on a luminance distribution in the bright line image.

Thereby, for example, as shown in FIG. 43, it is possible to estimate the appropriate position of the subject based on the brightness distribution.

Also, there is provided an information communication method for transmitting a signal according to a change in luminance, which comprises a first determining step of determining a first pattern of change in luminance by modulating a first signal to be transmitted, and a first determining step for transmitting. A second determining step of determining a second pattern of intensity changes by modulating the second signal; a change in intensity of the light emitter according to the determined first pattern; And a transmission step of transmitting the first and second signals by alternately performing the luminance change according to the second pattern.

Thereby, for example, as shown in FIG. 37A, the first signal and the second signal can be transmitted without delay.

Further, in the transmitting step, when the luminance change is switched between the luminance change according to the first pattern and the luminance change according to the second pattern, a buffering time may be opened before switching.

Thereby, for example, as shown in FIG. 37B, it is possible to suppress interference between the first signal and the second signal.

In addition, there is provided an information communication method for transmitting a signal according to a change in brightness, wherein a determining step of determining a pattern of brightness change by modulating a signal to be transmitted, and the light emitter changing the brightness according to the determined pattern. Transmitting the signal to be transmitted by transmitting the signal to be transmitted, the signal comprising a plurality of large blocks, each of the plurality of large blocks including first data and a preamble for the first data. And a check signal for the first data, wherein the first data is composed of a plurality of small blocks, and the small blocks include second data, a preamble for the second data, and the first data. And a check signal for 2 data.

Thereby, for example, as shown in FIG. 38, both a receiver that requires a blanking period and a receiver that does not require a blanking period can properly acquire data.

In addition, in the information communication method of transmitting a signal according to a change in brightness, a plurality of transmitters each modulate a signal to be transmitted, and a determining step of determining a pattern of change in brightness, And a transmitting step of transmitting the signal to be transmitted by changing the luminance according to the determined pattern, the transmitting body including a transmitter, wherein the transmitting step transmits signals having different frequencies or protocols. You may.

Thereby, for example, as shown in FIG. 40, interference of signals from a plurality of transmitters can be suppressed.

In addition, in the information communication method of transmitting a signal according to a change in brightness, a plurality of transmitters each modulate a signal to be transmitted, and a determining step of determining a pattern of change in brightness, And a transmitting step of transmitting the signal to be transmitted by changing the luminance according to the determined pattern, wherein the light-emitting body provided in the transmitter includes one of the plurality of transmitters. One transmitter may receive the signal transmitted from the other transmitter and transmit the other signal in a manner that does not interfere with the received signal.

Thereby, for example, as shown in FIG. 41, interference of signals from a plurality of transmitters can be suppressed.

(Embodiment 3)
In the present embodiment, description will be given of each application example using the receiver such as the smartphone according to the first or second embodiment and the transmitter that transmits information as a blinking pattern such as an LED or an organic EL.

FIG. 54 is a flowchart showing an example of operation of the receiver in Embodiment 3.

First, the receiver receives a signal with the illuminance sensor (8101). Next, the receiver acquires information such as position information from the server based on the received signal (8102). Next, the receiver activates an image sensor capable of imaging the light receiving direction of the illuminance sensor (8103). Next, the receiver receives part or all of the signal at the image sensor and checks whether part or all of the signal is the same as the signal received at the illuminance sensor (8104). Next, the receiver estimates the position of the receiver from the position of the transmitter in the captured image (captured image), the information from the 9-axis sensor provided in the receiver, and the position information of the transmitter. (8105). In this way, the receiver activates the illuminance sensor that consumes less power and activates the image sensor when the illuminance sensor receives a signal. Then, the receiver performs position estimation using the image pickup by the image sensor. As a result, the position of the receiver can be accurately estimated while suppressing power consumption.

FIG. 55 is a flowchart showing another example of the operation of the receiver in the third embodiment.

The receiver recognizes a periodic change in luminance from the sensor value of the illuminance sensor (8111). Next, the receiver activates an image sensor capable of capturing the light receiving direction of the illuminance sensor and receives a signal (8112). That is, similarly to the above, the receiver activates the illuminance sensor with low power consumption, and activates the image sensor when the illuminance sensor receives a periodic change in luminance. Then, the receiver receives an accurate signal by imaging with the image sensor. This makes it possible to receive an accurate signal while suppressing power consumption.

FIG. 56A is a block diagram showing an example of a transmitter in Embodiment 3.

The transmitter 8115 includes a power supply unit 8115a, a signal control unit 8115b, a light emitting unit 8115c, and a light emitting unit 8115d. The power supply unit 8115a supplies electric power to the signal control unit 8115b. The signal control unit 8115b distributes the electric power supplied from the power supply unit 8115a to the light emitting unit 8115c and the light emitting unit 8115d, and controls the luminance change of the light emitting unit 8115c and the light emitting unit 8115d.

FIG. 56B is a block diagram showing another example of the transmitter according to the third embodiment.

The transmitter 8116 includes a power supply unit 8116a, a signal control unit 8116b, a light emitting unit 8116c, and a light emitting unit 8116d. The power supply unit 8116a supplies electric power to the light emitting unit 8116c and the light emitting unit 8116d. Here, the signal control unit 8116b controls the brightness change of the light emitting unit 8116c and the light emitting unit 8116d by controlling the power supplied from the power supply unit 8116a. In this way, the power control unit 8116a controls the power supply unit 8116a that supplies power to each of the light emitting unit 8116c and the light emitting unit 8116d, so that the use efficiency of power can be increased.

FIG. 57 is a diagram showing a configuration example of a system including a plurality of transmitters according to the third embodiment.

This system includes a centralized control unit 8118, a transmitter 8117, and a transmitter 8120. The centralized control unit 8118 controls the transmission of signals according to the brightness changes of the transmitter 8117 and the transmitter 8120. For example, the centralized control unit 8118 causes each of the transmitter 8117 and the transmitter 8120 to transmit the same signal at the same timing, or causes only one of the transmitters to transmit a signal unique to the transmitter.

The transmitter 8120 includes two transmission units 8121 and 8122, a signal changing unit 8123, a signal storage unit 8124, a synchronization signal input unit 8125, a synchronization control unit 8126, and a light receiving unit 8127.

Each of the two transmission units 8121 and 8122 has the same configuration as the transmitter 8115 shown in FIG. 56A, and transmits a signal by changing the luminance. Specifically, the transmission unit 8121 includes a power supply unit 8121a, a signal control unit 8121b, a light emitting unit 8121c, and a light emitting unit 8121d. The transmission unit 8122 includes a power supply unit 8122a, a signal control unit 8122b, a light emitting unit 8122c, and a light emitting unit 8122d.

The signal changing unit 8123 modulates the signal to be transmitted into a signal indicating a luminance change pattern. The signal storage unit 8124 stores a signal indicating the brightness change pattern. The signal control unit 8121b of the transmission unit 8121 reads the signal stored in the signal storage unit 8124 and changes the brightness of the light emitting unit 8121c and the light emitting unit 8121d according to the signal.

The synchronization signal input unit 8125 acquires a synchronization signal under the control of the centralized control unit 8118. When the synchronization signal is acquired, the synchronization control unit 8126 synchronizes the brightness changes of the transmission unit 8121 and the transmission unit 8122. That is, the synchronization control unit 8126 controls the signal control unit 8121b and the signal control unit 8122b to synchronize the luminance change between the transmission unit 8121 and the transmission unit 8122. Here, the light receiving unit 8127 detects light emission from the transmission unit 8121 and the transmission unit 8122. The synchronization control unit 8126 performs feedback control to the signal control unit 8121b and the signal control unit 8122b according to the light detected by the light receiving unit 8127.

FIG. 58 is a block diagram showing another example of the transmitter according to the third embodiment.

The transmitter 8130 includes a transmission unit 8131 that transmits a signal according to a change in luminance, and a non-transmission unit 8132 that emits light without transmitting a signal.

The transmission unit 8131 has a structure similar to that of the transmitter 8115 illustrated in FIG. 56A and includes a power supply portion 8131a, a signal control portion 8131b, and light emitting portions 8131c to 8131f. Further, the non-transmission unit 8132 includes a power supply unit 8132a and light emitting units 8132c to 8132f, and does not include a signal control unit. That is, when there are a plurality of units including a power source and the synchronous control of the brightness change cannot be performed among these units, only one of the units is provided with the signal control unit as in the configuration shown in FIG. Change the brightness of only one unit.

Here, in such a transmitter 8130, the light emitting units 8131c to 8131f of the transmission unit 8131 are continuously arranged in a line. That is, the set of the light emitting units 8131c to 8131f does not include any of the light emitting units 8132c to 8132f of the non-transmission unit 8132. As a result, the size of the light-emitting body that changes in brightness increases, so that the receiver can easily receive the signal transmitted by the change in brightness.

FIG. 59A is a diagram illustrating an example of a transmitter in Embodiment 3.

The transmitter 8134 is configured as, for example, a signage, and includes three light emitting units (light emitting areas) 8134a to 8134c. The lights from the light emitting units 8134a to 8134c do not interfere with each other. Here, in the case where only one of the light emitting units 8134a to 8134c can be changed in brightness to transmit a signal, the light emitting unit 8134b in the center is set as shown in (a) of FIG. 59A. It is desirable to change the brightness. In the case where two of the light-emitting portions 8134a to 8134c can be changed in luminance, the light-emitting portion 8134b at the center and the light-emitting portion 8134a at the end or the light-emitting portion 8134b as illustrated in FIG. 59A (b). It is desirable to change the luminance of the portion 8134c. By changing the luminance of the light emitting unit in such a position, the receiver can properly receive the signal transmitted by the luminance change.

FIG. 59B is a diagram illustrating an example of a transmitter in Embodiment 3.

The transmitter 8135 is configured as, for example, a signage, and includes three light emitting units 8135a to 8135c. It should be noted that among the light emitting units 8135a to 8135c, lights from adjacent light emitting units interfere with each other. Here, when only one of the light emitting units 8135a to 8135c can be changed in luminance to transmit a signal, as shown in (a) of FIG. 59B, the light emitted at the end is emitted. It is desirable to change the luminance of the portion 8135a or the light emitting portion 8135c. As a result, it is possible to prevent the change in luminance for transmitting a signal from being interfered with the light from another light emitting unit. When two of the light emitting units 8135a to 8135c can be changed in luminance, as shown in FIG. 59B, the light emitting unit 8135b at the center and the light emitting unit 8135a at the end or the light emitting unit 8135a. It is desirable to change the luminance of the portion 8135c. By changing the luminance of the light emitting unit in such a position, the area of the luminance change becomes large, so that the receiver can appropriately receive the signal transmitted by the luminance change.

FIG. 59C is a diagram illustrating an example of a transmitter in Embodiment 3.

When the transmitter 8134 can change the brightness of two of the three light emitting units 8134a to 8134c, the transmitter 8134 can change the brightness of the light emitting units 8134a and 8134c at both ends as shown in FIG. 50C. Good. In this case, in the image pickup by the receiver, it is possible to widen the image pickup range in which the region where the luminance changes is reflected.

FIG. 60A is a diagram illustrating an example of a transmitter in Embodiment 3.

The transmitter 8137 is configured as, for example, a signage, and transmits a signal by changing the luminance of the character portion “A Shop” and the light emitting unit 8137a. The light emitting unit 8137a is formed, for example, in a rectangular shape that is long in the horizontal direction, and has a uniform luminance change. By uniformly changing the luminance of the light emitting unit 8137a, the receiver can appropriately receive the signal transmitted by the luminance change.

FIG. 60B is a diagram illustrating an example of a transmitter in Embodiment 3.

The transmitter 8138 is configured as, for example, a signage, and transmits a signal by changing the luminance of the character portion “A Shop” and the light emitting unit 8138a. The light emitting unit 8138a is formed in a frame shape, for example, along the edge of the signage, and the luminance changes uniformly. That is, the light emitting portion 8138a is formed so that the length of the continuous projected portion becomes maximum when the light emitting portion is projected on an arbitrary straight line. By uniformly changing the brightness of the light emitting unit 8138a, the receiver can more appropriately receive the signal transmitted by the brightness change.

FIG. 61 is a diagram showing an example of processing operations of the receiver, the transmitter, and the server in the third embodiment.

For example, the receiver 8142 configured as a smartphone acquires position information indicating its own position and transmits the position information to the server 8141. Note that the receiver 8142 uses GPS or the like, or acquires the position information thereof when receiving another signal. The server 8141 transmits to the receiver 8142 the ID list associated with the position indicated by the position information. For each ID such as “abcd”, the ID list includes the ID and information associated with the ID.

The receiver 8142 receives a signal from a transmitter 8143 configured as a lighting device, for example. At this time, the receiver 8142 may be able to receive only a part of the ID (for example, "b") as the above signal. In this case, the receiver 8142 searches the ID list for an ID including a part of the ID. If the unique ID is not found, then the receiver 8142 also receives a signal from the transmitter 8143 that contains the other part of the ID. This causes receiver 8142 to obtain a larger portion (eg, "bc") of that ID. Then, the receiver 8142 searches the ID list again for an ID including a part of the ID (for example, "bc"). By performing such a search, the receiver 8142 can specify all the IDs even if only a part of the IDs can be acquired. When receiving the signal from the transmitter 8143, the receiver 8142 receives not only a part of the ID but also a check part such as CRC (Cyclic Redundancy Check).

FIG. 62 is a diagram showing an example of processing operations of the receiver, the transmitter, and the server according to the third embodiment.

For example, the receiver 8152 configured as a smartphone acquires position information indicating its own position. Note that the receiver 8152 acquires the position information when using GPS, for example, or when receiving another signal. Further, the receiver 8152 receives a signal from a transmitter 8153 configured as, for example, a lighting device. At this time, the signal includes only a part of the ID (for example, "b") among the IDs. Here, the receiver 8152 transmits the position information and a part of the ID to the server 8151.

The server 8151 searches the ID list associated with the position indicated by the position information for an ID including a part of the ID. When the unique ID is not found, the server 8151 notifies the receiver 8152 that the identification of the ID has failed.

The receiver 8152 then receives a signal from the transmitter 8153 that includes the other portion of its ID. This causes the receiver 8152 to obtain a larger portion of that ID (eg, "be"). Then, the receiver 8152 transmits a part of the ID (for example, “be”) and the position information to the server 8151.

The server 8151 searches the ID list associated with the position indicated by the position information for an ID including a part of the ID. When the unique ID is found, the server 8151 notifies the receiver 8152 that the ID (for example, “abef”) has been specified, and also transmits the information associated with the ID to the receiver 8152.

FIG. 63 is a diagram showing an example of processing operations of the receiver, the transmitter, and the server in the third embodiment.

The receiver 8152 may transmit not only some of the IDs but also all of them to the server 8151 together with the position information. At this time, when the ID in the complete state (for example, “wxyz”) is not included in the ID list, the server 8151 notifies the receiver 8152 of an error.

FIG. 64A is an explanatory diagram for explaining synchronization of a plurality of transmitters according to the third embodiment.

The transmitter 8155a and the transmitter 8155b transmit signals by changing the luminance. Here, the transmitter 8155a changes the luminance in synchronization with the transmitter 8155b by transmitting the synchronization signal to the transmitter 8155b. In addition, the transmitter 8155a and the transmitter 8155b each acquire a signal from a transmission source and change the brightness according to the signal. Here, the time required for signal transmission from the transmission source to the transmitter 8155a (first delay time) may be different from the time required for signal transmission from the transmission source to the transmitter 8155b (second delay time). is there. Then, the round-trip time of the signal between the transmitters 8155a and 8155b and the transmission source is measured, and 1/2 of the round-trip time is specified as the above-mentioned first or second delay time. The transmitter 8155a changes the brightness in synchronization with the transmitter 8155b by transmitting the synchronization signal so that the difference between the first and second delay times is canceled.

FIG. 64B is an explanatory diagram for explaining synchronization of a plurality of transmitters in the third embodiment.

The light receiving sensor 8156 detects light from the transmitter 8155a and the transmitter 8155b, and outputs the result as a detection signal to the transmitter 8155a and the transmitter 8155b. When the transmitter 8155a and the transmitter 8155b receive the detection signal from the light receiving sensor 8156, the transmitter 8155a and the transmitter 8155b change the brightness in synchronization with each other or adjust the intensity of the signal based on the detection signal.

FIG. 65 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

For example, the transmitter 8165 configured as a television acquires an image and an ID (ID 1000) associated with the image from the control unit 8166. Then, the transmitter 8165 transmits the ID (ID 1000) to the receiver 8167 by displaying the image and changing the brightness. The receiver 8167 receives the ID (ID 1000) by imaging and displays the information associated with the ID (ID 1000). Here, the control unit 8166 changes the image output to the transmitter 8165 to another image. At this time, the control unit 8166 also changes the ID output to the transmitter 8165. That is, the control unit 8166 outputs the other ID (ID 1001) associated with the other image to the transmitter 8165 together with the other image. As a result, the transmitter 8165 transmits another ID (ID 1001) to the receiver 8167 by displaying another image and changing the brightness. The receiver 8167 receives the other ID (ID 1001) by imaging, and displays the information associated with the other ID (ID 1001).

FIG. 66 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

The transmitter 8170 is configured as, for example, a signage, and switches and displays images. Then, when displaying the image, the transmitter 8170 transmits the ID time information indicating the ID corresponding to the displayed image and the time when the image is displayed to the receiver 8171 by changing the brightness. For example, the transmitter 8170 displays an image showing a round figure at time t1, and displays the ID (ID:1000) corresponding to the image and the time (TIME:t1) at which the image is displayed. The ID time information shown is transmitted.

Here, the transmitter 8170 transmits not only the ID time information corresponding to the currently displayed image, but also at least one ID time information corresponding to the previously displayed image. For example, the transmitter 8170 displays an image showing a square figure at time t2, and also displays the ID (ID: 1001) corresponding to the image and the time (TIME: t2) at which the image is displayed. The ID time information shown is transmitted. Further, at this time, the transmitter 8170 transmits ID time information indicating the ID (ID: 1000) corresponding to the image showing the round figure and the time (TIME: t1) when the image was displayed. Similarly, at time t3, the transmitter 8170 displays an image showing a triangular figure, and also displays an ID (ID: 1002) corresponding to the image and a time (TIME: t3) at which the image is displayed. Is transmitted. Further, at this time, the transmitter 8170 transmits ID time information indicating the ID (ID: 1001) corresponding to the image showing the square figure and the time (TIME: t2) when the image was displayed. That is, the transmitter 8170 transmits a plurality of ID time information at the same timing.

For example, the user holds the image sensor of the receiver 8171 over the transmitter 8170 at time t2 when the image showing the square figure is displayed in order to obtain information related to the image showing the square figure. Imaging by the 8171 is started.

Here, the receiver 8171 cannot acquire the ID time information corresponding to the image while the image showing the square figure is displayed on the transmitter 8170 even if the image pickup is started at the time t2. There are cases. Even in such a case, as described above, since the ID time information corresponding to the image displayed in the past is also transmitted from the transmitter 8170, the receiver 8171 has a triangular shape at time t3. Not only the ID time information (ID: 1002, TIME: t3) corresponding to the image showing the figure, but also the ID time information (ID: 1001, TIME: t2) corresponding to the image showing the square figure can be acquired. Then, the receiver 8171 selects ID time information (ID: 1001, TIME: t2) indicating the time (t2) held over the transmitter 8170 from the ID time information, and uses the ID time information to select the ID time information. The indicated ID (ID: 1001) is specified. As a result, the receiver 8171 can obtain information about the image showing the square figure from the server or the like at the time t3 based on the specified ID (ID: 1001).

The above-mentioned time is not limited to an absolute time, but a time between the time when the receiver 8171 is held over the transmitter 8170 and the time when the receiver 8171 acquires the ID time information (so-called relative time). May be Further, the transmitter 8170 transmits the ID time information corresponding to the image currently displayed and the ID time information corresponding to the image displayed in the past, but corresponds to the image to be displayed in the future. The ID time information may be transmitted. Further, the transmitter 8170 may increase the number of past or future ID time information to be transmitted when the receiver 8171 is in a situation where reception is difficult.

Further, when the transmitter 8170 is configured as a television instead of a signage, the transmitter 8170 may transmit information indicating the channel corresponding to the displayed image instead of the ID time information. That is, when the image of the television program being broadcast is displayed on the transmitter 8170 in real time, the display time of the image displayed on the transmitter 8170 can be uniquely specified for each channel. Therefore, the receiver 8171 specifies the time when the receiver 8171 is held over the transmitter 8170, that is, the time when the receiver 8171 starts the imaging, based on the image obtained by the imaging and the channel thereof. You can Then, the receiver 8171 can obtain information regarding the image obtained by the image pickup from, for example, a server based on the channel and the time. Note that the transmitter 8170 may transmit information indicating the display time of the displayed image instead of the ID time information. In this case, the receiver 8171 searches all TV programs being broadcast at that time for a TV program including an image obtained by imaging, and selects the TV program channel and its display time. Based on that, information related to the image can be obtained from a server or the like.

FIG. 67 is a diagram showing an example of operations of a transmitter, a receiver, and a server in Embodiment 3.

As shown in (a) of FIG. 67, the receiver 8176 captures an image of the transmitter 8175 to acquire an image including a bright line, and identifies (acquires) the ID of the transmitter 8175 from the image. Further, the receiver 8176 transmits the ID to the server 8177 and acquires the information associated with the ID from the server 8177.

On the other hand, as shown in (b) of FIG. 67, the receiver 8176 may acquire an image including a bright line by capturing an image of the transmitter 8175 and transmit the image to the server 8177 as captured image data. Further, the receiver 8176 may perform preprocessing on the image including the bright line so that the information amount of the image is reduced, and transmit the preprocessed image to the server 8177 as imaging data. This preprocessing is, for example, image binarization processing. When the server 8177 acquires the imaging data, the server 8177 identifies (acquires) the ID of the transmitter 8175 from the image indicated by the imaging data. Further, the server 8177 transmits the information associated with the ID to the receiver 8176.

FIG. 68 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

When the user is at the position A, the receiver 8183 identifies the position of the receiver 8183 by acquiring the signal transmitted from the transmitter 8181 whose brightness changes. As a result, the receiver 8183 displays the point 8183b indicating the specified position together with the error range 8183a of the position.

Next, when the user moves from the position A and reaches the position B, the receiver 8183 is in a state where it cannot acquire a signal from the transmitter 8181. At this time, the receiver 8183 estimates its own position by using a 9-axis sensor or the like provided in itself. Then, the receiver 8183 displays the point 8183b indicating the estimated position together with the error range 8183a of the position. At this time, since the position is estimated by the 9-axis sensor, the error range 8183a is displayed broadly.

Next, when the user moves from the position B to the position C, the receiver 8183 identifies the position of the receiver 8183 by acquiring a signal transmitted from another transmitter 8182 whose brightness changes. As a result, the receiver 8183 displays the point 8183b indicating the specified position together with the error range 8183a of the position. Here, the receiver 8183 smoothly switches the point 8183b indicating the position estimated by using the 9-axis sensor and the error range 8183a without immediately switching to and displaying the position and the error range specified as described above. Move and switch between them. At this time, the error range 8183a becomes smaller.

FIG. 69 is a diagram illustrating an example of an outer appearance of the receiver in Embodiment 3.

The receiver 8183 is configured as, for example, a smartphone (high-performance mobile phone), and as shown in (a) of FIG. 69, an image sensor 8183c, an illuminance sensor 8183d, and a display 8183e are arranged on the front surface of the receiver 8183. ing. The image sensor 8183c acquires an image including a bright line by capturing the subject whose brightness changes as described above. The illuminance sensor 8183d detects the above-mentioned change in the brightness of the subject. Therefore, the illuminance sensor 8183d can be used instead of the image sensor 8183c depending on the state or condition of the subject. The display 8183e displays an image and the like. Here, the receiver 8183 may have a function as a subject whose luminance changes. In this case, the receiver 8183 transmits a signal by changing the brightness of the display 8183e.

Further, as shown in (b) of FIG. 69, an image sensor 8183f, an illuminance sensor 8183g, and a flash light emitting unit 8183h are arranged on the back surface of the receiver 8183. The image sensor 8183f is the same as the above-described image sensor 8183c, and acquires an image including a bright line by capturing the subject whose brightness changes as described above. The illuminance sensor 8183g is the same as the above-mentioned illuminance sensor 8183d, and detects the brightness change of the subject. Therefore, the illuminance sensor 8183g can be used instead of the image sensor 8183f depending on the state or situation of the subject. The flash light emitting unit 8183h emits a flash for imaging. Here, the receiver 8183 may have a function as a subject whose luminance changes, and in this case, the signal is transmitted by changing the luminance of the flash light emitting unit 8183h.

FIG. 70 is a diagram showing an example of operations of the transmitter, the receiver, and the server in the third embodiment.

For example, the transmitter 8185 configured as a smartphone displays information indicating, for example, "coupon 100 yen discount" by changing the brightness of a portion of the display 8185a other than the barcode portion 8185b, that is, by visible light communication. Send. Further, the transmitter 8185 displays the barcode on the barcode portion 8185b without changing the brightness of the barcode portion 8185b. This bar code shows the same information as the information transmitted by the visible light communication described above. Further, the transmitter 8185 displays a character or a picture indicating the information transmitted by the visible light communication, for example, the character "coupon 100 yen discount" on a portion of the display 8185a other than the barcode portion 8185b. By displaying such characters or pictures, the user of the transmitter 8185 can easily understand what information is being transmitted.

The receiver 8186 acquires the information transmitted by visible light communication and the information indicated by the barcode by capturing an image, and transmits the information to the server 8187. The server 8187 determines whether or not these pieces of information match or relate to each other, and when it determines that the pieces of information match or are related to each other, executes the processing according to the information. Alternatively, the server 8187 transmits the determination result to the receiver 8186 and causes the receiver 8186 to execute processing according to the information.

Note that the transmitter 8185 may transmit part of the information indicated by the barcode by visible light communication. Further, the URL of the server 8187 may be shown in the barcode. Also, the transmitter 8185 may acquire an ID as a receiver and transmit the ID to the server 8187 to acquire the information associated with the ID. The information associated with this ID is the same as the information transmitted by the visible light communication or the information indicated by the barcode. In addition, the server 8187 may transmit to the transmitter 8185 an ID associated with information (visible light communication information or barcode information) transmitted from the transmitter 8185 via the receiver 8186.

71 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG.

For example, the transmitter 8185 configured as a smartphone transmits a signal by changing the brightness of the display 8185a. The receiver 8188 includes a light-shielding cone-shaped container 8188b and an illuminance sensor 8188a. The illuminance sensor 8188a is stored inside the container 8188b and is arranged near the tip of the container 8188b. When a signal is transmitted from the transmitter 8185 by visible light communication, the opening (bottom) of the container 8188b of the receiver 8188 is directed to the display 8185a. As a result, light other than the light from the display 8185a does not enter the container 8188b, so that the illuminance sensor 8188a of the receiver 8188 appropriately receives the light from the display 8185a without being affected by the light that becomes noise. Can receive light. As a result, the receiver 8188 can properly receive the signal from the transmitter 8185.

72 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. [FIG.

The transmitter 8190 is configured as, for example, a beacon at a bus stop, and changes the brightness to transmit operation information indicating the operation status of the bus to the receiver 8183. For example, the destination of the bus, the time when the destination bus arrives at the bus stop, and operation information indicating the current location of the bus are transmitted to the receiver 8183. Upon receiving the operation information, the receiver 8183 displays the content indicated by the operation information on the display.

Here, for example, when buses of different destinations stop at the bus stop, the transmitter 8190 transmits operation information on the buses of those destinations. Upon receiving the operation information, the receiver 8183 selects the operation information of the destination bus that is frequently used by the user from the operation information and displays the content indicated by the operation information on the display. To do. Specifically, the receiver 8183 specifies the destination of the bus used by the user by using, for example, GPS and records the history of the destination. The receiver 8183 selects the operation information of the destination bus that is frequently used by the user by referring to this history. Alternatively, the receiver 8183 may display, on the display, the content indicated by the operation information selected by the user operation from the operation information. Alternatively, the receiver 8183 may preferentially display the operation information of the destination bus that is frequently selected by the user's operation.

FIG. 73 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

For example, the transmitter 8191 configured as a signage transmits the information of a plurality of stores to the receiver 8183 by changing the brightness. This information is a collection of information about a plurality of stores, and is not unique to each store. Therefore, when the receiver 8183 receives the information by imaging, the receiver 8183 can display the information regarding not only one store but a plurality of stores. Here, the receiver 8183 selects the information regarding the store (for example, “B shop”) included in the imaging range from the information regarding the plurality of stores, and displays the selected information. Further, when displaying the information, the receiver 8183 translates the language for expressing the information into a language registered in advance, and displays the information in the translated language. Further, the transmitter 8191 may display a message prompting the user to take an image with the image sensor (camera) of the receiver 8183 in characters or the like. Specifically, a message (for example, “Get information with camera”) is displayed on the transmitter 8191 informing that information can be provided by activating a dedicated application program and capturing an image with the camera. It

FIG. 74 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

For example, the receiver 8183 images a subject including a plurality of persons 8197 and a street lamp 8195. The street lamp 8195 includes a transmitter 8195a that transmits information according to a change in brightness. By this imaging, the receiver 8183 acquires an image in which the image of the transmitter 8195a appears as the above-described bright line pattern. Further, the receiver 8183 acquires the AR object 8196a associated with the ID indicated by the bright line pattern from, for example, the server. Then, the receiver 8183 superimposes the AR object 8196a on the normal captured image 8196 obtained by the normal capturing, and displays the normal captured image 8196 on which the AR object 8196a is superimposed.

FIG. 75A is a diagram showing an example of a configuration of information transmitted by the transmitter according to the third embodiment.

For example, the information transmitted by the transmitter includes a preamble part, a fixed-length data part, and a check part. The receiver checks the data part using the check part and normally receives the information composed of these parts. Here, when the receiver receives the preamble part and the data part and cannot receive the check part, the check using the check part is omitted. Even if such a check is omitted, the receiver can normally receive the information including the respective parts.

FIG. 75B is a diagram showing another example of the structure of information transmitted by the transmitter in Embodiment 3.

For example, the information transmitted by the transmitter comprises a preamble part, a check part, and a variable length data part. The next information transmitted by the transmitter also consists of a preamble part, a check part and a variable length data part. Here, when the receiver receives the preamble part during reception and further receives the next preamble part, the information from the previous preamble part to immediately before the next preamble part has one meaning. Recognize as information. Further, the receiver may specify the end of the data section received next to the check section by using the check section. In this case, even if the receiver cannot receive the next preamble part (all or part of the preamble part) described above, it properly receives the one meaning information transmitted immediately before. be able to.

FIG. 76 is a diagram showing an example of a four-value PPM modulation method by the transmitter in the third embodiment.

The transmitter modulates a signal to be transmitted into a luminance change pattern by the 4-value PPM modulation method. At this time, the transmitter can keep the brightness of the light whose brightness is changed constant regardless of the signal to be transmitted.

For example, when the brightness is kept at 75%, the transmitter transmits each of the transmission target signals “00”, “01”, “10” and “11” to any one of four consecutive slots. One modulates the brightness L (Low), and the other three modulate to a brightness change pattern that represents the brightness H (High). Specifically, the transmitter transmits the signal “00” to be transmitted, in which the first slot indicates the luminance L and the second to fourth slots indicate the luminance H in a luminance change pattern (L, H, H, H). That is, in this brightness change, there is a rise in brightness between the first slot and the second slot. Similarly, the transmitter transmits the signal “01” to be transmitted, in which the second slot indicates the luminance L and the first, third, and fourth slots indicate the luminance H in a pattern of luminance change (H, L, H, H). That is, in this brightness change, there is a rise in brightness between the second slot and the third slot.

Further, when the brightness is kept at 50%, the transmitter transmits each of the transmission target signals “00”, “01”, “10” and “11” to any two of the four slots. Luminance L (Low) is indicated, and the other two are modulated into a luminance change pattern indicating luminance H (High). Specifically, the transmitter transmits the signal “00” to be transmitted, in which the first and fourth slots indicate the luminance L, and the second and third slots indicate the luminance H in a luminance change pattern (L, L, H, H, L). That is, in this brightness change, there is a rise in brightness between the first slot and the second slot. Similarly, the transmitter transmits the signal “01” to be transmitted, in which the first and second slots indicate the luminance L and the third and fourth slots indicate the luminance H in a luminance change pattern (L, L, H, H). Alternatively, the transmitter transmits the signal “01” to be transmitted, in which the second and fourth slots indicate luminance L and the first and third slots indicate luminance H (H, L, H). , L). That is, in these changes in brightness, there is a rise in brightness between the second slot and the third slot.

Further, when the brightness is kept at 25%, the transmitter transmits each of the transmission target signals “00”, “01”, “10” and “11” to any three of the four slots. The brightness L (Low) is shown, and the other one is modulated into a brightness change pattern showing the brightness H (High). Specifically, the transmitter transmits the signal “00” to be transmitted, in which the first, third and fourth slots indicate the luminance L and the second slot indicates the luminance H (L, L). H, L, L). That is, in this brightness change, there is a rise in brightness between the first slot and the second slot. Similarly, the transmitter transmits the signal “01” to be transmitted, in which the first, second and fourth slots indicate the luminance L and the third slot indicates the luminance H (L, L). L, H, L). That is, in this brightness change, there is a rise in brightness between the second slot and the third slot.

The transmitter can suppress the flicker and easily adjust the brightness stepwise by the four-value PPM modulation method as described above. Further, the receiver can appropriately demodulate the pattern of the brightness change by specifying the position of the rise of the brightness. It should be noted that the receiver ignores the presence/absence of luminance rise at the boundary between the slot group consisting of four slots and the next slot group without using it for demodulating the luminance change pattern.

FIG. 77 is a diagram illustrating an example of a PPM modulation method performed by the transmitter in Embodiment 3.

The transmitter modulates the signal to be transmitted into a luminance change pattern in the same manner as the four-level PPM modulation method shown in FIG. 76, but performs PPM modulation without switching the luminance between L and H for each slot. Good. That is, the transmitter performs PPM modulation by switching the rising position of the luminance in the time width (hereinafter, referred to as a unit time width) of four continuous slots shown in FIG. 76 according to the signal to be transmitted. For example, as illustrated in FIG. 77, the transmitter modulates the transmission target signal “00” into a luminance change pattern in which the luminance rises at a position of 25% of the unit time width. Similarly, as shown in FIG. 77, the transmitter modulates the transmission target signal “01” into a luminance change pattern in which the luminance rises at a position of 50% of the unit time width.

Further, when the brightness is kept at 75%, the transmitter indicates the luminance “L” of the signal “00” to be transmitted at the position of 0 to 25% in the unit time width described above, and shows the luminance L at the position of 25 to 100%. Is modulated into a brightness change pattern showing brightness H. Here, in the case of maintaining the brightness at 99%, the transmitter indicates the signal “00” to be transmitted, the brightness L at the position of 24 to 25% in the unit time width described above, and the brightness L of 0 to 24%. Modulation is performed on the position and the brightness change pattern indicating the brightness H at the position of 25 to 100%. Similarly, when maintaining the brightness at 1%, the transmitter sets the signal "00" to be transmitted to the brightness L at the position of 0 to 25% and the position of 26 to 100% in the unit time width described above. It is modulated to a brightness change pattern showing the brightness H at the position of 25 to 26%.

In this way, the brightness can be continuously adjusted by switching the brightness between L and H at an arbitrary position of the unit time width without switching the brightness between L and H for each slot.

FIG. 78 is a diagram illustrating an example of a PPM modulation method in the transmitter in Embodiment 3.

The transmitter performs modulation in the same manner as the PPM modulation method shown in FIG. 77, but whatever signal the signal to be transmitted is, the signal always shows the brightness H at the beginning of the unit time width. In addition, at the end of the unit time width, the pattern is always modulated to the brightness change pattern showing the brightness L. As a result, a rise in luminance occurs at the boundary between the unit time width and the next unit time width, so that the receiver can appropriately identify the boundary. Therefore, the receiver and the transmitter can correct the clock shift.

FIG. 79A is a diagram showing an example of a luminance change pattern corresponding to the header (preamble portion) in the third embodiment.

When the transmitter transmits, for example, the header (preamble part) shown in FIGS. 75A and 75B, the brightness changes according to the pattern shown in FIG. 79A. That is, when the header is composed of 7 slots, the transmitter changes in luminance according to the pattern indicated by L, H, L, H, L, H, H. If the header is composed of 8 slots, the transmitter changes in brightness according to the pattern indicated by H, L, H, L, H, L, H, H. Since these patterns can be distinguished from the luminance change patterns shown in FIG. 76, it is possible to clearly inform the receiver that the signal shown by these patterns is a header.

FIG. 79B is a diagram showing an example of a luminance change pattern in the third embodiment.

As shown in FIG. 76, in the four-level PPM modulation method, when the signal “01” to be transmitted included in the data part is modulated with the brightness kept at 50%, the transmitter is Is modulated into one of two patterns. That is, the modulation is performed to the first pattern represented by L, L, H, H or the second pattern represented by H, L, H, L.

Here, when the pattern of the luminance change corresponding to the header is the pattern as shown in FIG. 79A, the transmitter transmits the above-mentioned signal “01” to be transmitted as the first signal indicated by L, L, H, and H. It is desirable to modulate the pattern. For example, the signals “11, 01, 11” to be transmitted, which are included in the above data part, are “H, H, L, L, L, L, H, H when the first pattern is used. , H, H, L, L” pattern. On the other hand, when the second pattern is used, the transmission target signals “11, 01, 11” included in the above-mentioned data part are “H, H, L, L, H, L, H, L. , H, H, L, L” pattern. In this case, the pattern "H, H, L, L, H, L, H, L, H, H, L, L" has the same pattern as the header pattern composed of 7 slots shown in FIG. 79A. Is included. Therefore, in order to clearly distinguish the header and the data part, it is desirable to modulate the signal “01” to be transmitted to the first pattern.

FIG. 80A is a diagram showing an example of a luminance change pattern in the third embodiment.

As shown in FIG. 76, in the four-level PPM modulation method, when the signal “11” to be transmitted is modulated, the transmitter outputs “H, H, H” so as to prevent rising of the luminance. , L” pattern, “H, H, L, L” pattern, or “H, L, L, L” pattern. However, as shown in FIG. 80A, in order to adjust the brightness, the transmitter transmits the signal “11” to be transmitted as a pattern of “H, H, H, H” or “L, L, L, L”. May be modulated into a pattern.

FIG. 80B is a diagram showing an example of a luminance change pattern in the third embodiment.

As shown in FIG. 76, in the 4-value PPM modulation method, when the signal “11:00” to be transmitted is modulated while maintaining the brightness at 75%, the transmitter outputs the signal as “H, H”. , H, L, L, H, H, H” pattern. However, when the luminance L is going to occur continuously, the luminance other than the last luminance L among the continuous luminance L may be changed to H so that the luminance L does not continue. In this case, the transmitter modulates the signal "11:00" into a pattern of "H, H, H, H, L, H, H, H".

As a result, the luminance L is not continuous, so that the load on the transmitter can be suppressed. Further, the capacity of the capacitor provided in the transmitter can be reduced, and the volume of the control circuit can be reduced. Further, since the load on the light source of the transmitter is small, the light source can be easily manufactured. Furthermore, the power efficiency of the transmitter can be increased. Further, since it is guaranteed that the luminance L is not continuous, the receiver can easily demodulate the pattern of the luminance change.

(Summary of this Embodiment)
The information communication method according to the present embodiment is an information communication method of transmitting a signal according to a luminance change, and a determining step of determining a pattern of luminance change by modulating a signal to be transmitted, and a light emitting body determining And a step of transmitting the signal to be transmitted by changing the luminance according to the pattern, wherein the pattern of luminance change has two different luminances at arbitrary positions in a predetermined time width. One of the values is a pattern that appears, and in the determining step, for each of the different signals to be transmitted, the brightness change position that is the rising position or the falling position of the brightness in the time width is different from each other. Further, the pattern of the brightness change is determined so that the integrated value of the brightness of the light emitter in the time width becomes the same value according to the preset brightness.

For example, as shown in FIG. 77, for each of the signals “00”, “01”, “10”, and “11” that are different from each other to be transmitted, the rising positions of brightness (brightness change positions) are different from each other, and , The luminance change so that the integrated value of the luminance of the light emitter in the predetermined time width (unit time width) becomes the same value according to the predetermined brightness (eg, 99% or 1%) Pattern is determined. This makes it possible to keep the brightness of the light emitter constant for each of the signals to be transmitted, suppress flicker, and allow the receiver that images the light emitter to detect the brightness based on the brightness change position. , The luminance change pattern can be demodulated appropriately. In addition, since the brightness change pattern is a pattern in which one of two different brightness values (brightness H (High) or brightness L (Low)) appears at any position in the unit time width, light emission The brightness of the body can be changed continuously.

Further, the information communication method further includes an image display step of sequentially switching and displaying each of the plurality of images, and in the determining step, the image displayed is displayed every time the image is displayed in the image display step. By modulating the identification information corresponding to the signal as the signal to be transmitted, the pattern of the luminance change with respect to the identification information is determined, and in the transmitting step, each time the image is displayed in the image displaying step, the pattern is displayed. The identification information may be transmitted by changing the brightness of the light emitter according to a brightness change pattern determined for the identification information corresponding to the existing image.

As a result, as shown in FIG. 65, for example, every time an image is displayed, the identification information corresponding to the displayed image is transmitted, so that the user receives the image based on the displayed image at the receiver. The identification information to be made can be easily selected.

In addition, in the transmitting step, each time an image is displayed in the image displaying step, the luminous body is further lit according to a brightness change pattern determined for identification information corresponding to an image displayed in the past. The identification information may be transmitted by changing.

Accordingly, for example, as shown in FIG. 66, even when the receiver cannot receive the identification signal transmitted before the switching because the displayed image is switched, the identification information corresponding to the currently displayed image is displayed. At the same time, since the identification information corresponding to the image displayed in the past is also transmitted, the identification information transmitted before the switching can be properly received again by the receiver.

Further, in the determining step, every time an image is displayed in the image displaying step, the identification information corresponding to the displayed image and the time when the image is displayed are modulated as the signal to be transmitted. By determining the pattern of the luminance change with respect to the identification information and the time, the transmission step, with respect to the identification information and time corresponding to the displayed image every time the image is displayed in the image display step. The identification information and the time are transmitted by changing the luminance of the luminous body according to the pattern of the luminance change determined as described above, and the identification information and the time corresponding to the image displayed in the past are determined. The identification information and the time may be transmitted by changing the luminance of the light emitter according to a pattern of luminance change.

Thereby, as shown in FIG. 66, for example, each time an image is displayed, a plurality of pieces of ID time information (information including identification information and time) are transmitted, so that the receiver receives a plurality of received ID times. It is possible to easily select, from the information, the identification signal transmitted in the past and not received, based on the time included in each of the ID time information.

In addition, the light emitter has a plurality of regions that respectively emit light, and lights of adjacent regions of the plurality of regions interfere with each other, and only one of the plurality of regions is determined. When the brightness changes according to the brightness change pattern, in the transmitting step, only the regions arranged at the ends of the plurality of regions may change the brightness according to the determined brightness change pattern.

As a result, as shown in (a) of FIG. 59B, for example, the luminance changes only in the regions (light emitting portions) arranged at the edges, so that the luminance changes only in the regions other than the edges. It is possible to suppress the influence of the light from the area on the luminance change. As a result, the receiver can appropriately capture the pattern of the brightness change by photographing.

When only two of the plurality of areas change in brightness according to the determined pattern of the change in brightness, the transmitting step includes an area arranged at an end of the plurality of areas, The area adjacent to the area arranged at the edge may change in brightness according to the determined pattern of the change in brightness.

As a result, for example, as shown in (b) of FIG. 59B, the area (light emitting portion) arranged at the end and the area (light emitting portion) adjacent to the area arranged at the end change in luminance, so that they are mutually different. Compared with the case where the distant area changes in luminance, the area of the range in which the luminance changes spatially continuously can be kept wide. As a result, the receiver can appropriately capture the pattern of the brightness change by photographing.

The information communication method according to the present embodiment is an information communication method of acquiring information from a subject, and a position information transmitting step of transmitting position information indicating a position of an image sensor used for photographing the subject, and the position information. A list receiving step of receiving an ID list including a plurality of pieces of identification information associated with the position indicated by, and an image obtained by photographing the subject by the image sensor, corresponding to an exposure line included in the image sensor. Exposure time setting step of setting the exposure time of the image sensor so that a bright line occurs according to the change in the brightness of the subject, and the image sensor shoots the subject whose brightness changes with the set exposure time. By performing an image acquisition step of acquiring a bright line image including the bright line, an information acquisition step of acquiring information by demodulating the data specified by the pattern of the bright line included in the acquired bright line image, A search step of searching the ID list for identification information including the acquired information.

As a result, as shown in FIG. 61, for example, since the ID list has been received in advance, even if the acquired information “bc” is only a part of the identification information, the appropriate identification information “ abcd" can be specified.

Further, in the search step, when the identification information including the acquired information is not uniquely specified, new information is acquired by repeating the image acquisition step and the information acquisition step, and the information communication is performed. The method may further include a re-search step of searching the ID list for identification information including the acquired information and the new information.

Thereby, as shown in FIG. 61, for example, even when the acquired information "b" is only a part of the identification information and the identification information is not uniquely specified by only the information, the new information " Since c "is acquired, the appropriate identification information "abcd" can be specified based on the new information and the ID list.

The information communication method according to the present embodiment is an information communication method for acquiring information from a subject, in which an image obtained by photographing the subject with an image sensor has a bright line corresponding to an exposure line included in the image sensor. An exposure time setting step of setting an exposure time of the image sensor so as to occur according to a change in brightness of the subject, and the image sensor, by photographing the subject whose brightness changes, at the set exposure time, An image acquisition step of acquiring a bright line image including the bright line, an information acquisition step of acquiring identification information by demodulating data specified by the pattern of the bright line included in the acquired bright line image, and acquired. The transmitting step of transmitting the identification information and the position information indicating the position of the image sensor, the ID list including a plurality of identification information associated with the position indicated by the position information, the acquired identification When there is no information, an error receiving step of receiving error notification information for notifying an error is included.

Thus, for example, as shown in FIG. 63, when the acquired identification information is not in the ID list, the error notification information is received, so that the user of the receiver who has received the error notification information has acquired the error notification information. It is possible to easily understand that the information associated with the identification information cannot be obtained.

(Embodiment 4)
In the present embodiment, an application example for each situation will be described that uses the receiver such as the smartphone in the first to third embodiments and the transmitter that transmits information as a blinking pattern such as an LED or an organic EL.

<Situation: In front of the store>
First, an application example in a situation where there is a user carrying a receiver in front of a store holding an advertising signboard configured as a transmitter will be described with reference to FIGS. 81 to 85.

FIG. 81: is a figure which shows an example of operation|movement of the receiver in the situation in front of a store.

For example, the user finds the signboard 8301 of the store while walking while carrying the receiver 8300 (terminal device) configured as a smartphone. The signboard 8301 is a transmitter (subject) that transmits a signal according to a change in luminance like the transmitter according to any of the first to third embodiments. Here, when the user is interested in the store and determines that the signboard 8301 is transmitting a signal due to a change in luminance, the user operates the receiver 8300 to use the receiver 8300 for visible light communication. Application software (hereinafter referred to as communication application) is started.

FIG. 82: is a figure which shows the other example of operation|movement of the receiver 8300 in the situation in front of a store.

The receiver 8300 may automatically start the communication application without receiving a user operation. For example, the receiver 8300 detects its own current location by using a GPS or a 9-axis sensor, and determines whether or not the current location has entered a specific area predetermined for the signboard 8301. Note that this specific area is an area around the signboard 8301. When the receiver 8300 determines that the current location of the receiver 8300 has entered the specific area, the receiver 8300 activates the communication application. The receiver 8300 may detect the operation of pushing out the receiver 8300 or the operation of rotating the receiver 8300 by using a built-in 9-axis sensor or the like, and activate the communication application. As a result, the operation of the user can be omitted and the usability can be improved.

FIG. 83: is a figure which shows an example of the following operation|movement of the receiver 8300 in the situation in front of a store.

The receiver 8300 that has activated the communication application as described above captures an image (visible light imaging) of the signboard 8301 configured as a transmitter that transmits a signal according to a change in brightness. That is, the receiver 8300 performs visible light communication with the signboard 8301.

FIG. 84: is a figure which shows an example of the following operation|movement of the receiver 8300 in the situation in front of a store.

The receiver 8300 acquires an image including a bright line by the above-mentioned imaging. Then, the receiver 8300 acquires the device ID of the signboard 8301 by demodulating the data specified by the bright line pattern. That is, the receiver 8300 acquires the device ID from the signboard 8301 by visible light imaging or visible light communication in the first to third embodiments. Further, the receiver 8300 transmits the device ID to the server and acquires the advertisement information (service information) associated with the device ID from the server.

Note that the receiver 8300 may acquire the advertisement information associated with the device ID from among the plurality of pieces of advertisement information held in advance. In this case, when the receiver 8300 determines that the current location of the receiver 8300 has entered the above-described specific area, the receiver 8300 notifies the server of the specific area or the current location, and all device IDs corresponding to the specific area are input. , And the advertisement information associated with each of the device IDs is acquired in advance from the server and held (cached). Accordingly, when the receiver 8300 acquires the device ID of the signboard 8301 within the specific area, the receiver 8300 associates the device IDs stored in advance with the device IDs without requesting the server for the advertisement information corresponding to the device IDs. The advertisement information associated with the device ID of the signboard 8301 can be quickly acquired from the obtained advertisement information.

Upon receiving the advertisement information associated with the device ID of the signboard 8301, the receiver 8300 displays the advertisement information. For example, the receiver 8300 displays the coupon of the store indicated by the signboard 8301, the vacant seat status, and the barcode indicating the same contents as those.

Here, the receiver 8300 may obtain not only the device ID but also privilege data from the signboard 8301 by visible light communication. This privilege data indicates, for example, a random ID (random number), or the time or time zone when the privilege data is transmitted. When receiving the privilege data, the receiver 8300 transmits the privilege data to the server together with the device ID. Then, the receiver 8300 acquires the advertisement information associated with the device ID and the privilege data from the server. Accordingly, the receiver 8300 can receive different advertisement information according to the privilege data. For example, if the time zone when the image of the signboard 8301 is captured is early in the morning, the receiver 8300 can acquire and display the advertisement information indicating the early morning discount coupon. That is, it is possible to change the advertisement according to the same signboard according to the privilege data (time zone, etc.). As a result, the user can be provided with a service suitable for the time of day. In this embodiment, the presentation (display) of information such as service information to the user is referred to as service provision.

In addition, the receiver 8300 may obtain three-dimensional information indicating the spatial arrangement of the signboard 8301 with high accuracy (within an error of 1 m) together with the device ID from the signboard 8301 by visible light communication. Alternatively, the receiver 8300 may acquire the three-dimensional information associated with the device ID from the server. Further, the receiver 8300 may acquire size information indicating the size of the signboard 8301 instead of or together with the three-dimensional information. When the receiver 8300 acquires the size information, the receiver 8300 displays the signboard from the receiver 8300 based on the difference between the size of the signboard 8301 indicated by the size information and the size of the signboard 8301 displayed in the image obtained by imaging. The distance to 8301 can be calculated.

Further, when the receiver 8300 transmits the device ID acquired by visible light communication to the server, the receiver 8300 may transmit holding information (adjunct information) held in advance to the server together with the device ID. For example, the retained information is personal information (such as gender or age) of the user of the receiver 8300 or a user ID. The server that has received such holding information together with the device ID receives the advertising information associated with the holding information (personal information or user ID) among the at least one piece of advertising information associated with the device ID. Send to. That is, the receiver 8300 can receive advertisement information of the store that matches personal information or the like, or advertisement information of the store that corresponds to the user ID. As a result, the user can receive a more useful service.

Alternatively, the retained information indicates the reception condition preset in the receiver 8300. The reception condition is the number of visitors when the store is a restaurant, for example. The server, which has received such holding information together with the device ID, transmits to the receiver 8300, among the at least one piece of advertisement information associated with the device ID, the advertisement information associated with the reception condition (the number of visitors). .. That is, the receiver 8300 can receive the advertisement information of the store suitable for the number of visitors, specifically, the information indicating the vacant seat status with respect to the number of visitors. In addition, the store can optimize the attracting of customers and profit by displaying the advertisement information with the discount rate changed according to the number of people visiting the store, the day of the week, and the time of day.

Alternatively, the retained information indicates the current location previously detected by the receiver 8300. The server that receives such holding information together with the device ID receives not only the advertisement information associated with the device ID, but also other device IDs corresponding to the current location (current location and its surroundings) indicated by the holding information. And the advertisement information associated with the device ID of 1. This allows the receiver 8300 to cache the other device ID and the advertisement information associated with the other device ID. Therefore, when the receiver 8300 performs visible light communication with another transmitter at its current position (current position and its surroundings), the advertisement associated with the device ID of the other transmitter without accessing the server. You can get information quickly.

FIG. 85: is a figure which shows an example of the following operation|movement of the receiver 8300 in the situation in front of a store.

Upon receiving the advertising information from the server as described above, the receiver 8300 displays, for example, a button described as “vacant seat available” as the vacant seat status indicated by the advertising information. Here, when the user performs an operation of touching the button “vacant seat” with a finger, the receiver 8300 notifies the server of the operation result. Upon receiving the notification, the server makes a tentative reservation for the store of the signboard 8301 and notifies the receiver 8300 that the tentative reservation has been completed. Upon receiving the notification from the server, the receiver 8300 displays the character string "temporary reservation" indicating that the temporary reservation has been completed, instead of the button "vacant seat". The receiver 8300 stores in the memory an image including the coupon of the store indicated by the signboard 8301, the character string “provisional reservation” certifying that the temporary reservation is made, and the bar code showing the same contents as the pre-acquired image. Store it.

Here, the server can log information regarding visible light communication performed between the signboard 8301 and the receiver 8300 by the operation described with reference to FIGS. 84 and 85. That is, the server is privileged data indicating the device ID of the transmitter (signboard) that has performed visible light communication, the location where visible light communication has been performed (the current location of the receiver 8300), the time zone during which visible light communication has been performed, and the like. , And personal information of the user of the receiver 8300 that has performed visible light communication can be logged. The server can analyze the value of the signboard 8301, that is, the degree of contribution of the signboard 8301 to the advertisement and promotion of the store, as an advertising effect by using at least one of the logged information.

<Situation: Inside the store>
Next, an application example in a situation where a user carrying the receiver 8300 enters a store corresponding to the displayed advertisement information (service information) will be described with reference to FIGS. 86 to 94.

FIG. 86: is a figure which shows an example of operation|movement of the display apparatus in the situation in a store.

For example, the user of the receiver 8300 that has performed visible light communication with the signboard 8301 described above enters the store corresponding to the displayed advertisement information. At this time, the receiver 8300 detects that the user has entered (stored in) the store corresponding to the advertisement information displayed using visible light communication. For example, the receiver 8300 obtains store information indicating the location of the store associated with the device ID of the signboard 8301 from the server after performing visible light communication with the signboard 8301. Then, the receiver 8300 determines whether or not the current location of the receiver 8300 obtained by using the GPS or the 9-axis sensor has entered the location of the store indicated by the store information. Here, the receiver 8300 detects the above-mentioned entry into the store by determining that the current location has entered the location of the store.

When the receiver 8300 detects entry, the receiver 8300 notifies the display device 8300b of entry via, for example, a server. Alternatively, the receiver 8300 notifies the display device 8300b of entering the store by visible light communication or wireless communication. Upon receiving the notification, the display device 8300b acquires product service information indicating a menu of products or services provided in the store, and displays the menu described by the product service information. The display device 8300b may be a mobile terminal carried by the user of the receiver 8300 or a store clerk, or may be a device provided in the store.

FIG. 87 is a diagram showing an example of the next operation of the display device 8300b in the in-store situation.

The user selects a desired product from the menu displayed on the display device 8300b. That is, the user performs an operation of touching the part of the menu where the name of the desired product is displayed with his/her finger. Accordingly, the display device 8300b receives the operation result of product selection.

FIG. 88 is a diagram showing an example of the next operation of the display device 8300b in the in-store situation.

The display device 8300b that has received the product selection operation result displays an image representing the selected product and the price of the product. As a result, the display device 8300b prompts the user to confirm the selected product. The image representing the above-mentioned product, information indicating the price of the product, and the like are included in the above-mentioned product service information, for example.

FIG. 89 is a diagram illustrating an example of the next operation of the receiver 8300 in the in-store situation.

The user who is prompted to confirm performs an operation for ordering the product. When the operation is performed, the receiver 8300 notifies the POS (Pint of Sale) system of the store of the payment information necessary for the electronic payment via the display device 8300b or the server. Further, the receiver 8300 determines whether or not there is the above-described pre-acquired image acquired and stored by using visible light communication with the signboard 8301 of the store. When the receiver 8300 determines that there is the pre-acquired image, the receiver 8300 displays the pre-acquired image.

Although the display device 8300b is used in this situation, the receiver 8300 may instead perform the process by the display device 8300b without using the display device 8300b. In this case, when the receiver 8300 detects entry, the receiver 8300 acquires product service information indicating a menu of products or services provided in the store from the server and displays the menu described by the product service information. In addition, when the receiver 8300 receives an operation for ordering a product, the receiver 8300 notifies the ordered product and the payment information necessary for electronic payment to the POS system of the store via the server.

FIG. 90 is a diagram showing an example of the next operation of the receiver 8300 in the in-store situation.

The store clerk applies the barcode scanner 8302 of the POS system to the barcode of the pre-acquired image displayed on the receiver 8300. The barcode scanner 8302 reads the barcode of the pre-acquired image. As a result, the POS system makes electronic payment according to the coupon indicated by the barcode. Then, the barcode scanner 8302 of the POS system transmits the payment completion information indicating that the electronic payment is completed to the receiver 8300 by changing the brightness. That is, the barcode scanner 8302 also has a function as a transmitter for visible light communication. When the receiver 8300 acquires the payment completion information by visible light communication, the receiver 8300 displays the payment completion information. This payment completion information indicates, for example, a message "Thank you for your purchase" and the amount of payment. By performing such electronic payment, the POS system, the server, and the receiver 8300 provide the service indicated by the advertisement information in the store corresponding to the advertisement information (service information) displayed in front of the store. It can be determined that it has been used.

As described above, the products are ordered in the store by the operations of the receiver 8300 and the POS system as shown in FIGS. 86 to 90. Therefore, when entering the store, the user can place an order for the product from the menu of the store that is automatically displayed on the display device 8300b or the receiver 8300. In other words, the store clerk does not need to show the menu to the user and directly receive the product order from the user. As a result, the burden on the store clerk can be significantly reduced. Further, in the above example, the barcode scanner 8302 reads the barcode, but the barcode scanner 8302 may not be used. For example, the receiver 8300 may send the information shown in the barcode to the POS system via the server. Then, the receiver 8300 may obtain the payment completion information from the POS system via the server. As a result, the work by the store clerk can be further reduced, and the user can place an order for the product without going through the store clerk. Alternatively, the display device 8300b and the receiver 8300 may exchange order and billing data by visible light communication, or may exchange the data by wireless communication using a key exchanged by visible light communication.

In addition, the signboard 8301 may be displayed by one of a plurality of stores belonging to the chain store. In such a case, the advertising information acquired using visible light communication with the signboard 8301 can be used in all stores belonging to the chain store. However, even in the same chain store, a service provided to the user may be different between a store (advertising store) that displays the signboard 8301 and a store (non-advertising store) that does not display the signboard 8301. For example, when the user enters a non-advertising store, the user receives a service with a discount rate (for example, 20%) according to the coupon shown in the pre-acquired image. Receive services with a discount rate higher than the coupon discount rate (for example, 30%). That is, when the receiver 8300 detects entry into the advertising store, the receiver 8300 acquires additional service information indicating a further discount of 10% from the server, and the discount rate of 30% (20%+10%). Is displayed instead of the pre-acquired image shown in FIG. 89. The receiver 8300 detects whether the user has entered an advertising store or a non-advertising store based on the above-mentioned store information acquired from the server. The store information indicates the location of each of the plurality of stores belonging to the chain store and whether the store is an advertising store or a non-advertising store.

Further, when there are a plurality of non-advertising stores in the same chain store, different services may be provided to the user at each non-advertising store. For example, a service according to the distance from the position of the signboard 8301 or the current position of the receiver 8300 when performing visible light communication with the signboard 8301 to the non-advertising store is provided to the user entering the non-advertising store. To be done. Alternatively, a service corresponding to the difference (time difference) between the time when the receiver 8300 and the signboard 8301 perform visible light communication and the time when the user enters the non-advertising store is provided to the user entering the non-advertising store. To be done. That is, the receiver 8300 acquires additional service information indicating a further discount, which differs depending on the distance (position of the signboard 8301) and the time difference, from the server, and the discount rate (eg, 30) that reflects the further discount. %) is displayed instead of the pre-acquired image shown in FIG. 89. It should be noted that such a service is determined by the server or the POS system, or by the mutual cooperation thereof. Further, such a service may be applied to all stores belonging to a chain store without distinguishing between advertising stores and non-advertising stores.

Further, when the user enters the non-advertising store and places an order using the advertising information, the POS system of the non-public notice store returns a part of the amount of money obtained by the order to the POS system of the advertising store. You may.

Further, the server can determine whether or not the advertisement information is used each time the advertisement information is displayed, and analyze the advertisement effect of the signboard 8301 by accumulating the determination results. You can Further, the server further includes at least one of the position of the signboard 8301, the time when the advertisement information is displayed, the position of the store where the advertisement information is used, the time when the advertisement information is used, and the time when the user enters the store. By collecting the two, it is possible to improve the accuracy of analyzing the advertising effect of the signboard 8301, and it is possible to find the position of the signboard 8301 having the highest advertising effect.

Further, the receiver 8300 acquires additional service information indicating a further discount for the number of times the advertisement information has been used for ordering the product from the server, and a discount rate (a discount rate reflecting the further discount for the number of times of use) is acquired. An image showing, for example, 30%) may be displayed instead of the pre-acquired image shown in FIG. 89. For example, the server may cooperate with the POS system to provide a service that further increases the discount rate if the number of times of use is large.

In addition, when the server receives the advertisement information associated with each of the device IDs of all the signboards 8301 issued by the store by the receiver 8300 (when the acquisition of all the advertisement information is completed). The advantageous service may be provided to the user who has entered the store of the signboard 8301. The profitable service is, for example, a service with an extremely high discount rate, or a service that provides products other than ordered products for free. That is, when the receiver 8300 detects that the user has entered the store, the server determines whether or not the receiver 8300 has performed a process including visible light communication on all of the signboards associated with the store. judge. Then, if it is determined that the processing has been performed, the receiver 8300 acquires additional service information indicating a further discount from the server as the above-described advantageous service, and the discount rate ( An image showing, for example, 50%) is displayed instead of the pre-acquired image shown in FIG. 89.

Further, the receiver 8300 displays additional service information indicating a further discount rate, which differs depending on the time when the advertisement information is displayed by performing visible light communication with the signboard 8301 and the time when the user enters the store. An image acquired from the server and showing a discount rate (for example, 30%) in which a further discount is reflected may be displayed instead of the pre-acquired image shown in FIG. 89. For example, the receiver 8300 acquires additional service information indicating a higher discount rate as the difference is smaller from the server.

FIG. 91 is a diagram illustrating an example of the next operation of the receiver 8300 in the in-store situation.

After receiving the order and the electronic payment, the receiver 8300 receives a signal transmitted by a change in brightness from a transmitter configured as a lighting device in the store, and transmits the signal to the server, so that the user's seat position ( For example, an in-store guide map indicating a black circle is acquired. Further, the receiver 8300 specifies the position of the receiver 8300 using the received signal, as in any of Embodiments 1 to 3 above. Then, the receiver 8300 displays the specified position (for example, a star) of the receiver 8300 in the guide map. With this, the user can easily understand how to proceed in the store to get to his/her seat.

Further, the receiver 8300 performs visible light communication with a nearby transmitter configured as a lighting device in the store even when the user is moving, thereby identifying the position of the receiver 8300 as described above. I am going from time to time. Therefore, the receiver 8300 sequentially updates the displayed position of the receiver 8300 (for example, a star). As a result, the user can be properly guided to the seat.

FIG. 92 is a diagram illustrating an example of the next operation of the receiver 8300 in the in-store situation.

When the user arrives at the seat, the receiver 8300 identifies the position of the receiver 8300 by performing visible light communication with the transmitter 8303 configured as a lighting device, and determines that the position is at the user's seat position. To do. Then, the receiver 8300 notifies the terminal in the store via the server that the user has taken a seat together with the user name or nickname. This allows the store clerk to know which user is seated in which seat.

FIG. 93 is a diagram illustrating an example of next operation of the receiver 8300 in the in-store situation.

The transmitter 8303 transmits a signal including a customer ID and a message notifying that the ordered product is completed by changing the brightness. Note that, for example, when the receiver 8300 acquires, for example, the product service information indicating the menu of the product from the server, the receiver 8300 also acquires and holds the customer ID described above from the server. The receiver 8300 receives the above-mentioned signal by photographing the transmitter 8303 with visible light. Further, the receiver 8300 determines whether or not the customer ID included in the signal matches the customer ID held in advance. Here, if the receiver 8300 determines that they match, the receiver 8300 displays a message included in the signal (for example, “a product is completed”).

FIG. 94 is a diagram illustrating an example of the next operation of the receiver 8300 in the in-store situation.

The clerk who has delivered the ordered product to the user's seat directs the handy terminal 8302a to the receiver 8300 in order to prove that the ordered product has been delivered. The handy terminal 8302a is configured as a transmitter, and transmits a signal indicating that the ordered product has been delivered to the receiver 8300 by changing the brightness. The receiver 8300 receives the signal by capturing an image of the handy terminal 8302a, and displays a message (for example, "enjoy a meal") indicated by the signal.

<Situation: Looking for a store>
Next, an application example in a situation in which the user carrying the receiver 8300 is searching for a store of interest will be described with reference to FIGS. 95 to 97.

FIG. 95 is a diagram illustrating an example of operation of the receiver 8300 in a store search situation.

The user finds the signage 8304 on which the restaurant of interest is posted. At this time, when the user determines that the signage 8304 is transmitting a signal due to a change in luminance, the user operates the receiver 8300 to start the communication application of the receiver 8300, as in the example shown in FIG. 81. Let Note that, similarly to the example shown in FIG. 82, the receiver 8300 may automatically start the communication application.

FIG. 96 is a diagram illustrating an example of next operation of the receiver 8300 in the store search situation.

The receiver 8300 captures an image of the entire signage 8304 or a part of the signage 8304 in which the restaurant in which the user is interested is posted, and thereby the ID for identifying the signage 8304 or the restaurant is obtained. To receive.

FIG. 97 is a diagram illustrating an example of next operation of the receiver 8300 in the store search situation.

Upon receiving the above ID, the receiver 8300 transmits the ID to the server, acquires the advertisement information (service information) associated with the ID from the server, and displays the advertisement information (service information). At this time, the receiver 8300 may notify the server of the number of people (attached information) scheduled to enter the restaurant together with the ID. Thereby, the receiver 8300 can acquire the advertisement information according to the number of people. For example, the receiver 8300 can acquire advertisement information indicating whether or not there are vacant seats for the notified number of people at the restaurant.

<Situation: movie advertisement>
Next, an application example in a situation in which the user carrying the receiver 8300 is in front of a signage in which a movie advertisement of interest is placed will be described with reference to FIGS. 98 to 101.

FIG. 98 is a diagram illustrating an example of operation of the receiver 8300 in a movie advertisement situation.

The user finds a signage 8305 in which a movie advertisement of interest is posted and a signage 8306 configured as, for example, a liquid crystal display and displaying a moving image for movie advertisement. The signage 8305 includes, for example, a transparent film on which an image showing a movie advertisement is drawn, and a plurality of LEDs arranged on the back side of the film to illuminate the film. That is, the signage 8305 brightly displays the image drawn on the film as a still image by the light emission of the plurality of LEDs. Further, the signage 8305 is configured as a transmitter that transmits a signal by changing the brightness.

Here, when the user determines that the signage 8305 is transmitting a signal due to a change in luminance, the user operates the receiver 8300 to activate the communication application of the receiver 8300, as in the example shown in FIG. 81. Let Note that, similarly to the example shown in FIG. 82, the receiver 8300 may automatically start the communication application.

FIG. 99 is a diagram illustrating an example of next operation of the receiver 8300 in the movie advertisement situation.

The receiver 8300 acquires the ID of the signage 8305 by imaging the signage 8305. Then, the receiver 8300 transmits the ID to the server, downloads the movie image moving image data associated with the ID as service information from the server, and reproduces it.

FIG. 100 is a diagram illustrating an example of next operation of the receiver 8300 in the movie advertisement situation.

The moving image displayed by reproducing the moving image data downloaded as described above is the same as the moving image displayed by the signage 8306, for example. Therefore, when the user wants to see a moving image for movie advertisement, the user can see the moving image at any place without stopping before the signage 8306.

FIG. 101 is a diagram illustrating an example of next operation of the receiver 8300 in the movie advertisement situation.

The receiver 8300 may download, as service information, not only the moving image data, but also the showing information indicating the show time of the movie together with the moving image data. Thereby, the receiver 8300 can display the content of the screening information and notify the user, and can also share the screening information with other terminals (such as other smartphones).

<Situation: Museum>
Next, an application example in a situation where a user carrying the receiver 8300 enters a museum and appreciates each exhibit in the museum will be described with reference to FIGS. 102 to 107.

FIG. 102 is a diagram illustrating an example of operation of the receiver 8300 in a museum situation.

For example, when the user tries to enter an art museum, the user finds the information bulletin board 8307 hung at the entrance of the art museum. At this time, when the user determines that the guidance bulletin board 8307 is transmitting a signal due to a change in luminance, the user operates the receiver 8300 to operate the communication application of the receiver 8300 as in the example shown in FIG. 81. To start. Note that, similarly to the example shown in FIG. 82, the receiver 8300 may automatically start the communication application.

FIG. 103 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

The receiver 8300 acquires the ID of the guidance bulletin board 8307 by imaging the guidance bulletin board 8307. Then, the receiver 8300 transmits the ID to the server, and downloads the guidance application program of the museum (hereinafter referred to as the museum application) from the server and activates it as service information associated with the ID.

FIG. 104 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

When the museum application is activated, the receiver 8300 displays a guide map inside the museum according to the museum application. Further, the receiver 8300 identifies the position of the receiver 8300 in the museum, as in any of Embodiments 1 to 3 above. Then, the receiver 8300 displays the specified position (for example, a star) of the receiver 8300 in the guide map.

In order to specify the position as described above, the receiver 8300 acquires form information indicating the size and shape of the information board 8307 from the server, for example, when downloading the museum application. Then, the receiver 8300 performs triangulation based on the size and shape of the guidance bulletin board 8307 indicated by the form information and the size and shape of the guidance bulletin board 8307 displayed in the image obtained by the above-described imaging. The relative position of the receiver 8300 with respect to the information bulletin board 8307 is specified according to the above method.

FIG. 105 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

When the user enters the museum, the receiver 8300 that activates the museum app as described above performs visible light communication with a nearby transmitter configured as a lighting device in the museum, thereby allowing the receiver 8300 to operate. The position is specified at any time. For example, the receiver 8300 acquires the ID of the transmitter 8308 from the transmitter 8308 by capturing an image of the transmitter 8308 configured as a lighting device. Then, the receiver 8300 acquires, from the server, position information indicating the position of the transmitter 8308 and form information indicating the size and shape of the transmitter 8308 associated with the ID. Then, the receiver 8300 performs triangulation based on the size and shape of the transmitter 8308 indicated by the morphology information and the size and shape of the transmitter 8308 displayed in the image obtained by the above-described imaging. The relative position of the receiver 8300 with respect to the transmitter 8308 is estimated according to the above method. In addition, the receiver 8300 is based on the position of the transmitter 8308 indicated by the position information acquired from the server and the relative position of the receiver 8300 estimated as described above. Specify the position.

Then, the receiver 8300 moves the displayed star mark to the latest specified position each time the position of the receiver 8300 is specified. Thus, the user in the museum can easily know where he is in the museum by looking at the guide map and the star mark displayed on the receiver 8300.

FIG. 106 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

When the user who enters the museum finds an exhibit 8309 of interest, the user holds the receiver 8300 over the exhibit 8309 so that the receiver 8300 can image the exhibit 8309. Here, the exhibit 8309 is illuminated by the light from the lighting device 8310. The lighting device 8310 is used exclusively for the exhibit 8309, and is configured as a transmitter that transmits a signal according to a change in luminance. Therefore, the exhibit 8309 is illuminated by the light whose brightness changes, and indirectly transmits the signal from the lighting device 8310.

When the receiver 8300 detects an operation of holding the receiver 8300 over the exhibit 8309, for example, based on the output from the built-in 9-axis sensor, the receiver 8300 captures an image of the exhibit 8309 to detect a signal from the lighting device 8310. To receive. This signal indicates, for example, the ID of the exhibit 8309. Then, the receiver 8300 acquires, from the server, the introduction information (service information) of the exhibit 8309 associated with the ID. This introduction information shows a diagram for introducing the exhibit 8309, and the text for the introduction is shown in each language such as Japanese, English, and French.

Upon receiving the introduction information from the server, the receiver 8300 displays the diagram and the sentence indicated by the introduction information. Here, when displaying a sentence, the receiver 8300 extracts a sentence in a language preset by the user from the sentences in each country's language and displays only the sentence in that language. Further, the receiver 8300 may change the language according to the selection operation by the user.

FIG. 107 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

When the display of the diagram and the text of the introduction information is finished by the user's operation, the receiver 8300 performs visible light communication again with a nearby transmitter (for example, the lighting device 8311) configured as the lighting device. , The position of the receiver 8300 is specified. Then, when a new position of the receiver 8300 is specified, the receiver 8300 moves the displayed star mark to the specified new position. As a result, the user who views the exhibit 8309 can easily move to the next desired exhibit by viewing the guide map and the star mark displayed on the receiver 8300.

<Situation: bus stop>
Next, an application example in a situation in which a user carrying the receiver 8300 is at a bus stop will be described with reference to FIGS. 108 to 109.

FIG. 108 is a diagram illustrating an example of operation of the receiver 8300 in the bus stop situation.

A user goes to a bus stop, for example, to get on a bus. Therefore, when the user determines that the sign tower 8312 at the bus stop is transmitting a signal due to a change in brightness, the user operates the receiver 8300 to operate the receiver 8300 as in the example shown in FIG. 81. Start a communication application. Note that, similarly to the example shown in FIG. 82, the receiver 8300 may automatically start the communication application.

FIG. 109 is a diagram illustrating an example of next operation of the receiver 8300 in the bus stop situation.

The receiver 8300 acquires the ID of the bus stop where the sign tower 8312 is located by capturing an image of the sign tower 8312. Then, the receiver 8300 transmits the ID to the server, and acquires the operation status information associated with the ID from the server. The operation status information is information indicating traffic status and is service information indicating services provided to the user.

Here, the server manages the operating status of each bus by collecting information from each bus operating in the area including the bus stop. Therefore, when the server acquires the ID of the bus stop from the receiver 8300, the server estimates the time until the bus arrives at the bus stop of the ID based on the managed operation status, and the estimated time. Is transmitted to the receiver 8300.

The receiver 8300 that has acquired the operation status information displays the time indicated by the operation status information, for example, “10 minutes after arrival”. This allows the user to easily understand the operating status of the bus.

(Supplement)
When the scanning direction on the image capturing side is the vertical direction (vertical direction) of the mobile terminal and the image is captured with a short exposure time, as shown in (a) of FIG. In addition, it is possible to capture an image of a bright line that is a white/black pattern in the same direction as the scanning direction. In (a) of FIG. 110, since the long side direction of the vertically long LED lighting device is imaged so as to be perpendicular to the scanning direction on the imaging side (the horizontal direction of the mobile terminal), the same direction as the scanning direction. In addition, a large number of white and black bright lines can be captured. That is, the amount of information that can be transmitted and received can be increased. On the other hand, as shown in (b) of FIG. 110, when the vertically long LED lighting device is imaged so as to be parallel to the scanning direction on the imaging side (vertical direction of the mobile terminal), a white/black pattern that can be imaged There are few bright lines. That is, the amount of information that can be transmitted becomes small.

In this way, when a large number of white/black pattern bright lines can be imaged depending on the orientation of the LED lighting device with respect to the scanning direction on the imaging side (the long side direction of the vertically long LED lighting device is perpendicular to the scanning direction on the imaging side). Image) and a case where only a few bright lines of white/black patterns can be imaged (when the long side direction of the vertically long LED lighting device is parallel to the scanning direction on the imaging side). Occurs.

In the present embodiment, a method of controlling an illumination device that can image a large number of bright lines even if only a small number of white and black bright lines can be imaged will be described.

FIG. 111 shows an example of a lighting device in which a plurality of LEDs are arranged in the vertical direction and a drive signal thereof. FIG. 111A shows an illuminating device having a plurality of LEDs arranged in the vertical direction. It is assumed that each LED element corresponds to the minimum unit of horizontal stripes that encodes the visible light communication signal and that it corresponds to the encoded ON/OFF signal. In this way, by generating a white/black pattern, turning on/off each LED element, and illuminating, it is assumed that the scanning direction on the imaging side and the long side direction of the vertically long LED lighting device are parallel. , It becomes possible to photograph a white/black pattern for each LED element.

111(c) and (d) show an example in which a white/black pattern is generated, and each LED element is turned on/off to illuminate. When the illumination device illuminates as a white/black pattern, the light may have unevenness even for a short time. Therefore, an example in which patterns of opposite phases are generated and alternately illuminated is shown in (c) and (d) of FIG. 111. In FIG. 111(c), the element that has been turned on is off in FIG. 111(d), and in FIG. 111(c) the element that has been turned off is shown in FIG. It is turned on in d). In this way, by illuminating the white and black patterns alternately with the positive-phase pattern and the reverse-phase pattern, the unevenness of the light does not occur, and the scanning direction on the imaging side and the illumination A large amount of information can be transmitted and received in visible light communication without being affected by the relationship with the orientation of the device. Further, not only the case where two types of patterns of the positive phase pattern and the pattern of the opposite phase are alternately generated and illuminated, but three or more types of patterns may be generated and illuminated. FIG. 112 shows an example of sequentially illuminating four types of patterns.

In general, a configuration is possible in which the entire LED illumination blinks ((b) in FIG. 111), a white/black pattern is generated for a predetermined time, and illumination is performed in LED element units. For example, a configuration is conceivable in which the entire LED illumination blinks during the transmission/reception of a predetermined data unit, and then the white/black pattern is illuminated in LED element units in a short time. Here, the predetermined data unit refers to, for example, a data unit from the first header to the next second header. At this time, when the image is picked up in the direction of (a) of FIG. 110, the signal is received from the bright line obtained by imaging the blinking of the entire LED illumination, and when it is received in the direction of (b) of FIG. Receive a signal from the emission pattern.

Note that the present embodiment is not limited to the LED lighting device, and any element may be used as long as it can control ON/OFF in small element units, like the LED element. The device is not limited to the lighting device, and may be a device such as a television, a projector, or a signage.

Further, although an example of illuminating with a white/black pattern has been described in the present embodiment, a color may be used instead of the white/black pattern. For example, among RGB, RG may be constantly turned on and only B may be used to blink. It is more difficult for humans to recognize B than R and G, and flicker can be suppressed. As another example, instead of the white/black pattern, use complementary colors in the additive color mixture (red and cyan patterns, green and magenta patterns, yellow and blue patterns, etc.) to display ON/OFF. Good. Flicker can be suppressed by using a complementary color in the additive color mixture.

Further, although the present embodiment has been described by using the example in which the LED elements are arranged in one dimension, the LED elements are not arranged in one dimension but are arranged in two dimensions and displayed like a two-dimensional barcode. You may go.

(Summary of this Embodiment)
A service providing method according to the present embodiment is a service providing method for providing a service to a user of the terminal device by using a terminal device including an image sensor having a plurality of exposure lines, and each exposure line of the image sensor is provided. Exposure is sequentially started at different times, and the exposure time of each exposure line is 1/480 seconds or less so that it partially overlaps with an adjacent exposure line. An image acquisition step of acquiring image data by capturing an image of a subject, and a visible light communication for acquiring identification information of the subject by demodulating a bright line pattern corresponding to each exposure line appearing in the image data. And a service presenting step of presenting to the user service information associated with identification information of the subject.

Thereby, the service information related to the subject is presented to the user of the terminal device by utilizing that the subject and the terminal device communicate with each other as a transmitter and a receiver, respectively. It can be provided to users in various forms. For example, in the service presenting step, information indicating an advertisement, vacancy status or reservation status of a store related to the subject, information indicating a discount rate of the price of goods or services, a moving image for movie advertisement, and a show time. At least one of the information indicating the above, the information for guiding inside the building, the information for introducing the exhibit, and the information indicating the traffic condition is presented as the service information.

The service providing method may further include an identification information transmitting step in which the terminal device transmits identification information of the subject to a server, and the terminal device transmits the service information associated with the identification information of the subject to the server. In the service presenting step, the terminal device may present the acquired service information to the user.

This allows the server to manage the service information in association with the identification information of the subject, so that maintenance such as updating the service information can be facilitated.

Further, in the identification information transmitting step, the additional information is transmitted to the server together with the identification information of the subject, and in the service obtaining step, the service information associated with the identification information of the subject and the additional information is obtained. You may.

This makes it possible to provide a more appropriate service for the user according to the attached information. For example, like the operation described with reference to FIGS. 84 and 97, in the identification information transmitting step, the personal information of the user, the identification information of the user, the number of persons information indicating the number of persons in the group including the user, or Position information indicating the position of the terminal device is transmitted as the attached information.

Further, the service providing method further includes a position transmitting step in which the terminal device transmits position information indicating a position of the terminal device to the server, and at least one position within a predetermined range including the position indicated by the position information. The terminal device further includes a pre-acquisition step of acquiring the identification information of the device and at least one service information associated with each of the identification information from the server, and holding the pre-acquisition step. The terminal device selects service information associated with the identification information of the subject from the at least one service information stored in the pre-article acquisition step, and presents the service information to the user. Good.

As a result, when the terminal device acquires the identification information of the subject, as in the operation described with reference to FIG. 82, for example, among the at least one service information stored in advance, the terminal device does not communicate with the server or the like thereafter. From, it is possible to obtain and present the service information associated with the identification information of the subject. Therefore, the provision of services can be speeded up.

Further, the service providing method further includes an entrance determination step of determining whether or not the user has entered the store corresponding to the service information presented in the service presentation step by specifying the position of the user. And in the entrance determination step, when it is determined that the user has entered the store, the product service in which the terminal device acquires the product service information related to the product or service of the store from the server and presents it to the user. The presenting step may be included.

With this, when the user enters the store, for example, like the operation described with reference to FIGS. 86 to 90, the menu of the store and the like can be automatically presented to the user as product service information. Therefore, the store clerk does not need to present a menu or the like to the user, and the user can easily place an order for the store.

Further, the service providing method further includes an entrance determination step of determining whether or not the user has entered the store corresponding to the service information presented in the service presentation step by specifying the position of the user. In the entrance determination step, when it is determined that the user has entered the store, the location of the subject and the time when the service information is presented vary depending on at least one of the stores. The additional service information may be presented by the terminal device presenting the additional service information to the user.

As a result, for example, as in the process described with reference to FIGS. 86 to 90, the closer the subject is to the store where the user entered, or the time when the user entered the store and the time when the service information was presented (or The closer it is to the time when the subject was photographed, the more useful service information for the user can be presented to the user as additional service information. Specifically, each of the plurality of stores belonging to the chain store is a store corresponding to the service information presented in the service presenting step, and a signboard that is a subject by one store (advertising store) of the stores May have been issued. In such a case, the advertising store of the plurality of stores belonging to the chain store is typically located closest to the subject (signboard). Therefore, the closer the subject is to the store where the user entered, or the closer the user entered the store and the time when the service information was presented, the higher the possibility that the store where the user entered was an advertising store. .. Therefore, when there is a high possibility that the user has entered the advertising store, it is possible to present more useful service information to the user as additional service information.

Further, the service providing method further includes an entrance determination step of determining whether or not the user has entered the store corresponding to the service information presented in the service presentation step by specifying the position of the user. And when it is determined in the store entrance determination step that the user has entered the store, an additional service of the store that differs depending on the number of times the user uses the service indicated by the service information in the store And a step of presenting information to the user by the terminal device.

As a result, as in the operation described with reference to FIGS. 86 to 90, the more the service is used, the more useful the service information to the user can be presented to the user as additional service information. For example, if the number of times the service information indicating a 20% discount on the product price is used exceeds a threshold value, additional service information indicating a further 10% discount can be presented to the user.

Further, the service providing method further includes an entrance determination step of determining whether or not the user has entered the store corresponding to the service information presented in the service presentation step by specifying the position of the user. And in the store entrance determination step, when it is determined that the user has entered the store, the image acquisition step is performed also for each of all other subjects associated with the store other than the subject. A complete determination step of determining whether or not processing including the visible light communication step and the service presentation step has been performed, and when it is determined that the processing has been performed in the complete determination step, the additional And a step of presenting the service information to the user by the terminal device.

Thereby, for example, as in the operation described with reference to FIGS. 86 to 90, for example, a store puts out some subjects as billboards, and an image acquisition step, a visible light communication step, and a service presentation for all of these billboards. When the step is performed, the service information most useful to the user can be presented to the user as additional service information.

Further, the service providing method further includes an entrance determination step of determining whether or not the user has entered the store corresponding to the service information presented in the service presentation step by specifying the position of the user. And in the store entrance determination step, when it is determined that the user has entered the store, the store that differs according to the difference between the time when the service information is presented and the time when the user enters the store The additional service information may be provided by the terminal device to the user.

Thereby, as in the operation described with reference to FIGS. 86 to 90, for example, the difference between the time when the service information is presented (or the time when the subject is photographed) and the time when the user enters the store is calculated. The smaller the value, the more useful service information for the user can be presented to the user as additional service information. That is, it is possible to additionally provide a more useful service to a user who takes a short time from receiving the presentation of service information by capturing an image of a subject to entering a store.

Further, the service providing method further includes a use determining step of determining whether or not the user has used the service indicated by the service information in a store corresponding to the service information presented in the service presenting step, Each time the service information is presented, the analysis step of accumulating the results determined in the usage determining step and analyzing the advertising effect of the subject based on the accumulated content may be included.

For example, when a service such as a 20% discount on the product price is indicated in the service information as in the operation described with reference to FIGS. 86 to 90, it is determined whether or not the service is used by electronic payment, for example. To be done. That is, every time a service is provided to the user when the subject is imaged, it is determined whether or not the service is used. As a result, for example, when it is often determined that the image has been used, it can be analyzed that the advertising effect of the subject is high. That is, the advertising effect of the subject can be appropriately analyzed based on the usage result.

In the analysis step, the position of the subject, the time when the service information is presented, the position of the store, and the time when the user enters the store, together with the result determined in the use determination step. , At least one of them may be accumulated, and the advertising effect of the subject may be analyzed based on the accumulated contents.

Thereby, the advertising effect of the subject can be analyzed in more detail. For example, when the position of the subject is changed, it is possible to compare the advertising effect before and after the position is changed, and as a result, the subject can be placed at a position where the advertising effect is high.

Further, the service providing method further includes a use determining step of determining whether or not the user has used the service indicated by the service information in a store corresponding to the service information presented in the service presenting step, When it is determined that the service has been used in the use determination step, a store determination step of determining whether or not a use store that is a store where the service is used is a specific store associated with the subject, When it is determined that the used store is not the specific store in the store determination step, at least a part of the amount of money settled using the service at the used store is returned to the specific store by using electronic commerce. And a reduction step of

Thereby, even if the service is not used in a specific store (for example, an advertising store that displays a signboard that is a subject), as in the operation described with reference to FIGS. 86 to 90, the specific store is For example, a profit can be obtained as a price for installing a signboard that is a subject.

Further, in the service presenting step, when the subject illuminated by the light whose brightness changes in the image acquiring step is photographed, the terminal device presents the service information for introducing the subject to the user. However, in the case where the lighting device whose brightness changes in the image acquisition step is photographed as the subject, the terminal device presents the service information for guiding the inside of the building in which the subject is arranged to the user. Good.

Thus, for example, as in the operations described with reference to FIGS. 105 and 106, it is possible to appropriately provide the user with a guidance service in a museum such as an art museum and an introduction service of an exhibit as a subject.

The information communication method according to the present embodiment is an information communication method for acquiring information from a subject having a plurality of light emitting elements, and an image obtained by photographing the subject with an image sensor is included in the image sensor. An exposure time setting step of setting the exposure time of the image sensor so that a bright line corresponding to an exposure line is generated in accordance with a change in the brightness of the subject, and the plurality of the plurality of pixels according to the brightness change pattern for indicating the first information. An image acquisition step of acquiring a bright line image that is an image including the bright line by the image sensor shooting the subject in which all of the light emitting elements change in luminance in the same manner, by the set exposure time; The first information acquisition step of acquiring the first information by demodulating the data specified by the pattern of the bright lines included in the generated bright line image and the plurality of light emitting elements are different from each other. The subject that emits light at one of the two luminance values is photographed, and the data specified by the arrangement of brightness and darkness along the direction parallel to the exposure line shown in the image obtained by photographing is demodulated. And a second information acquisition step of acquiring the second information.

Alternatively, the information communication method according to the present embodiment is an information communication method of transmitting a signal by a luminance change, and a determining step of determining a pattern of the luminance change by modulating the first signal to be transmitted, A first transmitting step of transmitting the first signal by changing the luminance of all of the plurality of light emitting elements included in the light emitting body in the same manner according to the determined pattern of the luminance change, and the plurality of light emitting elements. Each of the elements emits light with one of two brightness values different from each other, so that a bright and dark array of brightness appears in the space in which the light emitters are arranged. And a second transmitting step of transmitting the signal of.

As a result, even if the illuminating device, which is the subject or the light emitting body, is a long and thin device including a plurality of LEDs arranged in a line, as in the operation described with reference to FIGS. , Irrespective of the shooting direction, it is possible to appropriately acquire information or a signal from the lighting device. In other words, when the exposure line (the operation direction on the imaging side) of the image sensor provided in the receiver and the arrangement direction of the plurality of LEDs are not parallel, the receiver receives information from the change in the brightness of the entire lighting device. Alternatively, the signal can be properly acquired. Further, even when the receiver is parallel to the exposure line and the arrangement direction described above, the receiver can appropriately obtain the information or the signal from the arrangement of brightness and darkness along the direction parallel to the exposure line. it can. In other words, it is possible to suppress the dependence of the imaging direction on the reception of information.

(Embodiment 5)
In the present embodiment, an application example will be described in which the receiver such as the smartphone according to the first to fourth embodiments and the transmitter that transmits information as a blinking pattern such as an LED or an organic EL are used.

FIG. 113 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

Each of the transmitter 8321, the transmitter 8322, and the transmitter 8323 has a function similar to that of the transmitter of any of Embodiments 1 to 4 above, and is configured as a lighting device that transmits a signal (visible light communication) by luminance change. Has been done. Further, these transmitters 8321 to 8323 each transmit a signal by changing the luminance at frequencies different from each other. For example, the transmitter 8321 transmits the ID “1000” of the transmitter 8321 by changing the luminance at the frequency a (for example, 9200 Hz), and the transmitter 8322 changes the luminance at the frequency b (for example, 9600 Hz). The transmitter 8322 transmits the ID “2000” of the transmitter 8322, and the transmitter 8323 transmits the ID “3000” of the transmitter 8322 by changing the brightness at the frequency c (eg, 10000 Hz).

The receiver images the transmitters 8321 to 8323 (visible light imaging) in the same manner as in the first to fourth embodiments so that the transmitters 8321 to 8323 are all included in the angle of view. The bright line pattern corresponding to each transmitter appears in the image obtained by the imaging. From the bright line pattern, the frequency of the brightness change of the transmitter corresponding to the bright line pattern can be specified.

Here, if the frequencies of the transmitters 8321 to 8323 are the same, the frequencies specified from the bright line patterns corresponding to the transmitters are also the same. Further, when the bright line patterns are adjacent to each other, it is difficult to distinguish the bright line patterns because the frequencies specified from the bright line patterns are the same.

Therefore, as described above, the transmitters 8321 to 8323 change in luminance at frequencies different from each other, so that the receiver can easily distinguish the bright line patterns, and the data specified by each bright line pattern can be obtained. By demodulating, the IDs of the transmitters 8321 to 8323 can be appropriately acquired. That is, the receiver can properly distinguish the signals from the transmitters 8321 to 8323.

The frequencies of the transmitters 8321 to 8323 may be set by the remote controller or may be set randomly. Further, each of the transmitters 8321 to 8323 may communicate with the adjacent transmitter and automatically set the frequency of its own transmitter so as to be different from the frequency of the adjacent communication device.

FIG. 114 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

In the above example, each of the transmitters changes in brightness at a different frequency, but when there are five or more transmitters, it is not necessary for each of them to change in brightness at a different frequency. That is, each of the five or more transmitters may change the luminance at any one of the four frequencies.

For example, as shown in FIG. 114, even if the bright line patterns (rectangular regions in FIG. 114) corresponding to each of five or more transmitters are adjacent to each other, the number of types of frequency is the same as the number of transmitters. If there are four types (frequency a, b, c, d) that are not necessary, the frequencies of the bright line patterns adjacent to each other can be surely made different. This is justified by the four-color problem or the four-color theorem.

That is, in the present embodiment, each of the plurality of transmitters changes in luminance at any one of at least four types of frequencies, and two or more light emitters of the plurality of transmitters have the same brightness. The brightness changes with the frequency. In addition, when a plurality of transmitters are projected on the light receiving surface of the image sensor of the receiver, the brightness changes between all the transmitters (bright line patterns that are images of the transmitters) adjacent to each other on the light receiving surface. Each of the plurality of transmitters varies in intensity so that the frequency of the transmitter varies.

FIG. 115 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

The transmitter changes the brightness by outputting high-luminance light (H) or low-luminance light (L) for each predetermined time unit (slot), and transmits a signal by this brightness change. Here, the transmitter transmits a signal for each block including a header and a body. The header is expressed as (L, H, L, H, L, H, H) using 7 slots as shown in FIG. 79A, for example. The body is composed of a plurality of symbols (00, 01, 10 or 11), and each symbol is represented by using 4 slots (4 value PPM). A block is represented using a predetermined number of slots (19 in the example of FIG. 115). The ID is obtained by combining the bodies included in each of the four blocks, for example. The block may be expressed using 33 slots.

The bright line pattern obtained by the image pickup of the receiver includes a pattern (header pattern) corresponding to the header and a pattern (data pattern) corresponding to the body. The data pattern does not include the same pattern as the header pattern. Therefore, the receiver can easily find the header pattern from the bright line pattern, and can measure the number of pixels (the number of exposure lines corresponding to the block) between the header pattern and the next header pattern. In the receiver, the number of slots in one block (19 in the example of FIG. 115) is set to a fixed number regardless of the frequency, and therefore the frequency of the transmitter (time of one slot is set according to the measured number of pixels). The reciprocal of the width) can be specified. That is, the receiver specifies a lower frequency as the number of pixels is larger and a higher frequency as the number of pixels is smaller.

In this way, the receiver can acquire the ID of the transmitter and identify the frequency of the transmitter by capturing the image of the transmitter. Here, the receiver can determine whether or not the acquired ID is appropriate, that is, perform the error detection of the ID by using the frequency thus specified. Specifically, the receiver calculates a hash value for the ID and compares the hash value with the specified frequency. If the hash value and the frequency match, the receiver determines that the acquired ID is appropriate, and if they do not match, the acquired ID is incorrect (error). Judge that there is. For example, the receiver treats the remainder obtained by dividing the ID by a predetermined divisor as a hash value. Conversely, the transmitter transmits the ID of the transmission target by changing the luminance at the frequency (the reciprocal of the time width of one slot) having the same value as the hash value for the ID of the transmission target.

FIG. 116 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5.

Instead of using the frequency having the same value as the hash value as described above, the transmitter may use an arbitrary frequency and change the brightness at the arbitrary frequency. In this case, the transmitter sends a signal that indicates a value different from the ID of the transmitter. For example, if the transmitter ID is "100" and the transmitter uses 2 kHz as an arbitrary frequency, the transmitter transmits a signal "1002" that is a combination of the ID and the frequency. Similarly, when the ID of the other transmitter is “110” and the other transmitter uses 1 kHz as an arbitrary frequency, the other transmitter outputs a signal “1101” that is a combination of the ID and the frequency. To send.

In this case, the receiver uses the lower first digit of the signal transmitted and acquired from the transmitter for error detection, and extracts the remaining digits as the transmitter ID. Then, the receiver compares the frequency specified from the brightness pattern with the lower first digit value included in the acquired signal. The receiver determines that the extracted ID is appropriate when the numerical value in the lower first digit and the frequency match, and the extracted ID is inappropriate when it does not match. It is determined to be (error).

As a result, it is possible to increase the degree of freedom in setting the frequency of the luminance change in the transmitter while enabling the receiver to detect an error.

FIG. 117 is a diagram illustrating an example of operation of a receiver in Embodiment 5.

As shown in FIG. 117, in an image obtained by imaging (visible light imaging) by the receiver, the bright line pattern 8327a and the bright line pattern 8327b may partially overlap each other. In such a case, the receiver does not demodulate data from the portion 8327c where the bright line pattern 8327a and the bright line pattern 8327b overlap, but does not demodulate data from the portions other than the respective portions 8327c of the bright line pattern 8327a and the bright line pattern 8327b. To do. Accordingly, the receiver can acquire appropriate IDs from the bright line pattern 8327a and the bright line pattern 8327b.

FIG. 118 is a diagram illustrating an example of operation of a receiver in Embodiment 5.

As shown in (a) of FIG. 118, the transmitter switches the frequency of the luminance change for transmitting the block, for example, for each block. This allows the receiver to more easily discriminate between blocks and block delimiters.

Further, as shown in (b) of FIG. 118, the transmitter makes the frequency of the luminance change for transmitting the header of the block different from the frequency of the luminance change for transmitting the body of the block. This can prevent the same pattern as the header from appearing in the body. As a result, the receiver can more appropriately determine the distinction between the header and the body.

119 is a diagram illustrating an example of operation of a system including a transmitter, a receiver, and a server in Embodiment 5.

The system in this embodiment includes a transmitter 8331, a receiver 8332, and a server 8333. The transmitter 8331 has the same function as that of the transmitter according to any one of the above-described first to fourth embodiments, and is configured as an illumination device that transmits the ID of the transmitter 8331 (visible light communication) by a change in luminance. The receiver 8332 has a function as the receiver according to any of the above Embodiments 1 to 4, and acquires the ID of the transmitter 8331 from the transmitter 8331 by imaging the transmitter 8331 (visible light imaging). To do. The server 8333 communicates with the transmitter 8331 and the receiver 8332 via a network such as the Internet.

In addition, in this embodiment, the ID of the transmitter 8331 is not changed and is fixed. On the other hand, the frequency used for the luminance change (visible light communication) of the transmitter 8331 can be arbitrarily changed by setting.

In such a system, the transmitter 8331 first registers the frequency used for luminance change (visible light communication) in the server 8333. Specifically, the transmitter 8331 transmits its own ID, registered frequency information indicating the frequency, and related information related to the transmitter 8331 to the server 8333. When the server 8333 receives the ID of the transmitter 8331, the registered frequency information, and the related information, the server 8333 records them in association with each other. That is, the ID of the transmitter 8331, the frequency used for the luminance change of the transmitter 8331, and the related information are recorded in association with each other. As a result, the frequency used to change the brightness of the transmitter 8331 is registered.

Next, the transmitter 8331 transmits the ID of the transmitter 8331 according to the brightness change of the registered frequency. The receiver 8332 acquires the ID of the transmitter 8331 by imaging the transmitter 8331, and specifies the frequency of the luminance change of the transmitter 8331 as described above.

Next, the receiver 8332 transmits the acquired ID and specific frequency information indicating the specified frequency to the server 8333. When the server 8333 receives the ID and the specific frequency information transmitted by the receiver 8332, the server 8333 searches the frequency recorded in association with the ID (the frequency indicated by the registered frequency information), and detects the recorded frequency. , It is determined whether the frequency indicated by the specific frequency information matches. Here, when it is determined that they match, the server 8333 transmits relevant information (data) recorded in association with the ID and the frequency to the receiver 8332.

As a result, if the frequency specified by the receiver 8332 does not match the frequency registered in the server 8333, the server 8333 does not transmit the related information to the receiver 832. Therefore, if the receiver 8332 acquires the ID from the transmitter 8331 only once, the receiver 8332 is in a state of being able to receive the related information from the server 8333 at any time. Can be prevented by changing at any time. That is, the transmitter 8331 can prohibit acquisition of related information by the receiver 8332 that has acquired the ID before the change by changing the frequency registered in the server 8333 (that is, the frequency used for the brightness change). .. In other words, by changing the frequency, it is possible to set an expiration date for the acquisition of related information. For example, when the user of the receiver 8332 stays at the hotel in which the transmitter 8331 is installed, the administrator of the hotel changes the frequency after the stay. Accordingly, the receiver 8332 can acquire the related information only on the day when the user stays, and can prohibit the receiver 8332 from acquiring the related information after the stay.

Note that the server 8333 may register one ID in association with a plurality of frequencies. For example, every time the server 8333 acquires registered frequency information from the receiver 8332, the server 8333 always registers the frequency indicated by the latest four registered frequency information in association with the ID. As a result, even the receiver 8332 that has acquired the ID in the past can acquire the related information from the server 8333 until the frequency is changed three times. Further, the server 8333 may manage, for each registered frequency, the time or time zone when the frequency was set in the transmitter 8331. In this case, when the server 8333 receives the ID and the specific frequency information from the receiver 8332, the server 8333 confirms the time zone managed for the frequency indicated by the specific frequency information and the like, thereby The time zone in which the 8332 has acquired the ID can be specified.

FIG. 120 is a block diagram showing the configuration of the transmitter according to the fifth embodiment.

The transmitter 8334 has the same function as the transmitter according to any one of the above-described first to fourth embodiments, and also has a frequency storage unit 8335, an ID storage unit 8336, a check value storage unit 8337, a check value comparison unit 8338, and a check value. The calculation unit 8339, the frequency calculation unit 8340, the frequency comparison unit 8341, the transmission unit 8342, and the error notification unit 8343 are provided.

The frequency storage unit 8335 stores a frequency used for luminance change (visible light communication). The ID storage unit 8336 stores the ID of the transmitter 8334. The check value storage unit 8337 stores a check value for determining whether the ID stored in the ID storage unit 8336 is appropriate.

The check value calculation unit 8339 reads the ID stored in the ID storage unit 8336 and applies a predetermined function to the ID to calculate a check value (calculated check value) for the ID. The check value comparison unit 8338 reads the check value stored in the check value storage unit 8337 and compares the check value with the calculated check value calculated by the check value calculation unit 8339. When the check value comparing unit 8338 determines that the calculated check value is different from the check value, it notifies the error notifying unit 8343 of an error. For example, the check value storage unit 8337 stores a value “0” indicating that the ID stored in the ID storage unit 8336 is an even number as a check value. The check value calculation unit 8339 reads the ID stored in the ID storage unit 8336 and divides it by the value “2” to calculate the remainder as the calculated check value. Then, the check value comparison unit 8338 compares the check value “0” with the calculated check value that is the remainder of the above division.

The frequency calculation unit 8340 reads the ID stored in the ID storage unit 8336 via the check value calculation unit 8339, and calculates the frequency (calculated frequency) from the ID. For example, the frequency calculation unit 8340 calculates the remainder as a frequency by dividing the ID by a predetermined value. The frequency comparison unit 8341 compares the frequency (stored frequency) stored in the frequency storage unit 8335 with the calculated frequency. When the frequency comparison unit 8341 determines that the calculated frequency is different from the stored frequency, the frequency comparison unit 8341 notifies the error notification unit 8343 of an error.

The transmission unit 8342 transmits the ID stored in the ID storage unit 8336 by changing the brightness at the calculated frequency calculated by the frequency calculation unit 8340.

When at least one of the check value comparing unit 8338 and the frequency comparing unit 8341 notifies the error, the error notifying unit 8343 notifies the error by a buzzer sound, blinking or lighting. Specifically, the error notification unit 8343 includes a lamp for reporting an error, and notifies the error by turning on or blinking the lamp. Alternatively, when the power switch of the transmitter 8334 is turned on, the error notification unit 8343 uses a light source whose brightness changes to transmit a signal such as an ID for a predetermined period (for example, 10 seconds) by a human. An error is notified by flashing it in a recognizable cycle.

As a result, it is checked whether the ID stored in the ID storage unit 8336 and the frequency calculated from the ID are appropriate, so that an erroneous ID is transmitted and a luminance change at an erroneous frequency occurs. Can be prevented.

FIG. 121 is a block diagram showing a structure of the receiver in Embodiment 5.

The receiver 8344 has the same function as the receiver according to any of the above-described first to fourth embodiments, and also has a light receiving unit 8345, a frequency detecting unit 8346, an ID detecting unit 8347, a frequency comparing unit 8348, and a frequency calculating unit 8349. Is equipped with.

The light receiving unit 8345 includes, for example, an image sensor, and acquires an image including a bright line pattern by imaging (visible light imaging) a transmitter that changes in brightness. The ID detection unit 8347 detects the ID of the transmitter from the image. That is, the ID detection unit 8347 acquires the ID of the transmitter by demodulating the data specified by the bright line pattern included in the image. The frequency detection unit 8346 detects the frequency of the brightness change of the transmitter from the image. That is, the frequency detection unit 8346 specifies the frequency of the transmitter from the bright line pattern included in the image, as in the example described with reference to FIG. 115.

The frequency calculation unit 8349 calculates the frequency of the transmitter by dividing the ID detected by the ID detection unit 8347 with respect to the ID as described above, for example. The frequency comparison unit 8348 compares the frequency detected by the frequency detection unit 8346 with the frequency calculated by the frequency calculation unit 8349. Here, if these frequencies are different, the frequency comparison unit 8348 determines that the detected ID is incorrect, and causes the ID detection unit 8347 to detect the ID again. This can prevent the acquisition of an incorrect ID.

FIG. 122 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

The transmitter may distinguish and transmit each of the symbols “00, 01, 10, 11” by changing the position where the luminance change occurs in a predetermined time unit.

For example, when transmitting the symbol “00”, the transmitter transmits the symbol “00” by changing the luminance only in the first period which is the first period in the time unit. Further, when transmitting the symbol “01”, the transmitter transmits the symbol “01” by changing the luminance only in the second section which is the second section in the time unit. Similarly, when transmitting the symbol "10", the transmitter transmits the symbol "10" and transmits the symbol "11" by changing the luminance only in the third section which is the third section in the time unit. When doing so, the transmitter transmits the symbol “11” by changing the luminance only in the fourth section which is the fourth section in the time unit.

As described above, according to the present embodiment, when any symbol is transmitted, the luminance changes within one section, so that the entire section (slot) has a lower luminance than the above-described transmitter. The flicker can be suppressed.

FIG. 123 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

The transmitter may distinguish and transmit each of the symbols “0, 1” by changing the presence or absence of a luminance change in a predetermined time unit. For example, when transmitting the symbol "0", the transmitter transmits the symbol "0" by not changing the luminance in time units. Further, when transmitting the symbol “1”, the transmitter transmits the symbol “1” by changing the luminance in time units.

FIG. 124 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

The transmitter may distinguish and transmit each of the symbols “00, 01, 10, 11” by changing the intensity (frequency) of the luminance change in a predetermined time unit. For example, when transmitting the symbol "00", the transmitter transmits the symbol "00" by not changing the luminance in time units. Further, when transmitting the symbol “01”, the transmitter transmits the symbol “01” by changing the luminance in time units (changing the luminance at a low frequency). Further, when transmitting the symbol “10”, the transmitter transmits the symbol “10” by violently changing the luminance in time units (changing the luminance at a high frequency). Then, when transmitting the symbol “11”, the transmitter transmits the symbol “11” by changing the luminance more drastically in the time unit (changing the luminance at a higher frequency).

FIG. 125 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

The transmitter may distinguish and transmit each of the symbols "0, 1" by making the phase of the luminance change in a predetermined time unit different. For example, when transmitting the symbol “0”, the transmitter transmits the symbol “0” by changing the luminance in a predetermined phase in a time unit. Further, when transmitting the symbol “1”, the transmitter transmits the symbol “1” by changing the luminance in a phase opposite to the above phase in time unit.

FIG. 126 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

When transmitting a signal such as an ID, the transmitter may change the brightness for each color such as red, green, and blue. As a result, the transmitter can transmit a signal having a larger amount of information to the receiver that can recognize the brightness change for each color. Further, the brightness change of any color may be used for clock adjustment. For example, a change in red brightness may be used for clock adjustment. In this case, the brightness change of red color serves as a header. As a result, it is not necessary to use a header for the luminance change of colors other than red (green and blue), and redundant data transmission can be suppressed.

127 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG.

The transmitter may express the brightness of a combined color (for example, white) by combining a plurality of colors such as red, green, and blue. That is, the transmitter expresses the brightness change of the composite color by changing the brightness for each color such as red, green, and blue. Due to the change in the brightness of the composite color, the signal to be transmitted is transmitted as in the above visible light communication. Here, the brightness of any one or more of the colors of red, green, and blue may be used for adjustment for expressing the predetermined brightness of the composite color. Accordingly, it is possible to transmit the signal by the luminance change of the composite color and also transmit the signal by the luminance change of any two colors of red, green and blue. In other words, the transmitter can send signals to the receiver that can recognize only the brightness change of the composite color (for example, white) as described above, and can also recognize each color such as red, green, and blue. More signals can be transmitted to such receivers, for example, as supplementary information.

FIG. 128 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

The transmitter is equipped with four light sources. Each of these four light sources (for example, an LED lamp) emits light of a color represented by different positions 8351a, 8351b, 8352a, 8352b in the CIE xy chromaticity diagram shown in FIG.

The transmitter transmits each signal by switching between the first lighting transmission and the second lighting transmission. Here, the first lighting transmission is a process of transmitting the signal “0” by turning on the light source that emits the color light at the position 8351a and the light source that emits the light color at the position 8351b among the four light sources. Is. The second lighting transmission is a process of transmitting the signal “1” by turning on the light source that emits the light of the color of position 8352a and the light source that emits the light of the color of position 8352b. The image sensor of the receiver can distinguish the color represented by each of the positions 8351a, 8351b, 8352a, 8352b so that the receiver can properly receive the signal "0" and the signal "1". it can.

Here, when the first lighting transmission is performed, the color represented by the position intermediate between the positions 8351a and 8351b in the CIE xy chromaticity diagram appears to human eyes. Similarly, when the second lighting transmission is being performed, the color represented by the position intermediate between the positions 8352a and 8352b in the CIE xy chromaticity diagram is visible to the human eye. Therefore, by appropriately adjusting the color and the brightness of the four light sources, the position in the middle of the positions 8351a and 8351b and the position in the middle of the positions 8352a and 8352b can be matched (to the position 8353). Accordingly, even if the first lighting transmission and the second lighting transmission are switched, the emission color of the transmitter appears to the human eye as if it were fixed, so that it is possible to suppress human flicker. ..

FIG. 129 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5.

The transmitter includes an ID storage unit 8361, a random number generation unit 8362, an addition unit 8363, an encryption unit 8364, and a transmission unit 8365. The ID storage unit 8361 stores the ID of the transmitter. The random number generation unit 8362 generates different random numbers at regular time intervals. The addition unit 8363 combines the latest random number generated by the random number generation unit 8362 with the ID stored in the ID storage unit 8361, and outputs the result as the edited ID. The encryption unit 8364 generates an encrypted edit ID by encrypting the edit ID. The transmitter 8365 transmits the encrypted edit ID to the receiver by changing the brightness.

The receiver includes a receiving unit 8366, a decoding unit 8367, and an ID acquisition unit 8368. The receiving unit 8366 receives the encrypted edit ID from the transmitter by capturing an image (visible light capturing) of the transmitter. The decryption unit 8367 restores the edit ID by decrypting the received encrypted edit ID. The ID acquisition unit 8368 acquires the ID by extracting the ID from the restored edit ID.

For example, the ID storage unit 8361 stores the ID “100”, and the random number generation unit 8362 generates the latest random number “817” (example 1). In this case, the addition unit 8363 generates and outputs the edit ID “100817” by combining the random number “817” with the ID “100”. The encryption unit 8364 generates the encrypted edit ID “abced” by encrypting the edit ID “100817”. The decryption unit 8367 of the receiver restores the edit ID “100817” by decrypting the encrypted edit ID “abced”. Then, the ID acquisition unit 8368 extracts the ID “100” from the restored editing ID “100817”. In other words, the ID acquisition unit 8368 acquires the ID “100” by deleting the last three digits of the edit ID.

Next, the random number generation unit 8362 generates a new random number “619” (Example 2). In this case, the adding unit 8363 generates and outputs the edit ID “100619” by combining the random number “619” with the ID “100”. The encryption unit 8364 encrypts the edit ID “100619” to generate the encrypted edit ID “difia”. The decryption unit 8367 of the transmitter restores the edit ID “100619” by decrypting the encrypted edit ID “difia”. Then, the ID acquisition unit 8368 extracts the ID “100” from the restored editing ID “100619”. In other words, the ID acquisition unit 8368 acquires the ID “100” by deleting the last three digits of the edit ID.

In this way, the transmitter does not simply encrypt the ID, but encrypts a combination of random numbers that are changed at regular intervals, so the ID can be easily decoded from the signal transmitted from the transmission unit 8365. Can be prevented. In other words, when the encrypted ID is simply transmitted from the transmitter to the receiver several times, even if the ID is encrypted, if the ID is the same, the transmitter transmits the ID to the receiver. Since the transmitted signal is the same, its ID may be decrypted. However, in the example shown in FIG. 129, the random number that is changed at regular time intervals is combined with the ID, and the ID that combines the random numbers is encrypted. Therefore, even if the same ID is transmitted to the receiver several times, the signals transmitted from the transmitter to the receiver can be different if the transmission timings of the IDs are different. As a result, it is possible to prevent the ID from being easily decrypted.

130 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. [FIG.

The transmitter includes an ID storage unit 8371, a clock unit 8372, an addition unit 8373, an encryption unit 8374, and a transmission unit 8375. The ID storage unit 8371 stores the ID of the transmitter. The clock unit 8372 clocks and outputs the current date and time (current date and time). The addition unit 8373 combines the ID stored in the ID storage unit 8371 with the current date and time output from the time counting unit 8372 as the transmission date and time, and outputs the result as the edited ID. The encryption unit 8374 generates an encrypted edit ID by encrypting the edit ID. The transmitter 8375 transmits the encrypted edit ID to the receiver by changing the brightness.

The receiver includes a reception unit 8376, a decoding unit 8377, a validity determination unit 8378, and a time counting unit 8379. The receiving unit 8376 receives the encrypted edit ID from the transmitter by capturing an image (visible light capturing) of the transmitter. The decryption unit 8377 restores the edit ID by decrypting the received encrypted edit ID. The clock unit 8379 clocks and outputs the current date and time (current date and time). The validity determination unit 8378 acquires the ID by extracting the ID from the restored edit ID. Furthermore, the validity determination unit 8378 determines the validity of the ID by extracting the transmission date and time from the restored edit ID and comparing the transmission date and time with the current date and time output from the clock unit 8379. For example, the validity determination unit 8378 determines that the ID is invalid when the difference between the transmission date and time and the current date and time is longer than a predetermined time, or when the transmission date and time is newer than the current date and time. ..

For example, the ID storage unit 8371 stores the ID “100”, and the timer unit 8372 outputs the current date and time “20130111200” (12:00, May 1, 2013) as the transmission date and time (Example 1). In this case, the adding unit 8373 combines the ID “100” with the transmission date and time “2013015011200” to generate and output the edit ID “1002013015011200”. The encryption unit 8374 generates the encrypted edit ID “ei39ks” by encrypting the edit ID “1002013015011200”. The decryption unit 8377 of the receiver restores the edit ID "1002013050111200" by decrypting the encrypted edit ID "ei39ks". Then, the validity determination unit 8378 extracts the ID “100” from the restored edit ID “1002013030111200”. In other words, the validity determination unit 8378 acquires the ID “100” by deleting the last 12 digits of the edit ID. Further, the validity determination unit 8378 extracts the transmission date and time “201305011200” from the restored edit ID “1002013015011200”. If the transmission date and time “201105011200” is older than the current date and time output from the time counting unit 8379 and the difference between the transmission date and time and the current date and time is within 10 minutes, for example, the validity determination unit 8378 will identify that ID “100”. Is determined to be valid.

On the other hand, the ID storage unit 8371 stores the ID “100”, and the time counting unit 8372 outputs the current date and time “20140111730” (17:30 on January 1, 2014) as the transmission date and time (Example 2). In this case, the adding unit 8373 combines the ID “100” with the transmission date/time “2014011017730” to generate and output the edit ID “1002014010111730”. The encryption unit 8374 generates an encrypted edit ID “002jflk” by encrypting the edit ID “1002014010111730”. The decryption unit 8377 of the receiver restores the edit ID “1002014010117730” by decrypting the encrypted edit ID “002jflk”. Then, the validity determination unit 8378 extracts the ID “100” from the restored editing ID “1002014010111730”. In other words, the validity determination unit 8378 acquires the ID “100” by deleting the last 12 digits of the edit ID. Furthermore, the validity determination unit 8378 extracts the transmission date/time “2014011017730” from the restored edit ID “100201401011730”. Then, if the transmission date and time “20120110730” is newer than the current date and time output from the time counting unit 8379, the validity determination unit 8378 determines that the ID “100” is invalid.

In this way, the transmitter does not simply encrypt the ID, but encrypts a combination of the current date and time that is changed at regular intervals, so that the ID can be easily obtained from the signal transmitted from the transmitter 8375. It can be prevented from being decrypted. In other words, when the encrypted ID is simply transmitted from the transmitter to the receiver several times, even if the ID is encrypted, if the ID is the same, the transmitter transmits the ID to the receiver. Since the transmitted signal is the same, its ID may be decrypted. However, in the example shown in FIG. 130, the current date and time, which is changed at regular intervals, is combined with the ID, and the ID with the combined current date and time is encrypted. Therefore, even if the same ID is transmitted to the receiver several times, the signals transmitted from the transmitter to the receiver can be different if the transmission timings of the IDs are different. As a result, it is possible to prevent the ID from being easily decrypted.

Further, by comparing the transmission date and time when the encrypted edit ID is transmitted with the current date and time, it is determined whether or not the acquired ID is valid, so the validity of the ID is managed based on the transmission and reception time. be able to.

When the receiver shown in FIGS. 129 and 130 acquires the encrypted edit ID, the receiver may transmit the encrypted edit ID to the server and acquire the ID from the server.

(Guidance at the station)
FIG. 131 shows an example of a usage pattern of the present invention in a train platform. A user holds a mobile terminal over an electronic bulletin board or a light and obtains information displayed on the electronic bulletin board, train information of a station where the electronic bulletin board is installed, station premises information, or the like by visible light communication. Here, the information itself displayed on the electronic bulletin board may be transmitted to the mobile terminal by visible light communication, or the ID information corresponding to the electronic bulletin board is transmitted to the mobile terminal and the ID information acquired by the mobile terminal is displayed. The information displayed on the electronic bulletin board may be acquired by inquiring of the server. When the ID information is transmitted from the mobile terminal, the server transmits the content displayed on the electronic bulletin board to the mobile terminal based on the ID information. The train ticket information stored in the memory of the mobile terminal is compared with the information displayed on the electronic bulletin board, and when the ticket information corresponding to the user's ticket is displayed on the electronic bulletin board, On the display, an arrow indicating the destination to the home where the user's scheduled train arrives is displayed. The route to the vehicle close to the exit or the transfer route may be displayed when getting off. If a seat is designated, the route to that seat may be displayed. When displaying the arrow, the arrow can be displayed more clearly by using the same color as the color of the train route in the map or the train guidance information. In addition to displaying the arrow, the user's reservation information (home number, vehicle number, departure time, seat number) can be displayed. By displaying the reservation information of the user together, it is possible to prevent erroneous recognition. If the ticket information is stored in the server, contact the server from the mobile terminal to obtain the ticket information for comparison, or compare the ticket information on the server side with the information displayed on the electronic bulletin board. With this, it is possible to obtain information related to the ticket information. The target route may be estimated from the history of the user's transfer search, and the route may be displayed. Further, not only the contents displayed on the electronic bulletin board, but also the train information/premise information of the station where the electronic bulletin board is installed may be acquired and compared. Information related to the user may be highlighted or rewritten and displayed on the display of the electronic bulletin board on the display. When the user's boarding schedule is unknown, an arrow for guiding to the platform of each route may be displayed. When the station premises information is acquired, an arrow for guiding to a shop or restroom may be displayed on the display. The behavior characteristics of the user may be managed in advance by the server, and when the user often stops at the shop/toilet at the station, an arrow for guiding to the shop/toilet may be displayed on the display. It is possible to reduce the processing amount by displaying the arrow that guides the user to the shop/restroom, etc. only to the users who have the behavioral characteristics of visiting the shop/restroom and not displaying it to other users. . The color of the arrow that guides the shop or restroom may be different from the color of the arrow that guides you to your home. When both arrows are displayed simultaneously, different colors can be used to prevent erroneous recognition. Note that although an example of a train is shown in FIG. 131, display can be performed with a similar configuration on an airplane, a bus, or the like.

(Translation of the guide sign)
FIG. 132 shows an example in the case of obtaining information by visible light communication from an electronic guide display board installed in an airport, a station yard, a tourist resort, a hospital, or the like. The information on the display content is acquired from the electronic guide display board by visible light communication, the information on the display content is translated into the language information set in the mobile terminal, and then displayed on the display of the mobile terminal. Since the information is translated and displayed in the user's language, the user can easily understand the information. The translation of the language may be performed in the mobile terminal or in the server. When translating on the server, the information of the display content acquired by visible light communication and the language information of the mobile terminal are transmitted to the server, the server translates the information, transmits to the mobile terminal, and displays on the display of the mobile terminal. May be displayed. Further, when the ID information is acquired from the electronic guide display board, the ID information may be transmitted to the server and the display content information corresponding to the ID information may be acquired from the server. Furthermore, based on the nationality information, ticket information, and baggage deposit information stored in the mobile terminal, an arrow that guides the user to the next place may be displayed.

(Coupon popup)
FIG. 133 shows an example in which the coupon information acquired by visible light communication is displayed or a popup is displayed on the display of the mobile terminal when the user approaches the store. The user acquires the coupon information of the store from the electronic bulletin board or the like by visible light communication using the mobile terminal. Next, when the user enters within a predetermined range from the store, coupon information of the store or popup is displayed. Whether the user has entered the predetermined range from the store is determined using GPS information of the mobile terminal and store information included in the coupon information. The ticket information is not limited to the coupon information. When a store where coupons and tickets can be used, etc. is approached, an alert is automatically issued, so that the user can properly use the coupons and tickets.

FIG. 134 shows an example in which coupon information/ticket information or a pop-up is displayed on the display of a mobile terminal at a cash register, a ticket gate, or the like. Visible light communication is used to acquire position information from a lighting installed at a cash register or a ticket gate, and when the acquired position information matches the information included in the coupon information/ticket information, the display is performed. It should be noted that the barcode reader may have the light emitting unit and may obtain the position information by performing visible light communication with the light emitting unit, or may obtain the position information from the GPS of the mobile terminal. A transmitter may be installed near the cashier, and the user may hold the receiver over the transmitter to display coupon or payment information on the display of the receiver, or the receiver may communicate with the server. Payment processing may be performed. Further, when the coupon information/ticket information includes Wi-Fi information installed in a store or the like, and the user's mobile terminal acquires the same information as the Wi-Fi information included in the coupon information/ticket information. May be displayed.

(Launch operation application)
FIG. 135 shows an example in which a user uses a mobile terminal to obtain information from a home appliance by visible light communication. When the ID information or the information about the home appliance is acquired from the home appliance by the visible light communication, the application for operating the home appliance is automatically started. In FIG. 135, an example using a television is shown. With such a configuration, an application for operating a home electric appliance can be activated by simply holding the mobile terminal over the home electric appliance.

(Stop transmission while the barcode reader is operating)
FIG. 136 shows an example of suspending data communication for visible light communication in the vicinity of the barcode reader 8405a when the barcode reader 8405a reads a barcode of a product. By stopping visible light communication at the time of reading the barcode, it is possible to prevent the barcode reader 8405a from erroneously recognizing the barcode. The barcode reader 8405a transmits a transmission stop signal to the visible light signal transmitter 8405b when the barcode reading button is pressed, and when a finger is released from the button or when the button is released from the button, a predetermined signal is transmitted. When the time has elapsed, the transmission restart signal is transmitted to the visible light signal transmitter 8405b. The transmission stop signal and the transmission restart signal are transmitted by wired/wireless communication, infrared communication, or sound wave communication. The bar code reader 8405a transmits a transmission stop signal when it is estimated that the bar code reader 8405a has been moved from the measurement value of the acceleration sensor provided in the bar code reader 8405a, and that the bar code reader 8405a has not been moved for a predetermined time. The transmission restart signal may be transmitted when the estimation is performed. The bar code reader 8405a transmits a transmission stop signal when it is estimated that the bar code reader 8405a is grasped from the measurement values of the electrostatic sensor and the illuminance sensor provided in the bar code reader 8405a, and when it is estimated that the hand is released. A transmission restart signal may be transmitted. The bar code reader 8405a detects that the bar code reader 8405a has been lifted because the switch provided on the ground surface of the bar code reader 8405a is released from the pressed state, and transmits a transmission stop signal. It is also possible to detect that the bar code reader 8405a has been placed because the button has been pressed, and transmit the transmission restart signal. The bar code reader 8405a detects that the bar code reader 8405a has been lifted from the measurement value of the switch in the bar code reader storage area or the infrared sensor, transmits a transmission stop signal, and detects that the bar code reader 8405a has been returned. Then, the transmission restart signal may be transmitted. The cash register 8405c may transmit a transmission stop signal when the operation is started, and may transmit a transmission restart signal when the settlement operation is performed.

For example, the transmitter 8405b configured as lighting stops signal transmission when it receives a transmission stop signal, and operates so as to reduce ripple (change in luminance) of 100 Hz to 100 kHz. Alternatively, the luminance change of the signal pattern is reduced to continue the signal transmission. Alternatively, the cycle of the carrier wave is set longer than the reading time of the bar code of the bar code reader 8405a, or the cycle of the carrier wave is set shorter than the exposure time of the bar code reader 8405a. This can prevent malfunction of the barcode reader 8405a.

As shown in FIG. 137, a transmitter 8406b configured as, for example, an illumination detects that there is a person near the barcode reader 8406a with a motion sensor or a camera, and stops signal transmission. Alternatively, the same operation as when the transmitter 8405b receives the transmission stop signal is performed. The transmitter 8406b resumes signal transmission when it detects that there is no person near the barcode reader 8406a. The transmitter 8406b may detect the operation sound of the barcode reader 8406a and stop the signal transmission for a predetermined time.

(Sending information from a computer)
FIG. 138 shows an example of the usage pattern of the present invention.

The transmitter 8407a configured as a personal computer, for example, transmits a visible light signal through a display such as a built-in display, a connected display, and a projector. The transmitter 8407a transmits the URL of the website displayed by the browser, the information on the clipboard, and the information defined by the application having the focus. For example, the coupon information acquired on the website is transmitted.

(Database)
FIG. 139 shows an example of the configuration of the database held by the server that manages the ID transmitted by the transmitter.

The database has an ID-data table that holds data to be passed to an inquiry using the ID as a key, and an access log table that stores a record of the inquiry using the ID as a key. In the ID-data table, the ID transmitted by the transmitter, the data to be transferred in response to the inquiry using the ID as a key, the condition for transmitting the data, the number of times of access using the ID as the key, and the data after clearing the condition are passed. Has a number of times. The conditions for passing data include the date and time, the number of times of access, the number of times of successful access, information of the terminal of the inquiry source (terminal model, application that made the inquiry, current location of the terminal, etc.), and user information of the inquiry source ( Age, gender, occupation, nationality, language used, religion, etc.) By setting the number of times of successful access as a condition, it is possible to provide a service method such as “1 yen per access, but 100 yen is the upper limit and no data is returned thereafter”. The log table, when accessed using the ID as a key, the ID of the requesting user, the time of day, other supplementary information, whether the data was passed after clearing the conditions, and the passed data. Record the contents of.

(Reception start gesture)
FIG. 140 shows an example of a gesture operation for starting reception by this communication method.

The user starts reception by sticking out a receiver configured as, for example, a smartphone and rotating the wrist left and right. The receiver detects this operation from the measurement value of the 9-axis sensor and starts reception. The receiver may initiate reception if it detects at least one of these actions. The operation of sticking out the receiver has an effect that the transmitter is imaged larger and the reception speed and accuracy are improved as the receiver approaches the transmitter. The operation of rotating the wrist to the left and right has the effect of changing the angle of the receiver, eliminating the angle dependence of this method, and enabling stable reception.

Note that this operation may be performed only when the home screen is in the foreground. As a result, it is possible to prevent the main communication from being performed against the user's intention while using another application.

Note that the image sensor may be activated at the time of detecting the motion of the receiver protruding, and the reception may be stopped if there is no motion of shaking the wrist from side to side. Since it takes several hundred milliseconds to 2 seconds to start the image sensor, the responsiveness can be further improved.

(Control of transmitter by power line)
FIG. 141 shows an example of the transmitter of the present invention.

The signal control unit 8410g controls the transmission state of the transmitter 8410a (contents of a signal to be transmitted, presence/absence of transmission, intensity of luminance change used for transmission, and the like). The signal control unit 8410g sends the control content of the transmitter 8410a to the power distribution control unit 8410f. The power distribution control unit 8410f changes the voltage, the current, or the frequency supplied to the power supply unit 8410b of the transmitter 8410a, thereby transmitting the control content in the form of the magnitude of change and the time of change. The power supply unit 8410b performs a constant output without being affected by a slight change in voltage, current, or frequency. Therefore, a signal exceeding a stabilization capability of the power supply unit 8410b, for example, a timing or a time width of cutting power supply causes a signal to be output. A signal is transmitted by expressing. The brightness control unit 8410d receives the content transmitted by the power distribution control unit 8410f and changes the brightness change pattern of the light emitting unit in consideration of the conversion by the power supply unit 8410b.

(Encoding method)
FIG. 142 is a diagram for explaining one of the visible light communication image encoding methods.

In this encoding method, since the ratios of white and black are almost the same, the average luminance of the normal phase image and the negative phase image are almost the same, and there is an advantage that humans hardly feel the flicker.

(Coding method that can receive light even when imaged from an oblique direction)
FIG. 143 is a diagram for explaining one of the visible light communication image encoding methods.

The image 1001a is an image in which white and black lines are displayed with a uniform width. When this image 1001a is imaged obliquely, in the image 1001b obtained by the imaging, the left line appears thin and the right line appears thick. When the image 1001a is projected on a curved surface, lines having different thicknesses appear in the image 1001i obtained by capturing the image.

Therefore, a visible light communication image is created by the following encoding method. From the left, the visible light communication image 1001c is composed of a white line, a black line having a thickness three times that of the white line, and a white line having a thickness one-third the thickness of the black line. In this way, the preamble is encoded as an image in which a line three times thicker than the line on the left side and a line one-third the thickness of the line on the left side continue. As in the visible light communication images 1001d and 1001e, a line having the same thickness as the line on the left is encoded as “0”. As in the visible light communication images 1001f and 1001g, a line that is twice as thick as the left adjacent line or half the thickness of the left adjacent line is encoded as "1". That is, when the thickness of the encoding target line is different from the thickness of the line on the left side, the encoding target line is encoded as "1". As an example using this encoding method, a signal including "01011101011" following a preamble is represented by an image such as a visible light communication image 1001h. In addition, here, a line having the same thickness as the left adjacent line is coded as "0" and a line having a different thickness as the left adjacent line is coded as "1", but a line having the same thickness as the left adjacent line is coded. May be coded as "1" and a line having a different thickness from the line on the left may be coded as "0". Further, not only the comparison with the line on the left side but also the comparison with the line on the right side may be performed. That is, "1" and "0" may be encoded depending on whether the thickness of the encoding target line and the thickness of the line on the right are the same or different. In this way, on the transmitter side, the color is different from that of the line to be encoded, and by making the thickness of the adjacent line and the line of the line to be encoded the same, “0” is encoded and a different line thickness is obtained. By doing so, "1" is encoded.

On the receiver side, a visible light communication image is captured, and the thickness of the white or black line is detected in the captured visible light communication image. If the color of the line to be decoded is different from that of the line to be decoded and the line adjacent to the line to be decoded (on the left or on the right) is compared with the thickness of the line to be decoded, and the thickness is the same , The decoding target line is decoded as “0”, and when the thicknesses are different, the decoding target line is decoded as “1”. It may be decoded as "1" when the thickness is the same and as "0" when the thickness is different. Finally, the data is decoded based on the decoded data sequence of 1 and 0.

This encoding method uses a local line thickness relationship. As seen in the image 1001b and the image 1001i, the thickness ratios of the neighboring lines do not change significantly, and therefore, even if the image is taken from an oblique direction or projected on a curved surface, it is created by this encoding method. The visible light communication image can be correctly decoded.

In this encoding method, since the ratio of white and black is approximately the same, the average luminance of the normal-phase image and the inverse-phase image are approximately the same, and there is the advantage that humans are less likely to perceive flicker. In addition, since this encoding method is not limited to black and white, there is an advantage that the visible light communication image of either the positive phase signal or the negative phase signal can be decoded by the same algorithm.

Further, there is an advantage that it is easy to add a code. For example, the visible light communication image 1001j is a combination of a line that is quadruple in thickness on the left side and a line that is ¼ on the left side. In this way, there are many unique patterns such as "5 times and 1/5 on the left side" and "3 times and 2/3 on the left side". It can be defined. For example, a visible light communication image can represent one piece of data with multiple sheets, but the visible light is used as a cancellation signal that indicates that some of the data received so far has become invalid because the data to be transmitted has been changed. It is conceivable to use the communication image 1001j. The color is not limited to white and black, and may be any color as long as it is different. For example, complementary colors may be used.

(Coding method in which the amount of information varies depending on the distance)
144 and 145 are diagrams for explaining one of the coding methods for visible light communication images.

As shown in (a-1) of FIG. 144, when a 2-bit signal is expressed by making one part black and the remaining part white in the image divided into four, the average luminance of this image is When white is 100% and black is 0%, it is 75%. When the white and black portions are reversed as in (a-2) of FIG. 144, the average luminance is 25%.

In the image 1003a, the white portion of the visible light communication image created by the encoding method of FIG. 143 is represented by the image of (a-1) in FIG. 144, and the black portion is represented by the image of (a-2) in FIG. 144. 2 is a visible light communication image. This visible light communication image shows the signal A encoded by the encoding method of FIG. 143 and the signal B encoded by (a-1) and (a-2) of FIG. 144. When the nearby receiver 1003b captures the visible light communication image 1003a, a fine image 1003d is obtained, and both the signal A and the signal B can be received. When the distant receiver 1003c captures the visible light communication image 1003a, a small image 1003e is obtained. In the image 1003e, the detailed portion cannot be confirmed, and the portion (a-1) of FIG. 144 is white and the portion (a-2) of FIG. 144 is black, so only the signal A can be received. .. As a result, the shorter the distance between the visible light communication image and the receiver, the more information can be transmitted. As a method of encoding the signal B, the combination of (b-1) and (b-2) in FIG. 144 or the combination of (c-1) and (c-2) in FIG. 144 may be used.

By using the signals A and B, it is possible to represent basic and important information with the signal A and additional information with the signal B. When the receiver transmits the signals A and B as ID information to the server and the server transmits information corresponding to the ID information to the receiver, the information transmitted by the server may be changed depending on the presence or absence of the signal B. It will be possible.

(Encoding method with divided data)
FIG. 146 is a diagram for explaining one of visible-light communication image encoding methods.

The signal 1005a to be transmitted is divided into a plurality of data pieces 1005b, 1005c, 1005d. An address indicating the position of the data piece is added to each data piece, and further, a preamble, an error detection/correction code, a frame type description, etc. are added to form frame data 1005e, 1005f, 1005g. The frame data is encoded to create and display visible light communication images 1005h, 1005i, 1005j. When the display area is sufficiently large, a visible light communication image 1005k in which a plurality of visible light communication images are connected is displayed.

As shown in FIG. 146, as a method of inserting a visible light communication image in a video, in the case of a display device using a solid-state light source, the visible light communication signal image is normally displayed, and only during the display period. By implementing a method in which the fixed light source is turned on and the solid-state light source is turned off during other periods, a wide range of display devices, for example, a projector using DMD, a projector using liquid crystal including LCOS, Of course, display devices using MEMS can be applied. Further, even in a display device of a type in which an image display is divided into subframes and displayed, for example, in a display device that does not use a light source such as a backlight such as a PDP or an EL, a part of the subframes is converted into a visible light communication image. By replacing it, adaptation becomes possible. A semiconductor laser, an LED light source, or the like can be considered as the solid-state light source.

(Effect of inserting reverse phase image)
FIG. 147 and FIG. 148 are diagrams for explaining one of the encoding methods for visible light communication images.

As in (1006a) of FIG. 147, the transmitter inserts a black image between the video and the visible light communication image (normal phase image). The image captured by the receiver is as shown in (1006b) of FIG. 147. Since it is easy for the receiver to search for a portion in which the line of pixels to be exposed at the same time is black, it is easy to set the position where the visible light communication image is captured as the position of the pixel to be exposed at the next timing. Can be specified.

As shown in (1006a) of FIG. 147, the transmitter displays the visible light communication image (normal phase image) and then the reverse running visible light communication image in which black and white are reversed. The receiver obtains the difference between the pixel values of the normal-phase image and the negative-phase image, thereby obtaining an SN ratio twice as high as when only the normal-phase image is used. On the other hand, when the same SN ratio is ensured, the difference in brightness between black and white can be suppressed to half, and flicker when viewed by a human can be suppressed. Further, as in (1007a) and (1007b) of FIG. 148, the moving average of the expected value of the difference in luminance between the video and the visible light communication image is canceled in the normal phase image and the negative phase image. Since the time resolution of human vision is about 1/60 second, it is possible to make humans feel that the visible light communication image is not displayed by setting the time for displaying the visible light communication image to be less than this. ..

As shown in (1006c) of FIG. 147, the transmitter may further insert a black image between the normal phase image and the negative phase image. In this case, the image shown by (1006d) in FIG. 147 is obtained by imaging with the receiver. In the image shown in (1006b) of FIG. 147, since the normal phase image pattern and the negative phase image pattern are adjacent to each other, the pixel values may be averaged at the boundary portion. The image shown in) does not cause such a problem.

(Super resolution)
FIG. 149 is a diagram illustrating one of the methods for encoding a visible light communication image.

As shown in (a) of FIG. 149, when the video data and the signal data to be transmitted by the visible light communication are separated, the super resolution processing is performed on the video data and the obtained super resolution image is subjected to the visible light communication. Superimpose images. That is, the super-resolution processing is not performed on the visible light communication image. When the visible light communication image is already superimposed on the video data as shown in (b) of FIG. 149, (1) the data is shown by the edge of the visible light communication image (color difference such as white and black). (2) super-resolution processing is performed so that (2) the average image of the normal phase image and the negative phase image of the visible light communication image has uniform brightness. In this way, by changing the processing for the visible light communication image depending on whether or not the visible light communication image is superimposed on the video data, it is possible to perform the visible light communication more appropriately (reduce the error rate). Becomes

(Indication of compatibility with visible light communication)
FIG. 150 is a diagram illustrating one operation of the transmitter.

The transmitter 8500a displays that it is compatible with visible light communication by being superimposed on an image to be projected or displayed. This display is displayed, for example, only for a predetermined time after the transmitter 8500a is started.

The transmitter 8500a transmits the fact that it supports visible light communication to the connected device 8500c. The device 8500c displays that the transmitter 8500a is compatible with visible light communication. For example, the display of the device 8500c indicates that the transmitter 8500a is compatible with visible light communication. The device 8500c transmits data for visible light communication to the transmitter 8500a when the connected transmitter 8500a supports visible light communication. The indication that the transmitter 8500a is compatible with visible light communication may be displayed when the device 8500c is connected to the transmitter 8500a, or may be displayed from the device 8500c to the transmitter 8500a for visible light communication. It may be displayed when the data is sent. When displaying when the data for visible light communication is transmitted from the device 8500c, the transmitter 8500a obtains the identification information indicating the visible light communication from the data, and the identification information includes the data for the visible light communication. When it is included, it may be displayed that the transmitter 8500a is compatible with visible light communication.

In this way, by displaying on the projection screen or on the display of the device that the transmitter (lighting, projector, video display device) supports visible light communication, or whether or not it supports it. The user can easily understand whether the transmitter supports visible light communication. Therefore, it is possible to prevent a malfunction in which visible light communication cannot be performed even though the device transmits visible light communication data to the transmitter.

(Information acquisition using visible light communication signal)
FIG. 151 is a diagram illustrating one application example of visible light communication.

The transmitter 8501a receives the video data and the signal data from the device 8501c, and displays the visible light communication image 8501b. The receiver 8501d images the visible light communication image 8501b and receives the signal included in the visible light communication image. The receiver 8501d communicates with the device 8501c from the information (address, password, etc.) included in the received signal, and the video itself displayed by the transmitter 8501a and its accompanying information (video ID, URL, password, SSID, (Translation data, voice data, hash tags, product information, purchase information, coupons, vacant seat information, etc.). The device 8501c may transmit the transmission status to the transmitter 8501a to the server 8501e, and the receiver 8501d may obtain the information from the server 8501e.

(data format)
FIG. 152 is a diagram for explaining one of the formats of visible light communication data.

The data shown in (a) of FIG. 152 has a video address table showing the position of video data and a position address table showing the position of signal data transmitted by visible light communication in the storage area. In a video display device that does not support visible light communication, only the video address table is referred to, so even if the input includes the signal address table and signal data, the video display is not affected. Thereby, backward compatibility with an image display device that does not support visible light communication is maintained.

In the data format shown in (b) of FIG. 152, an identifier indicating that the data that follows is video data is placed before the video data, and an identifier that indicates that the data that follows is signal data. It is arranged in front. By using the identifier, it is inserted into the data only when there is video data or signal data, so that the overall code amount can be reduced. Further, identification information indicating whether the data is video data or signal data may be provided. Further, the program information may include identification information indicating whether or not the data for visible light communication is included. By including the identification information indicating whether or not the program information includes the data for visible light communication, the user can determine whether or not the visible light communication is possible when searching for the program. The program information may include an identifier indicating that the data for visible light communication is included. Furthermore, by adding an identifier/identification information for each data, it is possible to switch the processing such as switching the brightness of each data and switching the super-resolution, and reduce the error rate during visible light communication. It will be possible.

The data format shown in (a) of FIG. 152 is suitable for a situation in which data is read from a storage-type medium such as an optical disk, and the data format shown in (b) of FIG. 152 is suitable for streaming-type data such as television broadcasting. There is. The signal data includes the value of the signal transmitted by visible light communication, the transmission start time, the transmission end time, the location used for transmission on the display or projection surface, the brightness of the visible light communication image, and the bar of the visible light communication image. Includes information such as code orientation.

(Estimate and receive 3D shape)
153 and 154 are diagrams for explaining one of application examples of visible light communication.

As shown in FIG. 153, a transmitter 8503a configured as a projector, for example, projects a distance measurement image in addition to a video image and a visible light communication image. The dot pattern shown in the image for distance measurement is a dot pattern in which the positional relationship of a predetermined number of dots near an arbitrary dot is different from the positional relationship of dots in any other arbitrary combination. The receiver can specify a local dot pattern and estimate the three-dimensional shape of the projection surface 8503b by capturing the image for distance measurement. The receiver restores the visible light communication image distorted by the three-dimensional shape of the projection surface to a planar image and receives a signal. The distance measurement image and the visible light communication image may be projected by infrared rays that cannot be perceived by humans.

As shown in FIG. 154, a transmitter 8504a configured as, for example, a projector includes an infrared projection device 8504b that projects an image for distance measurement with infrared rays. The receiver estimates the stereoscopic shape of the projection surface 8504c from the image for distance measurement, restores the distortion of the visible light communication image, and receives the signal. Note that the transmitter 8504a may project an image with visible light and a visible light communication image with infrared light. The infrared projection device 8504b may project the visible light communication image with infrared rays.

(3D projection)
155 and 156 are diagrams for explaining one of the methods for displaying a visible light communication image.

When stereoscopic projection is performed or when a visible light communication image is displayed on a cylindrical display surface, visible light communication images 8505a to 8505f are displayed as illustrated in FIG. 155 so that reception from a wide angle is possible. Become. Displaying the visible light communication images 8505a and 8505b enables reception from a wide angle in the horizontal direction. By combining the visible light communication images 8505a and 8505b, it becomes possible to receive even if the receiver is tilted. The visible light communication image 8505a and the visible light communication image 8505b may be alternately displayed, or the visible light communication image 8505f that is a composite image of these images may be displayed. Furthermore, by adding the visible light communication image 8505c and the visible light communication image 8505d, it becomes possible to receive from a wide angle in the vertical direction. A boundary between visible light communication images may be expressed by providing a portion that is projected with intermediate colors or a portion that is not projected, such as the visible light communication image 8505e. By rotating the visible light communication images 8505a to 8505f, the visible light communication images 8505a to 8505f can be received from a wider angle. Although the visible light communication image is displayed on the cylindrical display surface in FIG. 155, the visible light communication image may be displayed on the cylindrical display surface.

When performing stereoscopic projection or displaying a visible light communication image on a spherical display surface, by displaying visible light communication images 8506a to 8506d as shown in FIG. 156, reception from a wide angle is possible. Become. Although the visible light communication image 8506a has a wide receivable area in the horizontal direction but a narrow receivable area in the vertical direction, the visible light communication image 8506a is displayed in combination with the visible light communication image 8506b having the opposite property. The visible light communication image 8506a and the visible light communication image 8506b may be alternately displayed, or the visible light communication image 8506c which is an image obtained by combining these images may be displayed. The portion where the barcode is concentrated, such as the upper portion of the visible light communication image 8506a, has a fine display, and there is a high possibility that the signal will be erroneously received. Therefore, it is possible to prevent a reception error by displaying this part in an intermediate color like the visible light communication image 8506d or by not projecting anything.

(Different communication protocol for each zone)
FIG. 157 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5.

The receiver 8420a receives the zone information from the base station 8420h, recognizes in which zone it is located, and selects the reception protocol. The base station 8420h is configured as, for example, a communication base station of a mobile phone, a Wi-Fi access point, an IMES transmitter, a speaker, or a wireless transmitter (Bluetooth (registered trademark), ZigBee, a specific low-power wireless station, etc.). Note that the receiver 8420a may specify the zone based on position information obtained from GPS or the like. As an example, it is assumed that communication is performed at a signal frequency of 9.6 kHz in zone A, and that communication is performed at a signal frequency of 15 kHz for ceiling lighting and 4.8 kHz for signage in zone B. At the position 8420j, the receiver 8420a recognizes that the current location is the zone A from the information of the base station 8420h, receives at the signal frequency of 9.6 kHz, and receives the signals transmitted by the transmitters 8420b and 8420c. At the position 8420l, the receiver 8420a recognizes that the current position is the zone B from the information of the base station 8420i, and further tries to receive the signal from the ceiling lighting because the in-camera is directed upward. Is estimated, reception is performed at a signal frequency of 15 kHz, and the signals transmitted by the transmitters 8420e and 8420f are received. At the position 8420m, the receiver 8420a recognizes that the current position is the zone B from above the base station 8420i, and further estimates that it is about to receive the signal transmitted by the signage from the motion of pushing the out-camera. Reception is performed at a signal frequency of 0.8 kHz, and a signal transmitted by the transmitter 8420g is received. At the position 8420k, the receiver 8420a receives signals from both the base station 8420h and the base station 8420i, and cannot determine whether the current position is zone A or zone B. Therefore, the receiver 8420a performs reception processing at both 9.6 kHz and 15 kHz. I do. In addition, not only the frequency but the server that inquires the modulation method, the signal format, and the ID of the transmission signal may be different in the part where the protocol differs depending on the zone. The base stations 8420h and 8420i may transmit the protocol in the zone to the receiver, or may transmit only the ID indicating the zone and the receiver may acquire the protocol information from the server using the zone ID as a key. Good.

The transmitters 8420b to 8420f receive the zone ID and protocol information transmitted from the base stations 8420h and 8420i, and determine the signal transmission protocol. The transmitter 8420d capable of receiving signals transmitted by both the base station 8420h and the base station 8420i uses the protocol of the zone of the base station having stronger signal strength, or uses both protocols alternately.

(Zone recognition and services for each zone)
FIG. 158 is a diagram illustrating an example of operation of a receiver and a transmitter in Embodiment 5.

The receiver 8421a recognizes the zone to which its own position belongs from the received signal. The receiver 8421a provides services (coupon distribution, point granting, route guidance, etc.) defined for each zone. As an example, the receiver 8421a receives the signal transmitted from the left side of the transmitter 8421b and recognizes that it is in the zone A. Here, the transmitter 8421b may transmit different signals depending on the transmission direction. Further, the transmitter 8421b may transmit a signal such that a different signal is received depending on the distance to the receiver by using a signal having a light emission pattern like 2217a. Further, the receiver 8421a may recognize the positional relationship with the transmitter 8421b and the zone in which the receiver 8421a is located, from the direction and size of the image taken by the transmitter 8421b.

Some of the signals indicating that they are located in the same zone may be common. For example, the first half is common to the IDs representing the zone A transmitted from the transmitter 8421b and the transmitter 8421c. As a result, the receiver 8421a can recognize the zone in which it is located only by receiving the first half of the signal.

(Summary of this Embodiment)
The information communication method according to the present embodiment is an information communication method of transmitting a signal according to a change in brightness, and a determining step of determining a plurality of brightness change patterns by modulating each of a plurality of signals to be transmitted. , Each of the plurality of light emitters changes in luminance according to any one of the plurality of determined patterns of luminance change, and thereby transmits a signal to be transmitted corresponding to any one of the patterns. In the transmitting step, in the transmitting step, each of the two or more light emitters of the plurality of light emitters has two types having different luminances for each predetermined time unit. Luminance is changed at different frequencies so that any one of the two lights is output and the predetermined time units for the two or more light emitters are different from each other. To do.

As a result, as in the operation described with reference to FIG. 113, the brightness of each of two or more light emitters (for example, a transmitter configured as a lighting device) changes at frequencies different from each other. The receiver that receives the signals to be transmitted (for example, the ID of the light emitter) can easily distinguish and acquire the signals to be transmitted.

Further, in the transmitting step, each of the plurality of light emitters changes in luminance at any one frequency of at least four kinds of frequencies, and two or more light emitters of the plurality of light emitters, The brightness may change at the same frequency. For example, in the transmitting step, when the plurality of light emitting bodies are projected on the light receiving surface of the image sensor for receiving the plurality of signals to be transmitted, all the light emitting bodies adjacent to each other on the light receiving surface are projected. The brightness of each of the plurality of light emitters changes so that the frequency of the brightness change varies between them.

Thus, as in the operation described with reference to FIG. 114, if there are at least four types of frequencies used for brightness change, even if there are two or more light-emitting bodies that change brightness at the same frequency, that is, Even if the number of types of frequencies is smaller than the number of multiple light emitters, the brightness change between all light emitters adjacent to each other on the light receiving surface of the image sensor based on the four-color problem or the four-color theorem. It is possible to surely make the frequency of the different. As a result, the receiver can easily distinguish and acquire each of the signals to be transmitted, which are transmitted from the plurality of light emitters.

Further, in the transmitting step, each of the plurality of light emitters may transmit the signal to be transmitted by changing the luminance at a frequency specified by the hash value of the signal to be transmitted.

Accordingly, as in the operation described with reference to FIG. 113, each of the plurality of light emitters changes in luminance at the frequency specified by the hash value of the signal to be transmitted (for example, the ID of the light emitter). When receiving the signal to be transmitted, the machine can determine whether or not the frequency specified by the actual luminance change matches the frequency specified by the hash value. That is, the receiver can determine whether the received signal (eg, the ID of the light emitter) has an error.

Further, in the information communication method, a frequency for calculating a frequency corresponding to the signal to be transmitted as a first frequency from the signal to be transmitted stored in the signal storage unit according to a predetermined function. A calculation step, a frequency determination step of determining whether or not the second frequency stored in the frequency storage unit and the calculated first frequency match, the first frequency and the second frequency A frequency error notifying step of notifying an error when it is determined that the frequencies do not match, and the determining step when it is determined that the first frequency and the second frequency match. Then, a brightness change pattern is determined by modulating the signal to be transmitted stored in the signal storage unit, and in the transmitting step, any one of the plurality of light emitters emits light. The signal of the transmission target stored in the signal storage unit may be transmitted by changing the luminance at the first frequency according to the determined pattern.

Thus, as in the operation described with reference to FIG. 120, the frequency stored in the frequency storage unit and the frequency calculated from the signal to be transmitted stored in the signal storage unit (ID storage unit) are It is determined whether or not they match, and if it is determined that they do not match, an error is notified, so that it is possible to easily detect an abnormality in the signal transmission function of the light emitter.

Further, the information communication method further includes a check value calculation step of calculating a first check value from a signal to be transmitted stored in the signal storage unit according to a predetermined function, and a check value storage unit. A check value determination step of determining whether or not the stored second check value and the calculated first check value match, the first check value and the second check value are If it is determined that they do not match, a check value error notifying step of notifying an error is included, and if it is determined that the first check value and the second check value match, the determining step is performed. Then, a brightness change pattern is determined by modulating the signal to be transmitted stored in the signal storage unit, and in the transmitting step, any one of the plurality of light emitters emits light. The signal of the transmission target stored in the signal storage unit may be transmitted by changing the brightness according to the determined pattern.

As a result, like the operation described with reference to FIG. 120, the check value calculated from the check value stored in the check value storage unit and the transmission target signal stored in the signal storage unit (ID storage unit). Whether or not the values match is determined, and when it is determined that the values do not match, an error is notified, so that it is possible to easily detect an abnormality in the signal transmission function of the light emitter.

Further, the information communication method according to the present embodiment is an information communication method for acquiring information from a subject, and corresponds to a plurality of exposure lines included in the image sensor in an image obtained by photographing the subject with an image sensor. An exposure time setting step of setting an exposure time of the image sensor so that a plurality of bright lines are generated according to a change in the brightness of the object; An image acquisition step of acquiring a bright line image including the plurality of bright lines by capturing with, and information is acquired by demodulating data specified by the plurality of bright line patterns included in the acquired bright line image. And a frequency specifying step of specifying the frequency of the brightness change of the subject based on the pattern of the plurality of bright lines included in the acquired bright line image. For example, in the frequency specifying step, a plurality of header patterns, which are included in the plurality of bright line patterns and are a plurality of predetermined patterns for indicating respective headers, are specified, and the number of pixels between the plurality of header patterns is specified. Is specified as the frequency of the luminance change of the subject.

As a result, as in the operation described with reference to FIG. 115, the frequency of the luminance change of the subject is specified. Therefore, when a plurality of subjects having different luminance change frequencies are photographed, information from those subjects is acquired. Can be easily distinguished and acquired.

Further, in the image acquisition step, the plurality of subjects each having a change in brightness are captured to acquire the bright line image including a plurality of patterns each represented by a plurality of bright lines, and in the information acquisition step, the acquired bright line images are acquired. When a part of each of the plurality of patterns included in the bright line image overlaps, by demodulating the data specified by a part excluding the part from each of the plurality of patterns, the plurality of patterns Information may be obtained from each of the patterns.

As a result, as in the operation described with reference to FIG. 117, since data is not demodulated from the portion where a plurality of patterns (a plurality of bright line patterns) overlap, incorrect information may be acquired. Can be prevented.

Further, in the image acquisition step, a plurality of bright line images are acquired by photographing the plurality of subjects a plurality of times at mutually different timings, and in the frequency identification step, the bright line images are included in the bright line image. The frequency for each of the plurality of patterns is specified, and in the information acquisition step, from the plurality of bright line images before, a plurality of patterns in which the same frequency is specified is searched and the searched plurality of patterns are combined and combined. The information may be acquired by demodulating the data specified by the plurality of generated patterns.

As a result, a plurality of patterns in which the same frequency is specified (a plurality of bright line patterns) are searched from the plurality of bright line images, the plurality of searched patterns are combined, and information is acquired from the combined plurality of patterns. Therefore, even when a plurality of subjects are moving, it is possible to easily distinguish and acquire information from the plurality of subjects.

In the information communication method, the identification information of the subject included in the information acquired in the information acquisition step and the frequency identification may be further provided to the server in which the frequency is registered for each of the identification information. A transmitting step of transmitting specific frequency information indicating the frequency specified in the step, the identification information, and a related information acquiring step of acquiring related information associated with the frequency indicated by the specific frequency information from the server. May be included.

Thus, as in the operation described with reference to FIG. 119, the identification information (ID) acquired based on the brightness change of the subject (transmitter) and the related information associated with the frequency of the brightness change are acquired. To be done. Therefore, by changing the frequency of the brightness change of the subject and updating the frequency registered in the server with the changed frequency, the receiver that has acquired the identification information before the frequency change acquires the related information from the server. Can be prevented. That is, the receiver that has acquired the identification information of the subject in the past can acquire the related information from the server indefinitely by changing the frequency registered in the server in accordance with the change in the frequency of the brightness change of the subject. It is possible to prevent the situation.

Further, the information communication method further includes: an identification information acquisition step of acquiring identification information of the subject by extracting a part from the information acquired in the information acquisition step; and an acquisition step in the information acquisition step. In addition, a setting frequency specifying step of specifying the number indicated by the remaining part other than the part of the information as the setting frequency of the brightness change set for the subject may be included.

Thus, as in the operation described with reference to FIG. 116, the information obtained from the patterns of the plurality of bright lines does not depend on the identification information of the subject and the set frequency of the luminance change set for the subject. Since it can be included, the degree of freedom between the identification information and the set frequency can be increased.

(Embodiment 6)
In the present embodiment, description will be given of each application example using the receiver such as the smartphone in the first to fifth embodiments and the transmitter that transmits information as a blinking pattern of the LED.

FIG. 159 is a diagram illustrating an example of a transmission signal of Embodiment 6.

The transmission signal D is divided into data pieces Dx of a predetermined size (for example, Dx=D1, D2, D3), and the frame check sequence FCS for error detection/correction calculated from each data piece and the header Hdr are used for each data piece. Added to. Further, a frame check sequence FCS2 for error detection/correction calculated from the original data and a header Hdr2 are added. The data composed of Hdr, Dx, and FCS is configured to be received by the image sensor. Since the image sensor is suitable for receiving continuous data in a short time, Hdr, Dx, and FCS are continuously transmitted. The data composed of Hdr2, Dx, and FCS2 is configured to be received by the illuminance sensor. It is desirable that Hdr and FCS received by the image sensor are short, but Hdr2 and FCS2 received by the illuminance sensor can be a longer signal train. By using a long signal sequence for Hdr2, header detection accuracy can be improved. By lengthening the FCS2, a code capable of detecting and correcting many bit errors can be adopted, and the error detection/correction performance can be improved. Note that Hdr2 and FCS2 may not be transmitted, and Hdr and FCS may be received by the illuminance sensor instead. The illuminance sensor may receive both Hdr and Hdr2, or both FCS and FCS2.

FIG. 160 is a diagram showing an example of a transmission signal of Embodiment 6.

The FCS2 has a long signal length, and if it is inserted frequently, the reception efficiency of the image sensor deteriorates. Therefore, the frequency of FCS2 insertion is reduced, and the signal PoFCS2 indicating the location of FCS2 is inserted instead. As an example, when 4PPM having a 2-bit information amount per unit time is used for signal expression, 16 units of transmission time is required when CRC32 is used for FCS2, but PoFCS2 with a range of 0 to 3 requires 1 unit time. You can send it at. Since the transmission time is shortened as compared with the case where only the FCS2 is inserted, the efficiency of image sensor reception can be improved. The illuminance sensor receives PoFCS2 following the transmission signal D, specifies the transmission time of FCS2 from PoFCS2, and receives FCS2. Further, PoFCS2 following FCS2 is received, the transmission time of the next FCS2 is specified, and the next FCS2 is received. If the FCS2 received first and the FCS2 received later are the same, the receiver estimates that it is receiving the same signal.

161A to 161C are diagrams illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6.

In the captured image shown in FIG. 161A, since the image is small, the number of bright lines is small. Therefore, only a small amount of data can be received from this captured image at one time. The captured image shown in FIG. 161B is an image captured using zoom and has a large number of bright lines because the image is large. Therefore, a large amount of data can be received at one time by capturing an image using the zoom. It can also receive data from a distance and can receive signals from small transmitters. Optical zoom or Ex zoom is used as the zoom method. Optical zoom is zooming by increasing the focal length of the lens. The Ex-zoom is a zoom method for enlarging a part of a captured image by capturing an image using only a part of the image sensor instead of the entire image sensor when capturing with a resolution lower than the capability of the image sensor. The captured image shown in FIG. 161C is an image captured using electronic zoom (enlargement of the image). Although the image becomes large, the bright lines become thicker when enlarged by the electronic zoom, and the number of bright lines does not change from before zooming, so the reception characteristics do not change from before zooming.

162A and 162B are diagrams illustrating an example of a captured image (bright line image) of a receiver in Embodiment 6.

The captured image shown in FIG. 162A is an image captured by focusing on a subject, and the captured image shown in FIG. 162B is an image captured without focus. In the captured image shown in FIG. 162B, the bright lines can be observed up to the actual surroundings of the transmitter because the image is blurred and more bright lines can be observed. Therefore, if the image is taken out of focus, a large amount of data can be received at one time, and the data can be received from a further distance. An image similar to the captured image shown in FIG. 162B can also be captured by capturing an image using the macro mode.

163A to 163C are diagrams showing an example of a captured image (bright line image) of the receiver in the sixth embodiment.

By setting the exposure time to be longer than that in the visible light communication mode and shorter than that in the normal imaging mode, an image as shown in FIG. 163A can be obtained. An imaging mode in which such an image is obtained is called a bright line detection mode (intermediate mode). In the image shown in FIG. 163A, the bright line of the transmitter can be observed in the left center, and a darker normal captured image appears in other portions. By displaying this image on the receiver, it is possible for the user to easily take the image of the receiver toward the intended transmitter. In the bright line detection mode, an image is captured darker than in the normal imaging mode. Therefore, by capturing in the high-sensitivity mode, it is possible to capture an image having a brightness close to that in the normal imaging mode and easily recognized by humans. If the sensitivity is set too high, the dark part of the bright line will become bright, so the sensitivity should be set so that the bright line can be observed. The receiver shifts to a visible light communication mode and receives a transmission signal of the transmitter imaged in a portion designated by a user by touching an image or the like. The receiver may automatically shift to the visible light communication mode and receive the signal when a bright line (transmission signal) is found in the captured image.

The receiver finds the transmission signal from the bright line in the captured image and emphasizes that portion as shown in FIG. 163B, so that the portion where the signal is transmitted can be presented to the user in an easy-to-understand manner. The bright line may be observed not only by the transmitted signal but also by the pattern of the subject. Therefore, instead of determining whether or not there is a transmission signal from the bright line of one image, it may be determined that there is a transmission signal when the positions of the bright lines change in a plurality of images.

Since the image captured in the bright line detection mode is darker and has lower visibility than the image captured in the normal image capturing mode, an image whose visibility is improved by image processing may be displayed. The image shown in FIG. 163C is an example of an image in which edges are extracted to emphasize the boundaries of the object to be imaged.

FIG. 164 is a diagram illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6. Specifically, FIG. 164 is a diagram in which a transmitter having a signal transmission cycle of 1/9600 seconds is imaged at the exposure time ratio shown in the lower part of the figure. When the exposure time is shorter than the transmission cycle of 1/9600 seconds, the captured images are almost the same, and clear bright lines can be captured. Although the outline of the bright line becomes blurred as the exposure time becomes longer, in the example of this signal expression, the bright line pattern can be observed and the signal can be received at the exposure time up to about 1.5 times the transmission cycle. Further, in this example of signal expression, the bright line can be observed if the exposure time is up to about 20 times the transmission cycle, and the exposure time in this range can be used as the exposure time in the bright line detection mode.

The exposure time up to which the signal can be received depends on the signal expression method. If the signal expression rule that the number of bright lines is small and the interval between the bright lines is long is used, the signal can be received even at a longer exposure time and the bright line can be observed at a longer exposure time, although the transmission efficiency is lowered.

(Exposure time in intermediate imaging mode)
From FIG. 164, a clear bright line can be observed when the exposure time is up to about 3 times the modulation cycle. Since the modulation frequency is 480 Hz or higher, the exposure time in the intermediate imaging mode (intermediate mode) is preferably 1/160 second or shorter.

Further, when the exposure time is 1/10000 second or less, it is difficult to see the non-luminous object even when imaged in the high sensitivity mode under illumination light. Therefore, the exposure time in the intermediate imaging mode is preferably 1/10000 second or more. However, this limitation can be eased by improving the sensitivity of the image sensor in the future.

FIG. 165 is a diagram illustrating an example of a transmission signal in Embodiment 6.

Since the receiver integrates multiple received data pieces and receives a series of signals, if the transmission signal is suddenly changed, the data pieces before and after the change will be mixed, and the signals cannot be integrated correctly. .. Therefore, as shown in (a) of FIG. 165, when the transmission signal is changed, the transmitter normally lights up for a predetermined time as a buffer band and does not transmit the signal. When the receiver cannot receive the signal for a predetermined time T2 shorter than the predetermined time T1, the receiver discards the data pieces received up to that time to avoid mixing of the data pieces before and after the change. can do. Alternatively, as shown in (b) of FIG. 165, the transmitter repeatedly transmits the signal X for notifying that when the transmission signal is changed. Repeated transmission prevents omission of reception of the transmission signal change notification X. Alternatively, as shown in (c) of FIG. 165, the transmitter repeatedly transmits the preamble when the transmission signal is changed. When the preamble is received in a cycle shorter than the cycle in which the preamble appears in the normal signal, the receiver discards the data pieces received so far.

FIG. 166 is a diagram illustrating an example of operation of a receiver in Embodiment 6.

The image shown in (a) of FIG. 166 is an image obtained by capturing an image of the transmitter with just focus. The receiver can capture an image as shown in (b) of FIG. 166 by removing the focus and capturing the image. When the focus is further removed, the captured image becomes the image shown in (c) of FIG. 166. In (c) of FIG. 166, the bright lines of a plurality of transmitters overlap and the receiver cannot receive a signal. Therefore, the receiver adjusts the focus so that the bright lines of the plurality of transmitters do not overlap with each other and captures an image. When there is only one transmitter in the imaging range, the receiver adjusts the focus so that the size of the transmitter is maximized in the captured image.

The receiver may compress the captured image in the direction parallel to the bright line, but does not compress the image in the direction perpendicular to the bright line. Alternatively, the receiver reduces the degree of compression in the vertical direction. As a result, it is possible to prevent a bright line from being blurred and a reception error from occurring due to compression.

167 and 168 are diagrams each illustrating an example of an instruction to the user which is displayed on the screen of the receiver in Embodiment 6.

The receiver can estimate the position of the receiver by triangulation from the position information of each transmitter and the position, size, and angle of each transmitter in the captured image by imaging the plurality of transmitters. Therefore, when only one transmitter is imaged in a receivable form, the user changes the direction of the receiver or moves backward to take an image of multiple transmitters. Displays an image including an arrow or the like to give an instruction on the imaging direction or the moving direction. 167(a) is a display example of an instruction to image the transmitter on the right side by pointing the receiver to the right, and FIG. 167(b) is an example of an instruction to image the transmitter on the front by descending backward. It is a display example. FIG. 168 is a display example of an instruction to image the other transmitter by shaking the receiver or the like because the position of the other transmitter is unknown to the receiver. Although it is preferable that a plurality of transmitters be photographed in one captured image, the positional relationship between the transmitters in the plurality of images may be estimated using image processing or the sensor value of the 9-axis sensor. The receiver may use the ID received from one transmitter, inquire the position information of the transmitters in the vicinity to the server, and may instruct the user to image the transmitter that is most easily imaged.

The receiver detects that the user is moving the receiver based on the sensor values of the 9-axis sensor, and after a predetermined time has elapsed after the movement ends, displays a screen based on the last received signal. As a result, when the user directs the receiver to the intended transmitter, the receiver receives signals from other transmitters while the receiver is moving, and performs processing based on the transmission signals from unintended transmitters. Can be prevented.

The receiver may continue the reception process even while being moved, and may perform a process based on the received signal, for example, information acquisition from the server using the received signal as a key. In this case, the reception process is continued after that process, and the process based on the last received signal is the final process.

The receiver may process the signal received a predetermined number of times or notify the user. The receiver may process the signal that was received the most times while being moved.

The receiver may include a notification unit that notifies the user when the signal is successfully received or when the presence of the signal is detected in the captured image. The notification means notifies by sound, vibration, display update (pop-up display or the like), or the like. This allows the user to know the existence of the transmitter.

FIG. 169 is a diagram illustrating an example of a signal transmission method in Embodiment 6.

For example, a plurality of transmitters configured as displays are arranged adjacent to each other. When transmitting the same signal, the plurality of transmitters synchronize the signal transmission timing and transmit the signal from the entire surface as shown in (a) of FIG. 169. This configuration allows the receiver to see the multiple displays as one large transmitter, allowing the receiver to receive signals faster and at greater distances. When the plurality of transmitters transmit different signals, the plurality of transmitters transmit signals by providing a buffer band (non-transmission area) in which no signals are transmitted, as shown in (b) of FIG. 169. With this configuration, the receiver can recognize that the plurality of transmitters are different from each other with the buffer band in between and can receive different signals.

FIG. 170 is a diagram illustrating an example of a signal transmission method in Embodiment 6.

As shown in (a) of FIG. 170, the liquid crystal display provides a backlight extinguishing period and changes the state of the liquid crystal while the backlight is extinguished, thereby making the image in the state changing invisible and dynamically resolving. It can enhance the feeling. As shown in FIG. 170(b), a signal is superimposed on the liquid crystal display performing the backlight control in accordance with the lighting cycle of the backlight. Receiving efficiency can be improved by continuously transmitting a set of data (Hdr, Data, FCS). Further, the light emitting portion is in a bright state (Hi) at the beginning and end of the backlight lighting period. This is because if the light emitting unit is in a dark state (Lo) continuously during the backlight extinguishing period, the receiver cannot determine whether Lo is transmitted as a signal or is in the dark state because of the backlight extinguishing period.

During the backlight extinguishing period, a signal having a low average brightness may be superimposed.

By superimposing the signals, the average luminance changes as compared with the case where the signals are not superposed. Therefore, the backlight lighting period is increased/decreased or the brightness when the backlight is turned on is increased/decreased so that the average brightness becomes equal.

FIG. 171 is a diagram illustrating an example of a signal transmission method in Embodiment 6.

The liquid crystal display can reduce the luminance change of the entire screen by performing the backlight control at different timings for each position. This is called a backlight scan. The backlight scan is usually performed so that the backlight is turned on sequentially from the end as shown in FIG. At this time, a captured image 8802a is obtained. However, in the captured image 8802a, the part where the bright line exists is divided, and it may not be possible to estimate that the entire screen of the display is one transmitter. Therefore, as shown in (b) of FIG. 171, when the vertical axis is displayed as the spatial axis in the division direction of the backlight scan and the horizontal axis is displayed as the time axis, all the portions that emit light (signals are superimposed) are displayed. A captured image 8802b can be obtained by setting the order of the backlight scan so that they are connected to each other. In the captured image 8802b, all the bright line portions are connected, and it can be easily estimated that the signal is a transmission signal from one transmitter. Moreover, since the number of bright lines that can be continuously received increases, a signal can be received quickly and from a distance. Moreover, since the size of the transmitter can be easily estimated, the position of the receiver can be accurately estimated from the position, size, and angle of the transmitter in the captured image.

FIG. 172 is a diagram illustrating an example of a signal transmission method in Embodiment 6.

In the time-division backlight scan, the backlight lighting period is short, and light is emitted (the signal is superimposed on a graph in which the vertical axis represents the space axis in the division direction of the backlight scan and the horizontal axis represents the time axis. ) When the portions cannot be connected, signals are superimposed on each light emitting portion in accordance with the light emission timing of the backlight as in the case of FIG. At this time, by controlling the backlight so as to maximize the distance from other backlight-lit parts on the graph, it is possible to prevent the bright lines of the adjacent parts from being mixed.

FIG. 173 is a diagram for explaining a use case in the sixth embodiment. The system according to the present embodiment includes a lighting fixture 100 that performs visible light communication, a wearable device 101 having a visible light communication function, a smartphone 102, and a server 103.

An object of the present embodiment is to save a series of troubles and shorten a shopping time when a user makes a shopping at a store by using visible light communication. Conventionally, when purchasing a product at a store, it was necessary to search the site of the store in advance to obtain coupon information. There is also a problem that it takes time to search for a product for which the coupon is targeted in the store.

As shown in FIG. 173, the lighting fixture 100 regularly transmits visible light communication information of its own in front of a store (assuming a home electronics mass retailer as an example). When the wearable device 101 of the user receives the illumination ID information, the wearable device 101 transmits the illumination ID information to the smartphone 102 using the short-range wireless communication. The smartphone 102 transmits user information and lighting ID information to the server 103 using a mobile line or the like. The smartphone 102 receives from the server 103 point information of a store in front of the user, coupon information, and the like. The user browses the information received from the server 103 with the wearable device 101 or the smartphone 102. The user can purchase the displayed product information of the store on the spot or can be guided to the exhibition place in the store. The details will be described below with reference to the drawings.

FIG. 174 is a diagram showing an information table transmitted from the smartphone 102 to the server 103. In addition to the member number of the store, the ID information of the store, the transmission time, and the position information stored in the smartphone 102, the taste information and biometric information of the user, the search history and the action history information stored in the smartphone 102 are stored in the smartphone 102. To send.

FIG. 175 is a block diagram of the server 103. The transmission/reception unit 201 receives the information transmitted from the smartphone 102. The control unit 202 performs overall control. The member information DB 203 stores the member number, the name and date of birth of the user of the member number, point information, purchase history, and the like. The store DB 204 stores store information such as store IDs, product information on sale at the store, store display information, and store map information. The notification information generation unit 205 generates coupon information and recommended product information according to the taste of the user.

FIG. 176 is a flowchart showing the overall processing of the above system. The wearable device 101 receives the lighting ID from the lighting fixture 100 (S301). Next, the wearable device 101 transmits the lighting ID to the smartphone 102 using close proximity wireless communication such as Bluetooth (registered trademark) (S302). Next, the smartphone 102 transmits the history information of the user owned by the smartphone 102, the member number, and the lighting ID shown in FIG. 174 to the server 103 (S303). When the server 103 receives the data, the data is first transmitted to the control unit 202 (S304). Next, the control unit 202 queries the member information DB 203 for the member number and acquires the member information (S305). Next, the control unit 202 queries the store DB 204 for the lighting ID and acquires store information (S306). The store information includes information on products in stock at the store, information on products that the store wants to promote sales, coupon information, in-store map information, and the like. The control unit 202 transmits the member information and the store information to the notification information generation unit (S307). The notification information generation unit 205 generates advertisement information suitable for the user from the member information and the store information, and transmits it to the control unit 202 (S308). The control unit 202 transmits the member information and the advertisement information to the transmission/reception unit 201 (S309). The member information includes user point information and expiration date information. The transmission/reception unit 201 transmits the member information and the advertisement information to the smartphone 102 (S310). The smartphone 102 displays the received information on the display screen (S311).

Furthermore, the smartphone 102 transfers the information received from the server 103 to the wearable device 101 (S312). If the notification setting of the wearable device 101 is ON, the wearable device 101 displays information (S314). When displaying information on the wearable device, it is desirable to alert the user by vibrating. This is because the user does not necessarily enter the store, and therefore the user may not notice even if the coupon information or the like is transmitted.

FIG. 177 is a diagram showing an information table transmitted from the server 103 to the smartphone 102. The store map DB is in-store guidance information such as which product is displayed at which position in the store. The product information of the store is information on products in stock at the store, price information of the products, and the like. The user member information includes user point information and membership card expiration date information.

FIG. 178 is a diagram illustrating a screen flow displayed on the wearable device 101 from when the user receives information from the server 103 in front of the store until when the user actually purchases the product. In front of the store, points given when the user visits the store and coupon information are displayed. When the user taps the coupon information, information according to the user's taste transmitted from the server 103 is displayed. For example, when the user taps on the TV, the recommended TV information is displayed. When the purchase button is pressed here, a receiving method selection screen is displayed, and it is possible to select home delivery or in-store pickup. In the present embodiment, since it is known which store the user is in, there is an advantage that the user can receive the store. When the guide to the sales floor is selected in the flow 403, the wearable device 101 shifts to the Guide Mode. This mode is a mode that guides the user to a specific place by using an arrow or the like, and can guide the user to a place where the selected product is actually displayed. When the wearable device 101 is guided to the front of the product shelf, the wearable device 101 transits to a screen for inquiring whether purchase is necessary. By actually looking at the corresponding product, the user can try the size, color, usability, etc. before making a purchase decision.

In the visible light communication according to the present invention, the position of the user can be accurately specified. Therefore, for example, as shown in FIG. 179, it is possible to give a warning when the user is likely to enter the dangerous area in the factory. Furthermore, since it is possible to determine on the wearable device side whether or not to issue a warning, it is possible to construct a warning system with a high degree of freedom, for example, to issue a warning to children under a specific age.

(Embodiment 7)
FIG. 180 is a diagram showing a service providing system using the receiving method described in the above-mentioned embodiment.

First, a company A ex8000 managing the server ex8002 is requested by another company B or an individual ex8001 to distribute information to a mobile terminal. For example, a request is made to deliver detailed advertisement information, coupon information, map information, or the like to a mobile terminal that has performed visible light communication with a signage. The company A ex8000 that manages the server manages information to be distributed to the mobile terminal in association with arbitrary ID information. The mobile terminal ex8003 acquires ID information from the subject ex8004 by visible light communication, and transmits the acquired ID information to the server ex8002. The server ex8002 transmits the information corresponding to the ID information to the mobile terminal and counts the number of times the information corresponding to the ID information is transmitted. The company A ex8000 that manages the server charges the company B and the individual ex8001 who requested the fee according to the counted number. For example, the larger the number of counts, the larger the charge amount.

FIG. 181 is a flowchart showing the flow of service provision.

In Step ex8000, the company A that manages the server receives a request for information distribution from another company B. Next, in Step ex8001, the information managed by the server managed by the company A is associated with the specific ID information. In Step ex8002, the mobile terminal receives specific ID information from the subject by visible light communication and transmits it to the server managed by the company A. Details of the visible light communication method have already been described in the other embodiments, and will be omitted. The server transmits information corresponding to the specific ID information transmitted from the mobile terminal to the mobile terminal. In Step ex8003, the server counts the number of times information has been distributed. Finally, in Step ex8004, the company B is charged with a fee according to the number of distributed information. In this way, by charging according to the count number, it becomes possible to charge the company B with an appropriate charge according to the advertising effect of information distribution.

FIG. 182 is a flowchart showing service provision in another example. A description of steps that are the same as those in FIG. 181 will be omitted.

In Step ex8008, it is determined whether or not a predetermined time has elapsed from the start of information distribution. If it is determined that it is within the predetermined time, in Step ex8011, the company B is not charged. On the other hand, if it is determined that the predetermined period has elapsed, in Step ex8009, the number of times information is distributed is counted. Then, in Step ex8010, the company B is charged with a fee corresponding to the information distribution count. As described above, since the information is distributed free of charge within the predetermined period, the company B can receive the billing service after confirming the advertising effect and the like.

FIG. 183 is a flowchart showing service provision in another example. A description of steps that overlap with FIG. 182 will be omitted.

In Step ex8014, the number of times information is distributed is counted. If it is determined in Step ex8015 that the predetermined period has not passed from the start of information distribution, the charging is not performed in Step ex8016. On the other hand, if it is determined that the predetermined period has elapsed, in step ex8017, it is determined whether or not the number of times information has been distributed is a predetermined value or more. If the number of times the information is delivered is less than the predetermined value, the count number is reset and the number of times the information is delivered is counted again. In this case, the company B is not charged for the predetermined period when the number of times the information is distributed is less than the predetermined value. In Step ex8017, if the count number is equal to or larger than the predetermined value, in Step ex8018 the count number is reset once, and the counting is restarted. In Step ex8019, the company B is charged with a fee according to the count number. In this way, when the number of counts in the free distribution period is small, the free distribution period is provided again, so that the company B can receive the charging service at an appropriate timing. Also, the company A may analyze the information content when the count number is small, and suggest to the company B to change the information content, for example, when the information does not correspond to the season. It will be possible. When the free information distribution period is provided again, the period may be shorter than the first predetermined period. By making the time shorter than the predetermined period of the first time, the burden on the company A can be reduced. Further, the free distribution period may be provided again after a certain period. For example, if the information is affected by the season, it is possible to set a certain period until the season changes and then set a free delivery period again.

Note that the charging fee may be changed according to the amount of data regardless of the number of times information is distributed. The distribution of a certain amount of data may be free, and the amount of data equal to or more than a predetermined amount may be charged. Further, as the amount of data increases, the charging fee may increase. A management fee may be charged when managing the information in association with the specific ID information. By charging as a management fee, the fee can be determined at the time of requesting information distribution.

(Embodiment 8)
In the present embodiment, each application example using the receiver such as the smartphone in each of the above embodiments and the transmitter that transmits information as the blinking pattern of the LED or the organic EL will be described.

(Modulation method that is easy to receive)
184A, 184B and 185 are diagrams showing an example of signal encoding in the eighth embodiment.

The transmission signal is composed of a header (H) and a body (Body). The header contains a unique signal pattern. The receiver finds this unique pattern in the received signal, recognizes which part of the received signal represents the header or the body based on its position, and receives the data.

When the transmission signal is modulated with the pattern of (a) of FIG. 184A, the receiver can receive the data when the header and the following body are continuously received. The length of time that the receiver can continuously receive a signal is determined by the size of the transmitter shown in a captured image (captured image). When the transmitter is small or when the transmitter is imaged from a long distance, the receiver has a short time in which signals can be continuously received. When the time when the receiver can continuously receive the signal (continuous reception time) is the same as the transmission time of one block including the header and body, the reception is started only when the transmission start time of the header is equal to the reception start time. The machine can receive data. FIG. 184A(a) shows a case where the continuous reception time is slightly longer than the transmission time of one block including the header and the body. The arrow line represents the continuous reception time.When the signal is received at the timing indicated by the thick line, data can be received, but when the signal is received at the timing indicated by the thin line, the Data cannot be received because the header and body are not aligned.

Therefore, by modulating the transmission signal with the pattern of (b) of FIG. 184A, it is possible to receive data at a wider reception timing. The transmitter transmits the modulated signal as a set of "body header body". Here, two bodies in the same set have the same signal. Even if the receiver does not receive all the signals contained in the body in succession, the body can be restored by connecting the parts of the body before and after the header, so all the signals contained in the header are If it can be received, the data can be received. In FIG. 184A, the reception timing at which data can be received is indicated by a thick line, and it can be seen that in the case of (b), data can be received at a wider reception timing than in the case of (a).

In the case of the modulation method of (b) in FIG. 184A, if the signal length of the body is a fixed value, the receiver can restore the body. Alternatively, if the header includes information about the signal length of the body, the receiver can restore the body.

That is, as shown in FIG. 184B, the receiver first finds a header composed of a unique bright line pattern from a captured image (bright line image) including bright lines. Then, the receiver sequentially reads each signal of the body following the header (direction (1) in FIG. 184B). At this time, each time the receiver reads a signal, it judges whether or not the signal of the body has been read, by the signal length of the body. That is, it is determined whether or not all the signals included in the body have been read. When it is determined that the read signal is not read, the receiver reads the signal that follows the read signal. However, if there is no subsequent signal, the receiver sequentially reads each signal of the body following the header (direction (2) in FIG. 184B). As a result, all the signals included in the body are read out. Here, if the signal length of the body is a fixed length, the receiver holds the signal length in advance and uses the signal length to make the above determination. Alternatively, the receiver identifies the signal length of the body from the header and uses the signal length to make the above determination.

In addition, even if the signal length of the body is variable, by setting the modulation method so that the body modulated by the same transmitter will have the same signal length, the receiver can determine the signal length between the two headers. The body can be restored by estimating the signal length of the body. At this time, in the modulation method of FIG. 184A (b), it is necessary to receive signals for two headers and two bodies at one time. However, with the modulation method shown in FIG. 185, the signal length of the body can be estimated by only receiving signals for two headers and one body. Note that FIG. 185 shows a modulation method in which a set of "body header body header 2 (H2)" is used, and the receiver can receive data if all the signals included in the header can be continuously received. is there.

As described above, the transmitter according to the present embodiment has the first luminance change pattern corresponding to the body that is a part of the signal to be transmitted and the second luminance change pattern indicating the header for specifying the body. A header and a body by determining a luminance change pattern, and changing the luminance in the order of a first luminance change pattern, a second luminance change pattern, and the first luminance change pattern. To send. The transmitter further determines a third brightness change pattern indicating another header different from the header, and the first brightness change pattern, the second brightness change pattern, and the first brightness change pattern. The header, the body, and the other header may be transmitted by changing the luminance in accordance with each pattern in the order of the third luminance change pattern.

(Combined use of bright line communication and image recognition)
FIG. 186 is a diagram showing an example of a captured image in Embodiment 8.

The receiver can read the signal from the bright line in the captured image and analyze the portion other than the bright line by image processing. Thereby, as an example, the receiver receives a signal from a transmitter configured as, for example, digital signage, and even if the same signal is received, it depends on the image displayed on the screen of the transmitter at that time. It becomes possible to display advertisements and the like.

Since the bright line becomes noise in the image processing, the image processing may be performed after interpolating the pixel value of the bright line portion from the pixels on the left and right of the bright line. Further, image processing may be performed on the image excluding the bright line portion.

(How to use an image sensor suitable for receiving visible light signals)
187A to 187C are diagrams each illustrating an example of a structure and operation of a receiver in Embodiment 8.

As shown in FIG. 187A, 8910a is an image sensor of the receiver. The image pickup element is composed of an effective pixel which is a portion for picking up an image, an optical black for measuring noise such as dark current, and an invalid area 8910b. Optical black further includes VOB for measuring noise in the vertical direction and HOB for measuring noise in the horizontal direction. Since the bright line is generated in the direction of 8910c (horizontal direction), the bright line is not obtained during the exposure of the VOB or the invalid area 8910b, and the signal cannot be received. Therefore, at the time of visible light communication, it is possible to increase the time during which a signal can be received by switching to an imaging mode in which the VOB and the invalid area 8910b are not used or are used minimally.

As shown in FIG. 187B, by not using the VOB and the invalid area 8910b, the exposure time in the effective pixel area, which is an area including the effective pixels, can be extended. Specifically, as shown in (a) of FIG. 187B, during normal shooting, one shot image is acquired between times t0 to t10, times t10 to t20, and times t20 to t30. Further, since the VOB and the invalid area 8910b are also used to acquire each captured image, the time during which the VOB is exposed in the effective pixel area (the time for reading out the charges, which is the shaded portion in FIG. 187B). ) Are times t3 to t10, times t13 to t20, and times t23 to t30.

On the other hand, in the visible light communication, by not using the VOB and the invalid area 8910b, as shown in (b) of FIG. 187B, the VOB and the invalid area 8910b are used in the effective pixel area only for the time. The exposure time can be extended. That is, it is possible to increase the receivable time in visible light communication. As a result, many signals can be received.

As shown in (a) of FIG. 187C, during normal shooting, the exposure of each exposure line in the effective pixel area is started after a predetermined time m has elapsed since the exposure of the adjacent exposure line was started. On the other hand, as shown in (b) of FIG. 187C, during visible light communication, when the exposure time in the effective pixel area is extended, the exposure line of each effective pixel area is exposed to the adjacent exposure line. Is started after a predetermined time n (n>m) has elapsed.

As described above, the receiver according to the present embodiment, when performing normal shooting, for each of the plurality of exposure lines in the area including the optical black of the image sensor, for the exposure line adjacent to the exposure line. The charge is read after a predetermined time has elapsed from the time when the charge was read. Then, when performing visible light communication, the receiver does not use the optical black for reading out electric charges, and for each of the plurality of exposure lines in the area other than the optical black in the image sensor, The charge is read out after a time longer than the predetermined time from the time when the charge is read out on the adjacent exposure line.

Further, during visible light communication, by setting an imaging mode in which the number of vertical pixels does not decrease due to processing such as demosaicing or clipping, it is possible to further increase the time during which signals can be received.

When the VOB or the invalid area 8910b is not used and the image is captured in the mode in which the vertical pixels are not reduced, the exposure timings of the lower end of the captured image and the upper end of the captured image of the next frame are continuous in time, and the signal is continuous. Can be received. Further, even if the VOB and the like cannot be completely disabled, the signal can be continuously received by modulating the transmission signal by a method capable of error correction.

In FIG. 187A, a horizontal bright line appears because the horizontal photodiodes are exposed at the same time. During visible light communication, this exposure mode and the exposure mode in which the photodiode in the vertical direction is exposed at the same time are alternately repeated, so that horizontal and vertical emission lines can be obtained. Even if there is a stable signal can be received.

(Continuous signal reception)
FIG. 187D is a diagram illustrating an example of a signal reception method in Embodiment 8.

The image pickup element includes an effective pixel which is a pixel for forming an image of the intensity of the received light and an invalid pixel which does not form the intensity of the received light as an image and is used as a reference of the intensity of dark current. Therefore, in the normal imaging mode, as shown in (a), there is a time during which only invalid pixels are receiving light, and signals cannot be received during that time. Therefore, in the visible light communication mode, reception can be performed by minimizing the time during which only invalid pixels receive light as shown in (b) or by setting so that valid pixels always receive as shown in (c). You can take a long time. In addition, this enables continuous reception. In the case of (b), there is a time when the signal cannot be received, but by using the error correction code for the transmission data, the entire signal can be estimated even when a part of the signal cannot be received.

(How to receive a signal from a transmitter that is imaged small)
FIG. 187E is a flowchart showing an example of a signal receiving method in Embodiment 8.

As shown in FIG. 187E, a start is performed in step 9000a, a signal is received in step 9000b, and a header is detected in step 9000c. In step 9000d, it is confirmed whether or not the data size of the body following the header is known. If YES, the process proceeds to step 9000f. If NO, the process proceeds to step 9000e, the data size of the body following the header is read from the header, and the process proceeds to step 9000f. In step 9000f, it is confirmed whether or not all the signals meaning the body have been received following the header. If YES, the process proceeds to step 9000g, the body portion is read from the signal received following the header, and the process ends in step 9000p. To do. If NO, the process proceeds to step 9000h. If the data lengths of the part received after the header and the part received before the header are combined, it is confirmed whether the data length of the body is sufficient. If YES, the process goes to step 9000i. Then, the body part is read by connecting the part received after the header and the part received before the header, and the process ends at step 9000p. If NO, the process proceeds to step 9000j to check whether there is a means for capturing many bright lines from the transmitter. If YES, the process proceeds to step 9000n to change to a setting capable of capturing many bright lines, and then step 9000b. Return to. In the case of NO, the process proceeds to step 9000k to notify that there is a transmitter but the imaged size is insufficient, and in step 9000m the direction of the transmitter is notified, and if it approaches there, reception is possible. This is notified, and the process ends at step 9000p.

With this method, it is possible to stably receive a signal even when there are few exposure lines passing through the transmitter in the captured image.

(Captured image size suitable for receiving visible light signals)
188 and 189A are diagrams showing an example of a reception method according to the eighth embodiment.

When the effective pixel area of the image sensor is 4:3, when the image is captured at 16:9, the upper and lower parts of the image are clipped. When the bright line appears in the horizontal direction, the bright line is lost by this clipping, and the time when the signal can be received is shortened. Similarly, when the effective pixel area of the image sensor is 16:9 and the image is captured at 4:3, the left and right parts of the image are clipped, and the time during which the signal can be received is shortened when the bright line appears in the vertical direction. Therefore, the aspect ratio at which clipping does not occur, that is, 4:3 in FIG. 188 and 16:9 in FIG. 189A is set as the imaging aspect ratio in the visible light communication mode. As a result, the receivable time can be extended.

That is, the receiver according to the present embodiment further sets the ratio of the vertical width and the horizontal width of the image obtained by the image sensor. Then, when performing visible light communication, the receiver determines whether or not the edge of the image in the direction perpendicular to the exposure line (bright line) is clipped according to the set ratio, and the edge is clipped. When it is determined that the ratio is set, the set ratio is changed to a non-clipping ratio at which the end is not clipped. Then, the image sensor of the receiver acquires the bright line image of the non-clipping ratio by photographing the subject whose brightness changes.

189B is a flowchart illustrating an example of a reception method in Embodiment 8.

In this receiving method, the receiving time is extended and the imaging aspect ratio for receiving a signal from a small transmitter is set.

That is, as shown in FIG. 189B, in step 8911Ba, start is performed, and in step 8911Bb, the imaging mode is changed to the visible light communication mode. In step 8911Bc, it is confirmed whether or not the aspect ratio setting of the captured image is the closest to the aspect ratio of the effective pixel. If YES, the process proceeds to step 8911Bd, and the aspect ratio setting of the captured image is set to the setting closest to the aspect ratio of the effective pixel. The process ends at step 8911Be. If NO, the process proceeds to step 8911Be and ends. By setting the aspect ratio in this way in the visible light communication mode, it is possible to reduce the time during which reception is not possible. It also allows you to receive signals from small and distant transmitters.

189C is a flowchart illustrating an example of a reception method in Embodiment 8.

In this receiving method, the imaging aspect ratio for increasing the number of samples per time is set.

That is, as shown in FIG. 189C, start is performed in step 8911Ca, and the imaging mode is changed to the visible light communication mode in step 8911Cb. In step 8911Cc, the bright line of the exposure line can be confirmed, but it is confirmed whether the number of samples per time is small and the signal cannot be received. If YES, the process advances to step 8911Cd to set the aspect ratio setting of the captured image to the most different setting from the aspect ratio of the effective pixel. In step 8911Ce, the imaging frame rate is increased, and the process returns to step 8911Cc. In the case of NO, the process proceeds to step 8911Cf to receive the signal and ends.

By setting the aspect ratio in this manner in the visible light communication mode, it is possible to receive a high frequency signal. In addition, it becomes possible to receive even in a noisy environment.

(Receiving visible light signal using zoom)
FIG. 190 is a diagram showing an example of a reception method in Embodiment 8.

The receiver finds an area where bright lines exist in the captured image 8913a and performs zooming so that as many bright lines as possible are generated. As in the captured image 8913b, the number of bright lines can be maximized by expanding the bright line region in the direction perpendicular to the direction of the bright lines until the bright line region reaches the upper and lower ends of the screen.

Further, the receiver may find an area in which the bright line is clearly displayed and perform zooming so that the area is enlarged as in 8913c.

If a plurality of bright line areas are present in the captured image, the above processing may be sequentially performed for each bright line area. Further, the above processing may be performed on the bright line area designated by the user from the captured image.

(Image data size compression method suitable for receiving visible light signals)
FIG. 191 is a diagram illustrating an example of a reception method in Embodiment 8.

If the image data size needs to be compressed when transmitting the captured image (a) from the image capturing unit to the image processing unit or when transmitting the captured image (a) from the image capturing terminal (receiver) to the server, as shown in (c). By performing reduction or pixel thinning in a direction horizontal to the bright line, the data size can be compressed without reducing the information amount of the bright line. When the reduction or pixel thinning-out as in (b) and (d) is performed, the number of bright lines is reduced or it becomes difficult to recognize the bright lines. Also in image compression, it is possible to prevent a reduction in reception efficiency by not compressing in the direction perpendicular to the bright line or by making the compression ratio in the vertical direction smaller than the compression ratio in the horizontal direction. The moving average filter may be applied to both the vertical direction and the horizontal direction, and is effective for both data size reduction and noise reduction.

That is, the receiver according to the present embodiment further generates a compressed image by compressing the bright line image in a direction parallel to each of the multiple bright lines included in the bright line image, and transmits the compressed image. ..

(Modulation method with high reception error detection accuracy)
FIG. 192 is a diagram illustrating an example of a signal modulation method in Embodiment 8.

Since the error detection by the parity bit detects the reception error of 1 bit, the mistake of "01" and "10" and the mistake of "00" and "11" cannot be detected. In the modulation method of (a), “01” and “10” are different from each other only in the position of L, so it is easy to confuse them, but in the modulation method of (b), “01”, “10”, and “00” The positions of L of "11" are different from each other. By using the modulation method of (b), the reception error can be detected with high accuracy. This also applies to the modulation schemes in FIGS. 76 to 78.

That is, in the present embodiment, two luminance change patterns in which the timings at which a predetermined luminance value (for example, L) appears are assigned to the same parity signal unit (for example, “01” and “10”). As described above, the brightness change patterns whose timings are different from each other are assigned in advance to the different signal units. Then, the transmitter according to the present embodiment determines, for each of the signal units included in the signal to be transmitted, a luminance change pattern assigned to the signal unit.

(Changes in receiver operation due to different situations)
193 is a diagram illustrating an example of operation of a receiver in Embodiment 8. [FIG.

The receiver 8920a operates differently depending on the situation in which reception is started. For example, when activated in Japan, it receives a signal modulated by the phase shift keying method of 60 kHz, and downloads data from the server 8920d using the received ID as a key. When activated in the United States, it receives a signal modulated by a frequency shift keying method of 50 kHz, and downloads data from the server 8920e using the received ID as a key. In the situation where the operation of the receiver is changed, the place (country or building) where the receiver 8920a exists, the base station or the wireless access point (Wi-Fi, Bluetooth (registered trademark)) communicating with the receiver 8920a are included. ), IMES, etc.), time, etc.

For example, the receiver 8920a receives the position information and the last accessed wireless base station (carrier communication network or base station such as Wi-Fi, Bluetooth (registered trademark) or IMES) or the last received ID by visible light communication, Send to server 8920f. The server 8920f estimates the position of the receiver 8920a based on the received information, and receives a reception algorithm that can receive the transmission signal of the transmitter near that position and the ID management that manages the ID of the transmitter near that position. Send server information (URI, etc.). The receiver 8920a receives the signals of the transmitters 8920b and 8920c using the received algorithm, and makes an inquiry to the received ID management servers 8920d and 8920e using the ID as a key.

By this method, different types of communication can be performed in units such as countries, regions, and buildings. Further, the receiver 8920a according to the present embodiment, when receiving a signal, switches the server to be accessed, switches the reception algorithm, etc. according to the frequency used for the modulation of the signal, as shown in FIG. The signal modulation method may be switched.

(Awareness of visible light communication to humans)
194 is a diagram illustrating an example of operation of a transmitter in Embodiment 8. [FIG.

The light emitting unit of the transmitter 8921a repeats blinking and visible light communication visible to humans, as shown in (a) of FIG. 194. By blinking visible to humans, it is possible to inform humans that visible light communication is possible. The user notices that visible light communication is possible because the transmitter 8921a is blinking, performs visible light communication with the receiver 8921b toward the transmitter 8921a, and performs user registration for the transmitter 8921a.

That is, the transmitter in this embodiment alternately repeats a step in which the light emitter transmits a signal due to a change in luminance and a step in which the light emitter blinks so that it can be visually recognized by human eyes.

The transmitter may be separately provided with a visible light communication unit and a blinking unit (communication status display unit) as shown in FIG. 194(b).

The transmitter operates as shown in (c) of FIG. 194 by using the modulation method of FIG. 77 or 78, so that visible light communication is performed while the light emitting unit is blinking to humans. I can show you. That is, the transmitter alternately repeats high-luminance visible light communication with a brightness of 75% and low-luminance visible light communication with a brightness of 1%, for example. For example, by performing the operation shown in (c) of FIG. 194 when an abnormal condition occurs in the transmitter and a signal different from usual is being transmitted, the user can be alerted without stopping visible light communication. You can

(Expansion of reception range by the diffuser plate)
195 is a diagram illustrating an example of a receiver in Embodiment 8. [FIG.

195(a) shows the state of the receiver 8922a in the normal mode, and FIG. 195(b) shows the state of the receiver 8922a in the visible light communication mode. The receiver 8922a includes a diffuser plate 8922b in front of the image pickup unit. The receiver 8922a moves the diffuser plate 8922b in front of the imaging unit in the visible light communication mode so that the light source spreads and an image is captured. At this time, the position of the diffuser plate 8922b is adjusted so that the lights from the plurality of light sources do not overlap. A macro lens or a zoom lens may be used instead of the diffuser plate 8922b. This allows signals from distant transmitters and small transmitters to be received.

Note that instead of moving the diffuser plate 8922b, the imaging direction of the imaging unit may be moved.

The area of the image sensor in which the diffuser plate 8922b is reflected may not be used in the normal imaging mode, but may be used only in the visible light communication mode. As a result, the above-described effect can be obtained without moving the diffuser plate 8922b or the image pickup unit.

(Method of synchronizing signal transmission from multiple transmitters)
196 and 197 are diagrams showing an example of the transmission system in the eighth embodiment.

When using a plurality of projectors for projection mapping or the like, in order to avoid interference, it is necessary for only one projector to transmit a signal or to synchronize the signal transmission timings of a plurality of projectors for one projection. FIG. 196 shows a mechanism for synchronization of transmission timing.

The projector A and the projector B that perform projection on the same projection plane transmit signals as shown in FIG. The receiver images and receives the image, calculates the time difference between the signal a and the signal b, and adjusts the signal transmission timing of each projector.

Since the projector A and the projector B are not synchronized at the start of the operation, by providing a time during which neither signal is transmitted (total pause time), it is possible to prevent the signal a and the signal b from overlapping and becoming unreceivable. The signal transmitted by the projector may be changed as the timing adjustment of the projector progresses. For example, the total pause time is set long at the start of the operation, and the total pause time is shortened as the timing adjustment progresses, whereby the timing adjustment can be efficiently performed.

In order to adjust the timing accurately, it is desirable that the signal a and the signal b are included in one captured image. The image pickup frame rate of the receiver is often 60 fps to 7.5 fps. The signal a and the signal b can be included in the image picked up at 7.5 fps by setting the signal transmission period to be 1/7.5 second or less. Further, by setting the signal transmission cycle to 1/60 second or less, it is possible to reliably store the signals a and b in the image captured at 30 fps.

FIG. 197 is a diagram illustrating a case where a transmitter including a plurality of displays is synchronized. The synchronized display is imaged so that it fits in one image, and the timing is adjusted.

(Reception of visible light signal by illuminance sensor and image sensor)
FIG. 198 is a diagram illustrating an example of operation of a receiver in Embodiment 8.

Since the image sensor consumes more power than the illuminance sensor, the receiver 8940a activates the image sensor 8940b and receives the signal when the illuminance sensor 8940c detects the signal. As shown in (a) of FIG. 198, the receiver 8940a receives the signal transmitted by the transmitter 8940d at the illuminance sensor 8940c. After that, the receiver 8940a activates the image sensor 8940b, receives the transmission signal of the transmitter 8940d by the image sensor, and recognizes the position of the transmitter 8940d. At this time, if a part of the signal is received by the image sensor 8940b and the part is the same as the signal received by the illuminance sensor 8940c, the receiver 8940a tentatively determines that the same signal is received, and the latter stage Processing such as displaying the current position is performed. The judgment is completed when all the signals can be received by the image sensor 8940b.

At the time of provisional judgment, it may be displayed that the judgment is not completed. For example, the display of the current position is made semitransparent, or the position error is displayed.

The part of the signal can be defined as, for example, 20% of the entire signal length or an error detection code part.

In the situation shown in (b) of FIG. 198, the illuminance sensor 8940c interferes with the signal and cannot be received, but it can be recognized that the signal exists. For example, when a peak appears in the modulation frequency of the transmission signal when the sensor value of the illuminance sensor 8940c is Fourier transformed, it can be estimated that the signal exists. When the receiver 8940a estimates that a signal exists from the sensor value of the illuminance sensor 8940c, it activates the image sensor 8940b and receives signals from the transmitters 8940e and 8940f.

(Reception start trigger)
FIG. 199 is a diagram illustrating an example of operation of a receiver in Embodiment 8.

Power is consumed when the image sensor and illuminance sensor (hereinafter collectively referred to as the light receiving sensor) are activated. Therefore, stop power consumption when not needed and activate it when necessary to improve power consumption efficiency. Can be improved. Since the power consumption of the illuminance sensor is lower than that of the image sensor, the illuminance sensor may always be activated and only the image sensor may be controlled below.

In (a) of FIG. 199, the receiver 8941a detects movement from the sensor value of the 9-axis sensor, activates the light receiving sensor, and starts reception.

In (b) of FIG. 199, the operation of tilting the receiver is detected from the sensor value of the 9-axis sensor, the light receiving sensor on the side facing upward is activated, and reception is started.

In (c) of FIG. 199, the operation of sticking out the receiver is detected from the sensor values of the 9-axis sensor, and the light receiving sensor in the sticking out direction is activated to start reception.

In (d) of FIG. 199, from the sensor value of the 9-axis sensor, the operation of turning the receiver up and the operation of shaking are detected, and the light receiving sensor on the upside is activated to start reception.

That is, the receiver according to the present embodiment further determines whether or not the receiver (reception device) has been moved in a predetermined manner, and when it is determined that it has been moved in the predetermined manner, the image Activate the sensor.

(Reception start gesture)
FIG. 200 is a diagram showing an example of a gesture operation for starting reception by this communication method.

For example, the receiver 8942a configured as a smartphone detects an operation of standing the receiver and sliding it in the lateral direction or an operation of repeatedly sliding in the lateral direction from the sensor value of the 9-axis sensor. After that, the receiver 8942a starts the reception and acquires the position of the transmitter 8942b based on the received ID. After that, the receiver 8942a acquires its own position from the relative positional relationship between itself and the plurality of transmitters 8942b. The receiver 8942a can stably image the plurality of transmitters 8942b by sliding, and can estimate the self-position with high accuracy by the triangulation method.

Note that this operation may be performed only when the home screen is in the foreground. As a result, it is possible to prevent the main communication from being performed against the user's intention while using another application.

(Application example to car navigation)
201 and 202 are diagrams each showing an example of application of the transmitting and receiving system in Embodiment 8.

For example, the transmitter 8950b configured as a car navigation transmits Bluetooth (registered trademark) pairing information, Wi-Fi SSID and password, IP address, and other information for wireless connection to the transmitter 8950b. For example, the receiver 8950a configured as a smartphone establishes a wireless connection with the transmitter 8950b based on the received information, and subsequent communication is performed via this wireless connection.

For example, the user inputs destination information, store information to be searched, or the like to the smartphone 8950a, the smartphone 8950a transmits the input information to the car navigation 8950b via a wireless connection, and the car navigation 8950b displays route information. In addition, for example, the smartphone 8950a operates as a controller of the car navigation system 8950b, and controls music and images reproduced by the car navigation system 8950b. In addition, for example, music and video held by the smartphone 8950a are reproduced by the car navigation system 8950b. In addition, for example, the car navigation system 8950b acquires information on surrounding stores and congestion information on roads and causes the smartphone 8950a to display the information. In addition, for example, the smartphone 8950a uses the microphone and speaker of the wirelessly connected car navigation system 8950b to perform a call process when an incoming call is received. Note that the smartphone 8950a may perform the above operation by establishing a wireless connection when an incoming call is received.

The car navigation system 8950b automatically connects wirelessly to the registered terminal when the wireless connection automatic connection mode is set. When the car navigation system 8950b is not in the automatic connection mode, the car navigation system 8950b transmits connection information using visible light communication and waits for connection. In addition, the car navigation system 8950b may transmit connection information using visible light communication and wait for connection even in the automatic connection mode. When the car navigation system is manually connected, the car navigation system may be released from the automatic connection mode, or may be disconnected from the automatically connected terminal.

(Application example to content protection)
FIG. 203 is a diagram illustrating an example of application of the transmitting and receiving system in Embodiment 8.

For example, the transmitter 8951b configured as a television transmits the content protection information stored in itself or the connected device 8951c. For example, the receiver 8951a configured as a smartphone receives the content protection information, and for a predetermined time after that, the content protection information can be reproduced so that the content protected by the content protection information of the transmitter 8951b or the device 8951c can be reproduced. I do. As a result, the content held in another device owned by the user can be reproduced by the receiver.

The transmitter 8951b may store the content protection information in the server, and the receiver 8951a may acquire the content protection information from the server by using the received ID of the transmitter 8951b as a key.

The receiver 8951a may further transmit the acquired content protection information to another device.

(Application example as electronic lock)
FIG. 204A is a diagram illustrating an example of an application example of the transmitting and receiving system in Embodiment 8.

The receiver 8952a receives the ID transmitted by the transmitter 8952b and transmits it to the server 8952c. When the ID of the transmitter 8952b is sent from the receiver 8952a, the server 8952c unlocks the door 8952d, opens the automatic door, and moves the elevator that moves to the floor registered in the receiver 8952a to the receiver. Call to the floor where 8952a is. Accordingly, the receiver 8952a serves as a key, and the user can unlock the door before reaching the door 8952d.

That is, the receiver according to the present embodiment acquires a first bright line image that is an image including a plurality of bright lines by photographing a subject whose brightness changes (for example, the above-described transmitter), and acquires the acquired first bright line image. The first transmission information (for example, the ID of the subject) is acquired by demodulating the data specified by the patterns of the plurality of bright lines included in the bright line image. Then, the receiver causes the door opening/closing drive device to open the door by transmitting a control signal (for example, the ID of the subject) after the first transmission information is acquired.

In order to prevent malicious operation, the server 8952c may use security protection such as a secure element of the receiver 8952a to confirm that the communication partner is the receiver 8952a. Also, in order to confirm that the receiver 8952a is indeed near the transmitter 8952b, the server 8952c issues a command to send a different signal to the transmitter 8952b when it receives the ID of the transmitter 8952b, The unlocking may be performed when the signal is sent from the receiver 8952a.

In the case where a plurality of transmitters 8952b configured as lighting devices are arranged along the path toward the door 8952d, the receiver 8952a receives the IDs from the transmitters 8952b and receives the receivers. It can be determined whether the 8952a is approaching the door 8952d. For example, if the values indicated by these IDs decrease in the order in which those IDs were acquired, the receiver determines that they are approaching the door. Alternatively, the receiver identifies the position of each transmitter 8952b based on those IDs, and further based on the positions of those transmitters 8952b and the positions of those transmitters 8952b shown in the captured image. , Estimate your position. Then, the receiver determines whether or not the door 8952d is approaching by comparing the position of the door 8952d, which is held in advance, with the estimated position of itself. Then, when it is determined that the receiver is approaching the door 8952d, the receiver transmits any of the acquired IDs to the server 8952c. As a result, the server 8952c performs processing such as opening the door 8952d.

That is, the receiver according to the present exemplary embodiment acquires the second bright line image that is an image including a plurality of bright lines by capturing another subject whose brightness changes, and includes the second bright line image in the acquired second bright line image. The second transmission information (for example, the ID of another subject) is obtained by demodulating the data specified by the patterns of the plurality of bright lines. Then, the receiver determines whether or not the receiver is approaching the door based on the acquired first and second transmission information, and when determining that the receiver is approaching the door, a control signal (for example, , ID of any subject) is transmitted.

FIG. 204B is a flowchart of the information communication method in this embodiment.

The information communication method according to the present embodiment is an information communication method for acquiring information from a subject, and includes steps SK21 to SK24.

That is, according to this information communication method, in the image obtained by photographing the first subject, which is the subject, by the image sensor, a plurality of bright lines corresponding to each exposure line included in the image sensor has a brightness of the first subject. A first exposure time setting step SK21 for setting a first exposure time of the image sensor so as to occur in response to a change, and the image sensor sets the first subject whose brightness changes to the set first exposure time. A first bright line image acquisition step SK22 for acquiring a first bright line image which is an image including the plurality of bright lines by shooting at an exposure time of 1; and the first bright line image included in the acquired first bright line image. A first information acquisition step SK23 for acquiring first transmission information by demodulating data specified by a plurality of bright line patterns, and transmitting a control signal after the first transmission information is acquired. A door control step SK24 for opening the door to the door opening/closing drive device.

FIG. 204C is a block diagram of the information communication device according to the present embodiment.

The information communication device K20 in the present embodiment is an information communication device that transmits a signal according to a change in luminance, and includes components K21 to K24.

That is, the information communication device K20 is configured so that an image obtained by photographing the subject with the image sensor has a plurality of bright lines corresponding to the exposure lines included in the image sensor according to the change in the luminance of the subject. An exposure time setting unit K21 that sets an exposure time of the image sensor, and a bright line image that is an image including the plurality of bright lines is acquired by photographing the subject that changes in brightness at the set exposure time. A bright line image acquisition unit K22 having an image sensor, an information acquisition unit K23 that acquires transmission information by demodulating data specified by the patterns of the plurality of bright lines included in the acquired bright line image, and the transmission information And a door control unit K24 for opening the door to the door opening/closing drive device by transmitting a control signal.

In the information communication method and the information communication device K20 shown in FIGS. 204B and 204C as described above, for example, as shown in FIG. 204A, a receiver equipped with an image sensor can be used like a door key, and a special receiver can be used. The electronic lock can be eliminated. As a result, it is possible to perform communication between various devices including devices with low computing power.

In addition, in each of the above-described embodiments, each component may be configured by dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory. For example, the program causes a computer to execute the information communication method shown by the flowchart in FIG. 204B.

(Application example for transmitting store visit information)
FIG. 205 is a diagram illustrating an example of an application example of the transmitting and receiving system in Embodiment 8.

The receiver 8953a transmits the ID transmitted by the transmitter 8953b to the server 8953c. The server 8953c notifies the store clerk 8953d of the order information associated with the receiver 8953a. The store clerk 8953d prepares a product or the like based on the order information. As a result, an order is prepared when the user enters the store or the like, so that the user can quickly receive the product or the like.

(Application example of order control according to location)
FIG. 206 is a diagram illustrating an example of an application example of the transmitting and receiving system in Embodiment 8.

The receiver 8954a displays a screen where an order can be placed only when the transmission signal of the transmitter 8954b is received. This allows the store to avoid orders from customers who are not nearby.

Alternatively, the receiver 8954a places the order by adding the ID of the transmitter 8954b to the order information. Thereby, the store can grasp the position of the orderer and the position to deliver the product. Alternatively, the store can estimate the time until the orderer arrives. The receiver 8954a may add the travel time to the store calculated from the travel speed to the order information. Further, the receiver may not accept the purchase which is unnatural judging from the current position (for example, the purchase of a boarding pass departing from a station other than the current position).

(Application to directions)
FIG. 207 is a diagram illustrating an example of an application example of the transmitting and receiving system in Embodiment 8.

The receiver 8955a receives, for example, the transmission ID of the transmitter 8955b configured as a guide board, acquires the data of the map displayed on the guide board from the server, and displays it. At this time, the server may send an advertisement suitable for the user of the receiver 8955a, and the receiver 8955a may also display this advertisement information. The receiver 8955a displays the route from the current location to the location designated by the user.

(Application example for contact information)
FIG. 208 is a diagram illustrating an example of an application example of the transmitting and receiving system in Embodiment 8.

The receiver 8956a receives the ID transmitted from the transmitter 8956b configured as, for example, home or school lighting, and transmits the position information acquired using the ID as a key to the terminal 8956c. Thereby, the guardian having the terminal 8956c can know that the child having the receiver 8956a has returned home or has arrived at the school. Also, the supervisor having the terminal 8956c can grasp the current position of the worker having the receiver 8956a.

(Application example for use log storage and analysis)
FIG. 209 is a diagram illustrating an example of an application example of the transmitting and receiving system in Embodiment 8.

The receiver 8957a receives the ID transmitted by the transmitter 8957b configured as a signboard, acquires the coupon information from the server, and displays the coupon information. The receiver 8957a is a server that performs subsequent user actions such as saving a coupon, moving to a store displayed on the coupon, shopping at the store, and leaving without saving the coupon. Save to 8957c. Thereby, the subsequent action of the user who has acquired the information from the signboard 8957b can be analyzed, and the advertising value of the signboard 8957b can be estimated.

(Application example to screen sharing)
210 and 211 are diagrams showing an example of an application example of the transmission/reception system in the eighth embodiment.

For example, the transmitter 8960b configured as a projector or a display transmits information (SSID, wireless connection password, IP address, password for operating the transmitter) for wireless connection to itself. Alternatively, an ID that is a key for accessing these pieces of information is transmitted. For example, the receiver 8960a configured as a smartphone, a tablet, a laptop computer, or a camera receives the signal transmitted from the transmitter 8960b, acquires the information, and establishes a wireless connection with the transmitter 8960b. This wireless connection may be made via a router, or may be made directly by Wi-Fi Direct, Bluetooth (registered trademark), Wireless Home Digital Interface, or the like. The receiver 8960a transmits the screen displayed by the transmitter 8960b. This allows the image of the receiver to be easily displayed on the transmitter.

When the transmitter 8960b is connected to the receiver 8960a, the transmitter 8960b informs the receiver 8960a that a password is required in addition to the information transmitted by the transmitter to display the screen, and the correct password is displayed. If it is not sent, the sent screen may not be displayed. At this time, the receiver 8960a displays a password input screen such as 8960d and prompts the user to input the password.

FIG. 211 is a diagram showing an example in which the screen of the transmitter 8961c is displayed on the transmitter 8960b via the receiver 8960a. For example, the transmitter 8961c configured as a notebook computer transmits information for connecting to itself or an ID associated with the information. The receiver 8960a receives the signal transmitted from the transmitter 8960b and the signal transmitted from the transmitter 8961c, establishes a connection with each transmitter, and displays an image to be displayed on the transmitter 8960b from the transmitter 8961c. To transmit. The transmitter 8960b and the transmitter 8961c may communicate directly, or may communicate via the receiver 8960a or a router. Accordingly, even when the transmitter 8961c cannot receive the signal transmitted by the transmitter 8960b, the image of the transmitter 8961c can be easily displayed on the transmitter 8960b.

Even if the receiver 8960a receives the signal transmitted by the transmitter 8960b and the time when the signal transmitted by the transmitter 8961c is received is equal to or less than a predetermined time, the above operation may be performed. Good.

The transmitter 8961c may transmit the image to the transmitter 8960b only when the correct password is received from the receiver 8960a.

(Application example of position estimation using wireless access point)
FIG. 212 is a diagram illustrating an example of application of the transmitting and receiving system in Embodiment 8.

For example, the receiver 8963a configured as a smartphone receives the ID transmitted by the transmitter 8963b. The receiver 8963a acquires the position information of the transmitter 8963b using the received ID as a key, and estimates its own position from the position and direction of the transmitter 8963b in the captured image. Further, the receiver 8963a receives a signal from a wireless transmitter 8963c configured as a Wi-Fi access point, for example. The receiver 8963a estimates its own position from the position information of the wireless transmitter 8963c and the radio wave transmission direction information included in the signal. The receiver 8963a can estimate the self-position with high accuracy by estimating the self-position by a plurality of means.

A method of estimating the self position by using the information of the wireless transmitter 8963c will be described. The wireless transmitter 8963c transmits synchronized signals from different antennas in different directions. Further, the wireless transmitter 8963c sequentially changes the signal transmission direction. The receiver 8963a estimates that the radio wave transmission direction when the radio wave intensity is the strongest is the direction from the radio transmitter 8963c to itself. In addition, the route difference is calculated from the difference in the arrival time of the radio waves transmitted from the different antennas and passed through the routes 8963d, 8963e, 8963f, respectively, and the radio transmitter 8963c and the wireless transmitter 8963c and themselves are calculated from the radio wave transmission angle differences θ12, θ13, θ23. Calculate the distance of. Furthermore, the self-position can be estimated with higher accuracy by using the surrounding electric field information and the radio wave reflection object information.

(Position estimation by visible light communication and wireless communication)
FIG. 213 is a diagram showing a configuration for performing position estimation by visible light communication and wireless communication. That is, FIG. 213 shows a configuration in which the position estimation of the terminal is performed using visible light communication and wireless communication.

The mobile terminal (smartphone terminal) acquires the ID of the light emitting unit by performing visible light communication with the light emitting unit. The server obtains the position information of the light emitting unit by inquiring the server of the acquired ID. As a result, the mobile terminal acquires the actual distance L1 and the actual distance L2, which are the distances between the MIMO (multiple-input and multiple-output) access point and the light emitting unit in the x direction and the y direction. Further, as already described in the other embodiments, the mobile terminal detects the inclination θ1 of the mobile terminal using a gyro sensor or the like.

On the other hand, when beamforming is performed from the MIMO access point toward the mobile terminal, the beamforming angle θ2 is set by the MIMO access point and has a known value. Therefore, the mobile terminal acquires the beamforming angle θ2 by wireless communication or the like.

As a result, the mobile terminal uses the actual distance L1 and the actual distance L2, the inclination θ1 of the mobile terminal, and the beamforming angle θ2 to determine the coordinate position (x1, y1) of the mobile terminal with respect to the MIMO access point. It can be calculated. Since the MIMO access point can form a plurality of beams, it is possible to perform more accurate position estimation by using a plurality of beam formings.

As described above, according to the present embodiment, it is possible to further improve the accuracy of position estimation by using both the position estimation by visible light communication and the position estimation by wireless communication.

The information communication method according to one or more aspects has been described above based on the embodiment, but the present invention is not limited to this embodiment. As long as it does not depart from the gist of the present invention, various modifications that a person skilled in the art can think of in the present embodiment, or a form constructed by combining components in different embodiments are also included in the scope of one or more aspects. May be included within.

Further, as shown in FIGS. 214, 215, and 216, the information communication method according to one embodiment of the present invention may be applied.

FIG. 214 is a diagram illustrating an example of an application example of the transmitting and receiving system in Embodiment 8.

The camera configured as a receiver for visible light communication performs image capturing in the normal image capturing mode (Step 1). By this imaging, the camera acquires an image file configured in a format such as EXIF (Exchangeable image file format). Next, the camera performs imaging in the visible light communication imaging mode (Step 2). The camera acquires a signal (visible light communication information) transmitted by visible light communication from a transmitter, which is a subject, based on the pattern of bright lines in the image obtained by this imaging (Step 3). Further, the camera treats the signal (reception information) as a key and accesses the server to obtain the information corresponding to the key from the server (Step 4). Then, the camera transmits a signal (visible light reception data) transmitted from the subject through visible light communication, information obtained from the server, and data indicating a position where the transmitter as the subject is shown in the image indicated by the image file. And data indicating the time (time in the moving image) at which the signal transmitted by the visible light communication is received are stored as the metadata in the image file. Note that, when a plurality of transmitters are displayed as subjects in an image (image file) obtained by image pickup, the camera provides some metadata corresponding to each transmitter for each transmitter. Save to file.

A display or projector configured as a transmitter of visible light communication transmits a signal according to metadata included in the image file by visible light communication when displaying an image shown by the image file. For example, the display or projector may send the metadata itself by visible light communication, or it may send a signal associated with the imaged transmitter as a key.

A mobile terminal (smartphone) configured as a receiver of visible light communication receives a signal transmitted from the display or projector by visible light communication by capturing an image of the display or projector. If the received signal is the above-mentioned key, the mobile terminal uses the key to obtain the metadata of the transmitter associated with the key from the display, the projector, or the server. If the received signal is a signal (visible light reception data or visible light communication information) transmitted from an existing transmitter by visible light communication, the portable terminal sends the visible light from a display, a projector or a server. Acquire information corresponding to received light data or visible light communication information.

FIG. 215 is a flowchart showing the operation of the camera (receiver) of the transmission/reception system according to the eighth embodiment.

First, when the camera detects that the image capturing button is pressed (step S901), the camera captures an image in the normal image capturing mode (step S902). Then, the camera performs imaging in the visible light imaging mode by increasing the shutter speed above a certain speed, that is, by setting the exposure time shorter than that in the normal imaging mode (step S903). As a result, the camera acquires the signal transmitted from the subject by visible light communication.

After that, the camera handles the signal (information) obtained by the visible light communication as a key to obtain the information associated with the key from the server (step S905). Next, the camera saves the signal, each piece of information and each piece of data in a metadata area (an area where EXIF meter data is stored, etc.) of an image file obtained by image pickup in the normal image pickup mode (step S905). .. That is, the camera has a signal acquired by visible light communication, information acquired from the server, and an image of a transmitter that is a subject that has transmitted the signal by visible light communication (an image acquired by imaging in the normal imaging mode). Saves location data, etc., indicating the location inside.

Then, the camera determines whether or not to capture a moving image (step S906), and when determining to capture a moving image (Y in step S906), repeats the processing from step S902. On the other hand, when the camera determines not to capture a moving image (N in step S906), the image capturing process ends.

FIG. 216 is a flowchart showing the operation of the display (transmitter) of the transmission/reception system in the eighth embodiment.

First, the display checks the metadata area of the image file to determine whether there is one transmitter or a plurality of transmitters shown in the image indicated by the image file (step S911). Here, when the display determines that there are a plurality of transmitters (a plurality of transmitters in step S911), the display further determines whether the split transmission mode is set as the visible light communication mode (step S912). Then, when the display determines that the divided transmission mode is set (Y in step S912), the display area (transmission portion) of the display is divided, and a signal is transmitted from each display area (step S914). Specifically, the display treats, for each transmitter, an area in which the transmitter is displayed or an area in which the transmitter and its surroundings are displayed as a display area, and the transmission area is transmitted from the display area. The signal corresponding to the machine is transmitted by visible light communication.

On the other hand, when the display determines in step S912 that the divided transmission mode is not set (N in step S912), the signals corresponding to each of the plurality of transmitters are transmitted from the entire display area of the display by visible light communication. It is transmitted (step S913). That is, the display sends keys associated with multiple pieces of information from the entire screen.

In addition, when the display determines in step S911 that there is one transmitter (one in step S911), it transmits a signal corresponding to the one transmitter from the entire display area of the display by visible light communication. Yes (step S915). That is, the display transmits from the entire screen.

Further, after any of steps S913 to S915, the display receives an access from a portable terminal (smartphone) that handles a signal (transmission information) transmitted by visible light communication as a key, and then the key in the image file is displayed. The corresponding metadata is passed to the mobile terminal that is the access source (step S916).

(Summary of Embodiments, etc.)
The information communication method according to the present embodiment is an information communication method for acquiring information from a subject, in which an image obtained by photographing the first subject, which is the subject, by an image sensor, each exposure included in the image sensor. A first exposure time setting step of setting a first exposure time of the image sensor so that a plurality of bright lines corresponding to the lines are generated according to a change in the brightness of the first subject; A first bright line image acquisition step of acquiring a first bright line image which is an image including the plurality of bright lines by photographing the changing first subject with the set first exposure time; A first information acquisition step of acquiring first transmission information by demodulating data specified by the patterns of the plurality of bright lines included in the acquired first bright line image; and the first transmission information. And a door control step of causing the door opening/closing drive device to open the door by transmitting a control signal.

Thereby, as shown in FIG. 204A, for example, the receiver equipped with the image sensor can be used like a door key, and a special electronic lock can be eliminated. As a result, it is possible to perform communication between various devices including devices with low computing power.

Further, in the information communication method, the image sensor may capture an image of a second subject having a change in brightness with the set first exposure time, so that the image sensor includes an image including a plurality of bright lines. A second bright line image acquisition step of acquiring a bright line image, and second transmission information is acquired by demodulating data specified by the plurality of bright line patterns included in the acquired second bright line image. A second information acquisition step, and an approach determination step of determining whether or not the receiving device including the image sensor is approaching the door based on the acquired first and second transmission information. In the door control step, the control signal may be transmitted when it is determined that the receiving device is approaching the door.

Thereby, as shown in FIG. 204A, for example, the door can be opened only when the receiving device (receiver) approaches the door, that is, only at an appropriate timing.

Further, in the information communication method, a second exposure time setting step of setting a second exposure time that is longer than the first exposure time, and the image sensor sets a third subject. A normal image acquisition step of acquiring a normal image showing the third subject by photographing at the second exposure time, wherein the normal image acquisition step includes a region including optical black of the image sensor. For each of the plurality of exposure lines in step S1, the charge is read out after a predetermined time has elapsed from the time when the charge was read out for the exposure line adjacent to the exposure line, and the first bright line image acquisition step is performed. Then, without using the optical black for reading charges, for each of the plurality of exposure lines in the region other than the optical black in the image sensor, the charges are read for the exposure line adjacent to the exposure line. The charge may be read out after the elapse of a time longer than the predetermined time from the point of time when the charge was released.

As a result, as shown in FIGS. 187A to 187E, for example, when the first bright line image is acquired, the electric charges are not read out (exposed) to the optical black, and therefore, the area other than the optical black in the image sensor. It is possible to lengthen the time required to read out (exposure) the charges from the effective pixel region. As a result, it is possible to prolong the time for receiving a signal in the effective pixel region and obtain a large number of signals.

Further, in the information communication method, further, in a pattern of the plurality of bright lines included in the first bright line image, a length in a direction perpendicular to each of the plurality of bright lines is less than a predetermined length. A length determining step of determining whether or not there is a frame, and a frame rate of the image sensor is set to the first bright line when it is determined that the length of the pattern is less than the predetermined length. A frame rate changing step of changing to a second frame rate slower than the first frame rate at the time of acquiring an image, and the image sensor changing the brightness of the first subject at the second frame rate. And a third bright line image acquisition step of acquiring a third bright line image, which is an image including a plurality of bright lines, by shooting with the set first exposure time, and the acquired third bright line image. A third information acquisition step of acquiring the first transmission information by demodulating the data specified by the patterns of the plurality of bright lines included in the bright line image may be included.

As a result, as shown in FIG. 224A, for example, when the signal length indicated by the bright line pattern (bright line region) included in the first bright line image is less than, for example, one block of the transmitted signal, the frame The rate is reduced, and the bright line image is acquired again as the third bright line image. As a result, the length of the pattern of the bright lines included in the third bright line image can be increased, and the transmitted signal can be acquired for one block.

Further, the information communication method further includes a ratio setting step of setting a ratio of a vertical width and a horizontal width of an image obtained by the image sensor, and the first bright line image acquisition step is performed by the set ratio. A clipping determination step of determining whether or not an edge of the image in a direction perpendicular to each exposure line is clipped, and when it is determined that the edge is clipped, the ratio set in the ratio setting step Is changed to a non-clipping ratio that is a ratio at which the edge is not clipped, and the image sensor captures the first object having a change in brightness, so that the first bright line of the non-clipping ratio is obtained. An acquisition step of acquiring an image may be included.

As a result, for example, as shown in FIGS. 188 and 189A to 189C, for example, the ratio of the horizontal width to the vertical width of the effective pixel area of the image sensor is 4:3, and the ratio of the horizontal width to the vertical width of the image is 16:. When the bright line along the horizontal direction appears, that is, when the exposure line extends along the horizontal direction, it is determined that the upper and lower ends of the image are clipped. That is, it is determined that the edge of the first bright line image is missing. In this case, the ratio of the image is changed to a ratio that is not clipped, for example, 4:3. As a result, it is possible to prevent the edge of the first bright line image from being missing, and it is possible to acquire a lot of information from the first bright line image.

The information communication method further includes a compression step of generating a compressed image by compressing the first bright line image in a direction parallel to each of the plurality of bright lines included in the first bright line image. And a compressed image transmitting step of transmitting the compressed image.

Thereby, as shown in FIG. 191, for example, the first bright line image can be appropriately compressed without missing the information indicated by the plurality of bright lines.

Further, the information communication method further includes a gesture determination step of determining whether or not the receiving device including the image sensor has been moved in a predetermined manner, and it has been determined that the receiving apparatus has been moved in the predetermined manner. Sometimes, it may include a startup step of booting the image sensor.

Thereby, as shown in FIG. 199, for example, the image sensor can be easily activated only when necessary, and power consumption efficiency can be improved.

(Embodiment 9)
In the present embodiment, each application example using the receiver such as the smartphone in each of the above embodiments and the transmitter that transmits information as the blinking pattern of the LED or the organic EL will be described.

FIG. 217 is a diagram illustrating an example of application of the transmitter and the receiver in Embodiment 9.

The robot 8970 has, for example, a function as a self-propelled cleaner and a function as a receiver in each of the above embodiments. The lighting devices 8971a and 8971b each have a function as a transmitter in each of the above embodiments.

For example, the robot 8970 performs cleaning while moving in the room, and photographs the lighting device 8971a that illuminates the room. The lighting device 8971a transmits the ID of the lighting device 8971a by changing the brightness. As a result, the robot 8970 receives the ID from the lighting device 8971a and estimates its own position (self position) based on the ID, as in the above embodiments. In other words, the robot 8970 moves based on the detection result of the 9-axis sensor, the relative position of the lighting device 8971a shown in the image obtained by photographing, and the absolute position of the lighting device 8971a specified by the ID, while moving. Is estimated.

Further, when the robot 8970 moves away from the lighting device 8971a by moving, the robot 8970 transmits a signal for turning off the lighting device 8971a (light-off command). For example, the robot 8970 transmits an extinguishing command when the robot 8970 is separated from the lighting device 8971a by a predetermined distance. Alternatively, the robot 8970 transmits a turn-off command to the lighting device 8971a when the lighting device 8971a disappears from the image obtained by photographing or when another lighting device appears in the image. When the lighting device 8971a receives the turn-off command from the robot 8970, the lighting device 8971a turns off the light according to the turn-off command.

Next, the robot 8970 detects that the robot 8970 approaches the lighting device 8971b based on the estimated self-position while moving and cleaning. That is, the robot 8970 holds information indicating the position of the lighting device 8971b, and when the distance between the self position and the position of the lighting device 8971b becomes less than or equal to a predetermined distance, the lighting device 8971b. Detects that you are approaching. Then, the robot 8970 transmits a signal (lighting command) instructing the lighting device 8971b to light. When the lighting device 8971b receives the lighting command, the lighting device 8971b lights according to the lighting command.

As a result, the robot 8970 can brighten only its surroundings while moving and easily perform cleaning.

FIG. 218 is a diagram illustrating an example of application of the transmitter in Embodiment 9.

For example, as shown in (a) of FIG. 218, a plurality of light emitting areas A to F are displayed side by side on the display, and the light emitting areas A to F change their luminances to transmit a signal. Here, in the example shown in (a) of FIG. 218, each of the light emitting areas A to F has a rectangular shape and is arranged along the horizontal direction and the vertical direction. In such a case, a non-brightness changing area that does not change in brightness crosses the display along the horizontal direction of the display, passing between the light emitting areas A, B, C and the light emitting areas D, E, F. There is. In addition, another non-brightness changing area that does not change in brightness crosses the display along the vertical direction of the display, passing between the light emitting areas A and D and the light emitting areas B and E. Furthermore, another non-brightness changing region that does not change in luminance crosses the display along the vertical direction of the display, passing between the light emitting regions B and E and the light emitting regions C and F.

Therefore, when the receiver in each of the above-described embodiments photographs its display with the exposure line of the receiver oriented in the horizontal direction, the non-luminance change along the horizontal direction of the image (captured image) obtained by the photographing No bright line appears in the portion corresponding to the region. That is, in the captured image, the area where the bright line appears (bright line area) becomes discontinuous. Further, when the receiver photographs the display with its exposure line oriented in the vertical direction, no bright line appears in the portion of the captured image corresponding to the two non-luminance change regions along the vertical direction. That is, even at this time, the bright line area becomes discontinuous in the captured image. If the bright line region becomes discontinuous in this way, it becomes difficult to receive a signal transmitted due to a change in luminance.

Therefore, the display 8972 in this embodiment has a function as a transmitter in each of the above embodiments, and the plurality of light emitting areas A to F are arranged so as to be shifted so that the bright line areas are continuous.

For example, as shown in (b) of FIG. 218, in the display 8972, the upper light emitting areas A, B and C and the lower light emitting areas D, E and F are horizontally shifted from each other. Alternatively, as shown in (c) of FIG. 218, the display 8972 displays parallelogrammic or rhombic light emitting regions A to F, respectively. As a result, it is possible to eliminate the non-brightness change region that crosses the display 8972 between the light emitting regions A to F along the vertical direction of the display 8972. As a result, even if the receiver photographs the display 8972 with its exposure line oriented in the vertical direction, the bright line region can be continuous in the photographed image.

Further, as shown in (d) and (e) of FIG. 218, in the display 8972, each of the light emitting areas A to F may be vertically shifted and arranged. Accordingly, it is possible to eliminate the non-brightness change region that crosses the display 8972 through the light emitting regions A to F along the horizontal direction of the display 8972. As a result, even if the receiver photographs the display 8972 with the exposure line oriented in the horizontal direction, the bright line region can be continuous in the photographed image.

Further, as shown in (f) of FIG. 218, the display 8972 may display the hexagonal light emitting regions A to F such that the sides of the respective regions are parallel to each other. Even in such a case, similarly to the above, it is possible to eliminate the non-brightness changing region that crosses the display 8972 through the light emitting regions A to F along the horizontal direction and the vertical direction of the display 8972. As a result, even if the receiver photographs the display 8972 with its exposure line oriented horizontally or the display 8972 with its exposure line oriented vertically, the bright line in the photographed image The areas can be continuous.

FIG. 219 is a flowchart of the information communication method in this embodiment.

The information communication method according to the present embodiment is an information communication method for transmitting a signal according to a change in luminance, and includes steps SK11 and SK12.

That is, in this information communication method, the determination step SK11 that determines the pattern of the brightness change by modulating the signal to be transmitted, and the brightness change of the plurality of light emitters according to the determined pattern of the brightness change, And a transmitting step SK12 for transmitting the signal to be transmitted. Then, in the surface on which the plurality of light emitters are arranged, a non-brightness change region that is outside the plurality of light emitters and does not change in brightness is provided along at least one of a vertical direction and a horizontal direction of the surface. The plurality of light emitters are arranged on the surface so as not to traverse the surface between the plurality of light emitters.

FIG. 220 is a block diagram of the information communication device according to the present embodiment.

The information communication device K10 in the present embodiment is an information communication device that transmits a signal according to a change in luminance, and includes components K11 and K12.

That is, the information communication device K10 is configured to modulate the signal to be transmitted to determine the brightness change pattern, and the brightness change of the plurality of light emitters according to the brightness change pattern thus determined. And a transmission unit K12 that transmits the signal to be transmitted. Then, in the surface on which the plurality of light emitters are arranged, a non-brightness change region that is outside the plurality of light emitters and does not change in brightness is provided along at least one of a vertical direction and a horizontal direction of the surface. The plurality of light emitters are arranged on the surface so as not to traverse the surface between the plurality of light emitters.

In the information communication method and the information communication device K10 shown in FIGS. 219 and 220 as described above, for example, as shown in FIG. 218, it is acquired by photographing the above-mentioned surface (display) by the image sensor provided in the receiver. In the captured image, the bright line area can be continuous. As a result, it is possible to easily receive the signal to be transmitted, and it is possible to perform communication between various devices including a device having a small computing power.

In addition, in each of the above-described embodiments, each component may be configured by dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory. For example, the program causes a computer to execute the information communication method shown by the flowchart of FIG. 219.

FIG. 221A is a diagram illustrating an example of application of the transmitter and the receiver in Embodiment 9.

The receiver 8973 is configured as a smartphone having a function as a receiver in each of the above embodiments. As shown in (a) of FIG. 221A, the receiver 8973 photographs the display 8972 and tries to read the bright line appearing in the photographed image. Here, when the display 8972 is dark, the receiver 8973 may not be able to read the bright line and may not be able to receive the signal from the display 8972. At this time, the receiver 8973 fires a flash at a predetermined rhythm, as shown in (b) of FIG. 221A. Upon receiving the flash, the display 8972 raises the brightness and performs a bright display as shown in (c) of FIG. 221A. As a result, the receiver 8973 can read the bright line appearing in the captured image and can receive the signal from the display 8972.

FIG. 221B is a flowchart showing operation of the receiver 8973 in Embodiment 9.

The receiver 8973 first determines whether or not a user operation or gesture for starting reception has been received (step S831). Here, if the receiver 8973 determines that the image has been received (Y in step S831), the receiver 8973 starts the reception by photographing using the image sensor (step S832). Next, the receiver 8973 determines whether a predetermined time has elapsed from the start of reception without completing the reception (step S833). Here, if the receiver 8973 determines that the predetermined time has elapsed (Y in step S833), it flashes the flash at a predetermined rhythm (step S834) and repeats the processing from step S833. When the process of step S833 is repeated, the receiver 8973 determines whether or not a predetermined time has passed since the flash was blinked without the reception being completed. Further, in step S834, the receiver 8973 transmits a signal to the display 8972 for generating a predetermined sound of a frequency that is difficult for humans to hear, or for indicating that it is waiting for reception, instead of blinking the flash. May be sent to the machine.

FIG. 222 is a diagram illustrating an example of application of the transmitter and the receiver in Embodiment 9.

The lighting device 8974 has a function as a transmitter in each of the above embodiments. The illumination device 8974 illuminates, for example, a route bulletin board 8975 at a railway station while changing the brightness. The receiver 8973 directed to the route bulletin board 8975 by the user photographs the route bulletin board 8975. Accordingly, the receiver 8973 acquires the ID of the route bulletin board 8975, and acquires the detailed information regarding each route described in the route bulletin board 8975, which is the information associated with the ID. Then, the receiver 8973 displays a guide image 8973a showing the detailed information. For example, the guide image 8973a indicates the distance to the route described in the route bulletin board 8975, the direction toward the route, and the time when the next train arrives on the route.

Here, when the user touches the guide image 8973a, the receiver 8973 displays the supplementary guide image 8973b. This supplementary guide image 8973b is, for example, selected by the user from one of the train timetable, information about another route different from the route shown by the guide image 8973a, and detailed information about the station. It is an image to be displayed accordingly.

FIG. 223 is a diagram illustrating an example of application of the transmitter in Embodiment 9.

The lighting devices 8976a to 8976c each have a function as a transmitter in each of the above embodiments and illuminate the signboard 8977 of the store. Here, as shown in (a) of FIG. 223, each of the lighting devices 8976a to 8976c may transmit the same ID by synchronously changing the luminance. Also, as shown in (b) of FIG. 223, only the lighting equipment 8976a and the lighting equipment 8976c arranged at both ends transmit the same ID by synchronously changing the luminance, and are arranged between the lighting equipments. The illuminated lighting device 8976b may illuminate the signboard 8977 without transmitting the ID due to the brightness change. Further, as shown in (c) of FIG. 223, in a state where the lighting device 8976b does not transmit the ID, the lighting device 8976a and the lighting device 8976c arranged at both ends transmit different IDs by changing the luminance. May be. In this case, since the lighting device 8976b located between the lighting device 8976a and the lighting device 8976c does not change the brightness for transmitting the ID, signals from the lighting device 8976a and the lighting device 8976c interfere with each other. You can prevent it from happening. The ID transmitted by the lighting device 8976a and the ID transmitted by the lighting device 8976c are different from each other, but the same information may be associated with these IDs.

FIG. 224A is a diagram illustrating an example of application of the transmitter and the receiver in Embodiment 9.

The lighting device 8978 has a function as a transmitter in each of the above embodiments, and constantly transmits a signal by changing the luminance as shown in (1) of FIG. 224A.

The receiver in this embodiment images the lighting device 8978. At this time, as illustrated in FIG. 224A, the imaging range 8979 of the receiver includes a lighting device 8978 and a portion other than the lighting device 8978. That is, each of the upper range a and the lower range c in the shooting range 8979 includes a portion other than the lighting device 8978, and the central range b in the shooting range 8979 includes the lighting device 8978.

As shown in (2) and (3) of FIG. 224A, the receiver acquires a captured image (bright line image) including a plurality of bright lines caused by a change in luminance of the lighting device 8978 by capturing an image of the lighting device 8978. In this bright line image, bright lines do not appear in the portions corresponding to the upper range a and the lower range c described above, but only in the portions corresponding to the central range b.

Here, when the receiver photographs the illumination device 8978 at a frame rate of, for example, 30 fps, as shown in (2) of FIG. 224A, the length b of the bright line area in the bright line image is short, and the receiver has, for example, 15 fps. When the illumination device 8978 is photographed at the frame rate of, the length b of the bright line area in the bright line image is long, as shown in (3) of FIG. 224. The length of the bright line area (bright line pattern) is the length in the direction perpendicular to each bright line included in the bright line area.

Therefore, the receiver according to the present embodiment photographs the illumination device 8978 at a frame rate of 30 fps, for example, and determines whether or not the length b of the bright line area in the bright line image is less than a predetermined length. The predetermined length is, for example, a length corresponding to one block of the signal transmitted by the change in the brightness of the lighting device 8978. When the receiver determines that the length is less than the predetermined length, the receiver changes the frame rate to 15 fps, for example. Accordingly, the receiver can collectively receive signals for one block from the lighting device 8978.

FIG. 224B is a flowchart showing operation of the receiver in Embodiment 9.

The receiver first determines whether or not a bright line is included in the captured image, that is, whether or not a stripe due to the exposure line is captured (step S841). Here, when the receiver determines that the image is captured (Y in step S841), the receiver determines which image capturing mode (image capturing mode) is set (step S842). When the receiver determines that the image capturing mode is the intermediate image capturing mode (intermediate mode) or the normal image capturing mode (normal image capturing mode), the receiver changes the image capturing mode to the visible light image capturing mode (visible light communication mode) (step S843). ..

Next, the receiver determines whether or not the length in the direction perpendicular to the bright line in the bright line region (bright line pattern) is equal to or greater than a predetermined length (step S844). That is, the receiver determines whether or not there is a stripe region having a predetermined size or more in the direction perpendicular to the exposure line. If it is determined that the length is not equal to or greater than the predetermined length (N in step S844), the receiver determines whether the optical zoom is available (step S845). Here, when it is determined that the optical zoom is available (Y in step S845), the receiver performs the optical zoom so that the bright line area becomes long, that is, the stripe area is enlarged (step S846). ). On the other hand, when it is determined that the optical zoom cannot be used (N in step S845), the receiver determines whether the Ex zoom (Ex optical zoom) is available (step S847). If it is determined that the Ex zoom can be used (Y in step S847), the Ex zoom is performed so that the bright line area becomes long, that is, the stripe area is enlarged (step S848). On the other hand, when it is determined that the Ex zoom cannot be used (N in step S847), the receiver lowers the imaging frame rate (step S849). Then, the receiver receives the signal by photographing the lighting device 8978 at the set frame rate (step S850).

Note that, in the example shown in FIG. 224B, the frame rate is lowered when the optical zoom and the Ex zoom cannot be used, but the frame rate may be lowered when those zooms can be used. The Ex-zoom is a function of limiting the use area of the image sensor, narrowing the shooting angle of view, and setting the apparent focal length to the telephoto position.

FIG. 225 is a diagram illustrating an operation of the receiver in Embodiment 9.

When the illumination device 8978, which is the transmitter, is small in the captured image 8980a, the receiver can obtain the captured image 8980b in which the illumination device 8978 is largely displayed by using the optical zoom or the Ex zoom. it can. That is, the receiver can obtain a bright line image (captured image) having a long bright line region in a direction perpendicular to the bright line by using the optical zoom or the Ex zoom.

FIG. 226 is a diagram illustrating an example of application of the transmitter in Embodiment 9.

The transmitter 8981 has a function as a transmitter in each of the above-described embodiments, and communicates with the operation panel 8982, for example. The operation panel 8982 includes a transmission switch 8982a and a power switch 8982b.

When the transmission switch 8982a is turned on, the operation panel 8982 instructs the transmitter 8981 to perform visible light communication. Upon receiving the instruction, the transmitter 8981 transmits a signal by changing the brightness. When the transmission switch 8982a is turned off, the operation panel 8982 instructs the transmitter 8981 to stop visible light communication. Upon receiving the instruction, the transmitter 8981 does not change the brightness and stops the signal transmission.

When the power switch 8982b is turned on, the operation panel 8982 instructs the transmitter 8981 to turn on the power of the transmitter 8981. Upon receiving the instruction, the transmitter 8981 turns on its own power source. For example, when the transmitter 8981 is configured as a lighting device, the transmitter 8981 illuminates the surroundings when the power is turned on, and when the transmitter 8981 is configured as a television, the transmitter 8981 is turned on to display an image. Is displayed. When the power switch 8982b is turned off, the operation panel 8982 instructs the transmitter 8981 to turn off the power of the transmitter 8981. Upon receiving the instruction, the transmitter 8981 turns off its power source and enters a standby state.

227 is a diagram illustrating an example of application of the receiver in Embodiment 9.

The receiver 8973 configured as a smartphone has, for example, a function as a transmitter in each of the above embodiments, and acquires an authentication ID and an expiration date from the server 8983. If the current time is within the expiration date, the receiver 8973 transmits the authentication ID to the peripheral device 8984 by changing the brightness of the display, for example. The peripheral device 8984 is, for example, a camera, a barcode reader, or a personal computer.

When the peripheral device 8984 receives the authentication ID from the receiver 8973, the peripheral device 8984 transmits the authentication ID to the server 8983 and requests verification. The server 8983 compares the authentication ID transmitted from the peripheral device 8984 with the authentication ID held by itself and transmitted to the receiver 8973, and if they match, notifies the peripheral device 8984 that they match. When the peripheral device 8984 receives the notification of the coincidence from the server 8983, the peripheral device 8984 unlocks the lock set for itself, performs payment processing for electronic payment, and performs processing such as login.

FIG. 228A is a flowchart illustrating an example of operation of a transmitter in Embodiment 9.

The transmitter according to the present embodiment has a function as the transmitter according to each of the above embodiments, and is configured as, for example, a lighting device or a display. Such a transmitter determines whether or not the dimming level (brightness level) is lower than a predetermined level, for example (step S861a). Here, when the transmitter determines that the level is lower than the predetermined level (Y in step S861a), the transmitter stops transmitting the signal due to the brightness change (step S861b).

FIG. 228B is a flowchart illustrating an example of operation of a transmitter in Embodiment 9.

The transmitter in the present embodiment determines whether or not the dimming level (brightness level) exceeds a predetermined level (step S862a). Here, when the transmitter determines that the level is higher than the predetermined level (Y in step S862a), it starts transmitting a signal due to a change in brightness (step S862b).

FIG. 229 is a flowchart showing an example of operation of the transmitter in this embodiment.

The transmitter in the present embodiment determines whether or not a predetermined mode is selected (step S863a). For example, the predetermined mode is an eco mode or a power saving mode. Here, when the transmitter determines that the predetermined mode is selected (Y in step S863a), the transmitter stops transmitting the signal due to the brightness change (step S863b). On the other hand, when the transmitter determines that the predetermined mode is not selected (N in step S863a), it starts transmitting a signal due to a change in brightness (step S863c).

FIG. 230 is a flowchart illustrating an example of operation of the imaging device in Embodiment 9.

The imaging device according to the present embodiment is, for example, a video camera and determines whether or not recording is in progress (step S864a). Here, when the imaging device determines that recording is in progress (Y in step S864a), the imaging device transmits a visible light transmission stop command to a transmitter that transmits a signal due to a change in luminance (step S864b). The transmitter which has received the visible light transmission stop command stops the transmission of the signal due to the change in the luminance (visible light transmission). On the other hand, when the imaging device determines that the recording is not in progress (N in step S864a), it further determines whether or not the recording is stopped, that is, whether or not it is immediately after the recording (step S864c). Here, when the imaging device determines that the recording is stopped (Y in step S864c), the imaging device transmits a visible light transmission start command to the above-described transmitter (step S864d). The transmitter which has received the visible light transmission start command starts transmission of a signal due to a change in luminance (visible light transmission).

FIG. 231 is a flowchart illustrating an example of operation of the imaging device in Embodiment 9.

The imaging device according to the present embodiment is, for example, a digital still camera, and it is determined whether the imaging button (shutter button) is pressed halfway or whether focusing is performed (step S865a). .. Next, the imaging device determines whether or not light and shade appear in the direction along the exposure line of the image sensor included in the imaging device (step S865b). Here, if it is determined that the light and shade are present (Y in step S865b), there is a possibility that the transmitter that is transmitting the signal due to the brightness change is near the imaging device. Therefore, the imaging device transmits a visible light transmission stop command to the transmitter (step S865c). Next, the imaging device acquires a captured image by performing imaging (step S865d). Then, the imaging device transmits a visible light transmission start command to the above transmitter (step S865e). As a result, the imaging device can acquire the captured image without being affected by the brightness change by the transmitter. Moreover, since the period during which the signal transmission due to the change in luminance is stopped is limited to the very short period during which the image is captured by the image capturing device, the period during which the visible light communication cannot be performed can be suppressed.

FIG. 232 is a diagram illustrating an example of a signal transmitted by the transmitter in Embodiment 9.

The transmitter in this embodiment has a function as the transmitter in each of the above embodiments, and transmits a signal by outputting high-luminance light (Hi) or low-luminance light (Lo) for each slot. To do. Specifically, the slot is a time unit of 104.2 μs. Further, the transmitter transmits a signal indicating 1 by outputting Hi, and transmits a signal indicating 0 by outputting Lo.

FIG. 233 is a diagram illustrating an example of a signal transmitted by the transmitter in Embodiment 9.

The transmitter described above sequentially outputs PHY (physical layer) frames which are signal units by outputting Hi or Lo for each slot. The PHY frame includes a preamble composed of 8 slots, an FCS (Frame Check Sequence) composed of 2 slots, and a body composed of 20 slots. Each part included in the PHY frame is transmitted in the order of preamble, FCS, and body.

The preamble corresponds to the header of the PHY frame and includes, for example, "01010111". The preamble may be composed of 7 slots. In this case, the preamble includes "0101011". The FCS includes “01” when the number of 1's included in the body is an even number, and includes “11” when the number of 1's included in the body is an odd number. The body includes 5 symbols each consisting of 4 slots. The symbol includes “0111”, “1011”, “1101”, or “1110” in the case of 4PPM modulation.

FIG. 234 is a diagram illustrating an example of a signal transmitted by the transmitter in Embodiment 9.

The above symbols are converted by the receiver into a 2-bit value. For example, the symbols “0111”, “1011”, “1101” and “1110” are converted into “00”, “01”, “10” and “11”, respectively. Therefore, the body (20 slots) of the PHY frame is converted into a 10-bit signal. The 10-bit body includes TYPE, which is 3 bits indicating the type of PHY frame, ADDR which is 2 bits indicating the address of the PHY frame or body, and DATA which is 5 bits indicating the substance of the data. For example, when the type of PHY frame is TYPE1, TYPE indicates “000”. ADDR indicates "00", "01", "10" or "11".

At the receiver, the DATA contained in the bodies of the four PHY frames are combined. The above-mentioned ADDR is used for this connection. That is, the receiver includes DATA included in the body of the PHY frame having ADDR “00”, DATA included in the body of the PHY frame having ADDR “01”, and DATA included in the body of the PHY frame having ADDR “10”. Data composed of 20 bits is generated by combining DATA included in the body of the PHY frame having ADDR “11”. As a result, four PHY frames are decoded. The generated data includes 16-bit effective DATA and 4-bit CRC (Cyclic Redundancy Check).

FIG. 235 is a diagram illustrating an example of a signal transmitted by the transmitter in Embodiment 9.

The types of PHY frames described above include TYPE1, TYPE2, TYPE3, and TYPE4. The length of the body, the length of ADDR, the length of DATA, the number of DATA to be connected (the number of connections), the length of effective DATA, and the type of CRC are different for each of these types.

For example, in the case of TYPE1, TYPE (TYPEBIT) indicates “000”, the body length is 20 slots, the ADDR length is 2 bits, and the DATA length is 5 bits. The number is 4, the effective DATA length is 16 bits, and the CRC type is CRC-4. On the other hand, in the case of TYPE2, TYPE (TYPEPEBIT) indicates “001”, the body length is 24 slots, the ADDR length is 4 bits, and the DATA length is 5 bits. The number is eight, the effective DATA length is 32 bits, and the CRC type is CRC-8.

Visible light communication can be appropriately performed by the signals shown in FIGS. 232 to 235.

FIG. 236 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9.

The system according to the present embodiment includes a transmitter 8991 having the same function as the transmitter according to each of the above embodiments, a receiver 8973 configured as a smartphone, a content sharing server 8992, and an ID management server 8993. With.

For example, the content creator uploads content such as audio/video data indicating a still image or a moving image for introducing a product and product information indicating a maker, a place of origin, a raw material, a specification, etc. of the product to the content sharing server 8992. To do. Then, the content sharing server 8992 registers the product information in the ID management server 8993 in association with the content ID for identifying the content.

Next, the transmitter 8991 downloads the content and the content ID from the content sharing server 8992 in accordance with the operation by the user, displays the content, and changes the brightness, that is, the visible light communication to obtain the content ID. Send. When the user views the content and is interested in the product introduced in the content, the user directs the receiver 8973 to the transmitter 8991 to take a picture. The receiver 8973 receives the content ID by shooting the content displayed on the transmitter 8991.

Next, the receiver 8973 accesses the ID management server 8993 and makes an inquiry about the content ID to the ID management server 8993. Accordingly, the receiver 8973 acquires the product information associated with the content ID from the ID management server 8993, and displays the product information. Here, when the receiver 8973 receives an operation prompting the purchase of the product corresponding to the product information, the receiver 8973 accesses the maker of the product and executes a process for purchasing the product.

Next, the ID management server notifies the maker indicated by the product information associated with the content ID of the inquiry information indicating the number of inquiries made to the content ID or the number of accesses. Upon receiving the inquiry information, the maker electronically sends an affiliate reward according to the number of inquiries indicated by the inquiry information to the content creator identified by the content ID via the ID management server 8993 and the content sharing server 8992. Pay by payment.

FIG. 237 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9.

In the case of the example illustrated in FIG. 236, the content sharing server 8992 registers the product information in the ID management server 8993 in association with the content ID when the content and the product information are uploaded. However, such registration is performed. You don't have to. For example, as shown in FIG. 237, the content sharing server 8992 searches the ID management server for a product ID for identifying the product in the uploaded product information, and embeds the product ID in the uploaded content.

Next, the transmitter 8991 downloads the content in which the product ID is embedded and the content ID from the content sharing server 8992 in response to the operation by the user, displays the content, and changes the brightness, that is, the visible light. The content ID and the product ID are transmitted by communication. When the user views the content and is interested in the product introduced in the content, the user directs the receiver 8973 to the transmitter 8991 to take a picture. The receiver 8973 receives the content ID and the product ID by photographing the content displayed on the transmitter 8991.

Next, the receiver 8973 accesses the ID management server 8993 and makes an inquiry to the ID management server 8993 about the content ID and the product ID. Accordingly, the receiver 8973 acquires the product information associated with the product ID from the ID management server 8993 and displays the product information. Here, when the receiver 8973 receives an operation prompting the purchase of the product corresponding to the product information, the receiver 8973 accesses the maker of the product and executes a process for purchasing the product.

Next, the ID management server notifies the manufacturer indicated by the product information associated with the product ID of inquiry information indicating the number of inquiries made to the content ID and the product ID, or the number of accesses. To do. Upon receiving the inquiry information, the maker electronically sends an affiliate reward according to the number of inquiries indicated by the inquiry information to the content creator identified by the content ID via the ID management server 8993 and the content sharing server 8992. Pay by payment.

FIG. 238 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9.

The system according to the present embodiment includes a content sharing server 8992a instead of the content sharing server 8992 shown in FIG. 237, and further includes an SNS server 8994. The SNS server 8994 is a server that provides a social networking service, and performs a part of the processing performed by the content sharing server 8992 shown in FIG. 237.

Specifically, the SNS server 8994 acquires the content and product information uploaded from the content creator, searches for the product ID corresponding to the product information, and embeds the product ID in the content. Then, the SNS server 8994 transfers the content in which the product ID is embedded to the content sharing server 8992a. The content sharing server 8992a receives the content transferred from the SNS server 8994, and transmits the content in which the product ID is embedded and the content ID to the transmitter 8991.

That is, in the example shown in FIG. 238, a unit including the SNS server 8994 and the content sharing server 8992a serves as the content sharing server 8992 shown in FIG. 237.

In such a system shown in FIGS. 236 to 238, it is possible to appropriately pay an appropriate affiliate incentive money for an advertisement (content) for which an inquiry is made using visible light communication.

The information communication method according to one or more aspects has been described above based on the embodiment, but the present invention is not limited to this embodiment. As long as it does not depart from the gist of the present invention, various modifications that a person skilled in the art can think of in the present embodiment, or a form constructed by combining components in different embodiments are also included in the scope of one or more aspects. May be included within.

Hereinafter, the present embodiment will be supplemented.

(Mixed modulation method)
239 and 240 are diagrams illustrating an example of operation of a transmitter in Embodiment 9.

As shown in FIG. 239, the transmitter modulates the transmission signal by a plurality of modulation schemes and transmits the modulated signal alternately or simultaneously.

By modulating the same signal with a plurality of modulation schemes and transmitting the same, a receiver that supports only one of the modulation schemes can receive the signal. Further, for example, by using a modulation method having a high transmission rate, a modulation method resistant to noise, and a modulation method having a long communication distance together, reception can be performed by an optimum method according to the environment on the receiving side.

When the receiver is capable of receiving a plurality of modulation schemes, the receiver receives a signal modulated by a plurality of methods. When modulating the same signal, the transmitter gives the same signal ID and transmits the modulated signal. With this, the receiver can recognize that the same signal is modulated by different modulation methods by confirming the signal ID, and synthesizes a signal having the same signal ID from a plurality of types of modulated signals. Therefore, the signal can be received quickly and accurately.

For example, the transmitter includes a signal division unit and modulation units 1 to 3. The signal division unit divides the transmission signal into the partial signal 1 and the partial signal 2, attaches the signal ID to the partial signal 1, and attaches another signal ID to the partial signal 2. The modulation unit 1 frequency-modulates the partial signal 1 to which the signal ID is added to generate a signal indicating a sine wave. The modulator 2 performs frequency modulation different from that of the modulator 1 on the partial signal 1 to which the signal ID is added to generate a signal indicating a rectangular wave. On the other hand, the modulator 3 performs pulse position modulation on the partial signal 2 with another signal ID to generate a signal showing a rectangular wave.

As shown in FIG. 240, the transmitter combines and transmits signals modulated by a plurality of modulation methods. In the example of FIG. 240, the receiver can receive only the signal modulated by the frequency modulation method using the low frequency by setting the exposure time to be long. Further, the receiver can receive the pulse position modulation method using the high frequency band by setting the exposure time short. At this time, the receiver averages the intensities of the received light in time by averaging the luminance in the direction perpendicular to the bright line, and can obtain the signal when the exposure time is long. ..

(Transmission signal verification and digital modulation)
241 and 242 are diagrams illustrating an example of a structure and operation of a transmitter in Embodiment 9.

As shown in FIG. 241, the transmitter includes a signal storage unit, a signal verification unit, a signal modulation unit, a light emitting unit, an abnormality notification unit, an original key storage unit, and a key generation unit. The signal storage unit stores a transmission signal and a signal conversion value obtained by converting the transmission signal using a verification key described later. A one-way function is used for this conversion. The original key storage unit stores an original key that is a value that is a source of a key, such as a circuit constant such as a resistance value or a time constant. The key generation unit generates a verification key from the original key.

The signal verification unit obtains a signal conversion value by converting the transmission signal stored in the signal storage unit using the verification key. Whether or not the signal has been tampered with is verified by whether the signal conversion value obtained here is equal to the signal conversion value stored in the signal storage unit. As a result, simply copying the signal in the signal storage unit to another transmitter makes it impossible to send the signal because the verification key is different at the other transmitter, thus preventing forgery of the transmitter. I can.

If the signal has been tampered with, the abnormality notification unit displays a message to that effect. As a method thereof, for example, there is a method of blinking the light emitting portion or making a sound at a cycle that can be visually recognized by humans. By limiting the abnormality notification only to the predetermined time immediately after the power is turned on, the transmitter can be used for purposes other than transmission even when there is an abnormality in the signal.

If the signal has not been tampered with, the signal modulator modulates the signal into a light emission pattern. Various modulation methods can be used for this modulation method. Modulation methods that can be used here include, for example, amplitude shift keying (ASK), phase shift keying (PSK), frequency shift keying (FSK), quadrature phase amplitude keying (QAM), and delta keying (DM). , Minimum deviation modulation (MSK), complementary code modulation (CCK), orthogonal frequency division multiplexing (OFDM), amplitude modulation (AM), frequency modulation (FM), phase modulation (PM), pulse width modulation (PWM) , Pulse amplitude modulation (PAM), pulse density modulation (PDM), pulse position modulation (PPM), pulse code modulation (PCM), frequency hopping spread spectrum (FHSS), direct sequence spread spectrum (DSSS), etc. It is selected according to the characteristics (whether analog or digital, whether continuous data transmission, etc.) and required characteristics (transmission speed, noise resistance, transmission distance). Furthermore, a modulation method combining these can be used.

It should be noted that the same effects can be obtained in Embodiments 1 to 9 even when a signal modulated by the modulation method described here is used.

As shown in FIG. 242, the transmitter may include a signal demodulation unit instead of the signal verification unit. In this case, the signal storage unit holds the encrypted transmission signal obtained by encrypting the transmission signal using the encryption key paired with the decryption key generated by the key generation unit. The signal demodulation unit uses the decryption key to decrypt the encrypted transmission signal. With this configuration, it is possible to make it difficult to forge a transmitter, that is, to make a transmitter that transmits an arbitrary signal.

(Embodiment 10)
In the present embodiment, each application example using the receiver such as the smartphone in each of the above embodiments and the transmitter that transmits information as the blinking pattern of the LED or the organic EL will be described.

(Reception of signals from multiple directions by multiple light receiving units)
FIG. 243 is a diagram showing a timepiece equipped with an optical sensor.

The timepiece is configured as a receiver for visible light communication, and includes a plurality of optical sensors and a condenser lens corresponding to each of the plurality of optical sensors. Specifically, as shown in the cross sectional view of FIG. 243, a condenser lens is arranged on the upper surface of the optical sensor. In FIG. 243, the condenser lenses are arranged with a predetermined inclination. The shape of the condenser lens is not limited to this, and may be any other shape as long as it can collect light. With such a configuration, the optical sensor can collect and receive the light from the light source in the outside by the lens. Therefore, visible light communication can be performed even with a small optical sensor mounted on a watch. In FIG. 243, it is divided into 12 regions, 12 optical sensors are mounted, and a condenser lens is arranged on the upper surface of each optical sensor. Thus, by dividing the inside of the timepiece into a plurality of areas and disposing a plurality of photosensors, it becomes possible to acquire information from a plurality of light sources. For example, in FIG. 243, the light from the light source 1 can be received by the first optical sensor, and the light from the light source 2 can be received by the second optical sensor. Further, it is also possible to use a solar cell as the optical sensor. By using a photovoltaic cell as an optical sensor, it is possible to perform solar power generation with a single optical sensor and at the same time perform visible light communication, thus reducing costs and achieving a compact shape. It will be possible. Furthermore, when a plurality of optical sensors are arranged, information from a plurality of light sources can be acquired at the same time, so that the position estimation accuracy can be improved. Although the optical sensor is provided in the timepiece in this embodiment, the present invention is not limited to this, and the optical sensor may be provided in another device as long as it is a mobile terminal such as a mobile phone or a mobile terminal.

FIG. 244 is a diagram illustrating an example of a receiver in Embodiment 10.

For example, the receiver 9020a configured as a wristwatch includes a plurality of light receiving units. For example, as shown in FIG. 244, the receiver 9020a includes a light receiving unit 9020b arranged at the upper end of the rotating shaft that supports the long and short hands of a wristwatch, and a character near the letter indicating 12 o'clock on the periphery of the wristwatch. Light receiving section 9020c. The light receiving unit 9020b receives the light traveling toward the light receiving unit 9020b along the direction of the rotation axis described above, and the light receiving unit 9020c travels toward the light receiving unit 9020c along the direction connecting the rotation axis and the character indicating 12 o'clock. Receive. Accordingly, when the user holds the receiver 9020a in front of the chest like when checking the time, the light receiving unit 9020b can receive light from above. As a result, the receiver 9020a can receive the signal from the ceiling lighting. Further, when the user holds the receiver 9020a in front of his/her chest to confirm the time, the light receiving unit 9020c can receive light from the front direction. As a result, the receiver 9020a can receive a signal from a front signage or the like.

Since these light receiving units 9020b and 9020c have directivity, signals can be received without interference even when there are a plurality of transmitters near each other.

FIG. 245 is a diagram illustrating an example of a receiver in Embodiment 10.

For example, as shown in (a) of FIG. 245, a receiver 9021 configured as a wristwatch includes 17 light receiving elements (light receiving units). These light receiving elements are arranged on the dial. In addition, 12 of these light receiving elements are arranged at positions corresponding to 1 o'clock to 12 o'clock on the dial, and the remaining 5 are arranged in the central portion of the dial. There is. Further, these 17 light receiving elements have directivities different from each other and receive light (signal) in the direction corresponding to each. In this way, by arranging a plurality of light receiving elements having directivity, the receiver 9021 can estimate the direction of the received signal. Further, as shown in FIG. 245(b), a prism for guiding light to the light receiving element may be arranged in front of the light receiving element. That is, the receiver 9021 includes eight light receiving elements arranged at equal intervals on the peripheral portion of the dial and a plurality of prisms that guide light to at least one of the light receiving elements. By providing such a prism, it is possible to estimate the fine direction of the transmitter even with a small number of light receiving elements. For example, when only the light receiving element 9021d among the eight light receiving elements receives light, the receiver 9021 can estimate that there is a transmitter in the direction connecting the center of the dial and the prism 9021a, and the light receiving element 9021d and the light receiving element 9021d. When 9021e receives the same signal, it can be estimated that there is a transmitter in the direction connecting the center of the dial and the prism 9021b. The windshield of the wristwatch may be provided with a directivity function and a prism function.

FIG. 246A is a flowchart of an information communication method according to an aspect of the present invention.

An information communication method according to an aspect of the present invention is an information communication method in which a mobile terminal acquires information, and includes steps SE11 and SE12.

That is, in this information communication method, at least one solar cell among the plurality of solar cells each having directivity, which is provided in the mobile terminal, has a direction corresponding to the directivity of the solar cell. A light receiving step (SE11) of receiving visible light emitted along the line and an information acquiring step (SE12) of acquiring information by demodulating a signal specified by the received visible light are included.

FIG. 246B is a block diagram of a mobile terminal according to one embodiment of the present invention.

A mobile terminal E10 according to one aspect of the present invention is a mobile terminal that acquires information, and includes a plurality of solar cells E11 each having directivity and an information acquisition unit E12. The information acquisition unit E12 receives visible light emitted by at least one solar cell E11 among the plurality of solar cells E11 along a direction corresponding to the directivity of the solar cell E11. In this case, the information is acquired by demodulating the signal specified by the received visible light.

In the information communication method and portable terminal E10 shown in FIGS. 246A and 246B, the solar cell E11 can be used for power generation while using it as an optical sensor for visible light communication. The cost of the mobile terminal E10 to be acquired can be suppressed, and the mobile terminal E10 can be made compact. In addition, since each of the plurality of solar photovoltaic cells E11 has a directivity, it is possible to estimate the direction in which the transmitter that emits the visible light is based on the directivity of the solar photovoltaic cell E11 that receives the visible light. .. Furthermore, since the plurality of solar cells E11 each have directivity, it is possible to separately receive the visible light emitted from the plurality of transmitters and appropriately acquire information from each of the plurality of transmitters. be able to.

Further, in the light receiving step (SE11), as shown in (b) of FIG. 245, the solar cell E11 (9021d, 9021e) is connected to the prism (9021a, 9021b or 9021c) provided in the mobile terminal E10 (9021). ) May be received. As a result, it is possible to highly accurately estimate the direction in which the transmitter that emits visible light is located, while suppressing the number of photovoltaic cells E11 provided in the mobile terminal E10. Further, as shown in FIG. 245, the portable terminal E10 is a wristwatch, and the plurality of solar cells E11 (light receiving elements) are arranged along the periphery of the dial of the wristwatch, and the plurality of solar cells E11 are arranged. The directions of visible light received by each of the above may be different from each other. Thereby, the information can be properly acquired by the wristwatch.

(Cooperation between wristwatch type receiver and smartphone)
FIG. 247 is a diagram illustrating an example of a reception system in Embodiment 10.

For example, the receiver 9022b configured as a wristwatch is connected to the smartphone 9022a and the glasses-type display 9022c via wireless communication such as Bluetooth (registered trademark). The receiver 9022b displays information indicating that the signal is received on the display 9022c when the signal is received or when it is confirmed that the signal exists. The receiver 9022b transmits the received signal (received signal) to the smartphone 9022a. The smartphone 9022a acquires the data associated with the received signal from the server 9022d and displays the acquired data on the glasses-type display 9022c.

(Directions by wrist watch type display)
FIG. 248 is a diagram illustrating an example of a reception system in Embodiment 10.

For example, the receiver 9023b configured as a wristwatch is connected to the smartphone 9022a via wireless communication such as Bluetooth (registered trademark). The receiver 9023b has a dial made of a liquid crystal display or the like, and can display information other than the time. The smartphone 9022a recognizes the current location from the signal received by the receiver 9023b, and displays the route and distance to the destination on the display surface of the receiver 9023b.

(Frequency shift modulation and frequency multiplexing modulation)
249A, 249B, and 249C are diagrams showing an example of a modulation method in Embodiment 10.

In FIG. 249A, (a) represents a specific signal as a specific modulation frequency. The receiving side performs frequency analysis of the light pattern (pattern of change in the brightness of the light source) to find the dominant modulation frequency, and restores the signal.

As shown in (a) of FIG. 249C, many values can be expressed by changing the modulation frequency with time. Since the image pickup frame rate of a general image sensor is 30 fps, continuous reception of one modulation frequency for 1/30 second or more enables reliable reception. In addition, as shown in (b) of FIG. 249C, when a frequency is changed, a time during which signals are not superimposed is provided, so that the receiver can easily recognize the change in the modulation frequency. The light pattern during the time when the signal is not superimposed can be distinguished from the signal superimposed portion by keeping the brightness constant or by setting a specific modulation frequency. If the specific modulation frequency used here is set to a frequency that is an integral multiple of 30 Hz, it is unlikely to appear in the difference image and it is less likely to interfere with the reception process. Reception is facilitated by setting the length of time during which the signals are not superimposed to be equal to or longer than that of the signal having the longest cycle in the optical pattern used for the signals. As an example, if the light pattern of the lowest modulation frequency is 100 Hz, the length of time during which signals are not superimposed is set to 1/100 second or more.

FIG. 249A (b) is an example (1) in which a specific bit and a specific modulation frequency are associated with each other, and an optical pattern is expressed as a waveform in which modulation frequencies whose corresponding bit is 1 are superimposed. Specifically, when transmitting the information in which the first bit is 1, the transmitter changes in luminance with an optical pattern of frequency f1=1000 Hz. Further, when transmitting the information in which the second bit is 1, the transmitter changes the luminance in the light pattern of the frequency f2=1100 Hz. When transmitting information in which the third bit is 1, the transmitter changes in luminance with an optical pattern of frequency f3=1200 Hz. Therefore, for example, when transmitting information including a bit string of “110”, the transmitter changes the brightness with the light pattern of the frequency f2 at the time T2 and the brightness with the light pattern of the frequency f1 at the time T1 longer than the time T2. Change. Further, for example, when transmitting information including a bit string of “111”, the transmitter changes the brightness with the light pattern of the frequency f2 at the time T2, and the brightness with the light pattern of the frequency f3 at the time T3 shorter than the time T2. Further, the luminance changes with the light pattern of the frequency f1 at time T1. In this case, a higher CN ratio (Carrier to Noise Ratio) is required as compared with the modulation method of (a), but more values can be expressed. In the example (1), when the number of bits that are turned on is large, that is, when the waveform includes many frequencies, the energy per frequency is small and a higher CN ratio is required. There is a problem.

Therefore, in the example (2) in which the light pattern is expressed, the number of frequencies included in the waveform is limited to a predetermined number or less, that is, the number of frequencies can be changed within a predetermined number or less. Alternatively, in the example (3) in which the light pattern is expressed, the number of frequencies included in the waveform is limited to a predetermined number. Thereby, the above-mentioned problem can be avoided. In the example (3), since the number of included frequencies is determined, the signal and the noise can be separated more easily than the examples (1) and (2), and the method is the most resistant to the noise. ing.

When a signal is expressed using n types of frequencies, 2 ⁿ −1 kinds of signals can be expressed in the example (1). Furthermore, when the number of types of frequencies is limited to m, in the example (2), (Σ(k=1 to _m ) _n C _k )−1 signals are expressed, and in the example (3), _n C _m signals are expressed. be able to.

As a method of superimposing a plurality of modulation frequencies, (i) a method of simply adding the respective waveforms, (ii) a method of weighted averaging the respective waveforms, and (iii) a waveform of each frequency There is a method to repeat in order. When frequency analysis such as discrete cosine series expansion is performed on the receiving side, the peak tends to become smaller as the frequency becomes higher. Therefore, in (ii), the peaks of the respective frequencies are adjusted and weighted so as to have the same magnitude. It is good to average. That is, the higher the frequency, the better the weight. In (iii), the output of the waveform of each frequency is not repeated once (one cycle at a time), but the ratio of the number of outputs (the number of cycles) is adjusted, so that the size of the frequency peak at the time of reception can be adjusted. Can be adjusted. The higher the frequency, the more the number of output cycles may be increased, or the higher the frequency, the longer the output time may be. By this adjustment, the size of the frequency peaks can be made uniform to facilitate the reception process, and by making the difference in the size of the frequency peaks meaningful, additional information can be expressed. For example, in the case where the order of the magnitude of the frequency peaks has meaning, if the number of frequencies included is n, the information amount of log ₂ (n!) bits can be added. The frequency may be changed every one cycle, the frequency may be changed every one cycle or every half cycle, the frequency may be changed at every constant multiple of the half cycle, or at every constant time. The frequency may be changed to. The timing of changing the frequency may be when the brightness becomes the highest, when it becomes the lowest, or when it becomes an arbitrary value. Flickering can be suppressed by equalizing the brightness before and after changing the frequency (=continuously changing the brightness). For that purpose, the frequency may be output for a time of an integral multiple of a half wavelength of each frequency to be transmitted. At this time, the output time of each frequency is different. Also, by outputting a signal of a certain frequency for a time that is an integral multiple of a half cycle, it is possible to easily recognize that the frequency is included in the signal on the receiving side by frequency analysis even in the case of digital output. You can Flickering may be more difficult to be perceived by human eyes or a camera when the same frequency is output continuously rather than continuously. For example, when outputting the cycle T ₁ twice, T ₂ twice, and T ₃ once, T ₁ T ₂ T ₃ T ₂ T rather than T ₁ T ₁ T ₂ T ₂ T _{3. 1} is better. Instead of repeating the output in a predetermined order, the output may be performed while changing the order. By giving meaning in this order, it is possible to express additional information. Although this order does not appear in the frequency peak, this information can be obtained by analyzing the order of the frequencies. Since it is necessary to set the exposure time shorter when analyzing the order of frequencies than when analyzing the frequency peaks, it is possible to set the exposure time shorter only when additional information is required. Only the receiver that can be set may be able to acquire this additional information.

FIG. 249B shows a case where the signal of FIG. 249A is expressed by a binary optical pattern. In the methods (i) and (ii) for overlapping frequencies, the analog waveform has a complicated shape, and even if the analog waveform is binarized, the complicated shape cannot be expressed. Therefore, the receiver cannot obtain an accurate frequency peak, and the reception error increases. In the method (iii), since the analog waveform does not have a complicated shape, the influence of binarization is small and a relatively accurate frequency peak can be obtained. Therefore, the method of (iii) is superior when using a light pattern digitized with a binary value or a small number of values. This modulation method can be interpreted as a type of frequency modulation by focusing on the fact that the signal is expressed at the frequency of the optical pattern, and the signal is expressed by adjusting the length of the pulse time width. Focusing on the point, it can be interpreted as a kind of PWM modulation.

By setting the unit of time when the brightness changes to a discrete value, it is possible to perform transmission/reception similarly to pulse modulation. Regardless of the length of the cycle of the frequency to be transmitted, the average luminance can be increased by setting the section having low luminance as the shortest time unit. At this time, the longer the cycle of the transmission frequency is, the higher the average brightness is. Therefore, the average brightness can be increased by increasing the number of outputs of the frequency having the long cycle. Even if the low-luminance section has the same length, the length of the high-luminance section is set to the period of the transmission frequency minus the length of the low-luminance section to perform frequency analysis. , A frequency peak appears in the transmission frequency. Therefore, even if the exposure time of the receiver is not set so short, the signal can be received by using the frequency analysis method such as the discrete cosine transform.

As shown in (c) of FIG. 249C, many values can be expressed by temporally changing the superposition of modulation frequencies as in (a) of FIG. 249C.

A signal with a high modulation frequency cannot be received unless the exposure time is set short, but it can be used up to a certain level of the modulation frequency without setting the exposure time. By transmitting signals modulated using frequencies from low modulation frequency to high modulation frequency, all terminals can receive signals expressed by low modulation frequency, and in the case of terminals that can set the exposure time short By receiving signals up to high modulation frequencies, can receive more information faster from the same transmitter. Alternatively, when a low frequency modulated signal is found in the normal imaging mode, the entire transmission signal including the high frequency modulated signal may be received in the visible light communication mode.

The frequency shift modulation method and the frequency multiplex modulation method have the effect of not causing the human eye to flicker even when using a lower modulation frequency than expressing the signal by the pulse position, so use many frequency bands. You can

It should be noted that the same effects can be obtained in the first to tenth embodiments even when the signal modulated by the reception system/modulation system described here is used.

(Separation of mixed signals)
249D and 249E are diagrams showing an example of separation of mixed signals in the tenth embodiment.

The receiver has the function of (a) of FIG. 249D. The light receiving section receives the light pattern. The frequency analysis unit maps the signal in the frequency domain by performing a Fourier transform on the light pattern. The peak detector detects the peak of the frequency component of the light pattern. If no peak is detected by the peak detector, the subsequent processing is interrupted. The peak time change analysis unit analyzes the time change of the peak frequency. The signal source identification unit identifies which combination of modulation frequencies of signals transmitted from the same transmitter when a plurality of frequency peaks are detected.

As a result, even when a plurality of transmitters are arranged near each other, reception can be performed while avoiding signal interference. In addition, when the light from the transmitter receives the light reflected from the floor, wall, ceiling, etc., the light from multiple transmitters is often mixed, but even in such a case, avoid signal interference. Can be received.

As an example, when the receiver receives an optical pattern in which the signal of the transmitter A and the signal of the transmitter B are mixed, a frequency peak as shown in (b) of FIG. 249D is obtained. Since fA1 disappears and fA2 appears, it can be specified that fA1 and fA2 are signals from the same transmitter. Similarly, it can be specified that fA1, fA2, and fA3 are signals from the same transmitter, and fB1, fB2, and fB3 are signals from the same transmitter.

By making the time interval in which one transmitter changes the modulation frequency constant, it is possible to easily identify a signal from the same transmitter.

When the timings at which the modulation frequencies of a plurality of transmitters change are equal, the above-described method cannot identify signals from the same transmitter. Therefore, by making the time interval for changing the modulation frequency of the transmitter different for each individual transmitter, it is possible to avoid the situation where the timings at which the modulation frequencies of multiple transmitters change are always the same. Will be able to identify the signal of.

As shown in (c) of FIG. 249D, by setting the time from the transmitter changing the modulation frequency to the next change to a value obtained from the current modulation frequency and the modulation frequency before the change, Even when a plurality of transmitters change the modulation frequency at the same timing, it is possible to specify which modulation frequency signal is transmitted from the same transmitter.

The transmitter may recognize the transmission signals of other transmitters and adjust so that the timings of the modulation frequency changes are not equal.

The above method is the same not only in the case of frequency shift keying modulation in which one transmission signal is composed of one modulation frequency, but also in the same method when one transmission signal is composed of a plurality of modulation frequencies. The effect of is obtained.

As shown in (a) of FIG. 249E, the signal from the same transmitter cannot be specified when the optical pattern is not temporally changed by the frequency multiplexing modulation method, but as shown in (b) of FIG. 249E. By including a section with no signal or changing to a specific modulation frequency, the signal from the same transmitter can be specified from the peak time change.

FIG. 249F is a flowchart illustrating processing of the information processing program in Embodiment 10.

This information processing program is a program for changing the brightness of the light emitter (or light emitting unit) of the transmitter described above in the light pattern shown in FIG. 249A (b) or FIG. 249B (b).

That is, this information processing program is an information processing program that causes a computer to process the information to be transmitted in order to transmit the information to be transmitted by a change in brightness. Specifically, this information processing program encodes the information of the transmission target to determine the frequency of the luminance change, and the light emitting body changes the luminance according to the determined frequency of the luminance change. The output step SA12 of outputting a signal indicating the determined frequency of the luminance change is executed by the computer so as to transmit the information to be transmitted. In the determining step SA11, the first frequency (for example, the frequency f1) and the second frequency different from the first frequency (for example, the frequency f2) are respectively determined as the frequency of the luminance change. In the output step SA12, the luminous body changes in luminance according to the first frequency in the first time (for example, time T1), and after the first time has elapsed, the second time (for example, time) different from the first time is elapsed. At T2), the signals indicating the first and second frequencies are output as signals indicating the determined frequency of the luminance change so that the luminance changes according to the second frequency.

Thereby, the information to be transmitted can be appropriately transmitted by the visible light signals of the first and second frequencies. Also, by making the first time and the second time different, it is possible to perform transmission adapted to various situations. As a result, communication between various devices can be enabled.

For example, as shown in FIGS. 249A and 249B, the first time is a time corresponding to one cycle of the first frequency. The second time is a time corresponding to one cycle of the second frequency.

Further, in the output step SA12, the signal indicating the first frequency and the signal indicating the second frequency are set so that the signals indicating the first frequency and the signal indicating the second frequency are output different times. At least one of and may be repeatedly output. As a result, it is possible to perform transmission that suits various situations.

Further, in the output step SA12, among the signals indicating the first and second frequencies, the signal indicating the first frequency is output so that the number of times the low frequency signal is output is greater than the number of times the high frequency signal is output. And at least one of the signals indicating the second frequency may be repeatedly output.

Accordingly, when the brightness of the light emitter changes according to the frequency indicated by each output signal, the light emitter can transmit the information of the transmission target with a bright brightness. For example, it is assumed that the change in brightness according to the first frequency, which is a low frequency, and the change in brightness according to the second frequency, which is a high frequency, have the same duration of dark brightness. In this case, in the brightness change according to the first frequency (that is, the low frequency), the bright brightness continues for a longer time than in the brightness change according to the second frequency (that is, the high frequency). Therefore, by outputting many signals indicating the first frequency, the light emitting body can transmit the information of the transmission target with bright brightness.

Further, in the output step SA12, among the signals indicating the first and second frequencies, the signal indicating the first frequency is set so that the number of times the high frequency signal is output is greater than the number of times the low frequency signal is output. And at least one of the signals indicating the second frequency may be repeatedly output. For example, as shown in FIGS. 249A and 249B, the frequency of outputting the signal of frequency f2 is larger than the frequency of outputting the signal of frequency f1.

As a result, when the luminance of the light emitter changes according to the frequency indicated by each output signal, it is possible to improve the reception efficiency of the information transmitted by the luminance change. For example, when information to be transmitted is transmitted to a receiver by a visible light signal expressed by a plurality of frequencies, the receiver performs frequency analysis such as Fourier transform on an image obtained by imaging. , The peak of the frequency included in the visible light signal is detected. At this time, the higher the frequency, the more difficult the peak detection. Therefore, as described above, among the signals indicating the first and second frequencies, each signal is output so that the number of times the high frequency signal is output is greater than the number of times the low frequency signal is output. , It is possible to facilitate high frequency peak detection. As a result, the reception efficiency can be improved.

Further, in the output step SA12, even if at least one of the signal indicating the first frequency and the signal indicating the second frequency is repeatedly output so that the signals indicating the same frequency are not continuously output. Good. For example, as shown in FIGS. 249A and 249B, the signal indicating the frequency f1 is not continuously output, and the signal indicating the frequency f2 is not continuously output.

Accordingly, when the luminance of the light emitting body changes according to the frequency indicated by each output signal, it is possible to make it difficult for human eyes or a camera to catch the flicker of light from the light emitting body.

FIG. 249G is a block diagram of an information processing device in Embodiment 10.

The information processing apparatus A10 is an apparatus for changing the brightness of the light emitter of the transmitter described above with the light pattern shown in FIG. 249A (b) or FIG. 249B (b).

That is, the information processing apparatus A10 is an apparatus that processes the information to be transmitted in order to transmit the information to be transmitted by changing the brightness. Specifically, the information processing apparatus A10 encodes the information of the transmission target to change the luminance according to the frequency determining unit A11 that determines the frequency of the luminance change and the light emitting body according to the determined frequency of the luminance change. Thus, the output unit A12 that outputs the signal indicating the determined frequency of the luminance change is provided so as to transmit the information of the transmission target. Here, the frequency determination unit A11 determines the first frequency and the second frequency different from the first frequency as the frequency of the luminance change. The output unit A12 is configured such that the luminous body changes in luminance according to the first frequency at the first time, and after the lapse of the first time, the luminance changes according to the second frequency at the second time different from the first time. The signals indicating the first and second frequencies are output as the signals indicating the determined frequency of the luminance change so as to change. In such an information processing apparatus A10, the same effects as those of the above information processing program can be obtained.

(Operation of home appliances through lighting by visible light communication)
FIG. 250A is a diagram showing an example of a visible light communication system in Embodiment 10.

For example, a transmitter configured as a ceiling lighting (illumination device) has a wireless communication function such as Wi-Fi or Bluetooth (registered trademark). The transmitter transmits, by visible light communication, information (light emitting device ID, authentication ID, etc.) for connecting to the transmitter by wireless communication. For example, the receiver A configured as a smartphone (mobile terminal) performs wireless communication with the transmitter based on the received information. Receiver A may connect to the transmitter using other information, in which case it may not have a receiving function. The receiver B is configured as an electronic device (control target device) such as a microwave oven. The transmitter transmits the information of the paired receiver B to the receiver A. The receiver A displays the information of the receiver B as an operable device. The receiver A transmits an operation command (control signal) of the receiver B to the transmitter through wireless communication, and the transmitter transmits the operation command to the receiver B through visible light communication. Accordingly, the user can operate the receiver B via the receiver A. A device connected to the receiver A via the Internet or the like can operate the receiver B via the receiver A.

Since the receiver B has a transmission function and the transmitter has a reception function, bidirectional communication can be performed. The transmission function may be realized as visible light by light emission, or communication may be performed by sound. For example, the transmitter includes a sound collecting unit, and by recognizing the sound emitted by the receiver B, the state of the receiver B can be recognized. For example, it is possible to notify the user by recognizing the operation end sound of the receiver B and transmitting it to the receiver A, and the receiver A displaying the operation end of the receiver B on the display.

The receiver A and the receiver B are equipped with NFC. The receiver A receives the signal from the transmitter and then communicates with the receiver B via NFC so that the receiver B can receive the signal from the transmitter that transmitted the signal received immediately before. It is registered in the receiver A and the transmitter. This is called pairing of the transmitter and the receiver B. When the receiver B is moved, the receiver A registers the pairing cancellation in the transmitter. If receiver B is paired with another transmitter, the newly paired transmitter conveys that information to the previously paired transmitter and removes the previous pairing.

FIG. 250B is a diagram for explaining a use case in the tenth embodiment. With reference to FIG. 250B, an embodiment will be described in the case where the receiving unit 1028 using the modulation method such as the PPM method, the FDM method, the FSK method, or the frequency allocation method of the present invention is used.

First, the light emitting operation of the light emitting device 1003 which is a lighting device will be described. In the light emitting device 1003 such as a lighting fixture or a TV monitor attached to a ceiling or a wall, the authentication ID generating unit 1010 first generates an authentication ID using the random number generating unit 1012 that changes with time. For the ID of the light emitting device 1003 and the authentication ID 1004, if there is no interrupt process (step 1011), the light emitting device 1003 determines that there is no “transmission data string” sent from the mobile terminal 1020, and (1) emits light. A machine ID, (2) an authentication ID, and an identifier for identifying whether or not there is a transmission data string 1009 transmitted from the electronic device 1040 which is a control target device via the mobile terminal 1020, that is, (3) a transmission data string An optical signal is sent to the outside from the light emitting unit 1016 such as an LED for three flags of 0 continuously or intermittently.

The sent optical signal is received by the photosensor 1041 of the electronic device 1040 (step 1042), and the electronic device 1040 determines the device ID and the authentication ID (device authentication ID and light emitter ID) of the electronic device 1040 in step 1043. Check if it is genuine. If the confirmation result is YES (regular), the electronic device 1040 checks whether or not the transmission data string flag=1 (step 1051). Only when the check result is YES (transmission data string flag=1), the electronic device 1040 executes the data of the transmission data string, for example, the user command such as the recipe execution of the dish (step 1045).

Here, the mechanism of optical transmission of the electronic device 1040 using the optical modulation method of the present invention will be described. The electronic device 1040 displays the device ID, the authentication ID for authenticating the device, and the light emitting device ID of the light emitting device 1003 received by the electronic device 1040 as described above, that is, the light receiving device 1003 can reliably receive the device ID, on the display unit 1047. It is sent using the LED backlight unit 1050 and the like (step 1046).

Since the optical signal of the present invention is transmitted by the PPM, FDM, or FSK system at a modulation frequency of 60 Hz or more, which is flicker-free, from the display unit 1047 such as a liquid crystal of a microwave oven or a POS machine, the optical signal is transmitted to general consumers. I do not know that the signal is being sent. Therefore, the display unit 1047 can independently display, for example, a menu of a microwave oven.

(ID detection method of the light emitting device 1003 that can be received by the electronic device 1040)
A user who intends to use a microwave oven or the like receives an optical signal from the light emitting device 1003 at the in-camera unit 1017 of the mobile terminal 1020, and receives the light emitting device ID and the light emitting device authentication ID at the in camera processing unit 1026. (Step 1027). The light emitting device ID that can be received by the electronic device 1040 is the light emitting device ID existing at the position recorded in the cloud 1032 or the mobile terminal, as well as the position information using the radio wave of the mobile phone such as 3G or Wi-Fi. It may be detected (step 1025).

When the user points the out-camera 1019 of the mobile terminal 1020 toward the display unit 1047 of the microwave oven (electronic device) 1040, the optical signal 1048 of the present invention can be demodulated using the MOS camera. You can receive faster data by increasing the shutter speed. The receiving unit 1028 receives the device ID, the authentication ID, the service ID of the electronic device 1040, or the URL of the cloud for service provision converted from the service ID and the device status.

In step 1029, the 3G Wi-Fi communication unit 1031 connects to the cloud 1032 inside the terminal or outside using the received URL and sends the service ID and the device ID. The cloud 1032 searches the database 1033 for data corresponding to the device ID and the service ID, and sends the data to the mobile terminal 1020. Based on this data, video data, command buttons, etc. are displayed on the screen of the mobile terminal. The user who sees this inputs a desired command by an input method such as pressing a button on the screen (step 1030). In the case of YES (input), the transmitter 1022 of the BTLE (Blue Tooth (registered trademark) Low Energy) transmitting/receiving unit 1021 is the device ID of the electronic device 1040 or the like, the device authentication ID, the light emitter ID, the light emitter authentication ID, and the step. A transmission data string composed of 1030 user commands and the like is transmitted.

In the light emitting device 1003, the reception unit 1007 of the BTLE transmission/reception unit 1004 receives the “transmission data string”, and the interrupt processing unit 1011 detects that the “transmission data string” is received (YES in step 1013), and “ The data of “transmission data string+ID+transmission data flag=1” is modulated by the modulation unit of the present invention, and is optically transmitted from the light emitting unit 1016 such as an LED. When the reception of the “transmission data string” is not detected (NO in step 1013), the light emitting device 1003 continuously sends the light emitting device ID and the like.

As described above, this electronic device 1040 has already received and confirmed that it can already receive the signal from the light emitting device 1003, so that it can be reliably received.

In this case, since the interrupt processing unit 1011 includes the light emitting device ID in the transmission data string, it can be seen that the electronic device to be transmitted exists within the light irradiation range of the light emitting device of the ID. Therefore, the signal can be sent only from the light emitting device in an extremely narrow position where the electronic device is located, without sending a signal from another light emitting device, so that the radio wave space can be efficiently used.

If this method is not adopted, the Bluetooth signal will reach far, and an optical signal will be sent from a light emitting device at a position different from the electronic device. Therefore, during a light emission period, a certain light emitting device cannot transmit or interferes with optical transmission to another electronic device which wants to transmit, and this solution is effective.

Next, countermeasures for malfunction of electronic equipment will be described.

The photo sensor 1041 receives the optical signal in step 1042. First, since the light emitting device ID is checked, the light emission signal of another light emitting device ID can be removed, so that malfunction is reduced.

In the present invention, the transmission data string 1009 includes the device ID and the device authentication ID of the electronic device to be received. Therefore, in step 1043, it is checked whether the device authentication ID and the device ID are the IDs of the electronic device 1040, and no malfunction occurs. There is an effect that it is possible to prevent a malfunction of a microwave oven or the like caused by the electronic device 1040 erroneously processing a signal transmitted to another electronic device.

A method for preventing malfunction of execution of user commands will be described.

When the transmission data flag=1 in step 1044, it is determined that there is a user command, and when the transmission data flag=0, the process is stopped. When the transmission data flag=1, the transmission data sequence such as the user command is executed after authenticating the device ID and the authentication ID of the user data sequence. For example, the electronic device 1040 retrieves the recipe and displays it on the screen. When the user presses the button, the recipe, that is, 600w for 3 minutes, 200w for 1 minute, and steam cooking for 2 minutes are started without malfunction.

When a user command is executed, in the case of a microwave oven, electromagnetic noise of 2.4 GHz is generated. In order to reduce this, when receiving a command via Bluetooth or Wi-Fi via a smartphone to operate, the intermittent drive unit 1061 intermittently stops the output of the microwave oven, for example, about 100 ms in 2 seconds. During this time, communication such as Bluetooth or Wi-Fi 802.11n becomes possible. For example, when the range is not stopped, sending a stop command from the smartphone to the light emitting device 1003 by BTLE is hindered. On the other hand, according to the present invention, the signal can be transmitted without being affected by the interference wave, and the range can be stopped or the recipe of the range can be changed by the optical signal.

The feature of the present embodiment is that the photo sensor 1041 having a cost of about a few yen is added to the electronic device with the display unit to enable the bidirectional communication with the smartphone linked with the cloud. It has the effect of being installed in white goods and becoming a smart home appliance. However, although the white goods are used as the embodiment, a POS terminal with a display unit, a supermarket electronic price tag, or a personal computer can achieve the same effect.

Further, in this embodiment, the light emitting device ID can be received only from the illuminator on the upper part of the electronic device. Since the receiving area is narrow, a small zone ID such as Wi-Fi is defined for each light emitting device, and the following IDs at the positions in each zone are assigned to reduce the number of digits of the ID of the light emitting unit. is there. In this case, the number of digits of the ID of the light emitting device that transmits using the above PPM, FSK, or FDM of the present invention is reduced, so that an optical signal can be received from a small light source, an ID can be acquired quickly, or a distant light source can be used. There is an effect that the data of can be received.

FIG. 250C is a diagram showing an example of a signal transmission/reception system in Embodiment 10.

The signal transmission/reception system includes a smartphone (smartphone) that is a multifunctional mobile phone, an LED light emitting device that is a lighting device, a home electric appliance such as a refrigerator, and a server. The LED light emitting device performs communication using BTLE (Bluetooth (registered trademark) Low Energy) and visible light communication using an LED (Light Emitting Diode). For example, the LED light emitter controls a refrigerator or communicates with an air conditioner by BTLE. Further, the LED light emitting device controls a power source of a microwave oven, an air purifier, a television (TV), or the like by visible light communication.

The television includes, for example, a solar power generation element and uses this solar power generation element as an optical sensor. That is, when a signal is transmitted due to a change in the brightness of the LED light emitting device, the television detects a change in the brightness of the LED light emitting device due to a change in the power generated by the solar power generation element. Then, the television acquires the signal transmitted from the LED light emitting device by demodulating the signal indicated by the detected luminance change. When the signal is a command indicating power ON, the television switches its main power ON, and when the signal is a command power OFF, it switches its main power OFF.

In addition, the server can communicate with the air conditioner via the router and the specified low power radio station (special size). Further, the air conditioner can communicate with the LED light emitters via BTLE, so the server can communicate with the LED light emitters. Therefore, the server can switch the power of the TV between ON and OFF via the LED light emitting device. In addition, the smartphone can control the power supply of the TV via the server by communicating with the server via, for example, Wi-Fi (Wireless Fidelity).

As illustrated in FIGS. 250A to 250C, in the information communication method according to the present embodiment, a mobile terminal (smartphone) uses a wireless communication (BTLE, Wi-Fi, or the like) different from visible light communication to perform a control signal (transmission data string). A wireless communication step of transmitting a user command) to the lighting device (light emitting device), a visible light communication step of performing visible light communication by the lighting device changing in luminance according to the control signal, and a control target device (electronic device). Range) detects a change in the brightness of the lighting device, acquires a control signal by demodulating a signal specified by the detected change in the brightness, and executes a process according to the control signal. Including. As a result, even if the mobile terminal cannot change the brightness for visible light communication, it can change the brightness of the lighting device instead of the mobile terminal by wireless communication, and appropriately control the control target device. can do. The mobile terminal may be a wristwatch instead of a smartphone.

(Reception without interference)
FIG. 251 is a flowchart showing a reception method with interference removed according to the tenth embodiment.

First, in step 9001a, start is checked, and in step 9001b, it is confirmed whether or not the intensity of the received light has a periodic change. If YES, the process proceeds to step 9001c. If NO, the process proceeds to step 9001d, the lens of the light receiving unit is set to a wide angle to receive a wide range of light, and the process returns to step 9001b. In step 9001c, it is confirmed whether or not the signal can be received. If YES, the process proceeds to step 9001e, the signal is received, and the process ends in step 9001g. If NO, the process proceeds to step 9001f, the lens of the light receiving unit is set to the telephoto position to receive light in a narrow range, and the process returns to step 9001c.

By this method, it is possible to receive signals from transmitters in a wide direction while eliminating interference of signals from a plurality of transmitters.

(Estimation of transmitter direction)
FIG. 252 is a flowchart showing a method for estimating the azimuth of a transmitter in the tenth embodiment.

First, in step 9002a, start is performed, in step 9002b, the lens of the light receiving unit is set to the maximum telephoto, and in step 9002c, it is confirmed whether there is a periodic change in the intensity of the received light. Proceed to 9002d. If NO, the process proceeds to step 9002e, the lens of the light receiving unit is set to a wide angle to receive a wide range of light, and the process returns to step 9002c. In step 9002d, the signal is received, in step 9002f, the lens of the light receiving unit is set to the maximum telephoto position, the light receiving direction is changed along the boundary of the light receiving range, and the direction in which the light receiving intensity is maximized is detected, and the transmitter transmits the signal. It is estimated that the vehicle is in the direction, and the process ends in step 9002d.

By this method, the direction in which the transmitter is present can be estimated. Note that the maximum wide angle may be set first, and then the telephoto may be gradually set.

(Start reception)
FIG. 253 is a flowchart showing a reception start method in Embodiment 10.

First, in step 9003a, start is performed, and in step 9003b, it is confirmed whether a signal from a base station such as Wi-Fi, Bluetooth (registered trademark) or IMES is received, and if YES, the process proceeds to step 9003c. If NO, the process returns to step 9003b. In step 9003c, the base station confirms whether or not the base station is registered in a receiver or a server as a reception start trigger. If YES, the process proceeds to step 9003d to start signal reception and ends in step 9003e. .. If NO, the process returns to step 9003b.

With this method, reception can be started without the user performing a reception start operation. In addition, power consumption can be suppressed as compared with the case where reception is always performed.

(Generation of ID using information from other media)
FIG. 254 is a flowchart showing an ID generation method using information of another medium together in the tenth embodiment.

First, start in step 9004a, and the ID of the carrier communication network, Wi-Fi, Bluetooth (registered trademark) or the like connected in step 9004b, or the position information obtained from the above ID or the position information obtained from GPS or the like. Is transmitted to the high-order bit ID index server. In step 9004c, the upper bits of the visible light ID are received from the upper bit ID index server, and in step 9004d, the signal from the transmitter is received as the lower bits of the visible light ID. In step 9004e, the upper bits and the lower bits of the visible light ID are combined and transmitted to the ID resolution server, and the processing ends in step 9004f.

By this method, it is possible to obtain high-order bits commonly used in a place near the receiver and reduce the amount of data transmitted by the transmitter. Also, the receiving speed of the receiver can be increased.

Note that the transmitter may be transmitting both the upper bits and the lower bits. In this case, the receiver using this method can synthesize the ID when the lower bits are received, and the receiver not using this method receives the entire ID from the transmitter to obtain the ID. ..

(Reception method selection by frequency separation)
FIG. 255 is a flowchart showing a method of selecting a reception system by frequency separation according to the tenth embodiment.

First, a start is performed in step 9005a, and the optical signal received in step 9005b is applied to a frequency filter circuit, or discrete Fourier series expansion is performed to perform frequency decomposition. In step 9005c, it is confirmed whether or not a low frequency component exists. If YES, the process proceeds to step 9005d, the signal expressed in the low frequency region such as frequency modulation is decoded, and the process proceeds to step 9005e. If NO, the process proceeds to step 9005e. In step 9005e, the base station confirms whether or not the base station is registered in the receiver or the server as a reception start trigger. If YES, the process proceeds to step 9005f, and a signal expressed in a high frequency region such as pulse position modulation. Is decoded and the process proceeds to step 9005g. If NO, the process proceeds to step 9005g. Reception of a signal is started in step 9005g and ended in step 9005h.

With this method, it is possible to receive signals modulated by a plurality of modulation methods.

(Signal reception when exposure time is long)
FIG. 256 is a flowchart showing a signal receiving method when the exposure time is long in the tenth embodiment.

First, start is performed in step 9030a, and if the sensitivity can be set in step 9030b, the sensitivity is set to the highest. If the exposure time can be set in step 9030c, it is set to a time shorter than that in the normal shooting mode. In step 9030d, two images are picked up to obtain the difference in brightness. When the position or direction of the image capturing unit changes while capturing two images, the change is canceled and an image as if the image was captured from the same position and direction is generated to obtain the difference. In step 9030e, the average value of the brightness values in the direction parallel to the exposure line of the difference image or the captured image is obtained. In step 9030f, the averaged values are arranged in the direction perpendicular to the exposure line and discrete Fourier transform is performed. In step 9030g, it is recognized whether or not there is a peak near a predetermined frequency, and the process ends in step 9030h.

By this method, the signal can be received even when the exposure time is long, such as when the exposure time cannot be set or when a normal image is simultaneously captured.

When the exposure time is automatically set, the exposure time is set to about 1/60 second to 1/480 second by the automatic exposure correction function when the camera is turned to the transmitter configured as the illumination. When the exposure time cannot be set, the signal is received under this condition. In the experiment, when the illumination is periodically blinked, if one cycle time is about 1/16 or more of the exposure time, stripes can be visually recognized in a direction perpendicular to the exposure line, and the blinking cycle is determined by image processing. I was able to recognize it. At this time, since the luminance of the portion where the illumination is reflected is too high and the stripes are hard to be confirmed, it is preferable to obtain the signal period from the portion where the illumination light is reflected.

When using a method that periodically turns on and off the light-emitting part, such as a frequency shift modulation method or a frequency multiplex modulation method, even if the same modulation frequency is used, it is more The flicker is hard to see, and the flicker does not easily appear in the moving images shot by the video camera. Therefore, a low frequency can be used as the modulation frequency. Since the time resolution of human vision is about 60 Hz, a frequency higher than this frequency can be used as the modulation frequency.

Note that when the modulation frequency is an integer multiple of the image capture frame rate of the receiver, pixels at the same position in the two images are imaged when the light patterns of the transmitter are in the same phase, so a bright line appears in the difference image. It is difficult to receive. Since the image pickup frame rate of the receiver is usually 30 fps, setting the modulation frequency to a value other than an integral multiple of 30 Hz facilitates reception. In addition, since there are various image-capturing frame rates of receivers, two relatively disjoint modulation frequencies are assigned to the same signal, and the transmitter transmits by alternately using the two modulation frequencies. The machine can easily recover the signal by receiving at least one signal.

FIG. 257 is a diagram illustrating an example of a dimming (adjusting brightness) method of the transmitter.

By adjusting the ratio of the high brightness section and the low brightness section, the average brightness changes and the brightness can be adjusted. At this time, the frequency peak can be kept constant by keeping the period T ₁ for repeating the high and low luminance constant. For example, in any of (a), (b), and (c) of FIG. 257, transmission is performed while keeping the time T1 between the first brightness change brighter than the average brightness and the second brightness change constant. When dimming the machine, make the time to illuminate brighter than the average brightness. On the other hand, when dimming the transmitter brightly, the time for illuminating the transmitter brighter than the average brightness is set longer. 257 (b) and (c) of FIG. 257 are dimmer darker than (a), and (c) of FIG. 257 is the darkest dimmer. Thereby, dimming can be performed while transmitting signals having the same meaning.

The average brightness may be changed by changing the brightness of the high brightness section, the brightness of the low brightness section, or both of the brightness values.

FIG. 258 is a diagram illustrating an example of a method of configuring the dimming function of the transmitter.

Since the accuracy of the components is limited, even if the same dimming setting is made, the brightness is slightly different from that of another transmitter. However, when the transmitters are arranged side by side, it is unnatural if the brightness of the adjacent transmitters is different. Therefore, the user operates the dimming correction operation unit to adjust the brightness of the transmitter. The dimming correction unit holds the correction value, and the dimming control unit controls the brightness of the light emitting unit according to the correction value. When the user operates the dimming operation unit to change the degree of dimming, the dimming control unit uses the changed dimming setting value and the correction value held in the dimming correction unit as a basis. Then, the brightness of the light emitting unit is controlled. Further, the dimming control unit transmits the dimming setting value to another transmitter through the interlocking dimming unit. When the dimming setting value is transmitted from another device through the interlocking dimming unit, the dimming control unit determines the light emitting unit based on the dimming setting value and the correction value stored in the dimming correction unit. Control the brightness of.

According to one embodiment of the present invention, there is provided a control method for controlling an information communication device that transmits a signal by changing a luminance of a light emitter, the control method including a plurality of different signals to a computer of the information communication device. , A determination step of determining a pattern of luminance change of different frequency for each different signal by modulating a signal to be transmitted, and a luminance change of a single signal modulated at a time corresponding to a single frequency. And a transmission step of transmitting a signal to be transmitted by changing the luminance of the light emitter so as to include only the pattern.

For example, when a time corresponding to a single frequency includes a brightness change pattern obtained by modulating a plurality of signals, the waveform of the brightness change over time becomes complicated and it becomes difficult to properly receive the waveform. However, by controlling so as to include only the brightness change pattern obtained by modulating a single signal at a time corresponding to a single frequency, it becomes possible to perform reception more appropriately when receiving.

According to an embodiment of the present invention, the determining step determines that the number of transmissions of one of the plurality of different signals is different from the number of transmissions of another signal within a predetermined time. May determine the number of transmissions.

Since the number of times of transmitting one signal is different from the number of times of transmitting another signal, it is possible to prevent flicker during transmission.

According to one embodiment of the present invention, the determining step may cause the number of times of transmission of a signal corresponding to a high frequency to be larger than the number of transmissions of other signals within a predetermined time.

When frequency conversion is performed on the receiving side, the luminance of a signal corresponding to a high frequency decreases, but by increasing the number of transmissions, it is possible to increase the luminance value during frequency conversion.

According to one embodiment of the present invention, the brightness change pattern may be a pattern in which the waveform of the brightness change over time is a rectangular wave, a triangular wave, or a sawtooth wave.

By using a rectangular wave or the like, it becomes possible to perform reception more appropriately.

According to one embodiment of the present invention, when increasing the value of the average luminance of the luminous body, in the time corresponding to a single frequency, the time when the luminance of the luminous body becomes larger than a predetermined value, The value of the average brightness of the light emitting body may be increased as compared with the case of decreasing the value.

By adjusting the time during which the brightness of the light emitter becomes larger than a predetermined value in the time corresponding to a single frequency, it becomes possible to transmit a signal and adjust the average brightness of the light emitter. For example, when the light emitting body is used as illumination, it becomes possible to transmit a signal while making the overall brightness darker or brighter.

The receiver can set the exposure time to a predetermined value by using an API (abbreviation of application programming interface, which means means for using the function of the OS) for setting the exposure time. The visible light signal can be stably received. Further, the receiver can set the sensitivity to a predetermined value by using the API for setting the sensitivity, and stably receives the visible light signal even when the brightness of the transmission signal is dark or bright. be able to.

(Embodiment 11)
In the present embodiment, each application example using the receiver such as the smartphone in each of the above embodiments and the transmitter that transmits information as the blinking pattern of the LED or the organic EL will be described.

(Exposure time setting)
259A to 259D are a flowchart illustrating an example of operation of a receiver in Embodiment 11.

In order for the image sensor to receive a visible light signal using the method of the present invention, it is necessary to set the exposure time to a time shorter than a predetermined time. The predetermined time is determined by the modulation method and the modulation frequency of the visible light signal. Generally, the higher the modulation frequency, the shorter the exposure time needs to be.

As the exposure time is shortened, bright lines can be clearly observed. On the other hand, when the exposure time is shortened, the amount of received light is reduced, and the entire captured image becomes dark. That is, the signal strength is attenuated. Therefore, the reception performance (reception speed and error rate) can be improved by shortening the exposure time within the range in which the presence of a visible light signal can be detected.

As shown in FIG. 259A, the receiver first sets the imaging mode to the visible light imaging mode (step S9201). At this time, the receiver determines whether or not it has a monochrome imaging function and receives a signal modulated only with luminance information (step S9202). Here, when it is determined that the signal having the monochrome imaging function is received and the signal modulated only by the luminance information is received (Y in step S9202), the receiver sets the mode regarding the color in the imaging mode to the monochrome imaging function. The monochrome imaging mode using is set (step S9203). As a result, when a signal modulated by only luminance information is received, that is, when a visible light signal representing information by only luminance change is received, the processing speed can be improved by not handling color information. it can. On the other hand, in step S9202, when it is determined that the monochrome image pickup function is present and the signal modulated only by the luminance information is not received (N in step 9202), that is, the visible light is utilized by using the color information. When the signal is represented, the receiver sets the color mode of the image capturing modes to the color image capturing mode (step S9204).

Next, the receiver determines whether or not the image pickup unit including the above-described image sensor has the function of designating the exposure time (step S9205). Here, if it is determined that it is provided (Y in step S9205), the receiver sets the exposure time to a time shorter than the above-described predetermined time by using the function so that the bright line appears in the captured image. (Step S9206). It should be noted that the receiver may set the exposure time such that the exposure time is as short as possible within the range in which the transmitter transmitting the visible light signal can be seen in the captured image.

On the other hand, in step S9205, when it is determined that the function for designating the exposure time is not provided (N in step S9205), the receiver further determines whether or not the imaging unit is provided with the function for setting the sensitivity (step S9205). Step S9207). Here, when it is determined that the function for setting the sensitivity is provided (Y in step S9207), the receiver sets the sensitivity to the maximum by the function (step S9208). As a result, when a captured image is obtained by capturing an image with the maximum sensitivity, the captured image becomes bright. Therefore, in the receiver, if the automatic exposure is set to be effective, the exposure time is set to be short so that the exposure falls within a certain range by the automatic exposure. Note that in the automatic exposure, the captured image obtained by the image capturing is used for the input of the automatic exposure every time the image capturing is performed, and the exposure time is set based on the captured image so that the exposure falls within a certain range. Adjusted from time to time. Details of automatic exposure will be described later.

Further, the receiver determines whether or not the imaging unit has a function of setting the F value (aperture) (step S9209). If it is determined that the receiver has a function of setting the F value (Y in step S9209), the receiver sets the F value to the minimum (opens the aperture) by the function (step S9210). As a result, when a captured image is obtained by capturing an image with the minimized F value, the captured image becomes bright. Therefore, in the receiver, if the automatic exposure is set to be effective, the exposure time is set to be short so that the exposure falls within a certain range by the automatic exposure.

Further, the receiver determines whether or not the image pickup unit has a function of designating the value of the exposure correction (step S9211). If it is determined that the function for designating the value of the exposure correction is provided (Y in step S9211), the receiver sets the value of the exposure correction to the minimum by the function (step S9212). As a result, in the receiver, if automatic exposure is set to be effective, the automatic exposure sets a short exposure time in order to lower the exposure.

A scene mode for capturing an image of a subject moving at high speed (high-speed scene mode) is generally defined by the names of “sport” and “action”.

As shown in FIG. 259B, in the receiver, after step S9211 or step S9212, the imaging unit has a function of setting the high speed scene mode, and the sensitivity is set to the scene mode setting by setting the high speed scene mode. Determines whether or not the condition that the F value is set lower than before, the F value is set higher than before setting the scene mode, and the exposure correction value is not set higher than before setting the scene mode is satisfied. Yes (step S9213). If it is determined that the above condition is satisfied (Y in step S9213), the receiver sets the scene mode to the high-speed scene mode (step S9214). As a result, in the receiver, if the automatic exposure is set to be effective, the exposure time is set to be short by the automatic exposure in order to capture an image of a subject moving at high speed without blurring.

Next, the receiver sets the automatic exposure to be valid (step S9215) and images the subject (step S9216).

As shown in FIG. 259C, after step S9215, the receiver determines whether or not the image capturing unit has a zoom function (step S9217). Here, when it is determined that the zoom function is provided (Y in step S9217), the receiver further determines whether or not the center position of the zoom can be designated, that is, the center position is arbitrarily set in the captured image. It is determined whether or not the position can be set to (step S9218). When it is determined that the center position can be designated (Y in step S9218), the receiver designates the center position of the zoom in the bright portion in the captured image, so that the subject corresponding to the bright portion becomes large in the center. Zoom is performed so as to be imaged (step S9219). On the other hand, when it is determined that the center position cannot be specified (N in step S9218), the receiver determines whether the center of the captured image is brighter than the brightness of the predetermined value, or the predetermined each in the captured image. It is determined whether or not it is brighter than the average of the brightness at the location (step S9220). Here, if it is determined that the image is bright (Y in step S9220), the receiver zooms (step S9221). That is, also at this time, the subject corresponding to the bright portion is largely imaged in the center.

In general, most devices equipped with an image capturing section use center-weighted metering as the light metering method. Therefore, if there is a bright portion in the center of a captured image, the bright portion will not be specified even if the light measuring position is not designated at another position. The exposure is adjusted based on. As a result, the exposure time is set short. Further, since the area of the bright portion is increased by zooming and the exposure is adjusted on the basis of the brighter screen, the exposure time is set to be short.

Next, the receiver determines whether or not it has a function of designating the photometric position or the focus position (step S9222). Here, if it is determined that the function is present (Y in step S9222), the receiver performs processing for finding a bright place in the captured image. That is, the receiver performs processing for finding the location of the area having a predetermined shape and size, which is brighter than the predetermined brightness, from the captured image. Specifically, the receiver first determines whether or not the formula for exposure evaluation for automatic exposure is known (step S9224). If it is determined that the area is known (Y in step S9224), the receiver evaluates the brightness of each area in the captured image by using the same calculation expression as the known calculation expression. Find the location of the bright area (step S9226). On the other hand, when it is determined that the calculation formula of the exposure evaluation is unknown (N in step S9224), the receiver calculates the average value of the brightness of each pixel in the region of the predetermined shape and size. The location of the bright area is found by evaluating the brightness of each area in the captured image using a calculation formula (step S9225). The predetermined shape is, for example, a rectangle, a circle, or a cross shape. Further, the area may be composed of a plurality of non-continuous areas. Further, the above-described average value may be calculated by using a weighted average in which a greater weight is given to the central portion, instead of a simple average.

The receiver determines whether the total area of all the bright areas found is smaller than a predetermined area (step S9227). Here, if it is determined to be small (Y in step S9227), the receiver performs zooming so that the total area of the bright region is imaged in a size equal to or larger than a predetermined area (step S9228). Next, the receiver determines whether or not the photometric position can be designated (step S9229). When it is determined that the photometric position can be designated (Y in step S9229), the receiver designates the location of the brightest region as the photometric position (step S9230). In the automatic exposure, the exposure is adjusted according to the brightness of the photometric position. Therefore, by designating the location of the brightest area as the photometric position, the exposure time is set shorter by the automatic exposure. On the other hand, if it is determined in step S9229 that the photometric position cannot be specified (N in step S9229), that is, if the focus position can be specified, the receiver specifies the location of the brightest region as the focus position. Various image pickup units can be mounted on the receiver, and in the automatic exposure of some image pickup units of the various image pickup units, the exposure is adjusted by the brightness of the focus position. Therefore, by designating the place of the brightest region as the focus position, the exposure time is set to be short by the automatic exposure. The location specified here may be different from the location of the area used for the brightness evaluation or calculation, and is set according to the setting method of the imaging unit. As an example, if the specified format is a format that specifies the center point, the receiver specifies the center of the brightest area, and if the specified format is a format that specifies a rectangular area, the receiver is the most Specifies a rectangular area that includes the center of the bright area.

Next, the receiver determines whether or not there is an area brighter than the area of the place designated as the photometric position or the focus position in the captured image (step S9232). Here, if it is determined that the area exists (Y in step S9232), the receiver repeatedly executes the processing from step S9217. On the other hand, if it is determined that there is no such area (N in step S9232), the receiver images the subject (step S9233).

Next, the receiver determines whether or not the automatic exposure should be ended based on the captured image obtained by the image capturing in step S9233, or whether or not a predetermined time has elapsed after the automatic exposure was set to be valid. Yes (step S9234). Here, for example, when it is determined that the automatic exposure should not be ended (N in step S9234), the receiver further determines the position or the imaging direction of the imaging unit based on the captured image obtained by the imaging in step S9233. It is determined whether it has changed (step S9235). When it is determined that the position of the imaging unit or the imaging direction has changed (Y of step S9235), the receiver performs the processing from step S9217 again. As a result, even when the location designated as the photometric position or the focus position moves in the captured image, the location of the brightest region at that time can be designated again. On the other hand, if it is determined in step S9235 that the position of the imaging unit or the imaging direction has not changed (N in step S9235), the receiver repeats the processing from step S9232. Note that the receiver may search for the brightest area each time one captured image is acquired by capturing an image and respecify the photometric position and the focus position.

Further, in step S9234, when it is determined that the automatic exposure should be ended, when the exposure time does not change, or when it is determined that the fixed time has elapsed (Y in step S9234), or the exposure is performed in step S9206. When the time is set, the receiver sets the automatic exposure to invalid (step S9236) and images the subject (step S9237). Then, the receiver determines whether or not a visible light signal has been received by the imaging (step S9238). Here, when it is determined that the visible light signal is not received (N in step S9238), the receiver further determines whether or not a predetermined time has elapsed (step S9239). When it is determined that the predetermined time has not elapsed (N in step S9239), the receiver repeats the processing from step S9237. On the other hand, when it is determined that the predetermined time has elapsed (Y in step S9239), that is, when the visible light signal cannot be received even after the predetermined time has elapsed, the receiver By repeatedly executing the processing from step S9232, the search for the brightest area is redone.

Note that the receiver may turn off the zoom at an arbitrary timing when using the zoom function. That is, when the zoom is off, the receiver may detect whether or not there is a bright subject in a range that is not imaged during zooming but is imaged if zooming is not performed. Note that this bright subject is likely to be a transmitter that transmits a visible light signal due to a change in luminance. As a result, it is possible to receive signals from transmitters existing in a wide range.

Here, the automatic exposure and photometric method will be described.

The automatic exposure in FIGS. 259A to 259D will be described below. The automatic exposure is an operation, process, or function in which the imaging unit of the receiver adjusts the exposure time, the sensitivity, and the aperture to automatically adjust the photometric result to a predetermined value.

As the photometric method for obtaining the photometric result, there are methods such as average photometry (overall photometry), center-weighted photometry, spot photometry (partial photometry), and split photometry. In the average photometry, the average brightness of the entire image obtained by imaging is calculated. In the center-weighted photometry, the weighted average value of the brightness weighted is calculated so as to be closer to the center (or the designated portion) of the image. In spot photometry, the average value (or weighted average value) of brightness in a predetermined one (or several) range defined around the center or designated portion of the image is calculated. In the divided photometry, the image is divided into a plurality of parts, and the photometry is performed in each part to calculate a total brightness value.

In the imaging unit having the automatic exposure function, even if the exposure time cannot be set to be shortened directly, the exposure time can be indirectly set by the function of the automatic exposure. For example, if the sensitivity is set high (for example, to the maximum value), the image obtained by imaging becomes brighter under other conditions, so that the exposure time can be set shorter by automatic exposure. Also, if the aperture is set to open (that is, open), the exposure time can be similarly set short. Further, when the value indicating the level of exposure correction is set low (for example, to the minimum value), the image obtained by the image pickup becomes dark due to the automatic exposure. That is, the exposure time is set to be short. The exposure time can be set short by designating the brightest place in the image as the photometric position. If the photometric method can be designated, spot exposure can be designated to shorten the exposure time. If the photometric range can be designated, the exposure time can be set short by designating the photometric range to the minimum. When the area of the bright portion in the image is large, the exposure time can be set short by designating the photometric range as large as possible so as not to exceed the bright portion. If a plurality of photometric positions can be designated, the exposure time can be set short by designating the same place as the photometric positions multiple times. The exposure time can be set short by enlarging a bright area in the image with the zoom and designating that area as the photometric position.

Here, the EX zoom will be described.

FIG. 260 is a diagram for explaining the EX zoom.

Zooming in FIG. 259C, that is, a method of obtaining a large image includes optical zooming in which the focal length of the lens is adjusted to change the size of the image captured in the image sensor, and interpolation of the image captured in the image sensor by digital processing. There are a digital zoom that obtains a large image and an EX zoom that obtains a large image by changing a plurality of image pickup elements used for image pickup. The EX zoom can be used when the number of image pickup devices included in the image sensor is larger than the resolution of the picked-up image.

For example, in the image sensor 10080a shown in FIG. 260, 32×24 image pickup devices are arranged in a matrix. That is, 32 image sensors are arranged horizontally and 24 image sensors are arranged vertically. When an image with a resolution of 16 horizontal×12 vertical is obtained by image capturing by the image sensor 10080a, as shown in (a) of FIG. 260, the image sensor of 32×24 image capturing elements included in the image sensor 10080a is used. Only 16×12 image pickup elements (for example, the image pickup elements indicated by the black squares in the image sensor 1080a in FIG. 260(a)) uniformly distributed in 10080a are used for image pickup. That is, only the odd-numbered or even-numbered image pickup elements are used for image pickup among the plurality of image pickup elements arranged in the vertical direction and the horizontal direction. As a result, the image 10080b having the desired resolution is obtained. Note that, in FIG. 260, the subject appears in the image sensor 1008a, but this is for the purpose of making it easier to understand the correspondence relationship between each image pickup element and the image obtained by the image pickup.

A receiver equipped with this image sensor 10080a evenly scans a wide range to search for a transmitter or receive information from many transmitters, and the image sensor 10080a is evenly distributed over the entire area. An image is captured using only some of the image sensors arranged in a dispersed manner.

In addition, when performing the EX zoom, the receiver, as shown in (b) of FIG. 260, in the image sensor 10080a, some of the image pickup elements locally arranged (for example, (b) of FIG. 260). 16×12 image pickup elements indicated by the black squares in the image sensor 1080a in FIG. As a result, a portion of the image 10080b corresponding to a part of the image sensor is zoomed, and an image 10080d is obtained. With such an EX zoom, a large image of the transmitter can be picked up so that the visible light signal can be received for a long time, the receiving speed is improved, and the visible light signal can be received from a distance.

In the digital zoom, the number of exposure lines that receive the visible light signal cannot be increased, and the reception time of the visible light signal does not increase, so it is better to use another zoom as much as possible. The optical zoom requires a physical movement time of the lens and the image sensor, but the EX zoom has an advantage that the zoom time is short because it is performed only by electronically changing the setting. From this viewpoint, the priority of each zoom is (1) EX zoom, (2) optical zoom, and (3) digital zoom. The receiver may select and use any one or more zooms depending on the priority and the necessity of the zoom magnification. Note that in the imaging method shown in FIGS. 260A and 260B, image noise can be suppressed by using an unused imaging element.

FIG. 261A is a flowchart showing processing of the reception program in the tenth embodiment.

This reception program is a program that causes a computer included in the receiver to execute the processes shown in FIGS. 259A to 260, for example.

That is, this receiving program is a receiving program for receiving information from the light emitter. Specifically, the receiving program sets an exposure time setting step SA21 for setting the exposure time of the image sensor by using automatic exposure, and causes the image sensor to capture an image of the luminous body whose brightness changes. A bright line image acquisition step SA22 for acquiring a bright line image which is an image including bright lines corresponding to each of a plurality of exposure lines included in the image sensor, and decoding a plurality of bright line patterns included in the acquired bright line image. The computer is caused to execute the information acquisition step SA23 for acquiring the information. Further, in the exposure time setting step SA21, as in step S9208 of FIG. 259A, the sensitivity of the image sensor is set to the maximum value within the range predetermined for the image sensor, and the sensitivity of the maximum value is set. Exposure time is set by automatic exposure.

As a result, even if the exposure time of the image sensor cannot be set directly, the exposure time is short enough to obtain an appropriate bright line image using the automatic exposure function installed in a general camera. Can be set. That is, in the automatic exposure, the exposure is adjusted based on the brightness of the image acquired by the image sensor. Therefore, if the sensitivity of the image sensor is set to a large value, the image becomes bright, and therefore the exposure time of the image sensor is set to be short in order to suppress the exposure. If the sensitivity of the image sensor is set to the maximum value, the exposure time can be set shorter and an appropriate bright line image can be acquired. That is, the information from the light emitter can be properly received. As a result, communication between various devices can be enabled. The sensitivity is ISO sensitivity, for example.

Further, in the exposure time setting step SA21, as in step S9212 of FIG. 259A, the value indicating the exposure correction level of the image sensor is set to the minimum value in the range preset for the image sensor, Automatic exposure sets the exposure time according to the maximum sensitivity and minimum level exposure compensation.

As a result, the value indicating the exposure correction level is set to the minimum value, so that the exposure time can be set shorter by the automatic exposure processing that attempts to suppress the exposure, and a more appropriate bright line image is acquired. be able to. The unit of the value indicating the exposure correction level is EV, for example.

Further, in the exposure time setting step SA21, as shown in FIG. 259C, a brighter part than other parts is specified in the first image obtained by imaging the subject including the light emitting body by the image sensor. Then, a part of the subject corresponding to the bright portion is enlarged by the optical zoom. Further, the exposure time is set by using the second image obtained by capturing an image of a part of the enlarged subject by the image sensor for inputting the automatic exposure. Further, in the bright line image acquisition step SA22, a bright line image is acquired by causing the image sensor to capture a part of the enlarged subject with the set exposure time.

As a result, a part of the subject corresponding to the bright portion of the first image is magnified by the optical zoom, that is, the bright illuminant is magnified by the optical zoom, so that the second image is larger than the first image. Can also be brightened overall. Then, since the bright second image is used for the input of the automatic exposure, the exposure time can be set shorter by the automatic exposure processing for suppressing the exposure, and an appropriate bright line image can be acquired. it can.

Further, in the exposure time setting step SA21, as shown in FIG. 259C, as shown in FIG. 259C, the central portion of the first image acquired by capturing an image of a subject including a light emitter is the brightness at a plurality of positions in the first image. It is determined whether it is brighter than the average of. Then, when it is determined that the central portion is bright, a part of the subject corresponding to the central portion is enlarged by the optical zoom. Further, the exposure time is set by using the second image obtained by capturing an image of a part of the enlarged subject by the image sensor for inputting the automatic exposure. Further, in the bright line image acquisition step SA22, a bright line image is acquired by causing the image sensor to capture a part of the enlarged subject with the set exposure time.

As a result, a part of the subject corresponding to the bright center portion of the first image is magnified by the optical zoom, that is, the bright illuminant is magnified by the optical zoom, and thus the second image is changed to the first image. Can be brighter than the whole. Then, since the bright second image is used for the input of the automatic exposure, the exposure time can be set shorter by the automatic exposure processing for suppressing the exposure, and an appropriate bright line image can be acquired. it can. In addition, the optical zoom enlarges the angle of view or the central portion of the image unless the center position of enlargement can be set arbitrarily. Therefore, even if the center position cannot be arbitrarily set, if the central portion of the first image is bright, the second image can be entirely brightened by using the optical zoom. it can. Further, even when the central portion of the first image is dark, if the enlargement is performed by the optical zoom, the second image becomes dark and the exposure time becomes long. Therefore, as described above, it is possible to prevent the exposure time from increasing by performing enlargement by the optical zoom only when it is determined that the central portion is bright.

Further, in the exposure time setting step SA21, as shown in FIG. 260, among the K (K is an integer of 3 or more) image pickup devices included in the image sensor, N arranged in the image sensor are evenly distributed. In the first image acquired by imaging the subject including the light-emitting body with only N (N is an integer less than or equal to K and 2 or more) imaging elements, a brighter portion than other portions is specified. Further, by using the second image acquired by the image pickup by only the dense N image pickup elements corresponding to the bright portion among the K image pickup elements included in the image sensor as the input of the automatic exposure, the exposure is performed. Set the time. Further, in the bright line image acquisition step SA22, a bright line image is acquired by causing only the densely arranged N image pickup devices included in the image sensor to take an image with the set exposure time.

Thus, the so-called EX zoom can brighten the second image as a whole even if the bright portion of the first image is not in the center, and the exposure time can be set shorter.

Further, in the exposure time setting step SA21, as shown in FIG. 259C, the photometric position in the image acquired by the image sensor capturing the subject is set, and the exposure time corresponding to the brightness of the set photometric position. Is set by automatic exposure.

As a result, if the bright part in the image acquired by imaging is set to the photometric position, the exposure time can be set shorter by the automatic exposure process that attempts to suppress the exposure, and an appropriate bright line image can be obtained. Can be obtained.

Further, the reception program further includes an imaging mode setting step of switching the imaging mode of the image sensor from a color imaging mode for acquiring a color image by shooting to a monochrome imaging mode for acquiring a monochrome image by shooting. It may be executed by a computer. In this case, in the exposure time setting step SA21, the exposure time is set by using the image acquired in the monochrome imaging mode for inputting the automatic exposure.

As a result, the image acquired in the monochrome imaging mode is used for inputting the automatic exposure, so that an appropriate exposure time can be set without being affected by the color information. Further, when the exposure time is set in the monochrome imaging mode, the bright line image is acquired by imaging according to the mode. Therefore, when the information is transmitted from the light emitting body only by the brightness change, the information can be appropriately acquired.

Further, in the exposure time setting step SA21, every time an image is captured by the image sensor capturing the light emitter, the captured image is used for input of automatic exposure to update the exposure time of the image sensor. Here, for example, as shown in step S9234 of FIG. 259D, when the fluctuation range of the exposure time updated at any time falls within a predetermined range, the exposure time is updated by terminating the update of the exposure time by automatic exposure. Set.

As a result, when the fluctuation of the exposure time becomes stable, that is, when the brightness of the image acquired by the image capturing falls within the target brightness of the automatic exposure, the exposure time at that time becomes the bright line image. It is used for imaging for acquisition of. Therefore, an appropriate bright line image can be acquired.

FIG. 261B is a block diagram of a receiver in Embodiment 10.

The receiving device A20 is, for example, the above-described receiver that executes the processes shown in FIGS. 259A to 260.

In other words, the receiving device A20 is a device for receiving information from the light emitter, and sets the exposure time setting unit A21 that sets the exposure time of the image sensor using automatic exposure and the light emitter that changes in brightness. An image capturing unit A22 that acquires a bright line image that is an image including bright lines corresponding to each of a plurality of exposure lines included in the image sensor by causing the image sensor to capture an image with the obtained exposure time; And a decoding unit A23 that acquires information by decoding a plurality of bright line patterns. The exposure time setting unit A21 sets the sensitivity of the image sensor to the maximum value within a predetermined range for the image sensor, and sets the exposure time according to the maximum sensitivity by automatic exposure. In this receiving device A20, the same effect as the above-mentioned receiving program can be obtained.

A reception program according to an aspect of the present invention is a reception program for receiving information from a light-emitting body whose brightness changes according to a signal output by the above information processing program, and which automatically exposes an exposure time of an image sensor. And an exposure time setting step for setting a subject including the light-emitting body that changes in luminance by the image sensor at the set exposure time, thereby setting each of a plurality of exposure lines included in the image sensor. A bright line image acquisition step of acquiring a bright line image that is an image including a bright line corresponding to, and an information acquisition step of acquiring information by decoding a pattern of the plurality of bright line included in the acquired bright line image In the exposure time setting step, the sensitivity of the image sensor is set to the maximum value within a predetermined range for the image sensor, and the exposure time according to the sensitivity of the maximum value is set. It is set by the automatic exposure.

As a result, as shown in FIGS. 259A to 261B, even if the exposure time of the image sensor cannot be directly set, an appropriate bright line can be obtained by using the automatic exposure function installed in a general camera. It is possible to set an exposure time that is short enough to obtain an image. That is, in the automatic exposure, the exposure is adjusted based on the brightness of the image acquired by the image sensor. Therefore, if the sensitivity of the image sensor is set to a large value, the image becomes bright, and therefore the exposure time of the image sensor is set to be short in order to suppress the exposure. If the sensitivity of the image sensor is set to the maximum value, the exposure time can be set shorter and an appropriate bright line image can be acquired. That is, the information from the light emitter can be properly received. As a result, communication between various devices can be enabled. The sensitivity is ISO sensitivity, for example.

Further, in the exposure time setting step, a value indicating a level of exposure correction of the image sensor is set to a minimum value in a range preset for the image sensor, and the sensitivity of the maximum value and the minimum value are set. The exposure time may be set by the automatic exposure according to the value level exposure correction.

As a result, the value indicating the exposure correction level is set to the minimum value, so that the exposure time can be set shorter by the automatic exposure processing that attempts to suppress the exposure, and a more appropriate bright line image is acquired. be able to. The unit of the value indicating the exposure correction level is EV, for example.

In the exposure time setting step, a brighter part than other parts is specified in the first image obtained by imaging the subject including the light emitter by the image sensor, and the one part of the subject corresponding to the bright part is identified. A second image obtained by capturing a part of the enlarged subject by the image sensor by using an optical zoom to set the exposure time by using the second image for inputting the automatic exposure; In the bright line image acquisition step, the bright line image may be acquired by causing the image sensor to capture a part of the enlarged subject with the set exposure time.

As a result, a part of the subject corresponding to the bright portion of the first image is magnified by the optical zoom, that is, the bright illuminant is magnified by the optical zoom, so that the second image is larger than the first image. Can also be brightened overall. Then, since the bright second image is used for the input of the automatic exposure, the exposure time can be set shorter by the automatic exposure processing for suppressing the exposure, and an appropriate bright line image can be acquired. it can.

Further, in the exposure time setting step, a central portion of the first image acquired by capturing an image of a subject including the light emitting body by the image sensor is higher than an average of brightness at a plurality of positions in the first image. It is determined whether or not it is bright, and when it is determined that the central portion is bright, a part of the subject corresponding to the central portion is magnified by optical zoom, and the image sensor magnifies the magnified subject. The exposure time is set by using the second image obtained by capturing a part of the image to input the automatic exposure, and in the bright line image obtaining step, a part of the enlarged subject is set. The bright line image may be acquired by causing the image sensor to capture an image at the exposure time.

As a result, a part of the subject corresponding to the bright center portion of the first image is magnified by the optical zoom, that is, the bright illuminant is magnified by the optical zoom, and thus the second image is changed to the first image. Can be brighter than the whole. Then, since the bright second image is used for the input of the automatic exposure, the exposure time can be set shorter by the automatic exposure processing for suppressing the exposure, and an appropriate bright line image can be acquired. it can. In addition, the optical zoom enlarges the angle of view or the central portion of the image unless the center position of enlargement can be set arbitrarily. Therefore, even if the center position cannot be arbitrarily set, if the central portion of the first image is bright, the second image can be entirely brightened by using the optical zoom. it can. Further, even when the central portion of the first image is dark, if the enlargement is performed by the optical zoom, the second image becomes dark and the exposure time becomes long. Therefore, as described above, it is possible to prevent the exposure time from increasing by performing enlargement by the optical zoom only when it is determined that the central portion is bright.

Further, in the exposure time setting step, among K (K is an integer of 3 or more) image pickup devices included in the image sensor, N (N is K) evenly arranged in the image sensor. In the first image acquired by capturing an image of a subject including the light-emitting body using only an image sensor of an integer less than or equal to 2), a portion brighter than other portions is identified, and K pieces of image capturing included in the image sensor are performed. Among the elements, the exposure time is set by using the second image acquired by imaging only the dense N image pickup elements corresponding to the bright portion for inputting the automatic exposure, and the bright line image is set. In the acquisition step, the bright line image may be acquired by causing only the dense N image pickup devices included in the image sensor to pick up an image with the set exposure time.

Thus, the so-called EX zoom can brighten the second image as a whole even if the bright portion of the first image is not in the center, and the exposure time can be set shorter.

Further, in the exposure time setting step, a photometric position in an image acquired by the image sensor capturing the subject is set, and the exposure time according to the brightness of the set photometric position is automatically adjusted by the automatic exposure. You may set up by exposure.

Thus, if the bright part in the image acquired by imaging is set to the photometric position, the exposure time can be set shorter by the automatic exposure process that attempts to suppress the exposure, and an appropriate bright line image is acquired. can do.

Further, the reception program further sets an imaging mode setting for switching the imaging mode of the image sensor from a color imaging mode for acquiring a color image by shooting to a monochrome imaging mode for acquiring a monochrome image by shooting. The step may be executed by the computer, and the exposure time may be set in the exposure time setting step by using an image acquired in the monochrome imaging mode for inputting the automatic exposure.

As a result, the image acquired in the monochrome imaging mode is used for inputting the automatic exposure, so that an appropriate exposure time can be set without being affected by the color information. Further, when the exposure time is set in the monochrome imaging mode, the bright line image is acquired by imaging according to the mode. Therefore, when the information is transmitted from the light emitting body only by the brightness change, the information can be appropriately acquired.

Further, in the exposure time setting step, every time an image is captured by the imaging of the light emitter by the image sensor, the captured image is used for inputting the automatic exposure, thereby updating the exposure time of the image sensor. The exposure time may be set by ending the update of the exposure time by the automatic exposure when the fluctuation range of the exposure time, which is updated at any time, falls within a predetermined range.

As a result, when the fluctuation of the exposure time becomes stable, that is, when the brightness of the image acquired by the image capturing falls within the target brightness of the automatic exposure, the exposure time at that time becomes the bright line image. It is used for imaging for acquisition of. Therefore, an appropriate bright line image can be acquired.

(Embodiment 12)
In the present embodiment, each application example using the receiver such as the smartphone in each of the above embodiments and the transmitter that transmits information as the blinking pattern of the LED or the organic EL will be described.

In the present embodiment, the exposure time is set for each exposure line or each imaging element.

262, 263, and 264 are diagrams showing an example of the signal receiving method according to the twelfth embodiment.

As shown in FIG. 262, in the image sensor 10010a, which is the imaging unit provided in the receiver, the exposure time is set for each exposure line. That is, a long exposure time for normal imaging is set to a predetermined exposure line (white exposure line in FIG. 262), and visible light imaging is set to another exposure line (black exposure line in FIG. 262). Short exposure time is set. For example, a long exposure time and a short exposure time are alternately set for each exposure line arranged in the vertical direction. This allows normal imaging and visible light imaging (visible light communication) to be performed almost at the same time when imaging a transmitter that transmits a visible light signal due to a change in luminance. The two exposure times may be alternately set for each line, may be set for several lines, or different exposure times may be set for the upper portion and the lower portion of the image sensor 10010a. By using the two exposure times in this way, the data obtained by the imaging of the plurality of exposure lines set to the same exposure time can be summarized respectively, and the normal captured image 10010b and the bright line image showing the pattern of the plurality of bright lines are collected. And a visible light captured image 10010c is obtained. In the normal captured image 10010b, a portion that has not been captured in a long exposure time (that is, an image corresponding to a plurality of exposure lines set to a short exposure time) is missing. Therefore, by interpolating that portion, a preview image is obtained. 10010d can be displayed. Here, information obtained by visible light communication can be superimposed on the preview image 10010d. This information is information associated with a visible light signal obtained by decoding a plurality of bright line patterns included in the visible light captured image 10010c. Note that the receiver stores the normal captured image 10010b or an image obtained by interpolating the normal captured image 10010b as a captured image, and stores the received visible light signal or information associated with the visible light signal. It may be added as additional information to the captured image to be saved.

As shown in FIG. 263, an image sensor 10011a may be used instead of the image sensor 10010a. In the image sensor 1011a, the exposure time is set not for each exposure line but for each column (hereinafter, referred to as vertical line) including a plurality of image pickup elements arranged in a direction perpendicular to the exposure line. That is, a long exposure time for normal imaging is set to a predetermined vertical line (white vertical line in FIG. 263), and visible light imaging is set to another vertical line (black vertical line in FIG. 263). Short exposure time is set. In this case, in the image sensor 10011a, similarly to the image sensor 10010a, the exposure is started at different timings for each exposure line, but the exposure time of each image sensor included in the exposure line is different for each exposure line. .. The receiver obtains a normal captured image 10011b and a visible light captured image 10011c by capturing with the image sensor 10011a. Further, the receiver generates and displays the preview image 10011d based on the normal captured image 10011b and the information associated with the visible light signal obtained from the visible light captured image 10011c.

Unlike the image sensor 10010a, the image sensor 10011a can use all exposure lines for visible light imaging. As a result, the visible light imaged image 10011c obtained by the image sensor 10011a includes more bright lines than the visible light imaged image 10010c, so that the visible light signal reception accuracy can be increased.

Further, as shown in FIG. 264, an image sensor 10012a may be used instead of the image sensor 10010a. In the image sensor 10012a, the exposure time is set for each image pickup device so that the same exposure time is not continuously set for each image pickup device along the horizontal direction and the vertical direction. That is, the exposure time is set for each image sensor so that the plurality of image sensors for which the long exposure time is set and the plurality of image sensors for which the short exposure time are set are distributed like a grid or a checkered pattern. Is set. In this case as well, similarly to the image sensor 10010a, the exposure is started at different timing for each exposure line, but the exposure time of each image sensor included in the exposure line is different for each exposure line. The receiver obtains a normal captured image 10012b and a visible light captured image 10012c by capturing with the image sensor 10012a. Further, the receiver generates and displays the preview image 10012d based on the normal captured image 10012b and the information associated with the visible light signal obtained from the visible light captured image 10012c.

Since the normal captured image 10012b obtained by the image sensor 10012a has data of a plurality of image pickup elements arranged in a grid or evenly arranged, the normal captured image 10012b and the normal captured image 10011b can be interpolated more accurately or. Can be resized. The visible light imaged image 10012c is generated by imaging using all exposure lines of the image sensor 10012a. That is, in this image sensor 10012a, unlike the image sensor 10010a, all exposure lines can be used for visible light imaging. As a result, the visible light imaged image 10012c obtained by the image sensor 10012a includes more bright lines than the visible light imaged image 10010c as in the visible light imaged image 10011c, and thus the reception of the visible light signal is increased. Can be done with precision.

Here, the interlaced display of the preview image will be described.

FIG. 265 is a diagram illustrating an example of a screen display method of a receiver in Embodiment 12.

The receiver including the image sensor 10010a shown in FIG. 262 described above has exposure times set to odd-numbered exposure lines (hereinafter referred to as odd-numbered lines) and even-numbered exposure lines (hereinafter referred to as even-numbered lines). And the exposure time to be changed at predetermined intervals. For example, as shown in FIG. 265, at the time t1, the receiver sets a long exposure time for each image sensor of the odd line and a short exposure time for each image sensor of the even line. The imaging is performed using the exposure time set in (3). Further, at time t2, the receiver sets a short exposure time for each of the image pickup elements in the odd lines, sets a long exposure time for each of the image pickup elements in the even lines, and sets the set exposure times. The imaging used is performed. Then, the receiver performs imaging at time t3 using each exposure time set as at time t1, and at time t4 uses each exposure time set as at time t2. Take an image.

Further, at the time t1, the receiver receives an image obtained from each of a plurality of odd lines by imaging (hereinafter referred to as an odd line image) and an image obtained from each of a plurality of even lines by imaging (hereinafter, an even line image). Image1 that includes and). At this time, since the exposure time is short in each of the plurality of even lines, the subject is not clearly displayed in each of the even line images. Therefore, the receiver generates a plurality of interpolated line images by performing pixel value interpolation on the plurality of odd line images. Then, the receiver displays a preview image including a plurality of interpolated line images instead of the plurality of even line images. That is, the odd number line image and the interpolation line image are alternately arranged in the preview image.

At time t2, the receiver acquires Image2 including a plurality of odd line images and even line images by imaging. At this time, since the exposure time is short in each of the plurality of odd lines, the subject is not clearly displayed in each of the odd line images. Therefore, the receiver displays the preview image including the odd line image of Image1 instead of the odd line image of Image2. That is, in the preview image, the odd line image of Image1 and the even line image of Image2 are alternately arranged.

Further, at time t3, the receiver acquires Image3 including a plurality of odd line images and even line images by imaging. At this time, as in the case of the time t1, the exposure time is short for each of the plurality of even lines, so that the subject is not clearly displayed in each of the even line images. Therefore, the receiver displays the preview image including the even line image of Image2 instead of the even line image of Image3. That is, in the preview image, the even-numbered line image of Image2 and the odd-numbered line image of Image3 are alternately arranged. Then, at time t4, the receiver acquires Image4 including a plurality of odd line images and even line images by imaging. At this time, as in the case of the time t2, the exposure time is short for each of the plurality of odd lines, so that the subject is not clearly displayed in each of the odd line images. Therefore, the receiver displays the preview image including the odd line image of Image3 instead of the odd line image of Image4. That is, in the preview image, the odd line image of Image3 and the even line image of Image4 are alternately arranged.

In this way, the receiver performs so-called interlaced display, in which an image including an even line image and an odd line image whose acquired timings are different from each other is displayed.

A receiver like this can display a fine preview image while performing visible light imaging. It should be noted that the plurality of image pickup elements for which the same exposure time is set may be a plurality of image pickup elements arranged in a horizontal direction to the exposure line like the image sensor 10010a, or may be an exposure line like the image sensor 10011a. It may be a plurality of image pickup elements arranged along a direction perpendicular to, or a plurality of image pickup elements arranged in a checkered pattern like the image sensor 10012a. In addition, the receiver may store the preview image as captured data.

Next, the spatial ratio between normal imaging and visible light imaging will be described.

FIG. 266 is a diagram illustrating an example of a signal reception method in Embodiment 12.

In the image sensor 10014b included in the receiver, a long exposure time or a short exposure time is set for each exposure line, similarly to the image sensor 10010a described above. In this image sensor 10014b, the ratio of the number of image pickup elements for which a long exposure time is set to the number of image pickup elements for which a short exposure time is set is 1:1. It should be noted that this ratio is a ratio between normal imaging and visible light imaging, and is hereinafter referred to as a spatial ratio.

However, in this embodiment, the space ratio does not need to be 1:1. For example, the receiver may include the image sensor 10014a. In this image sensor 10014a, the number of image pickup devices having a short exposure time is larger than that of image pickup devices having a long exposure time, and the spatial ratio is 1:N (N>1). The receiver may also include the image sensor 10014c. In this image sensor 10014c, the image sensor with a short exposure time is smaller than the image sensor with a long exposure time, and the spatial ratio is N (N>1):1. Also, instead of the image sensors 10014a to 10014c, the receiver has image sensors 10015a to 10015c each having an exposure time set for each vertical line and having a spatial ratio of 1:N, 1:1 or N:1. Either may be provided.

In such image sensors 10014a and 10015a, since there are many imaging elements with a short exposure time, it is possible to improve the reception accuracy or the reception speed of the visible light signal. On the other hand, in the image sensors 10014c and 10015c, since many image pickup devices have a long exposure time, it is possible to display a fine preview image.

In addition, the receiver may perform interlaced display as shown in FIG. 265 using the image sensors 10014a, 10014c, 10015a, 10015c.

Next, the time ratio between normal imaging and visible light imaging will be described.

FIG. 267 is a diagram illustrating an example of a signal reception method in Embodiment 12.

As shown in (a) of FIG. 267, the receiver may switch the imaging mode to the normal imaging mode and the visible light imaging mode for each frame. The normal imaging mode is an imaging mode in which a long exposure time for normal imaging is set for all the imaging elements of the image sensor of the receiver. The visible light imaging mode is an imaging mode in which a short exposure time for visible light imaging is set for all imaging elements of the image sensor of the receiver. In this way, by switching between long and short exposure times, it is possible to display a preview image by imaging for a long exposure time while receiving a visible light signal by imaging for a short exposure time.

When the long exposure time is determined by automatic exposure, the receiver ignores the image obtained by the short exposure time and only the brightness of the image obtained by the long exposure time. Automatic exposure may be performed on the basis. This makes it possible to determine a long exposure time as an appropriate time.

In addition, the receiver may switch the imaging mode between the normal imaging mode and the visible light imaging mode for each set of a plurality of frames, as illustrated in (b) of FIG. 267. When it takes a long time to switch the exposure time, or when it takes a long time to stabilize the exposure time, by changing the exposure time for each set of a plurality of frames as shown in (b) of FIG. 267, Visible light imaging (reception of visible light signal) and normal imaging can be compatible. Further, the more frames included in the set, the smaller the number of times the exposure time is switched, so that power consumption and heat generation in the receiver can be suppressed.

Here, the number of at least one frame continuously generated by the long exposure time imaging in the normal imaging mode and the at least one frame continuously generated by the short exposure time imaging in the visible light imaging mode The ratio (hereinafter, referred to as time ratio) to the number of 1 does not have to be 1:1. That is, in the cases shown in (a) and (b) of FIG. 267, the time ratio is 1:1 but the time ratio may not be 1:1.

For example, the receiver may have more frames in the visible light image capturing mode than in the normal image capturing mode, as shown in (c) of FIG. 267. Thereby, the receiving speed of the visible light signal can be increased. If the frame rate of the preview image is equal to or higher than a predetermined rate, the difference between the preview images depending on the frame rate cannot be recognized by human eyes. When the frame rate of imaging is sufficiently high, for example, when the frame rate is 120 fps, the receiver sets the visible light imaging mode for three consecutive frames and the visible light imaging for the next one frame. Set the mode. Accordingly, the receiver can receive the visible light signal at high speed while displaying the preview image at the frame rate of 30 fps, which is sufficiently higher than the above-mentioned predetermined rate. Further, since the number of times of switching is reduced, the effect described in (b) of FIG. 267 can be obtained.

Also, the receiver may have more frames in the normal imaging mode than in the visible light imaging mode, as shown in (d) of FIG. 267. As described above, by increasing the number of frames in the normal imaging mode, that is, the frames obtained by imaging with a long exposure time, the preview image can be displayed smoothly. Further, since the number of times of receiving the visible light signal is reduced, there is a power saving effect. Further, since the number of times of switching is reduced, the effect described in (b) of FIG. 267 can be obtained.

In addition, as shown in (e) of FIG. 267, the receiver first switches the imaging mode for each frame as in the case of (a) of FIG. 267, and then receives the visible light signal. Upon completion, as in the case shown in FIG. 267(d), the number of frames in the normal imaging mode may be increased. Thus, after the completion of receiving the visible light signal, the preview image can be displayed smoothly and the search for a new visible light signal can be continued. Further, since the number of times of switching is reduced, the effect described in (b) of FIG. 267 can be obtained.

FIG. 268 is a flowchart showing an example of a signal reception method in Embodiment 12.

The receiver starts visible light reception, which is a process of receiving a visible light signal (step S10017a), and sets the exposure time length setting ratio to a value specified by the user (step S10017b). The exposure time length setting ratio is at least one of the space ratio and the time ratio described above. The user may specify only the space ratio, only the time ratio, or both values of the space ratio and the time ratio, or may be automatically set by the receiver regardless of the user's specification.

Next, the receiver determines whether the reception performance is equal to or lower than a predetermined value (step S10017c). When it is determined that the value is equal to or less than the predetermined value (Y in step S10017c), the receiver sets the ratio of the short exposure time to be high (step S10017d). As a result, reception performance can be improved. The short exposure time ratio is the ratio of the number of image pickup devices for which the short exposure time is set to the number of image pickup devices for which the long exposure time is set for the space ratio. It is the ratio of the number of frames continuously generated in the visible light imaging mode to the number of frames continuously generated in the mode.

Next, the receiver receives at least a part of the visible light signal and determines whether or not a priority is set for at least a part of the received visible light signal (hereinafter, referred to as a received signal) ( Step S10017e). When the priority is set, the identifier indicating the priority is included in the received signal. When determining that the priority is set (Y in step S10017e), the receiver sets the exposure time length/short ratio according to the priority (step S10017f). That is, if the priority is high, the receiver sets a high ratio of short exposure times. For example, an emergency light configured as a transmitter emits an identifier indicating high priority due to a change in brightness. In this case, the receiver can increase the reception speed by increasing the ratio of the short exposure time and promptly display the evacuation route and the like.

Next, the receiver determines whether reception of all visible light signals has been completed (step S10017g). Here, when it is determined that the processing is not completed (N in step S10017g), the receiver repeatedly executes the processing from step S10017c. On the other hand, when it is determined that the exposure is completed (Y in step S10017g), the receiver sets the ratio of the long exposure time to be high and shifts to the power saving mode (step S10017h). The ratio of the long exposure time is the ratio of the number of image pickup devices for which the long exposure time is set to the number of image pickup devices for which the short exposure time is set for the space ratio. It is the ratio of the number of frames continuously generated in the normal imaging mode to the number of frames continuously generated in the imaging mode. As a result, the preview image can be displayed smoothly without receiving unnecessary visible light.

Next, the receiver determines whether or not another visible light signal is found (step S10017i). Here, when it is determined that it has been discovered (Y in step S10017i), the receiver repeatedly executes the processing from step S10017b.

Next, simultaneous execution of visible light imaging and normal imaging will be described.

FIG. 269 is a diagram illustrating an example of a signal reception method in Embodiment 12.

The receiver may set two or more exposure times on the image sensor. That is, as shown in (a) of FIG. 269, each of the exposure lines included in the image sensor is continuously exposed for the longest exposure time of the set two or more exposure times. For each exposure line, the receiver reads the imaging data obtained by the exposure of the exposure line at the time when the above-mentioned set two or more exposure times have elapsed. Here, the receiver does not reset the read imaging data until the longest exposure time elapses. Therefore, the receiver can obtain the imaging data at a plurality of exposure times by only performing the exposure for the longest exposure time by recording the cumulative value of the read imaging data. The image sensor may or may not record the cumulative value of the imaged data. When the image sensor does not perform, a component of the receiver that reads data from the image sensor performs cumulative calculation, that is, recording the cumulative value of the imaging data.

For example, when two exposure times are set, as shown in (a) of FIG. 269, the receiver outputs visible light imaging data including a visible light signal, which is generated by exposure of a short exposure time. Readout is performed, and subsequently, normal imaging data generated by exposure for a long exposure time is read out.

Accordingly, it is possible to simultaneously perform visible light imaging, which is imaging for receiving a visible light signal, and normal imaging, and it is possible to perform normal imaging while receiving a visible light signal. Further, by using the data of a plurality of exposure times, it is possible to recognize a signal frequency higher than the sampling theorem, and it is possible to receive a high frequency signal and a high density modulation signal.

Further, when outputting the imaging data, the receiver outputs a data string including the imaging data as an imaging data body as shown in FIG. 269(b). That is, the receiver has an imaging mode identifier indicating an imaging mode (visible light imaging or normal imaging), an imaging element identifier for identifying the imaging element or an exposure line to which the imaging element belongs, and the exposure number of the imaging data body. The above-mentioned data string is generated and output by adding additional information including the image pickup data number indicating whether it is the image pickup data of time and the image pickup data length indicating the size of the image pickup data body to the above-mentioned data string. .. In the method of reading the imaged data described with reference to FIG. 269(a), the respective imaged data are not always output in the order of the exposure lines. Therefore, by adding the additional information shown in (b) of FIG. 269, it is possible to specify which exposure line of the imaging data the imaging data belongs to.

FIG. 270A is a flowchart showing processing of the reception program according to the twelfth embodiment.

This reception program is a program that causes a computer included in the receiver to execute the processes shown in, for example, FIGS.

That is, this receiving program is a receiving program for receiving information from the light emitting body whose brightness changes. Specifically, this reception program causes a computer to execute step SA31, step SA32, and step SA33. In step SA31, the first exposure time is set for some of a plurality of image pickup devices (K is an integer of 4 or more) included in the image sensor, and the first exposure time is set for the K image pickup devices. A second exposure time shorter than the first exposure time is set for the remaining plurality of image pickup devices. In step SA32, the image sensor images the subject, which is a light-emitting body that changes in luminance, with the set first and second exposure times, and outputs from the plurality of image pickup elements with the first exposure time set. And a bright line corresponding to each of the plurality of exposure lines included in the image sensor, which is an image corresponding to the output from the plurality of image pickup elements for which the second exposure time is set. Acquire a bright line image that is an image including the bright line image. In step SA33, information is acquired by decoding a plurality of bright line patterns included in the acquired bright line image.

As a result, since the image pickup is performed by the plurality of image pickup elements for which the first exposure time is set and the plurality of image pickup elements for which the second exposure time is set, the normal image is obtained by one image pickup by the image sensor. And the bright line image can be acquired. That is, it is possible to simultaneously capture a normal image and acquire information by visible light communication.

In the exposure time setting step SA31, the first exposure time is set for some of the plurality of image pickup device rows (L is an integer of 4 or more) included in the image sensor. Then, the second exposure time is set for the remaining plurality of image pickup device rows of the L image pickup device rows. Here, each of the L image pickup device rows includes a plurality of image pickup devices included in the image sensor and arranged in a line.

As a result, the exposure time can be set for each image pickup device row that is a large unit without individually setting the exposure time for each image pickup device that is a small unit, and the processing load can be reduced. ..

For example, each of the L image sensor arrays is an exposure line included in the image sensor, as shown in FIG. Alternatively, as shown in FIG. 263, each of the L image pickup device rows includes a plurality of image pickup devices arranged in a direction perpendicular to the exposure line included in the image sensor.

Further, as shown in FIG. 265, in the exposure time setting step SA31, the same exposure time is set for each of the odd-numbered image sensor rows of the L image sensor rows included in the image sensor. One of the first and second exposure times is set, and the first and second exposures have the same exposure time for each of the even-numbered image sensor rows of the L image sensor rows. The other of the times may be set. Then, when the exposure time setting step SA31, the image acquisition step SA32, and the information acquisition step SA33 are repeated, in the repeated exposure time setting step SA31, the exposure time setting step SA31 immediately before is set for each of the odd-numbered imaging element arrays. The existing exposure time may be replaced with the exposure time set for each of the even-numbered image pickup device rows.

With this, each time a normal image is acquired, the plurality of imaging element arrays used for the acquisition can be switched to the odd-numbered imaging element arrays and the even-numbered imaging element arrays. As a result, it is possible to display each of the sequentially acquired normal images by interlacing. Also, by complementing two consecutively acquired normal images with each other, a new normal image including an image formed by the odd-numbered image pickup element rows and an image formed by the even-numbered image pickup element rows is generated. be able to.

Further, as shown in FIG. 266, in the exposure time setting step SA31, the setting mode is switched between the normal priority mode and the visible light priority mode, and when the normal priority mode is switched, the first exposure time is set. The number of image pickup elements may be larger than the number of image pickup elements for which the second exposure time is set. When the mode is switched to the visible light priority mode, the number of image pickup devices for which the first exposure time is set may be smaller than the number of image pickup devices for which the second exposure time is set.

With this, when the setting mode is switched to the normal priority mode, the quality of the normal image can be improved, and when switched to the visible light priority mode, the efficiency of receiving information from the light emitter is improved. can do.

Further, as shown in FIG. 264, in the exposure time setting step SA31, the plurality of image pickup elements for which the first exposure time is set and the plurality of image pickup elements for which the second exposure time is set are in a checkered pattern ( The exposure time of the image sensor may be set for each image sensor included in the image sensor so as to be distributed like a checkered pattern).

As a result, the plurality of image pickup elements for which the first exposure time is set and the plurality of image pickup elements for which the second exposure time is set are evenly distributed, so that the image quality is not biased in the horizontal direction and the vertical direction. No normal image and bright line image can be acquired.

FIG. 270B is a block diagram of a receiver in Embodiment 12.

The receiving device A30 is the above-mentioned receiver that executes, for example, the processes shown in FIGS.

That is, the receiving device A30 is a receiving device that receives information from a light emitting body whose brightness changes, and includes a multiple exposure time setting unit A31, an imaging unit A32, and a decoding unit A33. The multiple-exposure-time setting unit A31 sets the first exposure time to some of the K image sensors (K is an integer of 4 or more) included in the image sensor, and sets the first exposure time to K image sensors. A second exposure time shorter than the first exposure time is set for the remaining plurality of image pickup devices among the image pickup devices. The imaging unit A32 causes the image sensor to capture an image of a subject, which is a light-emitting body that changes in brightness, with the set first and second exposure times. A normal image corresponding to the output is acquired, and an image corresponding to the output from the plurality of image pickup devices for which the second exposure time is set, the bright line corresponding to each of the plurality of exposure lines included in the image sensor. A bright line image that is an image including is acquired. The decoding unit A33 acquires information by decoding a plurality of bright line patterns included in the acquired bright line image. In such a receiving device A30, the same effect as that of the above-described receiving program can be obtained.

Next, the display of the content regarding the received visible light signal will be described.

271 and 272 are diagrams showing an example of display on the receiver when a visible light signal is received.

As illustrated in (a) of FIG. 271, when the receiver images the transmitter 10020d, the receiver displays an image 10020a showing the transmitter 10020d. Further, the receiver generates and displays the image 10020b by superimposing the object 10020e on the image 10020a. The object 10020e is an image showing a place where the image of the transmitter 10020d is located and that the visible light signal from the transmitter 10020d is being received. The object 10020e may be an image that differs depending on the reception state of the visible light signal (reception state, state of searching for transmitter, degree of reception progress, reception speed, error rate, etc.). For example, the receiver changes the color of the object 1020e, the thickness of the line, the type of the line (single line, double line, dotted line, or the like), the interval between the dotted lines, or the like. This allows the user to recognize the reception state. Next, the receiver generates and displays the image 10020c by superimposing the image showing the content of the acquired data as the acquired data image 10020f on the image 10020a. The acquired data is data associated with the received visible light signal or the ID indicated by the received visible light signal.

When displaying the acquired data image 10020f, the receiver displays the acquired data image 10020f like a balloon from the transmitter 10020d or near the transmitter 10020d, as shown in (a) of FIG. 271. The acquired data image 10020f is displayed on the screen. Also, the receiver may display the acquired data image 10020f so that the acquired data image 10020f gradually approaches from the transmitter 10020d to the receiver side, as shown in FIG. 271(b). This allows the user to recognize from which transmitter the acquired data image 10020f is based on the visible light signal received. The receiver may also display the acquired data image 10020f so that the acquired data image 10020f gradually emerges from the edge of the display of the receiver, as shown in FIG. 272. This allows the user to easily recognize that the visible light signal has been acquired at that time.

Next, AR (Augmented Reality) will be described.

FIG. 273 is a diagram showing an example of a display of the acquired data image 10020f.

When the image of the transmitter moves in the display, the receiver moves the acquired data image 10020f according to the movement of the image of the transmitter. This allows the user to recognize that the acquired data image 10020f corresponds to the transmitter. Further, the receiver may display the acquired data image 10020f in association with another image instead of the image of the transmitter. As a result, AR display can be performed.

Next, saving and discarding of acquired data will be described.

FIG. 274 is a diagram showing an example of an operation for saving or discarding acquired data.

For example, as illustrated in (a) of FIG. 274, when the user swipes down the acquired data image 10020f, the receiver stores the acquired data shown by the acquired data image 10020f. .. The receiver arranges the acquired data image 10020f showing the acquired acquired data at the end of another acquired data image showing one or more acquired acquired data. This allows the user to recognize that the acquired data indicated by the acquired data image 10020f is the last saved acquired data. For example, as shown in (a) of FIG. 274, the receiver arranges the acquired data image 10020f in the forefront among the plurality of acquired data images.

Also, as shown in (b) of FIG. 274, when the user swipes the acquired data image 10020f to the right, the receiver discards the acquired data indicated by the acquired data image 10020f. Alternatively, the receiver may discard the acquisition data shown by the acquisition data image 10020f when the transmitter image is framed out of the display by the user moving the receiver. Note that the same effect as described above can be obtained regardless of whether the swipe direction is up, down, left, or right. The receiver may display the swipe direction corresponding to save or discard. This allows the user to recognize that the operation can save or discard.

Next, browsing of acquired data will be described.

FIG. 275 is a diagram illustrating a display example when browsing acquired data.

As illustrated in (a) of FIG. 275, the receiver displays the acquired data images of the plurality of acquired acquired data in a small size by being superimposed on the lower end of the display. At this time, when the user taps a part of the displayed acquired data image, the receiver displays each of the acquired data images in a large size, as shown in (b) of FIG. 275. This makes it possible to display the acquired data images in a large size only when it is necessary to browse the acquired data, and to effectively use the display for other display when unnecessary.

When the user taps the acquired data image to be displayed in the state shown in (b) of FIG. 275, the receiver displays the tapped acquired data image in a larger size as shown in (c) of FIG. 275. A lot of information is displayed in the acquired data image. Also, when the user taps the backside display button 10024a, the receiver displays the backside of the acquired data image and displays other data related to the acquired data.

Next, turning off the camera shake correction at the time of estimating the accident position will be described.

The receiver disables (turns off) the image stabilization, or converts the captured image according to the image stabilization direction and the image stabilization amount to obtain the accurate image capturing direction and accurately estimate the self position. You can do it. The captured image is an image obtained by the image capturing section of the receiver. Self-position estimation is that the receiver estimates its own position. In self-position estimation, specifically, the receiver identifies the position of the transmitter based on the received visible light signal, and based on the size, position, or shape of the transmitter shown in the captured image, reception is performed. Specify the relative positional relationship between the aircraft and the transmitter. Then, the receiver estimates the position of the receiver based on the position of the transmitter and the relative positional relationship between the receiver and the transmitter.

Also, during partial read-out in which imaging is performed using only a part of the exposure lines as shown in FIG. 262, that is, when imaging as shown in FIG. 262 is performed, the transmitter will frame out due to slight blurring of the receiver. Will end up. In such a case, the receiver can continuously receive the signal by enabling the image stabilization.

Next, self-position estimation using an asymmetrical light emitting unit will be described.

FIG. 276 is a diagram illustrating an example of a transmitter in Embodiment 12.

The above-described transmitter includes a light emitting unit, and the visible light signal is transmitted by changing the brightness of the light emitting unit. In the above-described self-position estimation, the receiver is based on the shape of the transmitter (specifically, the light emitting unit) in the captured image, and the receiver and the transmitter are defined as a relative positional relationship between the receiver and the transmitter. Find the relative angle to the transmitter. Here, for example, as shown in FIG. 276, when the transmitter includes the light emitting unit 10090a having a rotationally symmetrical shape, as described above, it is determined that the transmitter is based on the shape of the transmitter in the captured image. The relative angle with the receiver cannot be accurately determined. Therefore, it is desirable that the transmitter includes a light emitting unit having a shape not rotationally symmetrical. This allows the receiver to accurately determine the above relative angle. That is, since the azimuth sensor for acquiring the angle has a large error in the measurement result, the receiver can perform accurate self-position estimation by using the relative angle obtained by the above method.

Here, as shown in FIG. 276, the transmitter may include a light emitting unit 10090b that is not in a completely rotationally symmetrical shape. The shape of the light emitting unit 10090b is symmetrical with respect to rotation of 90°, but is not completely rotationally symmetrical. In this case, the receiver can roughly determine the relative angle between the receiver and the transmitter by obtaining the rough angle with the azimuth sensor and using the shape of the transmitter in the captured image. It is possible to perform accurate self-position estimation.

Further, the transmitter may include the light emitting unit 10090c shown in FIG. 276. The shape of the light emitting unit 10090c is basically rotationally symmetrical. However, since the light guide plate and the like are installed in a part of the light emitting unit 10090c, the shape of the light emitting unit 10090c is not rotationally symmetrical.

Further, the transmitter may include the light emitting unit 10090d shown in FIG. 276. Each of the light emitting units 10090d is provided with illumination having a rotationally symmetrical shape. However, the overall shape of the light emitting unit 10090d configured by arranging them in combination is not rotationally symmetrical. Therefore, the receiver can perform accurate self-position estimation by imaging the transmitter. Further, all the lights included in the light emitting unit 10090d do not have to be lights for visible light communication in which the brightness changes to transmit a visible light signal, and only some of the lights are lights for visible light communication. May be.

The transmitter may also include the light emitting unit 10090e and the object 10090f shown in FIG. 276. Here, the object 10090f is an object (for example, a fire alarm or a pipe) configured such that the positional relationship with the light emitting unit 10090e does not change. Since the shape of the combination of the light emitting unit 10090e and the object 10090f is not rotationally symmetric, the receiver can accurately perform self-position estimation by capturing an image of the light emitting unit 10090e and the object 10090f.

Next, a time series process of self-position estimation will be described.

Each time the receiver captures an image, it can perform self-position estimation from the position and shape of the transmitter in the captured image. As a result, the receiver can estimate the moving direction and distance of the receiver during imaging. In addition, the receiver can perform more accurate self-position estimation by performing triangulation using a plurality of frames or images. By integrating the estimation results using a plurality of images and the estimation results using a plurality of images of different combinations, the receiver can more accurately perform self-position estimation. At this time, the receiver can estimate the self-position more accurately by giving importance to the results estimated from the latest captured images and integrating them.

Next, skipping of optical black will be described.

FIG. 277 is a diagram illustrating an example of a reception method in Embodiment 12. Note that the horizontal axis of the graph shown in FIG. 277 represents time, and the vertical axis represents the position of each exposure line in the image sensor. Further, the solid line arrow in the graph indicates the time (exposure timing) when the exposure of each exposure line in the image sensor is started.

At the time of normal imaging, the receiver reads the horizontal optical black signal from the image sensor as shown in FIG. 277(a), but skips the horizontal optical black signal as shown in FIG. 277(b). May be. As a result, continuous visible light signals can be received.

Horizontal optical black is an optical black in the horizontal direction on the exposure line. The vertical optical black is a portion of the optical black other than the horizontal optical black.

Since the receiver adjusts the black level according to the signal read from the optical black, the black level can be adjusted by using the optical black at the start of visible light imaging as in the normal imaging. When the vertical optical black is available, the receiver can perform the black level adjustment using only the vertical optical black, so that the continuous reception and the black level adjustment can be performed. When the visible light imaging is continued, the receiver may adjust the black level using the horizontal optical black every predetermined time. In the case of alternately performing normal imaging and visible light imaging, the receiver skips the horizontal optical black signal when continuously performing visible light imaging, and otherwise reads the horizontal optical black signal. Then, the receiver can adjust the black level based on the read signal, thereby adjusting the black level while continuously receiving the visible light signal. The receiver may adjust the black level by setting the darkest portion of the visible light captured image to be black.

In this way, by using only the vertical optical black as the optical black from which the signal is read, continuous visible light signals can be received. Further, by providing a mode for skipping the horizontal optical black signal, it is possible to adjust the black level during normal imaging and to perform continuous communication as necessary during visible light imaging. Further, by skipping the horizontal optical black signal, the difference in the timing of starting the exposure between the exposure lines becomes large, so that it is possible to receive the visible light signal from the transmitter, which shows only a small image.

Next, an identifier indicating the type of transmitter will be described.

The transmitter may add a transmitter identifier indicating the type of transmitter to the visible light signal and transmit the visible light signal. In this case, the receiver can perform the receiving operation according to the type of the transmitter when the transmitter identifier is received. For example, when the transmitter identifier indicates digital signage, the transmitter transmits a visible light signal including a content ID indicating which content is currently displayed, in addition to the transmitter ID for performing individual identification of the transmitter. Are sending as. By separately handling these IDs based on the transmitter identifier, the receiver can display information according to the content currently displayed by the transmitter. Further, for example, when the transmitter identifier indicates digital signage or an emergency light, the receiver can reduce the reception error by increasing the sensitivity and capturing an image.

(Embodiment 13)
In the present embodiment, each application example using a receiver such as the smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

First, the header pattern in this embodiment will be described.

FIG. 278 is a diagram showing an example of the header pattern in the present embodiment.

The transmitter divides the data to be transmitted into packets and transmits the packets. The packet is composed of, for example, a header and a body. The brightness change pattern of the header, that is, the header pattern needs to be a brightness change pattern that does not exist in the body. As a result, the position of the packet can be specified in the data that is continuously transmitted.

For example, the transmitted data is modulated by the 4PPM modulation method. Specifically, in this 4PPM, the data to be transmitted is modulated into a luminance change pattern in which one of the four slots indicates "0" and the remaining three slots indicate "1". Therefore, when the data to be continuously transmitted is modulated, the number of consecutive slots indicating “0” is 2 or less, and the number of slots indicating “0” is included in 4 slots and the next 4 slots. Is 2 or less.

Therefore, in this embodiment, the header pattern is “111111111000” shown in (a) of FIG. 278, “111111110001” shown in (b), “111111101001” shown in (c), or “111111100101” shown in (d). It is expressed as. As a result, the header and the data (that is, the body) continuously transmitted can be identified. That is, in the header pattern shown in (a) of FIG. 278, it is possible to identify that the four slots are a part of the header only by looking at the four slots “1000” after the header pattern. In this case, the number of slots indicating “0” in the header is 3, and the maximum number of consecutive slots indicating “0” is 4, so that the receiver can easily recognize the brightness change. That is, the receiver can easily receive data from a small transmitter or a distant transmitter.

In the header pattern shown in (b) of FIG. 278, it is possible to identify that the five slots are a part of the header only by looking at the five slots "10001" after the header pattern. In this case, the maximum number of consecutive slots indicating “0” is 3, which is smaller than that in the case of (a) of FIG. 278, so that the flicker due to the brightness change can be suppressed. As a result, the load on the circuit of the transmitter or the design requirement can be suppressed. That is, the capacitor can be downsized, and the power consumption, the heat generation amount, or the load on the power supply can be suppressed.

Further, in the header pattern shown in (c) or (d) of FIG. 278, if only the 6 slots “101001” or “100101” after the header pattern are seen, it is identified that the 6 slots are part of the header. can do. In this case, the maximum number of consecutive slots indicating “0” is 2, which is smaller than that in the case of (b) of FIG. 278, so that the flicker due to the brightness change can be further suppressed.

FIG. 279 is a diagram for describing an example of a packet structure of a communication protocol in this embodiment.

The transmitter divides the data to be transmitted into packets and transmits the packets. A packet is composed of a header, an address part, a data part, and an error correction code part. The position of the packet in the continuous data can be specified by using the brightness change pattern of the header as a brightness change pattern that does not exist in other parts. The data part stores a part of the divided data, and the address part stores an address indicating which part of the entire data of the data part. The error correction code unit is a code for detecting or correcting a reception error of a part or all of a packet (specifically, ECC1, ECC2, or ECC3 shown in FIG. 279, and these are collectively called an error correction code. ) Is stored.

ECC1 is an error correction code in the address part. By providing the error correction code of the address part instead of the error correction code of the entire packet, the reliability of the address part can be made higher than the reliability of the whole. As a result, when a plurality of packets are received, by comparing the data parts of the packets having the same address, the data part can be verified and the reception error rate can be reduced. Even if the error correction code of the address part is made longer than the error correction code of the data part, the same effect can be obtained.

ECC2 and ECC3 are error correction codes in the data section. The number of error correction code parts may be other than two. By dividing the error correction code in the data section into multiple parts, error correction can be performed using the error correction codes that have been received, and highly reliable data can be received without receiving the entire packet. You can

The transmitter may transmit a predetermined number or less of error correction codes. As a result, the receiver can receive data at high speed. This transmission method is effective for a transmitter having a small light emitting portion and high brightness, such as a downlight. This is because when the brightness is high, the error occurrence probability is low, and therefore many error correction codes are not necessary. If ECC3 is not sent, the next header is sent from ECC2, which means that the high brightness state continues for four slots or more, and the receiver can recognize that part is not ECC3.

As shown in (b) of FIG. 279, the header, address part, and ECC1 are transmitted at a frequency lower than that of the data part, ECC2, and ECC3. Conversely, the data part, ECC2 and ECC3 are transmitted at a higher frequency than the header, address part and ECC1. As a result, the reception error rate for the data such as the header can be reduced, and the large data in the data portion can be transmitted at high speed.

As described above, in this embodiment, the packet includes the first error correction code (ECC2, ECC3) for the data part and the second error correction code (ECC1) for the address part. Then, when the receiver receives the packet, the receiver receives the address part and the second error correction code transmitted from the transmitter due to the luminance change according to the second frequency. Further, the receiver receives from the transmitter the data part and the first error correction code transmitted by the luminance change according to the first frequency higher than the second frequency.

Here, a receiving method for comparing data portions having the same address will be described.

FIG. 280 is a flowchart showing an example of a reception method in this embodiment.

The receiver receives the packet (step S10101) and performs error correction (step S10102). Then, the receiver determines whether or not the packet having the same address as the received packet has already been received (step S10103). Here, when it is determined that the data is received (Y in step 10103), the receiver compares the data. That is, the receiver determines whether the data parts are equal (step S10104). Here, when it is determined that they are not equal (N in step S10104), the receiver further determines whether the difference in the plurality of data portions is equal to or more than a predetermined number, specifically, the number of different bits or the luminance. It is determined whether the number of slots having different states is equal to or larger than a predetermined number (step S10105). Here, if it is determined that the number is equal to or more than the predetermined number (N in step S10105), the receiver discards the packet that has already been received (step S10106). This makes it possible to avoid interference with a packet received from a previous transmitter when starting to receive a packet from another transmitter. On the other hand, when determining that the number of packets is not equal to or more than the predetermined number (N in step S10105), the receiver sets the data of the data part having the largest number of packets having the same data part as the data of the address (step S10107). Or, the receiver takes the most significant bit of equal bits as the value of that bit of the address. Alternatively, the receiver demodulates the data of the address by setting the brightness state having the largest number of equal brightness states as the brightness state of the slot of the address.

As described above, in the present embodiment, the receiver first obtains the first packet including the data part and the address part from the plurality of bright line patterns. Next, the receiver, among the at least one packet already acquired before the first packet, includes at least one second packet which is a packet including the same address portion as that of the first packet. It is determined whether or not the packet exists. Next, when the receiver determines that the at least one second packet is present, the receiver determines whether or not the respective data parts of the at least one second packet and the first packet are all equal. judge. If it is determined that the respective data parts are not all equal, the receiver determines, in each of the at least one second packet, the first packet among the parts included in the data part of the second packet. It is determined whether or not the number of parts different from each part included in the data part of (3) is more than a predetermined number. Here, if there is a second packet in which it is determined that the number of different portions of the at least one second packet is greater than or equal to a predetermined number, the receiver determines the at least one second packet. Discard. On the other hand, if there is no second packet in which the number of different portions of the at least one second packet is greater than or equal to the predetermined number, the receiver determines that the first packet and the at least one second packet. Among these packets, a plurality of packets having the largest number of packets having the same data part are specified. Then, the receiver decodes at least a part of the visible light identifier (ID) by decoding the data part included in each of the plurality of packets as the data part corresponding to the address part included in the first packet. get.

With this, when a plurality of packets having the same address part are received, even if the data parts of the packets are different, the appropriate data part can be decoded, and at least a part of the visible light identifier can be decoded. You can get it right. That is, a plurality of packets having the same address part transmitted from the same transmitter basically have the same data part. However, when the receiver switches the transmitter that is the source of the packet, the receiver may receive a plurality of packets having different data parts even though they have the same address part. In such a case, in the present embodiment, as in step S10106 of FIG. 280, the already received packet (second packet) is discarded and the data part of the latest packet (first packet) is discarded. Can be decoded as the correct data part corresponding to the address part. Further, even if the transmitter is not switched as described above, the data parts of a plurality of packets having the same address part may be slightly different depending on the transmission/reception status of the visible light signal. In such a case, in the present embodiment, as in step S10107 of FIG. 280, an appropriate data part can be decrypted by so-called majority voting.

Here, a receiving method for demodulating the data of the data part from a plurality of packets will be described.

FIG. 281 is a flowchart illustrating an example of a reception method in this embodiment.

First, the receiver receives the packet (step S10111) and corrects the error in the address part (step S10112). At this time, the receiver does not demodulate the data part and holds the pixel value obtained by imaging as it is. Then, the receiver determines whether or not there is a predetermined number or more of packets having the same address among the plurality of packets that have already been received (step S10113). Here, if it is determined that the packet exists (Y in step S10113), the receiver performs demodulation processing by combining the pixel values of the portions corresponding to the data portions of a plurality of packets having the same address (step S10114).

As described above, in the receiving method according to the present embodiment, the first packet including the data portion and the address portion is acquired from the plurality of bright line patterns. Then, out of at least one packet already acquired before the first packet, there is a predetermined number or more of second packets which are packets including the same address part as the address part of the first packet. Or not. When it is determined that the second packet is present in the predetermined number or more, the pixel values of the partial area of the bright line image corresponding to the respective data portions of the second packet or more in the predetermined number and the first value The pixel value of a partial area of the bright line image corresponding to the data part of the packet of is matched. That is, the pixel values are added. At least a part of the visible light identifier (ID) is acquired by calculating the composite pixel value by the addition and decoding the data part including the composite pixel value.

Since the timings at which a plurality of packets are received are different from each other, the pixel value of the data part is a value that reflects the brightness of the transmitter at slightly different times. Therefore, the part demodulated as described above includes a larger amount of data (number of samples) than the data part of a single packet. As a result, the data section can be demodulated more accurately. Further, by increasing the number of samples, it is possible to demodulate a signal modulated at a higher modulation frequency.

As shown in (b) of FIG. 279, the data part and its error correction code part are modulated at a higher frequency than the header part, the address part, and the error correction code part of the address part. With the above demodulation method, it is possible to demodulate even after being modulated with a high modulation frequency in the data part and thereafter, so this configuration can shorten the transmission time of the entire packet, and from a farther distance and a smaller light source. However, the visible light signal can be received faster.

Next, a receiving method for receiving data of a variable length address will be described.

282 is a flowchart showing an example of a reception method in this embodiment.

The receiver receives the packet (step S10121) and determines whether or not a packet in which all the bits of the data part are 0 (hereinafter referred to as a 0 termination packet) is received (step S10122). Here, when it is determined that the packet has been received, that is, when it is determined that the 0-terminal packet is present (Y in step S10122), the receiver determines whether or not all the packets having the addresses equal to or lower than the address of the 0-terminal packet are prepared, that is, It is determined whether or not it is received (step S10123). The address is set to a value that increases for each of the packets generated by dividing the data to be transmitted according to the transmission order of those packets. When the receiver determines that all the packets are complete (Y in step S10123), it determines that the address of the 0-end packet is the last address of the packet transmitted from the transmitter. Then, the receiver restores the data by connecting the data of the packets of the respective addresses up to the 0 terminal packet (step S10124). Further, the receiver checks the restored data for errors (step S10125). This allows you to send and receive variable-length address data even if you do not know how many pieces of data are to be sent, that is, even if the address is variable-length instead of fixed-length. More IDs can be transmitted and received with high efficiency.

As described above, in the present embodiment, the receiver acquires a plurality of packets each including a data part and an address part from a plurality of bright line patterns. Then, the receiver determines whether or not there is a 0-terminated packet, which is a packet in which all the bits included in the data part indicate 0, among the acquired plurality of packets. If the receiver determines that there is a 0-termination packet, the receiver is N packets (N is an integer of 1 or more) that is the packet including the address portion associated with the address portion of the 0-termination packet among the plurality of packets. ) Determines whether or not all related packets exist. Next, when the receiver determines that all N related packets are present, the receiver obtains a visible light identifier (ID) by arranging and decoding the respective data parts of the N related packets. Here, the address part associated with the address part of the 0-end packet is an address part that is smaller than the address shown in the address part of the 0-end packet and indicates an address of 0 or more.

Next, a receiving method using an exposure time longer than the cycle of the modulation frequency will be described.

283 and 284 are diagrams for explaining a receiving method in which the receiver in this embodiment uses an exposure time longer than the cycle of the modulation frequency (modulation cycle).

For example, as shown in (a) of FIG. 283, if the exposure time is set to a time equal to the modulation cycle, the visible light signal may not be received correctly. The modulation cycle is the time for one slot described above. That is, in such a case, there are few exposure lines (exposure lines shown in black in FIG. 283) that reflect the state of the brightness of a certain slot. As a result, it is difficult to estimate the brightness of the transmitter when the pixel values of these exposure lines happen to contain a lot of noise.

On the other hand, for example, as shown in (b) of FIG. 283, when the exposure time is set to a time longer than the modulation cycle, the visible light signal can be correctly received. That is, in such a case, since many exposure lines reflect the luminance of a certain slot, the luminance of the transmitter can be estimated from the pixel values of many exposure lines, which is resistant to noise.

On the contrary, if the exposure time is too long, the visible light signal cannot be received correctly.

For example, as shown in (a) of FIG. 284, when the exposure time is equal to the modulation cycle, the luminance change received by the receiver (that is, the pixel value change of each exposure line) is used for transmission. Follow changes in brightness. However, as shown in (b) of FIG. 284, when the exposure time is three times the modulation period, the luminance change received by the receiver can sufficiently follow the luminance change used for transmission. Can not. Further, as shown in (c) of FIG. 284, when the exposure time is 10 times the modulation cycle, the luminance change received by the receiver cannot follow the luminance change used for transmission at all. That is, the longer the exposure time, the higher the noise resistance because the luminance can be estimated from many exposure lines, but the longer the exposure time, the lower the identification margin or the smaller the identification margin, and the lower the noise resistance. With these balances, the noise resistance can be maximized by setting the exposure time to about 2 to 5 times the modulation cycle.

Next, the packet division number will be described.

FIG. 285 is a diagram showing an efficient division number with respect to the size of transmission data.

When the transmitter transmits data by changing the brightness, if all the data to be transmitted (transmission data) is included in one packet, the data size of the packet is large. However, as described using FIG. 279, when the transmission data is divided into a plurality of partial data and the partial data is included in each packet, the data size of each packet becomes small. Here, the receiver receives the packet by imaging. However, the larger the data size of the packet, the more difficult it is for the receiver to receive the packet in one image pickup, and the image pickup must be repeated.

Therefore, as shown in (a) and (b) of FIG. 285, it is desirable for the transmitter to increase the number of divisions of the transmission data as the data size of the transmission data increases. However, if the number of divisions is too large, the transmission data cannot be restored unless all of the partial data are received, and conversely, the reception efficiency decreases.

Therefore, as shown in (a) of FIG. 285, the data size of the address (address size) is variable, and the data size of the transmission data is 2-16 bits, 16-24 bits, 24-64 bits, 66-. In the case of 78 bits, 78-128 bits, 128 bits or more, the transmission data is 1-2 pieces, 2-4 pieces, 4 pieces, 4-6 pieces, 6-8 pieces, 7 pieces or more, respectively. By dividing, the transmission data can be efficiently transmitted by a visible light signal. Further, as shown in (b) of FIG. 285, the data size of the address (address size) is fixed to 4 bits, and the data size of the transmission data is 2-8 bits, 8-16 bits, 16-30 bits, In case of 30-64 bits, 66-80 bits, 80-96 bits, 96-132 bits, 132 bits or more, 1-2, 2-3, 2-4, 4-5, respectively. When the transmission data is divided into 4-7 pieces, 6 pieces, 6-8 pieces, 7 pieces or more of partial data, the transmission data can be efficiently transmitted by a visible light signal.

In addition, the transmitter sequentially changes the brightness based on each packet including each of the plurality of partial data. For example, the transmitter changes the brightness based on the packet in the address order of each packet. Further, the transmitter may again perform the luminance change based on the plurality of partial data in an order different from the address order. This allows the receiver to reliably receive each partial data.

Next, a method of setting the notification operation by the receiver will be described.

FIG. 286A is a diagram showing an example of a setting method in this embodiment.

First, the receiver acquires the notification operation identifier for identifying the notification operation and the priority of the notification operation identifier (specifically, the identifier indicating the priority) from the server near the receiver. (Step S10131). Here, the notification operation is a reception that notifies the user of the receiver that each packet including each of the plurality of partial data is received by the receiver when the packet is transmitted by the brightness change. This is the operation of the machine. For example, the operation is sounding, vibration, or screen display.

Next, the receiver receives the packetized visible light signal, that is, each packet including each of the plurality of partial data (step S10132). Here, the receiver acquires the notification operation identifier and the priority of the notification operation identifier (specifically, the identifier indicating the priority) included in the visible light signal (step S10133).

Further, the receiver is configured to set the current notification operation of the receiver, that is, the notification operation identifier preset in the receiver and the priority of the notification operation identifier (specifically, an identifier indicating the priority). ) And are read (step S10134). The notification operation identifier preset in the receiver is set, for example, by a user operation.

Then, the receiver selects the identifier having the highest priority from among the preset notification operation identifiers and the notification operation identifiers acquired in steps S10131 and S10133 (step S10135). Next, the receiver performs the operation indicated by the selected notification operation identifier by resetting the selected notification operation identifier to itself, and notifies the user of the reception of the visible light signal (step S10136).

Note that the receiver may select one of the two notification operation identifiers having a high priority, without performing either step S10131 or step S10133.

Note that the priority of notification operation identifiers transmitted from servers installed in theaters or museums, or the priority of notification operation identifiers included in visible light signals transmitted in those facilities is set to high. Good. As a result, regardless of the user's setting, it is possible to prevent the reception notification from sounding in the facility. In other facilities, by setting the priority of the notification operation identifier to be low, the receiver can notify the reception by the operation according to the user's setting.

FIG. 286B is a diagram showing another example of the setting method in the present embodiment.

First, the receiver acquires the notification operation identifier for identifying the notification operation and the priority of the notification operation identifier (specifically, the identifier indicating the priority) from the server near the receiver. (Step S10141). Next, the receiver receives the packetized visible light signal, that is, each packet including each of the plurality of partial data (step S10142). Here, the receiver acquires the notification operation identifier and the priority of the notification operation identifier (specifically, the identifier indicating the priority) included in the visible light signal (step S10143).

Further, the receiver is configured to set the current notification operation of the receiver, that is, the notification operation identifier preset in the receiver and the priority of the notification operation identifier (specifically, an identifier indicating the priority). ) And are read (step S10144).

Then, the receiver includes an operation notification identifier indicating the operation of prohibiting the generation of the notification sound in the notification operation identifier set in advance and the notification operation identifiers acquired in step S10141 and step S10143, respectively. It is determined whether or not (step S10145). Here, if it is determined that it is included (Y in step S10145), the receiver sounds a notification sound for notifying reception completion (step S10146). On the other hand, if it is determined that it is not included (N in step S10145), the receiver notifies the user of the completion of reception by, for example, vibration (step S10147).

Note that the receiver does not perform either step S10141 or step S10143, and determines whether the two notification operation identifiers include an operation notification identifier indicating an operation that prohibits the generation of notification sound. You may.

In addition, the receiver may perform self-position estimation based on the image obtained by imaging, and notify the user of the reception by the operation associated with the estimated position or the facility at that position.

FIG. 287A is a flowchart showing processing of the information processing program in the thirteenth embodiment.

This information processing program is a program for changing the brightness of the above-described transmitter light emitter according to the number of divisions shown in FIG.

That is, this information processing program is an information processing program that causes a computer to process the information to be transmitted in order to transmit the information to be transmitted by the change in brightness. Specifically, this information processing program includes an encoding step SA41 for generating an encoded signal by encoding the information to be transmitted, and the number of bits of the generated encoded signal falls within the range of 24 to 64 bits. In some cases, the computer is caused to execute a division step SA42 for dividing the encoded signal into four partial signals and an output step SA43 for sequentially outputting the four partial signals. It should be noted that these partial signals are output as packets shown in (a) of FIG. 279. Further, the information processing program may cause the computer to specify the number of bits of the encoded signal and determine the number of partial signals based on the specified number of bits. In this case, the information processing program causes the computer to generate the determined number of partial signals by dividing the encoded signal.

Accordingly, when the number of bits of the encoded signal is in the range of 24 to 64 bits, the encoded signal is divided into four partial signals and output. As a result, when the luminance of the light emitter changes in accordance with the output four partial signals, each of the four partial signals is transmitted as a visible light signal and received by the receiver. Here, the larger the number of bits of the output signal, the more difficult it is for the receiver to properly receive the signal by imaging, and the reception efficiency decreases. Therefore, it is desirable to divide the signal into a signal having a small number of bits, that is, a small signal. However, if the signal is subdivided into too many small signals, the receiver will not be able to receive the original signal without receiving each of all the small signals individually, resulting in poor reception efficiency. Therefore, as described above, when the number of bits of the encoded signal is in the range of 24 to 64 bits, the encoded signal is divided into four partial signals and sequentially output to indicate the information to be transmitted. The encoded signal can be transmitted as a visible light signal with the best reception efficiency. As a result, communication between various devices can be enabled.

In the output step SA43, the four partial signals may be output according to the first order, and the four partial signals may be output again according to the second order different from the first order.

As a result, these four partial signals are repeatedly output in a different order, so that when each output signal is transmitted to the receiver as a visible light signal, the reception efficiency of these four partial signals is improved. It can be further increased. That is, even if the four partial signals are repeatedly output in the same order, the same partial signal may not be received by the receiver. However, by changing the order, it is possible to prevent such a case from occurring.

Further, as shown in FIGS. 286A and 286B, in the output step SA43, the notification operation identifier may be further attached to the four partial signals and output. The notification operation identifier is for identifying the operation of the receiver that notifies the user of the receiver that the four partial signals have been received when the four partial signals are transmitted by the luminance change and received by the receiver. Is the identifier of.

Thereby, when the notification operation identifier is transmitted as a visible light signal and is received by the receiver, the receiver receives four partial signals to the user according to the operation identified by the notification operation identifier. You can be notified. That is, the notification operation by the receiver can be set on the side that transmits the information to be transmitted.

Further, as shown in FIGS. 286A and 286B, in the output step SA43, a priority identifier for identifying the priority of the notification operation identifier may be further attached to the four partial signals and output.

Accordingly, when the priority identifier and the notification operation identifier are transmitted as a visible light signal and received by the receiver, the receiver handles the notification operation identifier according to the priority identified by the priority identifier. be able to. That is, when the receiver has acquired another notification operation identifier, the receiver is identified by the notification operation identifier identified by the notification operation identifier transmitted as the visible light signal and the other notification operation identifier. One of the notification operations can be selected based on its priority.

FIG. 287B is a block diagram of an information processing device in Embodiment 13.

The information processing apparatus A40 is an apparatus for changing the brightness of the light emitting body (light emitting unit) of the transmitter described above according to the division number shown in FIG.

That is, the information processing apparatus A40 is an apparatus that processes the information to be transmitted in order to transmit the information to be transmitted by changing the brightness. Specifically, the information processing apparatus A40 includes an encoding unit A41 that generates an encoded signal by encoding information to be transmitted, and the number of bits of the generated encoded signal is in the range of 24 to 64 bits. In the case of 1), the division unit A42 that divides the encoded signal into four partial signals and the output unit A43 that sequentially outputs the four partial signals are provided. In such an information processing apparatus A40, the same effects as those of the above information processing program can be obtained.

An information processing program according to an aspect of the present invention is an information processing program that causes a computer to process the information to be transmitted in order to transmit the information to be transmitted by a luminance change, and encodes the information to be transmitted. A coding step of generating a coded signal by doing so, and a dividing step of dividing the coded signal into four partial signals when the number of bits of the generated coded signal is in the range of 24 to 64 bits. And an output step of sequentially outputting the four partial signals, causing the computer to execute.

Thereby, as shown in FIGS. 284 to 287B, when the number of bits of the encoded signal is in the range of 24 to 64 bits, the encoded signal is divided into four partial signals and output. As a result, when the luminance of the light emitter changes in accordance with the output four partial signals, each of the four partial signals is transmitted as a visible light signal and received by the receiver. Here, the greater the number of bits of the output signal, the more difficult it is for the receiver to properly receive the signal by imaging, and the reception efficiency decreases. Therefore, it is desirable to divide the signal into a signal having a small number of bits, that is, a small signal. However, if the signal is subdivided into many smaller signals too much, the receiver will not be able to receive the original signal without individually receiving all of the smaller signals, thus reducing the reception efficiency. Therefore, as described above, when the number of bits of the encoded signal is in the range of 24 to 64 bits, the encoded signal is divided into four partial signals and sequentially output to indicate the information to be transmitted. The encoded signal can be transmitted as a visible light signal with the best reception efficiency. As a result, communication between various devices can be enabled.

In the output step, the four partial signals may be output according to a first order, and the four partial signals may be output again according to a second order different from the first order.

As a result, these four partial signals are repeatedly output in a different order, so that when each output signal is transmitted to the receiver as a visible light signal, the reception efficiency of these four partial signals is improved. It can be further increased. That is, even if the four partial signals are repeatedly output in the same order, the same partial signal may not be received by the receiver. However, by changing the order, it is possible to prevent such a case from occurring.

Further, in the output step, a notification operation identifier is further attached to the four partial signals, and the notification operation identifier is output when the four partial signals are transmitted by a luminance change and received by a receiver. In addition, it may be an identifier for identifying the operation of the receiver for notifying the user of the receiver that the four partial signals have been received.

Thereby, when the notification operation identifier is transmitted as a visible light signal and is received by the receiver, the receiver receives four partial signals to the user according to the operation identified by the notification operation identifier. You can be notified. That is, the notification operation by the receiver can be set on the side that transmits the information to be transmitted.

Further, in the output step, a priority identifier for identifying the priority of the notification operation identifier may be further output along with the four partial signals.

Accordingly, when the priority identifier and the notification operation identifier are transmitted as a visible light signal and received by the receiver, the receiver handles the notification operation identifier according to the priority identified by the priority identifier. be able to. That is, when the receiver has acquired another notification operation identifier, the receiver is identified by the notification operation identifier identified by the notification operation identifier transmitted as the visible light signal and the other notification operation identifier. One of the notification operations can be selected based on its priority.

Next, the registration of the network connection of the electronic device will be described.

FIG. 288 is a diagram for describing an application example of the transmitting and receiving system in this embodiment.

This transmission/reception system includes a transmitter 10131b configured as an electronic device such as a washing machine, a receiver 10131a configured as a smartphone, and a communication device 10131c configured as an access point or a router.

FIG. 289 is a flowchart showing the processing operation of the transmission/reception system in the present embodiment.

When the start button is pressed (step S10165), the transmitter 10131b sends information for connecting to itself such as SSID, password, IP address, MAC address, or encryption key to Wi-Fi, Bluetooth (registered trademark). ), or via Ethernet (registered trademark) or the like (step S10166), and waits for connection. The transmitter 10131b may directly or indirectly transmit these pieces of information. When transmitting indirectly, the transmitter 10131b transmits the ID associated with those pieces of information. The receiver 10131a receiving the ID downloads the information associated with the ID from the server or the like, for example.

The receiver 10131a receives the information (step S10151), connects to the transmitter 10131b, and connects to the communication device 10131c configured as an access point or a router (SSID, password, IP address, MAC address, Alternatively, the encryption key or the like) is transmitted to the transmitter 10131b (step S10152). The receiver 10131a registers information (MAC address, IP address, encryption key, etc.) for the transmitter 10131b to connect to the communication device 10131c in the communication device 10131c, and makes the communication device 10131c wait for the connection. Further, the receiver 10131a notifies the transmitter 10131b that the preparation for connection from the transmitter 10131b to the communication device 10131c is completed (step S10153).

The transmitter 10131b disconnects from the receiver 10131a (step S10168) and connects to the communication device 10131c (step S10169). If the connection is successful (Y in step S10170), the transmitter 10131b notifies the receiver 10131a of the successful connection via the communication device 10131c, and notifies the user of the successful connection by screen display, LED status, voice, or the like. (Step S10171). If the connection is unsuccessful (N in step S10170), the transmitter 10131b notifies the receiver 10131a of the connection failure by visible light communication, and notifies the user similarly to the successful case (step S10172). The successful connection may be notified by visible light communication.

The receiver 10131a connects to the communication device 10131c (step S10154), and if there is no notification of connection success or failure (N in step S10155 and N in step S10156), the receiver 10131a can access the transmitter 10131b via the communication device 10131c. It is confirmed whether or not (step S10157). If not possible (N in step S10157), the receiver 10131a determines whether or not the connection to the transmitter 10131b using the information received from the transmitter 10131b has been performed a predetermined number of times or more (step S10158). Here, when it is determined that the reception has not been performed the predetermined number of times or more (N in step S10158), the receiver 10131a repeats the processing from step S10152. On the other hand, if it is determined that the processing has been performed the predetermined number of times or more (Y in step S10158), the receiver 10131a notifies the user of the processing failure (step S10159). Further, when the receiver 10131a determines in step S10156 that the notification of successful connection has been made (Y in step S10156), the receiver 10131a notifies the user of processing success (step S10160). That is, the receiver 10131a notifies the user of whether or not the transmitter 10131b has been able to connect to the communication device 10131c, by a screen display, voice, or the like. Accordingly, the transmitter 10131b can be connected to the communication device 10131c without requiring the user to make complicated input.

Next, registration of the network connection of the electronic device (when connecting via another electronic device) will be described.

FIG. 290 is a diagram for describing an application example of the transmission/reception system in this embodiment.

This transmission/reception system includes an air conditioner 10133b, a transmitter 10133c configured as an electronic device such as a wireless adapter connected to the air conditioner 10133b, a receiver 10133a configured as a smartphone, a communication device configured as an access point or a router, for example. 10133d and another electronic device 10133e configured as, for example, a wireless adapter, a wireless access point, a router, or the like.

FIG. 291 is a flowchart showing the processing operation of the transmission/reception system in this embodiment. Hereinafter, the air conditioner 10133b or the transmitter 10133c is referred to as an electronic device A, and the electronic device 10133e is referred to as an electronic device B.

First, when the start button is pressed (step S10188), the electronic device A transmits information (individual ID, password, IP address, MAC address, encryption key, or the like) for connecting to itself (step S10189). , Waits for connection (step S10190). The electronic device A may directly transmit these pieces of information, or may indirectly transmit these pieces of information, as described above.

The receiver 10133a receives the information from the electronic device A (step S10181), and transmits the information to the electronic device B (step S10182). Upon receiving the information (step S10196), the electronic device B connects to the electronic device A according to the received information (step S10197). Then, the electronic device B determines whether or not the connection with the electronic device A is established (step S10198), and notifies the receiver 10133a of the success or failure (step S10199 or step S101200).

If the electronic device A is connected to the electronic device B within a predetermined time (Y in step S10191), the electronic device A notifies the receiver 10133a of the successful connection via the electronic device B (step S10192), and if not connected (step S10192). In step S10191 N), the receiver 10133a is notified of the connection failure by visible light communication (step S10193). In addition, the electronic device A notifies the user of the success or failure of the connection by the screen display, the light emitting state, the sound, or the like. Accordingly, the electronic device A (transmitter 10133c) can be connected to the electronic device B (electronic device 10133e) without requiring the user to perform complicated input. Note that the air conditioner 10133b and the transmitter 10133c shown in FIG. 290 may be integrally configured, and similarly, the communication device 10133d and the electronic device 10133e may be integrally configured.

Next, transmission of appropriate imaging information will be described.

FIG. 292 is a diagram for describing an application example of the transmission/reception system in this embodiment.

This transmission/reception system includes a receiver 10135a configured as, for example, a digital still camera or a digital video camera, and a transmitter 10135b configured as, for example, lighting.

FIG. 293 is a flowchart showing the processing operation of the transmission/reception system in the present embodiment.

First, the receiver 10135a sends an imaging information transmission command to the transmitter 10135b (step S10211). Next, the transmitter 10135b receives the imaging information transmission command, the imaging information transmission button is pressed, the imaging information transmission switch is turned on, and the power is turned on (step S10221). Y), the imaging information is transmitted (step S10222). The imaging information transmission command is a command for transmitting the imaging information, and the imaging information indicates, for example, color temperature of illumination, spectral distribution, illuminance or light distribution. The transmitter 10135b may directly transmit the imaging information or may indirectly transmit the imaging information, as described above. When transmitting indirectly, the transmitter 10135b transmits the ID associated with the imaging information. The receiver 10135a that has received the ID downloads the imaging information associated with the ID from the server or the like, for example. At this time, the transmitter 10135b uses a method for transmitting a transmission stop command to itself (frequency of radio wave, infrared ray, or sound wave transmitting the transmission stop command, or SSID, password or IP address for connecting to self-confidence, etc.). ) May be transmitted.

Upon receiving the imaging information (step S10212), the receiver 10135a transmits a transmission stop command to the transmitter 10135b (step S10213). Here, when the transmitter 10135b receives the transmission stop command from the receiver 10135a (step S10223), the transmitter 10135b stops transmitting the imaging information and uniformly emits light (step S10224).

Further, the receiver 10135a sets the imaging parameter according to the imaging information received in step S10212 (step S10214), or notifies the user of the imaging information. The imaging parameter is, for example, white balance, exposure time, focal length, sensitivity or scene mode. As a result, it is possible to pick up an image with optimum settings according to the illumination. Next, the receiver 10135a takes an image after the transmission of the imaging information from the transmitter 10135b is stopped (Y in step S10215) (step S10216). As a result, it is possible to capture an image without changing the brightness of the subject due to signal transmission. Note that the receiver 10135a may transmit a transmission start command for prompting the start of transmission of imaging information to the transmitter 10135b after step S10216 (step S10217).

Next, the display of the charge state will be described.

FIG. 294 is a diagram for describing an application example of the transmitter in this embodiment.

For example, the transmitter 10137b configured as a charger includes a light emitting unit, and transmits a visible light signal indicating the charge state of the battery from the light emitting unit. Thereby, the charge state of the battery can be notified without providing an expensive display device. When a small LED is used as the light emitting unit, the visible light signal cannot be received unless the LED is imaged from near. Further, in the transmitter 10137c having a protrusion near the LED, the protrusion makes it difficult to take a close-up of the LED. Therefore, the visible light signal from the transmitter 10137b having no protrusion in the vicinity of the LED can be received more easily than the visible light signal from the transmitter 10137c.

(Embodiment 14)
In the present embodiment, each application example using a receiver such as the smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

First, the transmission in the demo mode and in the failure will be described.

FIG. 295 is a diagram illustrating an example of operation of a transmitter in this embodiment.

If an error has occurred, the transmitter sends a signal indicating that an error has occurred, or a signal corresponding to the error code, and Can convey error content. The receiver can repair the error or report the error content to the service center appropriately by showing an appropriate response according to the error content.

If the transmitter is in demo mode, it will send a demo code. Thereby, for example, when the transmitter, which is a product, is being demoted at the store, the visitor can receive the demo code and obtain the product description associated with the demo code. Whether or not it is in demo mode is determined by setting the operation settings of the transmitter to demo mode, inserting a store CAS card, not inserting a CAS card, or inserting a recording medium. It can be judged from the points.

Next, signal transmission from the remote controller will be described.

FIG. 296 is a diagram illustrating an example of operation of a transmitter in this embodiment.

For example, when a transmitter configured as a remote controller for an air conditioner receives main body information, the transmitter transmits the main body information, so that the receiver receives information on a remote body from a nearby transmitter. can do. The receiver can also receive information from the main body existing in a place where visible light communication is impossible, such as over a network.

Next, a process of transmitting only in a bright place will be described.

FIG. 297 is a diagram illustrating an example of operation of a transmitter in this embodiment.

The transmitter transmits when the ambient brightness is above a certain level and stops the transmission when the ambient brightness is below a certain level. As a result, the transmitter configured as, for example, an advertisement for a train can automatically stop its operation when the vehicle enters the garage, and can reduce battery consumption.

Next, content distribution (association change/scheduling) according to the display of the transmitter will be described.

FIG. 298 is a diagram illustrating an example of operation of a transmitter in this embodiment.

The transmitter associates the content desired to be acquired by the receiver with the transmission ID at the display timing of the content to be displayed. Every time the display content is changed, the change in association is registered in the server.

If the display timing of the display content is known, the transmitter sets the server so that another content is passed to the receiver in accordance with the change timing of the display content. The server transmits the content according to the set schedule to the receiver when the receiver requests the content associated with the transmission ID.

Thereby, when the transmitter configured as, for example, digital signage is changing the display contents one after another, the receiver can acquire the content matched with the content displayed by the transmitter.

Next, content distribution (synchronization by time) according to the display of the transmitter will be described.

FIG. 299 is a diagram illustrating an example of operation of a transmitter in this embodiment.

In response to a content acquisition request associated with a predetermined ID, different content is delivered in advance depending on the time of day, which is registered in the server.

The transmitter synchronizes the time with the server, adjusts the timing so that a predetermined portion is displayed at a predetermined time, and displays the content.

Thereby, when the transmitter configured as, for example, digital signage is changing the display contents one after another, the receiver can acquire the content matched with the content displayed by the transmitter.

Next, content distribution (transmission of display time) according to the display of the transmitter will be described.

FIG. 300 is a diagram illustrating an example of operation of a transmitter and a receiver in this embodiment.

The transmitter transmits the display time of the content being displayed in addition to the ID of the transmitter. The content display time is information that can identify the content that is currently displayed, and can be expressed as, for example, the elapsed time from the start time of the content.

The receiver acquires the content associated with the received ID from the server and displays the content at the received display time. Thereby, when the transmitter configured as, for example, digital signage is changing the display contents one after another, the receiver can acquire the content matched with the content displayed by the transmitter.

In addition, the receiver changes the content to be displayed as time passes. Thereby, even if the signal is not received again when the display content of the transmitter changes, the content matched with the display content is displayed.

Next, uploading of data according to the user's permission status will be described.

FIG. 301 is a diagram illustrating an example of operation of a receiver in this embodiment.

If the user has registered an account, the receiver can access the information that the user has permitted access to when registering the account (location of the receiver, telephone number, ID, installed applications, user The age, sex, occupation, taste, etc.) is sent to the server together with the received ID.

If the user does not have account registration and the user has permitted the upload of the above information, the same is transmitted to the server, and if not permitted, only the received ID is transmitted to the server. ..

As a result, the user can receive the content that matches the situation at the time of reception and the personality of the user, and the server can utilize the information of the user for data analysis.

Next, activation of the content reproduction application will be described.

FIG. 302 is a diagram illustrating an example of operation of a receiver in this embodiment.

The receiver acquires the content associated with the received ID from the server. When the running application can handle the acquired content (can display and play), the running application displays and plays the acquired content. If not, check whether or not an app that can be used is installed in the receiver, and if it is installed, start the app and display/play the acquired content. If it is not installed, it automatically installs it, displays a message prompting the user to install it, displays a download screen, and displays/plays the acquired content after the installation.

As a result, it is possible to properly handle (display/playback, etc.) the acquired content.

Next, the activation of the designated application will be described.

FIG. 303 is a diagram illustrating an example of operation of a receiver in this embodiment.

The receiver acquires, from the server, the content associated with the received ID and the information (application ID) designating the application to be activated. If the running application is the designated application, the acquired content is displayed/played. When the designated application is installed in the receiver, the designated application is activated to display/play the acquired content. If it is not installed, it automatically installs it, displays a message prompting the user to install it, displays a download screen, and displays/plays the acquired content after the installation.

The receiver may acquire only the application ID from the server and activate the designated application.

The receiver may make the specified settings. The receiver may set the specified parameter and activate the specified application.

Next, selection between streaming reception and normal reception will be described.

FIG. 304 is a diagram illustrating an example of operation of a receiver in this embodiment.

When the value of the predetermined address of the received data is a predetermined value, or when the received data includes a predetermined identifier, the receiver determines that the signal is streamed and the method of receiving the streaming data is determined. To receive. If not, the normal reception method is used.

Thereby, it is possible to receive the signal regardless of whether the signal is transmitted by the streaming distribution or the normal distribution.

Next, private data will be described.

FIG. 305 is a diagram illustrating an example of operation of a receiver in this embodiment.

If the value of the received ID is within a predetermined range or if it contains a predetermined identifier, the receiver refers to the table in the application, and if the received ID exists in the table, it is specified in that table. Get content Otherwise, the content set as the reception ID is acquired from the server.

As a result, the content can be received without registration in the server. In addition, since communication with the server is not performed, a quick response can be obtained.

Next, the setting of the exposure time according to the frequency will be described.

FIG. 306 is a diagram illustrating an example of operation of a receiver in this embodiment.

The receiver senses the signal and recognizes the modulation frequency of the signal. The receiver sets the exposure time according to the cycle of the modulation frequency (modulation cycle). For example, a signal can be easily received by setting the exposure time to the same level as the modulation cycle. Further, for example, by setting the exposure time to an integral multiple of the modulation cycle or a value close to it (approximately ±30%), it is possible to facilitate reception of a signal by convolutional decoding.

Next, the optimum parameter setting of the transmitter will be described.

FIG. 307 is a diagram illustrating an example of operation of a receiver in this embodiment.

The receiver transmits the current position information and information related to the user (address, sex, age, preference, etc.) to the server in addition to the data received from the transmitter. The server sends parameters to the receiver so that the transmitter operates optimally according to the received information. The receiver sets the received parameter if it can be set in the transmitter. If it cannot be set, the parameter is displayed and the user is prompted by the transmitter to set the parameter.

This allows, for example, the washing machine to operate by optimizing the water quality of the area where the transmitter is used, or the rice cooker to cook rice in a manner that is most suitable for the type of rice the user is using. It can be done.

Next, the identifier indicating the data structure will be described.

FIG. 308 is a diagram illustrating an example of a structure of transmission data in this embodiment.

The information sent includes an identifier, which allows the receiver to know the composition of the parts that follow. For example, the length of the data, the type and length of the error correction code, the division point of the data, and the like can be specified.

As a result, the transmitter can change the type and length of the data body and the error correction code according to the characteristics of the transmitter and the communication path. Further, the transmitter can cause the receiver to acquire an ID corresponding to the content ID by transmitting the content ID in addition to the ID of the transmitter.

(Embodiment 15)
In the present embodiment, each application example using the receiver such as the smartphone in each of the above embodiments and the transmitter that transmits information as the blinking pattern of the LED or the organic EL will be described.

FIG. 309 is a diagram for describing operation of the receiver in this embodiment.

The receiver 1210a according to the present embodiment switches the shutter speed between a high speed and a low speed, for example, on a frame-by-frame basis when performing continuous shooting with the image sensor. Further, the receiver 1210a switches the processing for the frame to the barcode recognition processing and the visible light recognition processing based on the frame obtained by the shooting. Here, the barcode recognition process is a process of decoding a barcode shown in a frame obtained by a low shutter speed. The visible light recognition process is a process of decoding the above-described bright line pattern reflected in a frame obtained by a high shutter speed.

Such a receiver 1210a includes a video input unit 1211, a barcode/visible light identification unit 1212, a barcode recognition unit 1212a, a visible light recognition unit 1212b, and an output unit 1213.

The video input unit 1211 includes an image sensor, and switches the shutter speed for shooting with the image sensor. That is, the video input unit 1211 alternately switches the shutter speed between low speed and high speed in frame units, for example. More specifically, the video input unit 1211 switches the shutter speed to high speed for odd-numbered frames and switches the shutter speed to low speed for even-numbered frames. Shooting at a low shutter speed is shooting in the normal shooting mode described above, and shooting at a high shutter speed is shooting in the visible light communication mode described above. That is, when the shutter speed is low, the exposure time of each exposure line included in the image sensor is long, and a normally photographed image showing a subject is obtained as a frame. Further, when the shutter speed is high, the exposure time of each exposure line included in the image sensor is short, and the visible light communication image in which the above-mentioned bright line is projected is obtained as a frame.

The barcode/visible light identifying unit 1212 switches the process for the image by determining whether the barcode or the bright line appears in the image obtained by the image input unit 1211. For example, if the barcode appears in the frame obtained by shooting at a low shutter speed, the barcode/visible light identifying unit 1212 causes the barcode recognizing unit 1212a to perform processing on the image. On the other hand, if the bright line appears in the image obtained by shooting at the high shutter speed, the barcode/visible light identifying unit 1212 causes the visible light recognizing unit 1212b to perform processing on the image.

The barcode recognition unit 1212a decodes the barcode appearing in the frame obtained by shooting at a low shutter speed. The barcode recognition unit 1212a obtains barcode data (for example, barcode identifier) by the decoding, and outputs the barcode identifier to the output unit 1213. The barcode may be a one-dimensional code or a two-dimensional code (for example, QR code (registered trademark)).

The visible light recognizing unit 1212b decodes the bright line pattern appearing in the frame obtained by photographing at a high shutter speed. The visible light recognizing unit 1212b acquires the visible light data (for example, the visible light identifier) by the decoding, and outputs the visible light identifier to the output unit 1213. The visible light data is the visible light signal described above.

The output unit 1213 displays only the frames obtained by shooting at a low shutter speed. Therefore, when the subject captured by the video input unit 1211 is a barcode, the output unit 1213 displays the barcode. When the subject photographed by the video input unit 1211 is a digital signage that transmits a visible light signal, the output unit 1213 displays the image of the digital signage without displaying the bright line pattern. Then, when acquiring the barcode identifier, the output unit 1213 acquires the information associated with the barcode identifier from, for example, a server and displays the information. Further, when the output unit 1213 acquires the visible light identifier, the output unit 1213 acquires information associated with the visible light identifier from, for example, a server and displays the information.

That is, the receiver 1210a, which is a terminal device, includes an image sensor, and while alternately switching the shutter speed of the image sensor between the first speed and the second speed that is higher than the first speed, Take a series of photos with. Then, (a) when the subject photographed by the image sensor is a barcode, the receiver 1210a obtains an image showing the barcode by photographing when the shutter speed is the first speed, The barcode identifier is obtained by decoding the barcode shown in the image. Further, (b) when the subject to be photographed by the image sensor is a light source (for example, digital signage), the receiver 1210a includes a plurality of image sensors included in the image sensor by photographing when the shutter speed is the second speed. A bright line image that is an image including bright lines corresponding to the respective exposure lines is acquired. Then, the receiver 1210a acquires a visible light signal as a visible light identifier by decoding a pattern of a plurality of bright lines included in the acquired bright line image. Further, the receiver 1210a displays an image obtained by shooting when the shutter speed is the first speed.

In the receiver 1210a according to the present embodiment as described above, it is possible to decode the barcode and receive the visible light signal by switching between the barcode recognition processing and the visible light recognition processing. Furthermore, power consumption can be suppressed by switching.

The receiver according to the present embodiment may perform image recognition processing at the same time as visible light processing, instead of barcode recognition processing.

FIG. 310A is a diagram for explaining another operation of the receiver in this embodiment.

The receiver 1210b according to the present embodiment switches the shutter speed between a high speed and a low speed, for example, on a frame-by-frame basis when performing continuous shooting with the image sensor. Further, the receiver 1210b simultaneously executes the image recognition processing and the visible light recognition processing on the image (frame) obtained by the shooting. The image recognition process is a process of recognizing a subject in a frame obtained by a low shutter speed.

Such a receiver 1210b includes a video input unit 1211, an image recognition unit 1212c, a visible light recognition unit 1212b, and an output unit 1215.

The video input unit 1211 includes an image sensor, and switches the shutter speed for shooting with the image sensor. That is, the video input unit 1211 alternately switches the shutter speed between low speed and high speed in frame units, for example. More specifically, the video input unit 1211 switches the shutter speed to high speed for odd-numbered frames and switches the shutter speed to low speed for even-numbered frames. Shooting at a low shutter speed is shooting in the normal shooting mode described above, and shooting at a high shutter speed is shooting in the visible light communication mode described above. That is, when the shutter speed is low, the exposure time of each exposure line included in the image sensor is long, and a normally photographed image showing a subject is obtained as a frame. Further, when the shutter speed is high, the exposure time of each exposure line included in the image sensor is short, and the visible light communication image in which the above-mentioned bright line is projected is obtained as a frame.

The image recognition unit 1212c recognizes a subject appearing in a frame obtained by shooting at a low shutter speed, and specifies the position of the subject in the frame. As a result of the recognition, the image recognition unit 1212c determines whether the subject is an AR (Augmented Reality) target (hereinafter, referred to as an AR target). When the image recognition unit 1212c determines that the subject is an AR object, the image recognition unit 1212c generates image recognition data that is data for displaying information about the subject (for example, the position of the subject and the AR marker), and the like. The AR marker is output to the output unit 1215.

The output unit 1215, like the output unit 1213 described above, displays only the frames obtained by shooting at a low shutter speed. Therefore, when the subject photographed by the video input unit 1211 is a digital signage that transmits a visible light signal, the output unit 1213 displays the image of the digital signage without displaying the bright line pattern. Further, when the output unit 1215 acquires the image recognition data from the image recognition unit 1212c, the output unit 1215 superimposes a white frame-shaped indicator surrounding the subject on the frame based on the position of the subject in the frame indicated by the image recognition data. ..

FIG. 310B is a diagram showing an example of an indicator displayed by the output unit 1215.

The output unit 1215 superimposes a white frame-shaped indicator 1215b surrounding a subject image 1215a configured as, for example, digital signage, on the frame. That is, the output unit 1215 displays the indicator 1215b indicating the image-recognized subject. Further, when the output unit 1215 acquires the visible light identifier from the visible light recognition unit 1212b, the output unit 1215 changes the color of the indicator 1215b from white to red, for example.

FIG. 310C is a diagram showing a display example of an AR.

The output unit 1215 further acquires information about the subject, which is associated with the visible light identifier, as related information from, for example, a server. The output unit 1215 describes related information in the AR marker 1215c indicated by the image recognition data, and displays the AR marker 1215c in which the related information is described in association with the image 1215a of the subject in the frame.

In the receiver 1210b according to the present embodiment as described above, the AR using visible light communication can be realized by simultaneously performing the image recognition process and the visible light recognition process. Note that the receiver 1210a shown in FIG. 310A may also display the indicator 1215b shown in FIG. 310B, similarly to the receiver 1210b. In this case, when the barcode is recognized in the frame obtained by shooting at a low shutter speed, the receiver 1210a displays a white frame-shaped indicator 1215b surrounding the barcode. Then, when the barcode is decoded, the receiver 1210a changes the color of the indicator 1215b from white to red. Similarly, when the receiver 1210a recognizes a bright line pattern in a frame obtained by shooting at a high shutter speed, the receiver 1210a identifies a region in the low speed frame corresponding to a region having the bright line pattern. For example, if the digital signage is transmitting a visible light signal, the image of the digital signage in the slow frame is identified. The low-speed frame is a frame obtained by shooting at a low shutter speed. Then, the receiver 1210a superimposes and displays the white frame-shaped indicator 1215b surrounding the specified portion (for example, the image of the digital signage described above) in the low speed frame on the low speed frame. When the bright line pattern is decoded, the receiver 1210a changes the color of the indicator 1215b from white to red.

FIG. 311A is a diagram for describing an example of a transmitter in this embodiment.

The transmitter 1220a in the present embodiment transmits a visible light signal in synchronization with the transmitter 1230. That is, the transmitter 1220a transmits the same visible light signal as the visible light signal at the timing when the transmitter 1230 transmits the visible light signal. The transmitter 1230 includes a light emitting unit 1231, and the visible light signal is transmitted by changing the brightness of the light emitting unit 1231.

Such a transmitter 1220a includes a light receiving unit 1221, a signal analyzing unit 1222, a transmission clock adjusting unit 1223a, and a light emitting unit 1224. The light emitting unit 1224 transmits the same visible light signal as the visible light signal transmitted from the transmitter 1230 by changing the luminance. The light receiving unit 1221 receives the visible light from the transmitter 1230 to receive the visible light signal from the transmitter 1230. The signal analysis unit 1222 analyzes the visible light signal received by the light receiving unit 1221 and transmits the analysis result to the transmission clock adjustment unit 1223a. The transmission clock adjustment unit 1223a adjusts the timing of the visible light signal transmitted from the light emitting unit 1224 based on the analysis result. That is, the transmission clock adjusting unit 1223a controls the light emitting unit 1224 so that the timing at which the visible light signal is transmitted from the light emitting unit 1231 of the transmitter 1230 matches the timing at which the visible light signal is transmitted from the light emitting unit 1224. Adjust the brightness change timing.

Accordingly, the waveform of the visible light signal transmitted by the transmitter 1220a and the waveform of the visible light signal transmitted by the transmitter 1230 can be matched in timing.

FIG. 311B is a diagram for describing another example of the transmitter in this embodiment.

Like the transmitter 1220a, the transmitter 1220b in the present embodiment transmits a visible light signal in synchronization with the transmitter 1230. That is, the transmitter 1200b transmits the same visible light signal as the visible light signal at the timing when the transmitter 1230 transmits the visible light signal.

Such a transmitter 1220b includes a first light receiving unit 1221a, a second light receiving unit 1221b, a comparing unit 1225, a transmission clock adjusting unit 1223b, and a light emitting unit 1224.

The first light receiving unit 1221a receives a visible light signal from the transmitter 1230 by receiving the visible light from the transmitter 1230, similarly to the light receiving unit 1221. The second light receiving unit 1221b receives the visible light from the light emitting unit 1224. The comparing unit 1225 compares the first timing when the visible light is received by the first light receiving unit 1221a and the second timing when the visible light is received by the second light receiving unit 1221b. Then, the comparison unit 1225 outputs the difference between the first timing and the second timing (that is, the delay time) to the transmission clock adjustment unit 1223b. The transmission clock adjustment unit 1223b adjusts the timing of the visible light signal transmitted from the light emitting unit 1224 so that the delay time is shortened.

Accordingly, the waveform of the visible light signal transmitted by the transmitter 1220b and the waveform of the visible light signal transmitted by the transmitter 1230 can be more accurately matched in timing.

Note that in the example illustrated in FIGS. 311A and 311B, the two transmitters transmit the same visible light signal, but different visible light signals may be transmitted. That is, when transmitting the same visible light signal, the two transmitters transmit in synchronization with each other as described above. Then, when the two transmitters transmit different visible light signals, only one of the two transmitters transmits the visible light signal, while the other transmitter uniformly turns on or off. To do. After that, one transmitter turns on or off uniformly, while only the other transmitter transmits a visible light signal. Note that the two transmitters may simultaneously transmit different visible light signals.

FIG. 312A is a diagram for describing an example of synchronous transmission by a plurality of transmitters in this embodiment.

The plurality of transmitters 1220 in this embodiment are arranged in, for example, one row as shown in FIG. 312A. Note that these transmitters 1220 have the same configuration as the transmitter 1220a shown in FIG. 311A or the transmitter 1220b shown in FIG. 311B. Each of the plurality of transmitters 1220 as described above transmits a visible light signal in synchronization with one of the transmitters 1220 on both sides of the transmitter 1220.

This allows many transmitters to transmit visible light signals in synchronization.

FIG. 312B is a diagram for describing an example of synchronous transmission by a plurality of transmitters in this embodiment.

One transmitter 1220 among the plurality of transmitters 1220 in the present embodiment serves as a reference for synchronizing the visible light signal, and the remaining plurality of transmitters 1220 have visible light signals so as to match the reference. To send.

Thereby, many transmitters can transmit a visible light signal more accurately in synchronization.

FIG. 313 is a diagram for describing another example of synchronous transmission by a plurality of transmitters in this embodiment.

Each of the plurality of transmitters 1240 in the present embodiment receives a synchronization signal and transmits a visible light signal according to the synchronization signal. Thereby, the visible light signal is synchronously transmitted from each of the plurality of transmitters 1240.

Specifically, each of the plurality of transmitters 1240 includes a control unit 1241, a synchronization control unit 1242, a photocoupler 1243, an LED drive circuit 1244, an LED 1245, and a photodiode 1246.

The control unit 1241 receives the synchronization signal and outputs the synchronization signal to the synchronization control unit 1242.

The LED 1245 is a light source that emits visible light, and blinks (that is, changes in luminance) under the control of the LED drive circuit 1244. This causes the visible light signal to be transmitted from the LED 1245 to the outside of the transmitter 1240.

The photocoupler 1243 electrically insulates the synchronization control unit 1242 and the LED drive circuit 1244, and transmits a signal between them. Specifically, the photocoupler 1243 transmits a later-described transmission start signal transmitted from the synchronization control unit 1242 to the LED drive circuit 1244.

When the LED drive circuit 1244 receives the transmission start signal from the synchronization control unit 1242 via the photocoupler 1243, the LED drive circuit 1244 causes the LED 1245 to start transmitting the visible light signal at the timing when the transmission start signal is received.

The photodiode 1246 detects visible light emitted from the LED 1245 and outputs a detection signal indicating that the visible light has been detected to the synchronization controller 1242.

When receiving the synchronization signal from the control unit 1241, the synchronization control unit 1242 transmits a transmission start signal to the LED drive circuit 1244 via the photo coupler 1243. By transmitting this transmission start signal, the transmission of the visible light signal is started. Further, when the synchronization control unit 1242 receives the detection signal from the photodiode 1246 by transmitting the visible light signal, the synchronization control unit 1242 has a delay that is a difference between the timing at which the detection signal is received and the timing at which the synchronization signal is received from the control unit 1241. Calculate time. When receiving the next synchronization signal from the control unit 1241, the synchronization control unit 1242 adjusts the timing of transmitting the next transmission start signal based on the calculated delay time. That is, the synchronization control unit 1242 adjusts the timing of transmitting the next transmission start signal so that the delay time for the next synchronization signal becomes a preset delay time. As a result, the synchronization control unit 1242 transmits the next transmission start signal at the adjusted timing.

FIG. 314 is a diagram for explaining signal processing in the transmitter 1240.

Upon receiving the synchronization signal, the synchronization control unit 1242 generates a delay time setting signal that causes a delay time setting pulse to occur at a predetermined timing. It should be noted that receiving the synchronization signal is specifically receiving the synchronization pulse. That is, the synchronization control unit 1242 generates the delay time setting signal so that the delay time setting pulse rises at the timing when the above-mentioned setting delay time has elapsed from the fall of the synchronization pulse.

Then, the synchronization control unit 1242 transmits a transmission start signal to the LED drive circuit 1244 via the photocoupler 1243 at a timing delayed by the correction value N obtained last time from the fall of the synchronization pulse. As a result, the LED drive circuit 1244 transmits a visible light signal from the LED 1245. Here, the synchronization control unit 1242 receives the detection signal from the photodiode 1246 at the timing delayed by the sum of the intrinsic delay time and the correction value N from the fall of the synchronization pulse. That is, the transmission of the visible light signal is started from that timing. Hereinafter, that timing is referred to as a transmission start timing. The above-mentioned inherent delay time is a delay time caused by a circuit such as the photocoupler 1243, and is a delay time that occurs even if the synchronization control unit 1242 receives the synchronization signal and immediately transmits the transmission start signal. ..

The synchronization control unit 1242 specifies the time difference from the transmission start timing to the rising edge of the delay time setting pulse as the correction correction value N. Then, the synchronization control unit 1242 calculates and holds the correction value (N+1) by the correction value (N+1)=correction value N+correction correction value N. Thus, when the synchronization control unit 1242 receives the next synchronization signal (synchronization pulse), it transmits a transmission start signal to the LED drive circuit 1244 at a timing delayed by the correction value (N+1) from the falling edge of the synchronization pulse. To do. The correction correction value N can be a negative value as well as a positive value.

As a result, each of the plurality of transmitters 1240 transmits the visible light signal after the set delay time elapses after receiving the synchronization signal (synchronization pulse), so that the visible light signal can be transmitted accurately in synchronization. it can. That is, even if there is a variation in the intrinsic delay time due to the circuit such as the photocoupler 1243 in each of the plurality of transmitters 1240, the visible light from each of the plurality of transmitters 1240 is not affected by the variation. The transmission of the optical signal can be accurately synchronized.

The LED drive circuit consumes a large amount of power and is electrically insulated from the control circuit that handles the synchronization signal by using a photocoupler or the like. Therefore, when such a photocoupler is used, it is difficult to synchronize the transmission of visible light signals from a plurality of transmitters due to the above-mentioned variations in the intrinsic delay time. However, in the plurality of transmitters 1240 according to the present embodiment, the light emitting timing of the LED 1245 is detected by the photodiode 1246, the delay time from the synchronization signal is detected by the synchronization control unit 1242, and the delay time is set by a preset delay. The time is adjusted (the above-mentioned set delay time). As a result, even if there are individual variations in the photo couplers provided in a plurality of transmitters each configured as, for example, an LED illumination, a state in which visible light signals (eg, visible light ID) are highly accurately synchronized from the plurality of LED illuminations. You can send it at.

The LED illumination may be turned on or off during periods other than the visible light signal transmission period. When the light is turned on during the period other than the visible light signal transmission period in the previous period, the first falling edge of the visible light signal may be detected. When the light is turned off except during the visible light signal transmission period, the first rising edge of the visible light signal may be detected.

In the above example, the transmitter 1240 transmits the visible light signal each time the synchronization signal is received, but the visible light signal may be transmitted without receiving the synchronization signal. That is, once the transmitter 1240 transmits the visible light signal in response to the reception of the synchronization signal, the transmitter 1240 may sequentially transmit the visible light signal without receiving the synchronization signal. Specifically, the transmitter 1240 may sequentially transmit the visible light signal 2 to several thousand times for one reception of the synchronization signal. Further, the transmitter 1240 may transmit the visible light signal according to the synchronization signal once every 100 msec or once every several seconds.

Further, when the visible light signal is repeatedly transmitted according to the synchronization signal, the continuity of the light emission of the LED 1245 may be lost due to the set delay time. That is, a slightly longer blanking period may occur. As a result, the blinking of the LED 1245 may be visually recognized by a person, and so-called flicker may occur. Therefore, the transmitter 1240 may transmit the visible light signal according to the synchronization signal at a cycle of 60 Hz or higher. As a result, blinking is performed at high speed, and it becomes difficult for a person to visually recognize the blinking. As a result, the occurrence of flicker can be suppressed. Alternatively, the transmitter 1240 may transmit the visible light signal according to the synchronization signal at a sufficiently long cycle, for example, once every several minutes. This makes it possible to prevent the blinking from being visually recognized by a person, but it is possible to prevent the blinking from being visually recognized repeatedly and continuously, and it is possible to reduce the discomfort that a flicker gives to a person.

(Pre-processing of receiving method)
FIG. 315 is a flowchart showing an example of a reception method in this embodiment. In addition, FIG. 316 is an explanatory diagram illustrating an example of a reception method in this embodiment.

First, the receiver calculates an average value of pixel values of a plurality of pixels arranged in a direction parallel to the exposure line (step S1211). By the central limit theorem, when the pixel values of N pixels are averaged, the expected value of the noise amount becomes N to the power of 1/2, and the SN ratio improves.

Next, the receiver leaves only the portion where the pixel value changes in the same direction in the vertical direction for each of all the colors, and removes the change in the pixel value in the portion where the pixel value changes differently (step S1212). When the transmission signal (visible light signal) is expressed by the brightness of the light emitting unit included in the transmitter, the brightness of the illumination of the transmitter and the backlight of the display changes. At this time, as in the part (b) of FIG. 316, the pixel values of all the colors change in the same direction. In the portions (a) and (c) of FIG. 316, the pixel value changes differently for each color. In these portions, the pixel value varies due to the reception noise or the picture of the display or the signage. Therefore, by removing these variations, the SN ratio can be improved.

Next, the receiver obtains a brightness value (step S1213). Since the luminance is hard to be changed by the color, the influence of the picture on the display or the signage can be eliminated, and the SN ratio can be improved.

Next, the receiver low-pass filters the luminance value (step S1214). In the receiving method according to the present embodiment, since a moving average filter based on the length of the exposure time is applied, almost no signal is included in the high frequency region, and noise becomes dominant. Therefore, the SN ratio can be improved by using a low-pass filter that cuts the high frequency region. Since there are many signal components up to the frequency up to the reciprocal of the exposure time, it is possible to enhance the effect of improving the SN ratio by cutting off frequencies higher than that. When the frequency component included in the signal is finite, the SN ratio can be improved by cutting off a frequency higher than that frequency. A filter (Butterworth filter or the like) that does not include frequency vibration components is suitable for the low-pass filter.

(Reception method by convolutional maximum likelihood decoding)
FIG. 317 is a flowchart showing another example of the reception method in this embodiment. 318 and 319 are diagrams for explaining another example of the receiving method according to the present embodiment. Hereinafter, a reception method when the exposure time is longer than the transmission cycle will be described with reference to these drawings.

When the exposure time is an integral multiple of the transmission cycle, the reception can be performed most accurately. Even if it is not an integral multiple, reception can be performed within a range of about (N±0.33) times (N is an integer).

First, the receiver sets the transmission/reception offset to 0 (step S1221). The transmission/reception offset is a value for correcting the difference between the transmission timing and the reception timing. Since this deviation is unknown, the receiver gradually changes the value of the transmission/reception offset candidate and adopts the value that is most intimate as the transmission/reception offset.

Next, the receiver determines whether the transmission/reception offset is less than the transmission cycle (step S1222). Here, since the reception cycle and the transmission cycle are not synchronized, it is not always possible to obtain the reception value that matches the transmission cycle. Therefore, when the receiver determines in step S1222 that it is less than the transmission cycle (Y in step S1222), the reception value in the vicinity thereof is used for each transmission cycle to obtain a reception value (for example, a pixel value) that matches the transmission cycle. ) Is calculated by interpolation (step S1223). As the interpolation method, linear interpolation, nearest neighbor value, spline interpolation, or the like can be used. Next, the receiver obtains the difference between the reception values obtained for each transmission cycle (step S1224).

FIG. 318 shows an example in which the exposure time is three times the transmission cycle and the transmission signal is a binary value of 0 or 1. The reception value at a certain point is a value obtained by adding three transmission signals. The value of the newly received signal can be found by finding the difference from the received value at the next time point. At this time, since the difference between the received values includes noise, it is not clear which signal is received. Therefore, the receiver calculates the probability (estimated likelihood) of which signal it was (step S1225). If the transmitted signal is x and the difference between the received values is y, this probability can be expressed by the conditional probability P(x|y). However, since P(x|y) is difficult to obtain, the receiver uses Bayes' law to calculate using the value on the right side of P(x|y)∝P(y|x)P(x). ..

It is conceivable to perform this calculation for all received values. When the number of received values is N, the transition pattern of the convolutional state exists as 2 N powers, which is NP-hard, but it can be calculated efficiently by using the Viterbi algorithm.

Most of the state transition paths in FIG. 319 are paths that do not conform to the transmission format. Therefore, the format check is performed every time the state transition is performed, and if the route does not conform to the transmission format, the likelihood of the route being set to 0 can improve the accuracy of estimating the correct received signal. ..

The receiver adds a predetermined value to the transmission/reception offset (step S1226) and repeats the processing from step S1222. When the receiver determines in step S1222 that it is not less than the transmission cycle (N in step S1222), the receiver identifies the highest likelihood of the received signal likelihoods calculated for the respective transmission/reception offsets. Then, the receiver determines whether or not the highest likelihood is equal to or higher than a predetermined value (step S1227). When it is determined that the value is equal to or greater than the predetermined value (Y in step S1227), the receiver uses the received signal with the highest likelihood as the final estimation result. Alternatively, the receiver uses a reception signal having a likelihood equal to or higher than a value obtained by subtracting a predetermined value from the highest likelihood as a reception signal candidate (step S1228). On the other hand, if it is determined in step S1227 that the highest likelihood is less than the predetermined value (N in step S1227), the receiver discards the estimation result (step S1229).

If there is too much noise, it is often impossible to properly estimate the received signal, and at the same time the likelihood becomes low. Therefore, when the likelihood is low, the reliability of the received signal can be improved by discarding the estimation result. Further, even if the input image does not include a valid signal, there is a problem that the maximum likelihood decoding outputs a valid signal as an estimation result. However, in this case as well, the likelihood is low. Therefore, when the likelihood is low, this problem can be avoided by discarding the estimation result.

(Embodiment 16)
[1 Introduction]
Conventional visible light communication systems include a system using a general-purpose image sensor as a light receiving device and a system using a photo sensor or a special high-speed image sensor. The former method is, for example, "PikaPi Camera (registered trademark)" by Casio Computer Co., Ltd. Since the imaging frame rate of many general-purpose image sensors has an upper limit of 30 fps, the luminance change of the light source to be transmitted needs to be at a frequency lower than this. Since such a change in luminance at a slow frequency is visible to the human eye, the illumination cannot be used as a transmitter, and a dedicated transmitter needs to be used. The latter method includes IEEE802.15.7 and CP1223. The modulation frequency used in these methods is as high as 9.6 kHz or more, and since it appears to the human eye to be lit uniformly, it is possible to use illumination as a transmitter. However, the point that the smartphone cannot receive the signal because it requires a dedicated receiving device is an obstacle to its widespread use.

We have developed a method that can receive signals modulated at a speed that is not visible to the human eye by using a general-purpose image sensor installed in today's smartphones as a receiving device. The CMOS image sensor is superior to the CCD image sensor in terms of high-speed response, high integration, low power consumption, and low voltage driving, and is used in almost all smartphones and digital cameras. The image pickup method of the CMOS image sensor is a line scan method in which each line of an image is sequentially exposed, and is known as a cause of causing distortion in an image when a moving object is picked up. We have developed a method (Line Scan Sampling, LSS) for performing sampling at a frequency of 30 kHz or more, which is 1000 times that of the conventional method, by appropriately setting the exposure time by utilizing the characteristics of this line scan method. We also devised a modulation method suitable for this method and implemented it in lighting and displays. It was confirmed that a commercially available smartphone was equipped with the LSS reception method and that signals modulated at 10 kHz or higher could be received.

[2-line scan sampling]
The CMOS image sensor converts the received light into a pixel value and reads it as one-dimensional data in the following procedure.

1. When the photodiode in the pixel is exposed, electric charges are generated according to the exposure amount, and the electric charges are converted into a voltage by the amplifier.

2. The voltage is sent to the vertical signal line by the row selection switch. Here, fixed pattern noise is removed and temporarily stored.

3. The data is sequentially sent to the horizontal signal line by the column selection switch and read as one-dimensional data.

Image sensors used in smartphones and digital cameras today are highly miniaturized, and each pixel does not have a memory. Therefore, the exposure at 1 is not performed at all pixels at the same time, but is performed sequentially in accordance with the timing of 2 that is sequentially performed for each row. That is, since the start and end of exposure are performed at slightly different timings for each row, the image captured by the CMOS image sensor shows images at different times for each row. By using this mechanism, it is possible to sample the change in the luminance of the transmitter at a much higher speed than the sampling for each imaging frame. A line of pixels that are exposed at the same time is called an exposure line.

FIG. 320 is a diagram in which a signal modulated with a modulation frequency of 10 kHz is imaged while changing the exposure time to 1/100, 1/1000, and 1/10,000. The pixel value of the captured image is obtained as a value obtained by multiplying the integrated value of the luminance of the imaging target within the exposure time by a value determined by the brightness of the lens and the sensitivity setting value. Usually, when shooting indoors, an exposure time of about 1/30 to 1/200 is used. When the exposure time Te is sufficiently longer than the modulation cycle Ts, the difference in brightness between the exposure line capturing the brightest period and the exposure line capturing the darkest period can be approximated to Ts/Te. When Te=1/100 seconds and Ts=1/10,000 seconds (10 kHz), the difference in pixel value is only 1%. Therefore, this blinking is not recognized in a photograph taken under normal conditions. However, as shown in (c) of FIG. 320, when the exposure time is shortened, the blinking state clearly appears as the pixel value of the exposure line. By shortening the exposure time, it is possible to capture high frequency luminance changes.

Not all photodiodes in CMOS image sensors are used directly for imaging. The optical black portion is shielded from light, and by subtracting the output potential of this portion from the output potential of the effective pixel, the dark current due to thermal noise is canceled. In addition, there is an invalid portion that exists because of design. In addition, effective pixels are often configured with an aspect ratio close to 4:3, but when the size of the captured image is set to 16:9, the effective pixel portion is cut off at the top and bottom, resulting in invalidity. It is treated the same as the part. Since the image sensor sequentially reads not only the effective pixels but also the optical black and the ineffective portion row by row, the time from when the bottom line of an image is captured to when the top line of the next image is captured. , There is a time difference corresponding to the time for exposing the optical black and the invalid portion. This time is called blanking time.

The time when the luminance change of the light source can be sampled by the LSS is only the time when the exposure line where the light source is imaged is exposed. This state is shown in FIG. Further, even if the light source is displayed on the entire screen, the blanking time described above exists, so that the samples are discontinuous. Therefore, it is necessary to perform signal transmission by a protocol suitable for LSS on the assumption that signal reception is discontinuous. It should be noted that today's smartphones do not have such a function, but as shown in FIG. 322, if it is possible to specify the light source position and set it so that only that part is imaged, continuous reception becomes possible and communication efficiency is improved. Can be dramatically increased.

Sampling frequency If the imaging frequency is 30 fps and the vertical size of the image is 1080 pixels, the number of samplings by LSS is 30×1,080=32,400 times per second, but there is a period during which sampling cannot be performed due to blanking time. The sampling frequency will be faster than this. The blanking time varies depending on the setting for each model and the imaging conditions such as the frame rate and the image resolution, but is generally between 1 and 10 milliseconds, and the sampling frequency is about 33 to 46 kHz.

[3 transmitter conditions]
In order to use the illumination light as a light source for visible light communication, it is necessary that the change in luminance for signal expression is at a level that cannot be perceived by humans. For that purpose, the average luminance (effective luminance) needs to be constant regardless of the signal to be transmitted. Further, it is necessary that the frequency of the luminance change is sufficiently high or the change rate is sufficiently small. The human perceptual limit frequency is called critical flicker frequency (CFF), and is said to be about 60 Hz, although it varies depending on the conditions. However, this is a limit in periodic blinking, and a higher modulation frequency is required in the irregular change for signal representation. Also, when the image is taken by a camera or a video camera, the influence of the brightness change should not appear. As described above, when the exposure time is set in the range used for normal shooting, the influence of the brightness change is small, and it does not pose a problem for still images. However, in the case of capturing a moving image, a shadow like a scan line may be perceived even if the luminance changes at a frequency higher than CFF. This is due to the alias of the deviation between the frame frequency and the signal frequency of moving image shooting, and in order to remove this effect, it is necessary to use a frequency considerably higher than CFF or to reduce the rate of change.

The dimming of illumination includes control based on the amount of current passed through the light source (current control) and control based on the length of time during which light is emitted (PWM control). PWM control cannot be used because the change in brightness is used for signal expression. However, in order to change what was conventionally PWM-controlled, such as the backlight of a display, to current control, a large circuit change is required, which becomes a barrier to the introduction of visible light communication. Therefore, it is desirable that the modulation method includes a function of adjusting the average brightness.

Higher brightness is desirable as the original function of lighting. Since the number of LEDs serving as a light source and the withstand voltage are determined by the maximum brightness, a modulation method in which the ratio of the effective brightness to the maximum brightness (effective brightness rate, Effective Luminance Rate (ELR)) is high is desirable.

A signal can be transmitted from the display by controlling the brightness of the display backlight. However, it is necessary to pay attention to the following points as compared with the case where lighting is used as a transmitter. Since the brightness of the light source is low, the SN ratio is low. The picture on the screen becomes noise, and the SN ratio is further reduced when the picture is dark. To improve the resolution of moving images, it is necessary to turn off the backlight while changing the transmittance of the liquid crystal. is there. The screen refresh rate is higher for higher grade products, up to 240 Hz for current products. In this case, the signal is transmitted intermittently in units of 1/240 seconds.

[4 Modulation method suitable for LSS]
The most important feature of LSS is that reception is discontinuous. Modulation methods corresponding to discontinuous reception include a small symbol method and a large symbol method.

[4.1 Large symbol method]
In the Large symbol method, uniform symbols whose symbol transmission time is longer than the image capturing cycle are used. Note that a uniform symbol refers to a symbol such as a frequency modulation symbol that can be decoded by receiving a part of any part of the symbol. The receiver receives one symbol per image and combines the symbols received from a plurality of images to restore communication data. The method of receiving one symbol per image is similar to the conventional image sensor reception method, but this method has a much larger amount of information per symbol and humans cannot visually recognize the blinking of the light source. There are differences in points. It is possible to restore the communication data by connecting the received data in the order in which they were received.However, for example, if the processing of the imaging frame drops due to the processing load of the receiver, it can be restored normally. And lack reliability. By using a part of the signal for expressing the address, the data can be normally received even in the above case.

The coded signal by frequency modulation has a uniform signal and a large amount of information per symbol, and is suitable as a code of the Large symbol method. An example of frequency modulation by on/off control is shown in (b) of FIG. 323. With simple frequency modulation, the effective luminance rate is 50%, but the effective luminance rate can be increased by fixing the time of one cycle and taking a lot of times with high luminance. FIGS. 323(b) and (c) are examples of frequency analysis of signals of different ELRs at the same frequency, and it can be seen that the frequency represented by the signal can be recognized from the fundamental frequency.

Since the sampling of the signal by the LSS is the average value of the brightness during the exposure period, the moving average filter for the length of the exposure time is applied. The frequency characteristic of this filter is shown in FIG. Therefore, it is necessary to keep the exposure time of the receiver constant and note that the frequency cut off here cannot be used.

[4.2 Small symbol method]
In the Small symbol method, the receiver receives a plurality of symbols during a series of reception times and connects the received parts in a plurality of image frames to restore communication data. If the repetition period of the transmission signal is constant, it is possible to calculate the length of the non-reception period from the imaging frame rate and connect the reception parts. This method is unreliable because the imaging frame rate is controlled to vary depending on the temperature. Therefore, by dividing the communication data into a plurality of packets and adding the header indicating the boundary of the packet and the address indicating the packet number, the received data can be connected regardless of the length of the non-reception period. Further, in the former method, when the ratio of the reception cycle (imaging frame rate) and the transmission cycle is represented by a small integer, a situation occurs in which only the same portion of the communication data can be received, but in the latter method, This problem can be solved by making the packet transmission order random.

The pulse position modulation and frequency modulation are suitable for the Small symbol method because the symbol transmission time is short and the ELR can be increased.

Encoding methods that keep brightness constant during pulse position modulation include Manchester code and four
There is a pulse-position modu-lation (4PPM) code (FIGS. 325 and 326). In each case, the coding efficiency is 50%, but the effective luminance rate is 50% for the Manchester code and 75% for the 4PPM code, and the 4PPM code is superior. FIG. 325 is based on a 4PPM code and is based on a coding method (variable) compatible with brightness adjustment.
4PPM, V4PPM). With this encoding method, the effective luminance rate can be continuously changed from 25% to 75%. In addition, since the rising position of the signal is constant regardless of the brightness, the receiving side has a feature that it can receive without being aware of the brightness setting value. There is a variable PPM (VPPM) system as a brightness adjustment-compatible coding system based on Manchester code, but when the effective brightness rate can be changed from 25% to 75% in the VPPM system, the shortest recognizable pulse width is set. In consideration of the standard, as shown in FIG. 327, the pulse width of 25% of the symbol length becomes the same width as one pulse of 4PPM. At this time, the coding efficiency of V4PPM is twice that of VPPM, and it can be said that V4PPM is excellent.

The frequency-modulated symbol can be received by performing frequency analysis such as discrete cosine transform. This method has the advantage that it can be received even with a long exposure time. However, since the information on the order of the symbols is lost, the available frequency combinations considering the harmonics are limited. In the following experiments, V4PPM is used as the symbol modulation scheme of the Small symbol scheme.

[4.3 Performance Evaluation]
Performance evaluation of two modulation methods is performed. The smartphone P-03E is used as a receiver and the liquid crystal television TH-L47DT5 is used as a transmitter. When rewriting the liquid crystal, the backlight is turned off. The liquid crystal rewriting frequency is 240 Hz, and in the standard mode, the backlight is lit for 75% of the time, so the continuous signal transmission time is 1,000,000/240×0.75=3,125 microseconds. Become. When the exposure time is set to 1/10,000 and a 50% gray screen is displayed on the display screen and an on/off signal of 1 kHz is transmitted, the signal noise power measured by the transmitter and the receiver is shown in FIG. 328. It shows in (a). The following experiment was carried out using a simulation signal ((b) in FIG. 328) simulating this SN ratio. Further, as the received signal, a value obtained by averaging the pixel values of 256 pixels in the direction horizontal to the exposure line was used. The following results are the results of 1,000 trials under each condition.

A single frequency symbol is used as a symbol of the Large symbol method. The receiving error becomes lower as the effective luminance rate approaches 50%, but it is set to 75%, which is the same as the ELR used in the experiment of the Small symbol method. The received signal is calculated by the discrete cosine transform of pixel values in the direction perpendicular to the exposure line. The reception error (difference between the transmission signal frequency and the reception signal frequency) is shown in FIG. 329A. The reception error sharply increased around 9 kHz. This is because the LSS moving average filter shown in FIG. 324 reduces the signal power and is buried in noise. The reason why the reception error is large in the low frequency region is that only a signal with a small period can be transmitted during the transmission period. The reception error rate at each frequency margin is shown in FIGS. 329B to 329F. For example, if the allowable error rate is 5%, a value in the range of 1.6 kHz to 8 kHz can be assigned in 50 Hz steps, and therefore (8,000-1,600)/50=128=7 bits of Information can be expressed. For example, if 2 bits are an address and 5 bits are data, information of 20 bits can be expressed. Since the communication data can be decoded from the image of 4 frames at the highest speed, the effective communication speed when imaged at 30 fps is 150 bps. In actual use, it is necessary to include an error check code to detect a reception error.

V4PPM was used as a symbol of the Small symbol method. FIG. 330 shows the reception success rate at each symbol rate. However, this successful reception rate indicates the rate at which all the symbols in one packet were correctly received. Further, the modulation frequency mentioned here indicates the number of time slots of luminance change included in one second. That is, at a modulation frequency of 10 kHz, it contains 2,500 V4PPM symbols. If the allowable error rate is 5%, the modulation frequency can be 10 kHz. If the entire continuous transmission period is one packet, the boundary can be determined if the beginning of the transmission period is in the ON state (the state where the brightness is high), so the header indicating the packet boundary can be indicated by one slot. Therefore, one packet includes the number of V4PPM symbols shown in (Equation 1) below.

That is, it includes 14-bit information. By assigning 2 bits to an address and 12 bits to data, 48 bits of information can be expressed. Since a plurality of packets can be received if the transmitter in the captured image is large enough, the effective communication speed becomes maximum when all the packets can be received from one image, and becomes 1440 bps when capturing at 30 fps.

Since the Small symbol method has a larger number of bits that can be expressed, this was implemented and the operation was confirmed. The structure of the packet is as in the above-described example, and the 4-bit CRC code is included in the combined 48-bit data. In addition, when packets of different data are received at the same address, data that has received the packet of the same data a large number of times is adopted. When the number of packets of the same data is the same, the reception is continued until one of the data reaches the maximum number independently. When an error is detected by CRC, all received packets are discarded. The distance between the transmitter and the receiver was 4 m. At this distance, at least one packet image is included in one image. In 200 trials, the average reception time was 351 milliseconds, and no error remained after the CRC error check. The expected value of the number of packet receptions required to collect N types of packets can be calculated by the following (Formula 2).

Therefore, when N=4, the expected value is 8.33. Therefore, when there is no error in reception and one packet is received from one image, the expected reception time is 8,33×33=275 milliseconds. The average reception time is slower than this because it is necessary to receive multiple packets due to a reception error, and the reception time is improved by reducing the reception errors by taking measures such as including an error detection code in the packet. Can be made

[5 Conclusion]
Visible light communication is a type of wireless communication using electromagnetic waves in the visible light band visible to humans. It is receiving attention from the social application side that lighting becomes communication infrastructure. As a feature, it does not require the certification of the Radio Law, it does not affect the living body and is safe, does not affect other devices with electromagnetic waves, the source and communication path are visible, so the communication range can be seen at a glance, preventing unauthorized communication Easy, easy to shield, has high directivity and can communicate only with a specific partner, and the energy of communication can be shared with lighting. Further, in addition to the use as bidirectional communication similar to the conventional wireless communication represented by WiFi or the like, a method of using as a sign using one-way communication is considered. For example, it is expected that the location information will be transmitted from ceiling lighting to identify the location indoors where GPS cannot reach.

In this paper, we proposed high-speed sampling using the line scan characteristics of a CMOS image sensor, and confirmed that current smartphones can receive signals modulated at a modulation frequency of 10 kHz.

Since smartphones can receive visible light signals using lighting as a transmitter, various applications are possible. For example, an application in which position information is transmitted from ceiling lighting and the position is specified indoors where GPS does not reach can be expected. In addition, applications such as using a signboard as a transmitter to obtain a coupon with a smartphone or checking vacant seat information are possible.

The visible light communication system proposed in this paper has the following advantages over the illuminance sensor reception system, in addition to being able to use a smartphone as a receiver. Since the received light can be spatially separated, even when a plurality of transmitters are present near each other, signals can be individually received without interference. Further, since the light receiving direction can be specified, the relative position with the light source can be calculated. That is, by acquiring the absolute position of the light source from the received signal, the absolute position of the receiver can be obtained with an accuracy of several cm. In addition, communication can be performed using a display or a signboard as a transmitter. Since the display and signboard have lower brightness and illuminance than illumination, it is difficult for the photosensor to receive the signal. However, the image sensor receiving method can receive the signal regardless of the environmental illuminance. Further, although the movement of the screen causes noise in the display, in the image sensor receiving method, it is possible to select a flat portion with less noise and receive a signal from that portion.

In the future, we plan to improve the reception algorithm and study further improvements in communication performance. In addition, we plan to examine the application example of this visible light communication system and demonstrate its application in industrial applications.

(Embodiment 17)
In the present embodiment, a display system which is a system configured as the transmitter in each of the above embodiments and which displays a video and transmits a visible light signal will be described.

FIG. 331 is a block diagram illustrating a structure of the display system in this embodiment.

The display system in the present embodiment includes a video signal transmitter 1250 for generating and transmitting a video signal, and a video display 1270 for displaying a video and transmitting a visible light signal.

The video signal transmitter 1250 includes a video signal generation unit 1251, a visible light signal generation unit 1252, a visible light synchronization signal generation unit 1253, and a video standard signal transmission unit 1254.

The video signal generation unit 1251 generates a video signal and outputs it to the video standard signal transmission unit 1254. The visible light signal generation unit 1252 generates a visible light signal as an electric signal and outputs it to the video standard signal transmission unit 1254. The visible light synchronization signal generation unit 1253 generates a visible light synchronization signal and outputs it to the video standard signal transmission unit 1254.

The image standard signal transmission unit 1254 outputs the image signal, the visible light signal, and the visible light synchronization signal generated as described above to the image display 1270 via the image standard transmission path group 1260.

The video display device 1270 includes a video standard signal reception unit 1271, a video display unit 1272, and a visible light signal transmission unit 1273.

The video standard signal reception unit 1271 receives the video signal, the visible light signal, and the visible light synchronization signal from the video standard signal transmission unit 1254 via the video standard transmission path group 1260. Then, the video standard signal reception unit 1271 outputs the video signal to the video display unit 1272, and outputs the visible light signal and the visible light synchronization signal to the visible light signal transmission unit 1273.

The video display unit 1272 includes, for example, a liquid crystal display, an organic EL display, a plasma display, or the like, and when receiving a video signal from the video standard signal receiving unit 1271, displays a video corresponding to the video signal. When the image display 1270 is a projector or the like, the image display unit 1272 is composed of a light projecting mechanism including a light source and an optical system, and when the image signal is received from the image standard signal receiving unit 1271, the image signal is received. The image according to is projected on the screen.

The visible light signal transmission unit 1273 acquires the visible light signal and the visible light synchronization signal from the video standard signal reception unit 1271. Further, when the visible light signal transmission unit 1273 receives the visible light synchronization signal, the visible light signal transmission unit 1273 causes the image display unit 1272 to start blinking according to the already-received visible light signal at the timing of the reception. As a result, the image display unit 1272 displays an image and changes the brightness to transmit a visible light signal as a light signal. The visible light signal transmitter 1273 may include a light source such as an LED and change the brightness of the light source.

FIG. 332 is a diagram illustrating a transmission/reception mode between the video standard signal transmission unit 1254 and the video standard signal reception unit 1271.

The image standard signal transmission unit 1254 transmits the image signal, the visible light signal, and the visible light synchronization signal to the image standard signal reception unit 1271 using the plurality of image standard transmission lines included in the image standard transmission line group 1260.

Upon receiving the video signal, the visible light signal, and the visible light synchronization signal, the video standard signal reception unit 1271 outputs the visible light synchronization signal to the visible light signal transmission unit 1273 with priority over the interpretation of the video signal and the visible light signal. To do. As a result, it is possible to prevent the output of the visible light synchronizing signal from being delayed due to the interpretation of the video signal and the visible light signal.

FIG. 333 is a diagram illustrating an example of a specific transmission/reception mode between the video standard signal transmission unit 1254 and the video standard signal reception unit 1271.

The image standard signal transmission unit 1254 transmits the image signal, the visible light signal, and the visible light synchronization signal to the image standard signal reception unit 1271 using the plurality of image standard transmission lines included in the image standard transmission line group 1260. At this time, the video standard signal transmission unit 1254 transmits the video signal and the visible light signal via the video standard transmission path used in the video standard among the plurality of video standard transmission paths included in the video standard transmission path group 1260. It is sent to the video standard signal receiving section 1271. In addition, the video standard signal transmission unit 1254 transmits the visible light synchronization signal to the video standard signal via a video standard transmission line that is not used in the video standard among the plurality of video standard transmission lines included in the video standard transmission line group 1260. It is sent to the receiving unit 1271.

FIG. 334 is a diagram showing another example of a specific transmission/reception mode between the video standard signal sending unit 1254 and the video standard signal receiving unit 1271.

The video standard signal sending unit 1254 sends the video signal and the visible light signal to the video standard signal receiving unit 1271 via the video standard transmission line used in the video standard, as described above. On the other hand, for the visible light synchronization signal, the video standard signal sending unit 1254 may send the visible light sync signal to the video standard signal receiving unit 1271 via a video standard transmission line for future expansion. The video standard transmission line for future expansion is a video standard transmission line provided for future expansion according to the standard.

FIG. 335 is a diagram showing another example of a specific transmission/reception mode between the video standard signal sending unit 1254 and the video standard signal receiving unit 1271.

The video standard signal sending unit 1254 sends the video signal and the visible light signal to the video standard signal receiving unit 1271 via the video standard transmission line used in the video standard, as described above. On the other hand, for the visible light synchronization signal, the video standard signal transmission unit 1254 passes through the video standard transmission line (hereinafter referred to as power transmission line) used for transmitting the power consumed by the video display 1270. The visible light synchronizing signal may be sent to the video standard signal receiving section 1271. As a result, the visible light synchronizing signal is transmitted together with the power. In other words, the video standard signal sending unit 1254 superimposes the visible light synchronization signal on the power and sends the power.

336A and 336B are diagrams showing electric power transmitted through the transmission line for electric power transmission.

When the visible light synchronizing signal is not transmitted through the power transmission line, the voltage determined by the video standard is continuously applied to the power transmission line as shown in FIG. 336A. It On the other hand, when the visible light synchronization signal is transmitted via the power transmission transmission line, as shown in FIG. 336B, the power transmission transmission line has a visible light synchronization signal at a voltage defined by the video standard. Are superimposed. In this case, the visible light sync signal should be adjusted so that the maximum voltage of the visible light sync signal is less than or equal to the rated voltage upper limit of the video standard transmission line and the minimum voltage of the visible light sync signal is greater than or equal to the rated voltage lower limit of the video standard transmission line. Is superimposed on the power. Further, in this case, the visible light synchronizing signal is superimposed on the electric power so that the average value of the voltage during the period in which the visible light synchronizing signal is superimposed becomes equal to the voltage defined by the video standard.

FIG. 337 is a diagram showing another example of a specific transmission/reception mode between the video standard signal sending unit 1254 and the video standard signal receiving unit 1271.

The video standard signal sending unit 1254 sends the video signal and the visible light signal to the video standard signal receiving unit 1271 via the video standard transmission line used in the video standard, as described above. On the other hand, for the visible light synchronizing signal, the video standard signal sending unit 1254 sends the visible light synchronizing signal to the video standard signal receiving unit 1271 via the video standard transmission line used for sending the vertical synchronizing signal in the video standard. May be sent. The vertical synchronizing signal is a signal for synchronizing the video in the vertical direction. The video standard signal sending unit 1254 sends a visible light synchronizing signal as a vertical synchronizing signal.

FIG. 338 is a diagram showing another example of a specific transmission/reception mode between the video standard signal sending unit 1254 and the video standard signal receiving unit 1271.

The video standard signal sending unit 1254 sends the video signal and the visible light signal to the video standard signal receiving unit 1271 via the video standard transmission line used in the video standard, as described above. On the other hand, for the visible light synchronization signal, the video standard signal transmission unit 1254 passes through a video standard transmission line (hereinafter referred to as a mixed transmission line) used for transmitting the video signal, the control signal and the vertical synchronization signal in the video standard. The visible light synchronizing signal may be sent to the video standard signal receiving section 1271. The video standard signal sending unit 1254 sends a visible light synchronizing signal as a vertical synchronizing signal.

In this case, the video standard signal reception unit 1271 extracts the visible light synchronization signal from the signal transmitted and received via the mixed transmission path, prior to the interpretation of the video signal and the control signal, and extracts the visible light synchronization signal. The visible light synchronizing signal is output to the visible light signal transmitting unit 1273.

As described above, in the present embodiment, since the visible light synchronizing signal is extracted with priority over the interpretation of the video signal and the visible light signal, the visible light synchronizing signal is output by the interpretation of the video signal and the visible light signal. It is possible to suppress the delay.

(Embodiment 18)
The present disclosure relates to a display device capable of outputting a visible light communication signal and a control method thereof.

For example, JP2007-43706A and JP2009-212768A disclose communication techniques using visible light. In JP-A-2007-43706 and JP-A-2009-212768, in a video display device including a display, a projector, etc., communication is performed in which communication information by visible light is superimposed on a normal video display for communication. The technology is disclosed.

The present disclosure provides a display device capable of outputting a visible light communication signal without significantly deteriorating the image quality of a display image, and reducing a reception error of the output visible light communication signal, and a control method thereof.

A display device according to the present disclosure is a display device capable of outputting a visible light communication signal, and a display panel having a display surface for displaying a video, and displaying a video on the display surface of the display panel based on the video signal. A display control unit for controlling the display panel, a backlight having a light emitting surface for illuminating the display surface of the display panel from the backside, and a backlight control for generating the visible light communication signal based on the video signal. A signal processing unit to be superimposed on a signal, the light emitting surface of the backlight is divided into a plurality of regions, according to the backlight control signal output by the signal processing unit, to control the light emission in each of the plurality of regions, and And a backlight control unit that provides a period for controlling extinction at different timings in each of the plurality of regions, the signal processing unit, when superimposing the visible light communication signal on the backlight control signal, The visible light communication signal is not superimposed on a signal indicating that the backlight is turned off among the backlight control signals.

The display device according to the present disclosure can output a visible light communication signal without significantly deteriorating the quality of a display image, and can reduce reception errors of the output visible light communication signal.

[Discovery that is the basis of the present disclosure]
2. Description of the Related Art In recent years, in the field of display devices, particularly liquid crystal displays, projectors using liquid crystals, and the like, a technique called backlight scanning has been adopted to improve image quality. Here, the backlight scan is a backlight control method for improving the slow reaction speed of the liquid crystal and a moving image blur due to the hold type. The display screen is divided into several areas (backlight area), and the area is divided. The emission of the backlight is controlled so that the lights are sequentially turned on every time. More specifically, the backlight scan is a control method in which a backlight extinguishing period is provided and the timing of the extinguishing period (blanking period) periodically performed is different in each backlight region. Generally, the timing of the blanking period is often controlled so as to match the liquid crystal scanning timing.

However, as disclosed in Japanese Patent Laid-Open No. 2007-43706, visible light communication employs a method of superimposing a visible light communication signal by blinking a backlight. Therefore, the visible light communication signal cannot be transmitted during the backlight off time, and this stop period causes a signal transmission error.Therefore, it is necessary to stop the backlight scan and perform communication in a state where the image quality is deteriorated. There wasn't.

Therefore, the present disclosure provides a display device that can output a visible light communication signal without significantly deteriorating the image quality of a display image and reduce reception errors of the output visible light communication signal.

Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed description of well-known matters or duplicate description of substantially the same configuration may be omitted. This is to prevent the following description from being unnecessarily redundant and to facilitate understanding by those skilled in the art.

It should be noted that the applicant provides the accompanying drawings and the following description for those skilled in the art to fully understand the present disclosure, and is not intended to limit the subject matter described in the claims by these. ..

The eighteenth embodiment will be described below with reference to FIGS. 339 to 346.

[1. Constitution]
FIG. 339 is a schematic diagram showing an example of a visible light communication system according to the eighteenth embodiment.

[1.1. Configuration of visible light communication system]
The visible light communication system 1300 illustrated in FIG. 339 includes a display device 1400 and a smartphone 1350.

The display device 1400 is, for example, a television and can display an image on the display surface 1410. The display device 1400 can also superimpose a visible light communication signal on the display surface 1410.

The smartphone 1350 is an example of an electronic device that receives a visible light communication signal, and can receive the visible light communication signal transmitted from the display device 1400. From this, the user of the smartphone 1350 can receive the information related to the video displayed on the display device 1400.

In the present embodiment, the display device 1400 will be described by exemplifying a monitor such as a television or a display that displays video, but the display device 1400 is not limited to this. The display device 1400 may be a device that projects an image, such as a projector. Although the smartphone 1350 is given as an example of the electronic device that receives the visible light communication signal output by the display device 1400, the electronic device is not limited to the smartphone as long as the electronic device can receive the visible light communication signal. For example, the electronic device may be a receiving device compliant with JEITA-CP1222. Further, this electronic device is not limited to a smartphone and may be a general mobile terminal. The electronic device may receive the visible light communication signal and obtain the information by decoding the received visible light communication signal.

Further, the information transmission method for transmitting the visible light communication signal may be based on the standards such as JEITA-CP-1223 which is currently being developed to the international standard, or the already-created IEEE-P802.15. In other words, the electronic device may be configured using a receiving device compatible with these standards.

[1.2. Display device configuration]
FIG. 340 is a block diagram showing an example of a schematic configuration of a display device according to the eighteenth embodiment.

A display device 1400 illustrated in FIG. 340 is a display device that can output a visible light communication signal, and includes a first input portion 1420, a first processing portion 1430, a first control portion 1440, and a display panel 1450. A second input unit 1460, a second processing unit 1470, a second control unit 1480, and a backlight 1490.

A video signal regarding a video displayed on the display panel 1450 is input to the first input unit 1420. The video signal is transmitted from, for example, a broadcast wave, a video recorder/player, a PC, or the like through the first input unit 1420 through, for example, an antenna cable, a video signal line, a composite cable, an HDMI (registered trademark) cable, a PJLink cable, or a LAN cable. Entered in. Here, the video signal may be stored in various recording media by using a video recorder or a player.

The video signal is input to the first processing unit 1430 from the first input unit 1420. The first processing unit 1430 performs general image processing such as image quality processing on the video signal. The first processing unit 1430 transmits the image-processed video signal to the first control unit 1440. In addition, the first processing unit 1430 transmits information indicating the size, display timing, brightness, etc. of the subframe or the video signal to the first control unit 1440 and the second processing unit 1470.

Note that the first processing unit 1430 outputs the duty ratio calculated based on the video signal and the backlight control signal (hereinafter, referred to as BL control signal) for each area to the second processing unit. Good.

The display panel 1450 is, for example, a liquid crystal panel and has a display surface 1410 for displaying an image.

The first control unit 1440 is an example of a display control unit, and controls the display panel 1450 to display an image on the display surface 1410 of the display panel 1450 based on the image signal. In this embodiment, the first control unit 1440 controls to display an image on the display panel 1450 based on the image signal transmitted from the first processing unit 1430. More specifically, the first control unit 1440 controls the liquid crystal aperture of the display panel 1450 based on the video signal transmitted from the first processing unit 1430.

The second input unit 1460 receives a signal used for visible light communication (hereinafter referred to as a visible light communication signal), and transmits the input visible light communication signal to the second processing unit 1470. In the present embodiment, the visible light communication signal generated by, for example, a PC is input to the second input unit 1460 via a dedicated cable or LAN cable.

Note that the visible light communication signal may be superimposed on part of broadcast radio waves and input to the second input unit 1460 through an antenna cable. In addition, the visible light communication signals stored in various recording media by the video recorder or the replay may be input to the second input unit 1460. For example, a video recorder that records a visible light communication signal may be placed on a part of a line such as an HDMI (registered trademark) cable or a PJLink cable, and may be input to the second input unit 1460. In addition, a visible light communication signal separately created by a PC or the like may be superimposed on the video signal and input to the second input unit 1460 from the video recorder or the replay device.

Note that the second input unit 1460 uses information stored in the display device, such as the ID of the display device, in addition to receiving from the outside, and reads the server information through the Internet or the like to generate a visible light communication signal. You may get it.

The second processing unit 1470 encodes the visible light communication signal input from the second input unit 1460, generates an encoded signal, and calculates a duty based on the video signal and/or the visible light communication signal. Also, the second processing unit 1470 receives the B.D. signal input from the first processing unit 1430. The encoded signal is superimposed on the L control signal.

In the present embodiment, the coded signal will be described as a signal in which the ON period and the OFF period are mixed at a certain ratio. Further, it is assumed that the coded signal is coded by the I-4PPM method in principle. The encoded signal may be encoded by the Manchester method or the like. Further, the modulated signal is described as ON/OFF with a modulation rate of 100%, but the present invention is not limited to this. When the modulation rate is not 100% and High/Low modulation is used, ON/OFF in the following description may be read as High/Low or the like. The duty of the visible light communication signal is not only handled as a value that the ON period is standardized over the entire signal transmission period, but also corresponds to (High level×High period+Low level×Low period)/(Signal transmission period×High level) You may read and implement.

More specifically, the second processing unit 1470 is an example of a signal processing unit, and performs a process of superimposing a visible light communication signal on a backlight control signal generated based on a video signal. However, when superimposing the visible light communication signal on the backlight control signal, the second processing unit 1470 does not superimpose the visible light communication signal on the signal indicating the extinction of the backlight in the backlight control signal. .. In addition, the coded visible light communication signal (coded signal) may also be described as a visible light communication signal.

The second control unit 1480 is an example of a backlight control unit, divides the light emitting surface of the backlight 1490 into a plurality of areas, and outputs a backlight control signal (BL control) output by the second processing unit 1470. Signal), light emission is controlled in each of the plurality of regions, and control is performed to provide a period in which the light is extinguished at different timings in each of the plurality of regions. In this embodiment, the second control unit 1480 controls the brightness and timing of the backlight 1490 based on the backlight control signal (BL control signal) transmitted from the second processing unit 1470.

The backlight 1490 irradiates the display panel 1450 with light from the back side. More specifically, the backlight 1490 has a light emitting surface that illuminates the display surface 1410 of the display panel 1450 from the back surface. This allows the viewer to visually recognize the video displayed on the display panel 1450.

In this embodiment, the light emitting surface of the backlight 1490 is divided into a plurality of areas, and backlight scanning can be realized by sequentially performing light emission control for each area. Note that the plurality of regions on the light emitting surface of the backlight 1490 correspond to the plurality of regions on the display surface 1410.

[2. Operation of display device]
Next, the operation of the display device 1400 configured as above will be described.

In the display device 1400, a backlight scan is performed by sequentially scanning the entire screen of the display panel 1450 with a backlight and sequentially turning off the lights in accordance with writing of a video signal.

Generally, in a display panel including liquid crystal, the phase change speed of the liquid crystal is slow, and it takes time until the image is switched even if the video signal is switched to display different gradations. Therefore, the backlight scan is performed in order to improve moving image characteristics such as blurring caused by displaying the image being switched by temporarily turning off the backlight while scanning the backlight of the display panel. On the other hand, the scanning speed for switching has been increasing year by year, and it has become possible to increase from a normal scanning speed of 60 frames per second to a high scanning speed of 2 times or 4 times the 60 frames per second. When scanning at a high speed, a smooth moving image characteristic can be obtained by performing frame interpolation between normal frames and switching images little by little.

Therefore, backlight scanning, which turns off the light while scanning the backlight, is very important for improving the moving image characteristics, and it is better for the visible light communication signal not to output the signal when the backlight scanning turns off. Good.

For the above reason, the display device 1400 does not output the visible light communication signal during the extinguishing period by backlight scanning (hereinafter referred to as the blanking period).

In the following, even when the display device 1400 does not output the visible light communication signal during the blanking period of the backlight control signal (BL control signal), for example, the receiver side such as the smartphone 1350 has a high probability of transmitting the visible light communication signal. The method (operation) that can be received by will be described.

(Example 1)
[2.1.1. Example of operation of second processing unit]
FIG. 341A is a diagram illustrating the B.I. FIG. 341B is a diagram illustrating an example of a state before a visible light communication signal is superimposed on the L control signal, and FIG. 341B illustrates the B.L. It is a figure which shows an example of the state which superimposed the visible light communication signal on the L control signal.

In FIG. 341A and FIG. 341B, the backlight 1490 is controlled for each of the eight areas A to H, which are the display areas of the divided display surface 1410, so that the corresponding B.I. An example in which the L control signals A to H are input is shown. The hatched portion indicates the area where the encoded signal (visible light communication signal) is present.

The encoded signal shown in FIG. When the L control signals A to H are superposed with different phases and coded signals of different phases are mixed in the range received by the receiver side, an error (visible light) occurs when the receiver side decodes. Communication signal reception error) occurs.

Therefore, in the present embodiment, as shown in FIG. 341B, the phases of the coded signals (visible light communication signals) are superimposed on each other in a certain area of the display area.

Here, the expression of matching the phases is described by taking an example of synchronizing the rising timings, but is not limited thereto. Any time may be used as long as it is determined which time from the state before the start of rising to the state of the completion of rising is the rising time. In addition, since a delay time or the like occurs in the control signal voltage path, it does not mean that synchronization is achieved only when the timings match, and even when there is a fixed delay time or a fixed delay time, the phase It is included in the expression of matching. The same applies to the following embodiments (Embodiments 18 to 23).

Here, since the backlight is sequentially extinguished for each area by sequential scanning, it is difficult to superimpose the encoded signal without including the extinguishing period (blanking period) at all. Therefore, in the present embodiment, the coded signal is superimposed in a specific area (hereinafter referred to as a reference area) of the divided display areas immediately after the turn-off period (blanking period). The coded signal is superimposed on the coded signal of the reference area in the same phase as the coded signal of the reference area even in a region other than the specific region (reference region), but during the extinguishing period (blanking period) when the backlight is extinguished. The coded signal is not superposed.

In the example illustrated in FIG. 341B, the second processing unit 1470 uses the B.I. The region C to which the L control signal C is input is set as the reference region, and the B.L. The superposition timing of the coded signal is adjusted so that the head (rise timing) P2 of the coded signal comes to the rise timing P1 of the L control signal C, and then the coded signal is set to B.A. The L control signals A to H are superimposed in phase. Then, the second processing unit 1470 causes the B. When the coded signal is superimposed on the L control signals A to H, B. The code signal is superimposed during the ON period of the L control signal, but is not superimposed during the OFF period.

The reference area is not limited to the area C described above. Hereinafter, an example of the reference area that can be set in this embodiment will be described. For example, the reference area may be set to the brightest area (area having a short blanking period or area having the highest transmittance of the display panel) among the divided display areas.

Even if the brightest area is set as the reference area, further measures are required when changing the position of the reference area for each frame. This is because the position of the encoded signal that is superimposed on each frame changes, and the image center of gravity changes extremely for each frame, which may cause flicker. Alternatively, the receiving error occurs on the receiving side unless measures are taken such that the periods of the coded signals to be superimposed overlap each other between regions and one of the overlapping coded signals is cut off midway or not overlapped in the first predetermined period. This is because it can occur. Therefore, when the position of the reference area is changed for each frame, a period of at least one frame, and a period in which the encoded signal is not superimposed may be provided.

Further, when setting the bright region as the reference region, the first processing unit 1430 does not determine the bright region based on the brightness of the display region in each frame, but the brightness of the image based on the video signal. The bright area may be determined based on the transition of the center of the sag.

When the brightness of the entire display area does not change more than a certain amount, such as when scenes do not switch for a certain amount of time, the area that includes the brightest place in the display area is the reference area based on the average of the video signals for a certain amount of time. It may be set as. The reference area may be determined in advance.

[2.1.2. Effect, etc.]
As described above, in the present embodiment, the display device is a display device (1400) capable of outputting a visible light communication signal, and is based on the display panel (1450) having a display surface for displaying an image and the image signal. A display control unit (first control unit 1440) that controls the display panel so that an image is displayed on the display surface of the display panel; Of the backlight (1490), a signal processing unit (second processing unit 1470) for superimposing the visible light communication signal on a backlight control signal generated based on the video signal, and the backlight (1490). The light emitting surface is divided into a plurality of areas, light emission is controlled in each of the plurality of areas in accordance with a backlight control signal output from the signal processing unit (second processing unit 1470), and the plurality of areas is also controlled. And a backlight control unit (second control unit 1480) that provides a period for performing turn-off control at different timings, and the signal processing unit (second processing unit 1470) outputs the visible light communication signal to the visible light communication signal. When superposed on the backlight control signal, the coded signal is not superposed on the signal indicating the extinction of the backlight (1490) in the backlight control signal.

With this configuration, it is possible to provide a display device that can output a visible light communication signal without significantly deteriorating the image quality of a display image and reduce reception errors of the output visible light communication signal.

Further, the signal processing unit (second processing unit 1470) superimposes the visible light communication signal on the backlight control signal of each of the plurality of regions and superimposes on the plurality of regions. The visible light communication signals may be in phase with each other.

Thereby, the reception error of the visible light communication signal can be suppressed.

Here, for example, the signal processing unit may match the phases of the visible light communication signals superimposed on each of the plurality of areas with reference to the backlight control signal of a predetermined area of the plurality of areas. Good.

This makes it possible to minimize the period of the visible light communication signal that is not superimposed on the blanking period.

Further, the predetermined region may be the brightest region of the plurality of regions, or the predetermined region may be a region corresponding to an end of the display surface of the plurality of regions. Good.

As a result, it is possible to suppress the influence of the decrease in luminance due to the turning off of the backlight due to the visible light communication signal.

(Example 2)
Hereinafter, it is assumed that the time length of the blanking period is the same in each of the display areas.

The total time during which the backlight 1490 is extinguished (total extinction period) is B. L control signal O
The blanking period, which is the FF period, and the OFF period of the encoded signal are added together.

Therefore, even if the coded signal is superimposed immediately after the end of the blanking period in the reference region and the coded signal is completely included from this blanking period to the next blanking period, The OFF period of the backlight 1490 increases by the OFF period of the encoded signal superimposed on the L control signal. That is, in the reference area, when the coded signal is superimposed, it becomes darker than before the coded signal is superimposed.

On the other hand, in regions other than the reference region, for example, since the coded signal is not superimposed in the blanking period, only the OFF period in the coded signal period that overlaps the blanking period and in which the coded signal is not superimposed is greater than the back region from the reference region. The time when the light 1490 is turned off becomes shorter. That is, for example, in an area other than the reference area, when the coded signal is superimposed, the area may be brighter than the reference area.

In order to improve this, two methods of providing an adjustment period for turning on or off the backlight 1490 can be considered. That is, in the first method, since the total extinguishing period of the reference region is the longest, there is a method of adjusting the total extinguishing period of the other region to the total extinguishing period of the reference region. The second method is to combine the total extinguishing periods of all areas with the total extinguishing periods determined based on the original video signal.

[2.2.1. Example of operation of second processing unit according to first method]
First, the operation of the second processing unit 1470 according to the first method will be described with reference to FIGS. 342 and 343.

342 and 343 are timing charts for explaining the first method in the second example of the eighteenth embodiment. In FIG. 342, (a), B.I. of the reference area before the coded signal is superimposed. L control signal is shown, and in FIG. 342 (b), the B.L. of the reference area after the coded signal is superimposed. The L control signal is shown. In FIG. 343 (a), B.I. of another area before the coded signal is superposed. L control signal is shown, and in FIG. 343 (b), B.B. of another area after the coded signal is superimposed. The L control signal is shown.

More specifically, in FIG. 342, the second processing unit 1470 indicates that the B. At the rising timing (time t12) of the L control signal, the coded signal is adjusted to the beginning (rising timing) of the coded signal, and the coded signal is set to B.B. An example in which it is superimposed on the L control signal is shown. In FIG. 343, the second processing unit 1470 transmits the coded signal having the same phase as the coded signal superimposed on the reference region to the B.D. An example in which it is superimposed on the L control signal is shown.

That is, in FIGS. 342 and 343, the second processing unit 1470 indicates that the B.D. An example is shown in which a coded signal is superimposed on the L control signal at the same time as the blanking period of the reference region and at the same phase as the other regions. The blanking period of each area is prioritized as in the first embodiment, and the coded signal is not superimposed in the blanking period.

As shown in (b) of FIG. 342, in the reference region, for example, in addition to the blanking period B1 shown as the period of time t11 to time t12, for example, the coded signal period C1 shown as the period of time t12 to time t14. There is a coded signal extinguishing period T1 that is the total of the OFF periods of the coded signal in the inside.

Therefore, in the reference region shown in (b) of FIG. 342, for example, the total of the OFF periods of the coded signal (coded signal extinguishing period) in one frame, which is shown as the period from time t11 to time t13, is When the duty is used, it can be expressed as the coded signal extinction period T1=the coded signal period C1×(1−Duty).

As shown in (b) of FIG. 342, in the reference region, basically, there is no period in which the coded signal period C1 and the blanking period B1 overlap each other, and thus the total extinguishing period T2 of one frame=the blanking period B1+ The encoded signal extinguishing period T1 starts. That is, the total extinguishing period of the reference region is the longest as compared with the other regions.

On the other hand, in areas other than the reference area, the coded signal period and the blanking period may overlap. As described above, the blanking period is B. Since the L control signal has priority over the coded signal, the coded signal is not superimposed.

Therefore, as illustrated in (b) of FIG. 343, in a region other than the reference region, for example, as a period from time t22 to time t23 in the encoded signal period C1 shown as a period from time t21 to time t24. The total OFF period is shorter than the reference region by the OFF period of the encoded signal in the encoded signal period C1 that overlaps with the blanking period B2 shown.

Here, the total of the OFF periods of the encoded signal in the encoded signal period C1 (encoded signal extinguishing period) is (encoded signal extinguishing period)=when the overlapping period of the encoded signal and the blanking period is B2. (Encoded signal period C1−Blanking period B2)×(1−Duty)

By the way, as described above, when the total extinguishing period is different for each area of the screen (display area), the brightness varies depending on the area, so that the image quality is deteriorated.

Therefore, the second processing unit 1470 operates according to the first method in which the adjustment period for turning on or off the backlight 1490 is provided, so that the total extinguishing period of each area is matched within the screen.

More specifically, the second processing unit 1470 according to the first method of aligning the total extinguishing period of the other region with the total extinguishing period of the reference region, total extinguishment of one frame of the reference region in the other region. An adjustment period will be provided to adjust the difference from the period. As described above, in this embodiment, the time length of the blanking period of each area is the same as a premise.

Here, in FIG. 343(b), the adjustment period A1 shown as the period from time t24 to time t26 can be expressed as a blanking period B2×(1-Duty). That is, the adjustment period in each region other than the reference region can be calculated from the blanking period of each region including the reference region, the encoded signal period, and the phase of the encoded signal. In (b) of FIG. 343, an example is shown in which the adjustment period is arranged in one frame shown as a period from time t21 to time t25.

As described above, the display device 1400 according to the present embodiment causes the second processing unit 1470 to provide the adjustment period by the first method. As a result, the display device 1400 transmits the encoded signal to the B. Although the brightness of the entire screen (display area) is reduced by a certain amount by superimposing it on the L control signal, the encoded signal can be output without significantly changing the image quality itself.

When the adjustment period is provided immediately after the encoded signal period, the second processing unit 1470 is preferable because it can be stably arranged as close as possible to the blanking period where the phase change of the liquid crystal of the display panel 1450 is large. It is not limited to this case. The second processing unit 1470 may provide the adjustment period by the time when the next encoded signal is superimposed.

[2.2.2. Example of Operation of Second Processing Unit by Second Method]
Next, the operation of the second processing unit 1470 according to the second method will be described.

The adjustment period in which the backlight 1490 is turned on or off to adjust the total turn-off period can be generally defined as follows, for example. That is, the original extinguishing period of the backlight 1490 (blanking period and black video period) based on the video signal is T4, and the encoded signal in the encoded signal period that does not overlap the blanking period in the encoded signal period is When the total of the OFF periods is T5 and the blanking period after superimposing the visible light communication signal is T6, the adjustment period can be expressed as T4-T5-T6. As mentioned above, it is desirable to set the adjustment period as close as possible to the blanking period.

For example, in the reference area, T5 first calculates the total OFF period of the encoded signal in the encoded signal period, and then subtracts the total OFF period of the encoded signal period that overlaps the blanking period. This can be calculated.

Hereinafter, a specific example of the operation of the second processing unit 1470 according to the second method will be described with reference to FIGS. 344A to 345D.

344A to 345D are timing charts for explaining the second method in the second example of the eighteenth embodiment.

First, with reference to FIGS. 344A to 344D, the operation in which the second processing unit 1470 provides the adjustment period by the second method when the coded signal period and the blanking period do not overlap will be described.

In FIGS. 344A to 344D, B. B before the encoded signal is superposed on (a) in the upper stage. The L.L control signal is shown, and in the lower part (b) to (e), the B.L. L control signal and B.T. adjusted by the second method. L control signal is shown. In the figure, the blanking period is shown as B1 and the encoded signal period is shown as C1.

B. where the encoded signal is superimposed by the second method. The L control signal is adjusted in four cases shown in FIGS. 344A to 344D depending on the sum (time sum) of the adjustment period, the encoded signal period, and the blanking period and the positive/negative relation of the adjustment period. Be divided. Hereinafter, each case will be described.

[Adjustment Method when Coded Signal Period and Blanking Period Do Not Overlap (Case 1)]
FIG. 344A shows an example in which the adjustment period is 0 or more and (adjustment period+coded signal period+blanking period) is less than or equal to the length of one frame.

As shown in the upper part of (b) of FIG. 344A, a part of the adjustment period is arranged in a period from the end time P2 of the blanking period B1 to the start time P3 of the encoded signal period C1 and the rest of the adjustment period. Are arranged after the coded signal period, preferably immediately after (time P5).

By providing the adjustment period shown in the upper part of (b) of FIG. 344A, the second processing unit 1470 provides the B.D. with the encoded signal superimposed as shown in the lower part of (b) of FIG. 344A. Adjust the L control signal.

In this way, the second control unit 1480 controls the adjusted B.I. According to the L control signal, the backlight 1490 is turned off after the blanking period B1 and before the start of the coded signal period C1. Further, after the end of the coded signal period C1 and the coded signal period C1, from the adjustment period to P2-P3. The light is turned off until the period obtained by subtracting the period between.

When the adjustment period is shorter than the period between P2 and P3, the adjustment period may be provided only between P2 and P3. When P2=P3, the entire adjustment period may be provided after the encoded signal period C ends.

[Adjustment Method when Coded Signal Period and Blanking Period Do Not Overlap (Case 2)]
FIG. 344B shows an example in which the adjustment period is 0 or more and (adjustment period+coded signal period+blanking period) is longer than the length of one frame.

As shown in the upper part of (c) of FIG. 344B, a part of the adjustment period is arranged in a period up to time P3 when the encoded signal period C1 starts with the end time P2 of the blanking period B1 as a starting point, and the rest of the adjustment period. Are arranged starting from the end time P4 of one frame.

By providing the adjustment period shown in the upper part of (c) of FIG. 344B, the second processing unit 1470, as shown in the lower part of (c) of FIG. Adjust the L control signal.

In this way, the second control unit 1480 controls the adjusted B.I. According to the L control signal, after the blanking period B1, the backlight 1490 is turned off until the start time P3 of the encoded signal period C1 and the period from time P5 to time P4 before the end of the encoded signal period C1 is turned off. That is, from the time P5 at which the rest of the adjustment period overlaps the coded signal period C1 to the end time P10 of the coded signal period C1, the coded signal B. It is not superimposed on the L control signal (or the signal is turned off).

When P2=P3 (at the same time), the entire adjustment period is provided after the encoded signal period.

[Adjustment Method when Coded Signal Period and Blanking Period Do Not Overlap (Case 3)]
FIG. 344C shows an example in which the adjustment period is smaller than 0 and (adjustment period+coded signal period+blanking period) is equal to or less than the length of one frame. Here, the adjustment period smaller than 0 means an adjustment period in which the backlight 1490 is turned on.

As shown in the upper part of (d) of FIG. 344C, the adjustment period starts from the end time P2 of the blanking period B1 and goes back by a time corresponding to the absolute value of the adjustment period (the period from time P6 to time P2). ).

By providing the adjustment period shown in the upper part of (d) of FIG. 344C, the second processing unit 1470, as shown in the lower part of (d) of FIG. Adjust the L control signal.

In this way, the second control unit 1480 controls the adjusted B.I. In accordance with the L control signal, the backlight 1490 is turned on during the blanking period B1 from time P6 to time P2.

When P2=P3, the entire adjustment period may be arranged after the encoded signal period C1. When the adjustment period is longer than the blanking period, the duty ratio of the coded signal is considered, and the coded signal is not superposed on the coding period C1 until the lighting time for the shortage can be secured. It may be set to the off period.

[Adjustment Method when Coded Signal Period and Blanking Period Do Not Overlap (Case 4)]
FIG. 344D shows an example in which the adjustment period is smaller than 0 and (adjustment period+coded signal period+blanking period) is longer than the length of one frame.

As shown in the upper part of (e) of FIG. 344D, the adjustment period starts from time P2 when the blanking period B1 ends and goes back by a time corresponding to the absolute value of the adjustment period (from time P7 to time P2). Period). As a result, the backlight 1490 for the period from the time P7 to the time P2 in the blanking period B1 is turned on.

Although the blanking period and the encoded signal period do not overlap with each other and the adjustment period is negative, the absolute value of the adjustment period may be longer than the blanking period. .. In this case, if the entire adjustment period is arranged with the time P2 at which the blanking period B1 ends as the starting point, the time P7 becomes the time P1 or a time earlier than the time P1 and the blanking period does not exist. When it is necessary to not only light all during the blanking period but also light the backlight 1490 (brighten the area), the encoded signal is set as the remaining period of the adjustment period excluding the blanking period. The OFF period of the encoded signal of the period may be turned on. That is, the remaining period of the adjustment period may be arranged by the period (time P8) traced back from time P9, and the superposition of the encoded signal may be stopped and the lighting may be continued.

Here, it is necessary to determine the time P8 because the original blanking period B1 is equal to the total of the OFF periods during the period between the encoded signal period C1 and the period between the times P8 and P9. Specifically, the time P8 can be calculated based on the relationship of the blanking period B1=(coded signal period C1−(time P9−time P8))×(1−Duty).

As a result, the second processing unit 1470 causes the second control unit 1480 to continue lighting the backlight 1490 in addition to the blanking period B1 from time P8 until the start of the next blanking period. ． The L control signal can be adjusted.

When P2=P3, the entire adjustment period may be placed after the encoded signal period C1.

Next, with reference to FIGS. 345A to 344D, an operation in which the second processing unit 1470 provides the adjustment period by the second method when the coded signal period and the blanking period overlap will be described.

In FIGS. 345A to 345D, B.B before the encoded signal is superimposed on (a) in the upper stage. L control signal is shown, and in the lower part (b) to (e), the B.L. L control signal and B.T. adjusted by the second method. L control signal is shown. In the figure, the blanking period is shown as B1, the encoded signal period is shown as C1, and the time from Q1 to Q6 is shown as one frame period.

B. where the encoded signal is superimposed by the second method. The method of adjusting the L control signal can be divided into four methods shown in FIGS. 345A to 345D depending on the sum of the adjustment period, the encoded signal period, the blanking period, and the positive/negative relationship of the adjustment period. Hereinafter, each case will be described.

[Adjustment Method When Coded Signal Period and Blanking Period Overlap (Case 1)]
FIG. 345A shows an example in which the adjustment period is 0 or more, and (adjustment period+coded signal period+blanking period) is less than or equal to the length of one frame.

As shown in the upper part of (b) of FIG. 7A, the adjustment period is arranged with the time Q4 at which the encoded signal period C1 ends as a starting point.

By providing the adjustment period shown in the upper part of (b) of FIG. 344A, the second processing unit 1470, as shown in the lower part of (b) of FIG. 345A, the period from time Q4 to time Q5 which is the adjustment period. , And the period from time Q2 to time Q3, which overlaps with the blanking period B1, so that the coded signal is not overlapped. Adjust the L control signal.

In this way, the second control unit 1480 controls the adjusted B.I. In accordance with the L control signal, the backlight 1490 is turned off during the period from time Q4 to time Q5 together with the period from time Q2 to time Q3 overlapping with the blanking period B1. During the period from time Q4 to time Q5, the backlight 1490 is turned off and the encoded signal is not transmitted.

[Adjustment Method When Coded Signal Period and Blanking Period Overlap (Case 2)]
FIG. 345B shows an example in which the adjustment period is 0 or more and (adjustment period+coded signal period+blanking period) is one frame length or more.

As shown in the upper part of (c) of FIG. 345B, the adjustment period is arranged in the period from the time Q8 to the time Q6, which is the adjustment period, starting from the time Q6 when the encoded signal of the next frame starts. ..

By providing the adjustment period shown in the upper part of (c) of FIG. 345B, the second processing unit 1470, as shown in the lower part of (c) of FIG. 345B, the period from time Q8 to time Q6 which is the adjustment period. , And the period from time Q2 to time Q3, which overlaps with the blanking period B1, so that the coded signal is not overlapped. Adjust the L control signal.

In this way, the second control unit 1480 controls the adjusted B.I. According to the L control signal, the backlight 1490 is turned off during the period from time Q8 to time Q6 together with the period from time Q2 to time Q3 overlapping with the blanking period B1. During the period from time Q8 to time Q6, the backlight 1490 is turned off and the encoded signal is not transmitted.

[Adjustment Method When Coded Signal Period and Blanking Period Overlap (Case 3)]
FIG. 345C shows an example in which the adjustment period is smaller than 0 and (adjustment period+coded signal period+blanking period) is equal to or longer than the length of one frame.

As shown in the upper part of (d) of FIG. 345C, the adjustment period is set as a starting point at the end time Q3 of the blanking period B1 and is arranged in a period traced back by a time corresponding to the absolute value of the adjustment period.

By providing the adjustment period shown in the upper part of (d) of FIG. 345C, the second processing unit 1470 provides the adjustment period from time Q9 to time Q3 as shown in the lower part of (d) of FIG. 345C. To turn on the backlight 1490 in B. The B.L. control signal is adjusted and the encoded signal is adjusted so that the encoded signal is not superimposed in the ranking period B1. Adjust the L control signal.

In this way, the second control unit 1480 controls the adjusted B.I. In accordance with the L control signal, the backlight 1490 is turned on during the period from time Q9 to time Q3.

The coded signal may be superimposed during the adjustment period. In this case, the adjustment period may be lengthened by the total of the OFF periods of the encoded signal. Furthermore, when the adjustment period is longer than the blanking period, the shortage of the lighting time in the adjustment period is encoded in a predetermined period that is traced back from the end time of the encoding period C1 based on the duty ratio of the encoded signal. The backlight 1490 may be turned on by not superimposing signals.

[Adjustment Method When Coded Signal Period and Blanking Period Overlap (Case 4)]
FIG. 345D shows an example in which the adjustment period is smaller than 0 and (adjustment period+coded signal period+blanking period) is longer than the length of one frame.

As shown in the upper part of (e) of FIG. 345D, the adjustment period is arranged from the end time Q3 of the blanking B1 as a starting point to a period of time Q10 traced back by a time corresponding to the absolute value of the adjustment period.

As a result, the backlight 1490 during the period from time Q10 to time Q3, which overlaps the blanking period B1, is turned on.

The adjustment period may be lengthened by the total of the OFF periods of the coded signal, and the coded signal may be superimposed in the adjustment period.

Further, as in the case of (e) of FIG. 344D, when the adjustment period is very long and the absolute value thereof is larger than the blanking period B1, the rest of the adjustment period excluding the blanking period B1 is left. As the period, the OFF period of the encoded signal in the encoded signal period may be turned on.

Here, the time Q11 needs to be determined because the original blanking period B1 becomes equal to the total of the OFF periods during the time period between the coded signal period C1 and the time period Q11-Q12. Specifically, the time Q11 can be calculated based on the relationship of the blanking period B1=(coded signal period C1−(time Q12−time Q11))×(1−Duty).

As a result, the second processing unit 1470 controls the second control unit 1480 so that, in addition to the blanking period B1, the second processing unit 1470 continues lighting from the time Q11 to the start time Q7 of the next blanking period. The L control signal can be adjusted.

[2.2.3. Effect, etc.]
As described above, according to the present embodiment, the backlight control method for improving the moving image characteristics such as the backlight scan and the transmission of the visible light communication signal using the backlight are turned off by the encoded signal of the visible light communication. It is possible to achieve both by equalizing the period or adjusting the period to be equal to the original video signal.

Here, for example, in the display device of the present embodiment, when the signal processing unit (second processing unit 1470) superimposes the visible light communication signal on the backlight control signal, Of the plurality of regions, the period of the signal indicating that the backlight is turned off and the period of the visible light communication signal to be overlapped are overlapped, and when there is a region of the plurality of regions, A lighting adjustment period for adjusting the brightness of the overlapping region may be provided, and ON/OFF of the backlight control signal may be adjusted during the lighting adjustment period.

As a result, in the region where the visible light communication signal period and the backlight extinguishing period overlap, the adjustment period is provided, so that the B.I. When the visible light communication signal (coded signal) is superimposed on the L control signal, the difference in luminance in the display area can be made inconspicuous.

Although the reference region is described as a bright region in this embodiment, it may be read as a region in which the opening of the display panel 1450 is set large.

(Example 3)
[2.3.1. Example of Operation of Second Processing Unit by Second Method]
In the second embodiment, the brightness of the display surface 1410 (display area) of the display panel 1450 is made uniform by providing the adjustment period in which the backlight 1490 is turned on or off, but the present invention is not limited to this.

In this embodiment, a method without providing an adjustment period will be described with reference to FIG. 346.

FIG. 346 is a diagram illustrating the B.V. 6 is a timing chart for explaining a method of superimposing a visible light communication signal on an L control signal. Here, in (a) of FIG. 346, B.I. The L control signal is shown. In the present embodiment, it is assumed that the signal is detected only by the rising signal of the waveform.

As shown in FIG. 346, even if the adjustment period is not provided, the luminance of the area is adjusted by changing the duty ratio of the visible light communication signal, that is, the High period of the signal, by an amount corresponding to the adjustment period. Good.

Specifically, for example, when the adjustment period in the second embodiment is a positive number, that is, when the adjustment for turning off the backlight 1490 is performed, as shown in (b) of FIG. The High period of the L control signal may be shortened.

In addition, for example, when the adjustment period in the second embodiment is a negative number, that is, when the adjustment for turning on the backlight 1490 is performed, as in (c) of FIG. The High period of the L control signal may be lengthened.

It should be noted that B.I. It is possible that the duty ratio of the L control signal changes. In this case, B.I. In order to drive the L control signal with a constant duty ratio in the plane, the adjustment period in the second embodiment calculated again by including the change in the duty ratio, and the High period of the visible light communication signal as in the present embodiment A method of changing the period may be used in combination.

Furthermore, in the above, control of the backlight 1490 during the High period (PWM control: Pul)
Although the in-plane uniformity and the prevention of the deterioration of the image quality have been described by controlling the luminance using the se width modulation), the invention is not limited to this. The second control unit 180 that controls the backlight may control the current supplied to the backlight 1490 in each area to bring the brightness in the visible light communication area closer to the brightness in areas other than the visible light communication area. Furthermore, the brightness in the visible light communication area may be made closer to the brightness in the area other than the visible light communication area by a combination of PWM control of the backlight 1490 and current control.

[2.3.2. Effect, etc.]
As described above, according to the present embodiment, the backlight control method for improving the moving image characteristics such as the backlight scan and the transmission of the visible light communication signal using the backlight are turned off by the encoded signal of the visible light communication. It is possible to achieve both by equalizing the period or adjusting the period to be equal to the original video signal.

In the present embodiment, the signal detection is performed by using only the rising signal, but the present invention is not limited to this. Hold the position of the falling edge and change the rising edge B. In the case of the L control signal, the signal may be detected by the falling signal. In the present embodiment, B. Although the coded signal is superimposed on the basis of the rising edge of the L control signal, the B.L. The characteristic timing of the L control signal may be used as a reference, or the synchronization signal of the video signal itself may be used as a reference. Alternatively, a signal delayed for a certain time from the video synchronization signal may be created and used as a reference.

[3. effect]
As described above, according to the present embodiment, it is possible to realize a display device that can output a visible light communication signal without significantly deteriorating the image quality of a display image and reduce reception errors of the output visible light communication signal. can do.

(Embodiment 19)
In the eighteenth embodiment, B.I. The operation of the display device 1400 when the encoded signal period is shorter than the ON period of the L control signal has been described. In the present embodiment, B. The operation of the display device 1400 when the encoded signal period is longer than the ON period of the L control signal will be described.

[1. Operation of display device]
Hereinafter, the operation of the second processing unit 1470 will be mainly described.

FIG. 347 is a flowchart for explaining the operation of the second processing unit in the nineteenth embodiment.

First, in step S1301, the second processing unit 1470 re-encodes the visible light communication signal. More specifically, the second processing unit 1470 encodes the visible light communication signal and then generates (re-encodes) an encoded signal to which a header or the like is added. The second processing unit 1470 also calculates the transmission time of the encoded signal based on the carrier frequency of the encoded signal.

Next, in step S1302, second processing section 1470 determines that the encoded signal length is B.264. It is determined whether it is longer than the ON period (lighting time) of the L control signal.

More specifically, the second processing unit 1470 calculates the B.M.T. calculated by the first processing unit 1430. Based on the duty ratio of the L control signal, the time during which the backlight 1490 is turned on (lighting time) is compared with the transmission time length of the encoded signal (encoded signal length). When the second processing unit 1470 determines that the transmission time length of the encoded signal is shorter (No in S1302), the processing proceeds to step S1306 and determines that the transmission time length of the encoded signal is longer. (Yes in S1302), the process proceeds to step S1303.

Next, in step S1303, the second processing unit 1470 determines whether to perform visible light communication. If the second processing unit 1470 determines that visible light communication is to be performed (Yes in S1303), the processing proceeds to step S1304, and if it is determined that visible light communication is not to be performed (No in S1303), proceeds to step S1309.

Next, in step S1304, the second processing unit 1470 re-encodes the visible light communication signal. Specifically, the second processing unit 1470 is configured so that the duty ratio of the signal of the header portion is substantially OFF when the header portion is encoded by a signal array that cannot be used as normal data. Recreate (re-encode). After that, the second processing unit 1470 outputs the last signal (a signal indicating the ON state existing at the last end) of the header portion to the B.D. A process for accelerating the transmission start time of the encoded signal is performed so as to match the rising timing of the L control signal. The details will be described later.

Next, in step S1305, second processing section 1470 determines that the encoded signal length is B.264. It is determined whether it is longer than the ON period (lighting time) of the L control signal.

More specifically, the second processing unit 1470 uses the B.I. The lighting time of the backlight 1490 based on the duty ratio of the L control signal is compared with the transmission time length of the encoded signal. Then, when the second processing unit 1470 determines that the transmission time length of the encoded signal is shorter (No in S1305), step S1306 determines that the transmission time length of the encoded signal is longer. In that case (Yes in S1305), the process proceeds to step S1307.

Here, in step S1306, the second processing unit 1470 determines that the B.I. The encoded signal is superimposed on the portion other than the blanking period of the L control signal (that is, the ON period of the B.L control signal) and output to the second control unit 1480, and the process is ended.

On the other hand, in step S1307, second processing section 1470 determines whether or not to divide the encoded signal. More specifically, first, the second processing unit 1470 compares the transmission time length of the re-encoded coded signal with the lighting time of the backlight 1490. If the transmission time length of the encoded signal is longer, the second processing unit 1470 determines to divide the encoded signal (Yes in S1307), the process proceeds to step S1308, and the transmission time length of the encoded signal is increased. If is shorter, it is determined that the encoded signal is not divided (No in S1307), and the process proceeds to step S1309.

Next, in step S1308, the second processing unit 1470 divides the encoded signal so that the length is within the data length within the backlight lighting period. Then, the second processing unit 1470 adjusts the coded signal so as to be superimposed on the portion other than the blanking period of the backlight control signal (ON period of the BL control signal), and ends the process.

Note that in step S1309, second processing section 1470 does not transmit the encoded signal to second control section 1480. That is, the transmission of the visible light communication signal is stopped.

[2. Operation details]
Next, the details (specific example) of the operation of the display device 1400 of the nineteenth embodiment will be described with reference to FIGS. 348A to 348D and FIG. 349.

[2.1. Specific example 1]
348A to 348D are B. It is a figure for demonstrating the concrete method of superimposing a coded signal on an L control signal.

In the present embodiment, the second processing unit 1470 encodes the visible light communication signal using an encoding method such as 4PPM or I-4PPM. This is because, when encoded by 4PPM, I-4PPM, or the like, it is possible to relatively mitigate a large change in luminance depending on a signal, and to avoid instability in luminance. The visible light communication signal may be encoded by using an encoding method such as Manchester method.

For example, as shown in FIG. 348A, the encoded signal includes a header 1310 and a data portion 1311 that stores a codeword and the like. It is assumed that the header 1310 uses an impossible signal array as a data signal. Here, in the case of encoding using I-4PPM, the ratio of the High period in the signal period is determined to be 75% in principle. Further, it is common to put the ON state in the header in a form of continuing for 3 slots (minimum unit of encoded signal) or more. In many cases, the end of the header is turned off to separate the header.

In FIG. 348B, B. The case where the encoded signal period is shorter than the ON period of the L control signal is shown. That is, as shown in FIG. 348B, the entire encoded signal including the header is B.264. If it is shorter than the period excluding the blanking period within one frame of the L control signal (that is, the ON period of the B.L control signal), the B.L. The encoded signal can be superimposed without any problem during the ON period of the L control signal.

In contrast, B. When the coded signal period is longer than the ON period of the L control signal, the entire coded signal including the header is set to B.A. Since it cannot be included in the ON period of the L control signal, the encoded signal is divided and included as described in step S1307 above.

In FIG. 348C, the entire coded signal including the header is B.264. Since the length of one frame of the L control signal is exceeded, B. An example is shown in which a coded signal is divided and superimposed in the ON period of the L control signal. Specifically, the data part 1311 of the encoded signal is divided into a data part 1311-1 and a data part 1311-2, and the header 1310 and the header 92 are included in the B.B. It is superposed on the ON period of the L control signal. The header 92 includes a determination signal indicating that the data portion 1311-2 is the divided data portion 1311 and is data subsequent to the data portion 1311-1.

In addition, B. When the encoded signal period is longer than the ON period of the L control signal, as shown in FIG. The data portion 1311 is superimposed on the B.L. It may be superposed during the ON period of the L control signal.

[2.2. Specific example 2]
Next, another mode different from FIG. 348D will be described. That is, B. When the encoded signal period is longer than the ON period of the L control signal, only the header portion of the encoded signal is set to B.A. A specific example in which the L control signal is superimposed in the blanking period will be described.

FIG. 349 illustrates the B. It is a figure for demonstrating another concrete method of superimposing a coded signal on an L control signal.

In (a) of FIG. 349, a coded signal coded by I-4PRM is shown.

As shown in (b) of FIG. 349, only the header portion of (a) of FIG. 349 may be re-encoded by changing the encoding method from I-4PPM to 4PPM. In this case, as shown in (b) of FIG. 349, the header that has continued to be in the ON state and finally transitioned to the OFF state is changed to a header that continues to be in the OFF state and finally transitions to the ON state.

Then, as shown in (c) of FIG. 349, the encoded signal shown in (b) of FIG. It is superimposed on the L control signal. In the example shown in (c) of FIG. 349, the encoded signal composed of the header 1330 which is a signal in the OFF state, the header 1321 which is a signal in the ON state, and the data 1322 is B.264. It is superimposed on the L control signal.

More specifically, the second processing unit 1470 encodes the visible light communication signal to generate an encoded signal, superimposes the encoded signal as a visible light communication signal on the backlight control signal, and outputs the encoded signal. When superposed on the backlight control signal, a period of a signal indicating that the backlight is turned off in the backlight control signal and a period of the coded signal to be superposed are regions overlapping with each other. If there is a region, the header portion of the encoded signal is superimposed on the backlight control signal in the period of the signal indicating that the backlight 1490 is turned off, and the portion of the encoded signal other than the header portion is turned off of the backlight. Is superimposed on the backlight control signal in a period other than the period of the signal indicating.

As a result, the B. Even when the encoded signal period is longer than the ON period of the L control signal, the B.L. The data portion of the encoded signal can be superimposed during the ON period of the L control signal.

That is, for example, as shown in (c) of FIG. The encoding time length can be saved (shortened) by superimposing it in the blanking period of the L control signal.

When the adjustment period described in the eighteenth embodiment is provided, for example, the header 1310 of the encoded signal of FIG. It is necessary to subtract from the adjustment period the period during which the L control signal is overlapped with the blanking period and lighting is performed during the blanking period.

However, for example, as shown in (c) of FIG. 349, in the case of determining the phase in accordance with the end time of the header 1330 of the encoded signal (time when the ON state is last), the phase is determined by Since it is not turned on during the blanking period, there is no need to subtract from the adjustment period.

[3. Effect, etc.]
As described above, according to the present embodiment, it is possible to realize a display device that can output a visible light communication signal without significantly deteriorating the image quality of a display image and reduce reception errors of the output visible light communication signal. can do.

In the present embodiment, an example using a header of a coded signal coded using a general 4PPM coding method has been described, but the present invention is not limited to this. For example, when the average duty ratio of the header of the encoded signal is high, the header in which the ON signal and the OFF signal are reversed may be summed up in the blanking period. In this case, as described above, it is desirable to adjust the amount of decrease in the blanking period during the extinguishing period into the adjustment period.

In addition, B. When the entire encoded signal including the header can be superimposed in the ON period of the L control signal (the lighting period of the backlight 1490), the encoding may be performed so that the header has a high duty ratio.

Further, even when the header is superimposed on the blanking period, it may not be filled depending on the length of the blanking period. In that case, several types of headers may be prepared and used depending on the length of the blanking period.

(Embodiment 20)
In the present embodiment, B. A method of dividing the plurality of display regions into some groups and superimposing the coded signals so that the coded signals in all the coded signal periods can be superimposed in the ON period of the L control signal will be described.

[1. Operation of second processing]
In the following, a method of determining the timing of superimposing the coded signal based on the brightness of the area, centering on the brightest area, will be described as an example.

FIG. 350 is a flow chart for explaining the operation of the second processing in the twentieth embodiment.

First, in step S1311, the second processing unit 1470 encodes the visible light communication signal. More specifically, the second processing unit 1470 encodes the visible light communication signal and then generates an encoded signal to which a header or the like is added. The second processing unit 1470 also calculates the transmission time of the encoded signal based on the carrier frequency of the encoded signal.

Next, in step S1312, the second processing unit 1470 divides the display area into a plurality of areas.

Next, in step S1313, the second processing unit 1470 detects a bright area of the display. More specifically, the second processing unit 1470 detects the brightness of each of the divided areas, and based on this, selects the brightest area for display. Here, "bright as a display" means B. It does not mean that the duty ratio of the L control signal is large, but that it is the brightest signal level indicating the light emission energy of the image. The detection of bright areas will be described later in detail.

Next, in step S1314, second processing section 1470 matches the phase of the encoded signal with the bright area of the display. More specifically, the second processing unit 1470 determines that the B.I. B. of all regions or selected partial regions (selected multiple regions) according to the timing of the L control signal. A coded signal having the same phase is superimposed on the L control signal.

However, as in the other embodiments, the B.I. The encoded signal is not superimposed during the blanking period of the L control signal. This is because each B. This is equivalent to the operation of calculating the AND of the L control signal and the encoded signal. Note that the operations of steps S1301 to S1309 of FIG. 347 may be performed as necessary.

Next, in step S1315, the second processing unit 1470 determines whether the coded signal and the blanking period overlap. More specifically, the second processing unit 1470 determines, for each region, the coded signal period and the B.D. It is judged whether or not there is an overlapping portion between the blanking period of the L control signal and the B.L. When the blanking period of the L control signal does not overlap (Yes in S1315), the process proceeds to step S1316, and the second processing unit 1470 outputs the encoded signal to the B.L. It is superimposed on the L control signal, and the process ends. On the other hand, if there is an overlapping portion (No in S1315), the process proceeds to S1317.

On the other hand, in step S1317, second processing unit 1470 determines whether or not to perform a visible light communication signal. When the visible light communication is not performed (No in S1317), the process proceeds to step S1318. When performing the visible light communication signal (Yes in S1317), the process proceeds to step S1320, and the second processing unit 1470 adjusts the duty ratio so as not to transmit the encoded signal, and ends the process.

Next, in step S1318, second processing section 1470 changes the phase of the encoded signal, and outputs the encoded signal with the changed phase to the B.D. It is superimposed on the L control signal.

Next, in step S1319, the second processing unit 1470 determines whether the blanking period overlaps the bright area. If they do not overlap (No in S1319), the process proceeds to step S1320. If they overlap (Yes in S1319), the process proceeds to S1321.

Next, in step S1321, the second processing unit 1470 determines whether the processing has ended for each area. If the process is not completed (No in S1321), the process returns to S1315. If the process has been completed (Yes in S1321), the process proceeds to step S1322.

Next, in step S1322, the second processing unit 1470 determines whether or not there is an area in which the encoded signal is not superimposed. When there is a non-overlapping area (No in S1322), the process proceeds to step S1313. If there is no overlapping area (Yes in S1322), the process ends.

[2. Operation details]
Next, details (specific example) of the display device 1400 of the twentieth embodiment will be described with reference to FIGS. 351 and 352.

351 is a timing chart showing an example of group division of regions in the twentieth embodiment, and FIG. 352 is a timing chart showing another example of group division of regions in the twentieth embodiment. In FIGS. 351 and 352, a shaded portion (hatched portion) indicates a period (encoded signal period) in which the encoded signal is superimposed.

For example, as shown in FIG. 351, a plurality of display areas are divided into three groups. Specifically, the area A, the area B, and the area C are divided into a group G1, the area F, the areas G and H are divided into a group G2, and the area D and the area E are divided into a group G3. Then, as shown in FIG. 351, in each group, coded signals are superimposed at the same timing within the same period. For example, in the group G1, the brightest area C is used as a reference for superimposition, and in the group G2, the brightest area E is used as a reference for this group.

Note that, as shown in FIG. 352, a plurality of display areas may be divided into two groups. That is, the area A, the area B, the area C, and the area D may be divided into the group G1, and the area E, the area F, the area G, and the area H may be divided into the group G2. Then, in each group, the coded signal is superimposed at the same timing within the same period.

[3. Effect, etc.]
As described above, according to the display device of the present embodiment, the signal processing unit (second processing unit 1470) causes the backlight of each of a plurality of groups including a plurality of neighboring regions among the plurality of regions. For the control signal, the visible light communication signal is respectively superimposed, the visible light communication signal to be superimposed on each of the plurality of groups are in phase with each other, the backlight control signal of each of the plurality of groups During the period in which the emission of the backlight (1490) is controlled, all of the corresponding visible light communication signals are superimposed.

As a result, the present display device is Since the coded signals of all the coded signal periods can be superimposed on the ON period of the L control signal, reception errors of the output visible light communication signal can be reduced. In other words, B. Since the visible light communication signal can be superimposed without being lost during the ON period of the L control signal, it is possible to reduce reception errors of the output visible light communication signal.

Further, the signal processing unit (second processing unit 1470) is superimposed on each of the plurality of groups based on the backlight control signal of a predetermined region among the plurality of regions included in the plurality of groups. The phases of the visible light communication signals may be matched.

As a result, the display device can output the visible light communication signal for each of the selected plurality of groups without further loss.

Here, the predetermined area is the brightest area of the plurality of areas.

Accordingly, in the display device 1400, the difference in luminance within the display area can be made inconspicuous.

Further, the phase of the visible light communication signal superimposed on one of the plurality of groups and the phase of the visible light communication signal superimposed on other than the plurality of groups are different.

Accordingly, the display device 1400 can output the visible light communication signal for each of the selected groups without further loss.

In some cases, it is not possible to divide an area into groups as described above. That is, even if the region is divided into groups, there may be a region in which encoded signals of the same phase do not enter. The operation in this case will be described below.

FIG. 353 is a timing chart showing another example of area division in Embodiment 20. In FIG. 353, a shaded portion (hatched portion) indicates a period (encoded signal period) in which the encoded signal is superimposed.

For example, the example shown in FIG. 353 is a special example of FIGS. 351 and 352. As shown in FIG. 353, when the region is divided into groups, the coded signal may not be transmitted in a place where the coded signal of the same phase does not enter.

Specifically, the area A, the area B, the area C, and the area D are divided into other areas, and the encoded signals of the same phase are superimposed on the area A, the area B, the area C, and the area D. Here, in the region D, the coded signal is not superimposed in a period in which the coded signal overlaps the blanking period. Further, in the example shown in FIG. 353, the coded signal is not superimposed on the regions after the region D (regions E to H).

Even if the region is divided into groups, if there is a region where the coded signal of the same phase does not enter, simply define the reference region and superimpose the coded signal only around that region (nearby region). You may allow it. At this time, the range in which the coded signal is superimposed may be based on the above-described flowchart or may be limited to the range determined in advance.

Further, the adjustment period described above may be provided so that no difference in luminance occurs between the region where the coded signal is superimposed, the region where the coded signal is not superimposed, and the region where the coded signal is superimposed.

In the present embodiment, the B. Although the L control signal is superimposed on the basis of the rising edge of the L control signal, other B.L. The characteristic timing of the L control signal may be used as a reference, or the synchronization signal of the video signal itself may be used as a reference. Alternatively, a signal delayed for a certain time from the video synchronization signal may be created and used as a reference.

It is very difficult to find a period that is not the blanking period in all of the plurality of display regions, and even if there is such a period, it is very short. In the present disclosure, the encoded signal is B. Even when it is superimposed on the L control signal, priority is given to the blanking period as much as possible to suppress the lighting during the blanking period, thereby avoiding the deterioration of the image quality.

However, even if the blanking period and the coded signal period overlap in one region, the blanking period and the coded signal period often overlap in another region.

Therefore, this embodiment discloses a method for avoiding overlapping of the blanking period and the coded signal period in as many regions as possible of the plurality of regions of the display region. That is, in the present embodiment, a plurality of regions are divided into some groups, and coded signals are superimposed in each group at a constant phase. Thereby, it is possible to reduce the overlap between the blanking period and the coded signal in the corresponding group.

Although an example of dividing a plurality of regions into two groups or three groups has been described in the present embodiment, the present invention is not limited to this.

Further, as a method of grouping a plurality of regions, the regions may be divided into some groups in advance, and how to shift the phase may be set in advance.

Further, in the present embodiment, a plurality of regions are grouped so that the coded signal length (all of the coded signal periods) can be superimposed on the basis of the bright region, but the present invention is not limited to this. The number of groups may be limited because the number of groups divided by this criterion may be very large. The regions divided into groups do not necessarily need to be able to overlap all encoded signal periods.

Also, the coded signals to be superimposed on the areas in each group may be the same or different. If two or more signals are mixed in the encoded signal obtained on the receiver side, the probability of erroneous recognition or error occurrence increases. Therefore, when different coded signals from two or more signals can be received by the same receiver at the same time, two or more signals having the same coded signal but different phases are received by the same receiver at the same time. When possible, or a combination thereof is meant. As a result, the probability of erroneous recognition or error occurrence can be reduced.

Further, the group to be divided with some criterion is not limited to the above example, and may be divided into groups based on the signal processing result performed by the second processing unit 1470 based on the relationship between the video signal and the encoded signal. ..

Further, in a backlight using an LED or the like, its light source is a very small object close to a point light source. Therefore, a light guide plate, a diffusion plate, or the like is used to cover an area in order to emit light in a plane like an LCD. Spreading. Therefore, when controlling the LEDs in each region, adjacent regions are designed to overlap each other, and there is a certain amount or more of leakage light.

Therefore, in a backlight using LEDs and the like, even when divided into several groups, another signal enters as noise due to leaked light from at least the adjacent regions, so that the codes of the regions including the adjacent blocks are coded. It is necessary to avoid overlapping the encoded signals in time. Therefore, for example, the coded signal may not be transmitted in the corresponding frame in the corresponding location, or the coded signal that is temporally continuous or overlapped may be transmitted in the area at a distant place.

In the case where the coded signal is not transmitted in the corresponding frame at the corresponding place, it may be determined from which area the coded signal is to be output for each frame and separated, or at a specific place (link to the video signal). Coded signal) may be preferentially transmitted.

Further, even when the transmission periods of encoded signals of different phases in different areas overlap, it is sufficient that the areas are not continuous or have a certain interval. This is because the signal can be received when the signal is received in a limited area. Note that the interval between regions having different phases needs to be determined depending on the extent to which the leaked light of the backlight extends, and is therefore a value based on the performance of the display device used.

Further, each area may be divided into a plurality of blocks and the above method may be applied to each.

(Embodiment 21)
When a coded signal is received by using a light intensity sensor such as a photodiode having a very fast response speed, the phase difference between the image and the coded signal does not cause a big problem.

On the other hand, when the coded signal is captured by using an image sensor such as a camera or a digital still camera attached to a smartphone or a mobile phone, the exposure timing is ON due to a slight difference in phase. It may not be able to be acquired as a valid signal due to a very short time difference or at the same time as the −OFF edge portion or the start or/and end timing of a series of encoded signal periods. In other words, the image pickup cycle of the image sensor is generally 30 FPS. For example, when the image signal of 60 FPS and the encoded signal are synchronized, if the timing does not match at the time of image pickup of the image sensor, it will be forever. It is also conceivable that the timing of the imaging cycle of the image sensor and the cycle of the encoded signal do not match.

Therefore, in this embodiment, in order to avoid this, a method of shifting the phase of the encoded signal will be described.

[1. Operation of display device]
Hereinafter, the operation of the second processing unit 1470 will be mainly described.

FIG. 354 is a flowchart for explaining the operation of the second processing unit in the twenty-first embodiment.

First, in step S1331, the second processing unit 1470 shifts the synchronization of signals. More specifically, the second processing unit 1470 shifts the synchronization of the encoded signal when the synchronization between the display panel 1450 and the backlight 1490 is not fixed. This is effective in increasing the probability of capturing an image with the smartphone 1350.

Next, in step S1332, the second processing unit 1470 determines, based on the duty ratio based on the video signal output from the first processing unit 1430, that the B. The AND of the L control signal and the encoded signal is calculated.

Next, in step S1333, the second processing unit 1470 adjusts the duty ratio based on the video signal and/or the visible light communication signal.

More specifically, as described in the eighteenth embodiment, second processing section 1470 checks whether the coded signal period and the blanking period overlap and divides the cases to provide an adjustment period. .. The second processing unit 1470 controls the B.B. The duty ratio of the L control signal is based on the original video signal. When it is different from the duty ratio of the L control signal, the duty ratio is adjusted by utilizing the period during which the transmission of the encoded signal is stopped. Here, for example, the second processing unit 1470 adjusts the duty ratio by setting a period for turning off the backlight 1490 (OFF period of the BL control signal) in a period different from the blanking period. .. Then, the second processing unit 1470 determines that the B.D.B. The L control signal is output to the second control unit 1480.

In addition, when the relationship between the phases of the encoded signal and the video signal is restored in a certain period, these may be corrected to a predetermined phase difference.

Furthermore, at a frequency different from the frequency of the video signal, the phase of the encoded signal and the phase of the video signal temporally change, that is, if either one is not a multiple of the other, It is not necessary to control the phase adjustment. This is because, even if the phases of both are not matched, the relationship between the phases of both will be restored after a certain time elapses, and there will always be a time zone where it is difficult to receive a signal and a time zone where reception is easy. Because it will exist in.

355A and FIG. 355B are the same as those in Embodiment 21. It is a figure for demonstrating the relationship of the phase of a L control signal and a visible light communication signal.

For example, in FIG. An encoded signal based on a visible light communication signal based on the L control signal X and B.L. It shows that the L control signal X and the L control signal X have the same phase at a constant cycle. The shaded area in the figure indicates the period during which the encoded signal is actually being transmitted. Although the display is such that it is output at a shorter cycle and a longer cycle than the L control signal, it does not matter which of the signal lengths is longer as described above. Also, the actual encoded signal transmission period and B. The length of the L control signal and the length of the L control signal do not have to be long, but the coding transmission period is B. It is preferably shorter than the L control signal. Here, B. The encoded signal is to be repeated 7 times while the L control signal X is repeated 12 times. When the L control signal is 60 fps, both phases have the same relationship at intervals of 0.2 seconds. On the other hand, in FIG. Although there is no relationship between the L control signal X and the encoded signal, in f1, B. The first half of the L control signal, the second half of f2, the substantially middle position of f3, the start of the encoded signal transmission period for each frame, The relationship of the start phase of the L control signal has changed. However, in particular, although the two have the least common multiple and the phase relationship does not return to the original one, the defects due to the timing of imaging can be solved somewhere by sequentially shifting the phases. In addition, B. Even when the coded signal is transmitted in the period corresponding to the break of the L control signal, the coded signal is interrupted in the area X even at the time starting from f2 or f5, but the coded signal is always transmitted in some other area. There is no problem because it can be sent. Regarding the relationship between the video and the communication information, the memory may be stored in a buffer or the like, and the immediately preceding memory may be pulled out and encoded as a communication signal for use. In addition, when the time until the relationship between the two phases is restored is very long, for example, when it takes several seconds or more, the relationship between the phases may be forcibly restored. For example, after the encoded signal is completed at f8 in FIG. 355B, B.I. The phases of the L control signal and the coded signal may or may not be adjusted again. The timing adjustment cycle may be performed, for example, every one second, or may not be performed at all.

[2. Operation details]
Next, the details (specific example) of the operation of the display device 1400 of the twentieth embodiment will be described with reference to FIGS. 356A, 356B, and 356C.

356A, 356B, and 356C are timing charts for describing the operation of the second processing unit according to the twenty-first embodiment. The shaded area (hatched area) indicates the area where the encoded signal exists. FIG. 356A shows the B.264 signal before superimposing the coded signal. 356B shows a timing chart of the L control signal, and FIG. 356B shows the B.L. The timing chart of the L control signal is shown. FIG. 356C shows that the phase of the backlight control signal and the phase of the visible light communication signal are eventually set by setting the delay time from the rising or falling timing of the backlight control signal that is based on the encoded signal. An example in which the relationship of is changed with time is shown.

For example, as shown in FIG. The synchronization with the L control signal is shifted. This makes it possible to reliably generate a timing at which the receiver such as the smartphone 1350 can receive the encoded signal. Here, the adjustment period described above may be calculated and provided for each phase difference in each frame.

Note that, as shown in FIG. 356C, for example, taking the region A as a reference, the time difference β1 between the rising U2 of the backlight control signal and the start V2 of the visible light communication signal may be set in advance as the delay time and may be superimposed. The time difference β2 between the rising U3 of the next frame and the start V3 of the visible light communication signal may be the same as β1 or may be different. Also, in the example shown in FIG. 356C, β indicates a delay and a positive numerical value (time), but a negative numerical value (time) may precede it.

Further, frames in which β is 0 may be mixed. The reference area may be anywhere or may be selected according to the above-mentioned reference. Although the reference time is the rising edge of the backlight control signal, this may be another falling edge or other characteristic characteristic of the signal waveform. The reference time may be based on the sync signal of the video signal itself, in addition to the characteristic part of the backlight control signal in a predetermined area, or may be a signal delayed by a certain time from the video sync signal. It may be created and the signal may be used as a reference.

Further, in the present embodiment, since the video signal and the encoded signal do not correspond one-to-one, some encoded data and image data are buffered in advance in a memory (not shown) in the display device 1400, It is good to perform processing.

It should be noted that the cycle of the video signal (1 frame length) and the cycle of superimposing the coded signal should preferably have the least common multiple within 1 second, more preferably within 0.5 seconds. Further, when these two cycles are synchronized, and from the timing at which these two cycles are synchronized, tracking is performed for the period of the least common multiple or for each time of an integral multiple thereof, and the minute time generated by the error generated in each is The shift (phase difference) may be corrected.

Further, as described above, when the cycle or/and frequency of the video signal and the cycle or/and frequency of the encoded signal have a relationship of changing the phase relationship with respect to time, each cycle is within 1 second. Even if it does not have the least common multiple, if the rate of change is fast, for example, if the change of the same phase is repeated within 1 second, the relationship between the two phases may not be particularly controlled. The rate of change preferably has the relationship as in the above example, but is not limited to this.

[3. Effect, etc.]
As described above, in the display device of the present embodiment, the signal processing unit, with respect to the backlight control signal of each of the plurality of regions, based on the backlight control signal of any one of the plurality of regions as a reference. The signal processing unit (second processing unit 1470) changes the delay time for encoding the visible light communication signal with respect to the backlight control signal of each of the plurality of regions. The delay time for encoding the visible light communication signal (encoded signal) is temporally changed with reference to the backlight control signal in any one of the areas.

With this configuration, it is possible to reliably generate a timing at which the receiver such as the smartphone 1350 can receive the encoded signal.

The signal processing unit (second processing unit 1470) superimposes a visible light communication signal (encoded signal) on the plurality of backlight control signals at a cycle different from the cycle of the backlight control signals, In each of the plurality of regions, the relationship between the phase of the backlight control signal and the phase of the visible light communication signal may change with the frame.

Here, the cycle of superimposing a visible light communication signal different from the cycle of the backlight control signal may change with time.

Further, the phase of the visible light communication signal to be superimposed on the plurality of backlight control signals may be the same in all regions where the visible light communication signal is superimposed.

Further, the cycle of phase shift of the visible light communication signal superimposed on each of the plurality of areas and one frame cycle of the backlight control signal may have a least common multiple within 1 sec.

As a result, the timing at which the receiver such as the smartphone 1350 can receive the encoded signal can be reliably generated within a relatively short period of time.

Further, the signal processing unit (second processing unit 1470) is configured to perform a phase shift cycle of the visible light communication signal (encoded signal) superimposed on each of the plurality of regions and one frame of the backlight control signal. The start point of the phase shift cycle of the visible light communication signal (encoded signal) superimposed on each of the plurality of areas is set to one frame cycle of the backlight control signal at every least common multiple or integer multiple of the cycle. It may be corrected.

Accordingly, by correcting the phase shift, it is possible to prevent the timing at which the receiver side such as the smartphone 1350 can receive the encoded signal from extending for a long period of time.

Here, the time indicated by the least common multiple of the two types of periods is usually a number (time) that the reception can be performed within at least this time if the position and the environment are such that the communication signal can be received. I think that it must be within the time that the person who thinks it can hold the receiver and wait. Even for normal NFC, it is recommended to hold it for 1 second as a guideline for waiting time, and it is desirable that it is equal or less. Further, it is more preferable that the least common multiple is set within 0.5 seconds as a time that does not become a psychological burden.

(Embodiment 22)
In the eighteenth to twenty-first embodiments, a case has been described in which each area is sequentially controlled at the scanning speed driven at the normal speed to display the image signal, but each area is moved at the double speed driving scanning speed higher than the normal speed. The image signals may be displayed in sequence.

In the present embodiment, a case will be described as an example where a 2× speed image signal is sequentially controlled and displayed in each region at a scanning speed of 4× speed drive. Hereinafter, the blanking period will be described based on the time of double speed driving.

[1. Operation of display device]
Hereinafter, the operation of the second processing unit 1470 will be mainly described.

357A and 357B are timing charts for explaining the operation of the second processing unit according to the twenty-second embodiment. The shaded area (hatched area) indicates the area where the encoded signal exists. FIG. 357A shows the B.D. FIG. 357B shows a timing chart of the L control signal, and FIG. 357B shows the B.L. The timing chart of the L control signal is shown.

For example, as shown in FIG. L control signals AB. There is no period in which the L control signal H is simultaneously lit. That is, it indicates that the coded signal cannot be simultaneously superimposed on all the display areas.

Therefore, in the present embodiment, as shown in FIG. 357B, for example, the scanning period between regions of the blanking period is set to half the normal period. Then, among the plurality of areas (all areas may be sufficient), the B. A region H, which is a region where the blanking period of the L control signal starts late, is selected.

The second processing unit 1470 selects the area H selected at the timing (the time when the BL control signal H is turned ON) when the blanking period in the area H ends and the backlight 1490 starts lighting. The coded signal is superimposed on.

In the example illustrated in FIG. 357B, the second processing unit 1470 determines that the B. When the blanking period by the L control signal H is completed, The encoded signal is superimposed at the timing when the L control signal H is turned on.

Thereby, the second processing unit 1470 can set the period in which the coded signal is superimposed in any of the display regions for a period of at most ½ frame.

[2. Effect, etc.]
As described above, in the display device of the present embodiment, the display control unit (first control unit 1440) controls the display panel according to the high-speed scanning speed that is higher than the scanning speed indicated by the video signal. The display panel (1450) is controlled to display an image on the display surface.

Thereby, the display device can take a longer period in which the encoded signal can be output.

Note that the encoded signal length (encoded signal period) is long, and B. If the L control signal cannot be superimposed only during the ON period (a period other than the blanking period) and there is a region overlapping with the blanking period, the coded signal is not superimposed during the blanking period in the corresponding region.

In addition, B. An adjustment period for turning on the backlight 1490 in the blanking period may be provided for the turn-off time by the encoded signal superimposed on the ON period of the L control signal. At that time, the adjustment period may be created by the method described in the above embodiment, or the header of the encoded signal may be superimposed on the blanking period. Further, a plurality of areas of the display area may be decomposed into some groups and the encoded signal may be superimposed.

Further, the same work may be performed in a region (other region) other than the applicable region, or no signal may be output at all. In this case, by the method described in the above embodiment, the adjustment period of the light-off time may be provided so that the duty ratio based on the visible light communication signal and/or the video signal is the same on all screens. Further, as in the twentieth embodiment, the brightest region may be selected and the coded signal may be superimposed at a timing matched with the region. In the present embodiment, the B. Although the L control signal is superimposed on the basis of the rising edge of the L control signal, other B.L. The characteristic timing of the L control signal may be used as a reference, or the synchronization signal of the video signal itself may be used as a reference. Alternatively, a signal delayed for a certain time from the video synchronization signal may be created and used as a reference.

In the present embodiment, an example in which the 2× scanning speed is set to the 4× scanning speed has been described, but the present invention is not limited to this. The number of frames may be left unchanged and only the scanning speed may be increased.

Further, in the present embodiment, it is assumed that a signal is transmitted in such a form in advance, but the second processing unit uses the relationship between the video signal and the coded signal in the present embodiment. The method of transmitting the signal may be adopted. In this case, since the transmission of the signal from the second processing unit 1470 to the first processing unit 1430 in FIG. 340 enters, the arrow connecting these two blocks becomes bidirectional.

(Embodiment 23)
In the eighteenth to twenty-second embodiments, the backlight scan is described as the control method in which the period for performing the turn-off control at different timings in each of the plurality of regions is described as the backlight scan, but the present invention is not limited thereto. It may be local dimming.

In the present embodiment, an operation when local dimming is adapted will be described.

Here, in local dimming, the display area (screen) is divided into several areas, the transmittance of the liquid crystal in that area is made higher than usual, and the brightness of the backlight is reduced (the duty is decreased) by that amount. Is a backlight control method for lowering power. Power can be reduced by the above control when there is a margin to increase the transmittance (when the highest luminance has a relatively low value) in the pixel having the highest luminance in the corresponding area. Further, by reducing the duty of the backlight, the lighting period is reduced, and as a result, the contrast is improved.

[1. Backlight control by local dimming]
Next, B.D. controlled by local dimming. The L control signal will be described.

FIG. 358 is a timing chart showing backlight control during local dimming adaptation in the twenty-third embodiment.

When the backlight control is performed by applying the local dimming, as shown in FIG. 358, for example, the interval T of the start timing of the blanking period is uniform between adjacent regions, but the length of the blanking period is Not uniform.

Therefore, in the display device 1400 of the present embodiment, in each area of the display area, the B.D. After the blanking period of the L control signal is stored in memory, the process (operation) described below may be performed.

[2. Operation of display device]
Hereinafter, the operation of the second processing unit 1470 will be mainly described. Note that this embodiment shows signal control in the case where the OFF period per frame in each region of the display region is aligned.

[2.1. Example of operation of second processing unit]
FIG. 359 is a flowchart for explaining an example of operation of the second processing portion in Embodiment 23.

First, in step S1341, the second processing unit 1470 calculates the adjustment period. Specifically, the turn-off time in the encoded signal is N1, and the B.T. If the light-off time of the L control signal is N2, it can be expressed as a relational expression of the adjustment period N=N2-N1. Accordingly, the second processing unit 1470 can calculate (calculate) the adjustment period.

Next, in step S1342, second processing section 1470 determines whether the sum (N+C) of adjustment period N and coded signal period C is one frame period or less.

If the second processing unit 1470 determines that (N+C) is less than or equal to one frame period (Yes in S1342), the process proceeds to S1343. On the other hand, when the second processing unit 1470 determines that (N+C) is longer than one frame period (No in S1342), the processing proceeds to S1346, stops outputting the encoded signal, and ends the processing.

Next, in step S1343, the second processing unit 1470 determines whether the adjustment period N is 0 or more.

When N is 0 or more (Yes in S1343), the second processing unit 1470 proceeds to S1344, and sets an extinguishing period backward from the start of the next encoded signal by the adjustment period. In addition, the encoded signal is not output during this period, and the processing is ended.

On the other hand, when N is smaller than 0 (No in S1343), the second processing unit 1470 proceeds to S1345, and B. Starting from the end of the blanking period of the L control signal, the B.L. A lighting period (ON period) for the adjustment period is provided in the blanking period of the L control signal. In addition, the encoded signal is not output during this adjustment period.

FIG. 360 is a timing chart for describing an example of operation of the second processing portion in Embodiment 23. Here, the thick line indicates B.I. The ON period and the OFF period of the L control signal are shown, and the region A will be described below as a reference region. In addition, B. A region controlled by the L control signal X (X is A to H) is referred to as a region X.

For example, as shown in FIG. 360, second processing section 1470 superimposes a coded signal of the same phase on each area from the start timing of the frame of area A, which is the reference area, and provides an adjustment period. Note that the adjustment period may be provided according to the second method of the eighteenth embodiment, but the second method has been described above, and therefore description thereof will be omitted here.

In the present embodiment, the B.I. When the L control signal is in the OFF period (blanking period), the coded signal is not superimposed and the B.L. When the L control signal is in the ON period, the encoded signal is basically superimposed. Note that the adjustment period may be converted in consideration of the duty ratio of the encoded signal. In that case, if the adjustment period is a period for outputting the encoded signal, the encoded signal is superimposed and output. May be

[2.2. Example of operation of second processing unit]
Also during local dimming, it is possible to give priority to having a sequential blanking period as in the case of normal backlight scan control. The processing in this case will be described below.

FIG. 361 is a flowchart for explaining an example of operation of the second processing portion in Embodiment 23.

First, in step S2101, the second processing unit 1470 calculates the adjustment period. Specifically, assuming that the blanking period of the predetermined area is N1, the extinguishing time of the encoded signal is N2, and the blanking period of the corresponding area is N3, it can be expressed as a relational expression of adjustment period N=N1-N2-N3. Accordingly, the second processing unit 1470 can calculate (calculate) the adjustment period.

Next, in step S2102, the second processing unit 1470 determines whether the sum (N+C+N3) of the adjustment period N, the coded signal period C, and the blanking period N3 of the corresponding area is one frame period or less, Memorize the judgment result.

Next, in step S2103, second processing section 1470 determines whether adjustment period N is 0 or more, and stores the determination result.

After taking the steps as described above, the second processing unit 1470 provides the adjustment period, the adjustment period, etc. based on the determination results of N1 to N3, S2102, and S2103 stored for each area. , Visible light communication signal is displayed at the same time as image display.

The adjustment period may be created by a combination with the second method of the eighteenth embodiment or the method described in the nineteenth to twenty-second embodiments.

FIG. 362 is a timing chart for describing an example of operation of the second processing portion in Embodiment 23. In FIG. 362, the adjustment period is provided by the second method described in Embodiment 18. The thick line indicates B.I. The ON period and the OFF period of the L control signal are shown, and the region A will be described below as a reference region.

For example, as shown in FIG. 362, the second processing unit 1470 outputs each of the in-phase coded signals in a period from time P to time Q after a predetermined period has elapsed from the start timing of the frame of the reference region A. An adjustment period is provided while overlapping the area. Note that the adjustment period may be set according to the second method of the eighteenth embodiment, but the second method has been described above, and therefore description thereof will be omitted here.

In the present embodiment, the B.I. When the L control signal is in the OFF period (blanking period), the coded signal is not superimposed and the B.L. When the L control signal is in the ON period, the encoded signal is basically superimposed. Therefore, for example, in area A, B. Since it is a blanking period in which the L control signal A is OFF, the coded signal is not superimposed. Then, the adjustment period is provided after the encoded signal period C.

The adjustment period may be converted in consideration of the duty ratio of the coded signal. In that case, if the period is the adjustment period and the coded signal is output, the coded signal is superimposed and output. May be

[2.3. Example of operation of second processing unit]
FIG. 363 is a timing chart for describing an example of operation of the second processing portion in Embodiment 23.

When backlight control is performed by applying local dimming, basically B. The blanking period of the L control signal differs for each frame and each region. Therefore, a temporary blanking period (described as a prescribed blanking period) is set for convenience of calculation. Then, the adjustment period can be calculated according to the second method of the nineteenth embodiment from the specified blanking period, the coded signal period, their phase difference, and the original blanking period. Hereinafter, an example of this case will be described with reference to FIG. The thick line in FIG. 363 shows the waveform showing the original blanking period.

The prescribed blanking period is determined based on the average length of the blanking period on the screen and the shortest period. Here, it is assumed that the prescribed blanking period is an extinguishing period in which coded signals are not superimposed. The encoded signal period is a period in which the encoded signal is superimposed.

Further, the adjustment period may be provided by using the second method of the eighteenth embodiment. If the adjustment period is positive, the backlight 1490 is turned off during this period. It is sufficient to adjust the L control signal, and if the adjustment period is negative, the B.I. The L control signal may be adjusted. If the adjustment period is provided retroactively to the blanking period, the B.I. The L control signal may be adjusted. If the adjustment period is negative, the B. When the coded signal is superimposed on the L control signal, the adjustment period may be corrected based on the duty ratio.

[3. Effect, etc.]
As described above, in the display device of the present embodiment, the backlight control unit (second control unit 1480) follows the backlight control signal output by the signal processing unit (second processing unit 1470). , A period for performing light emission control according to the light emission amount of the backlight based on each video signal in each of the plurality of regions and performing a light-off control at different timings in each of the plurality of regions, and in each of the plurality of regions The duty of the backlight based on the video signal and the visible light communication signal is also changed.

In the present embodiment, the B. Although the L control signal is superimposed on the basis of the rising edge of the L control signal, other B.L. The characteristic timing of the L control signal may be used as a reference, or the synchronization signal of the video signal itself may be used as a reference. Alternatively, a signal delayed for a certain time from the video synchronization signal may be created and used as a reference.

As described above, in the present embodiment, the case of adapting the local dimming has been described. However, in the local dimming, a region is two-dimensionally divided, and a video signal may be simultaneously scanned and written in a certain direction. Although a combination of regions having the same phase but different blanking periods occurs, the contents described in the present embodiment can be applied.

As described above, the embodiments have been described as examples of the technology according to the present disclosure. To that end, the accompanying drawings and detailed description are provided.

Therefore, among the constituent elements described in the accompanying drawings and the detailed description, not only constituent elements essential for solving the problem but also constituent elements not essential for solving the problem in order to exemplify the above technology Can also be included. Therefore, it should not be immediately recognized that the non-essential components are essential, because the non-essential components are described in the accompanying drawings and the detailed description.

Further, the above-described embodiment is for exemplifying the technology in the present disclosure, and thus the technology in the present disclosure is not limited thereto, and various modifications and replacements are made within the scope of the claims or the scope of equivalents thereof. It can be added or omitted.

For example, in the above-described embodiment, B. The case where the superposition is performed with the rising edge of the L control signal as a reference has been described as an example, but the invention is not limited thereto. Others such as falling B. The characteristic timing of the L control signal may be used as a reference, or the synchronization signal of the video signal itself may be used as a reference. Alternatively, a signal delayed for a certain time from the video synchronization signal may be created and used as a reference.

The present disclosure can be applied to a display device that can output a visible light communication signal without significantly deteriorating the image quality of a display image and reduce reception errors of the output visible light communication signal. Specifically, since the display device according to the present disclosure can safely and actively acquire information other than images, it can be used not only for home TVs or devices such as PCs and tablets, but also for signage on the go. The information terminal and the information display device can be applied to various applications such as transfer of image supplementary information and information transmission in all situations in the sense that required information can be safely obtained because of its activeness.

Further, for example, the display device according to the eighteenth to twenty-third embodiments is a display device capable of outputting a visible light communication signal, and includes a display panel having a display surface for displaying a video, and the display panel based on the video signal. A display control unit that controls the display panel to display an image on a display surface, a backlight having a light emitting surface that illuminates the display surface of the display panel from the back, and the visible light communication signal based on the image signal. And a signal processing unit to be superimposed on the backlight control signal generated by dividing the light emitting surface of the backlight into a plurality of regions, in each of the plurality of regions according to the backlight control signal output by the signal processing unit. A backlight control unit that controls the emission of light and that provides a period for controlling the turning off at different timings in each of the plurality of regions, wherein the signal processing unit transmits the visible light communication signal to the backlight control signal. The visible light communication signal may not be superimposed on the signal indicating that the backlight is turned off among the backlight control signals.

Also, for example, the signal processing unit, the backlight control signal of each of the plurality of regions, the visible light communication signal is respectively superimposed, the visible light communication signal superimposed on each of the plurality of regions, They may be in phase with each other. Here, for example, in the display device according to any of the eighteenth to twenty-third embodiments, the signal processing unit is superimposed on each of the plurality of regions based on the backlight control signal of a predetermined region among the plurality of regions. The phases of the visible light communication signals may be matched.

Further, for example, the predetermined region may be the brightest region of the plurality of regions, and the predetermined region is a region of the plurality of regions that corresponds to an end portion of the display surface. May be

Further, for example, the signal processing unit, for each of the backlight control signal of each of a plurality of groups including a plurality of regions in the vicinity of the plurality of regions, superimposes the visible light communication signal, respectively, The visible light communication signals superimposed on each group are in phase with each other, and during the period for controlling the emission of the backlight of the backlight control signals of each of the plurality of groups, the corresponding visible light communication signals All may be superposed.

Here, for example, the signal processing unit, based on the backlight control signal of a predetermined region of the plurality of regions included in the plurality of groups, the visible light communication signal to be superimposed on each of the plurality of groups, The phases may be matched. Alternatively, the predetermined area may be the brightest area of the plurality of areas.

Further, for example, the phase of the visible light communication signal superimposed on one of the plurality of groups and the phase of the visible light communication signal superimposed on other than the plurality of groups may be different.

In addition, for example, the signal processing unit, when superimposing the visible light communication signal on the backlight control signal, a period of a signal indicating that the backlight is turned off among the backlight control signals, the superimposing When there is a region of the plurality of regions in which the period of the visible light communication signal overlaps, a lighting adjustment period for adjusting the brightness of the overlapping region is provided in the overlapping region, On/off of the backlight control signal may be adjusted during the lighting adjustment period.

Further, for example, the signal processing unit encodes the visible light communication signal to generate an encoded signal, superimposes the encoded signal as the visible light communication signal on the backlight control signal, and outputs the encoded signal. Is superimposed on the backlight control signal, a period of a signal indicating that the backlight is turned off in the backlight control signal and a period of the coded signal to be superimposed are regions that overlap, If there is a region out of the region, the header portion of the encoded signal is superimposed on the backlight control signal during the period of the signal indicating that the backlight is off, and the header portion of the encoded signal is included. Other portions may be superposed on the backlight control signal in a period other than the period of the signal indicating that the backlight is off.

Further, for example, the signal processing unit superimposes a visible light communication signal on the plurality of backlight control signals in a cycle different from the cycle of the backlight control signal, and controls the backlight in each of the plurality of areas. The relationship between the phase of the signal and the phase of the visible light communication signal may change with the frame. Here, the cycle of superimposing a visible light communication signal different from the cycle of the backlight control signal may change with time.

Further, for example, the signal processing unit delays the backlight control signal for each of the plurality of regions by encoding a visible light communication signal based on one of the backlight control signals of the plurality of regions. The time may be changed with time.

Further, for example, the phase of the visible light communication signal to be superimposed on the plurality of backlight control signals may be the same phase in all regions where the visible light communication signal is superimposed.

Further, for example, the cycle of phase shift of the visible light communication signal superimposed on each of the plurality of areas and one frame cycle of the backlight control signal may have a least common multiple within 1 sec.

In addition, for example, the signal processing unit may be a least common multiple or an integer multiple of the phase shift cycle of the visible light communication signal superimposed on each of the plurality of regions and one frame cycle of the backlight control signal, at each time. In addition, the start point of the phase shift cycle of the visible light communication signal superimposed on each of the plurality of regions may be corrected to one frame cycle of the backlight control signal.

Note that, for example, the display control unit may control the display panel to display an image on the display surface of the display panel at a high scanning speed that is higher than the scanning speed indicated by the video signal.

The backlight control unit controls light emission in each of the plurality of regions according to the backlight control signal output from the signal processing unit, and the backlight based on each video signal in each of the plurality of regions. It is also possible to provide a period in which the extinction control is performed at different timings according to the light emission amount, and change the duty of the backlight based on the video signal and the visible light communication signal in each of the plurality of regions.

Further, the control method of the display device according to the eighteenth to twenty-third embodiments is a control method of the display device capable of outputting a visible light communication signal, and the display device has a display panel having a display surface for displaying an image, A backlight having a light emitting surface that illuminates the display surface of the display panel from the backside, and the control method controls the display panel to display an image on the display surface of the display panel based on an image signal. A display control step; a signal processing step of superimposing the visible light communication signal on a backlight control signal generated based on the video signal; a light emitting surface of the backlight divided into a plurality of areas; In accordance with the backlight control signal output in, the light emission control in each of the plurality of areas, and, including a backlight control step of providing a period for controlling the turning off at different timing in each of the plurality of areas, In the signal processing step, when superimposing the visible light communication signal on the backlight control signal, the visible light communication signal is not superposed on a signal indicating turning off of the backlight among the backlight control signals. ..

According to the eighteenth to twenty-third embodiments, a display device capable of outputting a visible light communication signal without significantly degrading the image quality of a display image and reducing reception errors of the output visible light communication signal. Is applicable to. Specifically, since the display devices according to the eighteenth to twenty-third embodiments can safely and actively acquire information other than images, not only home TVs, PCs, tablets, and other devices, but also when going out. Applicable to various purposes such as transfer of image incidental information and information transmission in all situations in the sense that even in signage, information terminals, and information display devices, the required information can be safely obtained due to its activity. It is possible.

(Embodiment 24)
The present disclosure relates to a display device and a display method capable of outputting a visible light communication signal.

Japanese Unexamined Patent Application Publication No. 2007-43706 and Japanese Unexamined Patent Application Publication No. 2009-212768, which disclose visible light communication technology using a display backlight, disclose a display device that superimposes and displays communication information by visible light in a video signal. ing.

The present disclosure provides a display device that outputs a visible light communication signal that can be restored by a receiving device.

A display device according to the present disclosure is a display device capable of outputting a visible light communication signal composed of a plurality of signal units in a carousel system, wherein a display panel for displaying a video signal and a signal unit are encoded to form a plurality of blocks. And a visible light communication processing unit that generates a plurality of transmission frames using a plurality of blocks as a backlight control signal, and a backlight that emits light from the back surface of the display panel based on the backlight control signal. Prepare A plurality of transmission frames for one signal unit generated by the visible light communication processing unit have different orders of a plurality of blocks of at least two transmission frames.

The display device of the present disclosure can output a visible light communication signal that can be restored by the receiving device.

Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed description of well-known matters or duplicate description of substantially the same configuration may be omitted. This is to prevent the following description from being unnecessarily redundant and to facilitate understanding by those skilled in the art.

It should be noted that the accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the claimed subject matter by them.

Hereinafter, the twenty-fourth embodiment will be described with reference to FIGS.

[1-1. Configuration of visible light communication system]
FIG. 364 is a schematic diagram of the visible light communication system according to the twenty-fourth embodiment. In FIG. 364, the visible light communication system 1500S includes a display device 1500 and a receiving device 1520.

The display device 1500 is, for example, a display, and displays an image on the display surface 1510. Further, a visible light communication signal is inserted or superimposed on the image displayed on the display surface 1510 as information related to the displayed image.

The receiving device 1520 receives the visible light communication signal output by being displayed on the display surface 1510 of the display device 1500 by capturing an image displayed on the display surface 1510. The receiving device 1520 is configured as, for example, a smartphone having a built-in sequential exposure type image sensor. Thus, the user of the receiving device 1520 can receive information related to the video displayed on the display device 1500.

In the present embodiment, a display is given as an example of display device 1500, but the present invention is not limited to this. The display device 1500 may be a projection type display device such as a projector.

Although a smartphone is given as an example of the receiving device 1520, any electronic device capable of receiving a visible light communication signal may be used. For example, the electronic device is a “JEITA-CP1222 visible light ID system (Visible Light ID System)” defined by the Japan Electronics and Information Technology Industries Association (JEITA).
The receiving device may be a device conforming to the “stem” standard. Further, the electronic device may be a general communication terminal.

Also, "capable of receiving a visible light communication signal" means that a visible light communication signal can be received, and the received visible light communication signal can be decoded to obtain information.

In addition, the communication method of the visible light communication signal is, for example, “JEITA-CP-1223 visible light beacon system (Visible Light Beacon)” defined by JEITA.
System) or a communication method based on IEEE (The Institute)
of Electrical and Electronics Engineers,
Inc. ) Standardized by WPAN (Wireless Personal Area)
It may be a communication system based on IEEE-P802.15, which is a network standard.

In other words, the receiving device 1520 may be any electronic device that can communicate with these communication methods and can further receive a visible light communication signal.

[1-2. Display device configuration]
FIG. 365 is a block diagram of a display device according to the twenty-fourth embodiment. In FIG. 365, the display device 1500 includes a video signal input unit 1501, a video signal processing unit 1502, a display control unit 1503, a display panel 1504, a visible light communication signal input unit 1505, and a visible light communication signal processing unit 1506. And a backlight control unit 1507 and a backlight 1508.

The video signal input unit 1501 relates to a video displayed on the display panel 1504 through an antenna cable, a composite cable, an HDMI (registered trademark) (High-Definition Multimedia Interface:) cable, a PJLink cable, a LAN (Local Area Network) cable, or the like. A video signal is input. The video signal input unit 1501 outputs the input video signal to the video signal processing unit 1502.

The video signal may be a video signal stored in a recording medium.

The video signal processing unit 1502 performs general image processing such as decoding processing on the input video signal. The video signal processing unit 1502 transmits the video signal subjected to the image processing to the display control unit 1503 and the backlight control unit 1507. The video signal includes information about the brightness of the video and the like.

The display control unit 1503 controls the display panel 1504 so as to display an image on the display surface 1510 of the display panel 1504 based on the input image signal. More specifically, the display control unit 1503 controls the liquid crystal aperture of the display panel 1504 based on the video signal input from the video signal processing unit 1502.

The display panel 1504 is, for example, a liquid crystal panel and has a display surface 1510 for displaying an image.

The visible light communication signal input unit 1505 receives the visible light communication signal through a visible light communication signal dedicated cable or a LAN cable.

The visible light communication signal may be a visible light communication signal stored in a recording medium. Furthermore, the visible light communication signal may be superimposed on the video signal.

The visible light communication signal input unit 1505 outputs the input visible light communication signal to the visible light communication signal processing unit 1506.

The visible light communication signal processing unit 1506 performs processing such as encoding the input visible light communication signal by a predetermined encoding method and further determining the transmission order of the visible light communication signals. The visible light communication signal processing unit 1506 converts the encoded visible light communication signal into a backlight control signal. The visible light communication signal processing unit 1506 outputs the generated backlight control signal to the backlight control unit 1507.

The backlight control unit 1507 divides the light emitting surface of the backlight 1508 into a plurality of areas, controls the light emission in each of the plurality of areas, and controls the light emitting surface to have a light-off period at different timings. To do.

The backlight control unit 1507 controls the brightness and timing of the backlight 1508 based on the information regarding the brightness of the video included in the input video signal. Further, the backlight control unit 1507 controls the light emission of the backlight 1508 based on the input backlight control signal.

The backlight 1508 is installed on the back surface of the display panel 1504 and has a light emitting surface that illuminates the display surface 1510 of the display panel 1504 from the back surface. The backlight 1508 emits light from the back surface of the display panel 1504. The viewer can visually recognize the video displayed on the display panel 1504.

In this embodiment, the entire display surface 1510 is a visible light communication region.

FIG. 366 is a diagram for describing a generation example of a visible light communication signal. As shown in FIG. 366, the visible light communication signal input to the visible light communication signal input unit 1505 is composed of a plurality of signal units of a predetermined length. The visible light communication signal processing unit 1506 divides the signal unit into a predetermined number of data. In FIG. 366, one signal unit is composed of four data having the same data length. That is, one signal unit is divided into data 1, data 2, data 3, and data 4. The division of one signal unit is based on the carrier frequency of the visible light communication signal output from the display device 1500, the data length of the signal unit of the visible light communication signal, and further based on the period during which the backlight 1508 does not emit light. Just decide.

Note that, in FIG. 366, it is described that the data length of the data dividing one signal unit is the same, but the data length of the data dividing one signal unit may be different from each other, or one signal unit may be different. The data length of one piece of the data that divides may be different from the data length of the remaining data.

Next, the visible light communication signal processing unit 1506 encodes the divided data, adds a header part to each data, determines the transmission order, and generates a block. Specifically, block 1, block 2, block 3, and block 4 are generated from data 1, data 2, data 3, and data 4. The visible light communication signal processing unit 1506 transmits the generated blocks to the backlight control unit 1507 in the order of block 1, block 2, block 3, and block 4 as a backlight control signal.

The header part of the block is composed of "preamble", "address", and "parity". The preamble is a pattern indicating the start of a block and includes an identifier indicating that the data is a visible light communication signal. For example, a signal out of the coding rule such as four-valued pulse position modulation (4 PPM: 4 Pulse Position Modulation) or i-4PPM (Inverted 4 PPM) is used. Parity is used to detect data errors. The address indicates the transmission order of blocks in the signal unit.

The four blocks generated from one signal unit are called a transmission frame.

[1-3. Configuration of receiving device]
FIG. 367 is a block diagram of a reception device according to the twenty-fourth embodiment. In FIG. 367, the reception device 1520 includes an imaging unit 1521, a captured image generation unit 1522, and a captured image processing unit 1523.

The imaging unit 1521 captures an image displayed in the visible light communication area of the display device 1500. The imaging unit 1521 is, for example, a sequential exposure type image sensor. When the image sensor starts capturing an image, it sequentially performs exposure, and sends exposure data to the captured image generation unit 1522.

The captured image generation unit 1522 temporarily stores the exposure data sent from the imaging unit 1521 in a built-in memory. A captured image is generated based on the exposure data stored in the memory.

The captured image processing unit 1523 restores the visible light communication signal from the captured image generated by the captured image generation unit 1522.

[1-4. Output and reception of visible light communication signals]
Next, a basic operation of receiving the transmission frame output from the visible light communication area of the display device 1500 by the reception device 1520 will be described.

[1-4-1. Image taken for turning on/off the backlight]
FIG. 368 is a diagram illustrating a captured image of the reception device 1520 with respect to turning on/off of the backlight 1508 of the display device 1500.

The imaging unit 1521 is a sequential exposure type image sensor, and performs exposure while scanning each line temporally. In this embodiment, in order to simplify the description, it is assumed that the exposure element of the image sensor has 8 lines. It is assumed that the exposure line is formed in a vertically long strip shape of the receiving device 1520.

As shown in FIG. 368, the backlight 1508 of the display device 1500 is turned on and off as time elapses. The image sensor sequentially exposes from the 1st line to the 8th line, and when the 8th line is sequentially exposed, the captured image generation unit 1522 of the reception device 1520 outputs the captured image based on the exposure data of the 8th line. To generate. Here, the sequential exposure period of the image sensor is referred to as an imaging period, and a captured image generated based on the exposure data sequentially exposed by the image sensor during the imaging period is referred to as a reception frame L. When the exposure of the image sensor is performed up to the 8th line, the process returns to the 1st line and the next exposure is started from the 1st line. The captured image generated next is set as a reception frame L+1. There is a blanking period such as the time for storing the exposure data in the memory from the end of the exposure up to the 8th line until the start of the exposure of the next 1st line, and there is no exposure during this time. .. The reception frame L is the first line, the second line, the fifth line, the sixth line, and the eighth line of the exposure of the image sensor of the reception device 1520 when the backlight 1508 of the display device 1500 is turned on, and each line is bright. The third and fourth lines of exposure of the image sensor of the receiving device 1520 are when the backlight 1508 of the display device 1500 is off, and each line is dark. The visible light communication signal is restored based on the received frame L.

In the reception frame L+1, the first line, the second line, the third line, the seventh line, and the eighth line of the exposure of the image sensor of the reception device 1520 are when the backlight 1508 of the display device 1500 is turned on, and each line is bright. The fourth line, the fifth line, and the sixth line of the exposure of the image sensor of the reception device 1520 are when the backlight 1508 of the display device 1500 is off, and each line is dark. The visible light communication signal is restored based on the received frame L+1.

[1-4-2. Captured image for transmission frame]
FIG. 369 is a schematic diagram illustrating a captured image of the reception device 1520 for a transmission frame of the display device 1500.

As described with reference to FIG. 366, the visible light communication signal is composed of a plurality of signal units, and one signal unit is divided into four data, coded, and divided into four blocks.

In the visible light communication region which is the display surface 1510 of the display device 1500, there may be a period in which it is not possible to determine whether the backlight 1508 is on or off depending on the content of the video signal. The receiving device 1520 may not be able to receive the transmission frame output from the display device 1500 during this period.

Therefore, the transmission frame output from the backlight 1508 of the display device 1500 uses a carousel method in which a transmission frame generated from one signal unit is repeatedly output a plurality of times. In FIG. 369, the display device 1500 uses the visible light communication signal as one signal unit and continuously outputs the transmission frame twice.

As shown in FIG. 369, a transmission frame is output by turning on/off the backlight 1508 of the display device 1500 with the lapse of time. The exposure of the image sensor of the receiving device 1520 is sequentially performed from the first line to the eighth line. When the exposure of the image sensor is performed up to the 8th line, the captured image generation unit 1522 of the reception device 1520 generates a captured image based on the exposure data of the 8th line. The received frame L, which is a captured image, receives the block 1 on the first and second lines of exposure of the image sensor of the receiving device 1520, receives the block 2 on the third and fourth lines, and receives the block 2 on the fifth line. Block 6 is received on the 6th line, and block 4 is received on the 7th and 8th lines. The reception frame L corresponds to the first transmission frame of one signal unit output from the display device 1500.

Further, in FIG. 369, in the received frame L+1 which is a captured image, block 1 is received on the first and second lines of exposure of the image sensor of the receiving device 1520, and block 2 is received on the third and fourth lines. Then, the block 3 is received on the 5th and 6th lines, and the block 4 is received on the 7th and 8th lines. The reception frame L+1 corresponds to the second transmission frame of one signal unit output from the display device 1500.

In this way, by continuously outputting the transmission frames generated from one signal unit by the carousel method, even if a reception failure occurs in the transmission of the first transmission frame, the second transmission frame Blocks that could not be received in the first transmission frame can be received. One signal unit can be restored by receiving all blocks, that is, four blocks, for two transmission frames.

In the case where the transmission frames are continuously output by the carousel method, the display device 1500 outputs the reset signal indicating that the current signal unit is switched to the next signal unit before outputting the transmission frame of the next signal unit. May be output.

This reset signal may be included in the preamble or data of the block of the transmission frame.

[1-5. Issues in output and reception of visible light communication signals]
Next, problems in outputting and receiving a visible light communication signal will be described. FIG. 370 is a diagram illustrating the relationship between the frequency of the transmission clock of the display device 1500 and the frame rate of the imaging unit 1521 of the reception device 1520.

The drive frequency of the liquid crystal panel which is the display panel 1504 of the display device 1500 in this embodiment is 120 Hz.

Depending on the type of liquid crystal panel, some operate at a drive frequency of 60 Hz and some operate at a drive frequency of 240 Hz.

In addition, the frame rate of the image sensor of the imaging unit 1521 of the receiving device 1520 according to the present embodiment operates at 30 fps (frame per second).

At this time, the relationship between the drive frequency of the liquid crystal panel and the frame rate of the image sensor is an integral multiple or an integral fraction. Further, the timing of turning on and off the backlight 1508 of the display device 1500 may be synchronized with the drive frequency of the liquid crystal panel for the purpose of controlling the brightness and moving image resolution in the backlight control unit 1507 of the display device 1500. That is, as shown in FIG. 370, the transmission frame of the display device 1500 is output in synchronization with the drive frequency of the liquid crystal panel. FIG. 370 illustrates a case where a transmission frame generated from one signal unit output from the display device 1500 in such a situation is output three times by the carousel method.

For the first transmission frame output from the display device 1500, the image sensor is exposed during the imaging period of one frame rate. The reception device 1520 generates a reception frame L that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L. Only blocks 2 and 3 in which the received frame L includes all data can be restored as visible light communication signals.

For the second transmission frame output from the display device 1500, the image sensor is exposed during the imaging period of one frame rate. The reception device 1520 generates a reception frame L+1 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L+1. Only blocks 2 and 3 in which the received frame L+1 contains all the data can be restored as a visible light communication signal.

For the third transmission frame output from the display device 1500, the image sensor is exposed during the imaging period of one frame rate. The reception device 1520 generates a reception frame L+2 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L+2. Only blocks 2 and 3 in which the received frame L+2 contains all data can be restored as a visible light communication signal.

As described above, the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is an integral multiple or an integral fraction, and a transmission frame for one signal unit output from the display device 1500 is transmitted to the liquid crystal panel. When outputting in synchronization with the driving frequency, even if the same transmission frame is output three times by the carousel method, only the block 2, block 2, block 3, or block 4 can be restored as a visible light communication signal. , Block 3 only. Blocks 1 and 4 cannot be restored as visible light communication signals.

[1-6. Transmission frame generation method]
In order to solve the above problem, in order to restore all four blocks included in one signal unit output from the display device 1500 as visible light communication signals in the receiving device 1520, a carousel method is applied to one signal unit. The transmission frames output a plurality of times are not generated using the same transmission frame each time, but different transmission frames are generated and output each time. That is, a transmission frame that is output a plurality of times by the carousel method for one signal unit is generated so that the transmission order of blocks of the transmission frame for one signal unit does not become the same every time.

FIG. 371 is a diagram for explaining a first generation example of a transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 371 shows a case where one signal unit output from the display device 1500 is output three times by the carousel method, as in the case of FIG. 370. The difference from FIG. 370 is that the transmission order of the blocks of the three transmission frames output from the display device 1500 is not the same, but is different each time.

The order of the blocks of the first transmission frame output from the display device 1500 is block 1, block 2, block 3, and block 4. For the first transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The reception device 1520 generates a reception frame L that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L. Only blocks 2 and 3 in which the received frame L includes all data can be restored as visible light communication signals.

The order of blocks of the second transmission frame output from the display device 1500 is block 2, block 3, block 4, and block 1. The image sensor is exposed during the image pickup period of one frame rate for the second signal unit output from the display device 1500. The reception device 1520 generates a reception frame L+1 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L+1. Only blocks 3 and 4 in which the received frame L+1 includes all data can be restored as a visible light communication signal.

The order of the blocks of the third transmission frame output from the display device 1500 is block 3, block 4, block 1, and block 2. For the third transmission frame output from the display device 1500, the image sensor is exposed during the imaging period of one frame rate. The reception device 1520 generates a reception frame L+2 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L+2. Only blocks 4 and 1 in which the received frame L+2 contains all the data can be restored as a visible light communication signal.

If the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is an integral multiple or a fraction of an integer, and the transmission frame is output from the display device 1500 in synchronization with the driving frequency of the liquid crystal panel, 1 When the transmission frame for one signal unit is changed three times by changing the transmission order of blocks each time by the carousel method, all of block 1, block 2, block 3 and block 4 of one signal unit are regarded as visible light communication signals. Can be restored.

In the generation example of FIG. 371, since the second and third blocks of the transmission frame output from the display device 1500 are blocks that can be restored as a visible light communication signal, all the blocks are the second and third blocks when output three times. The transmission order of the blocks of the signal unit is changed to output the second.

Note that, in the generation example of FIG. 371, the transmission frame output multiple times by the carousel method for one signal unit is changed so that the transmission order of the blocks of the transmission frame for one signal unit is not the same every time. It is not limited to this. The transmission order of blocks may be changed so that the transmission order of a plurality of transmission frames output to a single signal unit in the carousel method is different from that of two adjacent transmission frames to one signal unit.

Furthermore, the generation example of the transmission frame output from the display device 1500 is not limited to this.

FIG. 372A is a diagram for explaining the second generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment.

FIG. 372A shows the transmission order of the blocks of the transmission frame in ascending order, that is, in the order of block 1, block 2, block 3, block 4, and in descending order, that is, the order of block 4, block 3, block 2, block 1. repeat.

When the reception frame generated by the reception device 1520 is composed of the first half portion or the second half portion of the transmission frame, the transmission frame as in the second generation example is output a plurality of times by the carousel method to block one signal unit. All of 1, block 2, block 3, and block 4 can be restored as a visible light communication signal.

FIG. 372B is a diagram for explaining the third generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372B omits one of the four blocks of the signal unit and changes the transmission order for each transmission frame. The order of the blocks of the first transmission frame output from the display device 1500 is block 1, block 2, block 3, and block 2 excluding block 4. The order of blocks of the second transmission frame output from the display device 1500 is block 3, block 4, block 1, block 3, with block 2 omitted. The order of blocks of the third transmission frame output from the display device 1500 is block 4, block 1, block 2, block 4 with block 3 omitted. By changing the transmission order as described above, all blocks can be transmitted the same number of times.

FIG. 372C is a diagram for explaining the fourth generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372C arranges blocks of signal units in the order of block 1, block 2, block 3, and block 4, and adds one block therein. The order of blocks of the first transmission frame output from the display device 1500 is block 1, block 1, block 2, and block 3 with block 1 added. The order of the blocks of the second transmission frame output from the display device 1500 is to start from block 4 which was not included in the first transmission frame, add block 2, and then block 4, block 1, block 2, block 2 in this order. is there. The order of the blocks of the third transmission frame output from the display device 1500 is block 3, which is not included in the second transmission frame, and block 3, block 4, block 1, and block 2 in that order.

In this way, while outputting the transmission frame as in the fourth generation example a plurality of times by the carousel method, all of the block 1, block 2, block 3, and block 4 of one signal unit are used as visible light communication signals. Can be restored.

FIG. 372D is a diagram for explaining the fifth generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372D randomly changes the order of blocks of the signal unit. The order of the blocks of the first transmission frame output from the display device 1500 is block 1, block 3, block 2, and block 4. The order of blocks in the second transmission frame output from the display device 1500 is block 3, block 1, block 2, and block 4. The order of the blocks of the third transmission frame output from the display device 1500 is block 2, block 3, block 1, and block 4. All of block 1, block 2, block 3, and block 4 of one signal unit are transmitted by visible light communication while randomly changing the order of the blocks of the transmission frame for one signal unit and outputting a plurality of times by the carousel method. It can be restored as a signal.

FIG. 372E is a diagram for explaining the sixth generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. In FIG. 372E, the same block is continuous twice in one transmission frame. The order of the blocks of the first transmission frame output from the display device 1500 is block 1, block 1, block 2, and block 2. The order of blocks of the second transmission frame output from the display device 1500 is block 3, block 3, block 4, and block 4. The order of the blocks of the third transmission frame output from the display device 1500 is block 1, block 1, block 2, and block 2.

[1-7. Operation of visible light communication signal processing unit]
Next, the operation of the visible light communication signal processing unit 1506 of the display device 1500 will be described. FIG. 373 is a flowchart illustrating the operation of the visible light communication signal processing unit 1506 of the display device 1500.

(Step S1501) The visible light communication signal processing unit 1506 determines whether or not a visible light communication signal is input from the visible light communication signal input unit 1505. If it is determined that the visible light communication signal is input (Yes), the process proceeds to step S1502. When it is determined that the visible light communication signal has not been input (No), the process of step S1501 is repeated.

(Step S1502) The input visible light communication signal is composed of a plurality of signal units. The visible light communication signal processing unit 1506 reads one signal unit.

(Step S1503) The visible light communication signal processing unit 1506 divides one read signal unit into a predetermined number of data, encodes each data, adds a header part to each data, and generates a block. ..

(Step S1504) The visible light communication signal processing unit 1506 determines the transmission order of blocks included in each transmission frame of a plurality of transmission frames to be transmitted by the carousel method, based on the generated blocks.

(Step S1505) The visible light communication signal processing unit 1506 generates a plurality of transmission frames and outputs them to the backlight control unit 1507.

(Step S1506) The visible light communication signal processing unit 1506 determines the presence or absence of the remaining signal units. When it is determined that the remaining signal units are “present” (Yes), the process returns to step S1501. When it is determined that the remaining signal units are “none” (No), the processing ends.

[1-8. Effect, etc.]
As described above, the display device according to the present embodiment is a display device capable of outputting a visible light communication signal composed of a plurality of signal units in a carousel system, and a display panel for displaying a video signal and a signal unit. Is encoded, divided into a plurality of blocks, and a visible light communication processing unit that generates a plurality of transmission frames by using the plurality of blocks as a backlight control signal and a display panel from the back based on the backlight control signal. And a backlight that emits light. A plurality of transmission frames for one signal unit generated by the visible light communication processing unit have different orders of a plurality of blocks of at least two transmission frames.

Accordingly, the display device 1500 outputs a plurality of transmission frames having different block transmission orders to one signal unit, so that the reception device 1520 can restore the visible light communication signal.

Further, in the display device according to the present embodiment, the plurality of transmission frames for one signal unit generated by the visible light communication processing unit include the same block in at least two adjacent transmission frames.

Accordingly, the display device 1500 allows the receiving device 1520 to restore the visible light communication signal by including the same block in at least two adjacent transmission frames for one signal unit.

Further, in the display device according to the present embodiment, a plurality of transmission frames for one signal unit generated by the visible light communication processing unit include a plurality of the same blocks in at least one transmission frame, and a plurality of transmission frames. Contains all blocks.

Accordingly, the display device 1500 includes the same block in a plurality of terms in one transmission frame and all blocks in a plurality of transmission frames, so that the reception device 1520 can restore the visible light communication signal.

In addition, in the display device according to the present embodiment, the visible light communication signal processing unit inserts a reset signal between two adjacent signal units.

Accordingly, the display device 1500 can indicate that the current signal unit is switched to the next signal unit.

In the display device 1500 of this embodiment, the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is an integral multiple or a fraction of an integer, and a transmission frame from the display device 1500 has a driving frequency of the liquid crystal panel. This is especially effective when output is synchronized with.

In the present embodiment, the number of transmissions of the transmission frame output from the display device 1500 in the carousel method has been described as three times, but the present invention is not limited to this. The number of transmissions of the transmission frame output by the carousel method may be any number as long as it is a plurality of times.

(Embodiment 25)
Hereinafter, the twenty-fifth embodiment will be described with reference to FIGS. 374 to 376.

[2-1. Configuration of visible light communication system]
The visible light communication system in the present embodiment has the same configuration as visible light communication system 1500S described in the twenty-fourth embodiment. The different points in the visible light communication system according to the present embodiment will be mainly described.

[2-2. Relationship between light and darkness of image and output of visible light communication signal]
The display panel 1504 of the display device 1500 in this embodiment is a liquid crystal panel. When displaying an image, the liquid crystal panel is visually recognized as an image by opening and closing a shutter of liquid crystal on the display surface 1510, or by controlling gradation and controlling the backlight 1508.

Therefore, even if the backlight 1508 is set to be very bright, a dark area is created in the visible light communication area when the video signal is dark. In the dark area of the video signal, the light of the backlight 1508 is blocked by the shutter of the liquid crystal of the display panel 1504. When outputting a visible light communication signal in a dark area, the visible light communication signal may not be restored from the captured image captured by the image capturing unit 1521 of the receiving device 1520.

Therefore, in the present embodiment, one signal unit is used when the ratio of the high-luminance region, which is a region having a predetermined brightness or more, to the visible light communication region, which is the entire display surface 1510 of display device 1500, is small. The visible light communication signal can be restored by outputting the number of times of transmission of the block included in the multiple times. On the contrary, when the ratio of the high-luminance region is large relative to the visible light communication region, the number of times of transmission of the block included in one signal unit is reduced as compared with the case where the ratio of the high-luminance region is small, or The block included in the unit is transmitted once.

[2-3. Operation of visible light communication signal processing unit]
The twenty-fifth embodiment differs from the twenty-fourth embodiment mainly in the operation of visible light communication signal processing section 1506. Next, the operation of the visible light communication signal processing unit 1506 will be described. FIG. 374 is a flowchart illustrating the operation of the visible light communication signal processing unit 1506 of the display device 1500 according to the twenty-fifth embodiment.

The operation from step S1501 to step S1503 is the same as the operation of the twenty-fourth embodiment.

(Step S1511) The visible light communication signal processing unit 1506 detects the high brightness area of the visible light communication area from the video signal input from the video signal processing unit 1502. The visible light communication signal processing unit determines the number of times of transmission of each block of the transmission unit based on the ratio of the high brightness area in the visible light communication area. The method of determining the number of transmissions will be described later.

(Step S1512) The visible light communication signal processing unit 1506 determines the transmission order of blocks based on the number of times of transmission of each block of the signal unit. A method of determining the transmission order of blocks will be described later.

The operations of steps S1505 and S1506 are the same as the operations of the twenty-fourth embodiment.

[2-4. Block transmission count determination method]
Next, a method of determining the block transmission count will be described. FIG. 375 is a diagram illustrating an example of a method of determining the number of times of transmission of an arbitrary block of a transmission frame for one signal unit.

In FIG. 375, the horizontal axis represents the ratio of the high-luminance region in the visible light communication region, and the vertical axis represents the number of times of transmission of any block in the signal unit.

FIG. 375 assumes that if the high-luminance region is about 80% or more in the visible light communication region, the number of times of transmission of any block in the signal unit is once and the receiving device 1520 can restore the visible light communication signal. However, it is assumed that the visible light communication signal can be restored by the receiving device 1520 by increasing the number of times of transmission of an arbitrary block in the signal unit as the ratio of the high brightness area in the visible light communication area decreases. Specifically, if the high-luminance region in the visible light communication region is 90% (point A), the number of times of transmission of any block in the signal unit is 1, and the high-luminance region in the visible light communication region is If it is 50% (point B), the number of times of transmission of an arbitrary block in the signal unit is three times, and if the high-luminance region in the visible light communication region is 10% (point C), the arbitrary block in the signal unit Is transmitted 6 times. In FIG. 375, the number of times of transmission of an arbitrary block in the signal unit is such that the ratio of the high-luminance region in the visible light communication region is from 80% to approximately 15%, and the number of times of transmission of any block in the signal unit is once. It is increasing each.

The ratio of the number of transmissions is not limited to this, and may be changed appropriately.

[2-5. Block transmission order determination method]
Next, a method of determining the transmission order of blocks for one signal unit will be described. FIG. 376 is a diagram for describing an example of generating a transmission frame for one signal unit according to the twenty-fifth embodiment. The drive frequency of the liquid crystal panel which is the display panel 1504 of the display device 1500 in this embodiment is 120 Hz, and the frame rate of the image sensor of the imaging unit 1521 of the reception device 1520 operates at 30 fps. Further, the transmission frame of the display device 1500 is output in synchronization with the driving frequency of the liquid crystal panel. FIG. 376 shows a case where one signal unit of the visible light communication signal output from the display device 1500 is output three times by the carousel method. It is assumed that one signal unit is composed of 6 pieces of data having the same data length and is encoded to generate 6 blocks.

In FIG. 376, the number of transmissions of blocks included in three transmission frames for one signal unit is determined according to the ratio of the high-luminance region in the visible light communication region.

In the first transmission frame output from the display device 1500, the ratio of the high-brightness region is 80%, so the number of times of transmission of any block of the signal unit is one. Therefore, the order of the blocks of the first transmission frame output from the display device 1500 is block 1, block 2, block 3, block 4, block 5, and block 6. For the first transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The reception device 1520 generates a reception frame L that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L. Block 2, block 3, block 4, and block 5 in which the received frame L includes all data can be restored as a visible light communication signal.

Next, in the second transmission frame output from the display device 1500, since the proportion of the high-luminance region is 50%, the number of transmissions of any block of the signal unit is three. Therefore, the order of the blocks of the second transmission frame output from the display device 1500 is to repeat block 1 and block 2 three times in order. For the second transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The reception device 1520 generates a reception frame L+1 that is a captured image based on the exposure data. In the received frame L+1, the block of the area which is not the high brightness area cannot be restored. The reception device 1520 restores the visible light communication signal from the reception frame L+1. Blocks 1 and 2 in which all data is included in the reception frame L+1 can be restored as a visible light communication signal.

Next, in the third transmission frame output from the display device 1500, since the proportion of the high-luminance region is 10%, the number of transmissions of any block of the signal unit is six. As for the order of blocks of the third transmission frame output from the display device 1500, the block 6 is continuously repeated 6 times. For the third transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The reception device 1520 generates a reception frame L+2 that is a captured image based on the exposure data. In the received frame L+2, a block in an area other than the high brightness area cannot be restored. The reception device 1520 restores the visible light communication signal from the reception frame L+2. The block 6 in which all the data is included in the reception frame L+2 can be restored as a visible light communication signal.

When the transmission order of the blocks of the transmission frame for one signal unit is determined based on the ratio of the high-brightness area and is output three times by the carousel method, the block 1, block 2, block 3, block 4 of one signal unit, All of the blocks 5 and 6 can be restored as a visible light communication signal.

[2-6. Effect, etc.]
As described above, in the display device according to the present embodiment, the visible light communication processing unit detects an area having a brightness equal to or higher than a predetermined value on the display panel, and, according to the size of the area, the same block included in the transmission frame. , And generate a plurality of transmission frames for the signal unit.

As a result, the display device 1500 changes the number of block transmissions for one signal unit according to the ratio of the high-luminance region and outputs a plurality of transmission frames, so that the reception device 1520 restores the visible light communication signal. it can.

In this embodiment, the carousel transmission frame is output three times for one signal unit output from the display device 1500, but the present invention is not limited to this. For example, a transmission frame in which transmission frames are output three or more times by the carousel method and the combination of repeating the block 1 and the block 2 three times is changed as the combination of the blocks of the second transmission frame may be used.

In the display device 1500 of this embodiment, the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is an integral multiple or a fraction of an integer, and a transmission frame from the display device 1500 has a driving frequency of the liquid crystal panel. This is especially effective when output is synchronized with.

(Embodiment 26)
Hereinafter, the twenty-sixth embodiment will be described with reference to FIGS. 377 to 380.

[3-1. Configuration of visible light communication system]
The visible light communication system in the present embodiment has the same configuration as visible light communication system 1500S described in the twenty-fourth embodiment. The different points in the visible light communication system according to the present embodiment will be mainly described.

[3-2. Relationship between transmission of visible light communication signal and distance from display device]
The case where the distance between the display device 1500 and the receiving device 1520 is relatively short is compared with the case where the distance is relatively long. When the distance between the display device 1500 and the receiving device 1520 is relatively short, the number of blocks included in the captured image captured by the receiving device 1520 is larger than that when the distance between the display device 1500 and the receiving device 1520 is relatively long. Become.

This is because when the display device 1500 and the receiving device 1520 are relatively close to each other, the captured image that can be generated by the imaging unit 1521 of the receiving device 1520 is relatively large, and the distance between the display device 1500 and the receiving device 1520 is relatively large. This is because the captured image that can be generated by the image capturing unit 1521 of the receiving device 1520 is relatively small when the distance is far from.

Therefore, display device 1500 according to the present embodiment changes the number of times of transmission of an arbitrary block of the transmission frame of one signal unit according to the distance from reception device 1520.

[3-3. Operation of visible light communication signal processing unit]
The difference of the twenty-sixth embodiment from the twenty-fourth embodiment is mainly the operation of visible light communication signal processing section 1506. The operation of the visible light communication signal processing unit 1506 will be described. FIG. 377 is a flowchart illustrating the operation of the visible light communication signal processing unit 1506 of the display device 1500 according to the twenty-sixth embodiment.

The operation from step S1501 to step S1503 is the same as the operation of the twenty-fourth embodiment.

(Step 1401) The visible light communication signal processing unit 1506 determines the number of transmissions of each block of the transmission unit according to the distance from the reception device 1520. The method of determining the number of transmissions will be described later.

(Step 1402) The visible light communication signal processing unit 1506 determines the transmission order of blocks based on the number of times of transmission of each block of the signal unit. The method of determining the transmission order will be described later.

The operations of steps S1505 and S1506 are the same as the operations of the twenty-fourth embodiment.

[3-4. Block transmission count determination method]
Next, a method of determining the block transmission count will be described. FIG. 378 is a diagram illustrating an example of a method of determining the number of times of transmission of an arbitrary block of a transmission frame for one signal unit.

In FIG. 378, the horizontal axis represents the distance between the display device 1500 and the receiving device 1520, and the vertical axis represents the number of times of transmission of an arbitrary block in the signal unit. When the distance is short, the number of transmissions of each block of the signal unit is reduced. In FIG. 378, if the distance is 3 m or less, the number of times of transmission of each block of the signal unit is once.

When the distance is long, the number of times each block of the signal unit is transmitted is increased. In FIG. 378, the number of transmissions of each block of the signal unit is increased once at a rate of 2 m when the distance is 3 m or more.

In addition, you may change this ratio suitably.

[3-5. Block transmission order determination method]
Next, a method of determining the transmission order of blocks for one signal unit will be described. FIG. 379 is a diagram illustrating an example of generation of a transmission frame for one signal unit output from the display device 1500 according to Embodiment 26. FIG. 379 shows the case where the distance is 3 m. The drive frequency of the liquid crystal panel which is the display panel 1504 of the display device 1500 in this embodiment is 120 Hz, and the frame rate of the image sensor of the imaging unit 1521 of the reception device 1520 operates at 30 fps. Further, the transmission frame of the display device 1500 is output in synchronization with the driving frequency of the liquid crystal panel. FIG. 379 shows a case where one signal unit of the visible light communication signal output from the display device 1500 is transmitted four times by the carousel method. It is assumed that one signal unit is composed of four pieces of data having the same data length and is encoded to generate four blocks.

In FIG. 378, when the distance is 3 m, the number of times of transmission of an arbitrary block of one transmission frame of the signal unit is 2 times. Therefore, as shown in FIG. 379, an arbitrary block is transmitted twice in one transmission frame.

The order of the blocks of the first transmission frame output from the display device 1500 is block 1, block 1, block 2, and block 2 so that block 1 and block 2 are output twice respectively. For the first transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The reception device 1520 generates a reception frame L that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L. Blocks 1 and 2 in which all data is included in the reception frame L can be restored as a visible light communication signal.

The order of blocks of the second transmission frame output from the display device 1500 is block 3, block 3, block 4, and block 4 so that block 3 and block 4 are output twice, respectively. For the second transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The reception device 1520 generates a reception frame L+1 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L+1. Blocks 3 and 4 in which all data is included in the reception frame L+1 can be restored as a visible light communication signal.

The order of the blocks of the third transmission frame output from the display device 1500 is block 1, block 1, block 2, and block 2 so that block 1 and block 2 are each output twice. For the third transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The reception device 1520 generates a reception frame L+2 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L+2. Blocks 1 and 2 in which the received frame L+2 contains all the data can be restored as visible light communication signals.

The order of the blocks of the fourth transmission frame output from the display device 1500 is block 3, block 3, block 4, and block 4 so that block 3 and block 4 are output twice, respectively. For the fourth transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The reception device 1520 generates a reception frame L+3 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L+3. Blocks 3 and 4 in which all the data is included in the reception frame L+3 can be restored as a visible light communication signal.

As described above, each reception frame can receive one block out of the output of any block included in the transmission frame twice. That is, two different blocks can be received from each reception frame.

FIG. 380 is a diagram for explaining another example of generation of a transmission frame for one signal unit output from the display device according to the twenty-sixth embodiment. FIG. 380 shows the case where the distance is 8 m. The drive frequency of the liquid crystal panel which is the display panel 1504 of the display device 1500 in this embodiment is 120 Hz, and the frame rate of the image sensor of the imaging unit 1521 of the reception device 1520 operates at 30 fps. Further, the transmission frame of the display device 1500 is output in synchronization with the driving frequency of the liquid crystal panel. FIG. 380 shows a case where one signal unit of the visible light communication signal output from the display device 1500 is transmitted four times by the carousel method. It is assumed that one signal unit is composed of four pieces of data having the same data length and is encoded to generate four blocks.

In FIG. 378, when the distance is 8 m, the number of times of transmission of any block of one transmission frame of the signal unit is four times. Therefore, as shown in FIG. 380, an arbitrary block is transmitted four times in one transmission frame.

Regarding the order of blocks in the first transmission frame output from the display device 1500, block 1 is output four times. For the first transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The reception device 1520 generates a reception frame L that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L. The block 1 in which all the data is included in the reception frame L can be restored as a visible light communication signal.

Regarding the order of blocks in the second transmission frame output from the display device 1500, block 2 is output four times. For the second transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The reception device 1520 generates a reception frame L+1 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L+1. The block 2 including all data in the reception frame L+1 can be restored as a visible light communication signal.

The block order of the third transmission frame output from the display device 1500 is that the block 3 is output four times. For the third transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The reception device 1520 generates a reception frame L+2 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L+2. The block 3 in which all the data is included in the reception frame L+2 can be restored as a visible light communication signal.

Regarding the order of blocks in the fourth transmission frame output from the display device 1500, the block 4 is output four times. For the fourth transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The reception device 1520 generates a reception frame L+3 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L+3. The block 2 including all data in the reception frame L+3 can be restored as a visible light communication signal.

As described above, each reception frame can receive one block out of the arbitrary blocks included in the transmission frame that are output four times. That is, one block can be received from each received frame.

[3-6. Effect, etc.]
As described above, in the present embodiment, the visible light communication processing unit determines the number of the same blocks included in the transmission frame according to the distance between the display device and the receiving device capable of receiving the output visible light communication signal. And generate multiple transmission frames for the signaling unit.

As a result, the display device 1500 changes the number of block transmissions and outputs a plurality of transmission frames according to the distance between the display device 1500 and the reception device 1520, so that the reception device 1520 can restore the visible light communication signal. ..

In the display device 1500 of this embodiment, the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is an integral multiple or a fraction of an integer, and a transmission frame from the display device 1500 has a driving frequency of the liquid crystal panel. This is especially effective when output is synchronized with.

Note that the distance between the display device 1500 and the reception device 1520 can be set in advance on the display device 1500, and is preferably changed as appropriate depending on the application and the installation condition of the display device 1500.

The reception device 1520 may request the display device 1500 to specify the distance via wireless communication such as Wi-Fi (Wireless Fidelity), Bluetooth (registered trademark), or LTE (Long Yrm Evolution).

Further, the distance may be estimated using a sensor or a camera on either the display device 1500 or the reception device 1520.

Further, in the present embodiment, the generated transmission frame is an example, and the present invention is not limited to this.

Further, in the present embodiment, when two blocks are output in a transmission frame a plurality of times, the same number of times is used, but the number of times may not be the same.

(Embodiment 27)
The twenty-seventh embodiment will be described below with reference to FIGS. 381 to 383.

[4-1. Configuration of visible light communication system]
The visible light communication system in the present embodiment has the same configuration as visible light communication system 1500S described in the twenty-fourth embodiment. The different points in the visible light communication system according to the present embodiment will be mainly described.

[4-2. Insert blank]
FIG. 381 is a diagram illustrating an example of generation of a transmission frame for one signal unit according to the twenty-seventh embodiment. The drive frequency of the liquid crystal panel which is the display panel 1504 of the display device 1500 in this embodiment is 120 Hz, and the frame rate of the image sensor of the imaging unit 1521 of the reception device 1520 operates at 30 fps. Further, the transmission frame of the display device 1500 is output in synchronization with the driving frequency of the liquid crystal panel. One signal unit of the visible light communication signal output from the display device 1500 is output four times by the carousel method. One signal unit is composed of four pieces of data having the same data length and is encoded to generate four blocks.

In this embodiment, a blank of the same size as the block is inserted in the transmission frame so that the same block does not have the same position.

In FIG. 381, the first transmission frame output from the display device 1500 is in the order of block 1, block 2, block 3, block 4, and blank. For the first transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The reception device 1520 generates a reception frame L that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L. Only blocks 2 and 3 in which the received frame L includes all data can be restored as visible light communication signals.

The second transmission frame output from the display device 1500 is in the order of block 1, block 2, block 3, block 4, and blank. The image sensor is exposed during the image pickup period of one frame rate for the second signal unit output from the display device 1500. The reception device 1520 generates a reception frame L+1 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L+1. Only blocks 1 and 2 in which the received frame L+1 includes all data can be restored as a visible light communication signal.

The third transmission frame output from the display device 1500 is in the order of block 1, block 2, block 3, block 4, and blank. For the third transmission frame output from the display device 1500, the image sensor is exposed during the imaging period of one frame rate. The reception device 1520 generates a reception frame L+2 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L+2. Only the block 1 in which all the data is included in the reception frame L+2 can be restored as the visible light communication signal.

The fourth transmission frame output from the display device 1500 is in the order of block 1, block 2, block 3, block 4, and blank. For the fourth transmission frame output from the display device 1500, the image sensor is exposed during the imaging period of one frame rate. The reception device 1520 generates a reception frame L+3 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L+3. Only the block 4 in which all the data is included in the reception frame L+3 can be restored as the visible light communication signal.

The blank signal pattern to be inserted may be any pattern as long as it is different from the data included in the signal unit.

As described above, the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is an integral multiple or a fraction of an integer, and the transmission frame is output from the display device 1500 in synchronization with the driving frequency of the liquid crystal panel. In this case, by inserting a blank in the transmission frame for one signal unit, it is possible to avoid synchronizing the timing of turning on and off the backlight 1508 of the display device 1500 with the drive frequency of the liquid crystal panel, and to transmit the same transmission frame for all four times. Can output all of the block 1, block 2, block 3, and block 4 of one signal unit as a visible light communication signal.

In addition, by making the size of the blank to be inserted the same as the size of the block, it is possible to prevent fluctuations in the brightness of the video signal, and it is also effective as a brightness adjustment period.

Although the size of the blank to be inserted is the same as the size of the block, the present invention is not limited to this. The size of the blank to be inserted may be determined so that the timing of turning on and off the backlight 1508 of the display device 1500 is not synchronized with the driving frequency of the liquid crystal panel.

The size of the blank to be inserted may not always be the same.

Furthermore, the generation example of the transmission frame in which the blank is inserted is not limited to this.

FIG. 382A is a diagram for explaining a second generation example of a transmission frame for one signal unit according to the twenty-seventh embodiment.

In FIG. 382A, a blank is inserted at the end of the transmission frame, and the transmission order of the blocks of the transmission frame is made different every time as described in the twenty-fourth embodiment. That is, the order of the blocks of the first transmission frame output from the display device 1500 is block 1, block 2, block 3, block 3, and blank. The second transmission frame output from the display device 1500 is in the order of block 4, block 3, block 2, block 1, and blank. The third transmission frame output from the display device 1500 is in the order of block 2, block 3, block 4, block 1, and blank.

FIG. 382B is a diagram illustrating a third generation example of a transmission frame for one signal unit according to the twenty-seventh embodiment.

FIG. 382B inserts a blank after each block of the transmission frame. That is, the transmission frames output from the display device 1500 are in the order of block 1, blank, block 2, blank, block 3, blank, block 4, and blank. The size of the blank to be inserted is block length×α (decimal number of 0<α≦1), and α is determined so that the timing of turning on and off the backlight 1508 of the display device 1500 is not synchronized with the drive frequency of the liquid crystal panel. To do.

FIG. 382C is a diagram illustrating a fourth generation example of a transmission frame for one signal unit according to the twenty-seventh embodiment.

FIG. 382C inserts a blank after any block of the transmitted frame. That is, the transmission frames output from the display device 1500 are in the order of block 1, blank, block 2, blank, block 3, and block 4.

[4-3. Operation of visible light communication signal processing unit]
The difference of the twenty-seventh embodiment from the twenty-fourth embodiment is mainly the operation of visible light communication signal processing section 1506. Next, the operation of the visible light communication signal processing unit 1506 will be described. FIG. 383 is a flowchart illustrating the operation of the visible light communication signal processing unit 1506 of the display device 1500 according to the twenty-seventh embodiment.

The operation from step S1501 to step S1502 is the same as the operation of the twenty-fourth embodiment.

(Step S1531) Visible light communication signal processing unit 1506 determines the position where the blank of the transmission unit is inserted.

(Step S1532) Visible light communication signal processing unit 1506 determines a blank size.

The operations of steps S1503 to S1506 are the same as the operations of the twenty-fourth embodiment.

[4-4. Effect, etc.]
As described above, in the display device of the present embodiment, the visible light communication processing unit inserts a blank into at least one transmission frame among a plurality of transmission frames for one signal unit.

Accordingly, by inserting a blank in the transmission frame for one signal unit, it is possible to avoid the timing of turning on and off the backlight 1508 of the display device 1500 from synchronizing with the drive frequency of the liquid crystal panel, and the receiving device 1520 uses visible light communication. The signal can be restored.

In the display device 1500 of this embodiment, the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is an integral multiple or a fraction of an integer, and a transmission frame from the display device 1500 has a driving frequency of the liquid crystal panel. This is especially effective when output is synchronized with.

(Other embodiments)
As described above, Embodiments 24 to 27 have been described as examples of the technology of the present disclosure. The technology of the present disclosure is not limited to this, and can be applied to embodiments in which changes, replacements, additions, omissions, etc. have been made. Further, it is also possible to combine the constituent elements described in the above-mentioned Embodiments 24 to 27 to form a new embodiment.

Note that the display device of the present disclosure has shown an example of generation of a transmission frame when the transmission frame is output in synchronization with the driving frequency of the liquid crystal panel, but the present invention is not limited to this.

For example, even when the transmission frame is output from the display device without synchronizing with the drive frequency of the liquid crystal panel, the present embodiment is effective when the carrier frequency at which the transmission frame is output is an integral multiple of the frequency of the image sensor. Is.

Further, although the case where the display panel of the display device is a liquid crystal panel has been described, the present invention is not limited to this.

For example, even if the display device is a signboard in which the image film is illuminated from behind by an illumination such as an LED, the carrier frequency of the transmission frame output from the display device is an integral multiple of the frequency of the image sensor of the receiving device. In this case, this embodiment is effective.

The display device according to the present disclosure is applied to a display device capable of outputting a visible light communication signal, for example, a device such as a television, a personal computer, or a tablet terminal at home, a signage terminal at an outing, an information terminal, or an information display device. It is possible.

(Summary)
A display device according to a first aspect of the present disclosure is a display device capable of outputting a visible light communication signal composed of a plurality of signal units by a carousel method, the display panel displaying a video signal, and the signal unit. And a visible light communication processing unit that divides into a plurality of blocks, generates a plurality of transmission frames using the plurality of blocks as a backlight control signal, and the display panel based on the backlight control signal. And a backlight that emits light from the back side, and the plurality of transmission frames for one of the signal units generated by the visible light communication processing unit are different in the order of the plurality of blocks of at least two of the transmission frames. ..

In the display device according to the second aspect of the present disclosure, the plurality of transmission frames for the one signal unit generated by the visible light communication processing unit include the same block in at least two adjacent transmission frames. The display device according to the first aspect.

In the display device according to the third aspect of the present disclosure, the plurality of transmission frames for the one signal unit generated by the visible light communication processing unit include a plurality of identical blocks in at least one of the transmission frames. It is a display device according to the first aspect, which includes all of the plurality of blocks in the plurality of transmission frames.

In the display device according to the fourth aspect of the present disclosure, the visible light communication processing unit detects an area having a brightness equal to or higher than a predetermined value on the display panel, and includes the area in the transmission frame according to the size of the area. It is a display device according to a third aspect, which determines the number of identical blocks and generates a plurality of the transmission frames for the signal unit.

In the display device according to the fifth aspect of the present disclosure, the visible light communication processing unit determines whether the transmission frame is displayed according to a distance between the display device and a receiving device that can receive the output visible light communication signal. It is a display device according to a third aspect, which determines the number of identical blocks to be included and generates a plurality of the transmission frames for the signal unit.

A display device according to a sixth aspect of the present disclosure is the display device according to the first aspect, in which the visible light communication signal processing unit inserts a reset signal between two adjacent signal units.

In the display device according to a seventh aspect of the present disclosure, the visible light communication processing unit inserts a blank into at least one of the transmission frames of the plurality of transmission frames for one of the signal units. It is a display device according to.

A display method according to an eighth aspect of the present disclosure is a display method capable of outputting a visible light communication signal composed of a plurality of signal units by a carousel method, wherein the signal unit is encoded and divided into a plurality of blocks. Then, a first step of generating a plurality of transmission frames to be output by a carousel method using the plurality of blocks and outputting the frames as a backlight control signal, and a first step of controlling the backlight based on the backlight control signal. The plurality of transmission frames for one of the signal units generated in the second step and the first step is a display method in which the order of the plurality of blocks of at least two of the transmission frames is different.

(Embodiment 28)
FIG. 384 is a diagram for explaining switching control of visible light communication (VLC) when the transmission device is a moving image display device such as a television.

Specifically, (a) of FIG. 384 is a diagram showing a moving image including a plurality of pictures, and (b) of FIG. 384 is a backlight of the moving image display device when visible light communication is OFF. FIG. 384 is a diagram illustrating ON/OFF control of the backlight of the moving image display device when visible light communication is ON.

As shown in (a) of FIG. 384, when a moving image 1600 composed of a plurality of pictures P1601, P1602, P1603, P1604, P1605, P1606,... Is reproduced, a plurality of pictures P1601, P1602, P1603, P1604, P1605, P1606,... Are displayed on the moving image display device at times t1601, t1603, t1605, t1607, t1609, t1611,. The time t1 is a display start time of the moving image 1600, and may be an absolute time or a time designated by the user. Further, times t1603, t1605, t1607, t1609, t1601,... Are times at predetermined time intervals Δt1600 from time t1. That is, the times t1603, t1605, t1607, t1609, t1611,... Are times determined in a constant cycle (predetermined time interval Δt1600).

When reproducing such a moving image 1600, in particular, in a liquid crystal display, in order to reduce the blurring and reproduction of the moving image 1600, there is a control for inserting a black picture between adjacent pictures. In the case of such a moving image display device, as shown in (b) of FIG. 384, times t1601, t1603, t1605, t1607 at which a plurality of pictures P1601, P1602, P1603, P1604, P1605, P1606,... Are displayed. , T1609, t1611,... At times t1602, t1604, t1606, t1608, t1610, t1612,..., Control for turning off the backlight of the moving image display device in order to insert a black picture. Is going. That is, at times t1601, t1603, t1605, t1607, t1609, t1611,... When a plurality of pictures P1601, P1602, P1603, P1604, P1605, P1606,... At t1602, t1604, t1606, t1608, t1610, t1612,..., The control for turning off the backlight is performed.

However, if the backlight is turned off during the visible light communication, the communication will be interrupted during the period when the backlight is off. Therefore, as shown in (c) of FIG. 384, when visible light communication is performed (that is, when VLC is ON), the backlight is kept ON even during reproduction of the moving image 1600. Take control. Thus, in the transmitting device in this case, when visible light communication is being performed, the backlight is continuously turned on as shown in (c) of FIG. 384, and when visible light communication is not being performed, The control for repeating ON/OFF of the backlight is switched as in (b) of 384. Accordingly, when visible light communication is performed, it is possible to suppress communication interruption, and when the visible light communication is not performed, it is possible to reduce blurring and reproduction of the moving image 1600.

(Embodiment 29)
In this embodiment, a visible light communication protocol transmission method will be described.

385 and 386 are diagrams showing a procedure for transmitting the logical data (eg, ID) used in the application layer by visible light communication.

First, the logical data error correction code addition unit 1701 adds a logical data correction code 1712, which is an error correction code, to the logical data 1711 used in the application layer.

Next, the logical data division unit 1702 generates a plurality of divided logical data 1713 by dividing the logical data 1711 and the logical data correction code 1712 into a data size that can be transmitted. Further, the logical data division unit 1702 gives a division type 1714 and an address 1715 to each divided logical data 1713.

The data modulator 1703 converts the data generated by the logical data divider 1702 into a data string that can be transmitted, thereby generating physical data 1716 for transmission.

The logical data error correction code addition unit 1701 uses encoding such as CRC or Reed-Solomon code according to the size of logical data or the state of the transmission path. Further, the logical data correction code 1712 may be inserted at a specific position of the logical data 1711 when the logical data correction code 1712 is added at the beginning or the end of the logical data 1711.

The logical data division unit 1702 can determine the limit distance and the reception speed that can be received by visible light communication by changing the size after division. Further, the logical data division unit 1702 can improve the resistance to burst errors in addition to the error resistance by the logical data correction code 1712 and the physical data correction code 1717 by changing the division method, and at the same time concealment during data decoding. It is possible to improve the property.

Note that the data modulator 1703 has device characteristics of the visible light communication transmitter (for example, in the case of lighting, it is necessary to keep brightness as much as possible, in the case of a display, regardless of modulation type such as PPM modulation or Manchester modulation). Brightness control or modulation rate control can be realized by varying the corresponding quantization number or sample value per bit of the logical data depending on the necessity of coexisting with a moving image or a still image. For example, the data modulation unit 1703 uses a binary value such as “1” of physical data when light is emitted, “0” of physical data when light is not emitted, and a brightness at light emission of 100. By switching between “2” when the brightness is 50%, “1” when the brightness is 50% when the light is emitted, and “0” when the brightness when the light is emitted is 0%. Brightness control can be realized. Further, the data modulation unit 1703 sets the physical data “1” when light is emitted, the physical data “0” when light is not emitted, and sets the logical data “01” to the physical data “0100”. It is possible to control the average brightness in the physical data transmission size by switching between the modulation to “1” and the modulation to “11001111”.

Next, the physical data error correction code addition unit 1704 adds a physical data correction code 1717, which is an error correction code, to the physical data 1716 generated by the data modulation unit 1703.

Next, the physical data header insertion unit 1705 adds a header 1718 indicating the start position of the physical data 1716 to the physical data 1716. The obtained data is transmitted as visible light communication data by the visible light communication transmitting unit.

The physical data error correction code addition unit 1704 uses encoding such as CRC or Reed-Solomon code according to the size of the physical data 1716 or the state of the transmission path. Further, the physical data correction code 1717 may be inserted at a specific position of the physical data 1716 when it is added to the physical data 1716 or when it is added at the end.

The physical data header insertion unit 1705 inserts, as a header, preamble data that allows the visible light communication reception unit to identify the beginning of physical data of visible light communication data. The preamble data to be inserted is a data string that does not appear in the combined data of the physical data 1716 to be transmitted and the physical data correction code 1717. The physical data header insertion unit 1705 can control the flicker condition and the required brightness of the visible light communication transmission unit by changing the size of the preamble data and the preamble data string. Furthermore, the preamble data can also be used for identifying the device type in the visible light communication receiving unit. For example, the flicker can be reduced by setting the preamble data so that the difference between the brightness during transmission of the data including the physical data 1716 and the physical data correction code 1717 and the brightness during transmission of the preamble data is minimized. Further, by reducing the light emitting period during the preamble, the brightness of the preamble data can be adjusted to be small.

In addition, a general interleaving method can be used for the division in the logical data division unit 1702. FIG. 387 is a diagram for explaining the division processing by the logical data division unit 1702.

FIG. 387 is a diagram illustrating an example of division when the logical data is “010011000111010” and the number of divisions is 3. For example, as shown in (a) of FIG. 387, the logical data division unit 1702 divides the logical data 1711 and the logical data correction code 1712 into 5 BIT from the beginning to generate a plurality of divided logical data 1713. Alternatively, as shown in (b) of FIG. 387, the logical data division unit 1702 allocates the logical data 1711 and the logical data correction code 1712 to each divided logical data 1713 by 1 BIT from the beginning, thereby dividing the logical data into plural divisions. The logical data 1713 is generated.

Further, as shown in FIG. 388, the logical data division unit 1702 defines the number of SKIPs required to divide the logical data, and sets the logical data 1711 and the logical data correction code 1712 from the beginning by BIT for each SKIP number. A plurality of pieces of divided logical data 1713 may be generated by allocating to the data 1713.

In this case, the logical data dividing unit 1702 can give confidentiality by arbitrarily designating the SKIP number so that the visible light communication receiving unit that does not know the set SKIP number cannot restore the logical data. Becomes Note that the logical data dividing unit 1702 may perform division using a hash value output by applying a hash function based on an arbitrary value, or an arbitrary value that uniquely identifies the division designation BIT by an arbitrary value. An arithmetic expression may be used.

Furthermore, the logical data division unit 1702 can secure confidentiality so that data can be received only at a specific time by using time as an arbitrary value. Further, the logical data division unit 1702 can be expanded to a service in which only a specific channel can be received by using the TV channel number as an arbitrary value. Further, the logical data division unit 1702 can use the data only at the place by using the numerical value about the place as an arbitrary value.

The present invention may include the following aspects.

The transmitter includes a visible light transmitter and a human sensor. The human sensor unit detects the presence of a person and starts transmission. Transmission is performed in the direction in which a human sensor detects that a person is present. Thereby, power consumption can be suppressed.

The receiver receives the ID of the transmitter, adds address information or current position information, and transmits it to the server. The server sends a code to the receiver to optimize the settings for the received address or location. The receiver displays the code received from the server on the screen and prompts the user to set the transmitter. Thereby, for example, the settings of the rice cooker and the washing machine can be set to the optimum settings for the water quality in the living area.

The receiver changes the exposure time setting for each imaging frame, receives a visible light signal in a frame with a short exposure time, and receives other signals or markers such as a two-dimensional bar code in a frame with a long exposure time. Or perform object recognition or image recognition. As a result, it is possible to simultaneously receive visible light and receive other signals or markers.

The receiver changes the exposure time little by little for each frame to perform imaging. As a result, even if the modulation frequency of the transmission signal is unknown, the image of any frame is captured with an appropriate exposure time, and the signal can be demodulated. Further, by capturing the same signal for a plurality of exposure times, the received signal can be demodulated more efficiently.

When the receiver receives an ID within a predetermined range, the receiver does not inquire of the server and passes the received ID as it is to another processing unit. As a result, a quick response can be obtained. Further, the processing can be performed even when the server cannot be connected. In addition, the operation can be confirmed before setting the content in the server.

The transmitter expresses a transmission signal by amplitude modulation. At this time, the plurality of symbols representing different signals have the same duration in either the low brightness state or the high brightness state. As a result, the signal can be expressed even with a low control clock.

The transmitter registers the transmission ID and the content in the server when it is activated. As a result, desired content can be transmitted from the server to the receiver.

It is assumed that a part of the ID can be freely set by the transmitter. As a result, the code indicating the status of the transmitter can be included in the ID. The receiver or the server may change the display content depending on this part, or may ignore it.

(Multi-valued amplitude pulse signal)
389, 390, and 391 are diagrams showing an example of a transmission signal in the present embodiment.

By giving meaning to the amplitude of the pulse, more information can be expressed per unit time. For example, if the amplitude is classified into three levels, as shown in FIG. 389, three values can be expressed by the transmission time of two slots while keeping the average luminance at 50%. However, if (c) of FIG. 389 is continuously transmitted, there is no change in luminance, so the presence of a signal is difficult to understand. Also, the three values are a little difficult to handle in digital processing.

Therefore, by using the four types of symbols in (a) to (d) of FIG. 390, it is possible to express four values by the average transmission time of 3 slots while keeping the average luminance at 50%. Although the transmission time differs depending on the symbol, the end point of the symbol can be recognized by setting the last state of the symbol to the low brightness state. The same effect can be obtained by switching the high brightness state and the low brightness state. (E) of FIG. 390 is not suitable because it cannot be distinguished from (a) being transmitted twice. In (f) and (g) of FIG. 390, since the intermediate luminance is continuous, it is somewhat difficult to recognize, but it can be used.

Consider using the patterns (a) and (b) of FIG. 391 as a header. Since these patterns strongly have a specific frequency component in frequency analysis, by using these patterns as headers, signal detection can be performed by frequency analysis.

As in (c) of FIG. 391, a transmission packet is configured using the patterns of (a) and (b). Data can be separated by using a pattern of a specific length as the header of the entire packet and using patterns of different lengths as separators. In addition, by including this pattern in the middle, signal detection can be facilitated. As a result, even if one packet is longer than the image capturing time of one frame image, the receiver can combine the data and decode the data. Further, this makes it possible to make the packet length variable by adjusting the number of separators. The length of the entire packet may be expressed by the length of the pattern of the packet header. Further, by using the separator as the packet header and the separator length as the data address, the receiver can combine the partially received data.

The transmitter repeatedly transmits the packet thus configured. The contents of packets 1 to 4 in (c) of FIG. 391 may be all the same or may be combined as different data on the receiving side.

(Embodiment 30)
In the present embodiment, each application example using the receiver such as the smartphone in each of the above embodiments and the transmitter that transmits information as the blinking pattern of the LED or the organic EL will be described.

FIG. 392A is a diagram for describing a transmitter in this embodiment.

The transmitter according to the present embodiment is configured as a backlight of a liquid crystal display, for example, and includes a blue LED 2303 and a phosphor 2310 including a green fluorescent component 2304 and a red fluorescent component 2305.

The blue LED 2303 emits blue (B) light. The phosphor 2310 emits yellow (Y) light when receiving the blue light emitted from the blue LED 2303 as excitation light. That is, the phosphor 2310 emits yellow light. More specifically, the fluorescent substance 2130 is composed of the green fluorescent component 2304 and the red fluorescent component 2305, and thus emits yellow light by the emission of these fluorescent components. Of these two fluorescent components, the green fluorescent component 2304 emits green light when receiving the blue light emitted from the blue LED 2303 as excitation light. That is, the green fluorescent component 2304 emits green (G) light. Of the above two fluorescent components, the red fluorescent component 2305 emits red light when receiving the blue light emitted from the blue LED 2303 as excitation light. That is, the red fluorescent component 2305 emits red (R) light. As a result, RGB light or Y(RG)B light is emitted, so that the transmitter outputs white light as a backlight.

This transmitter transmits a visible light signal of white light by changing the brightness of the blue LED 2303 in the same manner as in the above embodiments. At this time, the visible light signal having a predetermined carrier frequency is output by changing the brightness of the white light.

Here, the barcode reader irradiates the barcode with red laser light, and reads the barcode based on a change in the brightness of the red laser light reflected from the barcode. The reading frequency of the barcode in the red laser light may be the same as or close to the carrier frequency of the visible light signal output from a general transmitter currently put into practical use. Therefore, in such a case, when the bar code reader tries to read the bar code illuminated by the white light which is the visible light signal from the general transmitter, the brightness of the red light contained in the white light is read. Depending on the change, the reading may fail. That is, a bar code reading error occurs due to interference between the carrier frequency of the visible light signal (especially red light) and the bar code reading frequency.

Therefore, in the present embodiment, the red fluorescent component 2305 uses a fluorescent material whose afterglow duration is longer than that of the green fluorescent component 2304. That is, the red fluorescent component 2305 in the present embodiment changes in luminance at a frequency sufficiently lower than the frequency of the luminance change of the blue LED 2303 and the green fluorescent component 2304. In other words, the red fluorescent component 2305 dulls the frequency of the luminance change of red contained in the visible light signal.

FIG. 392B is a diagram showing changes in luminance of RGB.

The blue light from the blue LED 2303 is included in the visible light signal and output, as shown in (a) of FIG. 392B. As shown in (b) of FIG. 392B, the green fluorescent component 2304 emits green light when receiving blue light from the blue LED 2303. The persistence time of the afterglow in the green fluorescent component 2304 is short. Therefore, when the brightness of the blue LED 2303 changes, the green fluorescent component 2304 emits green light whose brightness changes at a frequency substantially the same as the frequency of the brightness change of the blue LED 2303 (that is, the carrier frequency of the visible light signal). ..

As shown in (c) of FIG. 392B, the red fluorescent component 2305 emits red light when it receives blue light from the blue LED 2303. The persistence time of the afterglow in the red fluorescent component 2305 is long. Therefore, when the brightness of the blue LED 2303 changes, the red fluorescent component 2305 emits red light whose brightness changes at a frequency lower than the frequency of the brightness change of the blue LED 2303 (that is, the carrier frequency of the visible light signal). ..

FIG. 393 is a diagram showing afterglow characteristics of the green fluorescent component 2304 and the red fluorescent component 2305 in the present embodiment.

For example, when the blue LED 2303 is turned on without changing the luminance, the green fluorescent component 2304 emits the green light with the intensity I=I0 without changing the luminance (that is, the light with the frequency f=0 for changing the luminance). .. Further, even if the blue LED 2303 changes in luminance at a low frequency, the green fluorescent component 2304 emits green light having an intensity I=I0, which changes in luminance at a frequency f substantially equal to the low frequency. However, when the blue LED 2303 changes in luminance at a high frequency, the intensity I of the green light emitted from the green fluorescent component 2304, which changes in luminance at a frequency f substantially equal to the high frequency, is affected by the afterglow in the green fluorescent component 2304. Is smaller than the intensity I0. As a result, the intensity I of the green light emitted from the green fluorescent component 2304 is maintained at I=I0 when the frequency f of the brightness change of the light is less than the threshold fb, as shown by the dotted line in FIG. However, when the frequency f exceeds the threshold value fb and becomes high, the frequency f becomes gradually small.

Further, the afterglow duration of the red fluorescent component 2305 in the present embodiment is longer than the afterglow duration of the green fluorescent component 2304. Therefore, as shown by the solid line in FIG. 393, the intensity I of the red light emitted from the red fluorescent component 2305 is I=until the frequency f of the brightness change of the light is less than the threshold fa lower than the threshold fb. It is kept at I0, but becomes smaller as the frequency f exceeds the threshold fb and becomes higher. In other words, the red light emitted from the red fluorescent component 2305 does not exist in the high frequency region of the frequency band of the green light emitted from the green fluorescent component 2304, but exists only in the low frequency region.

More specifically, for the red fluorescent component 2305 in the present embodiment, a fluorescent material in which the intensity I of red light emitted at the same frequency f as the carrier frequency f1 of the visible light signal is I=I1 is used. Be done. The carrier frequency f1 is a carrier frequency of luminance change by the blue LED 2303 provided in the transmitter. The intensity I1 described above is 1/3 of the intensity I0 or −10 dB of the intensity I0. For example, the carrier frequency f1 is 10 kHz, or 5 to 100 kHz.

That is, the transmitter according to the present embodiment is a transmitter that transmits a visible light signal, and receives a blue LED that emits blue light whose brightness changes and is included in the visible light signal, and the blue light. Thus, a green fluorescent component that emits green light as light included in the visible light signal and a red fluorescent component that emits red light as light included in the visible light signal by receiving the blue light are provided. The persistence time of the afterglow in the red fluorescent component is longer than the persistence time of the afterglow in the green fluorescent component. The green fluorescent component and the red fluorescent component may be included in a single phosphor that receives the blue light and emits yellow light as light included in the visible light signal. Alternatively, the green fluorescent component may be included in the green fluorescent substance, and the red fluorescent component may be included in the red fluorescent substance that is separate from the green fluorescent substance.

As a result, the persistence time of the afterglow in the red fluorescent component is long, so that the luminance of the red light can be changed at a frequency lower than the frequency in the luminance change of the blue and green light. Therefore, even if the frequency in the luminance change of the blue and green light included in the visible light signal of white light is the same as or close to the reading frequency of the barcode in the red laser light, it is included in the visible light signal of white light. The frequency of red light can be significantly different from the barcode reading frequency. As a result, it is possible to suppress the occurrence of barcode reading errors.

Here, the red fluorescent component may emit red light whose luminance changes at a frequency lower than the frequency of the luminance change of the light emitted from the blue LED.

Further, the red fluorescent component may include a red fluorescent material that emits red light by receiving blue light, and a low-pass filter that transmits only light in a predetermined frequency band. For example, the low pass filter transmits only the light in the low frequency band of the blue light emitted from the blue LED and applies the light to the red fluorescent material. The red fluorescent material may have the same afterglow characteristics as the green fluorescent component. Alternatively, the low-pass filter transmits only the light in the low frequency band of the red light emitted from the red fluorescent material by hitting the red fluorescent material with the blue light emitted from the blue LED. Let Even when such a low-pass filter is used, it is possible to suppress the occurrence of a barcode reading error as in the above.

Further, the red fluorescent component may be made of a fluorescent material having a predetermined afterglow characteristic. For example, the predetermined afterglow characteristics are (a) the intensity of the red light when the frequency f of the luminance change of the red light emitted from the red fluorescent component is 0, and (b) When the carrier frequency in the luminance change of the light emitted from the blue LED is f1, and the frequency f of the red light is f=f1, the intensity of the red light is 1/3 or less of the I0. , Or -10 dB or less.

As a result, the frequency of the red light included in the visible light signal can be made to greatly differ from the barcode reading frequency. As a result, it is possible to reliably suppress the occurrence of barcode reading errors.

Further, the carrier frequency f1 may be approximately 10 kHz.

As a result, since the carrier frequency used for transmitting the visible light signal, which is currently in practical use, is 9.6 kHz, it is possible to effectively generate the barcode reading error in the practically transmitting visible light signal. Can be suppressed to.

Further, the carrier frequency f1 may be approximately 5 to 100 kHz.

It is expected that carrier frequencies such as 20 kHz, 40 kHz, 80 kHz, or 100 kHz will be used in future visible light communication due to advances in image sensors (imaging elements) of receivers that receive visible light signals. Therefore, by setting the above-mentioned carrier frequency f1 to approximately 5 to 100 kHz, it is possible to effectively suppress the occurrence of a barcode reading error even in future visible light communication.

In the present embodiment, regardless of whether the green fluorescent component and the red fluorescent component are contained in a single phosphor, or whether each of these two fluorescent components is contained in a separate phosphor, Each of the above effects can be achieved. That is, even when a single phosphor is used, the afterglow characteristics, that is, the frequency characteristics of the red light and the green light emitted from the phosphor are different. Therefore, the effects described above can also be obtained by using a single phosphor that is inferior in the afterglow characteristic or frequency characteristic of red light and is superior in the afterglow characteristic or frequency characteristic of green light. It should be noted that poor afterglow characteristics or frequency characteristics means that the duration of afterglow is long or that the intensity of light in the high frequency band is weak. Is short or the intensity of light in the high frequency band is high.

Here, in the examples shown in FIGS. 392A to 393, the occurrence of the barcode reading error is suppressed by blunting the frequency of the red luminance change included in the visible light signal, but the carrier frequency of the visible light signal is reduced. By increasing the height, the occurrence of the read error may be suppressed.

FIG. 394 is a diagram for explaining a new problem that occurs in order to suppress the occurrence of a barcode reading error.

As shown in FIG. 394, when the carrier frequency fc of the visible light signal is about 10 kHz, the reading frequency of the red laser light used for reading the barcode is also about 10 to 20 kHz, so that the frequencies interfere with each other, Barcode reading error occurs.

Therefore, by raising the carrier frequency fc of the visible light signal from about 10 kHz to, for example, 40 kHz, it is possible to suppress the occurrence of barcode reading errors.

However, if the carrier frequency fc of the visible light signal is about 40 kHz, the sampling frequency fs for the receiver to sample the visible light signal by photographing needs to be 80 kHz or more.

That is, since the sampling frequency fs required in the receiver is high, a new problem arises that the processing load on the receiver increases. Therefore, in order to solve this new problem, the receiver in the present embodiment performs downsampling.

FIG. 395 is a diagram for describing downsampling performed by the receiver in this embodiment.

The transmitter 2301 in this embodiment is configured as, for example, a liquid crystal display, a digital signage, or a lighting device. Then, the transmitter 2301 outputs the frequency-modulated visible light signal. At this time, the transmitter 2301 switches the carrier frequency fc of the visible light signal between, for example, 40 kHz and 45 kHz.

The receiver 2302 in the present embodiment captures an image of the transmitter 2301 at a frame rate of 30 fps, for example. At this time, similarly to the receivers in the above-described embodiments, the receiver 2302 performs shooting with a short exposure time so that a bright line is generated in each image (specifically, each frame) obtained by shooting. Further, the image sensor used for the image capturing of the receiver 2302 has, for example, 1000 exposure lines. Therefore, in the one-frame shooting, the visible light signal is sampled by starting exposure at 1000 different exposure lines. As a result, in one second, 30 fps×1000=30000 samplings (30 ks/sec) are performed. In other words, the sampling frequency fs of the visible light signal is 30 kHz.

According to a general sampling theorem, at the sampling frequency fs=30 kHz, only a visible light signal having a carrier frequency of 15 kHz or less can be demodulated.

However, the receiver 2302 in this embodiment downsamples the visible light signal having the sampling frequency fs=30 kHz and the carrier frequency fc=40 kHz or 45 kHz. An alias occurs in the frame due to this downsampling, but the receiver 2302 in the present embodiment estimates the carrier frequency fc of the visible light signal by observing and analyzing the alias.

FIG. 396 is a flowchart showing the processing operation of the receiver 2302 in this embodiment.

First, the receiver 2302 captures an image of a subject to perform downsampling at a sampling frequency fs=30 kHz on a visible light signal at a carrier frequency fc=40 kHz or 45 kHz (step S2310).

Next, the receiver 2302 observes and analyzes the alias generated in the frame obtained by the downsampling (step S2311). Thereby, the receiver 2302 specifies the frequency of the alias as 5.1 kHz or 5.5 kHz, for example.

Then, the receiver 2302 estimates the carrier frequency fc of the visible light signal based on the frequency of the identified alias (step S2311). That is, the receiver 2302 restores the original frequency from the alias. As a result, the receiver 2302 estimates the carrier frequency fc of the visible light signal as, for example, 40 kHz or 45 kHz.

As described above, the receiver 2302 according to the present embodiment can appropriately receive the visible light signal having the high carrier frequency by performing the downsampling and the frequency restoration based on the alias. For example, the receiver 2302 can receive a visible light signal having a carrier frequency of 30 kHz to 60 kHz even if the sampling frequency is fs=30 kHz. Therefore, the carrier frequency of the visible light signal can be increased from the frequency (about 10 kHz) currently in practical use to 30 kHz to 60 kHz. As a result, the carrier frequency of the visible light signal and the reading frequency (10 to 20 kHz) of the barcode can be greatly different, and the interference of the frequencies can be suppressed. As a result, it is possible to suppress the occurrence of barcode reading errors.

The receiving method in the present embodiment as described above is a receiving method for acquiring information from a subject, and corresponds to a plurality of exposure lines included in the image sensor in a frame obtained by photographing the subject by the image sensor. An exposure time setting step of setting an exposure time of the image sensor so that a plurality of bright lines are generated according to a change in the brightness of the subject, and the plurality of exposure lines included in the image sensor are sequentially exposed at different times. By repeating the start of the step, the image sensor captures the subject whose brightness changes at a predetermined frame rate and at the set exposure time, and for each frame obtained by the capturing. And an information acquisition step of acquiring information by demodulating the data specified by the patterns of the plurality of bright lines included in the frame. Then, in the photographing step, the exposure of each of the plurality of exposure lines is sequentially repeated at different times, so that the sampling frequency is lower than the carrier frequency of the visible light signal transmitted by the change in the luminance of the subject. , The visible light signal is down-sampled, and in the information acquisition step, for each frame obtained by the photographing, the frequency of the alias specified by the pattern of the plurality of bright lines included in the frame is specified and specified. The information is obtained by estimating the frequency of the visible light signal from the alias frequency and demodulating the estimated frequency of the visible light signal.

In such a receiving method, the visible light signal having a high carrier frequency can be properly received by performing the downsampling and the frequency restoration based on the alias.

Further, in the downsampling, a visible light signal having a carrier frequency higher than 30 kHz may be downsampled. Thereby, the interference between the carrier frequency of the visible light signal and the barcode reading frequency (10 to 20 kHz) can be avoided, and the barcode reading error can be more effectively suppressed.

(Embodiment 31)
FIG. 397 is a diagram illustrating a processing operation of the reception device (imaging device). Specifically, FIG. 397 is a diagram for describing an example of a process of switching between the normal imaging mode and the macro imaging mode when receiving visible light communication.

Here, the receiving device 1610 receives visible light emitted by a transmitting device including a plurality of light sources (four light sources in FIG. 397).

First, when the receiving apparatus 1610 transitions to the visible light communication mode, the receiving apparatus 1610 activates the imaging unit in the normal imaging mode (S1601). Note that the reception device 1610 displays a frame 1611 for capturing an image of the light source on the screen when transitioning to the mode for performing visible light communication.

After a predetermined time, the reception device 1610 switches the imaging mode of the imaging unit to the macro imaging mode (S1602). Note that the timing of switching from step S1601 to step S1602 may be not when a predetermined time has passed from step S1601 but when the receiving device 1610 determines that the image is captured so that the light source fits within the frame 1611. By switching to the macro imaging mode in this manner, the user can easily put the light source in the frame 1611 with a clear image in the normal imaging mode before the image is blurred in the macro imaging mode. Can fit in.

Next, the reception device 1610 determines whether or not a signal from the light source is received (S1603). If it is determined that the signal from the light source is received (Yes in S1603), the process returns to the normal imaging mode of step S1601, and if it is determined that the signal from the light source is not received (No in S1603), the macro imaging of step 1602 is performed. Continue the mode. Note that in the case of Yes in step S1603, a process based on the received signal (for example, a process of displaying an image shown in the received signal) may be performed.

According to this receiving device 1610, a user touches the display unit of the light source 1611 of the smartphone with his/her finger to switch from the normal image capturing mode to the macro image capturing mode, so that a plurality of light sources can be captured in a blurred state. Therefore, the image captured in the macro imaging mode includes more bright regions than the image captured in the normal imaging mode. In particular, between two adjacent light sources of a plurality of light sources, the light from the two light sources overlaps with each other, so that the striped images are separated as shown in the left diagram of FIG. The problem of not being received as a signal can be demodulated as a continuous reception signal for forming continuous stripes as shown in the right figure. Since a long code can be received at one time, there is an effect that the response time becomes short. As shown in (b) of FIG. 397, when a photographed image is first photographed with a normal shutter and a normal focus, a beautiful normal image is obtained. However, if the light sources are far apart as in the case of letters, even if the shutter speed is increased, continuous data cannot be obtained and demodulation cannot be performed. Next, when the shutter speed is increased and the focus driving unit of the lens is set to a short distance (macro), the light source is blurred and spreads, and the four light sources are connected, so that data can be received. Then refocus and set the shutter speed back to normal to get the original beautiful image. By recording and displaying a beautiful image in the memory on the display unit as in (c), there is an effect that only the beautiful image is displayed on the display unit. The image captured in the macro imaging mode includes more bright areas than the predetermined brightness than the image captured in the normal imaging mode. Therefore, in the macro imaging mode, it is possible to increase the number of exposure lines capable of generating bright lines for the subject.

FIG. 398 is a diagram illustrating a processing operation of the reception device (imaging device). Specifically, FIG. 398 is a diagram for describing another example of the switching processing between the normal imaging mode and the macro imaging mode when receiving visible light communication.

Here, the receiving device 1620 receives visible light emitted by the transmitting device including a plurality of light sources (four light sources in FIG. 398).

First, when the receiving device 1620 transitions to the visible light communication mode, the receiving device 1620 activates the imaging unit in the normal imaging mode and captures an image 1623 in a wider range than the image 1622 displayed on the screen of the receiving device 1620. .. Then, the image data indicating the captured image 1623 and the posture information indicating the posture of the receiving device 1620 detected by the gyro sensor, the geomagnetic sensor, and the acceleration sensor of the receiving device 1620 when the image 1623 is captured are held in the memory. Yes (S1611). Note that the captured image 1623 is an image in a wide range of a predetermined width in the vertical direction and the horizontal direction with the image 1622 displayed on the screen of the reception device 1620 as a reference. In addition, the reception device 1620 displays a frame 1621 for capturing an image of the light source on the screen when the mode transitions to the visible light communication mode.

After a predetermined time, the reception device 1620 switches the imaging mode of the imaging unit to the macro imaging mode (S1612). Note that the timing of switching from step S1611 to step S1612 is not after a predetermined time from step S1611, but even when it is determined that the image 1623 is captured and the image data indicating the captured image 1623 is stored in the memory. Good. At this time, the receiving device 1620 displays the image 1624 of the size corresponding to the screen size of the receiving device 1620 among the images 1623 based on the image data stored in the memory.

The image 1624 displayed on the receiving device 1620 at this time is a part of the image 1623, and is the posture of the receiving device 1620 indicated by the posture information acquired in step S1611 (shown by a white broken line). It is an image of a region predicted to be imaged by the current receiving device 1620 from the difference between the position) and the current attitude of the receiving device 1620. That is, the image 1624 is a partial image of the image 1623, and is an image of a region corresponding to the imaging target of the image 1625 actually captured in the macro imaging mode. That is, in step S1612, the posture (imaging direction) changed from the time of step S1611 is acquired, the image capturing target estimated to be currently captured is identified from the acquired current posture (imaging direction), and the image is captured in advance. An image 1624 corresponding to the current posture (imaging direction) is specified from the image 1623, and processing for displaying the image 1624 is performed. Therefore, as shown by the image 1623 in FIG. 398, when the receiving device 1620 moves from the position indicated by the white broken line in the direction of the white arrow, the receiving device 1620 cuts out the image 1623 according to the moving amount. The area 1624 can be determined, and the image 1624 that is the image 1623 in the determined area can be displayed.

As a result, the receiving device 1620 does not display the image 1625 captured in the macro image capturing mode, even if the image is captured in the macro image capturing mode, and instead of the image 1623 captured in the clearer normal image capturing mode. The image 1624 cut out according to the current posture of the receiving device 1620 can be displayed. In the method of the present invention in which continuous visible light information is obtained from a plurality of light sources that are distant from the defocused image, and at the same time, the stored normal surface image is displayed on the display unit, the user shoots with a smartphone. When this happens, camera shake may occur, and the direction of the actual captured image and the still image displayed from the memory may deviate, causing a problem that the user cannot adjust the direction to the target light source. In this case, it is not possible to receive the data from the light source, so it is necessary to take measures. However, according to the improved invention, even if the camera shakes, the camera shake is detected by the image shake detecting means and the shake detecting means of the vibrating gyroscope, and the target image in the still image is shifted in a predetermined direction. The user can see the deviation from the direction. This display allows the user to aim the camera at the target light source, allowing multiple divided light sources to be optically linked and photographed while displaying a normal image and continuously receiving signals. can do. As a result, since the normal image is displayed, it is possible to receive a plurality of divided light sources. In this case, it is possible to easily adjust the posture of the reception device 1620 so that the plurality of light sources match the frame 1621. Note that when the focus is blurred, the light sources are dispersed, and the brightness is equivalently reduced. Therefore, there is an effect that the visible light data can be more reliably received by increasing the sensitivity of the ISO or the like of the camera.

Next, the reception device 1620 determines whether or not a signal from the light source is received (S1613). If it is determined that the signal from the light source is received (Yes in S1613), the process returns to the normal imaging mode in step S1611, and if it is determined that the signal from the light source is not received (S1).
No in step 613), and the macro imaging mode in step 1612 is continued. Note that in the case of Yes in step S1613, processing based on the received signal (for example, processing of displaying an image shown in the received signal) may be performed.

Similar to the reception device 1610, the reception device 1620 can also capture an image including a brighter area in the macro imaging mode. Therefore, in the macro imaging mode, it is possible to increase the number of exposure lines capable of generating bright lines for the subject.

FIG. 399 is a diagram illustrating a processing operation of the reception device (imaging device).

Here, the transmission device 1630 is a display device such as a television, and transmits different transmission IDs by visible light communication at predetermined time intervals Δ1630. Specifically, at times t1631, t1632, t1633, and t1634, the transmission IDs ID1631, ID1632, ID1633, and ID1634 that are respectively associated with the data corresponding to the displayed images 1631, 1632, 1633, and 1634 are transmitted. To do. That is, the ID 1631 to ID 1634 are transmitted from the transmission device 1630 one after another at predetermined time intervals Δt1630.

The receiving device 1640 requests the server 1650 for the data associated with each transmission ID based on the transmission ID received by visible light communication, receives the data from the server, and displays the image corresponding to the data. Specifically, images 1641, 1642, 1643, and 1644 corresponding to ID1631, ID1632, ID1633, and ID1634 are displayed at times t1631, t1632, t1633, and t1634, respectively.

When receiving the ID 1631 received at time t1631, the receiving device 1640 may acquire ID information indicating the transmission ID scheduled to be transmitted from the transmitting device 1630 from the server 1650 at the subsequent times t1632 to t1634. In this case, the reception device 1640 uses the acquired ID information to obtain the data associated with the IDs 1632 to 1634 at the times t1632 to t1634 without receiving the transmission ID from the transmission device 1630 each time. The received data can be displayed at each time t1632 to t1634.

Further, if the receiving device 1640 does not acquire the information indicating the transmission ID scheduled to be transmitted from the transmitting device 1630 from the server 1650 at the subsequent times t1632 to t1634, it can request the data corresponding to the ID1631 at the time t1631. The data associated with the transmission ID corresponding to the subsequent times t1632 to t1634 may be received from the server 1650, and the received data may be displayed at the respective times t1632 to t1634. That is, when the server 1650 receives the request for the data associated with the ID 1631 transmitted from the receiving device 1640 at the time t1631, the server 1650 receives the data associated with the transmission ID corresponding to the subsequent times t1632 to t1634. Even if there is no request from 1640, it is transmitted to the receiving device 1640 at each time t1632 to t1634. That is, in this case, the server 1650 holds the association information in which each time t1631-1634 is associated with the data associated with the transmission ID corresponding to each time t1631-1634, and based on the association information. At a predetermined time, predetermined data associated with the predetermined time is transmitted.

As described above, if the reception device 1640 can acquire the transmission ID 1631 by visible light communication at time t1631, the data corresponding to each time t1632 to t1634 from the server 1650 at the subsequent times t1632 to t1634 without performing visible light communication. Can be received. Therefore, the user does not need to keep pointing the receiving device 1640 at the transmitting device 1630 to obtain the transmission ID by visible light communication, and can easily cause the receiving device 1640 to display the data obtained from the server 1650. In this case, if the receiving device 1640 acquires the data corresponding to the ID from the server each time, the response time becomes long due to the time delay from the server. Therefore, in order to speed up the response, the data corresponding to the ID is stored in advance in the storage unit of the receiver from the server or the like, and the data corresponding to the ID in the storage unit is displayed, so that the response time Can be fast. In this system, if the time information for outputting the next ID is included in the transmission signal from the visible light transmitter, the receiver side will be able to reach that time without continuously receiving the visible light signal. For example, since it is possible to know the transmission time of the next ID, there is an effect that it is not necessary to keep the receiving device facing the light source. With this method, when visible light is received, the transmitter's time information (clock) is simply synchronized with the receiver's time information (clock). After synchronization, the transmitter data is not received. Also, there is an effect that the screen synchronized with the transmitter can be continuously displayed.

Further, in the above-described example, the receiving device 1640 displays images 1641, 1642, 1643, and 1644 corresponding to the transmission IDs ID1631, ID1632, ID1633, and ID1634 at times t1631, t1632, t1633, and t1634, respectively. Displayed respectively. Here, as shown in FIG. 400, receiving apparatus 1640 may present not only the image but also other information at each of the above times. That is, the receiving device 1640 displays the image 1641 corresponding to the ID 1631 at time t1631, and outputs the sound or voice corresponding to the ID 1631. At this time, the receiving device 1640 may further display, for example, the purchase site of the product displayed in the image. The output of the sound and the display of the purchase site are similarly performed at times t1632, t1633, and t1634 other than the time t1631.

Next, in the case of a smartphone having two stereoscopic left and right cameras as shown in (b) of FIG. 397, a normal shutter speed is displayed for the left eye and a normal image quality image is displayed at a normal focus. At the same time, in the camera for the right eye, the shutter is faster than the left eye, and/or the focus or macro at a short distance is set to obtain the striped bright line of the present invention and demodulate the data. As a result, it is possible to obtain an effect that an image of normal image quality is displayed on the display unit and that the right-eye camera can receive optical communication data of a plurality of light sources divided in distance.

(Embodiment 32)
Here, an application example of audio synchronous reproduction will be described below.

FIG. 401 is a diagram showing an example of an application in the thirty-second embodiment.

For example, the receiver 1800a configured as a smartphone receives a signal (visible light signal) transmitted from the transmitter 1800b configured as, for example, a street digital signage. That is, the receiver 1800a receives the timing of image reproduction by the transmitter 1800b. The receiver 1800a reproduces sound at the same timing as the image reproduction. In other words, the receiver 1800a performs the synchronized reproduction of the sound so that the image reproduced by the transmitter 1800b and the sound are synchronized with each other. Note that the receiver 1800a may reproduce the same image as the image reproduced by the transmitter 1800b (reproduced image) or a related image related to the reproduced image together with sound. Further, the receiver 1800a may cause a device connected to the receiver 1800a to reproduce sound or the like. After receiving the visible light signal, the receiver 1800a may download the content such as the sound or the related image associated with the visible light signal from the server. The receiver 1800a performs synchronous reproduction after the download.

As a result, even if the sound from the transmitter 1800b cannot be heard, or even if the sound from the transmitter 1800b is not reproduced because the street sound reproduction is prohibited, the user can hear the sound according to the display of the transmitter 1800b. Can be heard. In addition, even if there is a distance such that it takes time to reach the voice, it is possible to listen to the voice adapted to the display.

Here, multilingual support by audio synchronous reproduction will be described below.

FIG. 402 is a diagram showing an example of the application in the thirty-second embodiment.

Each of the receiver 1800a and the receiver 1800c acquires sound from the server and reproduces sound corresponding to the language set in the receiver and corresponding to the image such as a movie displayed on the transmitter 1800d. To do. Specifically, the transmitter 1800d transmits a visible light signal indicating an ID for identifying the displayed image to the receiver. When the receiver receives the visible light signal, the receiver sends a request signal including the ID shown in the visible light signal and the language set for itself to the server. The receiver acquires the sound corresponding to the request signal from the server and reproduces it. Thus, the user can enjoy the work displayed on the transmitter 1800d in the language set by the user.

Here, the audio synchronization method will be described below.

403 and 404 are diagrams showing an example of a transmission signal and an example of a voice synchronization method according to the thirty-second embodiment.

Different data (for example, data: 1 to 6 shown in FIG. 403) is associated with a time every fixed time (N seconds). These data may be, for example, an ID for identifying time, may be time, or may be audio data (eg, 64 Kbps data). Hereinafter, description will be made on the assumption that the data is an ID. The different IDs may have different additional information parts associated with the IDs.

It is desirable that the packets forming the ID be different. Therefore, it is desirable that the IDs are not consecutive. Alternatively, when packetizing the ID, it is desirable to use a packetizing method that configures the discontinuous portion as one packet. Since error correction signals tend to have different patterns even with consecutive IDs, the error correction signals may be dispersed in a plurality of packets instead of being collected in one packet.

The transmitter 1800d transmits the ID at the reproduction time of the displayed image, for example. The receiver can recognize the reproduction time (synchronization time) of the image of the transmitter 1800d by detecting the timing when the ID is changed.

In the case of (a), since the change time points of ID:1 and ID:2 are received, the synchronization time can be accurately recognized.

If the time N during which the ID is transmitted is long, such an opportunity is small and the ID may be received as shown in (b). Even in this case, the synchronization time can be recognized by the following method.

(B1) The midpoint of the reception section where the ID has changed is assumed to be the ID change point. Further, the time after an integer multiple of the time N from the ID change point estimated in the past is also estimated as the ID change point, and the midpoint of the plurality of ID change points is estimated as the more accurate ID change point. With such an estimation algorithm, the ID change point can be gradually and accurately estimated.

(B2) In addition to the above, it is possible that the reception section in which the ID has not changed and the time after an integer multiple of the time N do not include the ID change point, so that the ID change point may gradually occur. It is possible to estimate an accurate ID change point by reducing the number of certain sections.

By setting N to 0.5 seconds or less, accurate synchronization can be achieved.

By setting N to 2 seconds or less, it is possible to synchronize without making the user feel any delay.

By setting N to 10 seconds or less, waste of the ID can be suppressed and synchronization can be achieved.

FIG. 404 is a diagram illustrating an example of a transmission signal in Embodiment 32.

In FIG. 404, the waste of the ID can be avoided by performing the synchronization by the time packet. The time packet is a packet holding the transmitted time. When it is necessary to express a long time, the time packet is divided into a time packet 1 representing a fine time and a time packet 2 representing a coarse time to form the time packet. For example, time packet 2 indicates hours and minutes of the time, and time packet 1 indicates only seconds of the time. The packet indicating the time may be divided into three or more time packets. Since coarse time is less necessary, the receiver can recognize the synchronization time quickly and accurately by transmitting more fine time packets than coarse time packets.

That is, in the present embodiment, the visible light signal includes the second information (time packet 2) indicating the hour and minute of the time and the first information (time packet 1) indicating the second of the time. Is included to indicate the time at which the visible light signal is transmitted from transmitter 1800d. Then, the receiver 1800a receives the second information and receives the first information more times than the number of times of receiving the second information.

Here, the synchronization time adjustment will be described below.

FIG. 405 is a diagram illustrating an example of a process flow of the receiver 1800a in Embodiment 32.

It takes a certain amount of time from the signal is transmitted to the processing by the receiver 1800a until the sound or moving image is played back. Therefore, by performing the processing of playing back the sound or moving image in anticipation of this processing time, Synchronous reproduction can be performed.

First, a processing delay time is designated for the receiver 1800a (step S1801). This may be held in the processing program or may be designated by the user. By the user performing the correction, more accurate synchronization can be realized according to the individual receiver. The processing delay time can be more accurately synchronized by changing the temperature of the receiver and the CPU usage rate for each model of the receiver.

The receiver 1800a determines whether a time packet has been received, or whether an ID associated for voice synchronization has been received (step S1802). Here, if the receiver 1800a determines that the image has been received (Y in step S1802), the receiver 1800a further determines whether or not there is an image waiting to be processed (step S1804). If it is determined that there is a processing-waiting image (Y in step S1804), the receiver 1800a discards the processing-waiting image, or delays the processing of the processing-waiting image, and receives the latest reception processing from the latest image. Is performed (step S1805). As a result, an unexpected delay due to the processing waiting amount can be avoided.

The receiver 1800a measures at which position in the image the visible light signal (specifically, the bright line) is located (step S1806). That is, by measuring at which position in the direction perpendicular to the exposure line the signal appears from the first exposure line in the image sensor, the time difference from the image acquisition start time to the signal reception time (in-image delay time) can be determined. Can be calculated.

The receiver 1800a can accurately perform the synchronous reproduction by reproducing the audio or moving image at the time obtained by adding the processing delay time and the intra-image delay time to the recognized synchronization time (step S1807).

On the other hand, if it is determined in step S1802 that the receiver 1800a has not received the time packet or the voice synchronization ID, the receiver 1800a receives a signal from the image obtained by imaging (step S1803).

FIG. 406 is a diagram illustrating an example of a user interface of the receiver 1800a in Embodiment 32.

The user can adjust the above-mentioned processing delay time by pressing any of the buttons Bt1 to Bt4 displayed on the receiver 1800a as shown in (a) of FIG. 406. Further, the processing delay time may be set by the swipe operation as shown in FIG. 406(b). Thereby, it is possible to perform the synchronized reproduction more accurately based on the user's feeling.

Here, the earphone limited reproduction will be described below.

FIG. 407 is a diagram illustrating an example of a process flow of the receiver 1800a in Embodiment 32.

By the earphone limited reproduction shown by this processing flow, it is possible to perform sound reproduction without disturbing the surroundings.

The receiver 1800a confirms whether the earphone-only setting is set (step S1811). When the earphone-only setting is made, for example, the earphone-only setting is made in the receiver 1800a. Alternatively, the earphone-only setting is set in the received signal (visible light signal). Alternatively, the earphone limitation is recorded in the server or the receiver 1800a in association with the received signal.

Upon confirming that the earphones are limited (Y in step S1811), the receiver 1800a determines whether or not the earphones are connected to the receiver 1800a (step S1813).

When the receiver 1800a confirms that earphones are not limited (N in step S1811) or determines that the earphones are connected (Y in step S1813), the receiver 1800a reproduces audio (step S1812). When reproducing sound, the receiver 1800a adjusts the volume so that the volume is within the set range. This setting range is set similar to the earphone-only setting.

When the receiver 1800a determines that the earphone is not connected (N in step S1813), the receiver 1800a gives a notification prompting the user to connect the earphone (step S1814). This notification is performed by screen display, voice output, or vibration, for example.

If the setting that prohibits forced audio reproduction is not set, the receiver 1800a prepares an interface for forced reproduction and determines whether or not the user has performed the forced reproduction operation ( Step S1815). Here, when it is determined that the forced reproduction operation is performed (Y in step S1815), the receiver 1800a reproduces the sound even when the earphone is not connected (step S1812).

On the other hand, when it is determined that the forced reproduction operation is not performed (N in step S1815), the receiver 1800a holds the previously received audio data and the analyzed synchronization time, so that the earphone is connected. Promptly perform audio synchronous playback.

FIG. 408 is a diagram illustrating another example of the process flow of the receiver 1800a in Embodiment 32.

The receiver 1800a first receives the ID from the transmitter 1800d (step S1821). That is, the receiver 1800a receives the visible light signal indicating the ID of the transmitter 1800d or the ID of the content displayed on the transmitter 1800d.

Next, the receiver 1800a downloads the information (content) associated with the received ID from the server (step S1822). Alternatively, the receiver 1800a reads the information from the data holding unit inside the receiver 1800a. Hereinafter, this information is referred to as related information.

Next, the receiver 1800a determines whether or not the synchronous reproduction flag included in the related information indicates ON (step S1823). If it is determined that the synchronous reproduction flag is not ON (N in step S1823), the receiver 1800a outputs the content indicated by the related information (step S1824). That is, when the content is an image, the receiver 1800a displays the image, and when the content is a sound, the receiver 1800a outputs the sound.

On the other hand, when the receiver 1800a determines that the synchronous reproduction flag is ON (Y in step S1823), whether the time adjustment mode included in the related information is set to the transmitter reference mode. , It is determined whether the absolute time mode is set (step S1825). When it is determined that the absolute time mode is set, the receiver 1800a determines whether or not the last time adjustment has been performed within a fixed time from the current time (step S1826). The time adjustment at this time is a process of acquiring the time information by a predetermined method and using the time information to adjust the time of the clock provided in the receiver 1800a to the absolute time of the reference clock. The predetermined method is, for example, a method using GPS (Global Positioning System) radio waves or NTP (Network Time Protocol) radio waves. The above-mentioned current time may be the time when the receiver 1800a which is the terminal device receives the visible light signal.

When the receiver 1800a determines that the last time adjustment has been performed within the fixed time (Y in step S1826), the receiver 1800a outputs relevant information based on the time of the clock of the receiver 1800a, thereby displaying it on the transmitter 1800d. Content and related information are synchronized (step S1827). When the content indicated by the related information is, for example, a moving image, the receiver 1800a displays the moving image so as to be synchronized with the content displayed on the transmitter 1800d. When the content indicated by the related information is, for example, audio, the receiver 1800a outputs the audio so as to be synchronized with the content displayed on the transmitter 1800d. For example, when the related information indicates a voice, the related information includes each frame constituting the voice, and those frames are time-stamped. The receiver 1800a outputs a sound synchronized with the content of the transmitter 1800d by reproducing a frame having a time stamp corresponding to the time of its own clock.

When the receiver 1800a determines that the final time adjustment has not been performed within the fixed time (N in step S1826), it attempts to acquire the time information by a predetermined method and determines whether or not the time information could be acquired. It is determined (step S1828). Here, if it is determined that the time information can be obtained (Y in step S1828), the receiver 1800a updates the time of the clock of the receiver 1800a using the time information (step S1829). Then, the receiver 1800a executes the process of step S1827 described above.

When it is determined in step S1825 that the time adjustment mode is the transmitter reference mode, or when it is determined in step S1828 that the time information could not be obtained (N in step S1828), the receiver 1800a determines , Time information is acquired from the transmitter 1800d (step S1830). That is, the receiver 1800a acquires time information, which is a synchronization signal, from the transmitter 1800d by visible light communication. For example, the synchronization signals are time packet 1 and time packet 2 shown in FIG. 404. Alternatively, the receiver 1800a acquires time information from the transmitter 1800d by radio waves such as Bluetooth (registered trademark) or Wi-Fi. Then, the receiver 1800a executes the processes of steps S1829 and S1827 described above.

In this embodiment, as in steps S1829 and S1830, a process (time adjustment) for synchronizing the clock of the terminal device, which is the receiver 1800a, and the reference clock is performed by GPS radio waves or NTP radio waves. When the time when the visible light signal is transmitted from the transmitter 1800d is before the predetermined time from the time when the visible light signal is received by the terminal device, the clock of the terminal device and the clock of the transmitter are determined by the time indicated by the visible light signal. Synchronize between. Thereby, the terminal device can reproduce the content (moving image or sound) at the timing synchronized with the transmitter-side content reproduced by the transmitter 1800d.

FIG. 409A is a diagram for explaining a specific method for synchronous reproduction in Embodiment 32. The synchronized reproduction methods include methods a to e shown in FIG. 409.

(Method a)
In method a, the transmitter 1800d outputs the visible light signal indicating the content ID and the time during which the content is being reproduced by changing the brightness of the display, as in the above embodiments. The content reproduction time is the reproduction time of the data that is a part of the content and is being reproduced by the transmitter 1800d when the content ID is transmitted from the transmitter 1800d. If the content is a moving image, the data is a picture or sequence forming the moving image. If the content is sound, the data is a frame forming the sound. The reproduction time indicates, for example, the reproduction time from the beginning of the content as time. If the content is a moving image, the reproduction time is included in the content as PTS (Presentation Time Stamp). That is, the content includes the reproduction time (display time) of the data for each piece of data constituting the content.

The receiver 1800a receives the visible light signal by capturing an image of the transmitter 1800d as in each of the above-described embodiments. Then, the receiver 1800a transmits a request signal including the content ID indicated by the visible light signal to the server 1800f. The server 1800f receives the request signal and transmits the content associated with the content ID included in the request signal to the receiver 1800a.

When the receiver 1800a receives the content, the receiver 1800a reproduces the content from the time point of (content reproducing time+elapsed time from ID reception). The elapsed time from the reception of the ID is the elapsed time from when the content ID is received by the receiver 1800a.

(Method b)
In method b, the transmitter 1800d outputs the visible light signal indicating the content ID and the content reproducing time by changing the brightness of the display, as in the above embodiments. The receiver 1800a receives the visible light signal by capturing an image of the transmitter 1800d as in each of the above-described embodiments. Then, the receiver 1800a transmits a request signal including the content ID indicated by the visible light signal and the content reproducing time to the server 1800f. The server 1800f receives the request signal and transmits, to the receiver 1800a, only a part of the content after the content reproduction time, out of the content associated with the content ID included in the request signal.

Upon receiving the part of the content, the receiver 1800a reproduces the part of the content from the point of time (elapsed time from ID reception).

(Method c)
In method c, the transmitter 1800d outputs the visible light signal indicating the transmitter ID and the content reproducing time by changing the brightness of the display, as in the above-described embodiments. The transmitter ID is information for identifying the transmitter.

The receiver 1800a receives the visible light signal by capturing an image of the transmitter 1800d as in each of the above-described embodiments. Then, the receiver 1800a transmits a request signal including the transmitter ID indicated by the visible light signal to the server 1800f.

The server 1800f holds, for each transmitter ID, a reproduction schedule that is a timetable of the content reproduced by the transmitter having the transmitter ID. Further, the server 1800f includes a clock. Upon receiving the request signal, the server 1800f sets the content associated with the transmitter ID included in the request signal and the clock time (server time) of the server 1800f as the content being reproduced. , Specify from the replay schedule. Then, the server 1800f transmits the content to the receiver 1800a.

When the receiver 1800a receives the content, the receiver 1800a reproduces the content from the time point of (content reproducing time+elapsed time from ID reception).

(Method d)
In method d, the transmitter 1800d outputs a visible light signal indicating the transmitter ID and the transmitter time by changing the brightness of the display, as in the above embodiments. The transmitter time is a time indicated by a clock included in the transmitter 1800d.

The receiver 1800a receives the visible light signal by capturing an image of the transmitter 1800d as in each of the above-described embodiments. Then, the receiver 1800a transmits a request signal including the transmitter ID and the transmitter time indicated by the visible light signal to the server 1800f.

The server 1800f holds the above reproduction schedule. Upon receiving the request signal, the server 1800f identifies the content associated with the transmitter ID and the transmitter time included in the request signal as the content being reproduced from the reproduction schedule. Further, the server 1800f specifies the content reproducing time from the transmitter time. That is, the server 1800f finds the reproduction start time of the specified content from the reproduction schedule and specifies the time between the transmitter time and the reproduction start time as the content reproduction time. Then, the server 1800f transmits the content and the content reproducing time to the receiver 1800a.

When the receiver 1800a receives the content and the time during which the content is being reproduced, the receiver 1800a reproduces the content from the point of (content reproducing time+elapsed time from ID reception).

Thus, in the present embodiment, the visible light signal indicates the time when the visible light signal is transmitted from transmitter 1800d. Therefore, the receiver 1800a that is the terminal device can receive the content associated with the time when the visible light signal is transmitted from the transmitter 1800d (transmitter time). For example, if the transmitter time is 5:43, the content reproduced at 5:43 can be received.

Further, in the present embodiment, the server 1800f has a plurality of contents each associated with time. However, the content associated with the time indicated by the visible light signal may not exist in the server 1800f. In such a case, the receiver 1800a, which is the terminal device, is associated with the time closest to the time indicated by the visible light signal and after the time indicated by the visible light signal among the plurality of contents. Content may be received. Thereby, even if the content associated with the time indicated by the visible light signal does not exist in the server 1800f, it is possible to receive an appropriate content from the plurality of contents in the server 1800f.

In addition, the reproducing method according to the present embodiment includes a signal receiving step of receiving a visible light signal from a sensor of a receiver 1800a (terminal device) from a transmitter 1800d that transmits a visible light signal according to a luminance change of a light source, and a receiver. From 1800a, a transmission step of transmitting a request signal for requesting the content associated with the visible light signal to the server 1800f, a content reception step of the receiver 1800a receiving the content from the server 1800f, and reproducing the content. And a reproduction step. The visible light signal indicates the transmitter ID and the transmitter time. The transmitter ID is ID information. The transmitter time is the time indicated by the clock of the transmitter 1800d, and the time when the visible light signal is transmitted from the transmitter 1800d. Then, in the content receiving step, the receiver 1800a receives the content associated with the transmitter ID and the transmitter time indicated by the visible light signal. As a result, the receiver 1800a can reproduce the content appropriate for the transmitter ID and the transmitter time.

(Method e)
In method e, the transmitter 1800d outputs a visible light signal indicating the transmitter ID by changing the brightness of the display, as in the above embodiments.

The receiver 1800a receives the visible light signal by capturing an image of the transmitter 1800d as in each of the above-described embodiments. Then, the receiver 1800a transmits a request signal including the transmitter ID indicated by the visible light signal to the server 1800f.

The server 1800f holds the above-mentioned reproduction schedule and further has a clock. Upon receiving the request signal, the server 1800f identifies the content associated with the transmitter ID and the server time included in the request signal from the reproduction schedule as the content being reproduced. The server time is the time indicated by the clock of the server 1800f. Further, the server 1800f also finds the reproduction start time of the specified content from the reproduction schedule. Then, the server 1800f transmits the content and the content reproduction start time to the receiver 1800a.

Upon receiving the content and the content reproduction start time, the receiver 1800a reproduces the content from the time of (receiver time-content reproduction start time). Note that the receiver time is the time indicated by the clock provided in the receiver 1800a.

As described above, the reproducing method according to the present embodiment includes a signal receiving step of receiving a visible light signal from a sensor of the receiver 1800a (terminal device) from a transmitter 1800d that transmits a visible light signal according to a change in luminance of a light source. A transmission step of transmitting a request signal for requesting the content associated with the visible light signal to the server 1800f from the receiver 1800a, the receiver 1800a includes each time, and data reproduced at each time. It includes a content receiving step of receiving the content from the server 1800f, and a reproducing step of reproducing the data corresponding to the time of the clock provided in the receiver 1800a in the content. Therefore, the receiver 1800a can properly reproduce the data in the content at the correct time indicated in the content without reproducing the data at the wrong time. Further, also in the transmitter 1800d, if the content related to the content (transmitter-side content) is reproduced, the receiver 1800a can reproduce the content in proper synchronization with the transmitter-side content. ..

Even in the above methods c to e, like method b, the server 1800f may transmit only some of the contents after the content reproducing time to the receiver 1800a.

In the above methods a to e, the receiver 1800a transmits a request signal to the server 1800f and receives necessary data from the server 1800f. You may keep it.

FIG. 409B is a block diagram showing the configuration of a playback device that performs synchronous playback by the method e described above.

The reproduction device B10 is a receiver 1800a or a terminal device that performs synchronous reproduction by the above-described method e, and includes a sensor B11, a request signal transmission unit B12, a content reception unit B13, a clock B14, and a reproduction unit B15. I have it.

The sensor B11 is, for example, an image sensor, and receives the visible light signal from the transmitter 1800d that transmits the visible light signal according to the change in the brightness of the light source. The request signal transmitter B12 transmits a request signal for requesting the content associated with the visible light signal to the server 1800f. The content receiving unit B13 receives content including each time and data reproduced at each time from the server 1800f. The reproducing unit B15 reproduces the data corresponding to the time of the clock B14 in the content.

FIG. 409C is a flowchart showing the processing operation of the terminal device which performs the synchronous reproduction by the above method e.

The reproduction device B10 is the receiver 1800a or the terminal device that performs the synchronous reproduction by the above-described method e, and executes the processes of steps SB11 to SB15.

In step SB11, the visible light signal is received from the transmitter 1800d that transmits the visible light signal according to the change in the brightness of the light source. In step SB12, a request signal for requesting the content associated with the visible light signal is transmitted to the server 1800f. In step SB13, the content including each time and the data reproduced at each time is received from the server 1800f. In step SB15, of the contents, the data corresponding to the time of the clock B14 is reproduced.

As described above, with the reproducing apparatus B10 and the reproducing method according to the present embodiment, it is possible to properly reproduce the data in the content at the correct time indicated in the content without reproducing the data at the wrong time.

In addition, in the present embodiment, each component may be configured by dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory. Here, the software that realizes the playback device B10 or the like of the present embodiment is a program that causes a computer to execute each step included in the flowchart illustrated in FIG. 409C.

FIG. 410 is a diagram for explaining advance preparation for synchronous playback in the thirty-second embodiment.

The receiver 1800a adjusts the time of the clock provided in the receiver 1800a to the time of the reference clock in order to perform the synchronous reproduction. For this time adjustment, the receiver 1800a performs the following processes (1) to (5).

(1) The receiver 1800a receives a signal. This signal may be a visible light signal transmitted by a change in the brightness of the display of the transmitter 1800d or a radio signal based on Wi-Fi or Bluetooth (registered trademark) from a wireless device. Alternatively, the receiver 1800a acquires position information indicating the position of the receiver 1800a, for example, by GPS instead of receiving such a signal. Then, the receiver 1800a recognizes from the position information that the receiver 1800a has entered a predetermined place or building.

(2) When the receiver 1800a receives the above signal or recognizes that it has entered a predetermined place, the receiver 1800a sends a request signal requesting data (related information) associated with the signal or the place. It is transmitted to the server (visible light ID resolution server) 1800f.

(3) The server 1800f transmits the above-mentioned data and a time adjustment request for causing the receiver 1800a to adjust the time to the receiver 1800a.

(4) Upon receiving the data and the time adjustment request, the receiver 1800a transmits the time adjustment request to the GPS time server, the NTP server, or the base station of the telecommunications carrier (carrier).

(5) Upon receiving the time adjustment request, the server or the base station transmits time data (time information) indicating the current time (time of the reference clock or absolute time) to the receiver 1800a. The receiver 1800a adjusts the time by adjusting the time of its own clock to the current time indicated by the time data.

As described above, in the present embodiment, a GPS (Global Positioning System) radio wave or an NTP (Network Time Protocol) radio wave is used between the clock provided in the receiver 1800a (terminal device) and the reference clock, Synchronized. Therefore, the receiver 1800a can reproduce the data corresponding to the time at an appropriate time according to the reference clock.

FIG. 411 is a diagram illustrating an application example of the receiver 1800a in Embodiment 32.

The receiver 1800a is configured as a smartphone as described above, and is used by being held by a holder 1810 made of a member such as resin or glass having a light-transmitting property. The holder 1810 has a back plate portion 1810a and a locking portion 1810b provided upright on the back plate portion 1810a. The receiver 1800a is inserted between the back plate portion 1810a and the locking portion 1810b along the back plate portion 1810a.

FIG. 412A is a front view of receiver 1800a held by holder 1810 in the thirty-second embodiment.

The receiver 1800a is held by the holder 1810 in the inserted state as described above. At this time, the locking portion 1810b locks with the lower portion of the receiver 1800a and clamps the lower portion with the back plate portion 1810a. Further, the back surface of the receiver 1800a faces the back plate portion 1810a, and the display 1801 of the receiver 1800a is exposed.

FIG. 412B is a rear view of the receiver 1800a held by the holder 1810 in the thirty-second embodiment.

A through hole 1811 is formed in the back plate portion 1810a, and a variable filter 1812 is attached near the through hole 1811. When the receiver 1800a is held by the holder 1810, the camera 1802 of the receiver 1800a is exposed from the back plate portion 1810a through the through hole 1811. Further, the flash light 1803 of the receiver 1800a faces the variable filter 1812.

The variable filter 1812 is formed, for example, in a disk shape, and has three color filters (red filter, yellow filter, and green filter) each having a fan shape and the same size. The variable filter 1812 is rotatably attached to the back plate portion 1810a about the center of the variable filter 1812. The red filter is a filter having a red translucency, the yellow filter is a filter having a yellow translucency, and the green filter is a filter having a green translucency.

Therefore, the variable filter 1812 is rotated and, for example, the red filter is arranged at a position facing the flashlight 1803a. In this case, the light emitted from the flashlight 1803a is diffused inside the holder 1810 as red light by passing through the red filter. As a result, substantially the entire holder 1810 emits red light.

Similarly, the variable filter 1812 is rotated and, for example, the yellow filter is arranged at a position facing the flashlight 1803a. In this case, the light emitted from the flashlight 1803a is diffused inside the holder 1810 as yellow light by passing through the yellow filter. As a result, substantially the entire holder 1810 emits yellow light.

Similarly, the variable filter 1812 is rotated and, for example, the green filter is arranged at a position facing the flashlight 1803a. In this case, the light emitted from the flashlight 1803a is diffused inside the holder 1810 as green light by passing through the green filter. As a result, substantially the entire holder 1810 emits green light.

That is, the holder 1810 lights up in red, yellow, or green like a penlight.

FIG. 413 is a diagram for describing a use case of the receiver 1800a held by the holder 1810 in the thirty-second embodiment.

For example, the receiver with holder, which is the receiver 1800a held by the holder 1810, is used in an amusement park or the like. That is, the plurality of receivers with holders directed to the moving float in the amusement park blinks in synchronization with the music flowing from the float. That is, the float is configured as the transmitter in each of the above embodiments, and transmits the visible light signal according to the change in the brightness of the light source attached to the float. For example, the float transmits a visible light signal that indicates the ID of the float. Then, the receiver with a holder receives the visible light signal, that is, the ID, by photographing with the camera 1802 of the receiver 1800a, as in each of the above-described embodiments. The receiver 1800a that has received the ID acquires the program associated with the ID from, for example, a server. This program is composed of an instruction to turn on the flashlight 1803 of the receiver 1800a at each predetermined time. The respective predetermined times are set (in synchronization) with the music flowing from the float. Then, the receiver 1800a blinks the flash light 1803a according to the program.

As a result, the holder 1810 of each receiver 1800a that has received the ID repeats lighting at the same timing according to the music flowing from the float of the ID.

Here, each receiver 1800a causes the flashlight 1803 to blink according to the color filter that has been set (hereinafter referred to as the setting filter). The setting filter is a color filter facing the flashlight 1803 of the receiver 1800a. Further, each receiver 1800a recognizes the current setting filter based on the operation by the user. Alternatively, each receiver 1800a recognizes the current setting filter based on the color of the image obtained by the camera 1802.

That is, among the plurality of receivers 1800a that have received the ID, at a predetermined time, only the holders 1810 of the plurality of receivers 1800a that recognize that the setting filter is the red filter are simultaneously turned on. At the next time, only the holders 1810 of the plurality of receivers 1800a that recognize that the setting filter is the green filter are turned on at the same time. Further, at the next time, only the holders 1810 of the plurality of receivers 1800a that recognize that the setting filter is the yellow filter are simultaneously turned on.

In this way, the receiver 1800a held in the holder 1810 is synchronized with the float music and the receiver 1800a held in another holder 1810 in the same manner as the synchronized playback shown in FIGS. 401 to 407 described above. Then, the flashlight 1803, that is, the holder 1810 is made to blink.

FIG. 414 is a flowchart showing processing operations of the receiver 1800a held by the holder 1810 in Embodiment 32.

The receiver 1800a receives the float ID indicated by the visible light signal from the float (step S1831). Next, the receiver 1800a acquires the program associated with the ID from the server (step S1832). Next, the receiver 1800a executes the program to turn on the flashlight 1803 at each predetermined time corresponding to the setting filter (step S1833).

Here, the receiver 1800a may display an image according to the received ID or the acquired program on the display 1801.

FIG. 415 is a diagram illustrating an example of an image displayed by the receiver 1800a in Embodiment 32.

For example, when the receiver 1800a receives the ID from the Santa Claus float, the receiver 1800a displays an image of Santa Claus as shown in (a) of FIG. 415. Further, as shown in FIG. 415(b), the receiver 1800a may change the background color of the image of Santa Claus to the color of the setting filter at the same time when the flashlight 1803 is turned on. For example, when the color of the setting filter is red, the holder 1810 lights up in red by lighting the flashlight 1803, and at the same time, an image of Santa Claus having a red background color is displayed on the display 1801. That is, the blinking of the holder 1810 and the display on the display 1801 are synchronized.

FIG. 416 is a diagram showing another example of the holder according to the thirty-second embodiment.

The holder 1820 has the same structure as the holder 1810 described above, but lacks the through hole 1811 and the variable filter 1812. Such a holder 1820 holds the receiver 1800a with the display 1801 of the receiver 1800a facing the back plate portion 1820a. In this case, the receiver 1800a causes the display 1801 to emit light instead of the flashlight 1803. As a result, the light from the display 1801 is diffused over almost the entire holder 1820. Therefore, when the receiver 1800a causes the display 1801 to emit light with red light according to the above-described program, the holder 1820 lights up in red. Similarly, when the receiver 1800a causes the display 1801 to emit light with yellow light according to the above-described program, the holder 1820 lights up in yellow. Receiver 1800a
However, when the display 1801 is caused to emit light with green light in accordance with the above program, the holder 1820 lights up in green. By using such a holder 1820, setting of the variable filter 1812 can be omitted.

(Embodiment 33)
(Visible light signal)
417A to 417D are diagrams each illustrating an example of a visible light signal in Embodiment 33.

Similar to the above, the transmitter generates a visible light signal of 4PPM, for example, as shown in FIG. 417A, and changes the brightness according to this visible light signal. Specifically, the transmitter allocates four slots to one signal unit and generates a visible light signal composed of a plurality of signal units. The signal unit indicates High (H) or Low (L) for each slot. Then, the transmitter emits bright light in the H slot, and emits dark light or extinguishes in the L slot. For example, one slot is a period corresponding to a time of 1/9600 seconds.

Also, the transmitter may generate a visible light signal in which the number of slots assigned to one signal unit is variable, as shown in FIG. 417B, for example. In this case, in a signal unit, a signal indicating H in one or more consecutive slots and a signal indicating L in one slot following the H signal are included. Since the number of H slots is variable, the total number of slots per signal is variable. For example, as shown in FIG. 417B, the transmitter generates a visible light signal including these signal units in the order of 3 slot signal units, 4 slot signal units, and 6 slot signal units. In this case, the transmitter emits bright light in the H slot, and emits dark light or extinguishes in the L slot.

Also, the transmitter may allocate an arbitrary period (signal unit period) to one signal unit without allocating a plurality of slots to one signal unit, as shown in FIG. 417C, for example. The signal unit period includes an H period and an L period that follows the H period. The H period is adjusted according to the signal before modulation. The L period may be fixed and may correspond to the slot. The H period and the L period are, for example, 100 μs or more. For example, as illustrated in FIG. 417C, the transmitter uses a visible light signal including a signal unit with a signal unit period of 210 μs, a signal unit with a signal unit period of 220 μs, and a signal unit with a signal unit period of 230 μs in this order. To send. Also in this case, the transmitter emits bright light during the H period, and emits dark light or extinguishes during the L period.

Further, the transmitter may generate, as a visible light signal, a signal alternately indicating L and H as shown in FIG. 417D, for example. In this case, the L period and the H period of the visible light signal are adjusted according to the signal before modulation. For example, as shown in FIG. 417D, the transmitter exhibits H for 100 μs, then L for 120 μs, then H for 110 μs, and L for 200 μs. The visible light signal shown is transmitted. Also in this case, the transmitter emits bright light during the H period, and emits dark light or extinguishes during the L period.

FIG. 418 is a diagram showing the structure of the visible light signal in the thirty-third embodiment.

The visible light signal includes, for example, a signal 1, a brightness adjustment signal corresponding to the signal 1, a signal 2, and a brightness adjustment signal corresponding to the signal 2. When the transmitter generates the signal 1 and the signal 2 by modulating the unmodulated signal, the transmitter generates a brightness adjustment signal for these signals and generates the visible light signal described above.

The brightness adjustment signal corresponding to the signal 1 is a signal that compensates for the increase or decrease in brightness due to the brightness change according to the signal 1. The brightness adjustment signal corresponding to the signal 2 is a signal that compensates for an increase or decrease in brightness due to a change in luminance according to the signal 2. Here, the brightness B1 is expressed by the brightness change according to the signal 1 and the brightness adjustment signal of the signal 1, and the brightness change according to the signal 2 and the brightness adjustment signal of the signal 2 is expressed by The brightness B2 is expressed. The transmitter according to the present embodiment generates the respective brightness adjustment signals of signal 1 and signal 2 as a part of the visible light signal so that the brightness B1 and the brightness B2 are equal. Thereby, the brightness is kept constant and flicker can be suppressed.

Further, when generating the above-mentioned signal 1, the transmitter generates the signal 1 including the data 1, the preamble (header) following the data 1, and the data 1 following the preamble. Here, the preamble is a signal corresponding to data 1 arranged before and after it. For example, this preamble is a signal serving as an identifier for reading the data 1. Thus, since the signal 1 is composed of the two data 1 and the preamble arranged between them, even if the receiver reads the visible light signal from the middle of the preceding data 1, Data 1 (that is, signal 1) can be correctly demodulated.

(Bright line image)
FIG. 419 is a diagram illustrating an example of a bright line image obtained by imaging with a receiver in Embodiment 33.

As described above, the receiver captures a bright line image including a visible light signal transmitted from the transmitter as a bright line pattern by capturing an image of the transmitter that changes in brightness. With such imaging, a visible light signal is received by the receiver.

For example, as shown in FIG. 419, the receiver captures an image at time t1 by using N exposure lines included in the image sensor, and thus the bright lines including the regions a and b in which the bright line patterns appear, respectively. Get an image. Regions a and b are regions in which a bright line pattern appears when the transmitter, which is a subject, changes in luminance.

Here, the receiver demodulates the visible light signal from the bright line patterns of the areas a and b. However, when the receiver determines that the demodulated visible light signal alone is insufficient, it uses only M (M<N) continuous exposure lines corresponding to the area a among the N exposure lines. , At time t2. As a result, the receiver acquires a bright line image including only the area a of the areas a and b. The receiver repeatedly carries out such imaging also at times t3 to t5. As a result, a visible light signal with a sufficient data amount from the subject corresponding to the area a can be received at high speed. Further, the receiver takes an image at time t6 using only L (L<N) continuous exposure lines corresponding to the region b among the N exposure lines. As a result, the receiver acquires a bright line image including only the area b of the areas a and b. The receiver repeatedly carries out such imaging also at times t7 to t9. As a result, a visible light signal having a sufficient data amount from the subject corresponding to the region b can be received at high speed.

In addition, the receiver may acquire the bright line image including only the region a by performing the same image pickup at the times t2 to t5 at the times t10 and t11. Further, the receiver may acquire the bright line image including only the region b by performing the same imaging as that at times t6 to t9 at times t12 and t13.

Further, in the above-described example, when the receiver determines that the visible light signal is insufficient, the receiver continuously shoots the bright line image including only the area a at the times t2 to t5, but the imaging at the time t1 is performed. If bright lines appear in the image obtained by, the above continuous shooting may be performed. Similarly, when it is determined that the visible light signal is insufficient, the receiver continuously shoots the bright line image including only the region b at times t6 to t9, but the receiver obtained the images at time t1. If a bright line appears in the image, the above continuous shooting may be performed. Further, the receiver may alternately acquire the bright line image including only the area a and acquire the bright line image including only the area b.

Note that the M continuous exposure lines corresponding to the area a are the exposure lines that contribute to the generation of the area a, and the L continuous exposure lines corresponding to the area b are the generation of the area b. Is an exposure line that contributes to.

FIG. 420 is a diagram showing another example of the bright line image obtained by the image pickup by the receiver in the thirty-third embodiment.

For example, as shown in FIG. 420, the receiver captures an image at time t1 by using N exposure lines included in the image sensor, and thus the bright lines including the regions a and b in which the bright line patterns appear, respectively. Get an image. Similarly to the above, the areas a and b are areas in which the bright line pattern appears when the transmitter, which is the subject, changes in brightness. Further, each of the region a and the region b has a region overlapping with each other along the direction of the bright line or the exposure line (hereinafter referred to as an overlapping region).

Here, when the receiver determines that the visible light signal demodulated from the bright line patterns of the regions a and b is insufficient, P (P<N) corresponding to the overlapping region of the N exposure lines. An image is taken at time t2 using only the continuous exposure lines. As a result, the receiver acquires the bright line image including only the overlapping areas of the area a and the area b. The receiver repeatedly performs such imaging at times t3 and t4. As a result, it is possible to receive a sufficient amount of visible light signals from the subject corresponding to each of the regions a and b at substantially the same time and at high speed.

FIG. 421 is a diagram illustrating another example of the bright line image obtained by imaging with the receiver in Embodiment 33.

For example, as shown in FIG. 421, the receiver captures an image at time t1 using N exposure lines included in the image sensor, and clearly shows a portion a in which the bright line pattern appears indistinct and A bright line image including an area including the exposed portion b is acquired. This area is an area in which a bright line pattern appears due to a change in brightness of the transmitter, which is the subject, as described above.

In such a case, when the receiver determines that the visible light signal demodulated from the bright line pattern in the above area is insufficient, Q (Q<N) of the N exposure lines corresponding to the portion b The image is picked up at time t2 using only the continuous exposure line. As a result, the receiver acquires the bright line image including only the part b in the above-mentioned area. The receiver repeatedly performs such imaging at times t3 and t4. As a result, a visible light signal having a sufficient data amount from the subject corresponding to the above-described area can be received at high speed.

Further, the receiver may perform continuous shooting of the bright line images including only the portion a after performing continuous shooting of the bright line images including only the portion b.

As described above, when the bright line image includes a plurality of regions (or portions) in which the bright line pattern appears, the receiver assigns an order to each region, and according to the order, only that region is displayed. Sequential shooting of bright line images including. In this case, the order may be the order according to the size of the signal (width of the region or the portion) or the order of clarity of the bright line. Further, the order may be in accordance with the color of light from the subject corresponding to those areas. For example, the first continuous shooting is performed on the area corresponding to red light, and the second continuous shooting is performed on the area corresponding to white light. Further, only continuous shooting of the area for red light may be performed.

(HDR synthesis)
FIG. 422 is a diagram for describing adaptation of the receiver in Embodiment 33 to a camera system which performs HDR combination.

The vehicle is equipped with a camera system to prevent collision. This camera system performs HDR (High Dynamic Range) composition using an image obtained by imaging with a camera. An image with a wide luminance dynamic range can be obtained by this HDR combination. The camera system recognizes a surrounding vehicle, an obstacle, a person, or the like based on the image having the wide dynamic range.

For example, the camera system has a normal setting mode and a communication setting mode as setting modes. When the setting mode is the normal setting mode, for example, as shown in FIG. 422, the camera system performs four times of imaging at the same shutter speed of 1/100 second and different sensitivities at times t1 to t4. .. The camera system performs HDR composition using the four images obtained by the four imagings.

On the other hand, in the case where the setting mode is the communication setting mode, for example, as shown in FIG. 422, the camera system captures images three times at the same shutter speed of 1/100 seconds and different sensitivities at times t5 to t7. I do. Furthermore, at time t8, the camera system captures an image with a shutter speed of 1/10000 second and maximum sensitivity (for example, ISO=1600). The camera system performs HDR composition using the three images obtained by the first three imagings of the four imagings. Further, the camera system receives the visible light signal by the last imaging of the above-mentioned four times of imaging, and demodulates the bright line pattern appearing in the image obtained by the imaging.

Further, when the setting mode is the communication setting mode, the camera system may not perform HDR combination. For example, as shown in FIG. 422, the camera system performs imaging at time t9 with a shutter speed of 1/100 second and low sensitivity (for example, ISO=200). Further, the camera system performs image capturing three times at times t10 to t12 with a shutter speed of 1/10000 seconds and different sensitivities. The camera system recognizes a surrounding vehicle, an obstacle, a person, or the like from an image obtained by the first one of the four times of imaging. Further, the camera system receives the visible light signal by the last three times of the above four times of imaging, and demodulates the bright line pattern appearing in the image obtained by the imaging.

Note that, in the example shown in FIG. 422, imaging is performed with different sensitivities at times t10 to t12, but imaging may be performed with the same sensitivity.

In such a camera system, it is possible to perform HDR combination and also to receive a visible light signal.

(Security)
FIG. 423 is a diagram for explaining the processing operation of the visible light communication system in the thirty-third embodiment.

This visible light communication system includes, for example, a transmitter arranged at a cash register, a smartphone as a receiver, and a server. Note that communication between the smartphone and the server and communication between the transmitter and the server are performed via secure communication lines. Further, the communication between the transmitter and the smartphone is performed by visible light communication. The visible light communication system in the present embodiment ensures security by determining whether or not the visible light signal from the transmitter is correctly received by the smartphone.

Specifically, the transmitter transmits a visible light signal indicating, for example, a value “100” to the smartphone by changing the luminance at time t1. When the smartphone receives the visible light signal at time t2, the smartphone transmits a radio signal indicating the value “100” to the server. The server receives the radio signal from the smartphone at time t3. At this time, the server performs a process for determining whether or not the value “100” indicated by the radio wave signal is the value of the visible light signal received by the smartphone from the transmitter. That is, the server transmits a radio signal indicating, for example, the value “200” to the transmitter. The transmitter that has received the radio signal transmits the visible light signal indicating the value “200” to the smartphone by changing the luminance at time t4. When the smartphone receives the visible light signal at time t5, the smartphone transmits a radio signal indicating the value “200” to the server. The server receives the radio signal from the smartphone at time t6. The server determines whether the value indicated by the received radio signal is the same as the value indicated by the radio signal transmitted at time t3. If they are the same, the server determines that the value “100” indicated by the visible light signal received at time t3 is the value of the visible light signal transmitted from the transmitter to the smartphone and received. On the other hand, if they are not the same, the server determines that the value “100” indicated by the visible light signal received at time t3 is suspicious as the value of the visible light signal transmitted from the transmitter to the smartphone and received.

This allows the server to determine whether the smartphone has indeed received the visible light signal from the transmitter. That is, it is possible to prevent the smartphone from transmitting the signal to the server by pretending to be receiving the visible light signal even though the smartphone does not receive the visible light signal from the transmitter.

In the above example, the communication using the radio wave signal is performed between the smartphone, the server, and the transmitter, but even if the communication using the optical signal other than the visible light signal or the communication using the electric signal is performed. Good. The visible light signal transmitted from the transmitter to the smartphone indicates, for example, a billing value, a coupon value, a monster value, or a bingo value.

(Vehicle related)
FIG. 424A is a diagram illustrating an example of inter-vehicle communication using visible light in Embodiment 33.

For example, the leading vehicle recognizes that there is an accident in the traveling direction by a sensor (camera or the like) mounted on the vehicle. Upon recognizing the accident in this way, the leading vehicle transmits a visible light signal by changing the brightness of the tail lamp. For example, the leading vehicle transmits a visible light signal that prompts the following vehicles to decelerate. When the following vehicle receives the visible light signal by imaging with the camera mounted on the vehicle, the following vehicle decelerates according to the visible light signal and further transmits a visible light signal prompting the following vehicle to decelerate. To do.

In this way, the visible light signal that prompts deceleration is sequentially transmitted to the plurality of vehicles running in a line from the beginning, and the vehicle that receives the visible light signal decelerates. Since the visible light signal is transmitted to each vehicle quickly, the plurality of vehicles can decelerate in the same manner at substantially the same time. Therefore, traffic congestion due to an accident can be alleviated.

FIG. 424B is a diagram illustrating another example of inter-vehicle communication using visible light in Embodiment 33.

For example, the front vehicle may send a visible light signal indicating a message (eg, "thank you") to the rear vehicle by changing the brightness of the tail lamps. This message is generated, for example, by the user's operation on the smartphone. Then, the smartphone transmits a signal indicating the message to the previous vehicle described above. As a result, the front vehicle can send a visible light signal indicative of the message to the rear vehicle.

FIG. 425 is a diagram showing an example of a method for determining the positions of a plurality of LEDs in the thirty-third embodiment.

For example, a vehicle headlight has a plurality of LEDs (Light Emitting Diodes). The transmitter of this vehicle transmits a visible light signal from each of the LEDs of the headlight by individually changing the brightness of each of the LEDs. Other vehicle receivers receive visible light signals from their LEDs by imaging the vehicle with its headlights.

At this time, the receiver determines the position of each of the plurality of LEDs from the image obtained by the imaging in order to recognize from which LED the received visible light signal is the signal transmitted. .. Specifically, the receiver uses an acceleration sensor mounted on the same vehicle as the receiver, and a plurality of receivers are set based on the direction of gravity (for example, a downward arrow in FIG. 425) indicated by the acceleration sensor. Determine the position of each of the LEDs.

In addition, in the above-mentioned example, although LED was mentioned as an example of the light-emitting body which changes brightness|luminance, Light-emitting bodies other than LED may be sufficient.

FIG. 426 is a diagram showing an example of a bright line image obtained by imaging a vehicle in the thirty-third embodiment.

For example, the receiver mounted on the traveling vehicle acquires the bright line image shown in FIG. 426 by capturing an image of the vehicle behind (the following vehicle). The transmitter mounted on the following vehicle transmits a visible light signal to the vehicle in front by changing the brightness of two headlights of the vehicle. A camera for picking up an image of the rear is attached to a rear portion of a front vehicle, a side mirror or the like. The receiver acquires a bright line image by capturing an image of the following vehicle with the camera, and demodulates a bright line pattern (visible light signal) included in the bright line image. As a result, the visible light signal transmitted from the transmitter of the following vehicle is received by the receiver of the preceding vehicle.

Here, the receiver acquires the ID of the vehicle having the headlight, the speed of the vehicle, and the vehicle type of the vehicle from each of the visible light signals transmitted from the two headlights and demodulated. If the IDs of the two visible light signals are the same, the receiver determines that the two visible light signals are signals transmitted from the same vehicle. Then, the receiver identifies the length between the two headlights of the vehicle (inter-light distance) from the vehicle type of the vehicle. Further, the receiver measures the distance L1 between the two areas in which the bright line pattern appears, which is included in the bright line image. Then, the receiver calculates the distance (inter-vehicle distance) from the vehicle equipped with the receiver to the following vehicle by triangulation using the distance L1 and the inter-light distance. The receiver determines the risk of collision based on the inter-vehicle distance and the speed of the vehicle acquired from the visible light signal, and notifies the driver of the vehicle of a warning according to the determination result. Thereby, a collision of the vehicle can be avoided.

In the above example, the receiver specifies the inter-light distance from the vehicle type included in the visible light signal, but may specify the inter-light distance from information other than the vehicle type. Further, in the above example, when the receiver determines that there is a risk of collision, it issues a warning, but even if a control signal for causing the vehicle to perform an action to avoid the risk is output to the vehicle. Good. For example, the control signal is a signal for accelerating the vehicle or a signal for causing the vehicle to change lanes.

Further, in the above example, the camera captures the following vehicle, but it may capture the oncoming vehicle. In addition, when the receiver determines that fog is present around the receiver (that is, the vehicle equipped with the receiver) from the image obtained by the image pickup by the camera, the receiver enters the mode for receiving the visible light signal as described above. May be. Accordingly, the receiver of the vehicle can identify the position and speed of the oncoming vehicle by receiving the visible light signal transmitted from the headlight of the oncoming vehicle, even if the fog is trapped around the vehicle.

FIG. 427 is a diagram illustrating an example of application of the receiver and the transmitter in Embodiment 33. Note that FIG. 427 is a view of the automobile as seen from the rear.

For example, a transmitter (car) 7006a having two tail lamps (light emitting units or lights) of a car transmits the identification information (ID) of the transmitter 7006a to a receiver configured as a smartphone, for example. When the receiver receives the ID, the receiver acquires the information associated with the ID from the server. For example, the information includes the ID of the vehicle or the transmitter, the distance between the light emitting parts, the size of the light emitting parts, the size of the car, the shape of the car, the weight of the car, the number of the car, the appearance of the front, or the danger. It is information indicating the presence or absence of. Further, the receiver may directly obtain these pieces of information from the transmitter 7006a.

FIG. 428 is a flowchart illustrating an example of process operations of the receiver and the transmitter 7006a in Embodiment 33.

The ID of the transmitter 7006a and the information passed to the receiver that has received the ID are stored in the server in association with each other (7106a). The information to be passed to the receiver includes the size of the light emitting unit serving as the transmitter 7006a, the distance between the light emitting units, the shape of an object having the transmitter 7006a as a part of its components, the weight, the body number, and the like. Information such as the number, the situation of a place that is difficult to observe from the receiver, and the presence or absence of danger may be included.

The transmitter 7006a transmits the ID (7106b). The transmission content may include the URL of the server and information stored in the server.

The receiver receives the information such as the transmitted ID (7106c). The receiver acquires information associated with the received ID from the server (7106d). The receiver displays the received information and the information acquired from the server (7106e).

The receiver uses the size information of the light emitting unit and the size of the image of the light emitting unit that is imaged, or the distance between the light emitting units and the distance between the imaged light emitting units in the manner of triangulation. The distance to and is calculated (7106f). The receiver issues a warning of danger based on information such as the situation of the place that is difficult to observe from the receiver and the presence/absence of danger (7106g).

FIG. 429 is a diagram illustrating an example of application of the receiver and the transmitter in Embodiment 33.

For example, a transmitter (car) 7007b having two tail lamps (light emitting units or lights) of a car transmits information of the transmitter 7007b to a receiver 7007a configured as a transceiver of a parking lot, for example. The information of the transmitter 7007b indicates the identification information (ID) of the transmitter 7007b, the number of the car, the size of the car, the shape of the car, or the weight of the car. Upon receiving the information, the receiver 7007a transmits parking permission, billing information, or a parking position. The receiver 7007a may receive the ID and acquire information other than the ID from the server.

FIG. 430 is a flowchart illustrating an example of process operations of the receiver 7007a and the transmitter 7007b in Embodiment 33. Note that the transmitter 7007b includes an in-vehicle transmitter and an in-vehicle receiver because the transmitter 7007b performs not only transmission but also reception.

The ID of the transmitter 7007b and the information passed to the receiver 7007a that has received the ID are associated and stored in the server (parking lot management server) (7107a). The information to be passed to the receiver 7007a includes the shape of an object including the transmitter 7007b as a component, the weight, an identification number such as a vehicle body number, the identification number of the user of the transmitter 7007b, and information for payment. May be included.

The transmitter 7007b (vehicle-mounted transmitter) transmits the ID (7107b). The transmission content may include the URL of the server and information stored in the server. The parking lot receiver 7007a (parking lot transmission/reception device) transmits the received information to the server that manages the parking lot (parking lot management server) (7107c). The parking lot management server acquires the information associated with the ID by using the ID of the transmitter 7007b as a key (7107d). The parking lot management server investigates the availability of the parking lot (7107e).

The parking lot receiver 7007a (parking lot transmission/reception device) transmits whether or not parking is possible, parking position information, or an address of a server holding such information (7107f). Alternatively, the parking lot management server transmits these pieces of information to another server. The transmitter (vehicle-mounted receiver) 7007b receives the information transmitted above (7107g). Alternatively, the in-vehicle system acquires these pieces of information from another server.

The parking lot management server controls the parking lot to facilitate parking (7107h). For example, control of a multilevel parking lot is performed. The transmitting/receiving device in the parking lot transmits the ID (7107i). The vehicle-mounted receiver (transmitter 7007b) makes an inquiry to the parking lot management server based on the user information of the vehicle-mounted receiver and the received ID (7107j).

The parking lot management server charges according to the parking time and the like (7107k). The parking lot management server controls the parking lot so that the parked vehicle can be easily accessed (7107 m). For example, control of a multilevel parking lot is performed. The vehicle-mounted receiver (transmitter 7007b) displays a map to the parking position and performs navigation from the current position (7107n).

(On the train)
431] FIG. 431 is a diagram illustrating a structure of a visible light communication system applied to the inside of a train in Embodiment 33. [FIG.

The visible light communication system includes, for example, a plurality of lighting devices 1905 arranged in a train, a smartphone 1906 held by a user, a server 1904, and a camera 1903 arranged in the train.

Each of the plurality of lighting devices 1905 is configured as the above-described transmitter, illuminates a light, and transmits a visible light signal by changing the brightness. This visible light signal indicates the ID of the lighting device 1905 that transmits the visible light signal.

The smartphone 1906 is configured as the above-described receiver, and by capturing an image of the lighting device 1905, the visible light signal transmitted from the lighting device 1905 is received. For example, the user causes the smartphone 1906 to receive the visible light signal when a trouble (for example, a molester or a quarrel) is involved in the train. Upon receiving the visible light signal, the smartphone 1906 notifies the server 1904 of the ID indicated by the visible light signal.

Upon receiving the notification of the ID, the server 1904 identifies the camera 1903 whose imaging range is the range illuminated by the lighting device 1905 identified by the ID. Then, the server 1904 causes the specified camera 1903 to capture an image of the range illuminated by the lighting device 1905.

The camera 1903 captures an image according to an instruction from the server 1904, and transmits the image obtained by the capture to the server 1904.

As a result, it is possible to obtain an image showing the situation of trouble on the train. This image can be used as evidence of trouble.

In addition, the user may operate the smartphone 1906 to cause the server 1904 to transmit the image obtained by the image capturing by the camera 1903 to the smartphone 1906.

In addition, the smartphone 1906 may display an image capturing button on the screen and, when the image capturing button is touched by the user, may transmit a signal prompting the image capturing to the server 1904. As a result, the user can determine the timing of image capturing by himself/herself.

FIG. 432 is a diagram showing the configuration of the visible light communication system applied to a facility such as an amusement park in the thirty-third embodiment.

The visible light communication system includes, for example, a plurality of cameras 1903 arranged in a facility and an accessory 1907 attached to a person.

The accessory 1907 is, for example, a headband having a ribbon to which a plurality of LEDs are attached. The accessory 1907 is configured as the above-mentioned transmitter, and transmits the visible light signal by changing the brightness of the plurality of LEDs.

Each of the plurality of cameras 1903 is configured as the above-mentioned receiver and has a visible light communication mode and a normal imaging mode. In addition, each of the plurality of cameras 1903 is arranged at different points on the passage in the facility.

Specifically, the camera 1903 receives a visible light signal from the accessory 1907 when the accessory 1907 is imaged as a subject when the visible light communication mode is set. Upon receiving the visible light signal, the camera 1903 switches the set mode from the visible light communication mode to the normal imaging mode. As a result, the camera 1903 images a person wearing the accessory 1907 as a subject.

Therefore, when a person wearing the accessory 1907 is walking along the path in the facility, the cameras 1903 near the person will take images of the person one after another. As a result, it is possible to automatically acquire and save an image showing how the person is having fun at the facility.

Note that the camera 1903 may not perform imaging in the normal imaging mode immediately after receiving the visible light signal, but may perform imaging in the normal imaging mode when receiving an instruction to start imaging from a smartphone, for example. Accordingly, the user can cause the camera 1903 to capture an image of himself/herself at the timing of touching the image capturing start button displayed on the screen of the smartphone.

FIG. 433 is a diagram illustrating an example of a visible light communication system including a playground equipment and a smartphone in Embodiment 33.

The play equipment 1901 is configured as, for example, the above-described transmitter including a plurality of LEDs. That is, the play equipment 1901 transmits a visible light signal by changing the brightness of the plurality of LEDs.

The smartphone 1902 receives the visible light signal transmitted from the play equipment 1901 by imaging the play equipment 1901. Then, as illustrated in (a) of FIG. 433, when the smartphone 1902 receives the visible light signal for the first time, the smartphone 1902 receives the moving image 1 associated with the visible light signal from the first time from, for example, a server. Download and play. On the other hand, when the smartphone 1902 receives the visible light signal for the second time, as illustrated in (b) of FIG. 433, the smartphone 1902 receives the moving image 2 associated with the visible light signal and the second time from, for example, the server. Download and play.

That is, even if the smartphone 1902 receives the same visible light signal, the smartphone 1902 switches the moving image to be reproduced according to the number of times the visible light signal is received. The number of times the visible light signal is received may be counted by the smartphone 1902 or the server. Alternatively, the smartphone 1902 does not continuously reproduce the same moving image even if the same visible light signal is received multiple times. Alternatively, the smartphone 1902 lowers the appearance probability of an already played video among a plurality of videos associated with the same visible light signal, and preferentially downloads and plays a video with a high appearance probability. May be.

In addition, the smartphone 1902 may receive a visible light signal transmitted from a touch panel provided at an information center of a facility having a plurality of stores and display an image corresponding to the visible light signal. For example, when the touch panel is displaying an initial screen showing the outline of the facility, it transmits a visible light signal showing the outline of the facility by changing the brightness. Therefore, when the smartphone receives the visible light signal by capturing an image of the touch panel displaying the initial screen, the smartphone can display an image showing the outline of the facility on its own display. Here, when the touch panel is operated by the user, the touch panel displays, for example, a store image showing information on a specific store. At this time, the touch panel is transmitting a visible light signal indicating information on the specific store. Therefore, when the smartphone receives the visible light signal by capturing an image of the touch panel displaying the store image, the smartphone can display the store image indicating the information of the specific store. In this way, the smartphone can display the image synchronized with the touch panel.

(Summary of the above embodiment)
A reproducing method according to one aspect of the present invention includes a signal receiving step of receiving a visible light signal by a sensor of a terminal device from a transmitter that transmits a visible light signal according to a luminance change of a light source, and the visible light signal from the terminal device. A transmitting step of transmitting a request signal for requesting content associated with an optical signal to a server, the terminal device provides content including each time and data reproduced at each time from the server. It includes a content receiving step of receiving, and a reproducing step of reproducing data corresponding to time of a clock provided in the terminal device among the contents.

As a result, as shown in FIG. 409C, the content including each time and the data reproduced at each time is received by the terminal device, and the data corresponding to the time of the clock of the terminal device is reproduced. Therefore, the terminal device can appropriately reproduce the data in the content at the correct time shown in the content without reproducing the data at the wrong time. Specifically, as in method e in FIG. 409A, the receiver, which is the terminal device, reproduces the content from the time point of (receiver time-content reproduction start time). The data corresponding to the time of the clock of the above-mentioned terminal device is the data at the time of (receiver time-content reproduction start time) in the content. Further, also in the transmitter, if the content related to the content (transmitter-side content) is reproduced, the terminal device can reproduce the content in proper synchronization with the transmitter-side content. The content is a voice or an image.

Further, the clock provided in the terminal device and the reference clock may be synchronized by a GPS (Global Positioning System) radio wave or an NTP (Network Time Protocol) radio wave.

As a result, as shown in FIGS. 408 and 410, since the clock of the terminal device (receiver) and the reference clock are synchronized, an appropriate time according to the reference clock corresponds to that time. You can play back the data that you want.

Further, the visible light signal may indicate a time when the visible light signal is transmitted from the transmitter.

As a result, the terminal device (receiver) can receive the content associated with the time when the visible light signal is transmitted from the transmitter (transmitter time), as shown in the method d in FIG. 409A. For example, if the transmitter time is 5:43, the content reproduced at 5:43 can be received.

Further, in the reproducing method, the time when the process for synchronizing the clock of the terminal device and the reference clock is performed by the GPS radio wave or the NTP radio wave is the visible time of the terminal device. If it is before a predetermined time from the time of receiving the optical signal, the clock of the terminal device and the clock of the transmitter are synchronized by the time indicated by the visible light signal transmitted from the transmitter. Very good.

For example, if a predetermined time has elapsed after the processing for synchronizing the clock of the terminal device and the reference clock, the synchronization may not be properly maintained. In such a case, the terminal device may not be able to reproduce the content at the time synchronized with the transmitter-side content reproduced by the transmitter. Therefore, in the reproducing method according to one embodiment of the present invention, when a predetermined time elapses, as in steps S1829 and S1830 in FIG. 408, the time is set between the clock of the terminal device (receiver) and the clock of the transmitter. Synchronized. Therefore, the terminal device can reproduce the content at the time synchronized with the transmitter-side content reproduced by the transmitter.

Further, the server has a plurality of contents that are respectively associated with time, and in the content receiving step, when the content associated with the time indicated by the visible light signal does not exist in the server, Among the plurality of contents, the contents closest to the time indicated by the visible light signal and associated with the time after the time indicated by the visible light signal may be received.

As a result, as shown in the method d of FIG. 409A, even if the content associated with the time indicated by the visible light signal does not exist in the server, an appropriate content is received from the plurality of contents in the server. be able to.

In addition, a signal receiving step of receiving the visible light signal by a sensor of the terminal device from a transmitter that transmits the visible light signal according to a change in the brightness of the light source, and content associated with the visible light signal from the terminal device. A transmission step of transmitting a request signal for requesting to a server, the terminal device includes a content reception step of receiving content from the server, and a reproduction step of reproducing the content, wherein the visible light signal is ID information and a time when the visible light signal is transmitted from the transmitter are indicated, and in the content receiving step, even if the content associated with the ID information and the time indicated by the visible light signal is received. Good.

As a result, like method d in FIG. 409A, the visible light signal is associated with the time (transmitter time) at which the visible light signal is transmitted from the transmitter from among the plurality of contents associated with the ID information (transmitter ID). The received content is received and played. Therefore, it is possible to reproduce the content appropriate for the transmitter ID and the transmitter time.

Further, the visible light signal includes second information indicating hours and minutes of time and first information indicating seconds of time, so that the visible light signal is transmitted from the transmitter. In the signal receiving step, the second information may be received, and the first information may be received more times than the number of times the second information is received.

Thus, for example, when notifying the terminal device of the time at which each packet included in the visible light signal is transmitted, in units of seconds, a packet indicating the current time represented using all of hours, minutes, and seconds. It is possible to reduce the time and effort required to transmit every 1 second to the terminal device. That is, as shown in FIG. 404, if the hour and minute of the time at which the packet is transmitted has not been updated from the hour and minute indicated in the previously transmitted packet, a packet indicating only seconds (time packet 1 ) Need only be transmitted. Therefore, by reducing the second information transmitted by the transmitter, which is the packet indicating the second (time packet 1), which is the packet indicating the hour and minute (time packet 2), It is possible to suppress the transmission of packets containing redundant contents.

Further, the sensor of the terminal device is an image sensor, and in the signal receiving step, the shutter speed of the image sensor is set to a first speed and a second speed that is higher than the first speed. While alternately switching, continuous shooting with the image sensor is performed, and (a) when the subject of the shooting with the image sensor is a barcode, by shooting when the shutter speed is the first speed, A bar code identifier is obtained by acquiring an image in which the bar code is reflected and decoding the bar code in the image, and (b) in the case where the object photographed by the image sensor is the light source. By capturing when the shutter speed is the second speed, a bright line image that is an image including bright lines corresponding to each of a plurality of exposure lines included in the image sensor is acquired, and the acquired bright line image is obtained. The visible light signal is obtained as a visible light identifier by decoding a pattern of a plurality of included bright lines, and in the reproducing method, an image obtained by photographing when the shutter speed is the first speed is further obtained. It may be displayed.

As a result, as shown in FIG. 309, it is possible to appropriately obtain an identifier corresponding to the barcode or the visible light signal, and to display an image in which the barcode or the light source of the subject is projected. Can be displayed.

In the acquisition of the visible light identifier, a first packet including a data part and an address part is acquired from the plurality of bright line patterns, and at least one of the packets already acquired before the first packet is acquired. It is determined whether or not there are a predetermined number or more of second packets, which are packets including the same address portion as the first packet, among the packets, and the second packet has the predetermined number. If it is determined that there is more than one, the pixel values of the partial area of the bright line image corresponding to the respective data parts of the second packet of the predetermined number or more and the data part of the first packet At least a part of the visible light identifier is obtained by calculating a composite pixel value by matching the pixel value of a partial area of the corresponding bright line image and decoding a data part including the composite pixel value. You may.

As a result, as shown in FIG. 281, even if the data parts of a plurality of packets including the same address part are slightly different, by matching the pixel values of the data parts of those packets, an appropriate data part can be obtained. It can be decoded and at least a part of the visible light identifier can be correctly acquired.

Further, the first packet further includes a first error correction code for the data part and a second error correction code for the address part, and in the signal receiving step, the second packet from the transmitter is used. Of the address portion and the second error correction code transmitted by the luminance change according to the frequency of, and the data portion transmitted by the luminance change according to the first frequency higher than the second frequency. And the first error correction code may be received.

As a result, as shown in FIG. 279, it is possible to prevent erroneous reception of the address part and quickly acquire the data part having a large amount of data.

In the acquisition of the visible light identifier, a first packet including a data part and an address part is acquired from the plurality of bright line patterns, and at least one of the packets already acquired before the first packet is acquired. Of the packets, it is determined whether there is at least one second packet that is a packet including the same address part as the first packet, and if there is at least one second packet, If it is determined, it is determined whether or not the respective data portions of the at least one second packet and the first packet are all equal, and if it is determined that the respective data portions are not all equal. In each of the at least one second packet, the number of different parts of the parts included in the data part of the second packet from the parts included in the data part of the first packet is If there is a second packet in which it is determined whether or not there is a predetermined number or more, and it is determined that the number of different portions of the at least one second packet is the predetermined number or more, At least one second packet is discarded, and if there is no second packet determined to have the number of different portions of the at least one second packet equal to or more than the predetermined number, the first packet Among the packets and the at least one second packet, a plurality of packets having the largest number of packets having the same data part is specified, and the data part included in each of the plurality of packets is defined as the first packet. At least a part of the visible light identifier may be obtained by decoding the data part corresponding to the address part included in the.

Thus, as shown in FIG. 280, when a plurality of packets having the same address part are received, the appropriate data part can be decoded even if the data parts of the packets are different, and At least a part of the optical identifier can be correctly acquired. That is, a plurality of packets having the same address part transmitted from the same transmitter basically have the same data part. However, when the terminal device switches the transmitter that is the source of the packet, the terminal device may receive a plurality of packets having different data parts even though they have the same address part. In such a case, in the reproducing method according to the above aspect of the present invention, as in step S10106 in FIG. 280, the already received packet (second packet) is discarded and the latest packet (first packet) is deleted. Data part of the packet) can be decoded as a correct data part corresponding to the address part. Further, even if the transmitter is not switched as described above, the data parts of a plurality of packets having the same address part may be slightly different depending on the transmission/reception status of the visible light signal. In such a case, in the reproducing method according to the above aspect of the present invention, an appropriate data part can be decoded by what is called a majority decision, as in step S10107 of FIG. 280.

Further, in the acquisition of the visible light identifier, from the plurality of bright line patterns, a plurality of packets each including a data part and an address part are acquired, and among the plurality of acquired packets, all included in the data part It is determined whether or not there is a 0-terminating packet that is a packet in which the bit of 0 indicates 0. It is determined whether or not all N (N is an integer of 1 or more) related packets that are packets including the associated address part are present, and if it is determined that all of the N related packets are present. , The visible light identifier may be obtained by arranging and decoding the respective data parts of the N related packets. For example, the address part associated with the address part of the 0-terminated packet is an address part that is smaller than the address indicated by the address part of the 0-terminated packet and indicates an address of 0 or more.

Specifically, as shown in FIG. 282, it is determined whether or not all packets having an address less than or equal to the address of the 0-end packet are prepared as related packets. Each data part of the packet is decoded. As a result, the terminal device does not need to know in advance how many related packets are needed to obtain the visible light identifier, and furthermore, does not know the address of those related packets in advance. Also, it can be easily known when the 0-terminal packet is acquired. As a result, the terminal device can obtain an appropriate visible light identifier by arranging and decoding the data parts of the N related packets.

(Embodiment 34)
The variable length/variable division number compatible protocol will be described below.

434 to 438 are diagrams showing an example of the transmission signal in the present embodiment.

The transmission packet is composed of a preamble, a TYPE, a payload, and a check unit. The packets may be transmitted continuously or intermittently. By providing a period during which packets are not transmitted, it is possible to change the state of the liquid crystal when the backlight is turned off and to improve the dynamic resolution of the liquid crystal display. Interference can be avoided by making the packet transmission interval random.

A pattern that does not appear in 4PPM is used as the preamble. The reception process can be simplified by using a short basic pattern. As shown in (b) and (c) of FIG. 436, when the basic pattern is arranged at the back, the packet can be received even when the short section is received. As shown in (c) of FIG. 436, the preamble can be accurately received by setting the start and end of the preamble to 1 (state with high brightness).

By expressing the number of data divisions by the type of preamble, the number of data divisions can be made variable without using extra transmission slots.

By changing the payload length according to the value of TYPE, it is possible to make the transmission data variable length. In TYPE, the payload length may be expressed or the data length before division may be expressed. By expressing the address of the packet by the value of TYPE, the receiver can correctly arrange the received packets. Since the required address length differs depending on the number of divisions, the length of TYPE may be changed according to the number of divisions, as shown in (c) of FIG. 437. Also, the payload length (data length) represented by the TYPE value may be changed depending on the type of preamble or the number of divisions.

Efficient error correction (detection) can be performed by changing the length of the check unit according to the payload length. By setting the minimum length of the check section to 2 bits, it is possible to efficiently convert to 4PPM. Also, by changing the type of error correction (detection) code depending on the payload length, error correction (detection) can be performed efficiently. The length of the check section or the type of error correction (detection) code may be changed depending on the type of preamble or the value of TYPE.

There are combinations that have the same data length with different combinations of payload and division number. In such a case, even if the same data value is used, different values can be given to each combination to represent more values.

The high-speed transmission/intensity modulation protocol will be described below.

439 and 440 are diagrams showing an example of a transmission signal in this embodiment.

The transmission packet is composed of a preamble part, a body part, and a brightness adjusting part. The body includes an address part, a data part, and an error correction (detection) code part. By permitting intermittent transmission, the same effect as described above can be obtained.

As shown in FIG. 440, by combining the brightness changes of the two types of average brightness preamble parts, the three types of average brightness body parts, and the brightness adjustment part capable of continuously changing the average brightness, The average brightness can be continuously changed as a whole. The continuous brightness change of the brightness adjusting unit can be realized by changing the maximum brightness or changing the ratio of the time between the bright period and the dark period. Since there are two kinds of expression methods near the luminance of 50%, it is possible to use them separately with 50% as a boundary, and by expressing different data in the two kinds of expression methods, more values can be expressed. it can.

At the time of reception, maximum likelihood decoding is performed assuming four combinations of the preamble part and the body part, and the pattern with the highest likelihood is used as the received signal, so that the signal even if the dimming state is unknown. Can be received.

The preamble portion has a small change in luminance of two types, and since it is a signal detection start portion, a signal can be efficiently received by defining the luminance reference at this portion.

The 4PPM convolutional decoding pattern will be described below.

441 to 444 are diagrams showing an example of the reception algorithm in the present embodiment.

When the convolution length is 3 and the preamble in FIG. 440 is used, 1 and 0 are 3 consecutive, so that the preamble can be detected as it is without performing the convolutional decoding.

At each luminance of 4 PPM, the convolution state transits as shown in FIGS. 441 to 444. By performing maximum likelihood estimation in each of these patterns, signals can be received while performing convolutional decoding. The start states “000” and “111” in FIG. 444 do not exist in the end state, but the end state of the preamble is one of these states, so this pattern is used in maximum likelihood decoding of the first 4 slots of the body part. By using it, maximum likelihood decoding can be performed accurately.

Here, as the sensitivity is set higher or the exposure time is set shorter, noise associated with imaging increases. Therefore, noise is reduced by increasing the number of pixels used to obtain the average luminance in the exposure line. can do. When the sensitivity is increased by N times, the noise amount can be suppressed to the same level by setting the average number of pixels to be N times the square. On the contrary, when the sensitivity is set low and the exposure time is set long, the calculation load can be reduced by reducing the number of pixels to be averaged.

When the value obtained by dividing the maximum likelihood path likelihood at the time of performing maximum likelihood decoding by the path length is lower than a predetermined value, the decoding result is unreliable and the decoded signal is discarded. As a result, reception errors can be reduced.

If the likelihood at the time of maximum likelihood decoding is low or if an error is detected, correct decoding may be performed by performing the decoding process again with the average number of pixels increased and the noise reduced. . Also, in the same case, if the pixel value is high, the sensitivity may be lowered or the exposure time may be shortened to correctly receive the signal, and if the pixel value is low, the sensitivity may be lowered. Correct reception may be possible by increasing the length or lengthening the exposure time.

When the sampling rate of the receiver (=the reciprocal of the time difference of exposure timing between adjacent exposure lines) is f Hertz, the transmission frequency is Nf/2+k Hertz. However, N is an integer and k<f/2. At this time, when frequency analysis is performed on the received signal, no high frequency component is observed and k hertz is obtained as an alias. Therefore, for example, when representing four values, k={0,
By assigning f/4, f/2, f×3/4}, the receiver can identify each signal. At this time, the frequency becomes higher than that when only k hertz is used as the transmission signal, so that effects such as suppressing flicker can be obtained. For example, a frequency of several kilohertz to several tens of kilohertz causes a reading error of the bar code reader, but by using a higher frequency, this problem can be avoided.

Since the bar code reader reads the reflected light of the red ray, by removing the red component from the visible light communication signal, it is possible to remove the adverse effect on the reading of the bar code reader. In order to remove the red component from the visible light communication signal, there is a method of emitting red light having a phase opposite to that of the signal, or using a filter or a phosphor that increases the afterglow of a component having a long wavelength. The receiver can correctly receive the signal by receiving the signal from the brightness of the light of the green or blue component.

Depending on the user's last action on the transmitter, the transmitter changes the signal it transmits. As a result, it is possible to send information on a phenomenon that has occurred as a result of the operation, or to instruct a signal that the user wants to send.

The transmitter transmits a signal only for a predetermined time since the user last performed an operation. As a result, power consumption can be suppressed and the timer of the microcomputer can be assigned to another function.

The transmitter sends a signal only when the microcomputer timer is not used for another function. Alternatively, the transmitter alternately executes the function of using the timer of the microcomputer including signal transmission. As a result, the transmitter can be configured with a small number of timers.

The receiver performs different operations only for a predetermined time after receiving predetermined data by visible light communication. For example, within one hour after receiving the ID from the advertising signboard, the product or service can be discounted, or the campaign item used for the game can be downloaded. As a result, the offline-to-online service or the offline-to-online-to-offline service can be realized.

The receiver receives the setting information from the transmitter and stores the setting information in the storage in the receiver in association with the identification information of the transmitter. The receiver receives the identification information from the same or another transmitter and sets the associated setting information in the transmitter. The setting information may be, for example, the language setting of the receiver other than the information received from the transmitter. This allows the transmitter settings to be quickly restored. In addition, it is possible to make settings suitable for the user in the transmitter without inputting by the user.

The receiver normally takes an image and displays it on the preview screen during the reception process. As a result, a smoother preview can be displayed without deteriorating the reception performance.

The receiver continuously or intermittently captures a predetermined number of visible light images, stores them in the memory until the processing of the captured image of the previous frame is completed, and sequentially performs the reception processing. This allows the reception to be completed even when the receiver is short on the transmitter.

(Embodiment 35)
(Frame structure of Single frame transmission)
445 and 446 are diagrams showing an example of a transmission signal in this embodiment.

The transmission frame includes a preamble (PRE), a frame length (FLEN), an ID type (IDTYPE), a content (ID/DATA), and a check code (CRC), and may include a content type (CONTENTTYPE). The number of bits in each area is an example.

By specifying the length of ID/DATA with FLEN, it is possible to transmit variable length contents.

CRC is a check code for correcting or detecting an error in a portion other than PRE. By changing the CRC length according to the length of the inspection area, it is possible to maintain the inspection ability above a certain level. Also, by using different check codes depending on the length of the check area, it is possible to improve the check capability per CRC length.

As in (e) of FIG. 446, when CONTENTTYPE has a predetermined bit, it indicates that ID/DATA is an ID, and when it is not the predetermined bit, it indicates that ID/DATA is data. Further, when CONTENTTYPE is not the predetermined bit, the IDTYPE area is also set to the ID/DATA area, so that a larger data capacity can be transmitted.

As shown in (f) and (g) of FIG. 446, by changing the IDTYPE length according to the ID/DATA length, it is possible to define an appropriate amount of ID type according to the ID/DATA length. For example, when the ID/DATA length has a large number of types of ID, many IDs can be defined by lengthening the IDTYPE length. Alternatively, if the ID/DATA length is short and easy to use, many ID systems can be defined by increasing the IDTYPE length. Alternatively, by setting the IDTYPE length to 0 for a specific ID/DATA length, the length of the entire frame can be shortened, and quick transmission/reception and long-distance communication can be performed.

(Frame structure of Multiple frame transmission)
FIG. 447 is a diagram illustrating an example of a transmission signal in this embodiment.

The transmission frame includes a preamble (PRE), an address (ADDR), and a part of divided data (DATAPART), and may include a division number (PARTNUM) and an address flag (ADDRFRAG). The number of bits in each area is an example.

Long-distance communication can be performed by dividing the content into a plurality of parts and transmitting them.

By making the size of division into equal parts, the maximum frame length can be reduced and stable communication can be performed.

If equal division is not possible, by making some of the divided portions smaller than other divided portions, it is possible to transmit data of an appropriate size.

More information can be transmitted by making the sizes of division different and making the combination of division sizes meaningful. For example, even if the data has the same value of 32 bits, when 8 bits are transmitted 4 times, when 16 bits are transmitted twice, and when 15 bits are transmitted once and 17 bits are transmitted once. By treating them as different information, it is possible to express a larger amount of information.

By indicating the number of divisions in PARTNUM, the receiver can immediately know the number of divisions and can accurately display the progress of reception.

When ADDRFRAG is 0, it is not the last address, and when it is 1, it is the last address, so that the area indicating the number of divisions is unnecessary, and transmission can be performed in a shorter time.

The CRC is a check code that corrects or detects an error in a portion other than the PRE, as in the above. By this inspection, it is possible to detect interference when receiving transmission frames from a plurality of transmission sources. By making the CRC length an integral multiple of the DATAPART length, the interference can be detected most efficiently.

A check code for checking a portion other than the PRE of each frame may be added to the end of the divided frame (frame indicated by (a), (b), or (c) in FIG. 447).

The IDTYPE shown by (d) of FIG. 447 may have a fixed length such as 4 bits or 5 bits as in (a) to (d) of FIG. 445, or may be a fixed length such as (f) and (g) of FIG. 446. Alternatively, the IDTYPE length may be changed according to the ID/DATA length. As a result, the same effect as described above can be obtained.

(Specification of ID/DATA length)
FIG. 448 and FIG. 449 are diagrams showing an example of a transmission signal in this embodiment.

In the case of (a) to (d) of FIG. 445, by setting like the tables (a) and (b) shown in FIG. 448 and the tables (a) and (b) shown in FIG. Sometimes ucode can be represented.

(CRC length and generator polynomial)
FIG. 450 is a diagram showing an example of a transmission signal in this embodiment.

By setting the CRC length in this way, the inspection capability can be maintained regardless of the length of the inspection target.

The generator polynomial is an example, and another generator polynomial may be used. Also, check codes other than CRC may be used. With these, the inspection capability can be improved.

(Specification of DATAPART length and last address depending on preamble type)
FIG. 451 is a diagram illustrating an example of a transmission signal in this embodiment.

By indicating the DATAPART length by the type of preamble, the area indicating the DATAPART length is not needed, and information can be transmitted in a shorter transmission time. Further, by indicating whether or not it is the last address, the area indicating the number of divisions is not needed, and information can be transmitted in a shorter transmission time. In the case of (b) of FIG. 451, the DATAPART length is not known in the case of the last address, so it is estimated that it is the same as the DATAPART length of the frame that is not the last address received immediately before or after the reception of the frame. By performing the reception process by the reception, normal reception can be performed.

The address length may be different depending on the type of preamble. As a result, it is possible to increase the number of combinations of transmission information lengths and to transmit in a short time.

In the case of (c) in FIG. 451, the number of divisions is defined by the preamble and an area indicating the DATAPART length is added.

(Specify address)
FIG. 452 is a diagram illustrating an example of a transmission signal in this embodiment.

By indicating the address of that frame with the value of ADDR, the receiver can reconstruct the correctly transmitted information.

By indicating the number of divisions with the value of PARTNUM, the receiver can always know the number of divisions at the time of receiving the first frame, and can accurately display the progress of reception.

(Prevention of interference due to the difference in the number of divisions)
453 and 454 are a diagram and a flow chart showing an example of the transmission/reception system in the present embodiment.

When the transmission information is divided into equal parts and dividedly transmitted, the signals from the transmitter A and the transmitter B in FIG. 453 have different preambles, and therefore even when these signals are received at the same time, the receiver confuses the transmission source. It is possible to reconstruct the transmitted information without.

Since the transmitters A and B are provided with the division number setting unit, the user can set the division numbers of the transmitters installed nearby so as to be different from each other and prevent interference.

By registering the division number of the received signal in the server, the receiver can know the division number set for the transmitter, and other receivers can obtain the information from the server, The progress of reception can be displayed accurately.

The receiver obtains from the server or from the storage unit of the receiver whether the signal from the nearby transmitter or the corresponding transmitter is equal-length division. When the acquired information is equal length division, the signal is restored only from the frames of the same DATAPART length. If this is not the case, or if the situation in which all addresses are not aligned in a frame with the same DATAPART length continues for a predetermined time or longer, frames with different DATAPART lengths are combined to restore the signal.

(Prevention of interference due to the difference in the number of divisions)
FIG. 455 is a flowchart showing the operation of the server in this embodiment.

The server receives from the receiver the ID received by the receiver and the division configuration (what combination of DATAPART lengths the signal was received in). When the ID is an object of expansion by the split configuration, a numerical value of the pattern of the split configuration is used as the auxiliary ID, and the associated information is set using the extended ID including the ID and the auxiliary ID as a key. Hand over to.

If it is not the target of expansion by the division configuration, it is confirmed whether the division configuration associated with the ID exists in the storage unit, and whether it is the same as the received division configuration. If they are different, a reconfirmation command is sent to the receiver. This can prevent erroneous information from being presented due to a reception error of the receiver.

When the same split configuration with the same ID is received within a predetermined time after transmitting the reconfirmation command, it is determined that the split configuration has been changed, and the split configuration associated with the ID is updated. Accordingly, as described as the description of FIG. 453, it is possible to deal with the case where the division configuration is changed.

When the division configuration is not stored, when the received division configuration matches the stored division configuration, or when the division configuration is updated, the associated information is passed to the receiver by using the ID as a key, The division configuration is stored in the storage unit in association with the ID.

(Display of reception progress status)
456 to 461 are a flowchart and a diagram illustrating an example of operation of the receiver in this embodiment.

The receiver acquires, from the storage area of the server or the receiver, the type and ratio of the number of divisions of the transmitter to which the receiver corresponds or the transmitter near the receiver. If some of the divided data has already been received, the type and ratio of the number of divisions of the transmitter that is transmitting the information that matches some of the divided data is acquired.

The receiver receives the divided frames.

If the last address has already been received, the acquired number of divisions is only one type, or if the number of divisions supported by the receiving application being executed is only one, the number of divisions is Since it is known, the progress status is displayed based on the number of divisions.

Otherwise, if there are few available processing resources or in the energy saving mode, the receiver calculates and displays the progress status in the simple mode. On the other hand, when there are many available processing resources or the mode is not the energy saving mode, the progress status is calculated and displayed in the maximum likelihood estimation mode.

FIG. 457 is a flowchart showing a method of calculating the progress status in the simple mode.

First, the receiver acquires the standard division number Ns from the server. Alternatively, the receiver reads the standard division number Ns from its own data holding unit. The standard number of divisions is (a) the mode or expected value of the number of transmitters transmitting at that number of divisions, (b) the number of divisions determined for each packet length, (c) the number of divisions determined for each application. The number, or (d) the location where the receiver is located, is the number of divisions defined for each identifiable range.

Next, the receiver determines whether or not the packet indicating the final address is received. When it is determined that the packet is received, the address of the final packet is set to N. On the other hand, when it is determined that it has not been received, Ne is a number obtained by adding 1 or 2 or more to the maximum received address Amax. Here, the receiver determines whether Ne>Ns. Upon determining that Ne>Ns, the receiver sets N=Ne. On the other hand, when it is determined that Ne>Ns is not established, the receiver sets N=Ns.

Then, assuming that the number of divisions of the signal being received is N, the receiver calculates the ratio of the number of received packets among the packets necessary for receiving the entire signal.

In such a simple mode, the progress status can be calculated by a simpler calculation than in the maximum likelihood estimation mode, which is advantageous in terms of processing time or energy consumption.

FIG. 458 is a flowchart showing a method of calculating the progress status in the maximum likelihood estimation mode.

First, the receiver acquires the prior distribution of the number of divisions from the server. Alternatively, the receiver reads the prior distribution from the data holding unit inside itself. The prior distribution is (a) defined as a distribution of the number of transmitters to be transmitted by the number of divisions, (b) defined for each packet length, (c) defined for each application, or , (D) It is a place where the receiver is located and is defined for each identifiable range.

Next, the receiver receives the packet x and calculates the probability P(x|y) of receiving the packet x when the division number is y. Then, the receiver obtains the probability P(y|x) that the number of divisions of the transmission signal is y when the packet x is received, as P(x|y)×P(y)÷A (note that A Is a normalized multiplier). Further, the receiver sets P(y)=P(y|x).

Here, the receiver determines whether the division number estimation mode is the maximum likelihood mode or the likelihood average mode. In the maximum likelihood mode, the receiver calculates the ratio of the number of received packets with y having the maximum P(y) as the division number. On the other hand, in the likelihood average mode, the receiver calculates the ratio of the number of already received packets with the sum of y×P(y) as the number of divisions.

In such maximum likelihood estimation mode, the progress degree can be calculated more accurately than in the simple mode.

If the division number estimation mode is the maximum likelihood mode, the likelihood of what the last address is from the addresses received so far is calculated, and the maximum likelihood is estimated to be the division number. Display the progress of reception. This display method can display the progress status closest to the actual progress status.

FIG. 459 is a flowchart showing a display method in which the progress status is not reduced.

First, the receiver calculates the ratio of the number of received packets to the packets required to receive the entire signal. Then, the receiver determines whether the calculated ratio is smaller than the ratio being displayed. If it is determined that it is smaller than the displayed ratio, the receiver further determines whether the displayed ratio is a calculation result before a predetermined time or more. When it is determined that the calculation result is equal to or more than the predetermined time, the receiver displays the calculated ratio. On the other hand, if it is determined that the calculation result is not a result of a predetermined time or more, the receiver continues to display the ratio being displayed.

Further, when the receiver determines that the calculated ratio is equal to or higher than the ratio being displayed, the receiver adds Ne to the number obtained by adding 1 or 2 or more to the maximum received address Amax. The receiver then displays the calculated percentage.

When the final packet is received, it is unnatural that the calculation result of the progress status is smaller than that, that is, the progress status (progress degree) displayed is reduced. However, with the above-described display method, such an unnatural display can be suppressed.

FIG. 460 is a flowchart showing a method of displaying the progress status when there are a plurality of packet lengths.

First, the receiver calculates the ratio P of the number of received packets for each packet length. Here, the receiver determines whether the display mode is the maximum mode, the full display mode, or the latest mode. When it is determined that the maximum mode is set, the receiver displays the maximum ratio of the respective ratios P of the plurality of packet lengths. When it is determined that the display mode is the full display mode, the receiver displays all the ratios P. When it is determined that the latest mode is set, the receiver displays the packet length ratio P of the last received packet.

In FIG. 461, (a) shows the progress calculated as the simple mode, (b) shows the progress calculated as the maximum likelihood mode, and (c) shows the smallest division number obtained as the division number. This is the progress situation. Since the progress increases in the order of (a), (b), and (c), by displaying (a), (b), and (c) in an overlapping manner in this manner, all the progress can be displayed at the same time. .

(Divided transmission)
FIG. 462 is a diagram showing an example of a transmission signal in this embodiment.

The check codes used in this frame structure include the two CRCs shown in FIGS. 445 to 447. By increasing the CRC length as the bit length of the portion to be checked increases, it is possible to prevent deterioration of the inspection capability.

The table (a) of FIG. 462 shows the case where the CRC length is a multiple of 4, and the table (b) of FIG. 462 shows the case where the CRC length is a multiple of 2. In 4PPM, since each 2 bits are encoded, a multiple of 2 can be allocated without waste. When the bit length to be inspected is N, the CRC length is defined so as not to fall below 2 log(N). Note that log is a logarithm with base 2. As a result, it is possible to set the inspection performance so as not to fall below a certain level. The numerical values in the table are examples, and another CRC length may be assigned or a CRC length other than a multiple of 2 or 4 may be used.

The generator polynomial in the table is an example, and another generator polynomial may be used. Also, instead of CRC, another check code may be used.

By dividing the check code, it is possible to perform some check even if the frame cannot be received to the end, and it is possible to reduce the error rate.

(Light emission control by common switch and pixel switch)
In the transmission method according to the present embodiment, for example, by changing the brightness of each LED included in the LED display for video display according to the switching of the common switch and the pixel switch, a visible light signal (also referred to as a visible light communication signal). Send).

The LED display is configured as, for example, a large display arranged outdoors. In addition, the LED display includes a plurality of LEDs arranged in a matrix, and displays an image by blinking these LEDs according to an image signal. Such an LED display includes a plurality of common lines (COM lines) and a plurality of pixel lines (SEG lines). Each common line is composed of a plurality of LEDs arranged in a line in the horizontal direction, and each pixel line is composed of a plurality of LEDs arranged in a line in the vertical direction. Further, each of the plurality of common lines is connected to the common switch corresponding to the common line. The common switch is, for example, a transistor. Each of the plurality of pixel lines is connected to the pixel switch corresponding to the pixel line. A plurality of pixel switches corresponding to a plurality of pixel lines are provided in, for example, an LED driver circuit (constant current circuit). The LED driver circuit is configured as a pixel switch control unit that switches a plurality of pixel switches.

More specifically, one of the anode and the cathode of each LED included in the common line is connected to a terminal such as the collector of the transistor corresponding to the common line. The other of the anode and the cathode of each LED included in the pixel line is connected to a terminal (pixel switch) corresponding to the pixel line in the LED driver circuit.

When such an LED display displays an image, a common switch control unit that controls a plurality of common switches turns on those common switches in a time division manner. For example, the common switch control unit turns on only the first common switch of the plurality of common switches during the first period, and the second common switch of the plurality of common switches during the next second period. Turn on only the switch. Then, the LED driver circuit turns on each pixel switch in accordance with the video signal while any of the common switches is turned on. As a result, the LEDs corresponding to the common switch and the pixel switch are lit only while the common switch is on and the pixel switch is on. The brightness of the pixels in the video is expressed by the lighting period. That is, the brightness of the image pixel is PWM-controlled.

In the transmission method according to the present embodiment, a visible light signal is transmitted using such an LED display, a common switch and a pixel switch, and a common switch control unit and a pixel switch control unit. The transmission device (also referred to as a transmitter) in the present embodiment that transmits a visible light signal by such a transmission method includes the common switch control unit and the pixel switch control unit.

FIG. 463 is a diagram showing an example of a transmission signal in this embodiment.

The transmitter transmits each symbol included in the visible light signal according to a predetermined symbol period. For example, when transmitting the symbol “00” by 4PPM, the transmitter switches the common switch according to the symbol (luminance change pattern of “00”) in the symbol period consisting of 4 slots. Then, the transmitter switches the pixel switch according to the average luminance indicated by the video signal or the like.

More specifically, when the average luminance in the symbol period is set to 75% ((a) in FIG. 463), the transmitter turns off the common switch during the period of the first slot, and the second slot to the fourth slot. Turn on the common switch for the duration of the slot. Further, the transmitter turns off the pixel switch during the first slot and turns on the pixel switch during the second slot to the fourth slot. As a result, the LEDs corresponding to the common switch and the pixel switch are lit only while the common switch is on and the pixel switch is on. In other words, the LEDs change in brightness by lighting at the brightness of LO (Low), HI (High), HI, and HI in each of the four slots. As a result, the symbol “00” is transmitted.

Further, when the average luminance in the symbol period is 25% ((e) in FIG. 463), the transmitter turns off the common switch during the period of the first slot, and during the period of the second slot to the fourth slot. , Turn on the common switch. Further, the transmitter turns off the pixel switch during the first slot, the third slot, and the fourth slot, and turns on the pixel switch during the second slot. As a result, the LEDs corresponding to the common switch and the pixel switch are lit only while the common switch is on and the pixel switch is on. That is, the LEDs change in brightness by lighting like LO (Low), HI (High), LO, and LO in each of the four slots. As a result, the symbol "00" is transmitted. Since the transmitter according to the present embodiment transmits a visible light signal close to the above-described V4PPM (variable 4PPM), the average luminance can be variable even when transmitting the same symbol. That is, when transmitting the same symbol (for example, “00”) with different average luminance, the transmitter, as shown in (a) to (e) of FIG. 463, has a rising position (timing) of the luminance peculiar to the symbol. ) Is constant regardless of the average brightness. This allows the receiver to receive the visible light signal without being aware of the brightness.

The common switch is switched by the common switch control unit described above, and the pixel switch is switched by the pixel switch control unit described above.

As described above, the transmission method according to the present embodiment is a transmission method that transmits a visible light signal according to a change in luminance, and includes a determination step, a common switch control step, and a first pixel switch control step. In the determining step, the brightness change pattern is determined by modulating the visible light signal. In the common switch control step, a common switch for commonly turning on a plurality of light sources (LEDs) included in a light source group (common line) included in the display for expressing pixels in an image is displayed. Switching is performed according to the change pattern. In the first pixel switch control step, the common switch is turned on by turning on the first pixel switch for turning on the first light source of the plurality of light sources included in the light source group, and , The visible light signal is transmitted by turning on the first light source only during the period when the first pixel switch is on.

Thereby, a visible light signal can be appropriately transmitted from a display including a plurality of LEDs as a light source. Therefore, it is possible to perform communication between devices other than the lighting device. When the display is a display for displaying an image by controlling the common switch and the first pixel switch, the common switch and the first pixel switch may be used to transmit a visible light signal. it can. Therefore, it is possible to easily transmit the visible light signal without significantly changing the configuration for displaying an image on the display.

Further, by making the control timing of the pixel switch coincide with the transmission symbol (4PPM one time) and controlling as shown in FIG. 463, the visible light signal can be transmitted from the LED display without flicker. The image signal (that is, the video signal) normally changes at a cycle of 1/30 seconds or 1/60 seconds, but by changing the image signal in accordance with the symbol transmission cycle (symbol cycle), it is realized without changing the circuit. can do.

As described above, in the determining step of the transmission method according to the present embodiment, the luminance change pattern is determined for each symbol period. In the first pixel switch control step, the pixel switches are switched in synchronization with the symbol period. Thereby, even if the symbol cycle is, for example, 1/2400 seconds, the visible light signal can be appropriately transmitted in accordance with the symbol cycle.

When the signal (symbol) is “10” and the average brightness is around 50%, the brightness change pattern is close to 0101 and the brightness rises at two points. However, in that case, the receiver can correctly receive the signal by prioritizing the subsequent rising point. That is, the subsequent rising point is the timing at which the rising of the luminance peculiar to the symbol “10” is obtained.

As the average brightness is higher, a signal closer to the signal modulated by 4PPM can be output. Therefore, when the luminance of the entire screen or the portion where the power supply line is common is low, the HI interval can be lengthened and the error can be reduced by reducing the current and lowering the instantaneous value of the luminance. .. In this case, the maximum brightness of the screen will decrease, but if high brightness is not required in the first place, such as indoor applications, or if priority is placed on visible light communication, enable the switch that enables this to enable communication. The balance between quality and image quality can be set optimally.

In the first pixel switch control step of the transmission method according to the present embodiment, when displaying an image on a display (LED display), the pixel value of the pixel in the image corresponding to the first light source is expressed. The first pixel switch is switched so as to supplement the lighting period only during the lighting period in which the first light source is turned off for the transmission of the visible light signal. That is, in the transmission method according to the present embodiment, the visible light signal is transmitted when the image is displayed on the LED display. Therefore, the LED may be turned off for the transmission of the visible light signal during the period in which the LED should be turned on to express the pixel value (specifically, the brightness value) indicated by the video signal. In such a case, in the transmission method according to the present embodiment, the first pixel switch is switched so as to compensate the lighting period only during the period when the LED is turned off.

For example, when displaying an image represented by a video signal without transmitting a visible light signal, a common switch is turned on and a pixel switch is an average pixel value represented by the video signal during one symbol period. It is turned on only for the period according to the brightness. When the average brightness is 75%, the common switch is turned on in the first slot to the fourth slot of the symbol period. Further, the pixel switch is turned on in the first slot to the third slot of the symbol period. Thus, during the symbol period, the LED lights up in the first slot to the third slot, so that the pixel value described above can be expressed. However, for the transmission of the symbol "01", the second slot is turned off. Therefore, in the transmission method according to the present embodiment, the pixel switch is switched so that only the second slot in which the LED is turned off compensates the lighting period of the LED, that is, the LED is turned on in the fourth slot. ..

Further, in the transmitting method according to the present embodiment, the lighting period is supplemented by changing the pixel value of the pixel in the video. For example, in the above case, the pixel value with an average luminance of 75% is changed to the pixel value with an average luminance of 100%. When the average brightness is 100%, the LEDs try to turn on in the first slot to the fourth slot, but the first slot is turned off for the transmission of the symbol “01”. Therefore, even when the visible light signal is transmitted, the LED can be turned on with the original pixel value (average brightness 75%).

As a result, it is possible to prevent the image from being corrupted by the transmission of the visible light signal.

(Emission control shifted for each pixel)
FIG. 464 is a diagram showing an example of a transmission signal in this embodiment.

When transmitting the same symbol (for example, “10”) from a pixel A and pixels near the pixel A (for example, pixel B and pixel C) as in FIG. The light emission timing of the pixel is shifted. However, the transmitter causes the pixels to emit light without shifting the rising timing of the luminance peculiar to the symbol between the pixels. Each of the pixels A to C corresponds to a light source (specifically, LED). Further, the rising timing of the brightness peculiar to the symbol is the timing of the boundary between the third slot and the fourth slot if the symbol is “10”. In addition, such a timing is hereinafter referred to as a symbol specific timing. By specifying the symbol specific timing, the receiver can receive the symbol corresponding to the timing.

By shifting the light emission timing in this manner, the waveform showing the average luminance transition between pixels has a gentle rising or falling except for the rising at the symbol specific timing, as shown in FIG. That is, the rising edge at the symbol specific timing is steeper than the rising edges at other timings. Therefore, the receiver can specify the proper symbol specific timing by preferentially receiving the steepest rising edge among the plurality of rising edges, and as a result, can suppress the reception error.

That is, when the symbol “10” is transmitted from a predetermined pixel and the brightness of the predetermined pixel is an intermediate value of 25% to 75%, the transmitter opens the pixel switch corresponding to the predetermined pixel. Set the interval short or long. Further, the transmitter inversely adjusts the open interval of the pixel switch corresponding to the pixels in the vicinity of the predetermined pixel. In this way, the error can also be suppressed by setting the open interval of each pixel switch so that the overall luminance including the predetermined pixel and the neighboring pixels does not change. The open section is a section in which the pixel switch is on.

Thus, the transmission method according to the present embodiment further includes the second pixel switch control step. In this second pixel switch control step, by turning on the second pixel switch for turning on the second light source, which is included in the above-mentioned light source group (common line) and is around the first light source, The visible light signal is transmitted by turning on the second light source only during the period when the common switch is on and the second pixel switch is on. The second light source is, for example, a light source adjacent to the first light source.

Then, in the first and second pixel switch control steps, when the same symbol included in the visible light signal is simultaneously transmitted from each of the first and second light sources, the first and second pixel switch control steps are performed. Of a plurality of timings that are turned on or off to transmit the same symbol, the timings at which the rising of the brightness unique to the same symbol is obtained are made the same in each of the first and second pixel switches, The other timing is made different for each of the first and second pixel switches, and the overall average brightness of the first and second light sources during the period in which the same symbol is transmitted becomes a predetermined brightness. To match.

As a result, like the inter-pixel average luminance transition shown in FIG. 464, in the spatially averaged luminance, the rising can be made steep only at the timing when the rising of the luminance unique to the symbol is obtained. It is possible to suppress the occurrence of errors. That is, it is possible to suppress the reception error of the visible light signal by the receiver.

When transmitting the symbol “10” from a predetermined pixel and the brightness of the predetermined pixel is an intermediate value between 25% and 75%, the transmitter selects the predetermined pixel in the first period. The open section of the corresponding pixel switch is set to be short or long. Further, the transmitter reversely adjusts the open period of the pixel switch in the second period (eg, frame) that is before or after the first period in time. As described above, the error can also be suppressed by setting the open section of the pixel switch so that the entire time average luminance including the first period and the second period does not change in a predetermined pixel.

That is, in the above-described first pixel switch control step in the transmission method according to the present embodiment, for example, the visible light signal includes the first period and the second period subsequent to the first period. Send the same symbol. At this time, in each of the first and second periods, the rising edge of the luminance unique to the same symbol among the plurality of timings at which the first pixel switch is turned on or off to transmit the same symbol. The obtained timing is the same, and the other timings are different. Then, the average brightness of the first light source in the entire first and second periods is made to match the predetermined brightness. The first period and the second period may be a period for displaying a frame and a period for displaying the next frame, respectively. Also, the first period and the second period may be symbol periods respectively. That is, the first period and the second period may each be a period for transmitting one symbol and a period for transmitting the next symbol.

As a result, similarly to the inter-pixel average luminance transition shown in FIG. 464, in the temporally averaged luminance, the rise can be made steep only at the timing when the rise of the luminance unique to the symbol is obtained. It is possible to suppress the occurrence of reception error. That is, it is possible to suppress the reception error of the visible light signal by the receiver.

(Light emission control when the pixel switch can be driven at double speed)
FIG. 465 is a diagram showing an example of a transmission signal in this embodiment.

When the pixel switch can be opened/closed at a half cycle of the transmission symbol cycle, that is, when the pixel switch can be driven at double speed, as shown in FIG. 465, the same light emission pattern as V4PPM (see FIG. 325) can be used.

In other words, when the symbol cycle (the period during which a symbol is transmitted) consists of 4 slots, the pixel switch control unit such as the LED driver circuit that controls the pixel switch can control the pixel switch every 2 slots. That is, the pixel switch control unit can turn on the pixel switch for an arbitrary time in the period of two slots from the first point of the symbol period. Further, the pixel switch control unit can turn on the pixel switch for an arbitrary time in the period of two slots from the first time point of the third slot of the symbol period.

That is, in the transmission method according to the present embodiment, the pixel value may be changed at a cycle that is ½ of the symbol cycle described above.

In this case, the fineness per opening/closing of the pixel switch may be reduced (accuracy may be reduced). Therefore, by performing this only when the transmission priority switch is enabled, it is possible to optimally set the balance between image quality and transmission quality.

(Block of light emission control by pixel value adjustment)
FIG. 466 is a block diagram illustrating an example of a transmitter in this embodiment.

FIG. 466(a) is a block diagram showing a configuration of a device that does not transmit a visible light signal but only displays an image, that is, a display device that displays an image on the above-mentioned LED display. As shown in (a) of FIG. 466, this display device includes an image/video input unit 1911, an N-fold speed increasing unit 1912, a common switch control unit 1913, and a pixel switch control unit 1914.

The image/video input unit 1911 outputs a video signal indicating an image or video at a frame rate of 60 Hz, for example, to the N-times speeding unit 1912.

The N-fold speed increasing unit 1912 increases the frame rate of the video signal input from the image/video input unit 1911 by N (N>1) times and outputs the video signal. For example, the N-fold speed increasing unit 1912 increases the frame rate ten times (N=10), that is, the frame rate of 600 Hz.

The common switch control unit 1913 switches the common switch based on the image having the frame rate of 600 Hz. Similarly, the pixel switch control unit 1914 switches the pixel switch based on the image having the frame rate of 600 Hz. As described above, since the frame rate is increased by the N speed doubler 1912, it is possible to avoid flickering due to opening/closing of a switch such as a common switch or a pixel switch. Further, even when the LED display is imaged by the imaging device with a high-speed shutter, it is possible to cause the imaging device to capture an image without missing pixels or flicker.

FIG. 466(b) is a block diagram showing a configuration of a display device, that is, a transmitter (transmitting device) that transmits not only an image but also the above visible light signal. This transmitter includes an image/video input unit 1911, a common switch control unit 1913, a pixel switch control unit 1914, a signal input unit 1915, and a pixel value adjustment unit 1916. The signal input unit 1915 outputs a visible light signal including a plurality of symbols to the pixel value adjustment unit 1916 at a symbol rate (frequency) of 2400 symbols/second.

The pixel value adjustment unit 1916 duplicates the image input from the image/video input unit 1911 according to the symbol rate of the visible light signal, and adjusts the pixel value according to the above method. Accordingly, the common switch control unit 1913 and the pixel switch control unit 1914 in the subsequent stage from the pixel value adjusting unit 1916 can output the visible light signal without changing the brightness of the image or the image.

For example, in the case of the example shown in FIG. 466, if the symbol rate of the visible light signal is 2400 symbols/sec, the pixel value adjusting unit 1916 is included in the video signal so that the frame rate 60 Hz of the video signal becomes 4800 Hz. Duplicate the image. For example, the value of the symbol included in the visible light signal is “00”, and the pixel value (luminance value) of the pixel included in the first image before copying is 50%. In this case, the pixel value adjustment unit 1916 adjusts the pixel value to 100% for the first image after copying and to 50% for the second image. As a result, the brightness becomes 50% due to the AND of the common switch and the pixel switch as in the brightness change in the case of the symbol “00” shown in (c) of FIG. As a result, the visible light signal can be transmitted while maintaining the same brightness as the original image. The common switch and the AND of the pixel switch mean that the light source (that is, the LED) corresponding to the common switch and the pixel switch is turned on only during the period when the common switch is on and the pixel switch is on. ..

In the transmission method according to the present embodiment, the video display and the visible light signal transmission may not be performed simultaneously, but may be performed separately for the signal transmission period and the video display time.

That is, in the above-described first pixel switch control step in the present embodiment, the first pixel switch is turned on during the signal transmission period in which the common switch is switching according to the brightness change pattern. Then, in the transmission method according to the present embodiment, the common switch is turned on during a video display period different from the signal transmission period, and the first pixel switch is turned on according to the video to be displayed during the video display period. Thus, a video display step of displaying pixels in the video by turning on the first light source only during a period when the common switch is on and the first pixel switch is on may be included. ..

Thus, the display of the image and the transmission of the visible light signal are performed in different periods, so that the display and the transmission can be easily performed.

(Power change timing)
When the power supply line is changed, a signal off section is generated, but reception is not affected even if the last part of 4PPM is not emitting light. Therefore, by changing the power supply line in accordance with the transmission cycle of 4PPM symbols. , The power line can be changed without affecting the reception quality.

Further, by changing the power supply line during the LO period of 4PPM, the power supply line can be changed without affecting the reception quality. In this case, it is possible to further transmit while keeping the maximum brightness high.

(Drive timing)
Further, in the present embodiment, the LED display may be driven at the timings shown in FIGS. 467 to 469.

467 to 469 are timing charts when the LED display is driven by the optical ID modulation signal of the present invention.

For example, as shown in FIG. 468, when the common switch (COM1) is turned off to transmit a visible light signal (optical ID) (period t1), the LED is turned on at the brightness indicated by the video signal. Therefore, the LED is turned on after the period t1. With this, it is possible to appropriately transmit the visible light signal and to appropriately display the video without destroying the video represented by the video signal.

(Summary)
FIG. 470A is a flowchart illustrating a transmission method according to an aspect of the present invention.

A transmission method according to an aspect of the present invention is a transmission method that transmits a visible light signal by a change in luminance, and includes steps SC11 to SC13.

In step SC11, the luminance change pattern is determined by modulating the visible light signal as in the above-described embodiments.

In step SC12, a common switch, which is included in the light source group included in the display and is used for commonly lighting a plurality of light sources for representing pixels in the image, is switched according to the brightness change pattern.

In step S13, the common switch is turned on by turning on the first pixel switch (that is, the pixel switch) for turning on the first light source of the plurality of light sources included in the light source group, and , The visible light signal is transmitted by turning on the first light source only during the period when the first pixel switch is on.

FIG. 470B is a block diagram illustrating a functional configuration of the transmission device according to one embodiment of the present invention.

A transmission device C10 according to an aspect of the present invention is a transmission device (or a transmitter) that transmits a visible light signal according to a change in luminance, and includes a determination unit C11, a common switch control unit C12, and a pixel switch control unit C13. Equipped with. The determination unit C11 determines the brightness change pattern by modulating the visible light signal, as in each of the above-described embodiments. The determining unit C11 is included in the signal input unit 1915 shown in FIG. 466, for example.

The common switch control unit C12 switches the common switch according to the brightness change pattern. The common switch is a switch for commonly lighting a plurality of light sources included in a light source group included in the display for representing pixels in an image.

The pixel switch control unit C13 turns on the pixel switch for turning on the light source to be controlled among the plurality of light sources included in the light source group, so that the common switch is turned on and the pixel switch is turned on. The visible light signal is transmitted by turning on the light source to be controlled only during a certain period. The light source to be controlled is the above-mentioned first light source.

Thereby, a visible light signal can be appropriately transmitted from a display including a plurality of LEDs as a light source. Therefore, it is possible to perform communication between devices other than the lighting device. Further, when the display is a display for displaying an image by controlling the common switch and the pixel switch, the visible light signal can be transmitted by using the common switch and the pixel switch. Therefore, the visible light signal can be easily transmitted without making a significant change to the configuration for displaying an image on the display (that is, the display device).

(Frame structure of Single frame transmission)
FIG. 471 is a diagram illustrating an example of a transmission signal in this embodiment.

As shown in (a) of FIG. 471, the transmission frame includes a preamble (PRE), an ID length (IDLEN), an ID type (IDTYPE), a content (ID/DATA), and a check code (CRC). The number of bits in each area is an example.

By using the preamble as shown in (b) of FIG. 471, the receiver can distinguish from other parts encoded with 4PPM, I-4PPM, or V4PPM, and can find a signal break. ..

As shown in (c) of FIG. 471, by specifying the length of ID/DATA with IDLEN, variable-length content can be transmitted.

CRC is a check code for correcting or detecting an error in a portion other than PRE. By changing the CRC length according to the length of the inspection area, it is possible to maintain the inspection ability above a certain level. Also, by using different check codes depending on the length of the check area, it is possible to improve the check capability per CRC length.

(Frame structure of Multiple frame transmission)
472 and 473 are diagrams showing an example of a transmission signal in the present embodiment.

A partition type (PTYPE) and a check code (CRC) are added to the transmission data (BODY) to form Joined data. Joined data is divided into several DATAPARTs, and a preamble (PRE) and an address (ADDR) are added and transmitted.

PTYPE (or partition mode (PMODE)) indicates a division method or meaning of BODY. By setting it to 2 bits as shown in (a) of FIG. 472, 4PPM can be used for proper encoding. The transmission time can be shortened by setting it to 1 bit as shown in (b) of FIG. 472.

CRC is a check code for checking PTYPE and BODY. As defined in FIG. 450, by changing the CRC code length according to the length of the portion to be inspected, it is possible to keep the inspection capability above a certain level.

By determining the preamble as shown in FIG. 451, it is possible to shorten the transmission time while ensuring the variation of the division pattern.

By setting the address as shown in FIG. 452, the receiver can restore the data regardless of the order in which the frames are received.

FIG. 473 shows combinations of possible Joined data length and the number of frames. The underlined combination is a combination used when PTYPE described later is Single frame compatible.

(Body field structure)
FIG. 474 is a diagram showing an example of a transmission signal in this embodiment.

By configuring BODY as a field structure as shown in the figure, it is possible to transmit the same ID as in single frame transmission.

When the same ID is used for the same IDTYPE, the same meaning is given regardless of whether single-frame transmission or multi-frame transmission is used, or whether packet transmission is used. Can be sent.

The length of the ID is specified by IDLEN, and PADDING is transmitted for the remaining portion. All of this part may be 0 or 1, data for extending the ID may be transmitted, or a check code may be used. PADDING may be left-justified.

In (b), (c), or (d) of FIG. 474, the transmission time can be made shorter than that of (a) of FIG. At this time, the length of the ID is the maximum length that can be taken as the ID.

In the case of (b) or (c) of FIG. 474, the number of bits of IDTYPE is an odd number, but when combined with the 1-bit PTYPE shown in (b) of FIG. 472, it becomes an even number, and efficient encoding with 4PPM is possible. You can

In (c) of FIG. 474, a longer ID can be transmitted.

In (d) of FIG. 474, more IDTYPEs can be expressed.

(PTYPE)
FIG. 475 is a diagram illustrating an example of a transmission signal in this embodiment.

When PTYPE is a predetermined bit, it indicates that BODY is in single frame compatible mode. As a result, the same ID as in single frame transmission can be transmitted.

For example, when PTYPE=00, the ID or ID type corresponding to the PTYPE can be treated in the same manner as the ID or ID type transmitted by single frame transmission, and the management of the ID or ID type can be simplified. it can.

When PTYPE is a predetermined bit, BODY indicates the data stream mode. At this time, all the combinations of the number of transmission frames and the DATAPART length can be used, and different combinations of data can have different meanings. Depending on the bit of PTYPE, the different combinations may have the same meaning or may have different meanings. Thereby, the transmission method can be flexibly selected.

For example, when PTYPE=01, an ID having a size not defined for single frame transmission can be transmitted. Even if the ID corresponding to the PTYPE is the same as the ID for single frame transmission, the ID corresponding to the PTYPE can be treated as an ID different from the ID for single frame transmission. As a result, the number of expressible IDs can be increased.

(Field structure of Single frame compatible mode)
FIG. 476 is a diagram illustrating an example of a transmission signal in this embodiment.

When (a) of FIG. 474 is used, in the single frame compatible mode, the case where the combination of the tables shown in FIG. 476 is transmitted is the most efficient.

When using (b), (c) or (d) of FIG. 474, when the ID is 32 bits, the combination of 13 frames and DATAPART length of 4 bits is efficient. When the ID is 64 bits, a combination of 11 frames and DATAPART length of 8 bits is efficient.

By transmitting only the combinations in the table, different combinations can be determined as reception errors, and the reception error rate can be reduced.

INDUSTRIAL APPLICABILITY The present invention can be used in, for example, a transmitter that transmits a visible light signal from a display or the like, and particularly can be used in a transmitter that transmits a visible light signal from a large LED display or the like.

C10 transmitter (transmitter)
C11 determination unit C12 common switch control unit C13 pixel switch control unit

Claims (9)

A transmission method of transmitting a visible light signal according to a change in brightness,
By determining the brightness change pattern for each symbol period by modulating the visible light signal,
A common switch control step of switching a common switch, which is included in a light source group provided in the display, for commonly lighting a plurality of light sources for representing pixels in an image, according to the brightness change pattern,
By turning on a first pixel switch for turning on a first light source of a plurality of light sources included in the light source group, the common switch is turned on, and the first pixel switch is turned on. only during the period is on, turns on the first light source, in synchronization with the symbol period, by switching to Rukoto the pixel switch and a first pixel switch control step of transmitting the visible light signal Including transmission method. In the first pixel switch control step,
When displaying an image on the display, the first light source for transmitting the visible light signal in the lighting period for expressing the pixel value of the pixel in the image corresponding to the first light source. period only, so that the lighting period becomes longer, transmission method according to claim 1 for switching the first pixel switch but is turned off. The transmission method according to claim 2 , wherein the lighting period is supplemented by changing a pixel value of a pixel in the video. The transmission method according to claim 3 , wherein the pixel value is changed at a cycle of ½ of the symbol cycle. The transmission method further comprises
The common switch is turned on by turning on a second pixel switch, which is included in the light source group, for turning on a second light source around the first light source, and the second pixel switch is turned on. A second pixel switch control step of transmitting the visible light signal by turning on the second light source only while the pixel switch is on.
In the first and second pixel switch control steps,
When simultaneously transmitting the same symbol contained in the visible light signal from each of the first and second light sources,
Among a plurality of timings at which each of the first and second pixel switches is turned on or off to transmit the same symbol, the timing at which the rising of the brightness unique to the same symbol is obtained is the first And the second pixel switch are made the same, and the other timings are made different in each of the first and second pixel switches,
The transmission method according to claim 2 , wherein the overall average luminance of the first and second light sources during the period in which the same symbol is transmitted is made to match a predetermined luminance. In the first pixel switch control step,
When transmitting the same symbol included in the visible light signal in the first period and the second period following the first period,
In each of the first and second periods, the rising of the brightness unique to the same symbol is obtained among a plurality of timings when the first pixel switch is turned on or off to transmit the same symbol. The same timing, different other timing,
The transmission method according to claim 2 , wherein the average brightness of the first light source in the entire first and second periods is made to match a predetermined brightness. In the first pixel switch control step,
Turning on the first pixel switch during a signal transmission period in which the common switch is switching in accordance with the luminance change pattern,
The transmission method further comprises
The common switch is turned on by turning on the common switch during a video display period different from the signal transmission period, and turning on the first pixel switch in accordance with the video to be displayed during the video display period, The transmission method according to claim 1, further comprising a video display step of displaying pixels in the video by turning on the first light source only during a period when the first pixel switch is on. A transmitter for transmitting a visible light signal according to a change in brightness,
By deciding the luminance change pattern for each symbol period by modulating the visible light signal,
A common switch control unit that switches a common switch, which is included in a light source group included in the display, for commonly lighting a plurality of light sources for representing pixels in an image, according to the brightness change pattern,
By turning on the pixel switch for turning on the light source to be controlled among the plurality of light sources included in the light source group, the common switch is on, and only during the period when the pixel switch is on. turns on the the control target of the light source, the symbol period is synchronized, by switching to Rukoto the pixel switch, a transmission device and a pixel switch control unit for transmitting the visible light signal. A program for transmitting a visible light signal according to a change in brightness,
By deciding the luminance change pattern for each symbol period by modulating the visible light signal,
A common switch control step of switching a common switch, which is included in a light source group provided in the display, for commonly lighting a plurality of light sources for representing pixels in an image, according to the brightness change pattern,
By turning on the pixel switch for turning on the light source to be controlled among the plurality of light sources included in the light source group, the common switch is on, and only during the period when the pixel switch is on. turns on the the control target of the light source, by switching to Rukoto the pixel switch in synchronism with the symbol period, a program for executing the pixel switch control step of transmitting the visible light signal to the computer.