JP2012034119A - Terminal device and processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To smoothen the progress of a conference by speedily outputting an image relating to a drive-controlled image, in drive control of a camera.SOLUTION: A television conference terminal 110 which outputs the image of a participant taken by a camera 113 to a television conference terminal 110 at the base or another base records alternate image information indicating participant to be taken by the camera 113 in association with positional information of the participants. A participant to be taken by the camera 113 is determined. The positional information of the determined participant is identified and on the basis of the identified positional information, the camera 113 is drive-controlled so as to take an image of the participant. During the drive control, at least the alternate image indicating the participant is output.

Description

  The present invention relates to a terminal device and a processing method for transmitting / receiving information between terminal devices, and in particular, a conference is performed using information transmitted / received between its own base and another base connected via a network. The present invention relates to a terminal device and a processing method.

  The video conference system transmits and receives image information indicating the status of participants at each base and material information used for a conference between a plurality of terminal devices. The video conference system displays images, materials, and the like of participants at each base based on information transmitted and received by the terminal device. Participants of the conference at each site conduct the conference by confirming the images and materials of the participants displayed by the terminal device.

  Participants' images are captured by cameras installed at each site. The terminal device at each base outputs the captured image of the participant to another base. The terminal device at each site displays the received participant image on the display. A terminal device at a base where a plurality of participants exist can change the image of the participant to be imaged by drive control such as pan / tilt of the camera. In recent years, since a landscape image or the like that is different from a participant is not captured and output during pan / tilt, pan / tilt is performed when panning / tilting is performed to change an imaging target from a participant who is capturing to a different participant. A proposal has been made to output an image of a participant who was imaged before tilting (Patent Document 1).

JP-A-7-322116

  However, in the conventional technique described in Patent Document 1 described above, an example of a problem is that an image of a participant who performs pan / tilt and is to be imaged cannot be output until the image can be captured by pan / tilt. It is done. In particular, even when panning / tilting is performed to capture participants who are important to the conference, such as speakers, it is configured to output the participant's image before panning / tilting. An example of the problem is that the conference does not proceed smoothly over time.

  In order to solve the above-described problems, the present invention can smoothly output a conference by quickly outputting an image related to an image after drive control when performing drive control such as pan / tilt of the camera. An object of the present invention is to provide a terminal device and a processing method that can be used.

  In order to solve the above-described problems and achieve the object, a terminal device according to claim 1 is a terminal device that outputs a subject video imaged by an imaging unit to a terminal device at its own site or another site, Recording means for recording substitute image information indicating a subject imaged by the imaging unit in association with position information of the subject, detection means for detecting a state change related to the subject at its own base or another base, A determining unit that determines the subject to be imaged by the imaging unit based on a state change detected by the detecting unit; a specifying unit that specifies the position information of the subject determined by the determining unit; Based on the position information specified by the specifying means, drive control of the imaging unit is performed so as to image the subject determined by the determining means. A clear screen control means, in the drive control by the control unit, characterized by comprising output means for outputting the substitute image information indicative of the object determined by at least the determining means.

  According to a second aspect of the present invention, in the terminal device according to the first aspect of the invention, the detection means detects the user's utterance as the subject as the state change, and the determination means detects the detection. The user located in the detection direction in which the utterance is detected by the means is determined as the subject to be imaged.

  According to a third aspect of the present invention, in the terminal device according to the second aspect, the control unit performs drive control of the imaging unit when the utterance detected by the detection unit continues for a predetermined time or more. It is characterized by that.

  The terminal device according to a fourth aspect of the present invention is the terminal device according to any one of the first to third aspects, wherein the recording means stores the substitute image information based on the subject video imaged by the imaging unit. Recording is performed in association with the position information based on the imaged position.

  The terminal device according to a fifth aspect of the present invention is the terminal device according to any one of the first to fourth aspects, wherein the recording unit captures the subject image by the imaging unit that is driven and controlled by the control unit. If it is, the already recorded alternative image information is updated to the alternative image information based on the captured subject video.

  The terminal device according to a sixth aspect of the present invention is the terminal device according to any one of the first to fifth aspects, wherein the output means outputs the information to be output to the terminal device at the local site or the other site. The video image picked up by the image pickup unit during the drive control is switched to the substitute image information recorded by the recording means.

  The processing method according to claim 7 is a processing method for outputting a subject video imaged by an imaging unit to a terminal device at its own site or another site, and substitute image information indicating a subject imaged by the imaging unit On the basis of the recording process for recording in association with the position information of the subject, a detection process for detecting a status change related to the subject at its own base or another base, and a status change detected by the detection process, Based on the determination step of determining the subject to be imaged by the imaging unit, the specifying step of specifying the position information of the subject determined by the determining step, and the position information specified by the specifying step A control process for controlling the driving of the imaging unit so as to image the subject determined in the determination process, and a drive by the control process. Dear is characterized in that it comprises an output step of outputting the substitute image information indicative of the object determined by at least the determining step.

  According to the first aspect of the present invention, since the substitute image information of the subject imaged by the imaging unit can be output even during the drive control of the imaging unit, the subject to be imaged can be quickly confirmed. be able to.

  According to the second aspect of the present invention, it is possible to perform drive control of the imaging unit with the user who has spoken as an imaging target, and to output as alternative image information during the drive control. Can be confirmed.

  According to the third aspect of the present invention, drive control can be performed when there is an utterance for a predetermined time or longer, so that an object that is important as an imaging target can be optimized.

  According to the fourth aspect of the present invention, since the substitute image information can be recorded by the image pickup by the image pickup unit, the substitute image information suitable for the use environment of the device can be used.

  According to the fifth aspect of the present invention, the substitute image information can be recorded by the image pickup by the drive unit that is driven and controlled. Therefore, by using the latest substitute image information according to the progress of the process, the important subject can be recorded. Optimization can be achieved.

  According to the sixth aspect of the present invention, the information output during the drive control of the image capturing unit is switched from the captured image to the substitute image information, so that the amount of information to be output can be reduced.

  According to the seventh aspect of the invention, since the alternative image information of the subject imaged by the imaging unit can be output even during the drive control of the imaging unit, the subject to be imaged can be quickly confirmed. be able to.

  As described above, according to the terminal device and the processing method according to the present invention, when performing drive control such as panning and tilting of the camera, it is possible to smoothly output an image related to the image after drive control. There is an effect that the conference can be progressed.

It is explanatory drawing which shows an example of the video conference system concerning embodiment of this invention. It is explanatory drawing which shows an example of a functional structure of the video conference terminal concerning embodiment of this invention. It is explanatory drawing which shows an example of the image table of the video conference terminal concerning embodiment of this invention. It is explanatory drawing which shows an example of the imaging of the participant image by the video conference terminal concerning embodiment of this invention. It is explanatory drawing which shows an example of the display on the display of the participant image concerning embodiment of this invention. It is a flowchart which shows the content of the process of the video conference terminal concerning embodiment of this invention. It is a flowchart which shows the content of the process of the video conference terminal concerning the modification of this invention.

  Exemplary embodiments of a terminal device and a processing method according to the present invention will be explained below in detail with reference to the accompanying drawings.

(Embodiment)
(overall structure)
The case where the terminal device according to the embodiment of the present invention is applied to video conference terminals installed at a plurality of bases for a video conference system that performs a video conference will be described with reference to FIG. FIG. 1 is an explanatory diagram showing an example of a video conference system according to an embodiment of the present invention. In the present embodiment, the terminal device according to the present invention is realized by the video conference terminals 110 (110a, 110b) installed at the respective bases (A, B), and a plurality of devices are provided via the video conference terminal 110 and the network 150. A case where the processing of the processing method according to the present invention is executed by the video conference system 100 to which the video conference terminal 110 is connected will be described.

  In FIG. 1, a video conference system 100 is configured by connecting video conference terminals 110a and 110b installed at each of the bases A and B via a network NW. Specifically, the video conference system 100 is configured such that video conference terminals 110a and 110b installed at geographically separated locations A and B are connected via a network 150 such as the Internet, or are separated from each other in a building. Video conference terminals 110a and 110b installed at bases A and B are connected via a network 150 such as a LAN (local area network). In FIG. 1, the video conference terminals 110a and 110b are described as being connected to each other via the network 150, but are connected to each other via a management server or the like installed at an arbitrary position on the network 150. It may be configured. In the following description, when the bases are not distinguished, “a” and “b” which are symbols at the end of the reference numerals are omitted.

  The video conference system 100 allows each video conference terminal 110 to mutually transmit and receive audio from an image of a subject such as a participant in a video conference and speech from the participant at each site. The video conference terminal 110 is connected to a main body 111 including a functional unit such as a CPU (Central Processing Unit), a display 112 that displays various images, a camera 113 that captures a subject, and a microphone 114 that collects sound. And a speaker 115 for outputting various sounds.

  The video conference terminal 110 captures an image of its own base with the camera 113. The video conference terminal 110 transmits the captured image to the video conference terminal 110 at another site via the network 150. The video conference terminal 110 receives an image transmitted from the video conference terminal 110 at another base. The video conference terminal 110 displays on the display 112 an image captured at its own base or an image received from another base.

  The video conference terminal 110 collects sound at the local site by the microphone 114. The video conference terminal 110 transmits the collected sound to the video conference terminal 110 at another site via the network 150. The video conference terminal 110 receives audio transmitted from the video conference terminal 110 at another site. The video conference terminal 110 outputs the received voice through the speaker 115.

  That is, the video conference terminal 110 plays back images and sounds that are transmitted and received between the local site and other sites. Participants at each site conduct a video conference between participants located remotely by viewing the images and sounds of other sites reproduced by the video conference terminal 110 at their site.

  Further, the video conference terminal 110 changes the subject included in the imaging range of the camera 113 by performing drive control such as pan / tilt on the camera 113. The drive control of the camera 113 is performed based on the direction of the participant's utterance detected by the participant's operation or the microphone 114. Note that, in the embodiment of the present invention, a description will be given of capturing a participant image as a subject, but the present invention is not limited to this. The subject may be a document related to a video conference. That is, the video conference terminal 110 detects the presentation of materials related to the video conference by vibration, voice, light emission, or the like, and controls the imaging direction of the camera 113 based on the detection direction.

  When the video conference terminal 110 performs drive control of the camera 113, the video conference terminal 110 outputs alternative image information indicating a participant imaged after the drive control to the display 112 at the local site or the video conference terminal 110 at another site. Although details will be described with reference to FIGS. 3 to 5, when the drive control of the camera 113 is started, the video conference terminal 110 drives the image to be output from the participant image captured before the drive control. It switches to the alternative image information which is an image which shows the participant imaged after control. When the drive control of the camera 113 is completed, the video conference terminal 110 outputs a participant image captured by the camera 113 after the drive control.

  The substitute image information is an image of a participant captured in the past by the camera 113 or an image such as a character, a picture, or a photograph indicating the participant that is input separately, and is used as an image table in association with the position information of the participant. To be recorded. The participant's position information is recorded based on, for example, the detection direction in which the participant's utterance is detected, the imaging direction of the camera 113 that images the participant, the imaging direction by the participant's operation input, and the like.

(Functional configuration)
The functional configuration of the video conference terminal 110 will be described with reference to FIG. FIG. 2 is an explanatory diagram illustrating an example of a functional configuration of the video conference terminal according to the embodiment of the present invention.

  In FIG. 2, a video conference terminal 110 inputs / outputs various images to / from a CPU (Central Processing Unit) 201, a RAM (Random Access Memory) 202, a ROM (Read Only Memory) 203, a display 112 and a camera 113. Controls communication with an external device, a video I / F 204 that controls input / output, an audio I / F 205 that controls input / output of various audio to / from the speaker 115 and the microphone 114, an operation unit 206 that receives input of various information, and the like. A communication I / F 207 and a storage medium 208 that stores various types of information are provided. Each component of the video conference terminal 110 is connected by a bus 200.

  The CPU 201 controls the entire video conference terminal 110. The CPU 201 executes various programs read from the ROM 203 using the RAM 202 as a work area.

  The video I / F 204 displays various images on the display 112 under the control of the CPU 201. The video I / F 204 reads the participant image received from the video conference terminal 110 at another site from the storage medium 208 and causes the display 112 to display the participant image under the control of the CPU 201. The video I / F 204 may be configured to display on the display 112 a participant image at the local site, a processing screen related to a video conference with another site, and the like.

  The video I / F 204 controls the drive of the camera 113 in accordance with the control of the CPU 201 and images the participant who is the subject at the local site. The video I / F 204 pans and tilts the camera 113 so as to capture, for example, a participant who speaks among a plurality of participants. The drive control of the camera 113 is performed based on, for example, a detection direction in which an utterance of a predetermined time or more is detected by a microphone 114 described later, or an imaging direction in which an input by a participant is received by the operation unit 206. The video I / F 204 transmits a participant image captured by driving control to the video conference terminal 110 that is a transmission destination via a communication I / F 207 described later.

  The video I / F 204 outputs the participant image captured by the camera 113 to the storage medium 208 described later as substitute image information indicating the participant. The video I / F 204 outputs the imaging direction of the camera 113 when the participant is imaged to the storage medium 208 as the location information of the participant image. In the embodiment of the present invention, the video I / F 204 has been described as using the imaging direction captured by the camera 113 as the position information, but the present invention is not limited to this. Specifically, for example, the direction in which an utterance is detected when an uttered participant is imaged, the direction in which an operation input by the participant is accepted, and the like may be output to the storage medium 208 as position information. The storage medium 208 stores an image table in which the substitute image information output from the video I / F 204 is associated with the position information according to the control of the CPU 201.

  During the drive control of the video I / F 204, the CPU 201 transmits, instead of the image captured by the camera 113, alternative image information indicating the participant of the drive control destination via the communication I / F 207 as a video conference as a transmission destination. Transmit to terminal 110. Specifically, the CPU 201 is configured to read alternative image information from the image table and transmit it based on the position information of the participants who are to be imaged. When the drive control of the camera 113 is completed, the CPU 201 transmits a participant image captured by the camera 113 to the video conference terminal 110 that is a transmission destination via the communication I / F 207.

  Here, the output of the alternative image information according to the embodiment of the present invention will be described with reference to FIGS. In the following, a case where the participant image or the substitute image information is transmitted to the video conference terminal 110b at the base B by the video conference terminal 110a at the base A will be described. FIG. 3 is an explanatory diagram illustrating an example of an image table of the video conference terminal according to the embodiment of the present invention.

  In FIG. 3, the image table 300a is stored in the storage medium 208a, and the image information of the participants a1 and a2 that are subjects of the base A is associated with the position information of each participant a1 and a2. The position information is an imaging direction of the camera 113a when each participant a1, a2 is imaged by the camera 113.

  When a video conference is started with the video conference terminal 110b at another site, the video conference terminal 110a drives and controls the imaging direction of the camera 113a to capture images of all the participants a1 and a2 at the local site. The video conference terminal 110a creates an image table 300a using the images of the participants a1 and a2 after the start of the video conference as substitute image information and the imaging direction in which the participants a1 and a2 are captured as position information. Specifically, for example, for an image captured by the camera 113a, when a face image is captured by image recognition, an image including the face image is used as substitute image information, and an image capturing direction in which the face image is captured is used as position information. A table 300a is created. Further, instead of recognizing the face image, the face image may be specified for the captured image by the user's operation.

  With reference to FIG. 4, a case will be described in which the participant image captured by pan / tilt control of the camera 113a is changed from the participant a2 to the participant a1 by the video conference terminal 110a according to the embodiment of the present invention. FIG. 4 is an explanatory diagram illustrating an example of capturing participant images by the video conference terminal according to the embodiment of the present invention.

  In FIG. 4, in the state 1, the video conference terminal 110a images the participant a2 included in the imaging range 401 of the camera 113a. The video conference terminal 110a outputs the captured image of the participant a2 to the video conference terminal 110b at the base B. The video conference terminal 110b displays the participant image of the participant a2 output from the video conference terminal 110a on the display 112b.

  The display on the display 112b by the video conference terminal 110b according to the embodiment of the present invention will be described with reference to FIG. FIG. 5 is an explanatory diagram illustrating an example of display on the display of the participant image according to the embodiment of the present invention.

  In FIG. 5, in the state 1, the video conference terminal 110b displays the participant image 501 of the participant a2 output from the site A on the display 112b. Although not particularly illustrated, the display 112b may be configured to display a participant image at the local site, a processed image regarding a video conference, a document image, and the like in addition to the participant image 501 at the site A.

  Returning to FIG. 4, when the video conference terminal 110a detects the speech of the participant a1 with the microphone 114a in the state 1, the video conference terminal 110a performs pan / tilt control in the direction 400 in which the speech is detected by the camera 113a. And migrate. The utterance may be detected when, for example, the duration of the utterance is longer than a predetermined time or the volume of the utterance is higher than a predetermined value.

  When the video conference terminal 110a performs pan / tilt control of the camera 113a in the state 2, the video conference terminal 110a specifies the participant a1 who has spoken as the speaker. The video conference terminal 110a outputs the specified alternative image information of the speaker to the video conference terminal 110b at the base B.

  Specifically, for example, the speaker specifies based on the direction in which the utterance is detected. The direction of the utterance is determined based on, for example, the direction in which the utterance is detected by the directional microphone 114 or the direction specified by the sound volume detected by each microphone 114a when a plurality of microphones 114a are installed. To do. The video conference terminal 110a determines the imaging direction of the camera 113a based on the direction in which the utterance is detected. The video conference terminal 110a outputs, based on the image table 300a, image information having the determined imaging direction as position information to the video conference terminal 110b at the site B as substitute image information.

  In state 2, even if the participants a1 and a2 are no longer included in the imaging range 402 by pan / tilt control, the video conference terminal 110a outputs alternative image information regarding the participant a1 to the video conference terminal 110b. .

  Returning to FIG. 5, in the state 2, the video conference terminal 110b displays the substitute image information 510 of the participant a1 output from the site A on the display 112b. That is, in state 2, the video conference terminal 110b is not the images of the participants a1 and a2 that are in the imaging range during the pan / tilt control of the camera 113a of the video conference terminal 110a, but the speech that is the participant a1 who made the speech. An image related to the person will be displayed.

  Returning to FIG. 4, when the pan / tilt control in the state 2 is completed and the participant a1 is included in the imaging range 403 of the camera 113a, the participant image of the participant a1 is captured by the camera 113a, and the state 3 is entered. Transition. The video conference terminal 110a outputs the participant image of the participant a1 who is speaking to the video conference terminal 110b at the base B.

  Returning to FIG. 5, in the state 3, the video conference terminal 110b displays the participant image 403 of the participant a1 output from the base A on the display 112b. Thus, in a state where the pan / tilt control of the camera 113a is not executed by the video conference terminal 110a at the site A (states 1 and 3), the participant image captured by the camera 113a is displayed and the pan / tilt control is performed. In the middle state (state 2), it is possible to display alternative image information indicating the participant a1 imaged after the pan / tilt control. That is, instead of the surrounding moving scenery image that can be captured during the pan / tilt control of the camera 113a, it is possible to display the images of the participants related to the video conference.

  Returning to FIG. 2, the sound I / F 205 causes the speaker 115 to output various sounds according to the control of the CPU 201. The audio I / F 205 reads out the audio received from the video conference terminal 110 at another site from the storage medium 208 according to the control of the CPU 201 and causes the speaker 115 to output it. The voice I / F 205 may be configured to output guidance voice regarding a video conference with another base.

  The voice I / F 205 collects the participant voice at its own location by the microphone 114 under the control of the CPU 201. The audio I / F 205 outputs the participant audio collected by the microphone 114 to the storage medium 208 according to the control of the CPU 201.

  The voice I / F 205 detects the speaker by the participant voice at the local site collected by the microphone 114 according to the control of the CPU 201. The speaker may be detected when, for example, the duration of the utterance is longer than a predetermined time or the volume of the utterance is higher than a predetermined value.

  The operation unit 206 receives input of various types of information from participants. The operation unit 206 includes a touch panel, operation buttons, and the like. The operation unit 206 accepts input of information regarding a video conference and outputs an input signal to the CPU 201.

  The communication I / F 207 is connected to a network 150 such as the Internet through a communication line, and is connected to an external device such as another video conference terminal 110 or other information terminal via the network 150. A communication I / F 207 controls an interface between the network 150 and the video conference terminal 110 and controls data input / output with respect to an external device. As the communication I / F 207, for example, a modem or a LAN adapter can be employed.

  The communication I / F 207 receives images and sound transmitted from the video conference terminal 110 at another base. Communication I / F 207 outputs the received image and sound to recording medium 208 in accordance with the control of CPU 201.

  The communication I / F 207 transmits the image and sound stored in the storage medium 208 to the video conference terminal 110 at another base according to the control of the CPU 201.

  The storage medium 208 is an HD (hard disk) or an FD (flexible disk) as an example of a removable recording medium. The storage medium 208 has respective drive devices, and various data are recorded under the control of the CPU 201. Further, data is read from the storage medium 208 according to the control of each drive device.

  If each component is described in association with each function, the function of the recording unit according to the present invention is realized by the CPU 201 and the storage medium 208 shown in FIG. The CPU 201 realizes the functions of the determination unit, the detection unit, and the identification unit according to the present invention. The function of the control means according to the present invention is realized by the CPU 201 and the video I / F 204. The function of the output unit according to the present invention is realized by the CPU 201 and the communication I / F 207. In addition, although each function was implement | achieved by one video conference terminal, it is good also as implement | achieving in cooperation with several apparatuses.

(Contents of processing of the video conference terminal 110)
The contents of processing of the video conference terminal 100 according to the embodiment of the present invention will be described with reference to FIG. FIG. 6 is a flowchart showing the contents of processing of the video conference terminal according to the embodiment of the present invention.

  In the flowchart of FIG. 6, first, the CPU 201 determines whether or not a video conference has started (step S601). The start of the video conference is determined, for example, by making a connection request to another video conference terminal 110 via the communication I / F 207 based on the operation of the operation unit 206 by the participant. Also, the determination is made by receiving a response from another video conference terminal 110 via the communication I / F 207.

  In step S601, waiting for the start of the video conference and starting (step S601: Yes), the CPU 201 controls the video I / F 204 and takes an image of the entire local site by the camera 113 (step S601: Yes). Step S602). For example, the entire image of the local site is controlled by driving and controlling the camera 113 such as pan and tilt, and images of all participants participating in the video conference at the local site are captured.

  The CPU 201 creates an image table based on the participant image captured in step S602 and stores it in the storage medium 208 (step S603). Specifically, for example, the image table includes image information of each participant as a subject as in the image table 300a illustrated in FIG. 3, and position information that is the imaging direction of the camera 113 when each participant is imaged. Is a table in which and are associated with each other.

  The CPU 201 controls the video I / F 204 to capture and output a participant image with the camera 113 (step S604). The participant image in step S604 is captured by the camera 113 according to, for example, an initial setting direction such as the front of the video conference terminal 110 or an imaging direction according to an operation instruction from the participant. The participant image is output to the video conference terminal 110 at another base via the communication I / F 207, for example.

  The CPU 201 determines whether or not the video conference is ended (step S605). The video conference is terminated based on, for example, an instruction by the participant operating the operation unit 206 or an instruction from another video conference terminal 110 received via the communication I / F 207.

  In step S605, when the video conference is ended (step S605: Yes), the series of processes is ended as it is. If the video conference is not ended in step S605 (step S605: No), the CPU 201 determines whether or not the microphone 114 has detected an utterance from a participant different from the participant image captured in step S604. (Step S606).

  If no utterance is detected in step S606 (step S606: No), the process returns to step S604 and the process is repeated. In step S606, when an utterance is detected (step S606: Yes), the CPU 201 determines whether or not the utterance continues for a predetermined time or more (step S607).

  In step S607, when the utterance has not continued for a predetermined time or longer (step S607: No), the process returns to step S604 and the process is repeated. That is, by confirming an utterance for a predetermined time or longer, an utterance with only consent or an utterance less relevant to a video conference can be made irrelevant.

  In step S607, when the utterance continues for a predetermined time or longer (step S607: Yes), the CPU 201 specifies the speaker who is the participant whose utterance is detected in step S606 and step S607 (step S608). For example, the speaker is specified by specifying image information in the image table created in step S603 based on the imaging direction of the camera 113a determined based on the direction in which the utterance is detected.

  For the speaker identified in step S608, the CPU 201 uses the alternative image information, which is the image information in the image table created in step S603, as the speaker image to the video conference terminal 110 at another base via the communication I / F 207. Output (step S609).

  In order to image the speaker identified in step S608, the CPU 201 executes pan / tilt control of the camera 113 via the video I / F 204 with the direction in which the utterance is detected as the imaging direction (step S610).

  The CPU 201 determines whether a speaker is included in the imaging range of the camera 113 that is pan / tilt controlled in step S610 via the video I / F 204 (step S611). The determination as to whether or not the speaker is included may be made, for example, based on the similarity between the participant in the alternative image information and the participant to be imaged.

  In step S611, when the speaker is not within the imaging range of the camera 113 (step S611: No), the process returns to step S610 to repeat the process. If the speaker is within the imaging range of the camera 113 in step S611 (step S611: Yes), the CPU 201 sets the speaker as a participant image to be imaged by the camera 113 (step S612), and proceeds to step S604. To do. In other words, in step S612, the image to be output to the video conference terminal 110 at another site is switched from the substitute image information output in step S609 to the participant image of the speaker captured by the camera 113. It becomes.

  The communication method in each component of the present invention and each process or each function of the embodiment of the present invention will be described in association with each other in the recording process according to the present invention by the processing of the CPU 201 and the storage medium 208 in step S603. Processing is executed. By the processing of the CPU 201 in steps S606 to S608, the determination process, the detection process, and the specific process according to the present invention are executed. The processing of the control process according to the present invention is executed by the processing of the CPU 201 and the video I / F 204 in step S610. By the processing of the CPU 201 and the communication I / F 207 in step S609, the output process according to the present invention is executed.

  As described above, according to the embodiment of the present invention, when performing pan / tilt control of a camera, an image of a participant after pan / tilt control can be output during pan / tilt control.・ You can check the information after tilting. In particular, when changing the imaging target to another participant who has spoken when the participant is being imaged by the camera, during the pan / tilt control of the camera, output alternative image information indicating the participant who has spoken, Since participants who have spoken quickly can be confirmed, a smooth video conference can be promoted.

  In addition, according to the embodiment of the present invention, since it is configured to determine whether or not the detected utterance has continued for a predetermined time or more, an utterance only for consent or an utterance less relevant to the video conference can be performed. As irrelevant, it is possible to quickly confirm participants who have made utterances closely related to the video conference, and to prevent unnecessary switching of images.

(Other variations)
In the embodiment of the present invention, the case where the participant is imaged has been described in particular. However, the present invention is not limited to this. Specifically, an image of an article related to the video conference may be performed. In this case, instead of detecting an utterance, the vibration, sound, or light emission that occurs during the presentation as a change in the state of the article is detected, or the pan / tilt control of the camera is performed according to the input of the imaging instruction by the participant It becomes. By doing in this way, broad application of the imaging target object relevant to a meeting can be aimed at regarding this invention.

  Further, in the embodiment of the present invention, in the flowchart of FIG. 6, it has been described that the alternative image information of the speaker is output when the utterance continues for a predetermined time or more, but the present invention is not limited to this. Specifically, for example, when the volume of the utterance is equal to or higher than a predetermined value, when the utterance is a predetermined voice color indicating a pre-registered participant, or at least one of the utterance time, the volume, and the voice color The configuration may be such that alternative image information is output when one of the conditions is satisfied. By doing so, it is possible to preferentially select an important speaker, so that an accurate conference can be promoted.

  In the embodiment of the present invention, in particular, in the flowchart of FIG. 6, when the speaker is included, the output target is described as switching to the participant image of the speaker captured from the alternative image information. There is no limit. Specifically, for example, it may be determined whether pan / tilt up to the imaging direction controlled in step S610 is completed. By doing in this way, it becomes possible to switch the image of the output object by a simple mechanism.

  In the embodiment of the present invention, the alternative image information to be displayed on the display is described as a configuration for receiving the output from the video conference terminal that is the transmission source. However, the present invention is not limited to this. Specifically, for example, as a configuration in which an image table is output in addition to a participant image from the transmission source video conference terminal, when the transmission source video conference terminal detects an utterance, information about the speaker is being driven while the camera is being driven. Is output. The video conference terminal of the transmission destination may be configured to read the substitute image information from the already stored image table when the information of the speaker is received.

  In the embodiment of the present invention, the image of each participant taken after the start of the video conference is recorded as the substitute image information. However, the present invention is not limited to this. Specifically, an entire image including information registered in advance and participants may be used. Further, a configuration may be adopted in which a participant image captured during the progress is recorded as substitute image information in accordance with the progress of the video conference. With reference to FIG. 7, a case will be described in which an image table is created without imaging the entire image after the start of the conference. FIG. 7 is a flowchart showing the contents of processing of the video conference terminal according to the modification of the present invention.

  In the flowchart of FIG. 7, first, the CPU 201 determines whether or not a video conference has started (step S701). In step S701, after waiting for the video conference to be started and starting (step S701: Yes), the CPU 201 controls the video I / F 204 to capture and output a participant image by the camera 113. (Step S702). The participant image in step S702 is imaged by the camera 113 according to, for example, an initial setting direction such as the front of the video conference terminal 110, an imaging direction according to an operation instruction from the participant, or the like. The participant image is output to the video conference terminal 110 at another base via the communication I / F 207, for example.

  The CPU 201 updates the image table stored in the storage medium 208 based on the participant image captured in step S702 (step S703). In other words, when the participant image captured in step S702 exists in the image table, the image information is updated to the latest, and when the participant image does not exist in the image table, it is newly registered. Note that it is not always necessary to update the image table when it exists in the image table, and it may be updated when there is a change in image information or position information, or when a predetermined time has elapsed from past image information.

  The CPU 201 determines whether or not the video conference is ended (step S704). The video conference is terminated based on, for example, an instruction by the participant operating the operation unit 206 or an instruction from another video conference terminal 110 received via the communication I / F 207.

  In step S704, when the video conference is ended (step S704: Yes), the series of processes is ended as it is. If the video conference is not ended in step S704 (step S704: No), the CPU 201 determines whether or not the microphone 114 has detected an utterance from a participant different from the participant image captured in step S702. (Step S705).

  If no speech is detected in step S705 (step S705: No), the process returns to step S702 and the process is repeated. In step S705, when an utterance is detected (step S705: Yes), the CPU 201 determines whether or not the utterance has continued for a predetermined time or more (step S706).

  In step S706, when the utterance has not continued for a predetermined time or longer (step S706: No), the process returns to step S702 and the process is repeated. In step S706, when the utterance continues for a predetermined time or longer (step S706: Yes), the CPU 201 displays the alternative image information of the uttered participant in the image table based on the utterance direction detected in step S705. It is determined whether or not it has been registered (step S707).

  In step S707, if not registered (step S707: No), the process proceeds to step S709. If registered in step S707 (step S707: Yes), substitute image information that is image information registered in the image table for the speaker who is a participant whose utterance has been detected in step S705 and step S706. Is output to the video conference terminal 110 at another site via the communication I / F 207 as an image of the speaker (step S708).

  Since the CPU 201 captures an image of a speaker who is a participant whose utterance has been detected in steps S705 and S706, the pan / tilt control of the camera 113 is performed via the video I / F 204 with the direction in which the utterance is detected as the imaging direction. Is executed (step S709).

  The CPU 201 determines whether or not a speaker is included in the imaging range of the camera 113 that has been pan / tilt controlled in step S709 via the video I / F 204 (step S710). The determination as to whether or not the speaker is included may be made, for example, based on the similarity between the participant in the alternative image information and the participant to be imaged.

  If the speaker is not within the imaging range of the camera 113 in step S710 (step S710: No), the process returns to step S709 and the process is repeated. In step S710, if the speaker is within the imaging range of the camera 113 (step S710: Yes), the CPU 201 sets the speaker as a participant image to be imaged and captured by the camera 113 (step S711), and proceeds to step S702. Transition.

  As described above, according to the modification of the present invention, since the alternative image information can be marched to the latest participant image, the alternative image information to be output is made fresher information and the video conference is accurately performed. can do.

  In the above description, the embodiment and some of the modifications have been described as separate examples, but the present invention is not limited to this. That is, the methods according to the embodiments and some of the modifications may be used in appropriate combinations.

  Note that the processing methods described in the embodiments and modifications of the present invention can be realized by executing a communication program prepared in advance on a computer such as a personal computer or a workstation. The communication program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, and a DVD, and is executed by being read from the recording medium by the computer. The program may be a transmission medium that can be distributed via a network such as the Internet.

100 Video conference system 110 (110a, 110b) Video conference terminal 111 (111a, 111b) Main body 112 (112a, 112b) Display 113 (113a, 113b) Camera 114 (114a, 114b) Microphone 115 (115a, 115b) Speaker a1 , A2, b1 Participants 150 Network 200 Bus 201 CPU
202 RAM
203 ROM
204 Video I / F
205 Voice I / F
206 Operation unit 207 Communication I / F
208 storage media

Claims (7)

A terminal device that outputs a subject image captured by an imaging unit to a terminal device at its own base or another base,
Recording means for recording substitute image information indicating a subject imaged by the imaging unit in association with position information of the subject;
A detecting means for detecting a state change related to the subject at the own base or another base;
A determination unit that determines the subject to be imaged by the imaging unit based on a state change detected by the detection unit;
Specifying means for specifying the position information of the subject determined by the determining means;
Control means for performing drive control of the imaging unit to image the subject determined by the determination means based on the position information specified by the specification means;
During drive control by the control means, output means for outputting the substitute image information indicating at least the subject determined by the determination means;
A terminal device comprising: The detection means detects a user's utterance as the subject as the state change,
The terminal device according to claim 1, wherein the determining unit determines the user positioned in a detection direction in which an utterance is detected by the detecting unit as the subject to be imaged.   The terminal device according to claim 2, wherein the control unit performs drive control of the imaging unit when an utterance detected by the detection unit continues for a predetermined time or more.   4. The recording device according to claim 1, wherein the recording unit records the substitute image information based on the subject video imaged by the imaging unit in association with the position information based on the imaged position. A terminal device according to any one of the above.   When the subject video is captured by the imaging unit that is driven and controlled by the control unit, the recording unit converts the substitute image information already recorded into the substitute image information based on the captured subject video. The terminal device according to claim 1, wherein the terminal device is updated.   The output means switches the information to be output to the terminal device at the local site or the remote site from the video imaged by the imaging unit during the drive control to the substitute image information recorded by the recording unit. The terminal device according to any one of claims 1 to 5. A processing method for outputting a subject video imaged by an imaging unit to a terminal device at its own base or another base,
A recording step of recording substitute image information indicating a subject imaged by the imaging unit in association with position information of the subject;
A detection step of detecting a state change related to the subject at the local site or another site;
A determination step of determining the subject to be imaged by the imaging unit based on the state change detected by the detection step;
A specifying step of specifying the position information of the subject determined by the determining step;
A control step of performing drive control of the imaging unit so as to image the subject determined in the determination step based on the position information specified in the specifying step;
During the drive control by the control step, an output step of outputting the substitute image information indicating at least the subject determined in the determination step;
The processing method characterized by including.