JPH08316953A - Electronic conference system - Google Patents

Abstract

PURPOSE: To use the voice data transmitted and received in a multimedia electronic conference system and to realize the authentication of conference participants and the management of the right to speaking. CONSTITUTION: When a participant in a conference explains contents of characters and graphics with a voice while drawing them in a shared window, input events of document data, picture data, and voice data successively occur, and the speaker is identified by a speaker recognition process 114 with respect to voice data. If voice data from speakers other than preliminarily designated specific speakers is detected, the transmission processing of voice data is inhibited or the sound volume of transmitted voice data is reduced. Thus, the trouble that the voice or the like of a person who is not a just participant in the conference or a participant in the conference who doesn't have the right for speaking is sent as voice data to participants in the conference of the other client computers is prevented.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an electronic conference system, and more particularly to an electronic conference system using a plurality of computers connected to a network.

[0002]

2. Description of the Related Art In recent years, an electronic conferencing system using a computer network has been receiving attention. The conventional electronic conferencing system is realized by a host computer of a personal computer communication service using a telephone line.

That is, in a conventional electronic conferencing system using a personal computer communication service, conference data such as document data sent from each personal computer is temporarily stored in a host computer, and the data is used by other conference participants. It is read by a personal computer as appropriate. As a result, information is exchanged between personal computers at remote locations.

However, in such a system using the personal computer communication service, the data transfer rate is slow and data is exchanged via the host computer, so that it is difficult to realize the real-time property required for the conference. Is.

Therefore, recently, an electronic conference system using a LAN has begun to be developed. In this system, conference data is directly exchanged between a plurality of personal computers connected to the LAN without using a host computer. Therefore, the speed of data exchange can be increased as compared with the system configuration using the host computer described above.

[0006]

However, in the conventional system form using the LAN, the collation process of the conference participants is limited to the use of the personal computer's password function, and the personal computer is used. It was difficult to determine whether the person participating in the conference was a legitimate conference participant. In addition, in a system that can handle voice data as conference data, voices (for example, soliloquy, chat, etc.) from a person who is not a legitimate conference participant or a conference participant who does not have a speaking right can participate in other conferences as voice data. However, there was a problem that legitimate remarks became difficult to hear.

Further, the conventional system is only capable of exchanging conference data, and saving and storing the conference data.
Functions such as reproduction and minutes creation were not realized. The present invention has been made in view of these points,
It is an object of the present invention to provide a multimedia-compatible electronic conferencing system capable of realizing authentication of conference participants and management of a speaking right by using voice data transmitted / received as conference data.

[0008]

Means and Actions for Solving the Problems
In an electronic conferencing system for exchanging conference data including text, image, or audio data between a plurality of computers connected to a network, each computer is
In response to the occurrence of an input event of character, image, or audio data, conference data transmitting means for transmitting data corresponding to the occurred event as conference data to all other computers, and a conference from the other computer. A means for displaying the character or image transmitted as data together with the character or image generated by its own computer on the same screen, and reproducing the voice data transmitted as conference data from the other computer. Means, speaker identification means for identifying a speaker of the input voice data when an input event of the voice data occurs, and voice data from a speaker other than a specific speaker designated in advance. When it is detected, the transmission processing by the conference data transmission means is prohibited, or the conference data transmission means Therefore characterized by comprising a means for attenuating the sound volume of the audio data transmitted as conference data.

In this electronic conferencing system, for example, when a conference participant draws characters or figures in a shared window and explains the contents by voice, input events of document data, image data, and voice data are successively performed. Although generated, the speaker identification process is performed on the voice data. When it is detected that the voice data is from a speaker other than the specific speaker designated in advance,
The transmission process by the conference data transmitting unit is prohibited, or the volume of the audio data transmitted as the conference data is attenuated by the conference data transmitting unit. As a result, voices (for example, soliloquy, chats, etc.) from persons who are not legitimate conference participants or who do not have the right to speak can be prevented from being sent as voice data to other conference participants. It is possible to efficiently realize the authentication of the person and the management of the floor.

[0010]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows the basic configuration of an electronic conference system according to an embodiment of the present invention. This electronic conference system is a system using a computer network system, and is realized by a plurality of client computers 11 to 13 and a server computer 21 connected on a LAN 10. In this system,
Each conference participant, while using the corresponding client computer, attends a conference with a conference participant who operates another client computer at a remote location.

In this case, the shared windows W1 to W for displaying the same conference data are respectively displayed on the plurality of client computers 11 to 13 connected to the LAN 10.
3 is opened, and conference data is exchanged using the shared windows W1 to W3. In addition, the server computer 21 is mainly used for storing / reproducing the contents of the meeting, creating the minutes, and the like.

Each of the client computers 11 to 13 is realized by a personal computer, a workstation, or the like, and a CPU that controls processing of programs and various peripheral devices, a memory that stores programs and various data, an external storage device such as an HDD or FDD. , An input device including a keyboard, a mouse, a pen, a scanner, an output device such as a display monitor and a printer, and a communication device for connecting to the LAN 10.

Also, these client computers 1
1 to 13 include a headset with a microphone 11a to handle multimedia data as conference data.
13a and video cameras 11b to 13b are also provided.

The video cameras 11b to 13b are for inputting images to be sent to other client computers and the server computer 21, so that each conference participant can photograph his or her face or the conference material at hand. Used.

Each of the headsets 11a to 13a has a microphone and a speaker, and is used for inputting an audio signal to be sent to another client computer and the server computer 21 and for reproducing an audio signal transmitted from another client computer. To be done.

The shared windows W1 to W3 are screens provided by the window system executed on the client computers 11 to 13, respectively. The shared windows W1 to W3 have the same conference data (characters, graphics, moving images, etc.). ) Is displayed.

That is, the client computer 11
The conference participant who operates the
If you enter "cd ...", the character data "abcd ..."
Is the shared window W of the client computer 11.
1 and the shared windows W2 and W3 of the other client computers 12 and 13, respectively.

When a conference participant operating the client computer 12 draws a figure such as a circle by operating the mouse using a graphics tool supported by the window system, the circle figure is shared by the client computer 12. In addition to being displayed in the window W2, it is also displayed in the shared windows W1 and W3 of the other client computers 11 and 13, respectively.

Similarly, when a conference participant operating the client computer 13 draws a figure such as a square by mouse operation using a graphics tool supported by the window system, the figure of the square is displayed on the client computer 13. While being displayed in the shared window W3, the other client computers 11,
It is also displayed in each of the 12 shared windows W1 and W2.

As described above, in the shared windows W1 to W3, the data input by the client computers 11 to 13 are combined and displayed in the input order. As a result, all conference participants can hold a conference while viewing the same data.

Such control of the shared window is realized by the client computers 11 to 13 broadcasting the data input by each computer to all other computers.

In this case, since the data transmission is performed every time an input event occurs, the screen display of the client computer which receives the data becomes the same as the screen display of the client computer which inputs the data. Therefore, for example, when a conference participant who operates the client computer 13 draws a square by dragging the mouse, the dragging process (illustrated by the broken line) displayed in the shared window W3 of the client computer 13 is Will also be displayed on the shared windows W1 and W2 of the client computers 11 and 12.

Broadcasting of voice data is also performed each time the voice data is input, and, for example, a conference participant operating the client computer 11 uses the headset 1
When voice is input from 1a, the input voice data is reproduced from the headsets 12a and 13a of the client computers 12 and 13.

Further, each client computer is provided with a speaker recognition function to identify whether the input voice data is from a pre-defined conference participant or another person. It Audio data from conference participants will be sent to all other computers as normal, but audio data from other people will be blocked from sending or will be sent to other computers with the volume attenuated. Sent to all computers. In addition, the same speaker recognition is performed for reproduction of voice data received from another computer, and only voice data from a legitimate conference participant is reproduced. This makes it possible to hold an electronic conference including voice data only between predetermined conference participants.

For speaker recognition, for example, the speaker name (speaker ID)
For each time, a speaker dictionary that holds speaker identification information that indicates the characteristics of the speaker's voice data and a speaker dictionary that matches the speaker identification information extracted from the input voice data are used to identify the speaker. It can be realized by the function and. As speaker identification information,
For example, cepstrum analysis, linear predictive analysis, PARCO
It is possible to use parameters and the like extracted from the voice data for each sentence using R analysis or the like.

The server computer 21, which is realized by a personal computer or a workstation, receives conference data communicated between the client computers 11 to 13, saves / reproduces the conference data, and prepares minutes. To do. Similar to the client computer, the server computer 21 has a CPU that controls processing of programs and various peripheral devices, a memory that stores programs and various data, an HDD, and an FD.
An external storage device such as D, an input device including a keyboard, a mouse, a pen, a scanner, an output device such as a display monitor and a printer, and a communication device for connecting to the LAN 10 are provided.

On the display monitor of the server computer 21, a timetable window TTW as shown in the figure is displayed in synchronization with the communication of the conference data executed between the client computers 11-13.

The timetable window TTW displays the contents of the input event for each client computer on the Thai scale in order to visually present the progress of the conference.

The timetable window corresponding to each client computer is divided into three tice scales of a moving image, a sound and a figure, and an icon corresponding to an input event is displayed on each of these tice scales.

In this timetable window TTW, a voice input event is indicated by an "arrow" icon a1, and when a conference participant speaks, an "arrow" is displayed on the voice time scale of the client computer corresponding to the conference participant. The icon a1 is displayed.

The display position of the "arrow" icon a1 and the length of the arrow correspond to the occurrence time of a voice input event in the client computer and the voice input time (speaking time), respectively. An input event of character data, that is, text data is indicated by a "KB" icon a2, an input event of circular graphic data is indicated by a "circle" icon a3, and an input event of square graphic data is indicated by a "square" icon a4. .

In the icon arrangement of the timetable window TTW of FIG. 1, the conference participants of the client computer 11 input text data “abcd ...” While inputting voice, and the conference participants of the client computer 12 then input. This corresponds to a case where a circle figure is drawn and text data “xyz ...” Is input while inputting voice, and then a conference participant of the client computer 13 draws a square figure while inputting voice.

In addition, the timetable window TTW
Is used, for example, in a case of performing a conference data search function of extracting a speech of a specific conference participant, and editing and reproducing the conference data. The speaker recognition function described above is also used in the search of the conference data. That is,
By searching the conference data using the speaker identification information as the search condition, only the voice data of the specific speaker can be extracted and reproduced.

Next, referring to FIGS. 2 and 3, the shared windows W1 to W3 and the timetable window TTW will be described.
An example of a specific screen configuration of will be described. FIG. 2 shows the screen configuration of the shared window W1.

Here, the client computer 11
A conference participant operating is copying the conference material at hand using the camera 11b and explaining the contents of the conference material while drawing an illustration by hand using the pen tool on the shared window W1. It is shown.

In this case, the same image as the screen image on the shared window W1 is displayed on the shared windows W2 and W3 except for the cursor. Further, various operation buttons are arranged in each shared window. The operation buttons include a square button B1 for activating a square drawing tool and a circle button B for activating a circle drawing tool.
2. Pen button B3 for activating a drawing tool for drawing the trajectory of cursor operation, and character button B for activating a character input tool for displaying character input from the keyboard.
4. An eraser button B5 for activating a drawing tool that erases only the screen image under the trajectory of the cursor operation, a clear button B6 for activating the drawing tool that clears the entire screen image, and a broadcast that determines the input event transmission mode. There is a button B7.

The broadcast button B7 is used to specify either the shared data broadcast mode for notifying all the other client computers including the server of the input event or the personal data mode for notifying only the server of the input event. used. When conducting a conference, the shared data broadcast mode is usually selected. As a result, input events such as squares, characters, erasers, and clears are reflected in all shared windows W1 to W3.

FIG. 3 shows the timetable window TT.
An example of the screen configuration of W is shown. In this time table window TTW, the vertical axis shows the images of the faces of the conference participants who operate the client computers 11 to 13, and the horizontal axis shows the time scale showing the conference progress time. Here, it is shown that the timetable window of each conference participant is divided into a figure window corresponding to a character or figure input event and a window corresponding to a voice data input event. As described in Section 1, the timetable window corresponding to each conference participant may be divided into three tice scales of a moving image, a sound, and a figure.

In the graphic timetable window, in addition to the above-mentioned icons a1 to a2, icons corresponding to other tools prepared in the shared window of FIG. 2, for example, "pen" icon a5, "clear" icon a6. Is also displayed.

Further, in the timetable window TTW, various buttons B11 to B17 for calling various tools corresponding to the recording, reproducing, searching and editing functions of the conference data prepared in the server computer 21, and voice data. Buttons B18 to B22 for designating a recording / playback level and a playback speed are arranged.

Recording of conference data is performed in the following procedure. When the "record" button B15 is pressed, recording of conference data is started. In this case, the conference data is saved in a temporary file and can be saved later with a specified file name.

When the "pause" button B12 is pressed, recording is temporarily stopped. After that, when the "start / restart" button B11 is pressed, the recording operation is restarted. All voices and graphic data received during "interruption" are discarded.

When the "stop" button B13 is pressed, the recording operation is stopped. In this case, recording to the temporary file stops there. All stopped voices and graphic receptions are discarded.

The reproduction of the conference data is performed in the following procedure. When the participant's face on the timetable window TTW is clicked and the "play" button B14 is pressed, the conference data of only the specified individual is played.

Also, one icon on the timetable window TTW is clicked to click the "Play" button B14.
When is pressed, only the data corresponding to that icon is played. If you drag while clicking the icon, the playback range will be up to the point where you release it.

The selected icon is changed to the display in the selected state, and is returned to the original display when the reproduction is completed. During playback, you can specify the playback speed. The conference data is edited in the following procedure.

When the "Edit" button B17 is pressed, a dialog for using the edit tool for cutting and pasting is displayed. The recorded conference data can be edited by selecting a voice / graphic icon on the timetable window TTW using this cut / paste editing tool and cutting and pasting.

Next, a software configuration for realizing the electronic conference system of FIG. 1 will be described. In Figure 4,
Each process and data flow of the electronic conferencing system of this embodiment are conceptually shown.

In each of the client computers 11 to 13, a voice transmission process 111, a voice reception / reproduction process 112, a shared window process 113, a speaker recognition process 114, etc. are executed.

Sound transmission process (sound send) 111
Receives the voice spoken by each participant via the voice input device, and transmits it to the voice receiving / reproducing process 112 of the client computer of the other participant and the conference data saving / time log display process 21a of the server computer 21. Send.

Audio reception / playback process (sound recv) 1
Reference numeral 12 performs a process of receiving voice data transmitted from the voice transmission process 111 of the client computer of another participant or the voice reproduction process 21b of the server computer 21, and reproducing the voice data by the voice output device.

Shared window process (gtool) 113
Controls a shared window shared by the participants, and inputs a square, a circle, a pen, text, an eraser, and a dump image of the screen.
Input events related to these windows are displayed in the shared window of the client computer at the time when they are input, and the shared window process of the other participants' client computers and the conference data storage / time log display process of the server 21. 21a
Sent to.

Also, the shared window process 113
Shared window process 113 of another client computer or graphic reproduction process 21c of server 21
When data is received from, display it in a shared window.

The speaker recognition process 114 is for realizing the above-mentioned speaker recognition function, and performs speaker recognition using a speaker dictionary, whereby only voice data for a specific speaker is transmitted and received. To enable.

In the server computer 21, a conference data storage / time log display process 21a, a voice reproduction process 21b, a graphic reproduction process 21c, a speaker recognition process 21e, etc. are executed.

The conference data storage / time log display process (timelog) 21a receives voice data and shared window data from the client computers of the participants and stores the received data in the conference data storage file 21d.
Save as minutes data to. Further, the conference data storage / time log display process (timelog) 21a displays an icon in the above-mentioned Tice scale window TTW.

Further, the conference data storage / time log display process (timelog) 21a can retrieve and edit the conference record data stored in the file 21d, and delete, move, and delete icons on the Tice Scale window TTW.
According to operations such as copying, operations such as deleting figures and voice objects from the conference proceedings data, moving the data position, and closing idle time are performed.

Further, the conference data storage / time log display process (timelog) 21a has a reproduction control function of the conference record data, and in addition to the reproduction of the edited result, the reproduction of the whole conference record data and a specific participant. Play only data about,
It is possible to perform reproduction of a figure / speech object alone, reproduction of voice data relating to a specific speaker, or entire conference data including the voice data.

Sound play process (sound play) 21b
Saves the conference data / displays the time log (timelo
g) According to the information instructed from 21a, the conference data storage file 21d is read and the voice data is transmitted to the voice receiving / reproducing process 112 of each client computer. In this case, the audio data is transmitted in synchronization with the graphic reproduction process 21c.

Picture play process 21c
Saves the conference data / displays the time log (timelo
g) According to the information instructed from 21a, the conference data storage file 21d is read and the shared window data (characters, figures, etc.) is transmitted to the shared window process 113 of each client computer. In this case, the shared window data is transmitted by the voice reproduction process 21c.
It is done in synchronization with.

It should be noted that reproduction is not limited to broadcast, but by specifying the destination machine in the time log,
It is also possible to send only to a specific client.
The speaker recognition process 21e is for realizing the above-described speaker recognition function, and uses the speaker identification information registered in the speaker dictionary as a search condition to search the conference proceedings data to specify the speaker identification data. It enables to extract voice data about a speaker from conference proceedings data.

Next, with reference to FIG. 5, the configuration of the client computer required for data transmission will be specifically described. In the client computer, in addition to the voice transmission process 111, the voice reception / reproduction process 112, the shared window process 113, and the speaker recognition process 114 described in FIG. 4, a communication library 114, a keyboard device driver 115,
Kana-Kanji conversion process 116, mouse driver 117,
For example, the window system 118 typified by X window and the audio device driver 119 are executed.

The voice data is transmitted by the voice device driver 119 for controlling the headset and the speaker recognition process 1.
14, voice reception / playback process 112, voice transmission process 111, and communication library 114. Regarding the transmission of text data, a keyboard device driver 115 for controlling the keyboard, a kana-kanji conversion process 116, a shared window process 1
13, and controlled by the communication library 114,
The transmission of the graphic data is controlled by the mouse driver 117 that controls the mouse, the window system 118, the shared window process 113, and the communication library 114.

The transmission of voice data is executed in the following procedure. The audio device driver 119 is the headset 11
When a voice is input from a, it is output as voice data. This voice data is sent to the speaker recognition process 114, where it is recognized. That is, the speaker recognition process 114 analyzes the voice data output from the voice device driver 119, extracts the speaker identification information from the voice data, and compares the speaker identification information with the speaker dictionary 115. Recognize the speaker name of the input voice data. When the recognized speaker name is a conference participant designated in advance, the speaker recognition process 114 sends the audio data from the audio device driver 119 to the audio transmission process 111 as it is. On the other hand, if the recognized speaker name is a person other than the conference participant designated in advance,
The speaker recognition process 114 prohibits the transmission of voice data to the voice transmission process 111 so that the corresponding voice data is not reproduced on another client computer. Further, instead of prohibiting the transmission of the audio data, a volume attenuation instruction may be issued to the audio transmission process 111 and the audio transmission process 111 may be caused to execute a process of attenuating the volume of the corresponding audio data.

The voice transmission process 111 reads the raw voice data output from the voice device driver 119 via the speaker recognition process 114, adds a header part to the voice data, and generates a voice message packet,
It is sent to the communication library 116. The generation and transmission of voice message packets are repeated every time voice data is read. The header part includes a data ID indicating that the data is audio data, a sequence number indicating the generation order of the audio data, and the like.

When a volume attenuation instruction is received,
The voice transmission process 111 transmits voice message packets by processing voice data or adding volume decay instruction information to a header part or the like so that the volume at the time of reproduction of the relevant voice data is sufficiently attenuated. .

The voice message packet is sent to the clients and servers of all other participants via the communication library 114. In this case, a header for communication is added to the voice message packet, and the voice message packet is sent out on the LAN 10 as a communication packet.

Transmission of text data is executed in the following procedure. The kana-kanji conversion process 116 sequentially receives the key codes input from the keyboard device driver 115, converts them into kana-kanji characters, and generates character data corresponding to an actual character string. When such a character data input event occurs, the shared window process 11
3 adds a header part to the character data from the Kana-Kanji conversion process 116 to generate a character message packet. The header part of each character message packet contains a data ID indicating that the data is text data, a sequence number indicating the generation order of the text data, and the like.

The text message packet is sent to the clients and servers of all other participants via the communication library 114. In this case, a communication header is added to the character message packet and the character message packet is sent out on the LAN 10 as a communication packet.

The transmission of the graphic data is executed in the following procedure. The window system 118 notifies the shared window process 113 of a mouse event from the mouse device driver 117. Shared window process 113
In response to a mouse event, adds a header portion to event data (graphic data) for displaying graphics corresponding to the mouse event, and generates a graphic message packet. The header part of the graphic message packet is
It includes a data ID indicating that it is graphic data, a sequence number indicating the generation order of graphic data, and the like.

The graphic message packet is sent to the clients and servers of all other participants via the communication library 114. In this case, a header for communication is added to the graphic message packet, and the graphic message packet is sent out on the LAN 10 as a communication packet.

As described above, the voice, text, and graphic data are assembled into a message packet each time each data input event occurs, and these message packets are LA packets as communication packets in the order of occurrence.
Sent on N10.

Here, the data format of each of the message packet and the communication packet will be described with reference to FIGS. FIG. 7 shows the data format of voice, text, and graphic message packets.

The message header section added to each message packet includes sequence number information,
It includes data type information, data name information (data ID), data length information, and time information.

The sequence number information is a number indicating the order of event occurrence within the data type for each data type such as voice data, graphic data, and text data. In any of voice data, graphic data, and text data, The sequence number of the first data starts from 0, and the sequence numbers of the following data are incremented by 1.

The data type information indicates whether the corresponding data is a deviation of start data, continuous data, or end data, and this information is used for displaying an icon on the timetable window. For example, for voice data, when a participant generates a phrase,
Multiple voice message packets are typically generated. In this case, the data type information of the first voice message packet of the phrase becomes the start data, the data type information of the last voice message packet of the phrase becomes the end data, and the data type information of each voice message packet between them is the continuous data. Become. When the volume of audio data is attenuated, attenuation information can be added here.

The data name information (data ID) is for indicating the type of corresponding data, and indicates one of square, circle, pen, text, eraser, voice data, screen dump image, and clear. The data length information indicates the data length included in the corresponding data part. The time information indicates the time when the input event of the corresponding data occurs.

FIG. 8 shows the data format of each communication packet. Each communication packet is composed of a data part composed of the above-mentioned message packet and a communication header part added to it. The communication header portion includes source address information, socket number information, destination address information, etc., as shown in the figure.

The source address information indicates the address of the source computer of the communication packet. The socket number information indicates the process name (shared window process, voice receiving / reproducing process, etc.) of the destination computer that transmits the communication packet. In this case, when the data name information of the corresponding message packet indicates the data to be displayed in the window, the socket number information specifying the shared window process is used, and when the data name information of the corresponding message packet indicates the voice data. Is voice reception /
Socket number information is used to specify the reclaim process.

The destination address information indicates the addresses of all the destination computers that transmit the communication packet. In the shared data broadcast mode, the destination address information includes client addresses and server addresses corresponding to all other pre-registered conference participants. On the other hand, in the case of the personal data mode, the destination address information includes only the server address.

Next, with reference to FIG. 6, a specific configuration of the client computer required for data reception will be described. When the communication library 114 receives a communication packet from a client computer or a server computer of another participant, the communication library 114 distributes the communication packet to the voice receiving / reproducing process 112 and the shared window process 113 according to the socket number included in the communication packet. .

The voice receiving / reproducing process 112 sequentially receives voice message packets, writes those voice data in the sequence number sequence to the voice device driver 119 via the speaker recognition process 114, and reproduces voice. The writing of the voice data to the voice device driver 119 is repeated every time the voice message packet is received.

The voice data sent from the voice receiving / reproducing process 112 to the voice device driver 119 is once captured by the speaker recognition process 114, and the speaker recognition process is performed there. In this speaker recognition processing, the speaker recognition process 114 analyzes the voice data output from the voice reception / reproduction process 112, extracts the speaker identification information from the voice data, and then extracts the speaker identification information and the speaker dictionary. The speaker name of the voice data is recognized by collating 115. If the recognized speaker name is a pre-specified conference participant, the speaker recognition process 114 sends the audio data as it is to the audio device driver 119 to be reproduced. On the other hand, when the recognized speaker name is a person other than the conference participant designated in advance, the speaker recognition process 1
14 prohibits the audio data from being transmitted to the audio device driver 119 or issues a volume attenuation instruction to the audio device driver 119 so that the corresponding audio data is not reproduced so that the volume of the applicable audio data is sufficiently attenuated. Play it in the open state.

In this embodiment, as described with reference to FIG. 5, voice data filtering or sound volume attenuation processing is already performed by speaker recognition at the time of transmission. Therefore, the speaker recognition process at the time of reception is effective in improving reliability, but is not always necessary, and the speaker recognition may be performed either at the time of transmitting or receiving the speaker recognition.

The shared window process 113 sequentially receives the message packets relating to the window display and classifies them according to the data ID for each data type (circle, square, text, pen, eraser, screen dump image, clear). Then, the shared window process 113
The function (procedure) corresponding to the classified message packet is called, and the message packet is passed to it in sequence number order. As a result, for example, a circle message packet is sequentially sent to the procedure 1 for drawing a circle in sequence number order, and similarly, a square message packet is sequentially sent to the procedure 2 for drawing a square order by sequence number to display text. To the procedure 3, the character message packet is sequentially sent in the sequence number order. As a result, figures such as circles and squares and characters are displayed in the shared window.

As described above, in the conference system of this embodiment, for example, when a conference participant draws characters or figures in the shared window and explains the content by voice, the text, figures, and voice data Input events are sequentially generated, a data ID and a sequence number are added to each of the text, graphic, and voice data to generate a message packet. The message packet is broadcast and sent to all other client computers. In each client computer that receives the message packet, the data type of the message packet is identified according to the data ID and the like, and the data connection for each data type is recognized according to the sequence number. Therefore, the text,
Figure (circle, square, pen, eraser, screen dump image,
Clear) and voice data are mixed and input in any order, the type and connection of the data can be recognized. Then, the text and graphics are displayed in the shared window in the order of event occurrence, and the audio data is also reproduced in the order of event occurrence.

Therefore, it becomes possible to exchange a plurality of types of data such as characters, figures, and voices in real time between client computers while synchronizing them, and to sufficiently realize the real-time nature of data exchange required for a conference. It becomes possible to do. Further, since the event data such as voice is handled as packet data, even if the conference participant has a long speaking time, the packet of the character or figure in which the event occurred during the transmission of the voice data can be sent at any time. .

Further, since the voice data is filtered or the sound volume is attenuated by the speaker recognition processing performed at least at the time of transmitting or receiving, only the voice data of the designated conference participant can be transmitted and received.

Next, with reference to FIG. 9, the configuration of the server computer for storing the conference data and displaying the timetable will be specifically described. Upon receiving the communication packet from each client computer, the communication library 21e of the server computer 21 stores the communication packet in the conference data storage / time log display process 21a.
Pass to. Conference data storage / time log display process 21
a stores the message data of the received communication packet as conference record data in the conference data storage file 21d, and starts displaying an icon on the timetable window TTW.

When the conference data is stored in the conference data storage file 21d, each message data is actually stored in the temporary file, and then the conference data storage file 21 is stored as a different file for each participant.
stored in d. In this case, the index information of each file is generated. The index information manages each message data that constitutes a file by an event number, and the data recording position, data size, time information,
It has management information such as a data ID. The event number is also assigned to the icon displayed on the timetable window TTW, which links the icon on the timetable window TTW and the data in the conference data storage file 21d.

Also, the speaker recognition process 2 is included in the index information.
The speaker identification information generated by 1e is also used. That is, the speaker recognition process 21e extracts the speaker identification information from the voice data stored in the conference data storage file 21d to create the speaker dictionary 21f for the conference participants, and the speaker identification information as the index information. Generate as.
Therefore, the conference data stored in the conference data storage file 21d can be searched using the speaker identification information as the search condition, and the operation such as extracting and reproducing only the voice data related to a specific speaker becomes possible. Become.

In the icon display process, communication packets are classified for each client according to the source address, and the timetable window corresponding to the client is selected. Then, according to the data ID, the data type, and the time included in the message packet of the communication packet, the type of the icon, the arrangement position, and the like are determined.

Hereinafter, the procedure of the icon display processing executed by the conference data storage / time log display process 21a will be described in detail with reference to the flowchart of FIG.
The conference data storage / time log display process 21a performs the following processing each time a communication packet is received.

That is, the conference data storage / time log display process 21a first checks the source address included in the received communication packet (step S1).
1) Identify the client that transmitted the communication packet (steps S12 to S14).

If the packet is a communication packet from the client computer 11 (client A), the process P of steps S15 to S29 corresponding to the client A is performed. On the other hand, if the packet is a communication packet from the client computer 12 (client B) or the client computer 13 (client C), the process proceeds to the same process P prepared for each of those clients. In these processes P, the timetable windows for which icons are displayed are different from each other, but the steps constituting them are the same.

The case where a communication packet from the client A is received will be described below. The conference data storage / time log display process 21a checks the data ID included in the received communication packet (step S1).
5) It is determined whether the communication packet corresponds to a message packet of graphic data or voice data (steps S16 and S17). Here, all data types other than circle, square, pen, text, eraser, voice, screen dump image, and clear, except voice, are classified as graphic data.

If it is graphic data, the conference data storage / time log display process 21a determines whether it belongs to a circle, a square, a pen, a text, an eraser, a screen dump image, or a clear (steps S18, S19 ...). . If the figure data is a circle, the conference data storage / time log display process 21a performs the processing Q of steps S20 to S29.
Proceed to. On the other hand, if the graphic data is a square, a pen, a text, an eraser, a screen dump image, or a clear, the process proceeds to a similar process Q prepared corresponding to each data type. These processes Q are the same except that the types of icons to be displayed are different from each other. Also, when the communication packet is voice data, the process proceeds to a similar process Q prepared for each data type.

In process Q, the conference data storage / time log display process 21a first checks the data type information contained in the message packet (step S2).
0), it is identified whether the data of the message packet is start data, continuous data, or end data (steps S21 to S23).

If it is the start data, the conference data storage / time log display process 21a checks the time information included in the message packet to determine the icon display position on the time scale (step S24), and the message is displayed there. The left end portion of the icon corresponding to the data type of the packet is drawn (step S25).

In case of continuous data, save conference data /
The time log display process 21a checks the time information included in the message packet to determine the icon display position on the time scale (step S26), and draws the intermediate part of the icon corresponding to the data type of the message packet there. Yes (step S27).

In the case of end data, the conference data storage / time log display process 21a checks the time information included in the message packet to determine the icon display position on the time scale (step S28), and The right end portion of the icon corresponding to the data type of the message packet is drawn (step S29).

As described above, the data ID, data type, and time information included in the message packet are used to determine the timetable window in which an icon should be displayed, the type of icon to be displayed, and the icon display position on the timetable window. Is used to generate a timetable window TTW indicating the progress of the conference.

Next, the conference data search function provided in the server computer 21 will be described. In order to easily edit the conference data, this search function uses the icons on the timetable window TTW to specify search conditions, and the conference data stored in the conference data save file 21d according to the search conditions. Is a function to search for.

The procedure of the search process will be described below with reference to the flowchart of FIG. First, the user of the server 21 inputs search conditions (step S3).
1). To enter these search conditions, press the search button B16 on the timetable window TTW, and enter the client name and data ID in the input window that is displayed.
(Square, circle, pen, text, eraser, voice, screen image, clear), speaker name, or time.

The conference data storage / time log display process 21a checks whether or not valid search conditions have been input (step S32), and the specified search condition is other than the client name, data ID, speaker name, and time. If there is, an error is displayed (step S33).

If valid search conditions have been input, the conference data storage / time log display process 21a proceeds to the search processing of steps S34 to S38. That is, the conference data storage / time log display process 21a first checks whether or not the client name (machine name) is specified in the input search condition (step S34), and if the client name is specified, The search is started by referring to the conference data storage file corresponding to the client (step S35).

For example, as search conditions, a square, a circle, a pen,
When a data ID such as text, eraser, voice, screen image, or clear is specified, the conference data storage / time log display process 21a displays the ID of the data recorded in the corresponding conference data storage file. Search in sequence and find the first data that matches the search criteria. When time is specified as the search condition, the conference data storage / time log display process 21a
Searches the data size of the audio data recorded in the corresponding conference data save file in order and detects the first audio data with a data size that makes the playback time shorter than the time specified in the search condition. . Further, when the speaker name is specified as the search condition, the conference data regarding the specified specific speaker is detected by using the voice identification information of the speaker as the search condition.

The conference data storage / time log display process 21a sequentially performs such detection processing and sequentially displays or reproduces data that matches the search condition (step S).
36). At this time, if the edit button B17 is pressed and the cut function is selected, the data matching the search condition is automatically deleted from the conference data. As a result, for example, it is possible to search for very short speech content (things that are not important as conference data such as sneezing, coughing, whispering, and noise of conference participants) and delete them from the conference data.

If the reproduction button B14 is pressed to specify the reproduction of the conference data, only the data matching the search condition is broadcast to all the clients and reproduced. Thereby, for example, it is possible to perform an operation such as continuously reproducing only the voice data of a certain conference participant.

On the other hand, when the client name (machine name) is not specified in the input search conditions (step S3
4) All the conference data are searched, and the same processing as steps S35 and S36 is performed (step S).
37, S38).

Next, with reference to the flowchart of FIG. 12, a search process using the voice identification information will be described. Normally, a plurality of conference record data corresponding to a plurality of conferences are recorded in the conference data storage file 21d, and a case where only the conference data of a specific speaker is retrieved from the plurality of conference record data will be described as an example. .

First, the collation process for specifying the speaker from the input voice data is performed using the speaker dictionary 21f (step S301), and then the conference proceedings data in which the speaker participates is searched. It is determined whether or not (step 302). When performing a search, the voice identification information of the corresponding speaker is used as a search condition, and only the conference data of the specific speaker is retrieved from the plurality of conference record data (step S303).

It is also possible to extract only the conference data of a specific speaker by sequentially reading out the voice data from each conference record data, extracting the voice identification information, and collating the extracted voice identification information with the speaker dictionary 21f. it can.

Next, the speaker dictionary creating process performed by the speaker recognition processes 114 and 21e will be described with reference to the flowchart of FIG. Voice data is extracted from the conference data transmitted and received during the electronic conference, and the speaker identification information of the voice data is extracted by analyzing the voice data (steps S41 and S42). Then, for example, a speaker dictionary in which the correspondence between the speaker name (personal data ID of the speaker, etc.) input from the keyboard of each machine and the speaker identification information is defined is created (step S4).
3).

Next, the speaker recognition processing executed by the speaker recognition processes 114 and 21e will be described with reference to the flowchart of FIG. First, the names of conference participants (personal data IDs, etc.) are input from the server 21 or the like and notified to each client or the like (step S5).
1). Next, the speaker identification information is extracted from the conference data exchanged with the other machine or the voice data input from the machine (steps S52 and S53).

Next, the speaker identification information of the notified conference participant is selected from the speaker dictionary (step S54). Then, the speaker identification information extracted from the voice data is compared with the speaker identification information selected from the speaker dictionary, and it is determined whether or not the corresponding voice data belongs to the conference participant. (Step S55).

Next, with reference to the flowchart of FIG. 15, the sound volume attenuation processing of the voice data performed by the speaker recognition process 114 will be described. First, speaker verification processing is performed according to the procedure described with reference to FIG. 14, and it is determined whether or not the corresponding voice data is from a specific speaker (steps S61 and S62). If it is not from the specific speaker, the volume of the voice data is attenuated (step S63).

Next, the voice data filtering process performed by the speaker recognition process 114 will be described with reference to the flowchart of FIG. First, FIG.
Speaker verification processing is performed according to the procedure described in step 4, and it is determined whether or not the corresponding voice data is from a specific speaker (steps S71 and S72). If it is not from the specific speaker, the transmission / reception of the voice data is stopped (step S73).

As described above, in this embodiment, for example, when a conference participant using the client computer 11 draws characters or figures in the shared window and explains the contents by voice, the document data , Image data, and audio data input events are sequentially generated, and the speaker is identified for audio data. When it is detected that the voice data is from a speaker other than the specific speaker designated in advance, the transmission process of the voice data is prohibited or the volume of the transmitted voice data is attenuated. . As a result, a voice (for example, soliloquy, chat, etc.) from a person who is not an authorized conference participant or a conference participant who does not have the right to speak is sent as audio data to the conference participants of the other client computers 12 and 13. As a result, it becomes possible to efficiently realize the authentication of the conference participants and the management of the floor.

[0120]

As described above, according to the present invention, it is possible to realize the authentication of the participants in the conference and the management of the floor by using the voice data transmitted and received as the conference data.

[Brief description of drawings]

FIG. 1 is a block diagram showing a system configuration of an electronic conference system according to an embodiment of the present invention.

FIG. 2 is a diagram showing a shared window displayed on each client screen in the embodiment.

FIG. 3 is a diagram showing a timetable window displayed on the server screen in the embodiment.

FIG. 4 is a block diagram showing a software configuration for realizing the system of the embodiment.

FIG. 5 is a diagram for explaining message packet generation and transmission operations executed by a client provided in the system of the embodiment.

FIG. 6 is an exemplary view for explaining a message packet receiving and reproducing operation executed by a client provided in the system of the embodiment.

FIG. 7 is a diagram showing an example of a data format of a message packet generated by the client provided in the system of the embodiment.

FIG. 8 is a diagram showing an example of a data format of a communication packet transmitted from the client to the server in the system of the embodiment.

FIG. 9 is a diagram for explaining a communication packet receiving operation executed by the server provided in the system of the embodiment.

FIG. 10 is a flowchart for explaining a procedure of icon display processing on a timetable window executed by the server provided in the system of the embodiment.

FIG. 11 is a flowchart for explaining a procedure of conference data search processing executed by the server provided in the system of the embodiment.

FIG. 12 is a flowchart for explaining a procedure of voice data search processing executed by a server provided in the system of the embodiment.

FIG. 13 is a flowchart for explaining a procedure of a speaker dictionary creating process executed by a speaker recognition process provided in each machine of the system of the embodiment.

FIG. 14 is a flowchart for explaining a speaker recognition processing procedure executed by a speaker recognition process provided in each machine of the system of the embodiment.

FIG. 15 is a flow chart for explaining a procedure of volume reduction processing of voice data executed by a speaker recognition process provided in each machine of the system of the embodiment.

FIG. 16 is a flowchart for explaining a procedure of voice data filtering processing executed by a speaker recognition process provided in each machine of the system of the embodiment.

[Explanation of symbols]

10 ... LAN, 11, 12, 13 ... Client computer, 13 ... Server computer, 11a, 12a,
13a ... Headset, 11b, 12b, 13b ... Video camera, W1, W2, W3 ... Shared window, TTW
... Timetable window, 111 ... Voice transmission process, 112 ... Voice reception / reproduction process, 113 ... Shared window process, 114 ... Speaker recognition process, 116
... communication library 117 ... mouse driver 118 ...
Window system, 119 ... Audio device driver,
21a ... Conference data storage / time log display process, 2
1b ... voice reproduction process, 21c ... graphic reproduction process,
21d ... Conference data storage file.

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Office reference number FI Technical display location H04N 7/15 H04N 7/15

Claims (4)

[Claims] 1. An electronic conferencing system for exchanging conference data including character, image, or audio data between a plurality of computers connected to a network, wherein each computer receives an input event of the character, image, or audio data. In response to the occurrence, a conference data transmitting unit that broadcast-transmits data corresponding to the occurred event to all other computers as conference data, and a character or an image transmitted as the conference data from the other computer, Means for displaying on the same screen together with characters or images generated by its own computer, means for reproducing audio data transmitted as conference data from the other computer, and an input event for the audio data occurs Speaker that identifies the speaker of the input voice data when When it is detected that the data is voice data from a speaker other than a specific speaker designated in advance, the transmission processing by the conference data transmitting unit is prohibited, or the conference data transmitting unit transmits the conference data. And a means for attenuating the volume of the audio data transmitted as the electronic conferencing system. 2. The voice data reproducing means includes a speaker identifying means for identifying a speaker of the voice data transmitted as conference data from the other computer, and a speaker other than a pre-designated conference participant. 2. The electronic conferencing system according to claim 1, further comprising means for prohibiting the audio data reproduction process or attenuating the volume of the reproduced audio data when the audio data is detected from the audio data. 3. At least one of the plurality of computers
Or a predetermined server computer connected to the network, receives the conference data from each of the plurality of computers, and stores the received conference data as conference record data, and a means for storing the conference record data. A speaker search unit for generating speaker identification information for identifying a speaker from the voice data, and searching the conference proceedings data using the speaker identification information as a search condition. The electronic conference system described in 1. 4. An electronic conferencing system using a server computer and a plurality of client computers connected to a network, wherein each client computer, when an input event of character, figure, or voice data occurs, the event occurred For each data corresponding to, the time information indicating the event occurrence time and the data are characters,
Message generation means for generating message data by adding identification information indicating which type of figure or voice it belongs to and sequence number indicating the event occurrence order within the data type, and generation by this message generation means Means for broadcasting the created message data to all other computers via a network, and receiving the message data sent from the other computer and the message data created by the message creating means of the own computer, Means for displaying or reproducing each of the characters, figures, or voice data in ascending order of sequence number according to the identification information and the sequence number included in the message data, and inputting the input event of the voice data when the input event occurs. Speaker identification means for identifying the speaker of the generated voice data, and when it is detected that the voice data is from another speaker other than the specific speaker designated in advance, And a means for attenuating the volume of audio data transmitted to another computer as message data, wherein the server computer receives message data from each of the plurality of computers, and the message data And a icon indicating the type of an input event from each of a plurality of conference participants operating each of the plurality of computers according to the identification information and the time information included in the received message data. Generate a timescale screen that displays in chronological order And means for searching and editing the recorded conference proceedings data according to an icon operation on the time scale screen, which is a talk for identifying a speaker from voice data stored in the conference proceedings data. Means for generating speaker identification information and searching the conference proceedings data using the speaker identification information as a search condition; and a message included in the retrieved and edited conference data according to an icon operation on the time scale screen. An electronic conferencing system comprising: means for broadcasting data to all of the plurality of computers to reproduce the conference data.