JP2003186593A - Multiwindow display method and system thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a multiwindow display method and system thereof which is comprised of a structure enabling the layout construction of multiwindow displays that can realize smooth interaction by interlockingly switching display sizes and positions between the multiple windows displayed on a monitor. <P>SOLUTION: The multiwindow display method and system switch the display sizes between a window (80d) displayed as a main window among the multiple windows (80a to 80d) and a window (80b) selected as a main window next time among the rest of windows (80a to 80c) displayed as an auxiliary window, and also change the display position of the auxiliary window depending on the display position fluctuation of the main window. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system and the like for displaying a plurality of windows on a screen of a predetermined display device, and more particularly to a multi-window display method and system for enabling smooth dialogue progress in a bidirectional interactive system. is there.

[0002]

2. Description of the Related Art Hitherto, there has been proposed a two-way interactive system for realizing intimate dialogue such as counseling while displaying the images of the other interlocutors on the monitor in a state where the other interlocutors' eyes are aligned with each other. ing. In particular, Patent No. 29
No. 70661 discloses a technique of presenting various kinds of information together with the image of the other party's interlocutor displayed on the monitor in the two-way interactive system that enables such line-of-sight matching. Specifically, in order to present information other than the video of the other interlocutor's image while avoiding interruption of the dialogue, etc., enlargement / reduction of the video of the other interlocutor's image displayed on the monitor is centered around a specific part. The display is done.

[0003]

DISCLOSURE OF THE INVENTION The inventor has found the following problems as a result of detailed study on window display in a conventional interactive interactive system capable of line-of-sight matching. That is, when dialogue is conducted between remote places,
On the monitor faced by the interlocutor's self-confidence, in addition to the image of the interlocutor on the other side, windows such as explanatory diagrams and character boards are sequentially generated (displayed on the monitor). Then, depending on the situation at the time of the dialogue, a particular window (display content) naturally becomes the center of the topic. In particular, regarding the window displaying the image of the interlocutor, it is necessary to consider the line-of-sight matching (maintaining the line-of-sight matching state or intentionally not matching the lines of sight).

In the window display method described in Japanese Patent No. 2970661, the window of the other interlocutor is maintained while maintaining the line-of-sight matching state between the interlocutors, centering on a specific part in the window where the image of the other interlocutor is displayed. Enlargement / reduction is performed. However, in such a conventional window display method, no consideration is given to display control between windows when a plurality of windows are displayed on the monitor. In other words, in the conventional window display method, the line-of-sight matching state between the interlocutors during the dialogue is maintained with respect to the change in the display size, but since the display content does not follow the change in the topic, the smooth dialogue progresses. However, there was a problem in that

In addition, window generation in a conventional OS (operating system) such as a personal computer lacks a preset function of a window generation position and the like, and a correction operation is required after the display. There was a problem that it hindered the progress of the dialogue.

The present invention has been made in order to solve the above-mentioned problems, and by smoothly switching the display size and the display position between a plurality of windows displayed on the monitor, a smooth dialogue progress can be achieved. To provide a multi-window display method having a structure for enabling a feasible multi-window display layout construction, its system, a computer program for executing the display method, and a recording medium having the computer program recorded therein. It is an object.

[0007]

A multi-window display method and system according to the present invention is provided with a structure for sequentially switching and displaying a plurality of windows on a screen of a predetermined display device in a linked state.

Specifically, in order to realize the multi-window display method according to the present invention, the multi-window display system according to the present invention divides a plurality of windows into a main window and an auxiliary window having different ideographic sizes and displays them on one screen. Display means for displaying and display control means for performing smooth display switching between the main window and the auxiliary window so as to realize smooth dialogue progress are provided.

The display means displays a window selected as a main window among a plurality of windows to be displayed at a predetermined position on the screen, and displays the remaining windows as auxiliary windows in an area excluding the display area of the main window. Show window. The display sizes of the main window and the auxiliary window are preset, and the size of the auxiliary window is set smaller than the size of the main window. The display size when the main window is selected as the main window and the display size when the auxiliary window is selected may be preset for each window to be displayed.

The display control means switches a preset display size between the window displayed as the main window and the window selected next time among the windows displayed as the auxiliary windows. At the same time, the auxiliary window is rearranged according to the change in the display position of the main window.

The multi-window display method according to the present invention is realized by the multi-window display system including the display means and the display control means that perform the characteristic operation as described above.

Further, in the multi-window display system according to the present invention, the display control means newly displays a window previously linked with the selected window as the main window is displayed. It may be arranged as an auxiliary window other than the display area of the main window.

The multi-window display method and system as described above is applied to a two-way interactive system for indirectly carrying out two-way interaction in a state where the lines of sight are in line with each other indirectly via a transmission means, thereby providing a smoother dialog. Allow progress. Usually, in such a two-way interactive system, the line-of-sight match between the interlocutors is indirectly achieved by matching the line-of-sight of the interlocutor himself displayed on the monitor with the line-of-sight of the interlocutor himself. The display means causes the monitor on which the image of the other party's interlocutor is displayed in this manner to display another window in which information can be presented without complicated operations, thereby causing a state in which the dialogue is interrupted, such as occurrence of a line-of-sight disagreement state ( The necessary related information and knowledge will be sequentially provided while avoiding the intentional disabling of support).

Particularly in a two-way interactive system for realizing line-of-sight matching, the display means switches a window for displaying a video image of the interlocutor on the other side to switch a main window and an auxiliary window to a predetermined part (window). For example, it is preferable to switch the display size centering around the head of the other interlocutor corresponding to the position where the interlocutor himself / herself is imaged in order to enable smooth continuation of the conversation.

The multi-window display method as described above may be a program executed by a computer or the like, and in this case, the program may be distributed via a network whether wired or wireless. ,Also,
It may be stored in a recording medium such as a CD, a DVD, or a flash memory.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION The embodiments of the multi-window display method and system according to the present invention will be described below.
This will be described in detail with reference to FIGS. In the description of the drawings, the same parts will be denoted by the same reference symbols and redundant description will be omitted.

The present invention details the manner of communication dialogue between remote places using a character board (real-time input screen on which handwritten characters and characters input by a keyboard are displayed) displayed on a monitor screen and a slide show. As a result of examination, in such a communication dialogue, the dialogue progresses in the order of introductory talks between interlocutors at the beginning of the dialogue, followed by explanation slides, explanations and meetings using the character board, and finally, summary story. It was made because it was confirmed that the pattern nature was strong. That is, when displaying a character board or slide screen in addition to the video of the other interlocutor on the monitor, a certain layout pattern in which the screen at the center of the dialogue is the main window and the other is the auxiliary window is displayed. The inventor has found that a plurality of screens are prepared and the main windows are switched by interlocking each of these screens, whereby the dialogue proceeds very smoothly.

FIG. 1 is a diagram for explaining an example of a window switching display operation in the multi-window display method according to the present invention.

For example, as shown in FIG.
In the multi-window display method, the main window 80d (D: character board, slide screen, etc.) in which the main contents of the dialogue are displayed is displayed at a predetermined position on the multi-window screen 8a (monitor screen), Other auxiliary window 80a (A), auxiliary window 80b (B),
Each of the auxiliary windows 80c (C) is arranged in an area excluding the display area of the main window 80d (D).
Dialogue progresses in this state, and the topic is the auxiliary window 80b.
When moving to the display contents of (B), the interlocutor selects the auxiliary window 80d (B) as the next main window. By this selection (switching instruction by the interlocutor), FIG.
, The window 81b (B) is displayed as the main window on the multi-window screen 8b, while the window 81d (D) is switched to the auxiliary window.
Is redisplayed in an area other than the display area of the new main window 81b (B). The main window selection is
Even if you specify the auxiliary window you want to select as the main window with the cursor and you can specify it by dragging and dropping it on the main window, you also have an enlargement / reduction button for each window and press this button It may be possible to give an instruction by doing so.

As described above, according to the multi-window display method, the window 8 displayed as the main window is
0d (D) and the window 80b (B) selected as the next main window among the windows 80a to 80c (A to C) displayed as auxiliary windows, and the display size of the main window is changed. Rearrange the auxiliary windows in conjunction with display position changes.
That is, since the remaining windows are rearranged as auxiliary windows in conjunction with the redisplay of the main window that is sequentially selected from the plurality of windows displayed on the monitor screen, smooth operation via slides or characters is performed. It is possible to construct a layout of a multi-window display that can realize dialogue progress.

The main window and the auxiliary window are
These display sizes are set in advance, and the size of the auxiliary window is set smaller than the size of the main window. The display size when the main window is selected as the main window and the display size when the auxiliary window is selected may be preset for each window to be displayed.

Further, as shown in FIG. 1 (c), the window 82b previously linked with the display of the window 81b (B) selected as the main window is displayed.
b (B1) and window 82c (B2) may be newly arranged as auxiliary windows outside the display area of the main window. For example, a title display screen (either data input in real time or data pre-recorded in a database may be used) may be linked to focus the topic.

In addition, in the case of home learning and the like as various modifications, while the agenda and questions are displayed as an auxiliary window,
If the real-time input window is used as the answer sheet and the main window is displayed, home learning and the like will proceed smoothly. By displaying all the agenda items as an auxiliary window, it becomes easier to grasp the pace allocation until the end of the dialogue. By linking the question display window and the real-time input window to each other, a multi-window display corresponding to the progress of the sequential learning becomes possible. The real-time input window is a screen for accepting input such as handwritten characters, keyboard input characters, voice, etc. according to the dialogue at that time, and preferably has a paging function.

A multi-window display system (multi-window display system according to the present invention) for implementing the above-described multi-window display method according to the present invention is
For example, it has a structure as shown in FIG. That is, FIG. 2 is a diagram showing a schematic configuration of the multi-window display system according to the present invention.

The multi-window display system shown in FIG. 2 is a data processing device 2 such as a personal computer.
6 and a monitor 21. This data processing device 26 is
A data input / output unit 261 (indicated by I in the figure) for capturing the image of the other party's interlocutor transmitted via the transmission means 27.
/ O)), and a data input / output unit 265 (in the figure, for exchanging data between the memory 262 serving as screen information to be displayed and a work space and an external recording device such as a CD, a DVD, or a flash memory. I / O), control means 263 (display control means) for performing smooth display switching between the main window and the auxiliary window, and a screen of the monitor 21 according to a predetermined display layout instructed by the control means 263. Drawing unit 2 that displays multiple windows on top
64 (display means) is provided. A computer program for executing the multi-window display method according to the present invention may be recorded in an external recording device such as a CD.

Here, the drawing unit 264 displays a window selected as a main window among a plurality of windows to be displayed at a predetermined position on the screen, and an auxiliary window in an area excluding the display area of the main window. To display the rest of the windows.

The control means 263 switches the preset display size between the window displayed as the main window and the window selected next time among the windows displayed as the auxiliary windows. , Rearrange the display position of the auxiliary window according to the change of the display position of the main window.

Next, the operation of the multi-window display method according to the present invention will be described with reference to the flow chart shown in FIG. Note that this operation is executed by the data processing device 26 having the structure shown in FIG. 2, and in the following description, the data processing device 26 uses the layout of the multi-window screen 8a shown in FIG. The operation of changing to the multi-window screen 8b as shown in FIG. 1B will be described.

First, a window 80b (B) in which an interlocutor in front of the monitor 21 is displayed as an auxiliary window.
When a switching instruction is made to change to the main window (step ST10), the control means 263 secures the display area of the main window on the screen of the monitor 21 (step ST20). Then, the control means 263 secures an area excluding the display area of the main window as an auxiliary window display area (step ST30), and determines a window to be displayed in the display area of the auxiliary window and determines its arrangement (step ST30). Step ST40). In this example, the window 8 selected as the main window
0b (B) has no pre-linked windows, so the other windows 80a (A), 80c (C), 80
Only d (D) is selected as the auxiliary window. The window 80b selected as the main window
When there is a window linked in advance to (B), windows 82b (B1), 82c are added as new auxiliary windows as shown in FIG. 1 (c), for example.
(B2) may be selected.

As described above, when the main window and the auxiliary window to be displayed are selected, the control means 263 finally determines the layout on the screen of the monitor 21 (step ST50), and the monitor according to the determined layout. The drawing unit 264 is instructed to draw each window on the screen 21 (step ST60). By such a series of operations, the multi-window screen 8b as shown in FIG. 1B is displayed.

In the switching display from the multi-window screen 8a shown in FIG. 1A to the multi-window screen 8b shown in FIG. 1B, the windows selected as auxiliary windows do not overlap each other. However, if there are many windows selected for display in the display area of the auxiliary window, for example, as shown in FIG.
Alternatively, the windows to be displayed in the display area of the auxiliary window may be displayed in a partially overlapped state with each other.

INDUSTRIAL APPLICABILITY The multi-window display method and system according to the present invention are suitable for home learning such as counseling between remote places and English conversation based on the form of dialogue indirectly facing each other via transmission means. When applied to the interactive system, it enables smooth dialogue progression. In particular, in counseling or the like, it is preferable that the lines of sight of the two persons are matched with each other even when they are faced to each other via a network. Therefore, in this specification, it is considered that the multi-window display system can be applied, and the structure of a bidirectional interactive system that enables dialogue in a state of line-of-sight matching between remote places and realization of line-of-sight matching using the system. The method will be described. Note that such a system and a method for realizing line-of-sight matching are described in Japanese Patent No. 3074477.

FIG. 5 is a logical block diagram showing the entire interactive interactive system that enables line-of-sight matching between interlocutors at remote locations. This system realizes a two-way dialogue between an interlocutor 3A (first interlocutor) at point A (first point) and an interlocutor 3B (second interlocutor) at point B (second point). It is a system that does. In particular, this system is a counseling system that establishes intimate relationships of trust even between interlocutors in remote areas and is a system that aims for home learning such as English conversation, so at least one interlocutor, It is intended for counselors such as teachers, doctors, and psychologists. Therefore, this system has various functions and configurations so that these counselors can provide information necessary for counseling and an ideal counseling environment.

Specifically, at the point A, (a) a chair 4A on which the interlocutor 3A (for example, counselor) sits, and (b)
The table 2A to which the interlocutor 3A faces, and (c) the transmission means 2
A terminal device 20A equipped with a mechanism for matching the lines of sight between the interlocutors via 7 is installed. On the other hand, at the point B, (a) a chair 4B on which an interlocutor 3B (for example, a client who receives counseling) sits, (b) a table 2B to which the interlocutor 3B faces, and (c) an inter-interactive person via the transmission means 27 The terminal device 20B having a mechanism for matching the lines of sight of Here, the transmission means 27 enables transmission / reception of image information and audio information between the terminal device 20A at the point A and the terminal device 20B at the point B, so that the optical fiber transmission line 31 that enables large-capacity transmission. (Main transmission line)
And a satellite communication transmission line 32 (a backup transmission line) and the like. The transmission means 27 may be wired or wireless. In addition, the transmission means includes a network such as a telephone line network that has already been installed, and an electronic bulletin board system (BBS: Bulletin Boar) equipped with various databases.
d System) is also included.

At point A, the chair 4A described above functions to define the position of the interlocutor 3A. This chair 4A is preferably fixed in order to keep a constant distance from the display unit 21A of the terminal device 20A. However, even if the chair 4A is not fixed, the chair 4A can be installed by installing the table 2A on the interlocutor 3A and the display unit 21A.
It becomes possible to maintain a constant distance from the display unit 21A (the fact that the existence of the table 2A serves as a reference for the psychological distance of the interlocutor 3A has been confirmed). The configuration of the terminal device 20B at the point B where the other party's interlocutor 3B (for example, a person who receives counseling) is the same.

At the point A, the terminal device 20A (1)
A monitor TV arranged on the table 2A for displaying the image of the interlocutor 3B based on the image information transmitted from the terminal device 20B at the point B via the transmission means 27.
A display unit 21A provided with 1A, (2) an image pickup unit 6A for taking in the image of the interlocutor 3A picked up by the CCD camera 5A as image information, and transmitting it to the terminal device 20B at the point B, (3 ) A voice output unit 23A for outputting the voice of the interlocutor 3B based on the voice information transmitted from the terminal device 20B at the point B via the transmission means 27.
(Speaker), and (4) a voice input unit 24A (microphone) for collecting the voice of the interlocutor 3A as voice information and transmitting it to the terminal device 20B at the point B. In addition, this system, as a counseling system, realizes more detailed communication.
Input a character or a graphic, display the input character or graphic on the display unit 21A, and transmit the character or graphic information to the terminal device 20B at the point B as a character or graphic information (keyboard, pointing, (Device, interface such as a touch panel), and (6) a data processing device 26A provided between each of the above-described components and the transmission means 27 and performing signal processing, transmission control, and multi-window switching display. The terminal device 20B at the point B is also configured in the same manner as the terminal device 20A at the point A described above.

Further, the terminal devices 20 at the points A and B
In A and 20B, the configurations of the imaging devices 600A and 600B (imaging means) will be described. For convenience of description, when referring to components common to the terminal devices 20A and 20B at points A and B, for example, the imaging device 60
The description will be made by omitting the letters A and B that distinguish each point like 0. In addition, unless specifically described, the terminal device 20A at the point A is described in principle, and the terminal device 20 at the point B having a common configuration is described.
Overlapping description of B is omitted.

This image pickup device 600 includes an ultra-small CCD camera 5 (5A) as an image pickup section, and a support mechanism 10 for mounting the CCD camera 5 at a predetermined position in a supported state (see FIG. 2).
Reference unit) and an image pickup unit 6 (6A) for controlling the CCD camera 5. The monitor TV 1 (1A) is placed on the table 2 (2A), and the interlocutor 3
(3A) is a chair 4 (4) provided with a height adjusting mechanism 40 (40A) placed at a distance L (m) from the monitor TV 1.
Sitting in A). This distance L (LA) is set to 0.5 m or more and 5 m or less. An outer diameter of 20 mm or less and a length of about 100 mm between the interlocutor 3 and the monitor TV 1 to be photographed.
A cylindrical microminiature CCD camera 5 is provided. Further, the CCD camera 5 is installed so that its imaging direction is directed to the interlocutor 3 who is imaged. Then, the image of the interlocutor 3 imaged by the camera 5 is transmitted as image information from the imaging unit 6 to the other interlocutor side (transmitted to the terminal device 20B at the point B via the transmission means 27).
Depending on the interlocutor, the space between the display unit 21 and the interlocutor may be weakened. Therefore, in order to cope with such a situation, it is preferable that the CCD camera 5 has a deep focal depth.

Next, the specific installation position of the microminiature CCD camera 5 will be described with reference to FIG.

First, the image pickup unit 6 of the image pickup apparatus 600 is installed on the display unit 21 for displaying the transmitted image of the other interlocutor. An image of the interlocutor himself / herself taken by the camera 5 is captured in the body of the imaging unit 6 (captured by the imaging unit 6 via the camera cable 50)
In addition to the changeover switch 6a, various adjustment switches 6b are provided for switching between the display of 1) and the display of the transmitted image of the interlocutor 3B. The switch 210 includes a power switch for turning on / off the power of the monitor TV 1. The CCD camera 5 is installed by a support mechanism 10 (robot arm) at a predetermined position in front of the monitor screen 8 on which the transmitted image 9 of the interlocutor 3B is displayed. The monitor screen 8 is a screen of 4 inches or more and about 40 inches.

Specifically, the CCD camera 5 is at a position forward w (cm) from the monitor screen 8, and the monitor screen 8
It is arranged near the head 9a of the interlocutor image 9 displayed in FIG.
The central axis of the cylindrical CCD camera 5 is located at a position h (cm) above the eye position of the interlocutor image 9 shown by the dotted line.

As described above, since the microminiature CCD camera 5 is located in the vicinity of the head above the eye position of the interlocutor image 9 displayed on the screen 8 of the large monitor TV 1, it is special for bidirectional conversation. No trouble will occur. For example, the CCD camera 5 is provided above the eye position (the position shown by the dotted line in the figure) of the interlocutor image 9 displayed on the screen 8 at h = about 10 (cm), and between the monitor TV 1 and the interlocutor 3. In the case of the system configuration in which the distance L is set to about 2.5 (m), the parallax angle is 2.3 °.
It is possible to sufficiently fall below the detection limit parallax angle of 3 ° (the interval w between the monitor screen 8 and the CCD camera 5 is 10 (c
Even if there is about m), there is no particular effect on the variation of the parallax angle). That is, if the eyes of the other party (the eyes of the image of the other interlocutor 3B whose monitor is displayed on the surface 8) can be clearly seen at the time of line-of-sight matching, even if the microminiature camera 5 is provided around the head, It has been confirmed that if the screen 8 of the monitor TV 1 is large, there is almost no hindrance to the interactive dialogue. As a result of the experiment, the size of this screen 8 is 35 cm in width and 26 cm in height.
It has also been confirmed that good two-way dialogue can be realized if the size is at least a certain level. It should be noted that if mutual interlocutors are acquaintances, there is a result that there is a tendency that there are few psychological obstacles even if the size of the screen is small, and it is advisable to properly use the screen size according to the application.

Further, the parallax angle will be described with reference to FIG. In this specification, the parallax angle means, as shown in FIG. 7, the line of sight of the interlocutor 3 to be photographed, which is directed toward the interlocutor image 9 displayed on the monitor screen 8, and the direction toward the CCD camera 5. The angle θ formed by the line of sight of the interlocutor 3 is defined. In other words, it means the shift angle of the installation position of the CCD camera 5 with respect to the line of sight of the interlocutor 3. In addition, the line-of-sight matching here means the 4th Japan Electrical Engineers Union Meeting (N)
o. 1998), it means that the unnaturalness due to the installation position of the CCD camera 5 is below the allowable limit. Quantitatively, the parallax angle is 4.5 ° or less in the left and right direction (both temple sides of the image 9 with respect to the eye center A of the interlocutor image 9), and 12 ° directly above (with respect to the image center A, The head 9b side of the image 9) and 8 ° below (the body side of the image 9 with respect to the image center A) or less are within the line-of-sight matching range. Of course, the smaller the parallax angle, the better, and the detection limit is 3 ° or less in the horizontal and vertical directions. Therefore, the area where the CCD camera 5 can be installed is the conical area shown in FIG. In addition,
This conical area is the center point A of the interlocutor image 9 displayed on the monitor screen 8 and the eye 30 of the interlocutor 3 (in this embodiment, the center of both eyes of the interlocutor image 9 is the center point of the interlocutor image 9). Defined as A), the line of sight 31 of the interlocutor 3 connecting
It is defined by the line-of-sight matching region 9b on the monitor screen 8 and the position of the eye 30 of the interlocutor 3 when the monitor is separated by a predetermined distance L (m).

FIG. 9 is a diagram for explaining an example of the window switching display operation when the multi-window display system is applied to the screen display to each interlocutor in the two-way interactive system shown in FIG. Is.

In such a two-way interactive system, the multi-window screen 8e (monitor screen) viewed by the interlocutor is used as the main window as shown in FIG. 9A. Window 8 showing the image of
5a is arranged so that the line-of-sight of the interlocutors matches. On the other hand, as an auxiliary window, the window 85b (A),
85c (B) is also displayed. The window 85a, which is the main window, is a CCD that images the interlocutor from the perspective of the interlocutor in order to maintain the line-of-sight matching state between the interlocutors.
The camera 5 and the head of the other party's interlocutor image in the window 85a are arranged so as to overlap each other. Further, windows 85b (A) and 85c (B) which are auxiliary windows are a title screen showing an agenda, a slide screen, a real-time input screen for handwritten characters, keyboard input characters, etc., and a screen for displaying supplementary explanation of the main topic of the dialogue. And so on. Further, when playing BGM during the dialogue, a display screen such as a BGM song selection list may be displayed.

When the multi-window display system is applied to the interactive interactive system that enables the line-of-sight matching as described above, the switching of the display of the window 85a on which the image of the other party's interlocutor is displayed is performed by the interlocutor. It is preferable that the line-of-sight matching state be maintained. Therefore, from the multi-window screen 8e as shown in FIG. 9 (a) to the multi-window screen 8f as shown in FIG. 9 (b), the window 85a displaying the image of the interlocutor is displayed. Even when the display is redisplayed as an auxiliary window having a small display size, it is preferable to rearrange the CCD camera 5 and the head of the image of the interlocutor on the other side so as to overlap with each other when seen from the interlocutor. When the image of the other party's interlocutor displayed on the multi-window screen is small, the CCD camera 5 and the other party's interlocutor's image need not necessarily overlap each other. Rather, the head of the interlocutor on the other side is hidden by the CCD camera 5 and becomes difficult to see. As described above, the condition for line-of-sight matching is that the eyes of the CCD camera 5 and the image of the other party's interlocutor are present in the area of the parallax angle of 3 °, and specifically about 3 cm at a position 50 cm away from the screen (Fig. It is h) in 6. Therefore, within this range, the line-of-sight matching state is maintained even if the heads of the CCD camera 5 and the image of the interlocutor on the other side are shifted. This image of the interlocutor and CCD camera 5
To shift the position, for example, adjust the window display position on the screen or adjust the support part of the CCD camera 5 to C
It is realized by changing the position of the CD camera 5 itself.

FIG. 10 is a flow chart for explaining a part of the window switching display operation shown in FIG. 9 in the multi-window display system including the special factor of maintaining the line-of-sight matching state as described above. In addition,
This operation is executed by the data processing device 26 having the structure shown in FIG. 2 and substantially the same operation as the flowchart shown in FIG. 3 is performed, but the display switching processing for other windows has priority. Then, the display switching process of the window in which the image of the other party's interlocutor is displayed (hereinafter, referred to as a portrait window) is performed. In the following description, the operation of the data processing device 26 changing the layout of the multi-window screen 8e shown in FIG. 9A to the multi-window screen 8f as shown in FIG. 9B will be described.

First, a window 85b (A) in which an interlocutor in front of the monitor 21 is displayed as an auxiliary window.
When a switching instruction is issued to change the window to the main window (step ST10), the control means 263 shifts to the person image window switching processing.

The control means 263 first confirms the current display state of the portrait window 85a (step ST
11) When the person image window 85a is the main window, a display area is secured so that the display size can be arranged as the size of the auxiliary window as shown in FIG. 9B (step ST13). The display area of the main window is also secured (step ST20).
When the person image window is switched from the main window to the auxiliary window in this way, the person is focused on the head of the image of the other party's interlocutor (the part overlapping the CCD camera 5 as seen from the interlocutor himself). This is done by reducing the size of the image window. On the other hand, when the person image window is the auxiliary window, it is confirmed whether or not it is selected as the main window for the next time (step ST12). If it is not selected as the main window, the above step S
If T13 (secure display area as auxiliary window) is executed and selected as the main window, the display area of the person image window is secured for the main window (step ST14). Even when the person image window is switched from the auxiliary window to the main window in this manner, the head of the image of the other party's interlocutor (the part overlapping with the CCD camera 5 as seen from the interlocutor himself) is centered,
This is performed by enlarging and displaying the person image window.

Subsequently, the control means 263 secures an area excluding the display area of the main window (including the display area of the person image window when the person image window is an auxiliary window) as the auxiliary window display area (step ST30), the selection of the window to be displayed in the display area of the auxiliary window and its arrangement are determined (step ST40). In this example, since the window 85b (A) selected as the main window has a window linked in advance, as shown in FIG. 9 (b), it is an auxiliary window including the person image window 86a. Main window 86 together with the window 86c (C) of
Window 86d (A1) pre-linked to b (A)
Will be placed as a new auxiliary window.

As described above, when the main window and the auxiliary window to be displayed are selected, the control means 263 finally determines the layout on the screen of the monitor 21 (step ST50), and the monitor according to the determined layout. The drawing unit 264 is instructed to draw each window on the screen 21 (step ST60). By such a series of operations, the multi-window screen 8f as shown in FIG. 9B is displayed.

In the switching display from the multi-window screen 8e shown in FIG. 9A to the multi-window screen 8f shown in FIG. 9B, the windows selected as auxiliary windows do not overlap each other. However, if there are many windows selected for display in the display area of the auxiliary window, for example, the remaining auxiliary windows are displayed as shown in FIG. 4 with the portrait window displayed in the foreground. The windows to be displayed in the display area may be displayed in a state where they overlap each other.

In the multi-window screens 8e and 8f shown in FIGS. 9A and 9B, the main window 85
auxiliary windows 85b, 85c and 86a, 86 displayed in the display area excluding the display areas a, 86b
Although d and 86c are displayed so as not to overlap each other, even if the other auxiliary windows are displayed so as to partially overlap as shown in FIG. 4, with the person image window displayed in the foreground. Good.

The means for realizing line-of-sight matching described in this specification is an example, and for example, OPTRONICS (199
9), No. 3, pp. 134-139, a CCD may be embedded in the monitor, or another line-of-sight matching method using a half mirror may be adopted.

[0055]

As described above, according to the present invention, the remaining windows are sequentially rearranged as auxiliary windows in association with the change in the display position of the main window. Allows you to more easily build layout changes for multi-window displays.

Further, according to the present invention, the main windows for displaying the information and video relating to the central topic can be switched according to the progress of the dialogue. However, the main windows switched and displayed in this way are merely for obtaining information. For each interlocutor who interacts via the monitor screen, it can be a psychological interface for producing a scene in which the thoughts and emotions in the dialogue are different.

[Brief description of drawings]

FIG. 1 is a diagram for explaining an example of a window switching display operation in a multi-window display method according to the present invention.

FIG. 2 is a diagram showing a schematic configuration of a multi-window display system according to the present invention.

FIG. 3 is a flowchart for explaining the operation of the multi-window display method according to the present invention.

FIG. 4 is a diagram for explaining another example of the window switching display operation in the multi-window system according to the present invention.

FIG. 5 is a diagram showing a configuration of a two-way interactive system that realizes line-of-sight matching between remote places via transmission means.

FIG. 6 is a diagram showing the image pickup device shown in FIG. 5, and a diagram for explaining a method of installing a CCD camera for line-of-sight matching.

FIG. 7 is a diagram illustrating a parallax angle.

FIG. 8 is a diagram showing an area where a CCD camera is to be installed.

9 is a diagram for explaining an example of a window switching display operation when the multi-window display system is applied to screen display to each interlocutor in the interactive dialog system shown in FIG.

10 is a flowchart for explaining a part of the window switching display operation shown in FIG.

[Explanation of symbols]

1A, 1B, 21 ... Monitor TV (display device), 3 (3
A, 3B) ... Interlocutors A, B, 5A, 5B ... CCD camera, 6 (6A, 6B) ... Imaging unit, 600 (600
A, 600B) ... Imaging device (imaging means), 26 (26
A, 26B) ... Data processing device, 263 ... Control means (display control means), 264 ... Drawing unit (display means).

Claims (10)

[Claims] 1. A multi-window display method for displaying a plurality of windows on a screen of a predetermined display device, wherein the window selected as a main window preset to a predetermined display size among the plurality of windows is used. While displaying at a predetermined position on the screen, in the area excluding the area where the main window is displayed on the screen, the remaining windows are displayed as auxiliary windows that are preset to a display size smaller than the main window, The display size is switched between the window displayed as the main window and the window selected as the next main window among the windows displayed as the auxiliary windows, and is linked to the change in the display position of the main window. Multi-window table to rearrange auxiliary windows Method. 2. Along with the display of the window selected as the main window, a window linked in advance with the selected window is newly displayed as an auxiliary window outside the display area of the main window. The multi-window display method according to claim 1, characterized in that 3. One of the plurality of windows is
For one of the interlocutors who interact indirectly through the transmission means,
The multi-window display method according to claim 1 or 2, wherein the window is a window for displaying an image in a state where the line-of-sight of the other party's interlocutor is matched. 4. A window for displaying the image of the other party's interlocutor is switched between a main window and an auxiliary window by focusing on a predetermined portion in the window displaying the image of the other party's interlocutor. 4. The display size switching display is performed.
Multi-window display method described. 5. A multi-window display system for displaying a plurality of windows on a screen of a predetermined display device, wherein the window selected as a main window preset to a predetermined display size among the plurality of windows is the window. A display that is displayed at a predetermined position on the screen and that displays the remaining windows as auxiliary windows that are preset to a display size smaller than the main window in the area excluding the area where the main window is displayed on the screen Means, the window displayed as the main window, and the window selected as the main window next time among the windows displayed as the auxiliary windows, and the display position of the main window changes. Table for rearranging auxiliary windows in conjunction with Multi-window display system and a control unit. 6. The display control means, in association with the display of the window selected as the main window, a window previously linked to the selected window is newly displayed as an auxiliary window in the display area of the main window. The multi-window display system according to claim 5, wherein the multi-window display system is arranged in a place other than the above. 7. The display means displays, in one of the plurality of windows, an image in a state where the other party's interlocutor's line of sight matches one of the interlocutors who indirectly interact with each other via the transmission means. The multi-window display system according to claim 5 or 6, characterized in that: 8. The display control means switches a window for displaying the image of the other party's interlocutor between a main window and an auxiliary window so that a predetermined portion in the window for displaying the image of the other party's interlocutor is displayed. 8. The multi-window display system according to claim 7, wherein the display size of the window is switched starting from. 9. A computer program that causes a computer to execute the method of claim 1. 10. A recording medium on which a computer program for causing a computer to execute the method according to claim 1 is recorded.