JP6670450B2 - Information processing apparatus and method, and program - Google Patents

Description

  The present disclosure relates to an information processing apparatus, method, and program, and more particularly, to an information processing apparatus, method, and program capable of improving the expressiveness of communication.

  2. Description of the Related Art In recent years, various systems and services such as a social networking service (SNS) and a chat system as well as an electronic mail and an electronic bulletin board have become widespread as communication tools between individual users via a network such as the Internet. .

  As a means of communication, a communication tool using various data such as images and sounds as well as text data has been developed (for example, see Patent Document 1). For example, in a network system described in Patent Literature 1, a chat system that exchanges an avatar image reflecting a user's facial expression with a message is provided.

JP 2013-9073 A

  However, with the improvement of information processing technology and communication equipment, the demand for communication tools is increasing, and the method described in Patent Document 1 is not sufficient as a method of communicating, and other methods are also required. .

  The present disclosure has been made in view of such a situation, and is intended to improve the expressiveness of communication.

According to one aspect of the present technology, both a first display indicating a history of communication between users and a second display indicating a history of communication with a virtual character that can respond on behalf of the user and the user are the same. A history display screen displayed in a state of being arranged in time series on a communication screen displaying a face image of the virtual character and one message of the user or the virtual character exchanged by the communication, The information processing apparatus includes a display control unit that causes the display unit to superimpose and display the information.

  The history of communication between the users includes a history of communication between a first user who is a predetermined user and another user other than the first user, and a history of communication between the virtual character and the user includes: The history of communication between the first user and the virtual character corresponding to the other user, or the history of communication between the virtual character corresponding to the first user and the other user may be included. it can.

  The first display includes text information representing a message sent from the first user to the other user, or a message sent from the other user to the first user, Is a message representing a message sent from the first user to a virtual character corresponding to the other user, or a text representing a message sent to the first user from a virtual character corresponding to the other user. Text representing information or a message sent from the virtual character corresponding to the first user to the other user, or a message sent from the other user to the virtual character corresponding to the first user Information can be included.

The communication history further includes a third display that expresses an emotion assigned when each message is exchanged, and the display control unit, on the history display screen, transmits and receives the message together with the message. Can be displayed on the display unit.

The emotion expression includes an expression of a face image of a user or a virtual character serving as a sender of the message, and the display control unit performs an expression indicating the emotion of the message, along with each message , on the history display screen . A face image of a user or a virtual character that is the sender of the message can be displayed on the display unit.

The emotion expression includes an effect image representing the emotion of the message, and the display control unit causes the display unit to display an effect image representing the emotion of the message on the history display screen along with each message. it can.

The emotion expression includes a balloon shape representing the emotion of the message, and the display control unit causes the display unit to display a balloon having a shape representing the emotion of the message on the history display screen along with each message. Can

  The emotional expression may be based on information obtained by sensing the first user or the other user who is a recipient of the message when the message is exchanged.

The display control unit, on the history display screen, from a message sent from the first user to the other user or a virtual character corresponding to the other user, and a virtual character corresponding to the first user. The message sent to the other user is displayed on the display unit in a distinguishable state, and is sent from the other user to the first user or a virtual character corresponding to the first user. A message and a message sent to the first user from the virtual character corresponding to the other user can be displayed on the display unit in a state where they can be identified from each other.

The display control unit further includes an instruction receiving unit that receives an instruction to rearrange the messages, wherein the display control unit receives the instruction to rearrange the messages when the instruction receiving unit receives the instruction to rearrange the messages. Each message can be rearranged according to the message rearrangement condition and displayed on the history display screen .

The display control unit further includes an instruction receiving unit that receives an instruction to select the message, and the display control unit receives the instruction to select the message when the instruction receiving unit receives the instruction to select the message. A message can be selected according to the selection condition, and the selected message can be displayed on the history display screen .

The display control unit may cause the user to slide the finger from the top to the bottom of the display unit so that the history display screen is superimposed on the communication screen and displayed on the display unit .

Communication between the user of the information processing apparatus itself and a user of the other information processing apparatus by communicating with the other information processing apparatus via the communication unit to exchange the message, Communication processing for realizing communication between a user and a virtual character corresponding to a user of the other information processing apparatus, and communication between a virtual character corresponding to a user of the information processing apparatus itself and a user of the other information processing apparatus Further comprising a unit, wherein the display control unit displays the history of communication between the user of the information processing device itself and the user of the another information processing device, performed by the communication processing unit, on the history display screen. A first display and a user of the information processing apparatus itself The history of communication with the virtual character corresponding to the user of the other information processing device, or the history of communication between the virtual character corresponding to the user of the information processing device itself and the user of the other information processing device. Both of the second displays can be displayed in a state of being arranged along the same time series .

  The communication processing unit may include, on the communication screen, a face image of a virtual character corresponding to a user of the other information processing apparatus or a user of the other information processing apparatus, the other information processing apparatus. One message sent from the virtual character corresponding to the user of the other information processing apparatus or the virtual character corresponding to the user of the other information processing apparatus or the user of the other information processing apparatus. One sent message can be displayed.

  The communication processing unit may further display, on the communication screen, an emotional expression assigned to a message displayed on the communication screen.

  The communication processing unit, as the emotion expression assigned to a message sent from a user of the other information processing device or a virtual character corresponding to the user of the other information processing device, a facial expression representing an emotion of the message. Further, the face image of the virtual character corresponding to the user of the other information processing device or the user of the other information processing device can be displayed on the communication screen.

  The communication processing unit is an effect image representing an emotion of the message as the emotion expression assigned to a message sent from a virtual character corresponding to a user of the other information processing apparatus or a user of the other information processing apparatus. Can be displayed at an arbitrary position on the communication screen.

  The communication processing unit, as the emotional expression assigned to a message sent from a virtual character corresponding to the user of the information processing device itself or the user of the information processing device itself, a rendering image representing the emotion of the message, The communication screen can be displayed in a region where a face image of a virtual character corresponding to the user of the other information processing device or the user of the other information processing device is displayed.

One aspect of the present technology also provides for both a first display showing a history of communication between users and a second display showing a history of communication with a virtual character that can respond on behalf of the user and the user. A communication screen that displays a history display screen displaying the virtual characters in a state in which they are arranged in the same chronological order, a face image of the virtual character, and one message of the user or the virtual character transmitted and received by the communication. This is an information processing method of superimposing the information on a display unit.

One aspect of the present technology further provides the computer with a first display showing a history of communication between the users, and a second display showing the history of communication between the virtual character and the user who can respond on behalf of the user. a history display screen displayed in a state arranged in chronological order both display the same display the face image of the virtual character, and a batch of messages of the user or the virtual character which has been exchanged by the communication This is a program for functioning as a display control unit that superimposes on a communication screen to be displayed on a display unit.

In one aspect of the present technique, the first display indicating communication history between users, as well as both the second indications indicating communication history with the virtual character and the user can respond on behalf of the user A history display screen that is displayed in a state of being arranged along the same time series has a face image of the virtual character and a communication screen that displays one message of the user or the virtual character transmitted and received by the communication, The information is superimposed and displayed on the display unit.

  According to the present disclosure, information on communication can be processed. In particular, the expressiveness of communication can be improved.

It is a figure explaining the outline of a communication system. It is a figure showing the main examples of composition of a communication system. FIG. 3 is a block diagram illustrating a main configuration example of a user management server and the like. It is a functional block diagram showing an example of a function realized by each server. It is a block diagram which shows the main structural examples of a terminal device. It is a functional block diagram showing an example of a function which a terminal unit realizes. FIG. 3 is a diagram illustrating an outline of initial settings. 9 is a flowchart illustrating an example of the flow of each process regarding initialization. It is a figure explaining the example of a screen. It is a figure explaining the example of a screen. It is a figure explaining the example of a screen. It is a figure explaining the example of a screen. It is a figure explaining the example of a screen. It is a figure explaining the example of a square screen. It is a figure explaining the outline of processing about a square. It is a flowchart explaining the example of the flow of a process regarding a square screen. It is a flowchart explaining the example of the flow of a process regarding a square screen. It is a flowchart explaining the example of the flow of a process regarding a square screen. It is a figure explaining the outline of communication between users. It is a flowchart explaining the example of the flow of a process regarding communication between users. It is a figure explaining the example of a screen. FIG. 3 is a diagram illustrating a configuration example of a screen. FIG. 6 is a diagram illustrating an example of an input screen. It is a figure explaining the example of an effect image. It is a figure explaining the example of a display of an effect image. FIG. 9 is a diagram for describing an outline of a process regarding communication with a virtual character. It is a flowchart explaining the example of the flow of a process regarding communication with a virtual character. It is a flowchart explaining the example of the flow of a process regarding the communication with the virtual character using the stamp image. It is a figure showing an example of a situation of a stamp image input. It is a figure explaining an example of restriction of an emotion expression according to the number of stamps. FIG. 9 is a diagram illustrating an outline of a process related to communication between another user and a virtual character. It is a flowchart explaining the example of the flow of a process regarding the communication between another user and the virtual character. It is a figure explaining the example of a screen. It is a figure explaining the example of a screen. It is a figure explaining the example of a screen. It is a figure explaining the outline of processing about setting, learning, etc. 9 is a flowchart illustrating an example of a flow of processing related to block mode setting. It is a flowchart explaining the example of the flow of a process regarding block mode release. It is a flowchart explaining the example of the flow of a process regarding friend deletion. 11 is a flowchart illustrating an example of a flow of processing related to log learning. FIG. 7 is a diagram illustrating an example of a history display screen. FIG. 7 is a diagram illustrating an example of a history display screen. 9 is a flowchart illustrating an example of a flow of a process related to history display. It is a figure showing an example of an icon.

Hereinafter, embodiments for implementing the present disclosure (hereinafter, referred to as embodiments) will be described. The description will be made in the following order.
1. First embodiment (communication system)

<1. First Embodiment>
<Expression of communication tools>
2. Description of the Related Art Conventionally, various systems and services have been widely used as communication tools between individual users via a network such as the Internet, for example, a system described in Patent Document 1.

  However, for such communication tools, conventional systems and communication methods are not sufficient, and further other methods are required. For example, it is required to have richer expressive power.

  The expressiveness of communication affects the directionality and entertainment of communication. As richer and more diverse expressions are possible, the user can enjoy communication more. In other words, when the expressiveness of the communication is reduced, the taste and entertainment of the communication are reduced, the satisfaction and interest of the user are reduced, and as a result, the motivation to use the communication tool may be reduced. In addition, the user may become accustomed to the existing expressive power over time, and the satisfaction and interest may be reduced. Therefore, with the improvement of information processing technology and communication environment, further expressiveness is required for communication tools.

<Overview of Communication System>
FIG. 1 is a diagram illustrating an overview of a communication system that is an embodiment of a system that implements communication to which the present technology is applied.

  The communication system 100 illustrated in FIG. 1 includes, for example, communication between users (users 101-1 to 101-3) and virtual characters (virtual characters 103-1 to 103) provided in the virtual space 102 by each user. -3) and a system for realizing communication between the user and the like.

  As will be described later, the virtual space 102 is a virtual space (unreal space) realized by, for example, a server, an information processing device, a network, or the like. The virtual characters 103-1 to 103-3 are virtual characters (unreal characters) corresponding to the users 101-1 to 101 set in the virtual space 102, respectively.

  In the following, the users 101-1 to 101-3 will be referred to as the users 101 when it is not necessary to distinguish them from one another. In addition, when it is not necessary to distinguish the virtual characters 103-1 to 103-3 from each other, the virtual characters 103-1 to 103-3 are referred to as virtual characters 103.

  For example, the user 101-1 can communicate with the user 101-2 (double arrow 104-1) using the communication system 100. In addition, the user 101-1 communicates (double arrow 104-2) with the virtual character 103-2 corresponding to the user 101-2 provided in the virtual space 102 using the communication system 100, for example. You can do it too. Further, the user 101-1 communicates with the virtual character 103-1 corresponding to the user 101-1 provided in the virtual space 102 using the communication system 100 (double arrow 104-3). You can do it too.

  Similarly, the user 101-1 can communicate with the user 101-3 (double arrow 104-4) using the communication system 100. Further, the user 101-1 similarly communicates with the virtual character 103-3 corresponding to the user 101-3 provided in the virtual space 102 by using the communication system 100 (double arrow 104-5). ) Can also be performed.

  That is, by using the communication system 100, the user 101-1 can similarly communicate with each of a plurality of other users (or virtual characters corresponding to the other users).

  Similarly, the user 101-2 uses the communication system 100 to communicate with the virtual character 103-1 corresponding to the user 101-1 provided in the virtual space 102 (double arrow 104-6). be able to. In addition, the user 101-2 uses the communication system 100 to communicate (double arrow 104-7) with the virtual character 103-2 corresponding to the user 101-2 provided in the virtual space 102. You can do it too.

  Similarly, the user 101-3 uses the communication system 100 to communicate with the virtual character 103-1 corresponding to the user 101-1 provided in the virtual space 102 (double arrow 104-8). be able to. Further, the user 101-3 communicates (double arrow 104-9) with the virtual character 103-3 corresponding to the user 101-3 provided in the virtual space 102 by using the communication system 100, for example. You can do it too.

  Although illustration is omitted, communication between the user 101-2 (or the virtual character 103-2) and the user 101-3 (or the virtual character 103-3) is also possible.

  That is, each user 101 uses the communication system 100 to communicate with one or more other users 101, communicate with the virtual character 103 corresponding to one or more other users 101, It can communicate with the virtual character 103 corresponding to itself.

  As described above, by enabling communication not only between the users 101 but also between the user 101 and the virtual character 103, even when each user 101 cannot communicate with the virtual character 103, Instead of the user 101, a virtual character 103 corresponding to the user 101 can communicate. Therefore, the user 101 can communicate in more various situations and time zones. That is, the convenience of the communication system 100 can be improved. Further, the virtual character 103 can respond without an instruction from the user corresponding to the virtual character 103. Therefore, the user 101 and the virtual character 103 corresponding to the user 101 are not completely identical (identical) as communication partners. That is, there is a high possibility that the way of responding, that is, the way of taking communication between the user 101 and the virtual character 103 corresponding to the user 101 changes. Therefore, by providing the virtual character 103 in this way, the communication system 100 can realize more various communication. That is, the expressiveness of communication can be improved.

  Of course, the number of users 101 (that is, the number of virtual characters 103) of the communication system 100 is arbitrary, and may be two or less, or four or more.

  The content of the communication is optional. In the following, a description will be given assuming that messages are mainly exchanged using text data as communication. Further, description will be made assuming that data other than text data such as images and sounds can be appropriately transmitted and received.

<Description of communication system>
FIG. 2 is a diagram illustrating an example of a main physical configuration of the communication system 100 described with reference to FIG.

  As shown in FIG. 2, the communication system 100 includes servers such as a user management server 111, a virtual character management server 112, a communication management server 113, and an analysis server 114, a network 121, and terminal devices 131-1 to 131-1. It has a device 131-N (N is an arbitrary natural number).

  The user management server 111 provides a service for managing information on the user 101 of the communication system 100. The virtual character management server 112 provides a service for managing information on the virtual character 103 corresponding to each user 101 set in the virtual space 102 of the communication system 100. The communication management server 113 provides a service for managing communication performed between the user 101 and the virtual character 103 of the communication system 100. The analysis server 114 provides a service for analyzing the content of communication performed between the user 101 and the virtual character 103 of the communication system 100.

  Note that the above-described service is an example, and any service may be provided as long as the service is related to the realization of the communication system 100. That is, a service other than the service of the above-described example may be provided. For example, a server other than the user management server 111 to the analysis server 114 may be provided.

  The physical configuration of each server is arbitrary, and the user management server 111 through the analysis server 114 may be configured as different devices from each other as in the example of FIG. You may. For example, some or all of the user management server 111 to the analysis server 114 may be configured as a virtual server realized by one device (server 110). Further, the server 110 (the user management server 111 to the analysis server 114) may be realized as so-called cloud computing, and the above-described service provided by each server may be provided as a cloud service.

  The network 121 is a communication network serving as a communication medium. The network 121 may be any communication network, may be a wired communication network, may be a wireless communication network, or may be both of them. For example, a wired LAN (Local Area Network), a wireless LAN, a public telephone line network, a wide area communication network for a wireless mobile such as a so-called 3G line or 4G line, or the Internet may be used, or a combination thereof. You may. Further, the network 121 may be a single communication network or a plurality of communication networks. Further, for example, the network 121 is partially or entirely configured by a communication cable of a predetermined standard such as a USB (Universal Serial Bus) cable or an HDMI (registered trademark) (High-Definition Multimedia Interface) cable. May be performed. Further, for example, the network 121 is partially or wholly connected to an ad hoc mode of an IEEE (Institute of Electrical and Electronic Engineers) 802.11 wireless LAN, an optical communication such as an infrared ray such as an IrDA (InfraRed Data Association), or a Bluetooth (registered). (Trademark) or a method conforming to a predetermined standard, or may be configured by wireless communication using a unique communication method.

  The user management server 111 to the analysis server 114 and the terminal devices 131-1 to 131-N are connected to the network 121. In FIG. 2, connections between these devices and the network 121 are shown by straight lines, but the user management server 111 to the analysis server 114 and the terminal devices 131-1 to 131-N are wired. It may be connected to the network 121 via communication, or may be connected to the network 121 via wireless communication (via an access point, a base station, or the like, not shown).

  The terminal devices 131-1 to 131-N are information processing devices that operate as terminal devices of the communication system 100, and are operated by the user 101. In the following, the terminal devices 131-1 to 131 -N are referred to as the terminal devices 131 when there is no need to distinguish them from one another.

  The terminal device 131 has a function capable of communicating with a user 101 of another device or a virtual character 103 corresponding to the user 101 as long as the terminal device 131 can communicate with another device via the network 121. (If any), configured by any information processing device. For example, the terminal device 131 may be configured by a portable (small) information processing device (a so-called mobile device) such as a mobile phone, a smartphone, and a tablet device. As described above, by making it possible to apply a highly portable information processing device as the terminal device 131, the user 101 can use the communication system 100 more easily at more various times and more various places. Will be able to That is, the convenience of the communication system 100 can be improved.

  The number of terminal devices 131 included in the communication system 100 is arbitrary. Further, as the terminal device 131, for example, a plurality of types of information processing devices such as a mobile phone and a smartphone may be applied.

<Server configuration>
Next, a configuration example of each server will be described. FIG. 3 is a block diagram illustrating a main configuration example of the user management server 111.

  As shown in FIG. 3, in the user management server 111, a CPU (Central Processing Unit) 151, a ROM (Read Only Memory) 152, and a RAM (Random Access Memory) 153 are interconnected via a bus 154. .

  The bus 154 is also connected to an input / output interface 160. An input unit 161, an output unit 162, a storage unit 163, a communication unit 164, and a drive 165 are connected to the input / output interface 160.

  The input unit 161 includes an input device that receives external information such as a user input. For example, the input unit 161 includes an operation button, a touch panel, a camera, a microphone, an input terminal, and the like. Further, various sensors such as an acceleration sensor, an optical sensor, and a temperature sensor may be included in the input unit 161.

  The output unit 162 includes an output device that outputs information such as images and sounds. For example, the output unit 162 includes a display, a speaker, an output terminal, and the like.

  The storage unit 163 includes, for example, a hard disk, a RAM disk, a nonvolatile memory, and the like. The communication unit 164 includes, for example, a network interface. For example, the communication unit 164 is connected to the network 121 and communicates with another device (for example, the terminal device 131) and another server connected via the network 121. The drive 165 drives a removable medium 171 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  The CPU 151 performs various processes by, for example, loading a program stored in the storage unit 163 into the RAM 153 via the input / output interface 160 and the bus 154 and executing the program. The RAM 153 also appropriately stores data necessary for the CPU 151 to execute various processes.

  The program executed by the CPU 151 can be provided to the user management server 111 by, for example, recording it on a removable medium 171 as a package medium or the like. In that case, the program can be installed in the storage unit 163 via the input / output interface 160 by attaching the removable medium 171 to the drive 165.

  In addition, this program can be provided to the user management server 111 via a wired or wireless transmission medium such as a LAN, the Internet, or digital satellite broadcasting. In that case, the program can be received by the communication unit 164 via a wired or wireless transmission medium and installed in the storage unit 163.

  In addition, this program can be installed in the ROM 152 or the storage unit 163 in advance.

  Each configuration of the virtual character management server 112 to the analysis server 114 is the same as that of the user management server 111. That is, the description of FIG. 3 can be applied to the description of each server from the virtual character management server 112 to the analysis server 114. When the server 110 is used, the configuration of the server 110 is the same as that of the user management server 111. That is, the description of FIG. 3 can be applied to the description of the server 110.

<Functional block configuration of server>
Next, functions realized by the user management server 111 to the analysis server 114 will be described. The CPU 151 of the user management server 111 to the analysis server 114 realizes various functions by executing the programs. FIG. 4 is a functional block diagram illustrating an example of main functions implemented in each server.

  As shown in FIG. 4, the user management server 111 implements functional blocks such as a user information management unit 181 and a friend management unit 182. The user information management unit 181 performs processing related to management of user information that is information on the user 101. The content of the user information is arbitrary. For example, the user information may include personal information of the user 101, an ID and password for logging in to the communication system 100, and other various setting information.

  The friend management unit 182 performs a process related to the management of a friend who is a communication partner of each user, set by each user 101. In the communication system 100, each user 101 can communicate with another user 101 set as a friend (or a virtual character 103 corresponding to the other user 101). In other words, in order to perform communication, the user 101 sets the other user 101 as the communication partner (or the other user 101 corresponding to the virtual character 103 as the communication partner) as a friend in advance. It is necessary to register in the management server 111 (friend management unit 182).

  The virtual character management server 112 implements functional blocks such as a setting management unit 183, a learning unit 184, a mode management unit 185, and a message setting unit 186.

  The setting management unit 183 performs processing related to management of the setting of the virtual character 103. The learning unit 184 performs a process related to learning about how the virtual character 103 responds. The mode management unit 185 performs a process for managing the operation mode of the virtual character 103. The message setting unit 186 performs a process related to setting of a message sent by the virtual character 103.

  The communication management server 113 implements functional blocks such as a square management unit 187, a state management unit 188, a communication management unit 189, and a history management unit 190.

  The square management unit 187 performs processing related to management of the square. The state management unit 188 performs processing related to the management of the state of the communication partner in the square. The communication management unit 189 performs processing related to communication management. The history management unit 190 performs processing related to communication history management.

  The analysis server 114 implements functional blocks such as a message analysis unit 191, an emotion setting unit 192, and an effect setting unit 193.

  The message analyzer 191 performs a process related to message analysis. The emotion setting unit 192 performs processing related to setting of the emotion corresponding to the message. The effect setting unit 193 performs a process related to setting of the effect image (effect).

<Configuration of terminal device>
Next, a configuration example of the terminal device 131 will be described. FIG. 5 is a block diagram illustrating a main configuration example of the terminal device 131.

  As illustrated in FIG. 5, the terminal device 131 includes a system-on-a-chip (SOC) 201, a random access memory (RAM) 202, an input unit 211, an output unit 212, a storage unit 213, a communication unit 214, It has a drive 215.

  The SOC 201 is a chip on which various functions realized in the terminal device 131 are mounted. The SOC 201 loads and executes programs and data into the RAM 202, for example.

  The input unit 211 includes various input devices. The input unit 211 includes, for example, an operation unit 221, a touch panel 222, a photographing unit 223, a voice input unit 224, a sensor unit 225, and an input terminal 226.

  The operation unit 221 includes, for example, any input device operated by the user such as a key, a button, a switch, and a lever, receives a user input by a user operation on the input device, and transmits the received user input to, for example, the SOC 201 or the like. Supply. The touch panel 222 is formed, for example, by superimposing it on a display unit 231 described later, and performs a user operation (for example, movement of a user's finger or a stylus pen or the like) performed based on a GUI (Graphical User Interface) displayed on the display unit 231. The information (ie, user input) indicated by the position is received, and the received user input is supplied to, for example, the SOC 201 or the like.

  The photographing unit 223 includes, for example, a lens, an aperture, and an image sensor, obtains a photographed image by photographing a subject, and supplies data of the obtained photographed image to, for example, the SOC 201 or the like. The voice input unit 224 has a voice input device such as a microphone, receives a voice input, and supplies the received voice data to, for example, the SOC 201 or the like. The sensor unit 225 includes, for example, various sensors such as an acceleration sensor, an optical sensor, and a temperature sensor, obtains information corresponding to the sensor by sensing, and supplies the obtained various sensor information to, for example, the SOC 201 or the like. The input terminal 226 has an arbitrary signal input terminal such as an analog signal input terminal, and supplies input data to, for example, the SOC 201 or the like.

  The output unit 212 has a display unit 231, an audio output unit 232, and an output terminal 233. The display unit 231 has an arbitrary display such as an LCD (Liquid Crystal Display) or an OELD (Organic ElectroLuminescence Display), and displays, for example, image information supplied from the SOC 201 or the like. The audio output unit 232 includes, for example, a speaker, and outputs audio information supplied from, for example, the SOC 201 or the like. The output terminal 233 has an arbitrary signal output terminal such as an analog signal output terminal, and outputs information supplied from the SOC 201 or the like to the outside of the terminal device 131.

  The storage unit 213 includes, for example, a hard disk, a RAM disk, a nonvolatile memory, and the like. The communication unit 214 includes, for example, a network interface. The communication unit 214 includes, for example, a wireless communication unit 241, a wired communication unit 242, and an NFC (Near Field Communication) unit 243. The wireless communication unit 241 has a wireless communication interface for communicating with another device by wireless communication of a predetermined communication standard. The wired communication unit 242 has a wired communication interface for communicating with another device by wired communication of a predetermined communication standard. The NFC unit 243 is a close proximity wireless transfer interface that performs close proximity wireless transfer with another communication device that has been brought into close proximity. For example, the NFC unit 243 exchanges information and the like of the terminal device 131 necessary for participation in the communication system 100 with another terminal device 131 by the close proximity wireless communication.

  The drive 215 drives, for example, a removable medium 251 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and reads information stored in the removable medium 251 or writes information to the removable medium 251.

  These processing units operate under the control of the SOC 201.

  The SOC 201 performs various processes by, for example, loading a program stored in the storage unit 213 into the RAM 202 and executing the program. The RAM 202 also stores data necessary for the SOC 201 to execute various types of processing, as appropriate.

  The program executed by the SOC 201 can be provided to the terminal device 131 by being recorded on a removable medium 251 as a package medium or the like, for example. In that case, the program can be installed in the storage unit 213 via the SOC 201 by attaching the removable medium 251 to the drive 215.

  The program can also be provided to the terminal device 131 via a wired or wireless transmission medium such as a LAN, the Internet, or digital satellite broadcasting. In that case, the program can be received by the communication unit 214 (the wireless communication unit 241 or the wired communication unit 242) via a wired or wireless transmission medium and installed in the storage unit 213.

  In addition, this program can be installed in the ROM or the storage unit 213 in the SOC 201 in advance.

  The configuration described above is an example, and the terminal device 131 may include a processing unit other than the processing unit described above. Further, the terminal device 131 may omit some of the above-described processing units.

<Configuration of functional block of terminal device>
Next, functions realized by the terminal device 131 will be described. The SOC 201 of the terminal device 131 realizes various functions by executing a program. FIG. 6 is a functional block diagram illustrating an example of main functions implemented in the terminal device 131.

  As shown in FIG. 6, the terminal device 131 implements functional blocks such as an initial setting unit 261, a square processing unit 262, a communication processing unit 263, an interrupt processing unit 264, a mode setting unit 265, and a history display control unit 266. I do.

  The initial setting unit 261 performs a process related to the initial setting. The square processing unit 262 performs processing relating to the square. The communication processing unit 263 performs processing related to communication. The interrupt processing unit 264 performs a process related to a communication interrupt. The mode setting unit 265 performs a process related to mode setting. The history display control unit 266 performs a process related to the display of the communication history.

<Overview of initial settings>
Next, various processes performed in the communication system 100 as described above will be described. First, an outline of the initial setting will be described with reference to FIG. In order to use the communication system 100, it is necessary to register information (user information) of the user 101 in the user management server 111. Further, as described above, in the communication system 100, communication using the virtual character 103 is also performed, so that the virtual character needs to be registered in the virtual character management server 112.

  Thus, the user 101 operates the terminal device 131 to communicate with the user management server 111 and the virtual character management server 112, and performs processing such as registration of user information and setting information of the virtual character 103 as initial settings.

<Processing flow for initial settings>
An example of the flow of the initialization process will be described with reference to the flowchart in FIG. Description will be made with reference to FIGS. 9 to 13 as necessary. This process is started, for example, when the user 101 activates an application for performing communication using the communication system 100 on the terminal device 131.

  When the process related to the initial setting is started, the initial setting unit 261 of the terminal device 131 accepts the registration of the nickname in step S101. The initial setting unit 261 causes the display unit 231 to display, for example, a nickname registration screen 301 as shown in FIG. As shown in FIG. 9A, the nickname registration screen 301 is provided with a nickname input field 302 and an enter button 303. When the user 101 inputs a nickname in the nickname input field 302 and operates the enter button 303, the initial setting unit 261 accepts the nickname input in the nickname input field 302 as the nickname of the user 101. This nickname is used, for example, when the virtual character 103 corresponding to the user 101 calls the user 101 in communication with the user 101.

  For example, when the input nickname is determined to be inappropriate or the number of characters exceeds the limit, the initial setting unit 261 causes the display unit 231 to display a registration failure notification screen 304 as shown in FIG. . As shown in FIG. 9B, on the registration failure notification screen 304, a message indicating that the registration of the nickname has failed, such as “Your desired nickname cannot be registered,” and an OK button 305 are displayed. When the user 101 confirms the registration failure and operates the OK button 305, the registration failure notification screen 304 disappears, the display on the display unit 231 returns to the state of A in FIG. 9, and the nickname registration is accepted again. Can be

  When the nickname is accepted, the initial setting unit 261 accepts gender registration in step S102. The initial setting unit 261 causes the display unit 231 to display a gender registration screen 306 as shown in, for example, C of FIG. As shown in FIG. 9C, on the gender registration screen 306, a male button 307 and a female button 308 are provided. The user 101 operates the male button 307 if the user is male, and operates the female button 308 if the user is female. The initial setting unit 261 accepts the gender corresponding to the button operated in this manner as the gender of the user 101.

  When the designation of the gender is accepted, the initial setting unit 261 accepts registration of an image (icon) in step S103. The “image (icon)” is an image that shows the user 101 and the virtual character 103 corresponding to the user 101 and also shows the emotion on the conversation screen displayed on the display unit 231 when communication is performed. Details of the conversation screen will be described later. The initial setting unit 261 causes the display unit 231 to display an icon registration screen 310 as shown in FIG. 10, for example.

  As shown in FIG. 10, the icon registration screen 310 is provided with a photograph button 311, an image selection button 312, a character selection button 313, and a setting reject button 314. The user 101 selects a method for obtaining the image (icon) to be registered by selecting any of these buttons.

  For example, when the user 101 operates the photographing button 311, a photographed image obtained by newly photographing is registered as an image (icon). That is, in this case, shooting is performed, and a shot image obtained by the shooting is registered as an image (icon). For example, when the user 101 operates the image selection button 312, an image registered in the album function (for example, a captured image obtained in the past or an image downloaded from the outside (another device)). Is registered as an image (icon). That is, in this case, the image selected from the image group registered in the album function is registered as an image (icon).

  Further, for example, when the user 101 operates the character selection button 313, an image of a character prepared in advance in the communication system 100 is registered as an image (icon). That is, in this case, a character is selected, and the image of the selected character is registered as an image (icon). For example, when the user 101 operates the setting reject button 314, registration of this image (icon) is omitted. That is, no image (icon) is registered here. The registration of the image (icon) may be performed at an arbitrary timing after the completion of the initial setting.

  For example, when the photograph button 311 is operated, the initial setting unit 261 causes the display unit 231 to display a captured image registration screen 320 as shown in FIG. As shown in FIG. 11A, on the captured image registration screen 320, a display area for displaying the captured image to be registered is provided for each emotion (for example, emotion, emotion, and so on) (display area 321-1 to display area 321-1). Region 321-4). When there is no need to distinguish the display regions 321-1 to 321-4 from each other, they are referred to as a display region 321. Before photographing is performed, the display area 321 may be left blank, or an expression indicating an emotion (for example, characters such as joy, anger, sadness, ease, etc.) may be displayed. .

  Further, on the photographed image registration screen 320, a photograph button is provided for each emotion (for example, emotion, anger, sorrow, etc.) (a photograph button 322-1 to a photograph button 322-4). When it is not necessary to distinguish the shooting buttons 322-1 to 322-4 from each other, the shooting buttons 322-1 to 322-4 are referred to as shooting buttons 322. Note that a message, a pattern, or the like indicating that shooting is performed by operation of the user 101 is displayed on each shooting button 322, such as “shooting” in the example of FIG. 11A. It may be. By doing so, the user 101 can more easily understand the meaning of operating the shooting button 322 (contents of processing performed by the user 101 operating the shooting button 322).

  When the user 101 operates the shooting button 322 of a desired emotion, shooting for obtaining a shot image representing the emotion is performed. For example, a face image of the user 101 having an expression corresponding to the emotion is captured.

  The initial setting unit 261 causes the display unit 231 to display a shooting screen 324 as shown in FIG. As shown in FIG. 11B, on the shooting screen 324, an emotion display area 325 for displaying a message or the like indicating an emotion corresponding to the shooting, an image display area 326 for displaying a captured image or a shot image, In addition, a shooting button 327 for shooting by operation of the user 101 is provided.

  In the emotion display area 325, a message or the like for obtaining an expression corresponding to the emotion corresponding to the shooting button 322 operated by the user 101 on the shooting image registration screen 320 is displayed. In the case of the example of FIG. 11B, since the user 101 operates the photographing button 322-1 on the photographed image registration screen 320 and the emotion corresponding to the photographing button 322-1 is “happy”, the emotion display area At 325, a message or the like requesting a smile such as "laugh" is displayed.

  Before the user 101 operates the shooting button 327, the captured image captured by the imaging unit 223 is displayed in the image display area 326. When the user 101 operates the shooting button 327, the captured image displayed in the image display area 326 is stored as a shot image. Therefore, the user 101 can perform photographing while checking the captured image displayed in the image display area 326, so that a photographed image with a desired expression can be obtained more easily.

  When the captured image is obtained, the captured image is displayed in the image display area 326 as shown in FIG. Then, an OK button 328 and a retake button 329 are displayed on the display unit 231 so as to be superimposed on the shooting screen 324. When the user 101 operates the retake button 329, the display on the display unit 231 returns to the state shown in FIG. 11B, and shooting is performed again. When the user 101 operates the OK button 328, the captured image is stored as an image registered as an image (icon). The initial setting unit 261 displays the captured image registration screen 320 on the display unit 231 as shown in D of FIG. At this time, a captured image stored as an image to be registered as an image (icon) is displayed in a display area 321 of the emotion desired by the user 101 on the captured image registration screen 320. For example, as described above, when an image of the emotion “pleasure” is taken, the taken image is displayed in the display area 321-1.

  When the above-described imaging is performed for each emotion, a captured image of each emotion is displayed in the image display area 321 as shown in FIG. If it is desired to redo shooting at this stage, the user 101 may operate the shooting button 322 for a desired emotion so that the user can redo shooting for that emotion. As shown in FIGS. 11A and 11D, the captured image registration screen 320 is provided with an enter button 323. When the captured images of all emotions are obtained, the user 101 operates the enter button 323. When the enter button 323 is operated, a captured image of each emotion is received as an image (icon).

  For example, when the image selection button 312 is operated, the initial setting unit 261 causes the display unit 231 to display a selected image registration screen 330 as shown in FIG. As shown in FIG. 12A, on the selected image registration screen 330, a display area for displaying the selected image is provided for each emotion (e.g. Region 331-4). When it is not necessary to distinguish the display regions 331-1 to 331-4 from each other, the display regions 331-1 to 331-4 are referred to as a display region 331. Before photographing is performed, the display area 331 may be blank, or an expression indicating an emotion (for example, characters such as joy, anger, sorrow, ease, etc.) may be displayed. .

  In the selected image registration screen 330, a selection button is provided for each emotion (for example, emotion, anger, sorrow, etc.) (a selection button 332-1 to a shooting button 332-4). The selection button 332-1 to the shooting button 332-4 will be referred to as a selection button 332 when it is not necessary to distinguish them from one another. In addition, a message, a pattern, or the like indicating that the user 101 operates to select an image, such as “select” in the example of FIG. You may do so. By doing so, the user 101 can more easily grasp the meaning of operating the selection button 332 (the content of the processing performed by the user 101 operating the selection button 332).

  When the user 101 operates the selection button 332 of a desired emotion, an image selection for obtaining an image (a captured image or the like) representing the emotion is performed.

  The initial setting unit 261 causes the display unit 231 to display an image selection screen 334 as shown in FIG. As shown in FIG. 12B, on the image selection screen 334, a list of images registered in the album (a list of thumbnail images) is displayed. When the user 101 selects a desired image (thumbnail image), the initial setting unit 261 displays a selected image display screen 335 on the display unit 231 as shown in FIG. On the selected image display screen 335, an image display area 336 and an enter button 337 are provided. In the image display area 336, an image selected by the user 101 on the image selection screen 334 is displayed. When the user 101 operates the enter button 337, a portion of the image selected on the image selection screen 334, which is displayed in the image display area 336, is trimmed (cut out) and registered as an image (icon). Will be saved.

  In the image display area 336, the user 101 can enlarge or reduce the image displayed in the image display area 336 and move (shift) the display range. By doing so, the user 101 can register an image according to the user's 101 preference.

  When the image selected as described above is obtained, the initial setting unit 261 displays the selected image registration screen 330 on the display unit 231 as shown in D of FIG. At this time, the selected image is displayed in the display area 331 of the emotion desired by the user 101 on the selected image registration screen 330. For example, as described above, when an image is selected for the emotion “pleasure”, the selected image is displayed in the display area 331-1.

  When the above image selection is performed for each emotion, a captured image of each emotion is displayed in the image display area 331 as shown in FIG. If the user wants to redo the image selection at this stage, the user 101 may operate the selection button 332 for the desired emotion to be able to redo the image selection for that emotion. As shown in FIGS. 12A and 12D, the selected image registration screen 330 is provided with a decision button 333. When all the emotion images are selected, the user 101 operates the enter button 333. When the enter button 333 is operated, the selected image of each emotion is received as an image (icon).

  For example, when the character selection button 313 is operated, the initial setting unit 261 causes the display unit 231 to display a character introduction screen 341 as shown in FIG. For example, as shown in FIG. 13, the character introduction screen 341 may be constituted by introduction screens for introducing each character (character introduction screens 341-1 to 341-4). At that time, the introduction screen for introducing each character may have a size corresponding to the display area of the display unit 231. That is, the introduction screen may be displayed for each character in the display area of the display unit 231 indicated by the dotted frame 342 in FIG. In this case, the user 101 can display an introduction screen of each character in the display area of the display unit 231 by, for example, scrolling left and right. When the user 101 displays an introduction screen of a desired character in the display area and operates the enter button 343, the character is selected, and an image of each emotion of the character is received as an image (icon).

  Referring back to FIG. 8, when the registration of the image is received as described above, the initial setting unit 261 activates the virtual character in step S104. In step S105, the initial setting unit 261 receives the setting of the virtual character. Here, the user 101 sets, for example, the character of the virtual character, a favorite thing or a word (keyword) of the virtual character, and the like.

  The above settings have not been registered in the server yet.

  In step S106, the initial setting unit 261 receives registration of an account such as a user ID and a password. This account may be made to be able to use an existing predetermined service ID or password, for example. In the account registration process, the initial setting unit 261 supplies the user management server 111 with the account and user information including various settings received in the above-described process.

  In step S111, the user information management unit 181 of the user management server 111 attempts to register the supplied user information. When the registration is successful, the user information management unit 181 supplies the virtual character management server 112 with information on the setting of the virtual character included in the user information. In step S121, the setting management unit 183 of the virtual character management server 112 registers the supplied virtual character setting information.

  When the setting information of the virtual character is registered in the virtual character management server 112, the user information management unit 181 of the user management server 111 notifies the terminal device 131 that the registration of the user information has been successful. Upon receiving the notification, the initial setting unit 261 of the terminal device 131 completes the registration of the account.

  Further, in order to communicate in the communication system 100, it is necessary to register another user as a communication partner as a "friend" in advance.

  Therefore, in step S107, the initial setting unit 261 accepts the registration of a friend. When accepting the registration of a friend by the operation of the user 101, the initial setting unit 261 supplies information about the friend to the user management server 111. The friend management unit 182 of the user management server 111 registers and manages information about the friend. That is, the friend management unit 182 registers and manages the user specified in the supplied information on the friend as the friend of the user 101. When the registration is completed, the friend management unit 182 notifies the terminal device 131 to that effect. Upon receiving the notification, the initial setting unit 261 completes the registration of the friend.

  As described above, various processes can be performed as initial settings.

<Square>
Next, the square will be described. In the terminal device 131, a list of images (icons) of friends of the user 101 is displayed on the square screen. The user 101 selects a friend to communicate with on this square screen.

  FIG. 14 shows an example of a square screen. In the example of FIG. 14, the square screen 350 is wider than the display area of the display unit 231, and only a part (the square screen 350-1) is displayed on the display unit 231. That is, the square screen 350-2 and the square screen 350-3 show portions of the square screen 350 that are not displayed on the display unit 231. That is, the square screen 350 includes the square screen 350-1 to the square screen 350-3. The square screen 350-2 and the square screen 350-3 can be displayed by, for example, scrolling left and right on the displayed square screen 350-1.

  On the square screen 350, images (icons) indicating other users registered as friends of the user 101 of the terminal device 131 or virtual characters corresponding to the other users are displayed as images 351-1 to 351-9. ) Is displayed. The user 101 selects one of the images (icons) to designate another user (or a virtual character corresponding to the other user) indicated by the image (icon) as a communication partner. Can be. In this way, by enabling the communication partner to be specified using the list of images (icons), the user 101 can more easily identify each user and designate a desired user as a communication partner. You can choose.

  Note that the square screen 350 may be configured to be able to identify whether the image (icon) indicates the user 101 or the virtual character 103. For example, as shown in FIG. 14, a character "P" may be added to an image (icon) indicating the virtual character 103. By doing so (whether or not the character "P" is added), the user 101 can more easily identify whether the image (icon) indicates the user 101 or the virtual character 103. can do.

  The position of each image (icon) on the square screen 350 is arbitrary. For example, they may be arranged (arranged) based on a predetermined condition. This condition is optional. For example, age, gender, name, new density, emotion, and the like may be used, or other values may be used. Also, the information may be arranged based on a plurality of conditions.

  Further, a character or an image indicating the condition may be included in the background of the square screen 350 or the like. For example, when an image (icon) is arranged according to the emotion of each user (or virtual character), an image indicating the emotion (directly or indirectly) may be included in the background of the square screen 350. For example, when the emotion of the user 101 or the virtual character 103 indicated by the images 351-5 to 351-7 is “joy”, the joy is displayed in the area 352-1 of the square screen 350 where those images are arranged. A background image may be used. Further, for example, when the emotion of the user 101 or the virtual character 103 indicated by the images 351-8 and 351-9 is “sorrow”, sadness is expressed in the area 352-2 of the square screen 350 where those images are arranged. Such a background image may be used. By doing so, the user 101 can more easily grasp the emotion of another user or the virtual character.

  The square screen 350 may be provided with a sort button 353 and a filter button 354. The rearrangement button 353 is operated by the user 101 when changing the arrangement (arrangement) of images (icons). When the user 101 operates the rearrangement button 353, input or selection of conditions becomes possible, and images (icons) are rearranged according to the input or selected conditions. The user 101 operates the filter button 354 to display only a desired image (icon). When the user 101 operates the filter button 354, it becomes possible to input or select a condition, an image (icon) is selected (refined) according to the input or selected condition, and only the selected image (icon) is selected. Is displayed on the square screen 350.

<Overview of processing related to the open space>
Next, with reference to FIG. 15, an outline of the processing related to the open space will be described. As described above, the user 101 selects a communication partner on the square screen 350. Processing related to the square, such as display of the square screen 350, rearrangement and filtering of images (icons) on the square screen 350, and designation of a communication partner on the square screen 350, is performed by the terminal device 131 and the communication device as shown in FIG. This is performed by the management server 113, the user management server 111, and the like.

<Flow of processing related to the open space>
First, an example of the flow of processing related to display of a square screen will be described with reference to the flowchart of FIG.

  In step S141, the square processing unit 262 of the terminal device 131 controls, for example, the operation unit 221 and the like, and receives a display request for the square screen 350. Upon receiving the display request for the square screen 350, the square processing unit 262 controls, for example, the wireless communication unit 241 or the wired communication unit 242 in step S142, and transmits the request to the communication management server 113.

  In step S151, the open space management unit 187 of the communication management server 113 controls, for example, the communication unit 164 and acquires the request. In step S152, the open space management unit 187 accesses the user management server 111 and obtains information (friend information) on the friend of the user of the terminal device 131. In step S161, the friend management unit 182 of the user management server 111 supplies the requested friend information of the user 101 to the communication management server 113. The friend information includes, for example, an image indicating the user 101 as a friend or a virtual character corresponding to the user 101 (hereinafter, also referred to as a friend image). The square management unit 187 extracts a friend image from the supplied friend information, and generates a square screen 350 to which the obtained friend image is added.

  In step S153, the square management unit 187 determines the state of the friend (user 101) indicated by each image (icon) added to the square screen 350 based on the information on the state of each user 101 managed by the state management unit 188. (That is, whether each image (icon) indicates the user 101 or the virtual character 103 corresponding to the user 101). Further, the open space management unit 187 sets a message, an image, and the like to be added to an image (icon) based on friend information, history information managed by the history management unit 190, and the like.

  When the square screen 350 is generated in this way, in step S154, the square management unit 187 controls the communication unit 164, for example, and supplies the data of the generated square screen 350 to the terminal device 131. In step S143, the square processing unit 262 of the terminal device controls, for example, the wireless communication unit 241 or the wired communication unit 242 to acquire data of the square screen 350.

  In step S144, the square processing unit 262 causes the display unit 231 to display the square screen 350 thus supplied.

  As described above, the square processing unit 262 can display the square screen 350 including the image (icon) of each friend on the display unit 231 reflecting the state of the friend and the like. Therefore, the user 101 can more easily select a communication partner.

  Next, an example of a flow of processing relating to sorting and filtering of friend images on the square screen will be described with reference to the flowchart in FIG.

  In step S171, the square processing unit 262 of the terminal device 131 controls, for example, the operation unit 221 or the like, and receives a request to rearrange images (icons) of friends on the square screen 350. For example, when a request for rearranging the images (icons) of friends is received by operating the rearrangement button 353 on the square screen 350 or the like, the square processing unit 262 determines in step S172 that, for example, the wireless communication unit 241 or It controls the wired communication unit 242 to transmit the request to the communication management server 113.

  The open space management unit 187 of the communication management server 113 acquires the request in step S181. In step S182, the square management unit 187 rearranges the friend images on the square screen 350 according to the sort condition and the like included in the request. In step S183, the square management unit 187 supplies the data of the square screen 350 in which the images of the friends are rearranged to the terminal device 131 that is the request source.

  The open space processing unit 262 of the terminal device 131 receives the data in step S173. In step S174, the square processing unit 262 causes the display unit 231 to display the square screen 350 in which the images of the friends are rearranged.

  By performing the rearrangement as described above, the user 101 can more easily rearrange the image of the friend displayed on the square screen 350 in the order desired by the user 101. By performing such rearrangement, it is possible to move a friend's image, which is likely to be a partner with which the user 101 wants to communicate, to a more prominent position. Therefore, the user 101 more easily searches for a desired friend and communicates with the other user 101 who is the friend (or the virtual character 103 corresponding to the friend (the other user 101)). Can be.

  Further, in step S175, the square processing unit 262 of the terminal device 131 controls, for example, the operation unit 221 and the like, and receives a request for filtering (selecting an image to be displayed) of a friend image (icon) on the square screen 350. . For example, when a request for filtering a friend's image (icon) is received by operating the filter button 354 of the square screen 350 or the like, the square processing unit 262 determines in step S176 that the request is for example the wireless communication unit 241 or wired communication. The request is transmitted to the communication management server 113 by controlling the section 242.

  The open space management unit 187 of the communication management server 113 acquires the request in step S184. In step S185, the square management unit 187 selects a friend image to be displayed on the square screen 350 according to the filtering conditions included in the request. In step S186, the open space management unit 187 supplies the data of the open space screen 350 obtained by filtering the image of the friend (not displaying some of the images of the friend) to the terminal device 131 that is the request source.

  The square processing unit 262 of the terminal device 131 receives the data in step S177. In step S178, the square processing unit 262 causes the display unit 231 to display the square screen 350 obtained by filtering the image of the friend.

  By performing the filtering as described above, the user 101 can more easily narrow down the friend images displayed on the square screen 350 under the conditions desired by the user 101. For example, when the user 101 has many friends, the user 101 must select a desired friend's image from the many friends' images on the square screen 350 when selecting a communication partner. And there is a possibility that a complicated operation is required. On the other hand, as described above, by narrowing down according to the conditions desired by the user 101, the number of friend images displayed on the square screen 350 can be reduced in a manner according to the user 101 's desire. Can be. That is, it is possible to narrow down the image of the friend displayed on the square screen 350 so as to leave the image of the friend who is likely to be a partner with which the user 101 wants to communicate. Therefore, the user 101 more easily searches for a desired friend and communicates with the other user 101 who is the friend (or the virtual character 103 corresponding to the friend (the other user 101)). Can be.

  Next, an example of the flow of processing related to selection of a communication partner on the square screen will be described with reference to the flowchart in FIG.

  In step S191, the square processing unit 262 of the terminal device 131 controls, for example, the operation unit 221 or the like, and receives selection of a friend image (icon) on the square screen 350. That is, the square processing unit 262 accepts a user's selection of a communication partner using the image (icon) of the friend on the square screen 350.

  Upon receiving the selection of the communication partner, the open space processing unit 262 controls, for example, the wireless communication unit 241 or the wired communication unit 242 in step S192, and transmits the request to the communication management server 113.

  The open space management unit 187 of the communication management server 113 acquires the request in step S201. In step S202, the communication management unit 189 of the communication management server 113 generates a conversation screen that is a GUI for communicating with the friend selected by the user 101 in accordance with the request. Details of the conversation screen will be described later. When the conversation screen is generated, in step S203, the communication management unit 189 controls, for example, the communication unit 164 and supplies the data of the conversation screen to the terminal device 131 that is the request source.

  In step S193, the communication processing unit 263 of the terminal device 131 controls, for example, the wireless communication unit 241 or the wired communication unit 242 to receive the data of the conversation screen. In step S194, the square processing unit 262 controls the display unit 231 to end the display of the square screen 350. In step S195, the communication processing unit 263 causes the display unit 231 to display the conversation screen acquired in step S193.

  As described above, by performing each process, selection of a communication partner using the square screen 350 can be realized, and the user 101 can more easily select a communication partner simply by selecting an image of a friend. Can be selected.

<Overview of communication between users>
Next, communication between the users 101 will be described. For example, when the user 101-1 communicates with the user 101-2 as shown by a double-headed arrow 104-1 in FIG. 1, the user 101-1 created in the terminal device 131-1 as shown in FIG. The message No. 1 is transmitted to the terminal device 131-2 of the user 101-2 and displayed. Conversely, the message of the user 101-2 created in the terminal device 131-2 is transmitted to the terminal device 131-1 of the user 101-1 and displayed.

  The exchange of such a message is managed by the communication management server 113. The communication management server 113 manages such communication using, for example, information of the user management server 111.

  Further, at that time, the communication management server 113 analyzes the transmitted / received message using the analysis server 114 or the like, and gives an emotional expression corresponding to the message to the message. As a result, the terminal 131 on the message receiving side displays the emotional expression together with the message. Although the details of the emotion expression will be described later, for example, the emotion expression is realized by a facial expression of the user 101 who is the message sender or a predetermined effect image (effect). By doing so, the expressiveness of communication can be improved. The emotional expression may be displayed on the message transmitting side (the details will be described later).

<Processing flow for communication between users>
An example of the flow of processing related to communication between users will be described with reference to the flowchart in FIG. Description will be made with reference to FIGS. 21 to 25 as necessary. Here, a description will be given of a case where the message transmission source is the user 101-1 (terminal device 131-1) and the transmission destination is the user 101-2 (terminal device 131-2). Since the users 101 and the terminal devices 131 are not particularly distinguished from each other, a case where a message is transmitted from any user 101 (terminal device 131) to any user 101 (terminal device 131) is the same as described below. (The following description can be applied).

  In step S211, the communication processing unit 263 of the terminal device 131-1 that is the message transmission source controls, for example, the operation unit 221 and receives an input of a message. The user 101-1 inputs a message on a conversation screen displayed on the display unit 231 of the terminal device 131-1.

  For example, a conversation screen 360 as shown in FIG. 21 is displayed on the display unit 231 of the terminal device 131 that performs communication. This conversation screen 360 is displayed on the entire display area of the display unit 231. As in the example shown in FIG. 21, the conversation screen 360 may be configured by a menu bar 361, an operation bar 362, and a communication display area 363.

  The menu bar 361 displays various menus such as setting and information display. The user 101 can execute various functions by operating the menu bar 361 (selecting a desired menu). The operation bar 362 displays a message input field 371 for inputting a message to be transmitted, a transmission button 372 for instructing transmission of the input message, and the like. The user 101 can communicate (transmit a message) by operating the operation bar 362 (inputting a message or operating the transmission button 372).

  The communication display area 363 displays the state of communication, that is, information such as messages and images to be transmitted and received. By browsing the display of the communication display area 363, the user 101 can grasp the state of the communication being performed by himself / herself (messages and images exchanged and the like).

  Then, as shown in FIG. 21, the communication display area 363 occupies most of the conversation screen 360 (that is, most of the display area of the display unit 231). That is, a wider area is secured as an area indicating the state of communication. Therefore, more various expressions are possible. That is, the expressiveness of communication can be improved.

  In a system that provides communication between individual users, if a user can use a portable information processing device such as a mobile phone, a smartphone, a tablet device, or the like as a terminal device, the user becomes more diverse. The system can be used more easily at an appropriate time and place, and the convenience of the system is improved. That is, if the above-described portable information processing device can be applied as the terminal device 131, the convenience of the communication system 100 can be improved.

  However, the display of such a portable information processing device is generally relatively small. For this reason, for example, a display indicating a state of communication such as a message or an image tends to be small, and the expressiveness of communication may be reduced. When the expressiveness of communication is reduced, the taste and entertainment of the communication are reduced, the satisfaction and interest of the user are reduced, and as a result, the desire to use the communication tool may be reduced. Therefore, as described above, by securing a wider area as the area indicating the state of communication, it is possible to suppress a reduction in the expressiveness of communication.

  Further, as shown in FIG. 21, only the state of the latest communication may be displayed in the communication display area 363. That is, only the most recent communication state may be displayed by using most of the display area of the display unit 231. For example, in a communication display area 363 shown in FIG. 21, an image display area 381 displaying an image (icon) indicating a communication partner (and the emotion of the partner), a transmission message displaying a message immediately before transmission to the partner is displayed. Although a display area 382 and a received message display area 383 for displaying a message immediately before transmission by the other party are formed, a message history or the like which is a list of messages transmitted and received in the past is not displayed.

  Generally, in communication, a message transmitted / received immediately before is more important than a message transmitted / received in the past. However, when the message history and the like are displayed, a large part of the communication display area 363 is occupied for displaying the message history, and the most important state of the latest communication is displayed in the message history. Could be smaller due to In other words, there is a possibility that the expressive power of the most important part of communication is reduced.

  Therefore, the configuration of the communication display area 363 may be simplified as much as possible so that the display of the most important state of the latest communication can be performed in a wider area. By doing so, the expressive power of each of the image display area 381, the transmission message display area 382, and the reception message display area 383 can be increased. As a result, it is possible to suppress a decrease in the satisfaction and interest of the user 101 with respect to the communication system 100.

  Further, in the above-described communication, the emotional expression of the user 101 may be transmitted and received together with the message. Although a specific method of the emotion expression is arbitrary, for example, the emotion expression may be performed by an expression of an image indicating the message sender (the user 101 or the virtual character 103).

  For example, in the image display area 381, an image corresponding to the emotion estimated from the transmitted / received message may be selected and displayed from the images for each emotion registered in the above-described initial setting. . For example, when the emotion “pleasure” is estimated from the transmitted / received message (the message displayed in the received message display area 383), the image display area 381 displays the emotion “ An image (for example, a photographed image of the smiling face of the user 101) corresponding to “joy” may be displayed.

  By doing so, it is possible to increase the expressiveness of the communication as compared with the case where the transmitted and received messages are simply displayed. This image need not be an image that directly shows the expression of the message sender (user 101 or virtual character 103). For example, it may not be the face image of the message sender. However, it is desirable that the image be one in which the emotion can be grasped more easily.

  In addition, for example, emotional expression may be performed using an effect image (effect) other than the image indicating the message sender. This effect image (effect) is optional. For example, it may be a predetermined stamp image representing an emotion, a design of a display area of a message displayed in the communication display area 363 (any visual effect such as size, shape, pattern, color, etc.) or a message. (An arbitrary visual effect such as font type, size, shape, pattern, color, etc.) or an effect that changes the display of part or all of the communication display area 363. Is also good.

  For example, in the case of the example of FIG. 21, a stamp image 384 of a design indicating a cold or rain is displayed in the communication display area 363 in response to the message “Today is cold” displayed in the received message display area 383. Have been. By doing so, it is possible to produce an effect that enhances “coldness” more than simply displaying the message “It's cold today”. That is, the expressiveness of communication can be increased.

  For example, in the case of the example in FIG. 21, the transmission message display area 382 and the reception message display area 383 have a so-called balloon shape. The emotion may be expressed by the design of the balloon, the design of the message displayed in the balloon, and the like. In this case, the design of the transmission message display area 382 and the design of the message displayed in the transmission message display area 382 are based on the emotional expression of the user 101 (or the virtual character 103) of the terminal device 131 (in the transmission message). The design of the received message display area 383 and the design of the message displayed in the received message display area 383 indicate the emotional expression of the user 101 (or the virtual character 103 thereof) as the communication partner. (Emotion expression corresponding to the received message). By doing so, it is possible to increase the expressiveness of communication as compared with the case where a transmitted / received message is simply displayed as text data.

  Note that a plurality of effect images (effects) may be combined. For example, a plurality of types of effect images (effects) may be combined, such as a change in the design of a stamp image and a balloon.

  Details of the effect image (effect) will be described later. In this way, by expressing the emotion using the effect image (effect), the expressiveness of the communication can be increased as compared with the communication simply transmitting and receiving the message.

  In the emotion expression as described above, the emotion may be determined and added based on the analysis result of the message. As will be described in detail later, by doing so, a means for detecting a user's facial expression, such as a camera, becomes unnecessary, and costs can be reduced. In addition, the user does not need to perform complicated operations such as creating an expression according to the content of the message.

  The conversation screen 360 as described above may have, for example, a hierarchical structure as shown in FIG. For example, the first level 391 (A in FIG. 22) including the menu bar 361 and the operation bar 362, the second level 392 (B in FIG. 22) including the stamp image 384, the transmission message display area 382 and the reception message display area 383. There is a third hierarchy 393 (C in FIG. 22) including a fourth hierarchy 394 (D in FIG. 22) including an image display area 381, and a fifth hierarchy 395 (E in FIG. 22) including a background image. The conversation screen 360 is formed by superimposing these hierarchical images.

  Therefore, the image of each layer can be easily laid out regardless of the display area of the image of another layer. For example, it is also possible to easily realize that images of different layers overlap each other. When the conversation screen 360 has one layer, the components in the screen do not overlap with each other, so it is necessary to comprehensively consider the layout of each component, and there are many layout restrictions. Further, if an image is obtained as a result in which the components overlap each other, such an image must be newly generated.

  On the other hand, the conversation screen 360 is hierarchized (multi-layered) as described above, so that the layouts of the images in the respective hierarchies can be controlled independently of each other, so that the degree of freedom in the layout of the images is improved. be able to. That is, by layering the conversation screen 360 in this way, the expressive power in the communication display area 363, that is, the expressive power of communication can be easily increased.

  Of course, the hierarchical structure shown in FIG. 22 is an example, and the conversation screen 360 may have another hierarchical structure. The images formed in each layer are also arbitrary, the order in which the layers overlap is also arbitrary, and the number of layers is also arbitrary.

  In the conversation screen 360 as described above, the user inputs a message (transmitted message). For example, when the user selects the message input field 371 of the operation bar 362, a message input screen 411 as shown in FIG. 23 is displayed so as to rise from the bottom of the conversation screen 360 in the figure. When the user 101 operates the message input screen 411, a message corresponding to the operation is displayed in the message input field 371. When the input of the message is completed and the user 101 operates the transmission button 372, the message input in the message input field 371 is transmitted as a transmission message to the terminal device 131-2 operated by the user 101-2 of the communication partner. Are displayed on the display unit 231.

  That is, returning to FIG. 20, when the input of the message is received in step S211, the communication processing unit 263 of the terminal device 131-1 transmits the received message or the like to the communication management server 113 in step S212. The communication management unit 189 of the communication management server 113 receives the message or the like in step S221, and supplies the received message or the like to the analysis server 114 in step S222.

  The message analysis unit 191 of the analysis server 114 acquires the message or the like in step S231, and analyzes the message in step S232. In step S233, emotion setting section 192 sets an emotion corresponding to the message based on the message analysis result. In step S234, the effect setting unit 193 sets an effect (effect image) corresponding to the set emotion.

  In step S235, the effect setting unit 193 supplies information on the set emotion and effect to the communication management server 113. When the communication management unit 189 of the communication management server 113 acquires the information on the emotion and the effect in step S223, a message corresponding to the emotion is generated based on the user information managed by the user information management unit 181 of the user management server 111. An image indicating the sender is acquired, and the conversation screen 630 is generated using the image, the effect, and the like together with the transmitted message.

  In step S224, the communication management unit 189 supplies the data of the conversation screen 630 on which the emotional expression is given together with the message as described above to the terminal device 131-2 on the message receiving side. The communication processing unit 263 of the terminal device 131-2 acquires the data of the conversation screen 630 in step S241. The communication processing unit 263 of the terminal device 131-2 displays the conversation screen 360 on the display unit 231 in step S242. That is, an image, a message, an effect, and the like indicating the message sender set by the processing of the communication management server 113 and the analysis server 114 are displayed on the conversation screen 360.

  FIG. 24 shows an example of the effect image (effect). In the example of A in FIG. 24, as a staging image (effect), the communication display area 363 is displayed in the communication display area 363 such that a joke “Futokoro ga sai wa wa” in the received message is clicked (to emphasize the joke). The effect is such that all displayed images freeze and break.

  In the example of FIG. 24B, the stamp image of the exclamation mark is displayed in the communication display area 363, the design of the received message display area 383, and the design of the message displayed in the received message display area 383. However, the design has been changed to emphasize surprise.

  Further, in the example of C in FIG. 24, a stamp image such as a cupcake, a fork, a spoon, or the like that makes a sweet image is displayed in the communication display area 363 in response to a topic message about eating.

  Further, in the example of D in FIG. 24, in response to the received message “Donkoidaida !!”, a stamp image that causes an explosion is displayed over the entire communication display area 363 (image display area 381, transmission message display). Area 382, the received message display area 383, etc.).

  As described above, the effect of the effect image (effect) can be further increased by reducing the restriction as much as possible. For example, it is desirable that the effect image (effect) can be displayed at an arbitrary position in the display area. For example, it is desirable that the effect image (effect) can be displayed in the entire display area (wider area). Also, for example, it is desirable that the effect image (effect) can be displayed so as to be superimposed on another message or image at an arbitrary hierarchy. For example, the effect image (effect) may be displayed on the near side of the message or the image representing the user's emotion, or may be displayed on the back side of the message or the image representing the user's emotion. As described above, by improving the degree of freedom in displaying effect images (effects), the expressiveness of communication can be further increased.

  Of course, the effect image (effect) is arbitrary, and may be other than the examples in FIG. For example, sound may be output in accordance with an image.

  The effect image (effect) may be displayed not only on the message receiving side but also on the message transmitting side. In that case, when the effect image (effect) is displayed on the message transmitting side in the same manner as the message receiving side, it is determined whether the effect image (effect) is for the transmitted message or the effect image (effect) for the received message. It may be difficult for the user 101 to understand. Therefore, the way of displaying the effect image (effect) may be changed between the case where the message is displayed on the receiving side of the message and the case where the effect image is displayed on the transmitting side of the message. .

  For example, even if the effect image (effect) is displayed on the entire communication display area 363 as shown in FIG. 25B on the conversation screen 360 on the message receiving side, the conversation screen 360 on the message transmitting side is displayed on the message transmission side. The effect image (effect) may be displayed only in the image display area 381 as shown in FIG.

  By doing so, the user 101-1 on the message transmitting side can easily understand that the effect image (effect) is intended for the communication partner (user 101-2). Further, the user 101-1 on the message transmitting side grasps what effect image (effect) is performed on the message receiving side by the message transmitted by the user by displaying such effect image (effect). can do. That is, by doing so, the expressiveness of communication can be improved.

  Returning to FIG. 20, in step S225, the history management unit 190 of the communication management server 113 records and manages the message transmitted as described above and the emotional expression attached to the message as a log. The recording location of this log may be the storage unit 163 of the communication management server 113 or another data server (not shown).

  As described above, by performing each process, communication between users can be realized, and the expressiveness of the communication can be improved.

<Overview of communication between user and virtual character>
Next, communication between the user 101 and the virtual character 103 will be described. First, communication between the user 101 and the virtual character 103 of another user 101 will be described (for example, double arrow 104-2 in FIG. 1). In this case, the message of the virtual character is generated by the virtual space 102, that is, the server 110 or the like. That is, the message of the user 101 created in the terminal device 131 is supplied to the server 110 as shown in FIG. Each of the servers 110 cooperates to generate a response message of the virtual character 103 to the message from the user 101 and an emotional expression corresponding to the response message. The response message and the emotional expression are transmitted to the terminal device 131 of the message transmission source and displayed on the display unit 231.

  That is, the communication management server 113 manages such communication using, for example, information of the user management server 111. Further, the communication management server 113 creates a message of the virtual character 103 using the virtual character management server 112. Further, the communication management server 113 uses the analysis server 114 to generate and add an emotional expression according to the message of the virtual character 103.

  That is, even when communicating with the virtual character 103, not only the message but also the emotional expression of the virtual character 103 is displayed on the display unit 231 of the terminal device 131. For example, it is difficult to express the emotion of such a virtual character 103 by a method in which a facial image of a user is photographed at the time of transmitting a message and is used as an emotional expression. As described above, the emotional expression is generated based on the transmitted / received message. Therefore, such an emotional expression of the virtual character 103 can be performed in the same manner as in the case of the user 101. That is, the expressiveness of communication can be improved.

<Flow of processing related to communication between user and virtual character>
An example of the flow of processing related to communication between the user and the virtual character will be described with reference to the flowchart in FIG.

  In step S251, the communication processing unit 263 of the terminal device 131 that is the message transmission source controls, for example, the operation unit 221 or the like, and receives a message input. The user 101 inputs a message on a conversation screen 360 displayed on the display unit 231 of the terminal device 131.

  When the input of the message is received in step S251, the communication processing unit 263 of the terminal device 131 transmits the received message or the like to the communication management server 113 in step S252. The communication management unit 189 of the communication management server 113 receives the message or the like in step S261, and supplies the received message or the like to the analysis server 114 in step S262.

  The message analysis unit 191 of the analysis server 114 acquires the message or the like in step S271, and analyzes the message in step S272. In step S273, the message analysis unit 191 of the analysis server 114 supplies the message and the analysis result of the message to the virtual character management server 112.

  In step S281, the message setting unit 186 of the virtual character management server 112 acquires the message and an analysis result of the message. In step S282, the message setting unit 186 sets the message of the virtual character 103 as a response to the message based on the message and the analysis result of the message and the setting information of the virtual character 103 managed by the setting management unit 183. In step S283, the message setting unit 186 supplies the message of the set virtual character 103 to the analysis server 114.

  In step S274, the message analysis unit 191 of the analysis server 114 analyzes the message of the virtual character 103. In step S275, emotion setting section 192 sets the emotion of virtual character 103 based on the message analysis result. In step S277, the effect setting unit 193 sets an effect (effect image) corresponding to the set emotion.

  In step S278, the effect setting unit 193 supplies information on the set emotion and effect to the communication management server 113. When the communication management unit 189 of the communication management server 113 acquires the information about the emotion and the effect in step S263, the communication management unit 189 based on the user information and the like managed by the user information management unit 181 of the user management server 111 performs a virtual operation corresponding to the emotion. An image showing the character 103 is acquired, and a conversation screen 630 is generated using the image, the effect, and the like together with the message of the virtual character 103.

  In step S264, the communication management unit 189 supplies the data of the conversation screen 630 on which the emotional expression is given together with the message as described above to the terminal device 131 that is the source of the message. The communication processing unit 263 of the terminal device 131 acquires the data of the conversation screen 630 in step S253. In step S254, the communication processing unit 263 of the terminal device 131 displays the conversation screen 360 on the display unit 231. That is, an image indicating the (feeling of) the virtual character 103, a message of the virtual character 103, an effect indicating the feeling of the virtual character 103, and the like set by the processing of the communication management server 113 and the analysis server 114 are displayed on the conversation screen 360. You.

  In step S265, the history management unit 190 of the communication management server 113 records and manages the message transmitted as described above and the emotional expression given to the message as a log. The recording location of this log may be the storage unit 163 of the communication management server 113 or another data server (not shown).

  As described above, by performing each process, communication between the user and the virtual character can be realized, and the expressiveness of the communication can be improved.

<Other communication>
In the communication between the user 101 and the virtual character 103 as described above, the data transmitted and received by the user 101 as communication is arbitrary and is not limited to message data. For example, image data of a predetermined pattern such as a stamp image is used. There may be.

  In addition, not only when the virtual character 103 transmits data (such as a message or a stamp image) but also when the virtual character 103 receives data, the emotional expression of the virtual character 103 can be expressed. Good. For example, when receiving a preset specific message or image data of a specific symbol, the terminal device 131 of the data transmission source may express the emotion of the virtual character 103.

  Further, by transmitting predetermined data to the virtual character 103, the emotion expression ability of the virtual character 103 may be increased.

<Processing flow related to other communication with the virtual character>
An example of the flow of processing related to other communication with the virtual character will be described with reference to the flowchart in FIG. Description will be made with reference to FIGS. 29 and 30 as necessary. Here, description will be made assuming that data of a stamp image is transmitted and received as communication instead of a message.

  In step S291, the communication processing unit 263 of the terminal device 131 that is the source of the stamp image controls, for example, the operation unit 221 or the like, and accepts an input of a message. The user 101 inputs a message on a conversation screen 360 displayed on the display unit 231 of the terminal device 131.

  For example, as shown in FIG. 29, the user 101 can perform a predetermined operation for inputting a stamp image and tap the conversation screen 360 to input the stamp image at the tapped position. In the example of FIG. 29, a heart-shaped stamp image 421 input by the user is displayed on the conversation screen 360. The position of the stamp image 421 may be anywhere within the communication display area 363.

  When a stamp image in a predetermined shape (for example, a heart shape) is input in this way, the emotional expression of the virtual character 103 that is the data receiving side of the stamp image is expressed on the conversation screen 360 of the terminal device 131. That is, in this case, a response using data such as a message or a stamp image is not performed, but a response using an emotional expression is performed. For example, as shown in FIG. 29, when a heart-shaped stamp image 421 is input (that is, transmitted to the virtual character 103), the expression of the image showing the virtual character 103 displayed in the image display area 381 becomes “joy”. (For example, a smile).

  Further, the emotional expression ability of the virtual character 103 may be improved according to the number of the input stamp images 421. For example, as shown in FIG. 30, the type of facial expression of the image showing the virtual character 103 representing each emotion may be increased according to the number of stamp images 421 input as described above. In other words, the more the stamp images 421 are obtained, the more the expressiveness of the communication can be improved.

  Returning to FIG. 28, when the input of the stamp image is received in step S291, the communication processing unit 263 of the terminal device 131 transmits the received stamp image or the like to the communication management server 113 in step S292. In step S301, the communication management unit 189 of the communication management server 113 receives the stamp image and the like, and supplies the received stamp image and the like to the analysis server 114 in step S302.

  The message analysis unit 191 of the analysis server 114 acquires the stamp image or the like in step S311, and analyzes the stamp image in step S312. That is, it is determined whether the stamp image is a predetermined stamp image 421 or not. In step S313, the message analysis unit 191 of the analysis server 114 supplies the stamp image and the analysis result of the stamp image to the virtual character management server 112.

  In step S321, the setting management unit 183 of the virtual character management server 112 acquires the supplied stamp image and the analysis result of the stamp image. If the supplied stamp image is a predetermined stamp image 421, the setting management unit 183 totals the number of the acquired stamp images 421 in step S322. In step S323, the setting management unit 183 updates the setting of the virtual character 103 according to the result of the aggregation (that is, the number of the obtained stamp images 421). For example, as shown in FIG. 30, an image (expression) used as an image representing each emotion is updated (the more stamp images are obtained, the more images can be used). .

  In step S324, the setting management unit 183 supplies information such as the totalization result of the stamp images and updated settings to the analysis server 114. In step S314, the emotion setting unit 192 of the analysis server 114 sets the emotion of the virtual character 103 based on information such as the tally result of the stamp image and updated settings. In step S316, the effect setting unit 193 sets an effect (effect image) corresponding to the set emotion.

  In step S317, the effect setting unit 193 supplies information on the set emotion and effect to the communication management server 113. When the communication management unit 189 of the communication management server 113 acquires the information on the emotion and the effect in step S303, based on the user information and the like managed by the user information management unit 181 of the user management server 111, the virtual communication corresponding to the emotion is performed. An image showing the character 103 is acquired, and a conversation screen 630 is generated using the image, effects, and the like.

  In step S304, the communication management unit 189 supplies the data of the conversation screen 630 on which the emotional expression is given together with the message as described above to the terminal device 131 that is the transmission source of the stamp image. In step S293, the communication processing unit 263 of the terminal device 131 acquires the data of the conversation screen 630. The communication processing unit 263 of the terminal device 131 displays the conversation screen 360 on the display unit 231 in step S294. That is, an image indicating (the emotion of) the virtual character 103, an effect indicating the emotion of the virtual character 103, and the like set by the processing of the communication management server 113 and the analysis server 114 are displayed on the conversation screen 360.

  In step S305, the history management unit 190 of the communication management server 113 records and manages the stamp image transmitted as described above and the emotional expression given to the stamp image as a log. The recording location of this log may be the storage unit 163 of the communication management server 113 or another data server (not shown).

  As described above, by performing each process, it is possible to realize more diverse communication between the user and the virtual character, and to improve the expressiveness of the communication.

  Although the communication between the user 101 and the virtual character 103 of another user has been described above, the communication between the user 101 and the virtual character 103 of the user 101 is not limited to the communication between the user 101 and the virtual character 103 of the other user. Processing can be performed in the same manner as in the case of communication with the communication 103. That is, the above description can be applied to the communication between the user 101 and the virtual character 103 of the user 101.

<Overview of processing related to communication between another user and virtual character>
Note that while another user is communicating with the virtual character 103 of the user 101, information indicating the content of the communication may be displayed on the display unit 231 of the terminal device 131 of the user 101. Good.

  For example, as shown in FIG. 31, a user 101-2 as another user operates the terminal device 131-2 to communicate with the server 110, and communicates with the virtual character 103-1 of the user 101-1. During the operation, information indicating the content of the communication may be supplied from the server 110 to the terminal device 131-1 and displayed.

  This “image indicating the content of communication (between another user and the virtual character 103 of the user 101)” may be any image. For example, all communications (e.g., transmitted and received messages and emotional expressions) performed between another user and the virtual character 103 of the user 101 may be performed, or a part of the transmitted and received messages and emotional expressions. (Summary), or an image or sound other than the transmitted / received message, emotional expression, or the like, which expresses the state of communication concretely or abstractly.

  Further, the user 101 may be configured so that another user can participate in communication with the virtual character 103 of the user 101 from the middle (that is, replace the virtual character 103). In this case, while another user is communicating with the virtual character 103 of the user 101, an image for accepting mid-game participation may be displayed on the display unit 231 of the terminal device 131 of the user 101.

  For example, as shown in FIG. 31, a user 101-2 as another user operates the terminal device 131-2 to communicate with the server 110, and communicates with the virtual character 103-1 of the user 101-1. The user 101 may operate the terminal device 131-1 to access the server 110, and may participate in the communication halfway. When participation in the middle is performed, the state becomes as shown in the example of FIG. 19, and communication between users is performed.

<Processing flow for communication between another user and virtual character>
An example of the flow of processing related to communication between another user and the virtual character will be described with reference to the flowchart in FIG. Description will be made with reference to FIGS. 33 to 35 as necessary.

  In step S361, the communication processing unit 263 of the terminal device 131-2, which is the message transmission source, controls, for example, the operation unit 221 and receives an input of a message. The user 101-2 inputs a message on the conversation screen 360 displayed on the display unit 231 of the terminal device 131-2.

  When the input of the message is received in step S251, the communication processing unit 263 of the terminal device 131-2 transmits the received message or the like to the communication management server 113 in step S362. The communication management unit 189 of the communication management server 113 receives the message or the like in step S341, and supplies the received message or the like to the analysis server 114 in step S342.

  The message analysis unit 191 of the analysis server 114 acquires the message or the like in step S351, and analyzes the message in step S352. In step S353, the emotion setting unit 192 of the analysis server 114 sets the emotion of the virtual character 103. In step S354, the effect setting unit 193 sets an effect (effect image) corresponding to the set emotion.

  When generating a response message of the virtual character 103 as in the example of FIG. 27, for example, the virtual character management server 112 generates a response message as in the case of FIG. 27, and the analysis server 114 sets the emotion. Or set effects (effect images).

  In step S354, the effect setting unit 193 supplies information on the set emotion and effect to the communication management server 113. When the communication management unit 189 of the communication management server 113 acquires the information on the emotion and the effect in step S343, based on the user information and the like managed by the user information management unit 181 of the user management server 111, the virtual communication corresponding to the emotion is performed. An image showing the character 103 is acquired, and a conversation confirmation screen is generated using the image, the effect, and the like together with the message of the virtual character 103.

  In this case, as described with reference to FIG. 27 and the like, the conversation screen is supplied to the terminal device 131-2, but the description is omitted.

  In step S344, the communication management unit 189 supplies the data of the conversation confirmation screen to the terminal device 131-1 on the virtual character side. In step S331, the interrupt processing unit 264 of the terminal device 131-1 acquires the data of the conversation confirmation screen. The communication processing unit 263 of the terminal device 131-1 displays the conversation confirmation screen on the display unit 231 in step S332.

  FIG. 33 is a diagram illustrating an example of the conversation confirmation screen. As shown in FIG. 33, on the conversation confirmation screen 430, an image indicating that another user and the virtual character 103 are communicating with each other is displayed, and the user 101 operates when participating in the middle. The halfway participation screen 431 is displayed. On the halfway participation screen 431, a display such as "OO is talking with XXXP" indicating that another user and the own virtual character 103 are communicating is displayed. When the user 101 taps the “intrude” link on the halfway participation screen 431, the user 101-1 becomes a communication partner of the user 101-2 instead of the virtual character 103-1 and communication between the users starts. .

  Further, the exchanged message may be displayed as in a conversation confirmation screen 432 shown in FIG. 34A. The conversation confirmation screen 432 is provided with a GO button 432A. When the user 101 operates the GO button 432A, the user 101-1 becomes a communication partner of the user 101-2 instead of the virtual character 103-1. Communication between each other starts.

  Further, as in a conversation confirmation screen 433 shown in FIG. 34B, the display may be changed according to the analysis result of the transmitted / received message. For example, as shown in FIG. 34, when a message of a predetermined emotion such as “I, no more .. I do not have myself.” Is displayed, the color and pattern of the conversation confirmation screen 433 change (the example in FIG. In this case, a diagonal pattern is formed). The conversation confirmation screen 433 is provided with a GO button 433A. When the user 101 operates the GO button 433A, the user 101-1 changes to the virtual character 103-1 and becomes a communication partner of the user 101-2. Communication between each other starts. In this way, the display may be such as to prompt the user 101 to join in the middle according to the situation.

  Further, a conversation confirmation screen 434 as shown in FIG. On the conversation confirmation screen 434, a picture of a shoji as shown in FIG. 35A is normally displayed. This state indicates that nothing special has occurred in the communication between the other user and the own virtual character. In this state, if the user 101 performs an operation to open a shoji, the user 101-1 changes to the virtual character 103-1 and becomes a communication partner of the user 101-2, and communication between users is started (that is, the user 101-2 starts communication). 101 participates on the way).

  Further, for example, in a communication between another user and his or her own virtual character, when a good thing that the user 101 wants to participate in the middle occurs, the conversation confirmation screen 434 is shown in FIG. 35B. Thus, the shoji is opened halfway, and a fun pattern such as a musical note mark behind the shoji becomes visible. In this state, if the user 101 performs an operation to open a shoji, the user 101-1 changes to the virtual character 103-1 and becomes a communication partner of the user 101-2, and communication between users is started (that is, the user 101-2 starts communication). 101 participates on the way).

  Further, for example, in a communication between another user and his or her own virtual character, if a situation occurs in which the user 101 cannot participate in the middle, the conversation confirmation screen 434 is shown in FIG. 35C. Thus, the image is as if the shoji was burning. In this state, if the user 101 performs an operation to open a shoji, the user 101-1 changes to the virtual character 103-1 and becomes a communication partner of the user 101-2, and communication between users is started (that is, the user 101-2 starts communication). 101 participates on the way). In this way, the display may be such as to prompt the user 101 to join in the middle according to the situation.

  Of course, examples other than these may be used.

  Referring back to FIG. 32, in step S333, the interrupt processing unit 264 accepts an input of a midway participation instruction by the user 101 or the like performed on the above-described conversation confirmation screen. When the midway participation instruction is received, the interrupt processing unit 264 supplies the received midway participation instruction to the communication management server 113 in step S334.

  In step S346, the status management unit 188 of the communication management server 113 updates the status of the user 101-1 and the virtual character 103-1. That is, the user 101-1 is a communication partner.

  In step S347, the communication management unit 189 of the communication management server 113 generates a conversation screen with which the user 101-1 communicates, and supplies the conversation screen to the terminal device 131-2.

  Upon acquiring the conversation screen in step S363, the communication processing unit 263 of the terminal device 131-2 displays the conversation screen on the display unit 231 in step S364 (updates the conversation screen).

  Further, in step S348, the history management unit 190 of the communication management server 113 records and manages the message transmitted as described above and the emotional expression given to the message as a log. The recording location of this log may be the storage unit 163 of the communication management server 113 or another data server (not shown).

  As described above, by performing each process, communication between the user and the virtual character can be realized, and the expressiveness of the communication can be improved.

<Overview of block mode>
As described above, in communication using the present communication system 100, a mode for changing the way of response may be provided. For example, as described above, in order to communicate with a certain user, it is necessary to register that user as a friend in advance. In other words, if it is desired to refuse communication with a certain user, the user may be removed from friends. However, when such a measure is taken, the relationship up to that point is reset, and the message history and the like are also discarded from the system. To communicate with the user again, it is necessary to register as a friend again.

  If communication can be rejected only by such a method, inconvenience may occur when, for example, it is desired to temporarily reject communication.

  Therefore, a block mode having a different response method from the normal state may be provided. If the state in which communication is possible between the user 101 and the virtual character 103 described above is the normal mode, only communication using the virtual character 103 can be performed in the block mode. In the block mode, the virtual character 103 returns a response message different from the response message in the normal mode.

  By providing such a block mode, the user 101 can perform communication according to more various situations. That is, the expressiveness of communication can be increased.

  36, the user 101 operates the terminal device 131 to exchange information with the user management server 111, the virtual character management server 112, the communication management server 113, and the like, as shown in FIG. Can be performed.

<Processing flow for setting block mode>
An example of the flow of processing related to the setting of the block mode will be described with reference to the flowchart in FIG.

  In step S371, the mode setting unit 265 of the terminal device 131-1 receives a block mode setting instruction. For example, when a block mode setting instruction to the user 101-2 is received, such as when the user 101-1 inputs an instruction, in step S372, the mode setting unit 265 converts the received block mode setting instruction to a virtual It is transmitted to the character management server 112.

  In step S391, the mode management unit 185 of the virtual character management server 112 receives the block mode setting instruction, and in step S392, according to the instruction, the user 101 of the virtual character 103-1 corresponding to the user 101-1. The mode for −2 is set to the block mode.

  The mode management unit 185 supplies a block mode setting instruction to the communication management server 113 in step S393.

  The square management unit 187 of the communication management server 113 acquires the block mode setting instruction in step S401, and in step S402, the friend (user) who is the target of the block mode on the square screen of the requesting user 101-1 The image of 101-2) is set to the block mode. In step S403, the open space management unit 187 transmits the updated open space screen to the terminal device 131-1.

  The square processing unit 262 of the terminal device 131-1 receives the updated square screen in step S373, and causes the display unit 231 to display the updated square screen in step S374. That is, a square screen in which the image of the friend who is the target of the block mode is changed to the image of the block mode is displayed on the display unit 231 of the terminal device 131-1. Thereby, the user 101-1 who has set the block mode can more easily know which friend has set the block mode on the square screen.

  In step S404, the open space management unit 187 of the communication management server 113 converts the image of the request source friend (user 101-1) into the virtual character 103 on the open space screen of the request destination user 101-2. In step S405, the open space management unit 187 transmits the updated open space screen to the terminal device 131-2.

  The square processing unit 262 of the terminal device 131-2 receives the updated square screen in step S381, and causes the display unit 231 to display the updated square screen in step S382. That is, a square screen in which the image of the friend who is the request source of the block mode is a virtual character is displayed on the display unit 231 of the terminal device 131-2.

  When the block mode is set in this way, the user 101-2 (terminal device 131-2) cannot communicate with the user 101-2 that has set the block mode for the user 101-2 (virtual character). 103-1). In the case of the block mode, the virtual character 103-1 returns a response different from that in the case of the normal mode. For example, the virtual character 103-1 may return a predetermined message irrespective of the message from the user 101-2.

  Also in this case, the user 101-2 can communicate with the user and the virtual character with respect to other users who have not set the block mode for the user 101-2 (the normal mode is maintained). Is). Similarly, the user 101-1 can communicate with both the user and the virtual character with respect to users other than the user 101-2 who has set the block mode (the normal mode is maintained).

  In this state, the user 101-2 may be able to determine whether or not the user 101-1 (virtual character 103-1) is in the block mode with respect to the user 101-2. May not be possible.

<Process flow related to release of block mode>
An example of the flow of processing related to the release of the block mode will be described with reference to the flowchart in FIG. For example, assume that the block mode for the user 101-2 is set for the virtual character 103-1 corresponding to the user 101-1.

  In step S411, the mode setting unit 265 of the terminal device 131-1 receives a block mode release instruction. For example, when an instruction to release the block mode to the user 101-2 is received, such as when the user 101-1 inputs an instruction, the mode setting unit 265 transmits the received block mode release instruction to the virtual device in step S372. It is transmitted to the character management server 112.

  In step S431, the mode management unit 185 of the virtual character management server 112 receives the block mode release instruction, and in step S432, sets the virtual character 103-1 corresponding to the user 101-1 according to the instruction. , Release the block mode for the user 101-2.

  The mode management unit 185 supplies a block mode release instruction to the communication management server 113 in step S433.

  In step S441, the open space management unit 187 of the communication management server 113 acquires the block mode release instruction, and in step S442, a friend (user) who is a target of the block mode on the open space screen of the requesting user 101-1. The image of 101-2) is set to the normal mode. In step S443, the square management unit 187 transmits the updated square screen to the terminal device 131-1.

  The square processing unit 262 of the terminal device 131-1 receives the updated square screen in step S413, and causes the display unit 231 to display the updated square screen in step S414. That is, a square screen in which the image of the friend set as the target of the block mode is changed to the image of the normal mode is displayed on the display unit 231 of the terminal device 131-1. Thus, the user 101-1 who has released the block mode can more easily grasp that the block mode has been released on the square screen.

  In step S444, the open space management unit 187 of the communication management server 113 changes the image of the request source friend (user 101-1) to the user 101 on the open space screen of the request destination user 101-2. In step S445, the square management unit 187 transmits the updated square screen to the terminal device 131-2.

  The square processing unit 262 of the terminal device 131-2 receives the updated square screen in step S421, and causes the display unit 231 to display the updated square screen in step S382. That is, the square screen in which the image of the friend who has requested the block mode is returned to the user 101 is displayed on the display unit 231 of the terminal device 131-2.

  When the block mode is released in this way, the user 101-2 (terminal device 131-2) can communicate with both the user 101-1 and the virtual character 103-1 (enter the normal mode). The response of the virtual character 103-1 is also a response in the normal mode.

  In the above description, the image of the friend on the square screen from which the block mode has been released has been described as the user 101, but the virtual character 103 may be used as the virtual character 103 (in this case, the normal mode is also used).

<Processing flow for friend deletion>
With reference to the flowchart of FIG. 39, an example of the processing flow in the case of deleting a desired user from a friend instead of the block mode will be described.

  In step S451, the mode setting unit 265 of the terminal device 131 receives a friend deletion instruction. For example, when the user 101-1 inputs an instruction or the like and receives a friend deletion instruction for another predetermined user, the mode setting unit 265 transmits the received friend deletion instruction to the user in step S452. It is transmitted to the management server 111.

  The user information management unit 181 of the user management server 111 receives the friend deletion instruction in step S471, and deletes the specified other user from the friend of the user 101 according to the instruction in step S472. . Then, in step S473, the user information management unit 181 notifies the communication management server 113 that the user has been deleted from the friend.

  In step S461, the open space management unit 187 of the communication management server 113 acquires the notification, and in step S462, deletes the image of the friend to be processed in the open space screen of the requesting user 101. In step S463, the square managing unit 187 transmits the updated square screen to the terminal device 131.

  The square processing unit 262 of the terminal device 131 receives the updated square screen in step S453, and causes the display unit 231 to display the updated square screen in step S454. That is, the square screen from which the image of the friend to be processed is deleted is displayed on the display unit 231 of the terminal device 131.

  By performing the various processes as described above, it is possible to realize the block mode which is an operation mode different from the normal mode and the friend deletion.

<Flow of learning process>
The virtual character 103 may learn how to respond based on the response method of the user 101 corresponding to the virtual character 103. By doing so, the virtual character 103 can approach the way of the response of the user 101 more, and can return a more individualized response. That is, the expressiveness of communication can be improved.

  With reference to the flowchart of FIG. 40, an example of the flow of processing regarding learning of the response of the virtual character 103 will be described.

  For example, it is assumed that the communication processing unit 263 of the terminal device 131 performs a process relating to a conversation (such as transmission and reception of a message) in step S481, and the communication management unit 189 of the communication management server 113 performs processing in step S491. When the message is exchanged in this manner, the history management unit 190 of the communication management server 113 supplies the conversation log (log of the exchanged message) to the virtual character management server 112 in step S492.

  After acquiring the conversation log in step S501, the learning unit 184 of the virtual character management server 112 learns a response method based on the acquired conversation log in step S502.

  For example, the exchange of messages specifically performed by the user 101 may be stored, and if the message corresponds to the pattern, a response may be made using the message. Further, pattern analysis may be performed. Also, an optimal response message may be detected by weighting based on the frequency of use of the response message by the user 101 or the like.

<History display>
As described above, the message history is basically not displayed on the conversation screen 360. However, the message history may be displayed. In that case, the message history may be displayed on a screen different from the conversation screen 360. By doing so, the message history can be displayed while maintaining the above-described features of the conversation screen 360. That is, also in this case, the expressiveness of communication can be increased.

  For example, as shown in FIG. 41, when the user 101 slides the finger 441 from the top to the bottom of the conversation screen 360 on the conversation screen 360, a state where the finger 441 is superimposed on the conversation screen 360 from the upper side. Then, a message history display screen 442 for displaying the message history may be displayed.

  FIG. 42 shows a display example of the message history display screen 442. As shown in FIG. 42, the message history display screen 442 may be a screen wider than the display area of the display unit 231. In the case of the example of FIG. 42, the area surrounded by the dotted frame 443 indicates the display area of the display unit 231. In this case, the user 101 can display the entire message history on the display unit 231 by sliding the message history display screen 442 up and down.

  As described above, in communication using the communication system 100, in addition to communication between the users 101, communication between the user 101 and the virtual character 103 can be performed. The user 101 and the virtual character 103 correspond to each other and are not unrelated. Therefore, if these communications are displayed as different histories, the display may be difficult for the user 101 to understand.

  For example, during the communication, the person who communicates can change from the user 101 to the virtual character 103 or from the virtual character 103 to the user 101, but the communication history between the users 101 and the user 101 If the communication history with the virtual character 103 is displayed separately, the connection of the conversation becomes unclear, and the value as the history may be reduced.

  Therefore, as in the example shown in FIG. 42, the first display indicating the history of communication between the users and the second display indicating the history of communication between the virtual character capable of responding on behalf of the user and the user. You may make it display on the display part 231 in the state where both of the display were arranged along the same time series.

  In the example shown in FIG. 42, both the message of the user 101 and the message of the virtual character 103 are displayed side by side in one time series (for example, from top to bottom).

  By doing so, it is possible to refer to the history as one time series regardless of whether the user 101 or the virtual character 103 communicates, and the message history becomes easy to see. Therefore, communication using the user 101 and the virtual character 103 can be realized without reducing the visibility of the message history, and more various communication can be performed. That is, the expressiveness of communication can be improved. In the above description, the message history is described, but information other than the message may be included. That is, the communication history may be used.

  Note that the history of communication between users described above may include the history of communication between a first user who is a predetermined user and another user other than the first user. Further, the communication history between the virtual character and the user described above may be the communication history between the first user and the virtual character corresponding to another user, or the communication history between the virtual character corresponding to the first user and another user. May be included.

  The above-described first display may include text information indicating a message sent from the first user to another user, or a message sent from another user to the first user. Further, the second display described above displays a message sent from the first user to a virtual character corresponding to another user, or a message sent from the virtual character corresponding to another user to the first user. Text information representing a message transmitted from the virtual character corresponding to the first user to another user, or text information representing a message transmitted from another user to the virtual character corresponding to the first user. It may be included.

  That is, the message histories of any of the communication patterns described above can be similarly displayed.

  The communication history described above may further include a third display expressing the emotion assigned when each message is exchanged. In addition, an emotional expression assigned when the message is exchanged may be displayed on the display unit 231.

  In the example of FIG. 42, the emotional expression assigned to the message is displayed together with each message. By doing so, the contents of the message history can be grasped more intuitively. Therefore, more various communication can be performed without reducing the visibility of the message history.

  Further, the emotional expression may include a facial expression of a face image of a user or a virtual character as a message sender. Along with each message, a face image of a user or a virtual character as a sender of the message, which has a facial expression expressing the emotion of the message, may be displayed on the display unit 231.

  For example, in the case of FIG. 42, a face image of the user 101 (or the virtual character 103) is added to the message 453 and the message 454 as an emotion expression. By doing so, the emotion corresponding to the message can be grasped more intuitively. Therefore, more various communication can be performed without reducing the visibility of the message history.

  Further, the emotional expression may include an effect image representing the emotion of the message. In addition, an effect image representing the emotion of the message may be displayed on the display unit 231 together with each message.

  For example, in the case of FIG. 42, a question mark is added to the message 456, and an exclamation mark is added to the message 457. Thus, the emotional expression is not limited to the face image.

  Further, the emotion expression may include a speech balloon shape indicating the emotion of the message. In addition, a balloon having a shape representing the emotion of the message may be displayed on the display unit 231 together with each message.

  In the example of FIG. 42, each of the messages 451 to 458 is displayed in a balloon of various designs. The emotion expression may be performed using the design of the balloon (arbitrary visual effects such as size, shape, pattern, and color). In addition, the emotional expression may be expressed using not only the balloon but also the design of the message itself (arbitrary visual effects such as font type, size, shape, pattern, and color).

  In addition, the emotional expression may be based on information obtained by sensing a first user who is a recipient of the message or another user when the message is exchanged.

  For example, the user 101 at the time of receiving the message may be photographed by the photographing unit 223 of the terminal device 131, and emotion expression may be performed based on the photographed image. Also, for example, the sensor unit 225 of the terminal device 131 senses the pulse, brain wave, sweating, and the like of the user 101 when the message is received, and expresses emotion based on information (biological sensing information) obtained by the sensing. May be performed. In these cases, the method of expressing emotions is also arbitrary. For example, the photographing unit 223 of the terminal device 131 photographs the face of the user 101 when the message is received, and expresses the emotion using the photographed image (that is, the face image of the user 101 when the message is received). It may be made to be performed.

  A state in which a message sent from the first user to another user or a virtual character corresponding to another user and a message sent from the virtual character corresponding to the first user to another user can be distinguished from each other. May be displayed on the display unit 231. Also, a message sent from another user to the first user or the virtual character corresponding to the first user and a message sent from the virtual character corresponding to the other user to the first user are distinguished from each other. You may make it display on the display part 231 in a possible state.

  In the example of FIG. 42, these messages can be identified by changing the color (pattern) of the balloon. For example, the messages 451 and 457 displayed in a white balloon on the left side of the message history display screen 442 are messages sent from the first user to another user or a virtual character corresponding to another user. The message 453 and the message 455 displayed in the hatched balloon on the left side of the message history display screen 442 are messages sent from the virtual character corresponding to the first user to other users. Also, the messages 452, 454, and 456 displayed on the white balloon on the right side of the message history display screen 442 are messages sent from other users to the first user or the virtual character corresponding to the first user. It is. Further, a message 458 displayed in a hatched balloon on the right side of the message history display screen 442 is a message sent from the virtual character corresponding to another user to the first user.

  In this way, these messages can be more easily identified. Therefore, more various communication can be performed without reducing the visibility of the message history.

  Note that an instruction receiving unit that receives an instruction to rearrange messages may be further provided. When a message rearrangement instruction is received by the instruction reception unit, the messages are rearranged according to the message rearrangement conditions received by the instruction reception unit, and displayed on the display unit 231. Is also good.

  As such an instruction receiving unit, for example, an operation unit 221 or a touch panel 222 may be provided. Also, by enabling the user 101 and the like to sort messages according to desired conditions, it is possible to display the history in a format that is more easily viewable for the user 101. Therefore, more various communication can be performed without reducing the visibility of the message history.

  Further, an instruction receiving unit that receives an instruction to select a message may be further provided. Further, when a message selection instruction is received by the instruction reception unit, a message is selected according to the message selection condition received by the instruction reception unit, and the selected message is displayed on the display unit 231. You may.

  As such an instruction receiving unit, for example, an operation unit 221 or a touch panel 222 may be provided. In addition, by enabling the user 101 or the like to select a message in accordance with a desired condition, it is possible to reduce the display of a history unnecessary for the user 101 and display the history in a more easily viewable format. become. Therefore, more various communication can be performed without reducing the visibility of the message history.

  The specific contents of such conditions for rearrangement and selection are arbitrary. For example, the condition may be a sender or a receiver of the message, such as the user 101 or the virtual character 103. Further, for example, an emotion (emotional expression) associated with the message may be used as a condition.

  Communication between the user of the information processing apparatus itself and the user of the other information processing apparatus, communication between the user of the information processing apparatus itself and other information by communicating with another information processing apparatus via the communication unit and exchanging messages. A communication processing unit that realizes communication with the virtual character corresponding to the user of the processing device and communication between the virtual character corresponding to the user of the information processing device itself and the user of another information processing device may be further provided. Good. In addition, a first display indicating the history of communication between the user of the information processing apparatus itself and the user of another information processing apparatus performed by such a communication processing unit, and the user of the information processing apparatus itself and another display The second display of the history of communication with the virtual character corresponding to the user of the information processing device or the history of communication between the virtual character corresponding to the user of the information processing device itself and the user of another information processing device You may make it display on the display part 231 in the state which both were arranged along the same time series.

  That is, a message history display screen 442 as shown in FIG. 42 may be generated in the terminal device 131, for example.

  Further, communication is realized in a state where the communication screen, which is a communication screen, is displayed on the display unit 231, and a history display screen different from the communication screen is displayed on the display unit 231. You may make it display both the said 1st display and the said 2nd display in the state arranged along the same time series.

  That is, as shown in the example of FIG. 42, the communication history may be displayed on a message history display screen 442 different from the conversation screen 360 which is a communication screen.

  Further, on the communication screen, a face image of a virtual character corresponding to a user of another information processing device or a user of another information processing device as a communication partner, a user of another information processing device or another information processing device. One message sent from a virtual character corresponding to a user and one message sent to a virtual character corresponding to a user of another information processing device or a user of another information processing device are displayed. You may.

  That is, the communication screen may be realized as the conversation screen 360 having the configuration described with reference to FIG. By using such a conversation screen 360, the expressiveness of communication can be increased.

  Further, on the communication screen, an emotional expression assigned to a message displayed on the communication screen may be displayed. As described with reference to FIG. 21, on the conversation screen 360, an emotional expression may be displayed in addition to the message. By using such an emotional expression, the expressiveness of communication can be increased.

  Also, as an emotional expression assigned to a message sent from a user of another information processing device or a virtual character corresponding to a user of another information processing device, an expression representing the emotion of the message is given. A face image of a virtual character corresponding to a user or a user of another information processing apparatus may be displayed on the communication screen.

  As described with reference to FIG. 21, in the image display area 381 of the conversation screen 360, the image corresponding to the emotion estimated from the transmitted / received message from among the images for each emotion registered in the initial setting. You may make it select and display.

  In addition, as an emotional expression assigned to a message sent from a user of another information processing device or a virtual character corresponding to a user of another information processing device, a rendering image representing the emotion of the message is displayed at an arbitrary position on the communication screen. May be displayed.

  This effect image (effect) is optional. For example, it may be a predetermined stamp image representing an emotion, a design of a display area of a message displayed in the communication display area 363 (any visual effect such as size, shape, pattern, color, etc.) or a message. (An arbitrary visual effect such as font type, size, shape, pattern, color, etc.) or an effect that changes the display of part or all of the communication display area 363. Is also good.

  Also, as an emotional expression assigned to a message sent from the user of the information processing device itself or a virtual character corresponding to the user of the information processing device itself, a rendering image representing the emotion of the message is displayed on another information processing screen of the communication screen. You may make it display in the area | region where the face image of the virtual character corresponding to the user of the apparatus or the user of another information processing apparatus is displayed.

  For example, on the conversation screen 360 on the message receiving side, an effect image (effect) displayed on the entire communication display area 363 as shown in FIG. 25B is displayed, and on the conversation screen 360 on the message transmitting side, the effect image shown in FIG. As described above, the image may be displayed only in the image display area 381. By doing so, the expressiveness of communication can be improved.

  The display unit 231 may be further provided.

<Process flow related to history display>
Next, an example of the flow of the processing related to such history display will be described with reference to the flowchart in FIG.

  For example, in step S511, the history display control unit 266 of the terminal device 131 controls the operation unit 221 and the touch panel 222 and receives an input of a history display instruction. For example, when an input of a history display instruction instructing to display a message history performed by the user 101 or the like is received, the history display control unit 266 transmits the received history display instruction to the communication management server 113 in step S512. Send to

  In step S541, the history management unit 190 of the communication management server 113 receives the history display instruction. In step S542, the history management unit 190 generates a history display screen (for example, the message history display screen 442 in FIG. 42) according to the instruction. The history management unit 190 transmits the generated history display screen to the terminal device 131 in step S543.

  In step S513, the history display control unit 266 of the terminal device 131 receives the history display screen. The history display control unit 266 causes the display unit 231 to display the history display screen in step S514.

  The history display screen is displayed as described above.

  When the message histories are rearranged, the history display control unit 266 of the terminal device 131 controls the operation unit 221 and the touch panel 222 to receive a history rearrangement instruction to instruct the rearrangement of the histories in step S515. For example, when the input of the history sorting instruction performed by the user 101 or the like is received, the history display control unit 266 transmits the received history sorting instruction to the communication management server 113 in step S516.

  In step S544, the history management unit 190 of the communication management server 113 receives the history sorting instruction. In step S545, the history management unit 190 sorts the histories on the history display screen under the conditions specified in the history sorting instruction. The history management unit 190 transmits the history display screen in which the histories are rearranged to the terminal device 131 in step S546.

  In step S517, the history display control unit 266 of the terminal device 131 receives the history display screen. In step S518, the history display control unit 266 causes the display unit 231 to display a history display screen on which the histories are arranged.

  As described above, the histories are rearranged on the history display screen.

  When narrowing down the message histories to be displayed, the history display control unit 266 of the terminal device 131 controls the operation unit 221 and the touch panel 222 to receive a history selection instruction instructing selection of a history to be displayed in step S519. . For example, when the input of the history selection instruction performed by the user 101 or the like is received, the history display control unit 266 transmits the received history selection instruction to the communication management server 113 in step S520.

  In step S547, the history management unit 190 of the communication management server 113 receives the history selection instruction. In step S548, the history management unit 190 selects a history to be displayed on the history display screen under the conditions specified in the history selection instruction. The history management unit 190 transmits the history display screen on which the history has been selected to the terminal device 131 in step S549.

  In step S521, the history display control unit 266 of the terminal device 131 receives the history display screen. In step S522, the history display control unit 266 causes the display unit 231 to display a history display screen on which the history has been selected.

  The history is selected on the history display screen as described above.

  By performing each process as described above, not only the history of the user 101 but also the history of the virtual character 103 can be displayed in an easy-to-read manner. Can be realized. That is, the expressiveness of communication can be increased.

<Software>
The series of processes described above can be executed by hardware or can be executed by software. When the series of processes described above are executed by software, a program constituting the software is installed from a network or a recording medium.

  For example, as shown in FIGS. 3 and 5, this recording medium is provided by a removable medium 171 or 251 on which the program is recorded, which is distributed to distribute the program to the user separately from the apparatus main body. Be composed. The removable medium 171 and the removable medium 251 include a magnetic disk (including a flexible disk) and an optical disk (including a CD-ROM and a DVD). Further, a magneto-optical disk (including an MD (Mini Disc)), a semiconductor memory, and the like are also included.

  In this case, for example, in the user management server 111 to the analysis server 114 or the like (or the server 110), the program can be installed in the storage unit 163 by attaching the removable medium 171 to the drive 165. In the terminal device 131, for example, the program can be installed in the storage unit 213 by attaching the removable medium 251 to the drive 215.

  In addition, this program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In this case, for example, in the user management server 111 to the analysis server 114 (or the server 110), the program can be received by the communication unit 164 and installed in the storage unit 163. For example, in the terminal device 131, the program can be received by the communication unit 214 and installed in the storage unit 213.

  In addition, this program can be installed in a storage unit or a ROM in advance. For example, in the case of the user management server 111 to the analysis server 114 (or the server 110), the program may be installed in the storage unit 163, the ROM 152, or the like in advance. In the case of the terminal device 131, for example, the program can be installed in advance in the storage unit 213, the ROM in the SOC 201, or the like.

  When the series of processes described above is executed by software, a program constituting the software may be provided to the user 101, for example, as an application that provides a function related to a user interface or a function related to communication of the communication system 100. Good. In this case, the user 101 executes the program, that is, activates an application, thereby displaying the initial settings described above, displaying various screens such as the square screen 350 and the conversation screen 360, and transmitting and receiving messages and emotional expressions. Is provided to the user 101. That is, the user 101 can communicate with other users using the communication system 100.

  A method of activating such an application is arbitrary, and a predetermined command may be input at a predetermined CLI (Command Line Interface), or a predetermined icon is operated at a predetermined GUI (Graphical User Interface) ( For example, a tap or the like may be used, or another method may be used.

  When the application is activated by operating the icon, for example, an icon of a predetermined design is displayed in advance in the display area of the display unit 231 of the terminal device 131, and when the user 101 uses the communication system 100, An icon may be operated.

  The design of this icon is optional. For example, a design as shown in FIG. In FIG. 44A, the icon is composed of a gray cloud-shaped design portion and a white character “Pecha” on the inside thereof, but is not limited to this example. For example, the gray part may be red. That is, this icon may be configured by a red cloud-shaped design portion and a white character “Pecha” inside the portion. Further, the character may be other than “Peta” or may be a color other than white.

  Further, for example, as shown in FIG. 44B, this icon is constituted by only the outer frame (line representing the outer shape) of the cloud-shaped design portion shown in FIG. 44A. You may. The color of the outer frame (line representing the outer shape) is arbitrary. For example, it may be black or any other color. The color inside the outer frame is also arbitrary. For example, it may be white, another color, or may be transmitted.

  Further, for example, icons having designs such as those shown in FIGS. 44A and 44B may be used in the various screens described above. For example, it may be used as a design of the transmission button 372 (FIG. 21) of the conversation screen 360. Of course, the design of this icon may be used for other parts of the conversation screen 360 and any part of any screen other than the conversation screen 360. As described above, by using the icon design on the screen displayed as the user interface of the activated application, the user 101 can confirm that the activated application corresponds to the icon operated by the user 101. It can be grasped more easily.

  Note that the program executed by the computer may be a program in which processing is performed in chronological order in the order described in this specification, or may be performed in parallel or at a necessary timing such as when a call is made. It may be a program that performs processing.

  In this specification, the step of describing a program to be recorded on a recording medium may be performed in a chronological order in the order described, but is not necessarily performed in a chronological order. This also includes processes executed individually.

  Further, the processing of each step described above can be executed in each device described above or in any device other than each device described above. In that case, the device that executes the processing may have the above-described functions (functional blocks and the like) necessary to execute the processing. In addition, information necessary for processing may be appropriately transmitted to the device.

  In this specification, a system means a set of a plurality of components (devices, modules (parts), and the like), and it does not matter whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and one device housing a plurality of modules in one housing are all systems. .

  Further, in the above, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configuration described above as a plurality of devices (or processing units) may be configured as one device (or processing unit). Also, a configuration other than those described above may be added to the configuration of each device (or each processing unit). Further, if the configuration and operation of the entire system are substantially the same, a part of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or other processing unit). .

  As described above, the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is apparent that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Is naturally understood to belong to the technical scope of the present disclosure.

  For example, the present technology can take a configuration of cloud computing in which one function is shared by a plurality of devices via a network and processed jointly.

  In addition, each step described in the above-described flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

  Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by one device or can be shared and executed by a plurality of devices.

  In addition, the present technology is not limited to this, and any configuration mounted on such a device or a device configuring the system, for example, a processor as a system LSI (Large Scale Integration), a module using a plurality of processors, a plurality of modules, It can also be implemented as a unit using the module or the like, a set in which other functions are added to the unit, or the like (that is, a configuration of a part of the device).

Note that the present technology can also have the following configurations.
(1) Both the first display indicating the history of communication between users and the second display indicating the history of communication between the virtual character that can respond on behalf of the user and the user are the same time series. An information processing apparatus including a display control unit that causes a display unit to display the information in a state in which the display control unit is arranged along a line.
(2) The history of communication between the users includes a history of communication between a first user who is a predetermined user and another user other than the first user,
The communication history between the virtual character and the user may be the communication history between the first user and the virtual character corresponding to the other user, or the virtual character corresponding to the first user and the other user. The information processing apparatus according to (1), including a history of communication with the device.
(3) the first display includes text information representing a message sent from the first user to the other user or a message sent from the other user to the first user;
The second display is a message sent from the first user to a virtual character corresponding to the other user, or a message sent from the virtual character corresponding to the other user to the first user. Or a message sent from the virtual character corresponding to the first user to the other user, or a message sent from the other user to the virtual character corresponding to the first user The information processing apparatus according to (2), including text information representing
(4) The communication history further includes a third display expressing an emotion assigned when each message is exchanged,
The information processing device according to (3), wherein the display control unit causes the display unit to display an emotional expression assigned when the message is exchanged, together with the message.
(5) The emotional expression includes a facial expression of a user or a virtual character as a sender of the message,
The information processing according to (4), wherein the display control unit causes the display unit to display, along with each message, a face image of a user or a virtual character that is a sender of the message and has a facial expression expressing an emotion of the message. apparatus.
(6) The emotional expression includes an effect image representing the emotion of the message,
The information processing device according to (4) or (5), wherein the display control unit causes the display unit to display an effect image representing an emotion of the message together with each message.
(7) The emotion expression includes a speech balloon shape expressing the emotion of the message,
The information processing device according to any one of (4) to (6), wherein the display control unit causes the display unit to display a balloon having a shape representing an emotion of the message together with each message.
(8) The emotional expression is based on information obtained by sensing the first user or the other user as a message recipient when the message is exchanged. (4) to (7) The information processing apparatus according to any one of the above.
(9) The display control unit includes:
A message sent from the first user to the other user or a virtual character corresponding to the other user, and a message sent from the virtual character corresponding to the first user to the other user, Displayed on the display unit in a state where they can be distinguished from each other,
A message sent from the other user to the first user or a virtual character corresponding to the first user, and a message sent to the first user from the virtual character corresponding to the other user. The information processing apparatus according to any one of (3) to (8), wherein the information is displayed on the display unit in a state where the information can be distinguished from each other.
(10) The apparatus further includes an instruction receiving unit that receives an instruction to rearrange the messages.
The display control unit, when an instruction to rearrange the messages is received by the instruction reception unit, rearranges each message according to the message rearrangement condition received by the instruction reception unit, and displays the message. The information processing device according to any one of (3) to (9).
(11) an instruction receiving unit that receives an instruction to select the message;
The display control unit, when an instruction to select the message is received by the instruction receiving unit, selects a message according to the message selection conditions received by the instruction receiving unit, the selected message, The information processing apparatus according to any one of (3) to (10), which displays the information on a display unit.
(12) communication between the user of the information processing apparatus itself and the user of the other information processing apparatus by communicating with the other information processing apparatus via the communication unit to exchange the message; A communication between the user of the apparatus itself and a virtual character corresponding to the user of the other information processing apparatus, and a communication between the virtual character corresponding to the user of the information processing apparatus itself and the user of the other information processing apparatus are realized. And a communication processing unit
The first display indicating a history of communication between a user of the information processing apparatus itself and a user of the another information processing apparatus, the first display being performed by the communication processing unit; History of communication between the user itself and the virtual character corresponding to the user of the other information processing apparatus, or communication history between the virtual character corresponding to the user of the information processing apparatus itself and the user of the other information processing apparatus The information processing apparatus according to any one of (3) to (11), wherein both of the second displays indicating the history are displayed on the display unit in a state of being arranged in the same time series.
(13) The communication processing unit realizes the communication in a state where a communication screen which is the communication screen is displayed on the display unit,
The display control unit displays a history display screen different from the communication screen on the display unit, and in the history display screen, displays both the first display and the second display in the same time series. The information processing apparatus according to (12), wherein the information is displayed in a state of being arranged.
(14) The communication processing unit may include, on the communication screen, a face image of a virtual character corresponding to a user of the other information processing device or a user of the other information processing device, which is a communication partner. One message sent from the virtual character corresponding to the user of the information processing device or the user of the other information processing device, and the message corresponding to the user of the other information processing device or the user of the other information processing device The information processing apparatus according to (13), wherein one message sent to the virtual character is displayed.
(15) The information processing device according to (14), wherein the communication processing unit further causes the communication screen to display an emotional expression assigned to a message displayed on the communication screen.
(16) The communication processing unit, as the emotional expression assigned to a message sent from a user of the other information processing device or a virtual character corresponding to the user of the other information processing device, expresses an emotion of the message. The information processing apparatus according to (15), wherein a face image of a virtual character corresponding to the user of the other information processing apparatus or the user of the other information processing apparatus, which has a facial expression, is displayed on the communication screen.
(17) The communication processing unit, as the emotional expression assigned to a message sent from a user of the other information processing device or a virtual character corresponding to a user of the other information processing device, expresses an emotion of the message. The information processing device according to (15) or (16), wherein the effect image to be displayed is displayed at an arbitrary position on the communication screen.
(18) The communication processing unit is an effect that represents an emotion of the message as the emotion expression assigned to a message sent from a user of the information processing device itself or a virtual character corresponding to the user of the information processing device itself. An image is displayed on the communication screen in a region where a face image of a virtual character corresponding to the user of the other information processing device or the user of the other information processing device is displayed. (15) to (17) An information processing device according to any one of the above.
(19) The information processing device according to any one of (1) to (18), further including the display unit.
(20) Both the first display indicating the history of communication between users and the second display indicating the history of communication between the virtual character that can respond on behalf of the user and the user have the same time series. An information processing method in which the information is displayed on the display unit in a state where the information is arranged along the line.
(21) Computer
Both the first display showing the history of communication between users and the second display showing the history of communication between the virtual character that can respond on behalf of the user and the user are provided along the same time series. A program that functions as a display control unit that is displayed on the display unit in a lined up state.

  100 communication system, 101 user, 102 virtual space, 103 virtual character, 110 server, 111 user management server, 112 virtual character management server, 113 communication management server, 114 analysis server, 121 network, 131 terminal device, 151 CPU, 181 user Information management section, 182 friend management section, 183 setting management section, 184 learning section, 185 mode management section, 186 message setting section, 187 open space management section, 188 state management section, 189 communication management section, 190 history management section, 191 Message analysis section, 192 emotion setting section, 193 effect setting section, 201 SOC, 261 initial setting section, 262 square processing section, 263 communiqué Deployment unit, 264 an interrupt processing unit 265 mode setting unit, 266 history display control unit

Claims (20)

First display indicating communication history between users, and arranges both of the second display indicating communication history with the virtual character and the user can respond on behalf of the user along the same chronological The history display screen displayed with the
A communication screen displaying a face image of the virtual character and one message of the user or the virtual character transferred by the communication,
An information processing apparatus including a display control unit that superimposes and displays the information on a display unit. The history of communication between the users includes a history of communication between a first user who is a predetermined user and another user other than the first user,
The communication history between the virtual character and the user may be the communication history between the first user and the virtual character corresponding to the other user, or the virtual character corresponding to the first user and the other user. The information processing device according to claim 1, wherein the information processing device includes a history of communication with the device. The first display includes text information representing a message sent from the first user to the other user, or a message sent from the other user to the first user,
The second display is a message sent from the first user to a virtual character corresponding to the other user, or a message sent from the virtual character corresponding to the other user to the first user. Or a message sent from the virtual character corresponding to the first user to the other user, or a message sent from the other user to the virtual character corresponding to the first user The information processing apparatus according to claim 2, wherein the information processing apparatus includes text information representing The communication history further includes a third display expressing an emotion assigned when each message is exchanged,
Wherein the display control unit, in the history display screen, the information processing apparatus described in conjunction with the message, to claim 3 for displaying the emotional expression to be allocated upon exchanging the message. The emotional expression includes an expression of a face image of a user or a virtual character that is a sender of the message,
5. The information according to claim 4, wherein the display control unit displays, on the history display screen, a face image of a user or a virtual character who is a sender of the message, with a facial expression expressing an emotion of the message, together with each message. Processing equipment. The emotional expression includes an effect image representing the emotion of the message,
Wherein the display control unit, in the history display screen, along with each message, the information processing apparatus according to claim 4 or claim 5 for displaying an effect image representing the emotion of the message. The emotional expression includes a balloon shape representing the emotion of the message,
The information processing device according to claim 4, wherein the display control unit displays, on the history display screen, a balloon having a shape representing an emotion of the message, together with each message. The said emotional expression is based on the information obtained by sensing the said 1st user who becomes a receiver of a message, or the said other user at the time of giving / receiving the said message. An information processing device according to any one of the above. The display control unit, on the history display screen,
A message sent from the first user to the other user or a virtual character corresponding to the other user, and a message sent from the virtual character corresponding to the first user to the other user, Display them in a state that they can be distinguished from each other,
A message sent from the other user to the first user or a virtual character corresponding to the first user, and a message sent to the first user from the virtual character corresponding to the other user. The information processing apparatus according to any one of claims 3 to 8, wherein the information is displayed in a state where the information can be distinguished from each other. An instruction receiving unit that receives an instruction to rearrange the messages is further provided.
Wherein the display control unit, when the sort of indication of the message by the instruction reception unit is received, the accepted by the instruction accepting unit, and rearranges each message according to the sort of condition of the message, the history The information processing device according to claim 3, wherein the information processing device displays the information on a display screen . An instruction receiving unit that receives an instruction to select the message is further provided,
Wherein the display control section, when the instruction receiving unit by the selection of the message indication is received, the accepted by the instruction accepting unit, select a message according to the condition of selection of the message, the selected message, the The information processing apparatus according to claim 3, wherein the information is displayed on a history display screen . The display control unit causes the user to slide the finger from the top to the bottom of the display unit to cause the history display screen to be displayed on the display unit in a manner superimposed on the communication screen.
The information processing device according to claim 1 . Communication between the user of the information processing apparatus itself and a user of the other information processing apparatus by communicating with the other information processing apparatus via the communication unit to exchange the message, Communication processing for realizing communication between a user and a virtual character corresponding to a user of the other information processing apparatus, and communication between a virtual character corresponding to a user of the information processing apparatus itself and a user of the other information processing apparatus Part further,
The display control unit, on the history display screen, the first display representing the history of communication between the user of the information processing device itself and the user of the other information processing device, performed by the communication processing unit, A communication history between a user of the information processing apparatus itself and a virtual character corresponding to the user of the other information processing apparatus, or a virtual character corresponding to the user of the information processing apparatus and the other information processing apparatus Both of the second displays representing the history of communication with the user are displayed in the same time-series.
The information processing apparatus according to any one of claims 1 to 12. The communication processing unit may include, on the communication screen, a face image of a virtual character corresponding to a user of the other information processing apparatus or a user of the other information processing apparatus, the other information processing apparatus. One message sent from the virtual character corresponding to the user of the other information processing apparatus or the virtual character corresponding to the user of the other information processing apparatus or the user of the other information processing apparatus. The information processing device according to claim 13, wherein one sent message is displayed. The information processing device according to claim 14, wherein the communication processing unit causes the communication screen to further display an emotional expression assigned to a message displayed on the communication screen. The communication processing unit, as the emotion expression assigned to a message sent from a user of the other information processing device or a virtual character corresponding to the user of the other information processing device, a facial expression representing an emotion of the message. The information processing apparatus according to claim 15, wherein a face image of a virtual character corresponding to a user of the other information processing apparatus or a user of the other information processing apparatus is displayed on the communication screen. The communication processing unit is an effect image representing an emotion of the message as the emotion expression assigned to a message sent from a virtual character corresponding to a user of the other information processing apparatus or a user of the other information processing apparatus. The information processing apparatus according to claim 15, wherein the information processing apparatus displays the information at an arbitrary position on the communication screen. The communication processing unit, as the emotional expression assigned to a message sent from a virtual character corresponding to the user of the information processing device itself or the user of the information processing device itself, a rendering image representing the emotion of the message, 18. The communication screen according to claim 15, wherein the communication screen is displayed in a region where a face image of a virtual character corresponding to the user of the another information processing device or the user of the other information processing device is displayed. Information processing device. First display indicating communication history between users, and arranges both of the second display indicating communication history with the virtual character and the user can respond on behalf of the user along the same chronological The history display screen displayed with the
A communication screen displaying a face image of the virtual character and one message of the user or the virtual character transferred by the communication,
An information processing method that is superimposed and displayed on a display unit. Computer
First display indicating communication history between users, and arranges both of the second display indicating communication history with the virtual character and the user can respond on behalf of the user along the same chronological The history display screen displayed with the
A communication screen displaying a face image of the virtual character and one message of the user or the virtual character transferred by the communication,
A program for functioning as a display control unit that is superimposed and displayed on the display unit.

Images