KR20240143236A - Conversation evaluation system for language education - Google Patents

Abstract

The present invention relates to a conversation evaluation system for language education that enables more efficient language education at a low cost by evaluating and correcting the speaking ability of participants in a video communication using artificial intelligence.

Description

{Conversation evaluation system for language education}

The present invention relates to video chat communication technology, and more specifically, to a conversation evaluation system for language education that evaluates and corrects a user's foreign language speaking ability by analyzing voice chat messages sent and received in a video chat.

Foreign language speaking tests are conducted through face-to-face interviews and computer-based assessment methods. Face-to-face interviews are conducted in the form of questions and answers between the examinee and the evaluator on specific topics and situations. Such face-to-face interviews are expensive due to the lack of physical time and resources, and are difficult to apply to the general public.

Computer-based assessments are an assessment tool that measures how effectively and appropriately language can be used in real life, and are used more frequently than face-to-face interviews, as the examinees record their answers to questions on the computer. These computer-based assessments also have limitations in deeply assessing the examinees’ actual conversational skills, as the examinees only respond to questions that are guided by a computer.

Meanwhile, artificial intelligence (AI) is revolutionizing business, organizational operations, lifestyles, and communication methods. Various information projects are being carried out to provide optimal services to the rapidly changing lifestyles of modern culture and the constantly changing needs of customers. Among them, big data and deep learning-related technologies have been rapidly developing recently, and AI technologies that are applied to real life in specific fields have been implemented, and they are also being applied to intelligent personal services that analyze specific data and integrate and utilize information from various fields specialized for each individual. Currently, the interaction between AI and humans is limited, but it is done in the form of everyday natural language, that is, conversation.

Conventional AI foreign language speaking tests evaluate conversational skills through reading a given text, describing a picture, and answering simple questions. In these speaking tests, simple pronunciation skills are measured, or after asking a question, the test taker thinks about and organizes the answer for several seconds and then answers within a given time limit. In terms of evaluation, these tests evaluate the accuracy of pronunciation, the ability to describe a picture, the understanding of the question, and the fluency and consistency of the answer. These AI speaking tests also have difficulty in deeply evaluating the test taker's conversational skills, and since questions are often curated based on the test taker's questionnaire input, there is a limit to receiving a high speaking ability evaluation score if the test taker learns the question pattern and prepares only the answers that match the pattern in advance. In addition, test takers can only receive a score and a general explanation of their evaluation results, and cannot receive detailed item-by-item feedback that analyzes what the test taker said, making it difficult to identify which areas they are lacking in.

Patent Document 1 (Korean Patent Publication No. 2007-0024918) discloses a method for providing a video communication service in a virtual space, which provides video communication in a virtual space and enables more realistic conversations to be performed in the virtual space according to various scenarios provided in the virtual space.

Patent Document 2 (Korean Patent Publication No. 2002-0066804) discloses a real-time online cyber learning system that provides various videos, documents, external applications, etc. necessary for the class to the learner in real time by connecting to the Internet, as a learning method that enables a live interface between the instructor and the learner, thereby enhancing the learning effect.

Patent Document 3 (Korean Patent Publication No. 2021-0059995) discloses a learning-based foreign language speaking evaluation method, including the steps of: receiving a user's voice answering a preset question in a foreign language, and converting the received user's voice into text; checking the user's speech volume based on analysis of the converted text, and extracting at least one of grammar, emotion, topic, and category for the answer to the question; determining at least one of word usage appropriateness, grammatical appropriateness, and question relevance for the answer to the question using a neural network based on a learning model that uses the converted text as input; and evaluating the user's foreign language speaking ability based on the user's speech volume, the extracted at least one, and the determined at least one.

KR 2007-0024918 A KR 2002-0066804 A KR 2021-0059995 A

In the case of existing language study groups, the leader listens to what others say and evaluates and corrects them. However, there is a significant shortage of people who play this leadership role, and especially leaders with a certain level of language proficiency are even more in short supply.

Therefore, the first technical task of the present invention is to reduce educational costs and improve educational quality by replacing the leaders of such language study groups with artificial intelligence (AI).

In addition, the present invention has as its second technical task to enable more efficient language education at a low cost by evaluating and correcting the speaking ability of participants in a video communication using artificial intelligence.

In order to achieve the above-described purpose, the first aspect of the present invention provides a conversation evaluation system for language education, which comprises the steps of: in an environment where a plurality of user terminals, each of which has a video chat application installed to provide a video chat service between a plurality of users, and a chat server that mediates video chat messages between the plurality of user terminals, are connected via a network; in the chat server, opening a conversation session for video chatting to send and receive video chat messages using the video chat application installed in the plurality of user terminals; adding a chat bot, which is a conversational artificial intelligence, as a friend to the conversation session; obtaining the video chat message by the chat bot, analyzing the video chat message, and evaluating the user's foreign language speaking ability; and executing the step of the chat bot correctly correcting the video chat message based on the evaluation level of the user's foreign language speaking ability, and responding the corrected message to the corresponding user terminal.

As another second aspect of the present invention, a conversation evaluation system for language education is characterized in that, in the first aspect, the Chat Bot includes: a voice data analysis unit that receives the video chat message, converts voice conversation data of the chat message into text conversation data, and divides the text conversation data into word units; a speaking evaluation unit that determines at least one of word usage appropriateness, grammar appropriateness, and pronunciation appropriateness based on the text conversation data divided into word units, and evaluates the user's foreign language speaking ability based on the determination; and a speaking correction unit that generates a message including correctly corrected words, grammar, and pronunciation for incorrect words, grammar, and pronunciation in the voice conversation data, and responds to the corresponding user as a chat message.

As another third aspect of the present invention, a conversation evaluation system for language education is characterized in that, in the second aspect, the chat server stores the voice conversation data, its evaluation information, and corrected chat messages as one data set.

As another fourth aspect of the present invention, a conversation evaluation system for language education is characterized in that, in the third aspect, it further includes a deep learning server for upgrading the artificial intelligence algorithm of the chatbot by machine learning the data sets into big data.

According to the present invention, since one's foreign language speaking ability can be evaluated and corrected by conversational artificial intelligence during a natural chat process between users, efficient language education is possible at a low cost, eliminating the need to study abroad or pay expensive lesson fees to learn a language.

The following drawings attached to this specification illustrate preferred embodiments of the present invention and, together with the detailed description of the invention described below, serve to further understand the technical idea of the present invention; therefore, the present invention should not be interpreted as being limited to matters described in such drawings.
Figure 1 is a schematic diagram of a video chat system using artificial intelligence according to a preferred embodiment of the present invention.
FIG. 2 is a functional block diagram illustrating the configuration of a chat server according to a preferred embodiment of the present invention.
FIG. 3 is a screen capture of an example of a chat window for video chat according to the present invention.
Figure 4 is a functional block diagram illustrating the configuration of a chatbot according to a preferred embodiment of the present invention.

The present invention can have various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

When it is said that a component is "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but there may also be other components in between.

On the other hand, when it is said that a component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

The terminology used herein is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprises" or "has" and the like are intended to specify the presence of a feature, number, process, operation, component, part or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, processes, operations, components, parts or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms defined in commonly used dictionaries, such as those defined in common dictionaries, should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and shall not be interpreted in an idealized or overly formal sense unless expressly defined in this application.

The terms or words used in this specification and claims should not be interpreted as limited to their common or dictionary meanings, and should be interpreted as meanings and concepts that conform to the technical idea of the present invention based on the principle that the inventor can appropriately define the concept of the term in order to explain his or her own invention in the best way. In addition, if there is no other definition for the technical and scientific terms used, they have the meanings commonly understood by those of ordinary skill in the technical field to which this invention belongs, and the description of well-known functions and configurations that may unnecessarily obscure the gist of the present invention in the following description and the attached drawings are omitted. The drawings introduced below are provided as examples so that those skilled in the art can sufficiently convey the idea of the present invention. Therefore, the present invention is not limited to the drawings presented below and may be embodied in other forms. In addition, the same reference numerals represent the same components throughout the specification. It should be noted that the same components in the drawings are represented by the same symbols wherever possible.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. The attached drawings are not drawn to scale, and like reference numerals in each drawing indicate like components.

Figure 1 is a schematic diagram of a foreign language speaking evaluation system using artificial intelligence according to a preferred embodiment of the present invention.

Referring to FIG. 1, the foreign language speaking evaluation system using artificial intelligence according to the present invention includes a chat server (210) that provides a video chat service to a plurality of user terminals (110 to 130) through a wired or wireless network (104). This video chat service is provided from the chat server (210) through a video chat application (111) installed in the user terminals (110 to 130).

According to one embodiment of the present invention, while a video conversation is in progress between users of user terminals (110 to 130), the chat server (210) evaluates and correctly corrects a foreign language (e.g., English) voice conversation input by a user based on a chat bot (220), which is an interactive artificial intelligence. That is, the chat server (210) causes the chat bot (220) to participate as one of the participants in the video conversation. The chat bot (220) recognizes and analyzes the conversation between users, and evaluates and corrects the user's foreign language in real time.

The above chat server (210) may include an artificial intelligence chatbot internally for evaluating and correcting users' foreign language speaking ability, but may also collaborate with at least one external chatbot (220) for evaluating and correcting users' foreign language speaking ability.

According to one embodiment of the present invention, the chat server (210) stores voice data generated during a video chat conversation between users of user terminals (110 to 130), evaluation information for the voice data, and correction information as big data, and may collaborate with a deep learning server (230) to use the big data as learning data to perform deep learning to increase the accuracy of the chat bot (220).

The above user terminals (110 to 130) may be fixed terminals or mobile terminals implemented as computer devices. Examples of the user terminals (110 to 130) include smart phones, mobile phones, navigation devices, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, etc. For example, the user terminals (110 to 130) may communicate with other terminals and/or a chat server (210) through a network (104) using a wireless or wired communication method.

The communication method is not limited, and may include not only a communication method that utilizes a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network (104) may include, but also a short-range wireless communication between devices. For example, the network (104) may include any one or more of a network such as a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), and the Internet. In addition, the network (104) may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree, or a hierarchical network.

The chat server (210) may be implemented as a computer device or multiple computer devices that communicate with user terminals (110 to 130) through a network (104) to provide commands, codes, files, contents, services, etc. For example, the chat server (210) allows multiple users of user terminals (110 to 130) connected through a network (104) and a chat bot (220) to participate in a single interactive chat room, and provides an environment in which the participating users can communicate with each other through voice messages or text messages.

FIG. 2 is a functional block diagram illustrating the configuration of a chat server according to a preferred embodiment of the present invention.

Video chat according to the present invention is provided from the chat server (210) through a video chat application installed on a plurality of user terminals (110 to 130).

According to one embodiment of the present invention, while a conversation is in progress within a video chat session between users of user terminals (110 to 130), the chat server (210) allows a chatbot (220) to participate as a participant in the conversation, recognizes and analyzes voice messages (or text messages) input from the users, and evaluates and corrects the users' foreign language speaking skills.

In addition, the chat server (210) according to the present invention for providing a video chat service between users includes a connection management unit (211), a user information storage unit (212), a processing unit (213), a database (214), a first linkage unit (215), and a second linkage unit (216) as functional modules. The above functional modules of the chat server (210) are preferably implemented in software, but of course, can be implemented in hardware.

The above video chat application may be distributed from an application store or market as one of multiple applications installed on a user terminal (110).

Hereinafter, the 'video chat' means exchanging voice messages or text messages while viewing each other's appearance by transmitting one's movements in real time by mounting a camera on the user terminal (110). In addition, the 'chat window' or 'conversation window' means a window that performs a video chat service through the screen of the user terminal (110).

The connection management unit (211) of the chat server (210) can be understood as a physical/logical communication module that allows the chat server (210) and various user terminals (such as 101 to 103) to be connected via the network (104). Messages received from user terminals (101 to 103) are transmitted to the processing unit (213) via the connection management unit (211), and the processing unit (213) transmits messages (particularly, voice messages) between users to the chatbot (220), and the chatbot (220) can evaluate and correct the user's voice messages and transmit the results to the user terminals (101 to 103) via the connection management unit (211).

The above user information storage unit (220) represents an entire physical/logical configuration that can store and manage various user subscription information (age, nationality, gender, etc.) and information about conversation sessions being conducted by users.

And, the processing unit (230) receives a user's voice message in a one-to-one, one-to-many, or many-to-many chat service and transmits it to other users, and at the same time, participates the chatbot (220) in the chat between users as one of the chat participants to evaluate and correct the conversation between users.

FIG. 3 is a screen capture of an example of a chat window for a video chat according to the present invention. Referring to the drawing, it shows a state in which a chatbot, which is an interactive artificial intelligence (AI), is participating as a conversation participant in the chat window in a video chat according to the present invention.

The above database (240) stores various programs for a video chat service according to the present invention, particularly, the video chat application, and stores voice messages generated during a chat process according to the present invention and their evaluation information and correction information as one data set.

The above first linkage unit (215) represents an entire physical/logical configuration that interfaces the chatbot (220) for evaluating and correcting the user's voice message and the processing unit (213).

The above second linkage unit (216) represents an integral physical/logical configuration that interfaces the processing unit (213) of the chat server (210) and the deep learning server (230) to transmit the data set including the voice message according to the present invention and its evaluation information and correction information to the deep learning server (230).

In addition, various operations or applications performed by the chat server (210) are described below. Even if no configuration of the connection management unit (211) or the second linkage unit (216) is specified, the content that can be clearly understood and expected by a person skilled in the art of the present invention can be understood as a typical implementation, and the scope of the rights of the present invention is not limited by the name of a specific configuration or the physical/logical structure.

Figure 4 is a functional block diagram illustrating the configuration of a chatbot according to a preferred embodiment of the present invention.

The chatbot (200) according to an embodiment of the present invention obtains foreign language voice messages between users, recognizes and analyzes the obtained voice messages to evaluate the user's foreign language ability, and performs an operation of correcting errors. The chatbot (220) according to an embodiment of the present invention can perform a foreign language ability evaluation operation and a correction operation based on an artificial intelligence algorithm operation process. The chatbot (220) can perform an operation of obtaining chat data between users (Data collection), an operation of processing and preprocessing chat data into data for analysis (Processing), an operation of performing a convolution operation based on an artificial intelligence algorithm, statistical analysis (Statistical analysis), and an evaluation and correction operation.

Prior to performing the evaluation and correction operations of the voice message of the chatbot (220), the deep learning server (230) according to the embodiment of the present invention can continuously improve the function of the chatbot (220) by training an artificial intelligence algorithm based on a data set (voice message-evaluation information-correction information) corresponding to a specific standard set by the chatbot (220).

The chatbot (220) of the present invention can participate in a chat conversation between users and input voice data extracted from the user's chat data into an artificial intelligence algorithm for evaluating and correcting foreign language ability modeled by the deep learning server (220), thereby evaluating the user's foreign language conversation ability and performing an operation of correcting errors.

Specifically, the chatbot (220) can perform an operation of acquiring foreign language voice data during a video chat process between users. The chatbot (220) determines at least one of word usage appropriateness, grammatical appropriateness, and pronunciation appropriateness of the extracted user's conversation voice data using a neural network based on a learning model, and evaluates the user's foreign language speaking ability based on this determination. Then, for incorrect words, grammar, and pronunciation in the conversation voice data, it generates a message (voice message or text message) including correctly corrected words, grammar, and pronunciation, and responds to the corresponding user as a chat message.

The above conversational voice data may be composed of multiple sentences, and accordingly, the chatbot (220) may perform analysis on all of the multiple sentences that compose the voice data. The chatbot (220) evaluates and corrects word usage suitability, grammatical suitability, and pronunciation suitability for the entire conversational voice data based on the results of analyzing all of the sentences that compose the conversational voice data.

The operation of the chatbot (220) will be described in more detail with reference to Fig. 4. The chatbot (220) may be configured to include a voice data analysis unit (221), a speech evaluation unit (222), and a speech correction unit (223) depending on its function.

The above voice data analysis unit (221) can analyze the voice data of a user's chat message through three major operations. Specifically, the voice data analysis unit (221) can perform an operation of obtaining voice conversation data between video chat users, an operation of converting the obtained voice conversation data into text conversation data, and an operation of dividing the text conversation data into word units and analyzing it.

The above voice data analysis unit (221) receives a chat message generated from a video chat between users from the first link unit (215) of the chat server (210), converts the voice conversation data of the chat message into text conversation data using a TTS (Sound To Text) program, etc., and analyzes the text conversation data by dividing it into word units.

The above-mentioned speaking evaluation unit (222) determines at least one of word usage suitability, grammatical suitability, and pronunciation suitability based on text conversation data divided into word units by the above-mentioned voice data analysis unit (221), and evaluates the user's foreign language speaking ability based on this determination. For example, the above-mentioned speaking evaluation unit (222) may evaluate the user's foreign language speaking ability by classifying it into levels such as upper, middle, and lower, or may express it as a score based on a full score of 100.

The above-mentioned speech correction unit (223) generates a message (voice message or text message) including correctly corrected words, grammar, and pronunciation for incorrect words, grammar, and pronunciation of the corresponding voice conversation data when the evaluation level of the above-mentioned speech evaluation unit (222) falls below a certain standard, and returns it to the first linkage unit (215) of the above-mentioned chat server (210) to respond as a chat message to the corresponding user. The chat server (210) receives the evaluation level by the above-mentioned speech evaluation unit (222) and the message corrected by the above-mentioned speech correction unit (223) through the first linkage unit (215), and transmits them to the corresponding user terminal (110) through the access management unit (211).

Below, a method for evaluating and correcting a user's foreign language speaking ability based on a voice conversation message of a video chat according to the present invention is described.

First, a user who wishes to use the video chat service of the present invention downloads and installs a video chat application on a user terminal (110, 120, 130). Hereinafter, a case in which the user terminal is a smart phone (110) will be described as a representative example.

The above video chat application may be distributed from an application store or market as one of multiple applications installed on a user terminal (110). When a user runs the video chat application, a conversation session can be opened with registered friends and voice messages or text messages can be exchanged.

The above video chat application can access the contact list of the user terminal (110) and automatically add friends who have installed the same application to the friend list, and can also add friends directly. For example, the invitation can access the contact list in the user terminal (110) and recommend the installation of the application to a friend who does not use the video chat application. In addition, by performing QR code reading, it can add another user as a friend corresponding to the information included in the recognized QR code, and can manually add a friend by directly searching for the ID of another user.

A user who wishes to use video chat according to the present invention opens a chat room including at least one other user using the video chat application.

The processing unit (213) of the chat server (210) recognizes the opening of the chat room and allows the chatbot (220) to participate in the chat room as a conversation partner, as shown in Fig. 3. Accordingly, the chat server (210) and the chatbot (220) become capable of recognizing all chat messages between users in the chat room.

In this state, the connection management unit (211) of the chat server (210) receives a voice chat message entered by the user in the user conversation session and transmits it to the processing unit (213).

The processing unit (213) that receives the above voice chat message stores the voice chat message in a database (214) and then transmits the voice chat message to the user terminals (110) of other users participating in the conversation session through the connection management unit (211).

The video chat application of the other users displays the voice chat message in the chat conversation window (i.e., chat window) of the user terminal (111) of the user.

During this chat process, the chatbot (220) obtains the voice chat messages exchanged between users, analyzes the voice chat messages to evaluate the user's foreign language speaking ability, corrects errors, generates a corrected chat message, and returns it to the user.

And, the processing unit (213) of the chat server (210) receives speaking ability evaluation information and correction information from the chatbot (220) through the first link unit (215), and stores this together with the voice chat message as a data set in the database (214).

The data set saved in this manner is transmitted to the deep learning server (230) through the second linkage (216), and the deep learning server (230) continuously upgrades the speaking correction model, which is an artificial intelligence algorithm of the chatbot (220), by machine learning this data set into big data. Accordingly, the artificial intelligence algorithm for evaluating and correcting the foreign language speaking ability of the chatbot (220) is improved to be more accurate and precise.

Although the present invention has been described above by means of limited embodiments and drawings, the present invention is not limited thereto, and various modifications and variations are possible by those skilled in the art within the scope of the technical idea of the present invention and the equivalent scope of the patent claims to be described below.

110,120,130: user terminal, 104: network, 210: chat server, 211: access management unit, 212: user information storage unit, 213: processing unit, 214: database, 215: first linkage unit, 216: second linkage unit, 220: chatbot, 221: voice data analysis unit, 222: ㅁ마말하기 evaluation unit, 223: speech correction unit, 230: deep learning server

Claims (4)

In an environment where multiple user terminals are installed with a video chat application that provides a video chat service between multiple users, and a chat server that mediates video chat messages between the multiple user terminals is connected via a network,
In the above chat server,
A step of opening a conversation session for video chatting to send and receive video chat messages using a video chat application installed on the plurality of user terminals;
A step of adding a chatbot, a conversational artificial intelligence, as a conversation buddy to the above conversation session;
A step in which the above chat bot obtains the video chat message and analyzes the video chat message to evaluate the user's foreign language speaking ability;
A conversation evaluation system for language education, characterized in that the chat bot correctly corrects the video chat message based on the evaluation level of the user's foreign language speaking ability and executes a step of responding with the corrected message to the corresponding user terminal. In the first paragraph, the chat bot,
A voice data analysis unit that receives the above video chat message, converts the voice conversation data of the chat message into text conversation data, and divides the text conversation data into word units;
A speaking evaluation unit that judges at least one of word usage appropriateness, grammatical appropriateness, and pronunciation appropriateness based on text conversation data divided into the above word units, and evaluates the user's foreign language speaking ability based on such judgment; and
A conversation evaluation system for language education, characterized by including a speech correction unit that generates a message including correctly corrected words, grammar, and pronunciation for incorrect words, grammar, and pronunciation of the above-mentioned voice conversation data, and responds to the corresponding user as a chat message. In the second paragraph,
A conversation evaluation system for language education, characterized in that the above chat server stores the voice conversation data, its evaluation information, and corrected chat messages as a single data set. In the third paragraph,
A conversation evaluation system for language education, characterized in that it further includes a deep learning server for upgrading the artificial intelligence algorithm of the chatbot by machine learning the above data sets into big data.

Images