JP2006074514A - Image editing device, image reproducing device, file database, file distributing server, image editing method, image editing program, image reproducing method, image reproducing program, and computer readable storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively transmit content of a lecture etc. recorded in a video file to a user. <P>SOLUTION: A device comprises a speech recognition unit 5 for converting audio data to a text file after recognition, and a file producing unit 10 for reproducing LIVE textbook relating the text file to a video file based on time information indicating which time content of the text file corresponding to the video file. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a video editing apparatus or the like for efficiently transmitting to a user the contents of a video file that records a lecture or the like.

  Activities such as recording university lectures and distributing them online to students, renting them through the media, and providing them to society for a fee have been conducted at many universities in Japan and abroad for a long time. As remote lectures and e-learning have been introduced, the recording and distribution of university lectures has become easier and cheaper as information media equipment and network environments have been improved. Use is considered to become increasingly important as a service that utilizes intellectual property possessed only by universities (see Non-Patent Documents 1 to 4).

  In general, a 90-minute lecture cannot be understood without taking a 90-minute video recording (video, DVD, etc.). However, this is not very attractive for students who want to review the lecture before the exam because of the great effort to find the places they want to see. In order for review to be attractive, it is necessary to have a content structure and an efficient operation function that can quickly grasp the overall structure of the lecture and play back the scene you want to watch directly.

For that purpose, it is essential to structure the content of lecture recordings. Many proposals have been made on the technology for structuring the contents of recording, but broadly speaking, an indexing technology based on video recognition and voice recognition can be mentioned. With this indexing technology, for example, it is possible to create a search key by recognizing from a recorded content of a lecture that the scene changes, there is a face, there are many red colors, and it is a document screen.
Kazuo Iwasawa, Mitsuo Ishii, Norikazu Iwata, Katsumi Konishi, “Public Lecture Archive as Digital Content Research Base”, Computer and Educational Research Report, No.73, p47-54,2004 "Education support system that digitizes and distributes lecture videos-Problems and prospects seen from the test assuming introduction", Computer and Educational Research Report, No.71, p35-42,2003 Hirofumi Fujikura et al., "Proposal of a learning support system using video archives", Spoken Language Information Processing Research Report, No040-001, 2001 Masahiro Sakakibara, Yoshio Himawari, Fumio Hino, Kazuhiko Takatori, “Construction of simple pharmacy class archives by digitizing board and audio”, [online], 2003, [searched July 27, 2004], Internet <URL : Http:https://www.shijokyo.or.jp/LINK/journal/0304/06_01.html>

  However, the conventional search key was created using the characteristics of the recorded video of the lecture, and does not closely represent the content of the lecture, so that the content of the lecture can be reviewed using recorded materials. Is not a valid search key. That is, the search key based on the image feature is not related to the content of the lecture, and cannot be used as an aid for understanding the content of the lecture.

  The present invention has been made in view of the above-described conventional problems, and is a video editing device, a video playback device, a file database, and a file distribution server that can efficiently transmit the contents of a lecture or the like recorded in a video file to a user. An object is to provide a video editing method, a video editing program, a video playback method, a video playback program, and a computer-readable recording medium.

  In order to solve the above problems, the video editing apparatus of the present invention is a video editing apparatus for editing a video file in which moving image data and audio data are recorded, and recognizes the audio data and converts it into a text file. Voice recognizing means, and a file associating means for associating the text file and the video file based on time information indicating the time of the text file in the video file. It is said.

  The video editing method of the present invention is a video editing method for editing a video file in which moving image data and audio data are recorded, the first step of recognizing the audio data and converting it into a text file; A second step of associating the text file with the video file based on time information indicating at what time the text file is in the video file.

  According to the above configuration, the content of the video file can be displayed on the external device by the voice recognition means or the text file converted by the first step, so the content of the video file is converted into text and efficiently transmitted to the user. be able to.

  Furthermore, since the video file and the text file are associated by the file association means or the second step, the video file and the text file can be synchronized and displayed on the external device. That is, the video file and the text file can be displayed on the external device in a state of being associated with each other, and the contents of the video file can be transmitted to the user more efficiently.

  Furthermore, the video editing apparatus having the above-described configuration is adapted to categorize the contents of the text file based on scene change detection means for detecting a scene change, which is a location where a scene changes in the video file, and the time of the scene change. It is preferable to include a title creating means for creating a title name for each item.

  According to the above configuration, the title name for the text file is created by the title creation unit based on the scene change time detected by the scene change detection unit. Therefore, by displaying the title name on the external device, the user can roughly grasp the contents of the text file. Therefore, the contents of the video file can be transmitted to the user more efficiently.

  Furthermore, the video editing apparatus having the above configuration adjusts the number of items for dividing the text file based on the time interval between the scene changes when a plurality of scene changes are detected from the video file by the scene change detecting means. It is preferable to further include an item number adjusting means.

  That is, if the title names created for the text file increase, the content of the video file may be difficult to convey to the user. In view of this, the video editing apparatus having the above-described configuration particularly includes item number adjusting means.

  According to this item number adjusting means, it is possible to select an appropriate number of items in the text file that have a long time interval between scene changes and create a title name based on the time of the scene change. Thereby, the number of items can be adjusted as appropriate so that the content of the video file can be transmitted to the user more efficiently.

  Further, in the video editing apparatus having the above-described configuration, the file association unit further associates the data of the material distributed in the moving image recorded in the video file with at least one of the text file and the video file. It is preferable.

  According to the above configuration, at least one of the text file and the video file is associated with the data of the distribution material by the file association unit. This association may be performed based on time information indicating the time when the material is distributed, for example. As a result, when the text file and the video file are displayed on the external device, the data of the distribution material can be displayed in association with these files.

  Therefore, since the user can check the data of the distribution material when viewing the video file, the user can understand the contents of the video file more efficiently.

  In order to solve the above problems, the video playback device of the present invention is a video playback device that plays back a video file in which moving image data and audio data are recorded, and is a text file into which the audio data has been converted. The text file associated with the time information indicating the time content in the video file is displayed simultaneously with the video file.

  In addition, the video playback method of the present invention is a video playback method for playing back a video file in which moving image data and audio data are recorded in order to solve the above problems, and is a text file in which the audio data is converted. The text file associated with the time information indicating the time content in the video file is displayed simultaneously with the video file.

  According to the above configuration, the text file obtained by converting the audio data of the video file and the video file are displayed simultaneously. Therefore, the user can simultaneously obtain the content of the video file documented by the text file and the moving image and sound of the video file, so that the content of the video file can be understood more efficiently.

  Furthermore, it is preferable that the video reproducing apparatus having the above configuration highlights a part of the text file corresponding to the content currently spoken in the moving image recorded in the video file.

  According to the above configuration, the user can understand the content currently spoken in the moving image recorded in the video file as a sentence by visually recognizing the highlighted portion of the text file. Can understand the contents of video files.

  Furthermore, it is preferable that the video reproducing apparatus having the above configuration reproduces the video file from a portion corresponding to a portion selected by the user in the displayed text file.

  Furthermore, the video playback device having the above configuration displays a plurality of title names for categorizing the contents of the text file, and from the locations corresponding to the title name selected by the user among the plurality of title names. It is preferable to play a video file.

  Furthermore, the video playback device having the above configuration displays a plurality of thumbnail images corresponding to the location where the scene changes in the video file, and also includes a location corresponding to the thumbnail image selected by the user among the plurality of thumbnail images. It is preferable to play the video file from.

  According to the above configuration, if a user selects a part of a text file, a title name, or a thumbnail image, the user can play a video file from a location corresponding to that part. Furthermore, when a user selects a part of a text file, a title name, or a thumbnail image, the user can check the content of the video file to be played based on the content of the text file. You can select exactly where you want.

  Furthermore, it is preferable that the video reproduction apparatus having the above configuration displays a link to data of a material distributed in a moving image recorded in the video file in association with at least one of the text file or the video file. .

  According to the above configuration, since the link to the data of the distribution material is displayed in association with the text file or the video file, the user appropriately selects the link when these files are displayed, You can refer to handouts. Thereby, the user can understand the content of the video file more efficiently.

  Furthermore, in order to solve the above problems, the file database of the present invention is a text file obtained by converting audio data of a video file in which moving image data and audio data are recorded, and video file reproduction for reproducing the video file. The video file playback file includes the text file and the video file based on time information indicating at what time the text file is in the video file. The video file playback file is associated with a file, and when the external device obtains the video file playback file, the text file is associated with the video file and played back on the external device.

  According to the above configuration, since the text file and the video file are associated with each other by the video file playback file, by distributing the video file playback file to the outside using the file distribution server, the video is played on the external device. A file and a text file can be linked and played. Therefore, the user can simultaneously obtain the content of the video file documented by the text file and the moving image and sound of the video file, so that the content of the video file can be understood more efficiently.

  Note that the same effects as those of the video editing method of the present invention can be obtained using a computer by a video editing program that causes a computer to execute the video editing method. Furthermore, the video editing program can be executed on an arbitrary computer by storing the video editing program in a computer-readable recording medium.

  Further, the same effect as the video playback method of the present invention can be obtained using a computer by a video playback program that causes a computer to execute the video playback method. Furthermore, the video reproduction program can be executed on an arbitrary computer by storing the video reproduction program in a computer-readable recording medium.

  According to the present invention, the content of the video file can be documented and transmitted to the user by the text file, and the video file and the text file can be displayed in a state of being associated with each other. It can be communicated to the user more efficiently.

[1. Overall system configuration)
The configuration of a system that uses an embodiment of the video editing apparatus of the present invention will be described with reference to FIG.

  As shown in FIG. 2, in the LIVE textbook data file distribution system of this embodiment, a LIVE textbook creation device (video editing device) 1 is connected to a video file distribution server 2 and a plurality of learning terminals (video playback) via a network. Device) 3... The LIVE textbook data file means a file configured so that a user can easily review a lecture by structuring a video file in which the contents of the lecture are recorded according to a procedure described later.

  The video file distribution server 2 distributes video files in MPEG format or WMV format stored in the video file database 2a to the outside via a network. The network used for distributing the video file may be the Internet or a LAN (Local Area Network). The video file database is simply described as “video file DB” in FIG.

  The LIVE textbook creation device 1 receives a video file (for example, a file recording a lecture) from the video file delivery server 2 and creates a LIVE textbook data file. The procedure in which the LIVE textbook creation device 1 creates the LIVE textbook data file will be described later.

  The LIVE textbook creation device 1 stores the LIVE textbook data file created by itself in the LIVE textbook database (file database) 1a. In FIG. 2, the LIVE textbook database is simply described as “Live textbook DB” in FIG.

  The learning terminal 3 acquires a LIVE textbook data file from the LIVE textbook database 1a via a network, and reproduces and displays the file. The display mode of the LIVE textbook data file on the learning terminal 3 will be described later.

  In this way, in the LIVE textbook data file distribution system of the present embodiment, the LIVE textbook creation device 1 creates a LIVE textbook data file from the video file acquired from the video file database 2a, and learns the LIVE textbook data file as a learning terminal. 3 can be played back via the network.

[2. Configuration of LIVE textbook creation device 1]
Next, the configuration of the LIVE textbook creation apparatus 1 will be described more specifically with reference to FIG. As shown in FIG. 1, the LIVE textbook creation apparatus 1 includes a video file acquisition unit 4, a voice recognition unit (voice recognition unit) 5, a text file processing unit 6, and a scene change detection unit (scene change detection unit) 7. And an index image file creation unit 8, an overall configuration adjustment unit (title creation unit, item number adjustment unit) 9, and a LIVE textbook reproduction file creation unit (file association unit) 10.

  The video file acquisition unit 4 acquires a video file recording the lecture contents from the video file distribution server 2 (FIG. 2) via the network. Further, the video file acquisition unit 4 may convert the video file acquired from the video file distribution server 2 into a format that can be easily edited. For example, if video files are converted to MPEG2 format using Adobe Premiere (Adobe software product name), various editing operations are facilitated.

  The voice recognition unit 5 recognizes the voice recorded in the video file itself acquired by the video file acquisition unit 4 or a file in which the video file is converted into an easily editable format, and the voice is converted into a text file in units of words. It is to convert to. This voice recognition is automatically performed using commercially available voice recognition software. The text file converted by the voice recognition unit 5 is given information (time stamp) indicating at which time each piece of data constituting the text file corresponds to the voice to be reproduced in the video file. ing.

  The text file processing unit 6 collects the text files converted by the speech recognition unit 5 in phrase units. That is, the text file processing unit 6 sets the silent period as a segment break if the silent period of the voice recognized by the voice recognition unit 5 continues for a predetermined length (for example, 2 seconds) or longer. Furthermore, the text file processing unit 6 stores the text file collected in units of clauses in the LIVE textbook database 1a.

  The scene change detection unit 7 detects a portion (scene change) where the scene changes in the video file. A scene change can be detected as follows, for example.

  That is, the scene change detection unit 7 extracts still images from the video file at intervals of 1 second, for example. If the video file has a length of 90 minutes, 5400 still images are extracted. Then, the scene change detection unit 7 reduces one still image to a thumbnail image having a size of 96 × 72 pixels, for example. Further, the scene change detection unit 7 divides the thumbnail image into nine as shown in FIG. 3 to form nine divided images, and calculates the average of the pixel values of the pixels included in each divided image.

  Then, the scene change detection unit 7 calculates the average difference of the pixel values for the partial images corresponding to each other between the thumbnail images arranged every second. That is, as shown in FIG. 3, R = Av−Bv is calculated when the average of the pixel values related to the divided image A at time t is Av and the average of the pixel values related to the divided image B at time t + 1 is Bv.

  Furthermore, the scene change detection unit 7 calculates R for each of the r color, the g color, and the b color, and sets the maximum R among the Rs for each color as Rm. In addition, the scene change detection unit 7 calculates Rm calculated in this way for all the divided images included in the thumbnail image.

  Then, the scene change detection unit 7 recognizes that the scene has changed at time t + 1 when the number of divided images in which Rm exceeds a predetermined threshold (P) is greater than the predetermined number (q). It should be noted that the values of P and q can be arbitrarily set so that an appropriate scene change can be detected.

  The index image file creation unit 8 creates an index image file in the wmv format, for example, by arranging a plurality of thumbnail images recognized by the scene change detection unit 7 as having changed scenes in time series. By referring to the index file image file created in this way, the outline of the video file acquired by the video file acquisition unit 4 can be grasped. Further, the index image file creation unit 8 stores the index image file in the LIVE textbook database 1a.

  The video file acquired by the video file acquisition unit 4, the thumbnail image file detected by the scene change detection unit 7 that the scene has been changed, and the index image file created by the index image file creation unit 8 are all included. A time stamp is given.

  The overall configuration adjusting unit 9 refers to the time stamp of the thumbnail image included in the index image file, and configures the text file processed by the text file processing unit 6 in units of chapters. That is, the overall configuration adjusting unit 9 determines the time when a certain scene starts and the time when the next scene starts based on the time stamp of the thumbnail image file after the scene change, and the text file included in the range of these times is A text file that constitutes one chapter.

  Furthermore, the whole structure adjustment part 9 produces the text file which shows the name of a chapter. For example, while creating a text file indicating the time at which a chapter starts from a time stamp given to a thumbnail image whose scene has been changed, a text file indicating the first phrase issued in that chapter is extracted. A text file combined with a text file can be used as a text file indicating a chapter name. Note that a text file indicating an important word may be extracted from the text file processed by the text file processing unit 6, and the text file may indicate a chapter name.

  Furthermore, the overall configuration adjusting unit 9 can adjust the number of chapters. For example, in the case of a 90-minute video file, dozens of thumbnail images are detected by the scene change detection unit 7, but the number of chapters is preferably about 10. Therefore, for example, the number of chapters is reduced as follows.

  That is, when the time from a certain scene change to the next scene change is set as the thumbnail image presentation continuation time, about 10 thumbnail images included in the index image are extracted in order from the longest presentation continuation time. Then, based on the thumbnail image extracted in this way, the overall configuration adjusting unit 9 is made to create a text file indicating the name of the chapter.

  This thumbnail image extraction method is based on the tendency that the presentation duration of a scene that is talking together is long and the presentation duration of a scene that is not so important for reference is short. In this way, the number of chapters can be appropriately adjusted by adjusting the number of thumbnail images extracted based on the presentation duration time. Also, when commercial broadcast videos are used in lectures, these scenes are switched every few seconds. Therefore, if the thumbnail image extraction process based on the above-described presentation duration time is performed, it can be excluded that the commercial broadcast video is recognized as one chapter.

  In this way, the overall configuration adjustment unit 9 creates a text file indicating the name of the chapter, and creates a text file indicating the table of contents page of the video file by continuing the text files. The text file indicating this table of contents page is stored in the LIVE textbook database 1a.

  The LIVE textbook reproduction file creation unit 10 generates a text file and a video file based on the time stamp of the text file processed by the text file processing unit 6 and the time stamp of the index image file created by the index image file creation unit 8. A LIVE textbook reproduction file for reproduction in association is created, for example, in the HTML format. In the LIVE textbook reproduction file, the material file distributed during the lecture may be associated with the video file. The LIVE textbook reproduction file created in this way is stored in the LIVE textbook database.

  With the above configuration, the LIVE textbook creation apparatus 1 stores the index image file, the text file, and the LIVE textbook reproduction file in the LIVE textbook database 1a. Note that the text file includes a text file configured in units of clauses by the text file processing unit 6, a text file indicating the name of the chapter created by the overall configuration adjusting unit 9, and a text file indicating a table of contents page. It is.

  The LIVE textbook database 1a may store data files of materials distributed during the lecture. In this case, the LIVE textbook reproduction file creation unit 10 needs to associate the data file of the material with the index image file and the text file.

  That is, a time stamp indicating at which time the data file of the material corresponds to the lecture content is given to the data file, and the time stamp and each time stamp of the index image file and the text file Need to be associated.

[3. Processing flow
Next, a procedure for creating a LIVE textbook data file executed by the LIVE textbook creating apparatus 1 having the above configuration will be described with reference to the flowchart of FIG.

  As shown in FIG. 4, first, a video file is acquired by the video file acquisition unit 4 of the LIVE textbook creation apparatus 1 (step 1, hereinafter, the step is simply referred to as S). Thereafter, voice recognition of the video file acquired in S1 is executed by the voice recognition unit 5 (S2). Further, the text files recognized by the voice recognition unit 5 are grouped into phrase units by the text file processing unit 6 (S3).

  Further, based on the video file acquired by the video file acquisition unit 4, the scene change detection unit 7 detects a scene change (S4). Furthermore, in S5, thumbnail images that are recognized to have changed scenes are arranged in time series by the index image file creation unit 8 to create an index image file.

  Thereafter, the overall configuration adjusting unit 9 configures the text file processed by the text file processing unit 6 in units of chapters based on the index image file created in S5 (S6).

  Further, the overall configuration adjustment unit 9 adjusts the number of chapters based on the above-described presentation duration and creates a table of contents page (S7). Note that adjustment of the number of chapters is not necessarily performed.

  Then, a LIVE textbook reproduction file is created by the LIVE textbook reproduction file creation unit 10 based on the respective time stamps of the text file and the index image file created in the above steps (S8). Thereafter, the text file, the index image file, and the LIVE textbook reproduction file are stored in the LIVE textbook database 1a, thereby completing the creation of the LIVE textbook data file (S9).

[4. (Example of playback screen)
Next, an example of the reproduction screen of the LIVE textbook data file created as described above will be described with reference to FIGS. The reproduction screen described below is displayed on the learning terminal 3 side when the learning terminal 3 obtains a LIVE textbook reproduction file, an index image file, and a text file from the LIVE textbook database 1a via the network. Is. Note that the video file itself may be acquired from the video file database 2a by the learning terminal 3, or may be acquired from the LIVE textbook database 1a.

  FIG. 5 is a diagram showing a screen (hereinafter simply referred to as a home screen) showing an outline of the LIVE textbook data file displayed before reproduction. As shown in FIG. 5, the home screen displays a lecture name and a lecturer name window, a title window, a text window, an index image display window, a chapter selection button, a link to a distribution material file, and the like. The

  It is preferable to display the lecture name in the title window on the home screen. The title window may display the name of the chapter as will be described later.

  In addition, a lecture table of contents is displayed in a text window on the home screen. The table of contents displayed in the text window is displayed based on a text file indicating a table of contents page created by the overall configuration adjusting unit 9 (see FIG. 1) of the LIVE textbook creating apparatus 1.

  In the index image display window, the index image file created by the index image file creation unit 8 is displayed. The index image file is displayed so that a plurality of thumbnail images are arranged in time series as shown in FIG. The index image display window is provided with a scroll bar. Thereby, thumbnail images that are not displayed due to the relationship with the screen size can be displayed by sliding the index image with the scroll bar.

  The chapter selection button is used by the user to input a chapter number so that the user can see a desired chapter image. The chapter selection button may be displayed in a numeric keypad format as shown in FIG. 5, or may be displayed in another format, for example, a scroll bar format.

  The link to the distribution material file is a link for appropriately referring to the material distributed in the lecture. Of course, if there is no material distributed in the lecture, this link display may be omitted. In addition, textbooks related to lectures, reference books, and links to HP addresses may be displayed.

  On the home screen, an appropriate number of representative thumbnail images included in the index image may be extracted, and the thumbnail images may be displayed in a size larger than the size in the index image display window. As a result, the user can confirm the face of the instructor and roughly understand the atmosphere of the lecture.

  FIG. 6 is a view showing a screen (hereinafter referred to as a streaming screen) displayed when a LIVE textbook data file is reproduced. As shown in FIG. 6, the streaming screen displays a video file playback window, a title window, a text window, an index image display window, chapter selection buttons, distribution material links, and the like.

  The video file playback window displays the video file acquired by the video file acquisition unit 4 (see FIG. 1) as a moving image. In the title window on the streaming screen, the name of the chapter corresponding to the video file currently played back in the video file playback window is displayed.

  In the text window on the streaming screen, a text file corresponding to the video file currently reproduced in the video file reproduction window is displayed. In particular, in this text window, the text corresponding to the lecture content currently reproduced in the video file reproduction window is highlighted by, for example, reverse display. As a result, the user can instantly understand what is being spoken in the currently reproduced lecture.

  As a highlighting method, as shown in FIG. 8, text corresponding to the currently played lecture content may be enlarged and displayed in the enlarged display window. You will be able to understand the contents easily. Furthermore, the text corresponding to the currently replayed lecture content may be translated into English, and the lecture content may be displayed in English for international students.

  Further, in the index image display window on the streaming screen, the thumbnail image corresponding to the video file currently played back in the video file playback window is displayed with emphasis by a method such as being surrounded by a thick frame.

  In the home screen of FIG. 5 and the streaming screen of FIG. 6 displayed in this way, a thumbnail image file displayed in the index image display window, a text file indicating the contents of the table of contents or lecture displayed in the text window, and a video Video files displayed in the file playback window are related by time stamps. The association of these files can be grasped by the learning terminal 3 when the learning terminal 3 acquires the LIVE textbook reproduction file from the LIVE textbook database 1a.

  Therefore, by clicking a specific thumbnail image in the index image display window, the video file can be reproduced from the time stamp of the thumbnail image. In addition, by clicking a specific table of contents or clause in the text window, the video file can be played from the time stamp of the clicked text.

  For reference, FIG. 7 shows the structure of a LIVE textbook data file for displaying the home screen of FIG. 5 and the streaming screen of FIG. As shown in FIG. 7, in the LIVE textbook data file, a lecture video file, a thumbnail image file, a text file, and time stamps thereof are structured. It can also be seen that the distribution material file is also included in the LIVE textbook data file.

[5. Application example)
According to the LIVE textbook creation apparatus 1 of the present embodiment, the video file of the lecture content and the text file indicating the content spoken in the lecture are reproduced in synchronization as described above, so that in various scenes Is preferred. Hereinafter, some application examples of this embodiment will be introduced.

  First, the LIVE textbook creation apparatus 1 according to the present embodiment is suitable for a student to review a lecture at a school or research institution such as a university or high school. That is, in order to understand the content of the lecture, it is straightforward and efficient to visually recognize what is spoken in the lecture as characters. In this regard, according to the LIVE textbook creation apparatus 1 of the present embodiment, the lecture content is displayed as a text file on the learning terminal 3 side (see the text window in FIG. 6), so that the content of the lecture is efficiently understood and reviewed. it can.

  Moreover, the LIVE textbook creation apparatus 1 of this embodiment is suitable also from a viewpoint that the teacher in charge of a lecture makes his lecture better. In other words, even if it is not only a student but also a teacher, if the LIVE textbook data file created by the LIVE textbook creation device 1 is played back on the learning terminal 3, it is possible to review your lecture efficiently and improve it to the next lecture Can find a point.

  Furthermore, the user of the LIVE textbook creation apparatus 1 of the present embodiment is not limited to students and teachers. For example, according to the LIVE textbook creation apparatus 1 of the present embodiment, a video file and a text file can be reproduced in synchronization with each other, so that the hearing impaired person can understand the contents of the lecture more easily. Further, the purpose of use of the LIVE textbook creation apparatus 1 of the present embodiment is not limited to review of lectures, and may be used for the purpose of viewing lectures, documentaries, dramas, news programs, English conversation programs, for example. .

[6. Supplement)
The processing procedure of the LIVE textbook creation apparatus 1 of the present embodiment is such that a calculation unit such as a CPU executes a program stored in a storage unit such as a ROM (Read Only Memory) or a RAM, an input unit such as a keyboard, a display, or the like This can be realized by controlling the output means or the communication means such as an interface circuit.

  Therefore, various processes of the LIVE textbook creation apparatus 1 of the present embodiment can be realized simply by a computer having these means reading the recording medium on which the program is recorded and executing the program. In addition, by recording the program on a removable recording medium, the various functions and various processes described above can be realized on an arbitrary computer.

  As the recording medium, a program medium such as a memory (not shown) such as a ROM may be used for processing by the microcomputer, and a program reading device is provided as an external storage device (not shown). It may be a program medium that can be read by inserting a recording medium there.

  In any case, the stored program is preferably configured to be accessed and executed by the microprocessor. Furthermore, it is preferable that the program is read out, and the read program is downloaded to the program storage area of the microcomputer and the program is executed. It is assumed that the download program is stored in the main device in advance.

  The program medium is a recording medium configured to be separable from the main body, such as a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a flexible disk or a hard disk, or a disk such as a CD / MO / MD / DVD. Fixed disk system, card system such as IC card (including memory card), or semiconductor memory such as mask ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash ROM, etc. In particular, there are recording media that carry programs.

  In addition, if the system configuration is capable of connecting to a communication network including the Internet, the recording medium is preferably a recording medium that fluidly carries the program so as to download the program from the communication network.

  Further, when the program is downloaded from the communication network as described above, it is preferable that the download program is stored in the main device in advance or installed from another recording medium.

  As described above, the LIVE textbook creation device 1 of the present embodiment edits a video file in which moving image data and audio data are recorded, and recognizes the audio data and converts it into a text file. And a LIVE textbook reproduction file creation unit 10 for associating the text file and the video file on the basis of time information indicating what time the text file is in the video file.

  According to the above configuration, since the content of the video file can be displayed on the learning terminal 3 by the text file converted by the voice recognition unit 5, the content of the video file can be converted into text and efficiently transmitted to the user. it can.

  Further, since the video file and the text file are associated with each other by the LIVE textbook reproduction file creation unit 10, the video file and the text file can be synchronized and displayed on the learning terminal 3. That is, the video file and the text file can be displayed on the learning terminal 3 while being associated with each other, and the content of the video file can be transmitted to the user more efficiently.

  Furthermore, the LIVE textbook creation apparatus 1 having the above-described configuration includes a scene change detection unit 7 that detects a scene change that is a location where a scene changes in the video file, and items of the contents of the text file based on the time of the scene change. And an overall configuration adjustment unit 9 that creates title names for division for each item.

  According to the above configuration, based on the scene change time detected by the scene change detection unit 7, the overall configuration adjustment unit 9 creates a title name for the text file. Accordingly, by displaying the title name on the learning terminal 3, the user can roughly grasp the contents of the text file. Therefore, the contents of the video file can be transmitted to the user more efficiently.

  Further, when the scene change detecting unit 7 detects a plurality of scene changes from the video file, the overall configuration adjusting unit 9 separates the text file based on the time interval (presentation duration) between the scene changes. It is preferable to adjust the number.

  As a result, an item with a long time interval between scene changes can be selected as an appropriate number of items in the text file, and a title name can be created based on the time of the scene change. Thereby, the number of items can be adjusted as appropriate so that the content of the video file can be transmitted to the user more efficiently.

  Further, the LIVE textbook reproduction file creation unit 10 associates the data of the material distributed in the moving image recorded in the video file with at least one of the text file and the video file.

  According to the above configuration, the LIVE textbook reproduction file creation unit 10 associates at least one of the text file and the video file with the data of the distribution material. This association may be performed based on, for example, a time stamp indicating the time when the material is distributed. As a result, when the text file and the video file are displayed on the external device, the data of the distribution material can be displayed in association with these files.

  Therefore, since the user can check the data of the distribution material when viewing the video file, the user can understand the contents of the video file more efficiently.

  Further, the learning terminal 3 of the present embodiment reproduces a video file in which moving image data and audio data are recorded, and is a text file obtained by converting the audio data, and the text file is the video file. The text file associated based on the time information indicating at which time the content of the video is displayed simultaneously with the video file.

  According to the above configuration, the text file obtained by converting the audio data of the video file and the video file are displayed simultaneously. Therefore, the user can simultaneously obtain the content of the video file documented by the text file and the moving image and sound of the video file, so that the content of the video file can be understood more efficiently.

  Further, the learning terminal 3 having the above configuration highlights a part of the text file corresponding to the content currently spoken in the moving image recorded in the video file.

  According to the above configuration, the user can understand the content currently spoken in the moving image recorded in the video file as a sentence by visually recognizing the highlighted portion of the text file. Can understand the contents of video files.

  Furthermore, it is preferable that the learning terminal 3 having the above configuration reproduces the video file from a portion corresponding to a portion selected by the user in the displayed text file.

  Alternatively, a plurality of title names for categorizing the contents of the text file are displayed, and the video file is reproduced from a location corresponding to the title name selected by the user among the plurality of title names. May be.

  Further, the learning terminal 3 configured as described above displays a plurality of thumbnail images corresponding to the location where the scene changes in the video file, and also includes a location corresponding to the thumbnail image selected by the user among the plurality of thumbnail images. The above video file may be reproduced.

  According to the above configuration, if a user selects a part of a text file, a title name, or a thumbnail image, the user can play a video file from a location corresponding to that part. Furthermore, when a user selects a part of a text file, a title name, or a thumbnail image, the user can check the content of the video file to be played based on the content of the text file. You can select exactly where you want.

  Furthermore, it is preferable that the learning terminal 3 configured as described above displays a link to the data of the material distributed in the moving image recorded in the video file in association with at least one of the text file or the video file. .

  According to the above configuration, since the link to the data of the distribution material is displayed in association with the text file or the video file, the user appropriately selects the link when these files are displayed, You can refer to handouts. Thereby, the user can understand the content of the video file more efficiently.

  Furthermore, the LIVE textbook database 1a of the present embodiment stores a LIVE textbook reproduction file for reproducing a video file in which moving image data and audio data are recorded, and the LIVE textbook reproduction file is the above-mentioned file. The text file into which the audio data is converted is associated on the basis of time information (time stamp) indicating what time the text file is in the video file.

  According to the above configuration, since the text file and the video file are associated with each other by the LIVE textbook reproduction file, the learning terminal 3 distributes the LIVE textbook reproduction file to the outside using the file distribution server. A video file and a text file can be associated and played. Therefore, the user can simultaneously obtain the content of the video file documented by the text file and the moving image and sound of the video file, so that the content of the video file can be understood more efficiently.

  The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope shown in the claims, and the embodiment can be obtained by appropriately combining technical means disclosed in the embodiment. Is also included in the technical scope of the present invention.

  According to the present invention, the contents of the video file can be documented and transmitted to the user by the text file, and the video file and the text file can be displayed in a state of being associated with each other. It can be communicated to the user more efficiently. Therefore, it is suitable for students and teachers to review lectures recorded in video files.

It is a block diagram which shows the structure of the LIVE textbook preparation apparatus which concerns on one Embodiment of the video editing apparatus of this invention. It is a figure which shows the structure of the system using the LIVE textbook preparation apparatus of FIG. It is a figure for demonstrating the scene change detection performed by the LIVE textbook preparation apparatus of FIG. It is a flowchart which shows the flow of the process which concerns on one Embodiment of the video editing method of this invention. It is a figure which shows an example of the screen displayed in the learning terminal of FIG. It is a figure which shows an example of the screen displayed in the learning terminal of FIG. It is a figure which shows the structure of the LIVE textbook data file produced by the LIVE textbook production apparatus of FIG. It is a figure which shows an example of the screen displayed in the learning terminal of FIG.

Explanation of symbols

1 LIVE textbook creation device (video editing device)
1a LIVE textbook database (file database)
3 Learning terminal (video playback device)
5 Voice recognition unit (voice recognition means)
7 Scene change detection unit (scene change detection means)
8 Index image file creation unit 9 Overall configuration adjustment unit (title creation means, item number adjustment means)
10 LIVE textbook playback file creation unit (file association means)

Claims (19)

A video editing device for editing a video file recording moving image data and audio data,
Voice recognition means for recognizing the voice data and converting it into a text file;
A video editing apparatus, comprising: file association means for associating the text file with the video file based on time information indicating a time at which the text file is in the video file. Scene change detection means for detecting a scene change that is a location where a scene changes in the video file;
2. The video editing according to claim 1, further comprising: a title creating means for creating a title name for dividing the contents of the text file into items based on the time of the scene change. apparatus. When a plurality of scene changes are detected from the video file by the scene change detection means,
3. The video editing apparatus according to claim 2, further comprising item number adjusting means for adjusting the number of items for dividing the text file based on a time interval between the scene changes.   4. The file associating means further associates data of a material distributed in a moving image recorded in the video file with at least one of the text file and the video file. The video editing apparatus according to any one of the above.   5. The video editing apparatus according to claim 4, wherein the file association unit associates the data of the material based on time information indicating a time when the material is distributed. A video playback device for playing back a video file in which moving image data and audio data are recorded,
A text file in which the audio data is converted, and the text file associated with the time information indicating at what time in the video file the text file is displayed simultaneously with the video file. A featured video playback device.   7. The video reproduction apparatus according to claim 6, wherein a part of the text file corresponding to the content currently spoken in the moving image recorded in the video file is highlighted.   8. The video reproduction apparatus according to claim 6, wherein the video file is reproduced from a portion corresponding to a portion selected by the user in the displayed text file. In addition to displaying multiple title names to categorize the contents of the text file,
9. The video reproduction apparatus according to claim 6, wherein the video file is reproduced from a portion corresponding to the title name selected by the user among the plurality of title names. In addition to displaying a plurality of thumbnail images corresponding to the location where the scene changes in the video file,
10. The video reproduction apparatus according to claim 6, wherein the video file is reproduced from a portion corresponding to the thumbnail image selected by the user among the plurality of thumbnail images.   11. The link to data of material distributed in a moving image recorded in the video file is displayed in association with at least one of the text file or the video file. The video reproduction apparatus according to item 1. A file database in which a text file in which audio data of a video file in which moving image data and audio data are recorded is converted, and a video file playback file for playing back the video file is stored.
The video file playback file associates the text file with the video file based on time information indicating at which time the text file is in the video file.
A file database, wherein an external device acquires the video file playback file so that the text file is associated with the video file and played on the external device.   A file delivery server for delivering the video file playback file stored in the file database according to claim 12 to an external device. A video editing method for editing a video file recording moving image data and audio data,
A first step of recognizing the voice data and converting it into a text file;
A video editing method comprising: a second step of associating the text file with the video file based on time information indicating at what time the text file is in the video file.   A video editing program for causing a computer to execute each step of the video editing method according to claim 14.   A computer-readable recording medium on which the video editing program according to claim 15 is recorded. A video playback method for playing back a video file recording moving image data and audio data,
A text file in which the audio data is converted, and the text file associated with the time information indicating at what time in the video file the text file is displayed simultaneously with the video file. Characteristic video playback method.   A video playback program for causing a computer to execute the video playback method according to claim 17.   A computer-readable recording medium on which the video reproduction program according to claim 18 is recorded.

Images