JP2012053855A - Content browsing device, content display method and content display program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a content browsing device and the like with which important contexts and points thereof can be easily understood in a review of information having time information such as a video content.SOLUTION: A content browsing device in the present invention has a video content and audio and still pictures linked on a time axis, stores video scenes extracted from the video content, associates keywords extracted from the audio and the still pictures with importance degrees obtained by analyzing the respective keywords to store them, reads out the predetermined number of keywords of top importance degrees, also reads out audio or still pictures that are extraction sources of the read-out keywords and video scenes linked on the same time axis as display data, displays a group of a video scene, audio and/or a still picture, and a keyword linked on the same time axis among the read-out display data, and if there are plural groups, time-sequentially arranges and displays each group.

Description

  The present invention relates to a content browsing device, a content display method, and a content display program.

  Conventionally, the minutes of conferences (meetings) have been mainly recorded by, for example, document minutes. In this method, the clerk attends the meeting and creates the minutes. In some cases, the clerk creates a minutes of the statements made during the meeting (especially important meetings). In most cases, the clerk summarizes the contents while selecting the key points while understanding the contents of the meeting. We will make the minutes. When reviewing the contents of the meeting at a later date, the summarized document minutes contain important matters, so that the contents of the meeting can be easily grasped in a short time.

  In recent years, with the development of digital technology, the contents of meetings (meetings) can be easily recorded as video contents. If the content of the conference is recorded on the video camera, the pattern of the conference can be reproduced as it is, but here the video content recorded on the video camera is continuous information that flows as time passes. Therefore, unlike the document summaries described above, if you want to look back on the content (summary) of the meeting in a short time and efficiently, you need to play it back while repeating fast forward and rewind, so look for important points (scenes) It just takes a long time.

  Therefore, for the purpose of looking back on the video content shot at the conference, the video content is tagged for search, and the importance is calculated to locate the important points (scenes) of the video on the time axis. The technology to do is known. For example, in Patent Document 1, in a conference system using multimedia information such as voice and images, data in a conference is provided in order to provide a device and a user interface that can easily edit important items in the conference in a short time. Video input means, still picture input means, voice input means, pen input means, pointing means, and key input means, and the input data is stored in the data storage means, and the data time is stored. Analyzing the relationship to create a search file, storing the created search file, reading the corresponding data from the data storage means based on the reference result of the stored search file, and displaying and editing the data By doing so, a configuration for supporting the creation of the minutes is disclosed.

  However, a technique for looking back on a conventional video (for example, Patent Document 1) displays a stream of video content along a time axis, and displays a video with high context importance by color coding or the like. Therefore, in the case of a conference video that lasts for a long time, it takes time to search for an important scene. Even if only the color-coded video is displayed and the position of the important scene can be specified, it is difficult to grasp the contents of the main point without actually replaying the video scene and checking the content. is there. That is, the conventional technique still has room for improvement in the display method of content from the viewpoint of user friendliness when looking back on information with time information such as video content.

  The present invention has been made in view of the above points, and in reviewing information with time information such as video content, a content browsing device, a content display method, and a content browsing device capable of easily grasping important contexts and the contents of the main points are provided. An object is to provide a content display program.

  In order to achieve the above object, a content browsing device according to the present invention is a content browsing device that displays content on a display means, and is a video content and a voice, a still image linked to the video content on a time axis. Data storage means having at least one of images and a video scene extracted from the video content are stored, and a keyword extracted from at least one of the audio and the still image is analyzed for each keyword. Analysis data storage means in which the importance level is stored in association with each other, and the voice having the importance level extracted from the analysis data storage means and the voice from which the read keyword was extracted Or a video image linked to the still image and the same audio or still image on the same time axis. As a display data, and among the display data read by the read control means, a video scene, an audio and / or still image linked on the same time axis, and a group of keywords Display control means for arranging the plurality of groups and arranging them in time series for each group and displaying them on the display means.

  In order to achieve the above object, in the content browsing device, a second analysis data storage in which a video scene extracted from the video content and an importance level analyzed for each video scene are stored in association with each other And a voice and still image linked on the same time axis as the read video scene are read out from the second analysis data storage means Second reading control means for reading as two display data, and of the display data read by the second reading control means, a video scene linked on the same time axis, and an audio and / or still image A second display control means for displaying a group, and when there are a plurality of the groups, the display means arranges the data in time series and displays on the display means. Characterized in that it has a.

  In order to achieve the above object, in the content browsing apparatus, the display control means emphasizes and displays the voice or still image from which the extraction is performed, and the second display control means A predetermined number of video scenes with higher degrees are highlighted and displayed.

  In addition, what applied the arbitrary combination of the component of this invention, expression, or a component to a method, an apparatus, a system, a computer program, a recording medium, etc. is also effective as an aspect of this invention.

  According to the present invention, it is possible to provide a content browsing apparatus, a content display method, and a content display program capable of easily grasping an important context and the contents of the main points in looking back information with time information such as video content.

It is a figure which shows the flow of a series of processes of the conference recording system which concerns on embodiment. The example of a display screen concerning this embodiment is shown. It is a hardware block diagram which shows the main structures of one Embodiment of the conference recording system 1. It is a functional block diagram which shows the main functions of one Embodiment of the conference recording system 1 which concerns on this embodiment. It is a figure which shows the DB structural example of a data storage means and an analysis data storage means. The structural example of video content data DB510 is shown. The structural example of audio | voice data DB520 is shown. The structural example of whiteboard image DB530 is shown. The structural example of slide image DB540 is shown. The structural example of video scene DB511 cut out from video content data is shown. The structural example of the scene change DB 512 extracted from the video scene is shown. The structural example of speech recognition data DB521 extracted from speech data is shown. The structural example of speaker DB522 analyzed from speech recognition data is shown. The structural example of keyword DB550 analyzed from speech recognition data, a whiteboard image, and a slide image is shown. It is a flowchart explaining the reading and display processing of data. An example of the read analysis data is shown.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

[System configuration]
(Overview)
The present invention provides a content browsing device that makes it possible to easily grasp important contexts and the contents of the main points in looking back information with time information such as video content. Since it is necessary to acquire and analyze, in the following embodiments, the present invention is applied to a conference recording system (referred to as such for convenience) having a series of processing functions including acquisition, analysis, display (browsing), and playback of video content. An example where is applied.

  FIG. 1 is a diagram illustrating a flow of a series of processes of the conference recording system according to the embodiment. The conference recording system according to the embodiment includes a series of processing functions including acquisition, analysis, display (browsing), and playback of video content, and the processing proceeds in a flow of steps of S100 to S400 as shown in the figure. .

  S100: Data input step. In this embodiment, the data refers to conference data related to a conference, and specifically includes video content data, audio (may be included in the video content data), an image, a mouse, an input instruction data from a keyboard, and the like. As the conference proceeds, the conference recording system continues to input these conference data via each input means. When the conference is over, the data entry is also over.

  S200: Data analysis step. The data input in S100 is stored in storage means (storage means), and the conference recording system performs data analysis on this data. Data analysis includes, for example, dividing a video stream, calculating importance for each scene, extracting text from speech to identify the speaker, analyzing whiteboards and slides to extract keywords, etc. The importance is calculated. The data analysis result is stored in storage means (storage means).

  S300: Data reading and display step. This step is a scene in which the user looks back on the video content of the meeting. For example, when the user designates one meeting and performs a predetermined operation from the display means of the meeting recording system, the display means is displayed from the meeting contents. As many important contexts as the number of frames that can be displayed are displayed. Further, not only the content regarded as important, but also other contexts at the same time on the time axis are displayed so as to correspond.

  FIG. 2 shows an example of a display screen according to the present embodiment. In this conference, video content data, audio data, whiteboard images, and slide (electronic slide) images were input as input data. As a result of data analysis, the context of video scenes, speakers, whiteboards, slides, and keywords It is decomposed and analyzed. Among these, contexts with high importance (those with a strong frame display) are extracted, and the contexts with high importance and other contexts at the same time on the time axis are displayed in parallel. In this meeting, four highly important contexts are extracted and displayed, and the user recognizes that these four particularly important scenes (video scenes) in this meeting, and more specifically from the keywords. You can also recognize what the content was.

  S400: This is a scene selection and video playback step. Since some important contexts are displayed on the display means of the conference recording system, the user can select and reproduce a video scene to be reproduced. During the conference, the video content is actually reproduced from the point of the selected scene, and the user can look back on the conference locally. For example, when any context is selected and the “play” button is pressed, the screen is switched to the playback screen, and the video content is played from this time.

(hardware)
Here, the hardware configuration of the conference recording system (conference recording device) 1 will be briefly described. FIG. 3 is a hardware configuration diagram showing the main configuration of one embodiment of the conference recording system 1. The conference recording system 1 includes, as main components, a CPU 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an auxiliary storage device 104, a storage medium reading device 105, an input device 106, a display device 107, and a communication device. 108 and an interface device 109.

  The CPU 101 is composed of a microprocessor and its peripheral circuits, and is a circuit that controls the entire apparatus. The ROM 102 is a memory that stores a predetermined control program (software component) executed by the CPU 101. The RAM 103 executes various control operations by the CPU 101 executing a predetermined control program (software component) stored in the ROM 102. This is a memory used as a work area (work area) when performing.

  The auxiliary storage device 104 is a device that stores various information including a general-purpose OS (Operating System) and various programs, and an HDD (Hard Disk Drive) that is a nonvolatile storage device is used.

  The input device 106 is a device for the user to perform various input operations. The input device 106 includes a mouse, a keyboard, a touch panel switch provided so as to be superimposed on the display screen of the display device 107, and the like. The display device 107 is a device that displays various data on a display screen. For example, it is composed of LCD (Liquid Crystal Display), CRT (Cathode Ray Tube) and the like.

  The communication device 108 is a device that communicates with other devices via a network. Supports communication according to various network forms including wired and wireless networks.

  The interface device 109 is an interface for connecting to an external peripheral device. In the present embodiment, a video camera (for moving image), a digital camera (for still image), a microphone and the like are mainly connected to a data capture device, and data is input via the interface device 109.

(Functional configuration)
Next, the main functional configuration of the conference recording system 1 according to the present embodiment will be briefly described. FIG. 4 is a functional block diagram showing the main functions of one embodiment of the conference recording system 1 according to this embodiment. As shown in the figure, the conference recording system 1 includes, as main functions, a data input unit 201, a timing unit 202, a data registration unit 203, a data storage unit 204, a data analysis unit 205, an analysis data storage unit 206, and a display control unit 207. , Display data read control means 208, operation means 209, display means 210, and reproduction means 211.

  The data input unit 201 is a unit for inputting data. For example, the data that is realized and input by the above-described interface device 109 includes, for example, video content data of a video camera, audio data from a microphone, screen capture of a whiteboard by the camera (still image), and slide material data (still image). Etc.

  The time measuring means 202 measures the time in seconds or the like and returns the time at that time in response to a request from the data registration means 203.

  The data registration unit 203 stores the data input from the data input unit 201 in the data storage unit 204. If the data is not yet registered in the data storage unit 204, the time is inquired to the time measuring unit 202 and the time information is registered in the data storage unit 204 together. In the case of streaming information that has already been registered, such as video and audio, the time is not inquired of the time measuring means 202. Also in the case of a still image, the time is inquired to the time measuring means 202 and the time information is registered in the data storage means 204 together.

  The data storage unit 204 stores data requested for registration from the data registration unit 203. The time information is also stored. The stored data is retrieved in response to a request from the data analysis unit 205 or the display data read control unit 208. The data storage unit 204 is realized by the auxiliary storage device 104 described above, and may be an HDD (hard disk) or the like. The data storage unit 204 manages stored data by a DB (database) (specific examples will be described later).

  The data analysis unit 205 retrieves data stored in the data storage unit 204, performs a specific process, and retrieves metadata. The importance is calculated based on the metadata. The data analysis unit 205 includes, for example, a scene change extraction unit, a speaker identification unit, a key frame extraction unit, a slide extraction unit, a voice recognition unit, and a keyword extraction unit.

  The analysis data storage unit 206 stores the data analyzed by the data analysis unit 205. The analysis data storage means 206 takes out data in response to a request from the display data read control means 208. The analysis data storage means 206 manages the stored data by a DB (a specific example will be described later).

  The display control unit 207 issues a read request to the display data read control unit 208 based on the information operated by the operation unit 210. A display request is issued to the display means 209 according to the read information.

  The display data read control unit 208 requests the analysis data storage unit 206 to take out analysis data (analysis result) in response to a request from the display control unit 209. Further, the analysis data is compared and the data storage means 204 is requested to retrieve necessary data. The extracted data is transferred to the display control means 207 as display data.

  The display unit 209 displays the display requested by the display control unit 207. The display unit 209 is realized by the display device 107 described above, and may be a display or the like.

  The operation means 210 provides means for causing the user to perform an operation. The operation unit 210 is realized by the input device 106 described above, and may be a mouse, a keyboard, a pen input device, or the like.

  The reproduction unit 211 reproduces the video content stored in the data storage unit 204. When the target video content and playback time are specified by the user, the video content is played back. What is necessary is just to implement | achieve with a media reproduction | regeneration player etc.

  These functions are actually realized by a computer by a program executed by the CPU 101 of the apparatus.

[Information processing]
As described above, the conference recording system 1 according to the embodiment includes a series of processing functions including acquisition, analysis, display (browsing), and playback of video content, and the flow of steps S100 to 400 (FIG. 1) is largely performed. The process proceeds.

  FIG. 5 is a diagram illustrating a DB configuration example of the data storage unit and the analysis data storage unit. The following description will be made with reference to the same figure.

(Data input: S100)
In the conference recording system 1 according to the embodiment, peripheral devices for data input are connected, and data input in the present embodiment is video content data, audio data, whiteboard image (still image), slide image of a slide. (Still image).

  For example, a video camera is installed in the conference room so as to look over the entire conference room. Alternatively, the photographer can handle the video camera at any time and shoot freely. The video acquired from the video camera is streaming data including time information. Further, for example, a whiteboard is installed in the conference room, and a handwritten screen image of the whiteboard is captured at regular intervals or by a user operation. Also, for example, in the conference room, the electronic material data of the user PC is projected on the screen on the wall side, or the electronic material data is shared between the user PCs, and this slide image is captured.

  When the conference is started, the data starts to be input, and the input of the data is ended together with the end of the conference. During the meeting, the data is stored in the data storage unit 204 by the data registration unit 203. The data registration unit 203 inquires the time counting unit 202 about the time, and registers the time information together with the data storage unit 204.

  FIG. 6 shows a configuration example of the video content data DB 510. The data storage unit 204 manages data in the DB shown in the figure when storing the video content data. The video content data DB 510 includes an ID, a file location, a start time, and the like. The ID is an ID for uniquely specifying the video on the DB. The file location indicates the location where the video is stored. The storage location may be specified by a description such as a file path. For example, “0001.avi” is video content data (substance). The start time is given by the time measuring means 202 when the storage of the video is started, and is recorded, for example, in the UTC format. According to this example, since video content data having three IDs is stored (registered), it can be seen that videos for three conferences are stored in the DB.

  FIG. 7 shows a configuration example of the audio data DB 520. The data storage unit 204 manages data in the DB shown in the figure when storing the audio data. The audio data DB 520 includes an ID, a file location, a start time, and the like. The ID is an ID for uniquely specifying the voice on the DB. The file location indicates the location where the sound is stored. For example, “0001.wav” is audio data (substance). The start time is given by the time measuring unit 202 when the voice storage is started. According to this example, since voice data having three IDs is stored (registered), it can be seen that voices for three conferences are stored in the DB.

  FIG. 8 shows a configuration example of the whiteboard image DB 530. The data storage means 204 manages data in the DB shown in the figure when storing the whiteboard image. The whiteboard image DB 530 includes an ID, a file location, a start time, and the like. The ID is an ID for uniquely specifying the whiteboard image on the DB. The file location indicates the location where the whiteboard image is stored. For example, “w0001.jpg” is whiteboard image data (substance). The start time is given by the time measuring unit 202 when the whiteboard image storage is started. . The whiteboard image in this example is particularly suitable for the meeting corresponding to the ID “m1” in FIG. 6 and the ID “a1” in FIG. 7 if attention is paid to time information (2010-04-05 13: 10: 33-). It turns out that it is the captured whiteboard image data.

  FIG. 9 shows a configuration example of the slide image DB 540. The data storage unit 204 manages data in the DB shown in the figure when storing slide images. The slide image DB 540 includes an ID, a file location, a start time, and the like. The ID is an ID for uniquely identifying the slide image on the DB. The file location indicates the location where the slide image is stored. For example, “s0001.jpg” is slide image data (substance). The start time is given by the time measuring unit 202 when the slide image storage is started. The slide image in this example is captured in the meeting corresponding to the ID “m1” in FIG. 6 and the ID “a1” in FIG. 7, particularly when attention is paid to time information (2010-04-05 13: 10: 33˜). It can be seen that this is the slide image data.

(Data analysis: S200)
The data input as described above is stored in the data storage means 204, and the conference recording system 1 performs data analysis on this data. Data analysis includes, for example, dividing a video stream, calculating importance for each scene, extracting text from speech to identify the speaker, analyzing whiteboards and slides to extract keywords, etc. The importance is calculated. The data analysis result is stored in the analysis data storage unit 206. This will be described below with reference to FIG.

  FIG. 10 shows a configuration example of the video scene DB 511 cut out from the video content data. The data analysis unit 205 extracts an image for each video scene from the streaming of the video content data, and stores this in the video scene DB 511. Therefore, the video ID in FIG. 10 corresponds to the ID in FIG. For example, ID “si1” in FIG. 10 is extracted from “0001.avi” of ID “m1” in video content data 510 in FIG. The time zone when the video scene image is cut out is stored as time information.

  FIG. 11 shows a configuration example of the scene change DB 512 extracted from the video scene. The data analysis unit 205 extracts a video scene that is determined to be particularly important from the video scene, and stores it in the scene change DB 512. Therefore, the video scene ID in FIG. 11 corresponds to the ID in FIG. For example, the ID “sc1” in FIG. 11 is extracted as a result of determining that the ID “si1” in the video scene DB 511 in FIG. 10 is important. Then, the data analysis unit 205 calculates the importance for the video scene determined to indicate each scene change, and also stores it in the DB. The importance indicates the result of calculating how much information has changed with respect to the scene change as the importance, and a specific method for calculating the importance of the scene change is, for example, Japanese Patent No. 441254 of the present applicant, etc. Can be referred to.

  FIG. 12 shows a configuration example of the voice recognition data DB 521 extracted from the voice data. The data analysis unit 205 performs voice recognition processing from the streaming of the voice data, converts it into text, creates voice recognition data by dividing it into predetermined phrases, and stores this in the voice recognition data DB 521. Therefore, the voice ID in FIG. 12 corresponds to the ID in FIG. For example, ID “t1” in FIG. 12 is extracted from “0001.wav” of ID “a1” in the audio data DB 520 in FIG. The time information is calculated from the voice start time and elapsed time, and indicates the start time of the voice recognition data in the original voice data.

  FIG. 13 shows a configuration example of the speaker DB 522 analyzed from the speech recognition data. The data analysis unit 205 performs speech recognition processing from the speech data or speech recognition data, identifies a speaker (speaker ID), and stores this in the speaker DB 522. Therefore, the voice ID in FIG. 13 corresponds to the ID in FIG. For example, the ID “a1” in FIG. 13 is a speaker identified from “0001.wav” of the ID “a1” in the voice data DB 520 in FIG. The time information is calculated from the voice start time and elapsed time, and indicates the start time of the voice recognition data in the original voice data. As a method for specifying a speaker, for example, a presumed speaker name, a speaker ID, a speaker photo image, and each voiceprint are registered in association with each other, and collated with voice data or voice recognition data. Thus, the speaker can be specified.

  FIG. 14 shows a configuration example of the keyword DB 550 analyzed from the voice recognition data, the whiteboard image, and the slide image. The keyword DB 550 includes ID, content, keyword, importance, time information, and the like. The ID is an ID for uniquely specifying a keyword on the DB. The content indicates an extraction source of where the keyword is acquired from, and indicates IDs such as a speech recognition data DB, a whiteboard image DB, and a slide image DB. The keyword stores voice recognition data such as input data or analysis data analyzed from the input data, a keyword or a key phrase extracted from a whiteboard image or a slide image. The importance is a value obtained by calculating the keyword importance by calculation.

  The data analysis unit 205 performs keyword extraction processing from the voice recognition data, the whiteboard image, and the slide image, calculates the importance of the extracted keyword, and stores it in the DB of the analysis data storage unit 206. As a method for extracting keywords, for example, text recognition processing (OCR) is performed from whiteboard images and slide images to extract text (stored in whiteboard images OCRDB 531 and slide image images OCRDB 541). The speech data has already been taken out as text as speech recognition data (stored in the speech recognition data DB 521). These texts are decomposed into morphemes, and TF (Term Frequency: Word Frequency) -IDF for each morpheme. (Inverse Document Frequency) An index representing importance such as a value can be calculated, and a morpheme whose index (importance) is higher than a certain level can be used as a keyword.

  For example, in the figure, the keyword ID “k1” indicates that the keyword “System” is extracted from the content “sl1” (slide image), and the importance is “0.4”. The keyword ID “k2” indicates that the keyword “Architecture” is extracted from the content “w1” (whiteboard image) and the importance is “0.2”. Similarly, the keyword ID “k3” indicates that the keyword “system configuration” is extracted from the content “t1” (speech recognition data) and its importance is “0.6”. Note that “t1” (FIG. 12) of the speech recognition data has the content “The first agenda is about the system configuration.” The keyword “system configuration” indicating the agenda of the conference is relatively high. The importance “0.6” is calculated.

(Data reading and display: S300)
This step is a scene where the user browses the video content of the conference. For example, the user selects one conference from the recorded conference list from the display means of the conference recording system 1, and the important conference is selected for the selected conference. Perform an operation to display the context. In response, the conference recording system 1 extracts and displays an important context for the selected conference based on the analysis data of the analysis data storage unit 206.

  FIG. 15 is a flowchart for explaining data reading and display processing. In the conference recording system 1 according to the present embodiment, one conference is selected by the user, and an operation for displaying an important scene is received to start the process of the flowchart. At the time of operation, the user designates whether the context whose importance is to be judged is a video scene, a keyword, or both a video scene and a keyword. When displaying important scenes, the number of frames (number of scenes) to be displayed at one time is designated. Here, it is assumed that “both video scene and keyword” and “4 frames” are designated.

  S301: First, the display data read control unit 208 arranges keywords corresponding to the conference from the keyword DB 550 in order of importance based on the conference selected by the user. The keyword DB 550 may already be sorted in order of importance by the analysis data storage unit 206.

  S302: Similarly, the display data read control unit 208 arranges the video scenes corresponding to the conference from the video scene DB 511 in order of importance based on the conference selected by the user. Note that the video scene DB 511 may already be sorted in the order of importance by the analysis data storage unit 206.

  S303: The display data reading control means 208 acquires (reads) a predetermined upper number of keywords and video scenes as display data. The predetermined upper number is a value determined by the designated number of frames. Here, since the designated number of frames is “4 frames”, the predetermined upper number is 4, so the keywords and video scenes up to the upper fourth are acquired. As an acquisition method, when the content whose importance is to be determined is “keyword” (only), keywords having the highest importance are acquired up to the fourth highest. If the content whose importance is to be judged is “video scene” (only), the video scenes with the fourth highest importance are acquired. In addition, when the content whose importance is determined is “both video scenes and keywords”, two video scenes and keywords may be acquired up to two, that is, the second highest importance, If one of them is emphasized, priority may be given to obtain either by weighting. Alternatively, if the importance level of the video scene and the keyword is normalized (standardized), the video scene and / or the keyword having the fourth highest importance level is acquired from both the video scene and the keyword. You may do it.

  S304: Further, the display data reading control unit 208 acquires the acquired keyword and the context at the same time as the video scene as display data. Specifically, when a keyword is acquired based on importance, this keyword is extracted from one of speech recognition data, a whiteboard image, and a slide image, and the time information of this extraction source context is specified. To obtain another context at the same time as the specified time information.

  Specifically, it is assumed that, for example, in S303, the keyword “system configuration” is acquired because the keyword ID “k3” having the importance “0.6” is included in the predetermined upper number. Referring to the keyword DB 550, the extraction source of the keyword “system configuration” is the content “t1”, that is, the ID “t1” (FIG. 12) of the speech recognition data is the extraction source. In the voice recognition data ID “t1”, the time information is “2010-04-05 13:10:32”. Therefore, the voice recognition data ID “t1” which is the context at the same time as the keyword is acquired. As another context at the same time, a video scene of “2010-04-05 13:10:32”, a whiteboard image of “2010-04-05 13:10:32”, “2010-04-05 13: 10:32 ”slide images and speakers as of“ 2010-04-05 13:10:32 ”are acquired. The video scene can be read from the video scene DB 511, the white board image from the white board image DB 530 (or the white board image OCR DB 531), the slide image from the slide image DB 540 (or the slide image OCR DB 541), and the speaker from the speaker DB 522. . For the slide image DB 540 of FIG. 9, if there is no slide image at the same time, the most recent slide image is obtained instead (since the slide acquires an image every time it is switched).

  FIG. 16 shows an example of read analysis data. Through S303 and S304 described above, five contexts (elements) are extracted and read out as shown in the figure. Here, these five contexts are contexts that occur at the same time during the conference, and can be said to be a simultaneous context group. Here, as described above, the designated number of frames for displaying the important scene display at a time is “4 frames”. Therefore, in this manner, four simultaneous context groups are extracted and read out as display data. .

  S305: The display control unit 207 causes the display unit 209 to display the display data read by the display data read control unit 208. Reference is again made to FIG. As a display method, for example, as shown in FIG. 2, the same-time context groups are arranged vertically in the order of context types. In the case of the same time context group of “13:10:32” (FIG. 16), each context element in the group is “video scene”, “speaker”, “whiteboard”, “slide”, “keyword” from the top. Align vertically. In the same manner, the other three simultaneous time context groups are also aligned vertically. On the horizontal axis, the same-time context groups are arranged in chronological order. Since these contexts are extracted based on the importance level, the context that is the basis thereof is highlighted.

(Scene selection and video playback: S400)
In this step, since some important contexts are displayed on the display means of the conference recording system 1, the user can select and reproduce a video scene to be reproduced. Referring to FIG. 2 again, the user selects a video scene (or content in the same group) to be reproduced and presses a “play” button. The playback unit 211 acquires time information of the selected video scene, specifies a file corresponding to the video scene (FIG. 6), and starts playback from the time when the file is acquired. In the example so far, the video file “0001.avi” of this conference is reproduced from the time stream “13:10:32” (time information) including video and audio.

[Summary]
As described above, according to the conference recording system 1 according to the present embodiment, in reviewing video content data, only video content data and other input data having high importance are displayed in a list along the time axis. Further, among the displayed contexts, those with high importance are displayed by highlighting (including color coding) or the like so as to improve the visibility of the user. In addition, other contexts parallel to the same time in the meeting scene are also displayed simultaneously. As a result, the user can recall the conference scene from not only the video but also a plurality of other elements. Of these, by displaying the analyzed keyword, the user can grasp at a glance the key content at the time of the video scene as a keyword along with the video and context (speaker, whiteboard, slide) with high importance. I was able to do it.

  In other words, according to the above-described embodiment, it is possible to provide a content browsing device and the like that can easily grasp important contexts and the contents of the main points when looking back on time information such as video content.

  Although the present invention has been described based on each embodiment, the present invention is not limited to the requirements shown here, such as combinations with other elements listed in the above embodiments. With respect to these points, the present invention can be changed within a range that does not detract from the gist of the present invention, and can be appropriately determined according to the application form. Moreover, what applied the component, expression, or arbitrary combinations of the component of this invention to a method, an apparatus, a system, a computer program, a recording medium, etc. is also effective as an aspect of this invention.

1 Conference support system 101 CPU
102 ROM
103 RAM
104 Auxiliary storage device 105 Storage medium reading device 106 Input device 107 Display device 108 Communication device 109 Interface device 201 Data input means 202 Timekeeping means 203 Data registration means 204 Data storage means 205 Data analysis means 206 Analysis data storage means 207 Display control means 208 Display data read control means 209 Operation means 210 Display means 211 Playback means 510 Video content data DB
511 Video scene DB
512 Scene change DB
520 Audio data DB
521 Speech recognition data DB
522 Speaker DB
530 Whiteboard Image DB
531 Whiteboard Image OCRDB
540 slide image DB
541 Slide OCR image DB
550 Keyword DB

Japanese Patent No. 3185505

Claims (7)

A content browsing device for displaying content on a display means,
Data storage means having at least one of video content, audio linked to the video content on a time axis, and a still image;
Analysis data in which a video scene extracted from the video content is stored, and a keyword extracted from at least one of the audio and the still image is associated with an importance level analyzed for each keyword Storage means;
The keyword with the highest degree of importance is read from the analysis data storage means, and the voice or still image from which the read keyword is extracted and on the same time axis as the voice or still image Read control means for reading out the linked video scene as display data;
Displaying video scenes, audio and / or still images, and keyword groups linked on the same time axis among the display data read by the read control means, and when there are a plurality of the groups Display control means arranged in time series for each group and displayed on the display means;
A content browsing apparatus comprising: A second analysis data storage means in which a video scene extracted from the video content and an importance level analyzed for each video scene are stored in association with each other;
A predetermined number of video scenes with higher importance are read from the second analysis data storage means, and audio and still images linked on the same time axis as the read video scene are displayed as second display data. Second read control means for reading as:
When the display data read by the second read control means displays video scenes and audio and / or still image groups linked on the same time axis, and there are a plurality of the groups. Includes a second display control means arranged in time series for each group and displayed on the display means,
The content browsing apparatus according to claim 1, further comprising: The display control means emphasizes and displays the voice or still image that is the extraction source,
The second display control means emphasizes and displays the predetermined number of video scenes with the highest importance;
The content browsing apparatus according to claim 2. A content display method in a content browsing apparatus for displaying content on a display means,
The content browsing device
Data storage means having at least one of video content, audio linked to the video content on a time axis, and a still image;
Analysis data in which a video scene extracted from the video content is stored, and a keyword extracted from at least one of the audio and the still image is associated with an importance level analyzed for each keyword Storage means;
With
The keyword with the highest degree of importance is read from the analysis data storage means, and the voice or still image from which the read keyword is extracted and on the same time axis as the voice or still image A readout control procedure for reading out linked video scenes as display data;
Displaying video scenes, audio and / or still images, and keyword groups linked on the same time axis among the display data read by the read control procedure, and when there are a plurality of the groups Is arranged in time series for each group, and a display control procedure for displaying on the display means,
A content display method characterized by comprising: The content browsing device
A second analysis data storage means in which a video scene extracted from the video content and an importance level analyzed for each video scene are stored in association with each other;
With
A predetermined number of video scenes with higher importance are read from the second analysis data storage means, and audio and still images linked on the same time axis as the read video scene are displayed as second display data. A second reading control procedure to read as
In the display data read out by the second readout control procedure, a video scene and a group of audio and / or still images linked on the same time axis are displayed and there are a plurality of the groups. Includes a second display control procedure arranged in time series for each group and displayed on the display means;
5. The content display method according to claim 4, further comprising: The display control procedure emphasizes and displays the voice or still image that is the extraction source,
The second display control procedure is to emphasize and display the predetermined number of video scenes with higher importance.
The content display method according to claim 5.   A content display program for causing a computer to execute the content display method according to claim 4.

Images