JPH1145278A - Document processor, storage medium storing document processing program and document processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prepare a summary capable of accurately recognizing contents for respective topics by judging whether or not the plural topics are included in one document. SOLUTION: The document A is divided into plural sub documents (S14), document vectors (b) for the respective sub documents are obtained (S15) and the similarity degree (s) of the adjacent sub documents with each other is obtained from a cosine value between the adjacent document vectors (b) (S16). A break Xn predicted as the changing point of the topic is temporarily judged from the value of the similarity degree (s) (S17), a document vector B and the similarity degree S for a sub document group for the respective breaks are obtained (S19 and S20) and the changing point of the topic is finally judged from the similarity degree S (S21). Then, by preparing sub summaries for respective sub document groups finally judged as the changing point of the topic (S22) and synthesizing the sub summaries, the final summary of a summary object document A is attained.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a document processing apparatus,
The present invention relates to a storage medium storing a document processing program and a document processing method, and more particularly, to determining whether or not a created document includes a plurality of topics.

[0002]

2. Description of the Related Art Conventionally, various processes, such as automatic creation of summaries (including abstracts) and association with other documents, etc., are performed on various documents such as books, papers, and reports using a computer. That is being done. For automatic summarization of documents, for example, “Extraction and processing of semantic information from full-text information” (Information Processing Society of Japan 38th Annual Conference Proceedings, 222
1989). In this method, an important word in a document is first extracted from information such as a character type and a verb, and the most important word is determined from the appearance frequency of the important word. Next, by determining an important sentence from whether or not the important word and the most important word appear, it is possible to automatically create a summary. Further, a method described in Japanese Patent Application Laid-Open No. 3-191475 has been proposed in which a summary is created more accurately by reflecting the nature of a paragraph of a sentence. On the other hand, as for association with other data, a hyperlink on the Internet, an association in knowledge processing (expert system or the like) by a frame system or the like is performed.

[0003]

However, in conventional document processing, it is not possible to determine whether or not a document to be processed includes a plurality of topics (topics). Was being processed. For this reason, in the case of the above-mentioned conventional automatic summarization method, it is possible to create a relatively appropriate summary for a document including a single topic, but a plurality of topics are included in one document. It was not always possible to create an appropriate summary for documents containing. In other words, sentences were selected for important sentences ignoring the existence of a plurality of units having different statements of assertions and facts, and summaries were created by connecting the sentences to each other, so that summaries with low readability were generated. Further, even a document including a plurality of topics can be related only to the entire document.

The present invention has been made to solve such a conventional problem, and it is an object of the present invention to provide a document processing apparatus capable of determining whether or not one document includes a plurality of topics. This is the purpose of 1. It is a second object of the present invention to provide a storage medium storing a computer-readable document processing program capable of determining whether one document includes a plurality of topics. It is a third object of the present invention to provide a document processing method capable of determining whether one document includes a plurality of topics.

[0005]

According to the first aspect of the present invention, as shown in FIG. 11, a document acquisition unit 101 for acquiring a document of a predetermined format composed of a plurality of sentences is provided. A document dividing means for dividing the acquired document into a plurality of sub-documents;
A similarity calculating unit 103 for calculating the similarity between the sub-documents divided by the sub-document, and determining whether the document includes a plurality of topics based on the similarity between the sub-documents calculated by the similarity calculating unit 103. The first object is achieved by providing the document processing apparatus with the determination means 104. According to the second aspect of the present invention, as shown in FIG. 12, in the document processing apparatus according to the first aspect, a document vector determining unit that determines a document vector characterizing the sub-document divided by the document dividing unit 102 105, and the similarity calculating means 103 includes the document vector determining means 105
The similarity between the sub-documents is calculated based on the document vector of the sub-document determined in (1). In the invention described in claim 3, as shown in FIGS. 11 and 12, claim 1 or claim 2
In the document processing apparatus described in the above item, the determination means 104
Tentatively determines a topic change based on the similarity between sub-documents calculated by the similarity calculation means 103, and the similarity calculation means 103 re-divides the topic based on the topic change tentatively determined by the determination means 104. The similarity between the sub-document groups is further calculated, and the determination unit 104 determines whether the document includes a plurality of topics from the similarity between the sub-document groups calculated by the similarity calculation unit 103. judge. According to the invention described in claim 4, in the document processing device described in claim 1, claim 2, or claim 3, as in the example shown in FIG. It has a summary creation means for automatically creating a summary. When the determination means 104 determines that the document includes a plurality of topics, the summary creation means 106 creates a summary in units of topics. According to the invention described in claim 5, as in an example shown in FIG.
The document processing apparatus according to any one of claims 1 to 4, further comprising an associating unit 107 for associating predetermined data with other data, wherein the associating unit 107 The determined topic is associated with other data in units constituting the topic.
According to the invention described in claim 6, in the document processing apparatus according to any one of claims 1 to 5, as in an example shown in FIGS. When it is determined that a plurality of topics are not included, the dividing unit re-divides into sub-documents of different sizes, and the similarity calculating unit re-calculates the similarity between the sub-documents after the sub-dividing, The determination unit re-determines whether the document includes a plurality of topics based on the similarity after the recalculation. In the invention described in claim 7, as shown in FIG. 15, a document acquisition function 201 for acquiring a document composed of a plurality of sentences in a predetermined format, and a document acquired by the document A document division function 202 for dividing into documents, a similarity calculation function 203 for calculating the similarity between the sub-documents divided by the document division function 202,
Computer-readable document processing for causing a computer to implement a determination function 204 for determining whether the document includes a plurality of topics based on the similarity between sub-documents calculated by the similarity calculation function 203 The second object is achieved by storing the program in a storage medium. In the invention according to claim 8, as shown in FIG. 16, in the storage medium according to claim 7, the document division function 2
02 is provided with a document vector determination function 205 for determining a document vector characterizing the sub-document divided in 02, and the similarity calculation function 203 uses two adjacent document vectors based on the document vector of the sub-document determined by the document vector determination function 205. Calculate the similarity between sub-documents. According to the ninth aspect of the present invention, as shown in FIGS. 15 and 16, in the storage medium according to the seventh or eighth aspect, the determination function 2
04 tentatively determines a topic change based on the similarity between sub-documents calculated by the similarity calculator 203, and the similarity calculation function 203 determines a topic change based on the topic change tentatively determined by the determination function 204. The similarity between the sub-documents subdivided is further calculated, and the determination function 204 determines whether the document includes a plurality of topics based on the similarity between the sub-documents calculated by the similarity calculation function 203. Is determined. According to the tenth aspect, as in the example shown in FIG. 17, in the storage medium according to the seventh, eighth, or ninth aspect, a summary of a document including a plurality of documents is automatically set. Has a summary creation function 206 to create the summary,
The summarizing function 206 determines whether the document includes a plurality of topics by the determining function 204.
Create summaries in units that make up topics. Claim 11
In the storage medium according to any one of claims 7 to 10, as in the example shown in FIG. 18, the predetermined data is associated with other data. The associating function 207 performs the associating with other data in units constituting the topic determined by the determining function 204. In the invention according to claim 12, FIGS.
In the storage medium according to any one of claims 7 to 11, when the determination function 204 determines that a plurality of topics are not included, as in an example shown in FIG. The division function 202 re-divides into sub-documents of different sizes, and the similarity calculation function 203
The similarity between the sub-documents after the re-division is re-calculated, and the determination function 204 re-determines whether or not the document includes a plurality of topics based on the re-calculated similarity. In the invention according to claim 13, as shown in FIG. 19, a document in a predetermined format composed of a plurality of sentences is acquired 301, the acquired document is divided into a plurality of sub-documents 302, and the divided The third object is achieved by calculating 303 the similarity of the sub-documents and determining 304 from the calculated similarity between the sub-documents whether the document includes a plurality of topics. According to the fourteenth aspect of the present invention, as shown in FIG. 20, in the document processing method according to the thirteenth aspect, the similarity between adjacent sub-documents is determined by determining a document vector characterizing the divided sub-documents 303a. , 303b based on the determined document vector of the sub-document. In the invention described in claim 15, as in the example shown in FIG. 21, when it is determined in the invention described in claim 13 or 14 that a document includes a plurality of topics, the topic is formed. An abstract 305 is created in units.

[0006]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of a document processing apparatus, a storage medium storing a document processing program, and a document processing method according to the present invention will be described below with reference to FIGS.
This will be described in detail with reference to FIG. (1) Overview of Embodiment In this embodiment, a document is divided into a plurality of sub-documents (units), a document vector for each sub-document is obtained, and a document vector is set between sub-documents or between a set of a plurality of sub-documents. Take the difference. Subdivision is performed at places where the cosine value between these two consecutive sub-documents is extremely low. The overall similarity between the m units before and n units after the subdivision position is also determined, and when the similarity falls below a predetermined threshold T2, it is finally determined that the topic is a change of topic. Then, by applying the conventional summarization process for each single topic area, a summary of the entire document is generated as a set of a plurality of types of summaries instead of one summary. In addition, at the time of summarizing and combining each topic, the compound noun phrase set as each topic may be specified as the title of the subsection in the summary.

(2) Details of Embodiment FIG. 1 is a block diagram showing the configuration of a document processing apparatus. The document processing apparatus according to the present embodiment may be configured as a computer system including a personal computer, a word processor, and the like, and may be configured as a LAN (local area network) server or a computer (PC) communication host including the Internet. It is possible. The document processing apparatus includes a control unit 11 for controlling the entire apparatus as shown in FIG. This control unit 1
1 includes a keyboard 12 and a mouse 13 as input devices and a display device 1 via a bus line 21 such as a data bus.
4, printing device 15, storage device 16, storage medium drive device 1
7, a communication control device 18, an input / output I / F 19, and a character recognition device 20 are connected. The control unit 11 controls the CP
U111, ROM112, and RAM113 are provided.
The ROM 112 is a read-only memory in which various programs and data for the CPU 111 to perform various controls and calculations are stored in advance.

The RAM 113 is a random access memory used as a working memory by the CPU 111. In the RAM 113, the summarization target document storage area 1 is used as an area for performing the summarization process according to the present embodiment.
131, summary parameter storage area 1132, break position storage area 1133, document vector storage area 113
4. The summary storage area 1135 and other various areas are secured. Document vector storage area 1
The document vector 134 stores a document vector for the document to be summarized and a document vector for each sub-document described later. The summary storage area 1135 stores a sub-summary for each sub-document group including each topic discovered according to the present embodiment, and a summary for the entire summarization target document.

The keyboard 12 has various keys such as a kana key and a numeric keypad for inputting a kana character, a function key for executing various functions, a cursor key, and the like. The mouse 13 is a pointing device, and is an input device for designating a corresponding function by left-clicking a key, an icon, or the like displayed on the display device 14.
As the display device 14, for example, a CRT or a liquid crystal display is used. The display device displays the content of the document to be summarized, the content of the summary automatically generated by the present embodiment, and the like. The printing device 15 stores the text displayed on the display device 14 and the document storage unit 1 of the storage device 16.
64 for printing a document or the like stored in the storage device 64. Various printing apparatuses such as a laser printer, a dot printer, an ink jet printer, a page printer, a thermal printer, and a thermal transfer printer are used as the printing apparatus.

The storage device 16 is composed of a readable and writable storage medium and a drive device for reading and writing various information such as programs and data on the storage medium.
As a storage medium used for the storage device 16, a hard disk is mainly used, but a readable and writable storage medium among various storage media used in 17 described later may be used. The storage device 16 includes a kana-kanji conversion dictionary 161, a program storage unit 162, a data storage unit 163, a document database 164, a summary database 165, a document vector database 166, and other storage units (not shown) (for example, stored in the storage device 16). (A storage unit for backing up programs, data, and the like). In the program storage unit 162, in addition to various programs such as an automatic summarization processing program, a document vector creation processing program, and a summarization creation program according to the present embodiment, a kana character string input using the kana-kanji conversion dictionary 161 is stored in a kanji character. Various programs such as a kana-kanji conversion program for converting into a mixed sentence are stored. The data storage 163 stores various data such as default values of summary parameters. As the default value of the summary parameter, for example, the ratio of the summary to all documents = “25%”, or the emphasis on quantity such as date / time, price information, physical quantity (size, weight, temperature, etc.) = “No”
And values such as URL (Uniform Resource Locator) emphasis = “No”, long and simple sentence emphasis = “No”, and “/” / “/” selection = “No” are stored.

The document database 164 stores documents created by the kana-kanji conversion program and documents created by other devices and read from the storage medium driving device 17 or the communication control device 18. This document database 1
The format of each document stored in the H.64 is not particularly limited, and a text format document, HTML (Hyper Text Ma
It is possible to store documents in various formats, such as a document in an rkup language) format and a document in a JIS format. Document database 1
In addition to the document data in these formats, data of a break position X, which is a transition of a topic discovered according to the present embodiment, is stored in the file 64. Summary database 165,
The document vector database 166 stores summaries and document vectors corresponding to the documents stored in the document database 164.

FIG. 2 shows a document vector database 166.
Are conceptually represented. As shown in FIG. 2, the importance f (x) obtained for the keyword x automatically extracted from the document is stored as the element value f (x) of the document vector. This document vector is stored for each document (A, B, C...), And is associated with each document stored in the document database 164.
The dimension of each document vector is the number of keywords x (keywords) to be adopted. When the similarity between two documents is obtained from both document vectors, the number of unions of keywords of both documents is Be a dimension. In this case, the element value of the other document vector for the keyword included in only one document vector is defined as “0”.

For example, in FIG. 2, the keywords of document B are “important, important words, importance,...”, The keywords of document C are “important,..., Politics,. It is as follows. Document vector of document B = (1,18,19, ...) Document vector of document C = (18, ..., 21, ...) On the other hand, when calculating the similarity between document B and document C, Keywords for both documents are "important, important words, importance,
..., politics, ... ", and the document vectors of both documents are defined as follows. Document vector of document A = (1,18,19, ...,
0,...), Document vector of document C = (18, 0, 0,.
1,…)

The storage medium drive 17 is a drive for the CPU 111 to read a computer program or data including a document from an external storage medium. The computer programs and the like stored in the storage medium include programs for various processes executed by the document processing apparatus of the present embodiment, and dictionaries and data used therein. Here, the storage medium refers to a storage medium in which a computer program, data, and the like are stored, and specifically, a magnetic storage medium such as a floppy disk, a hard disk, and a magnetic tape, and a semiconductor storage medium such as a memory chip and an IC card. A storage medium such as a CD-ROM, an MO, a PD (phase change rewritable optical disk) or the like, from which information can be read optically, and a paper such as a paper card or a paper tape (and a medium having a function equivalent to the paper) are used. It includes a storage medium and a storage medium in which a computer program or the like is stored by various methods. As a storage medium used in the document processing apparatus of the present embodiment, a storage medium such as a CD-ROM or a floppy disk is mainly used. The storage medium drive 17 can read data stored in the RAM 113 or the storage device 16 into a writable storage medium such as a floppy disk in addition to reading a computer program from these various storage media. It is.

In the document processing apparatus of the present embodiment, the control unit 1
One CPU 111 reads a computer program from an external storage medium set in the storage medium drive 17 and stores (installs) it in each unit of the storage 16. When executing various processes such as the automatic summarization process according to the present embodiment, the corresponding program is read from the storage device 16 into the RAM 113 and executed.
However, not from the storage device 16 but the storage medium drive 1
7, it is also possible to read the program directly from the external storage medium into the RAM 113 and execute it. Further, depending on the document processing apparatus, the automatic summarization processing program and the like of the present embodiment may be stored in the ROM 112 in advance and the CPU 111 may execute the program. Further, various programs and data such as the automatic summarization processing program of the present embodiment may be downloaded from another storage medium via the communication control device 18 and executed.

The communication control device 18 can transmit and receive various data such as documents in various formats such as text format and HTML format and bitmap data to and from other personal computers and word processors. I have. The input / output I / F 19 is an interface for connecting various devices such as a speaker that outputs audio, music, and the like. The character recognition device 20 is a device for recognizing characters written on paper or the like in various formats such as a text format or HTML, and is configured by an image scanner, a character recognition program, and the like.

In this embodiment, in addition to a document created by an input operation of the keyboard 12 (stored in a predetermined storage area of the RAM 113), a document created externally and stored in a predetermined storage medium is read from the storage medium driving device 17. Documents, documents stored in advance in a document database, documents downloaded from the communication control device 18, and the character recognition device 2.
It is possible to acquire various documents such as a document whose characters have been recognized as 0 as a target document (document acquisition means).

The operation of the automatic summarization process for creating summaries according to the number of topics by the document processing apparatus of the present embodiment configured as described above will be described with reference to FIGS. FIG. 3 illustrates a main operation of the automatic summarization process, and FIGS. 4 to 8 conceptually illustrate processing in each step of the automatic summarization process. (A) to (I) shown on the right side of the flowchart in FIG. 3 correspond to (A) to (I) in FIGS. 4 to 8. The document vectors shown in FIGS. 4A to 8I are two-dimensionally displayed for easy conceptual understanding, but are actually N-dimensional vectors. The CPU 111 acquires the summary target document A (FIG. 4A) for which a summary is to be created, and stores the summary target document storage area 113 of the RAM 113.
1 (step 11). The document A to be summarized is a RAM 113 (in the case where the document has been created in the own device), a document database 164 of the storage device 16 (in the case where a document has not been created yet), and a storage medium driving device in accordance with a user's instruction. 17 (in the case of a document created by the own device or another device), and from the communication control device 18 (in the case of communication by personal computer communication, the Internet or the like).

Next, the CPU 111 obtains an input value when the user inputs a summary parameter from the keyboard 12 or the like, and obtains a default value of the summary parameter stored in the data storage unit 163 when there is no input by the user. Is acquired and stored in the summary parameter storage area 1132 (step 12).

Next, the CPU 111 obtains a document vector V (FIG. 4B) for the digest document A stored in the digest document storage area 1131 (step 13). FIG. 9 is a flowchart showing the operation of the document vector creation processing. The CPU 111 extracts a self-sustaining word from the digest target document A by performing morphological analysis (step 13).
Along with 1), candidate words (phrases) including noun phrases, compound noun phrases, etc. are extracted from the document A to be summarized and stored in a predetermined work area of the RAM 113 (step 132). Then, the importance f (x) of each candidate word (phrase) is determined from the appearance frequency of the extracted candidate word (phrase) in the document A to be summarized and the evaluation function (step 133). Here, as the evaluation function, for example, when a predetermined important word is specified in advance, weighting for the important word, weighting according to the type of a candidate word (phrase) such as a word, a noun phrase, a compound noun phrase, and the like are used. used. Further, the CPU 111 determines keywords a, b,... Of the document A to be summarized from the value of the determined importance f (x) (step 134). And the importance f of each keyword
The document vector V = (f (a), f
(B),...) Are stored in the document vector storage area 1134 of the RAM 113 (step 135), and the process returns to the automatic summarization processing routine of FIG.

When the document vector V is obtained, the CPU 111
Divides the document A to be summarized into a predetermined number of P sub-documents A1, A2,..., AP as shown in FIG. The sub-document may be divided in any manner. Specifically, the sub-document is divided by r characters, divided by s lines, divided by t pages, divided by u sentences, divided by 1 / R, and divided by 1 / R. 1 /
Split into S, all pages into 1 / T, all sentences into 1
/ U is divided. Further, it is also possible to divide a part of the document (for example, the head of the document, the center of the document, the end of the document, etc.) into a sub-document having a larger size than other parts without dividing the sub-document size into a certain size. Any one of these division methods may be defined in advance, or may be selectable by the user as one of the summary parameters. When the break due to the division is in the middle of the sentence, the CPU 111 sets the position where the entire sentence is included in the previous sub-document as the break position X of the sub-document, and the break position Xn (n = 1 to 1) of each sub-document. (P-1)) is obtained and stored in the break position storage area 1133 (step 14).

Next, the CPU 111 regards each of the sub-documents A1 to AP delimited by the delimiting position X as one document by the document vector creation processing described with reference to FIG. D)) is obtained (step 15). Then, as shown in FIG.
Sub-documents An and An + 1 (n = 1 to P-
The similarity snn + 1 between 1 and 2 is obtained by a cosine depending on the angle between the document vector bn and the document vector bn + 1 (step 16). That is, if the angle between the two document vectors bn and bn + 1 is q, the inner product of both document vectors is bn · bn + 1, and the size of both document vectors is | bn | and | bn + 1 | The degree snn + 1 is obtained by the following equation (1).

[0023]

## EQU00001 ## Similarity Snn + 1 = COS (q) = (bn.bn + 1) / (| bn | .times. | Bn + 1 |)

The value of the similarity s takes a value up to −1 ≦ s ≦ 1, and the closer the value is to 1, the closer the two document vectors are to each other and the closer the two sub-documents are. .

Next, the CPU 111 calculates the calculated similarity sn
A break Xn, which is expected to be a topic change from n + 1, is temporarily determined. That is, each similarity snn + 1 is compared with a predetermined threshold T1, and the similarity sn equal to or smaller than the threshold T1 is compared.
A delimiter Xn corresponding to n + is provisionally determined as a topic break (step 17). Here, the CPU 111 determines whether or not there is a section XC whose similarity s is equal to or smaller than the threshold value T1 (step 18).
N), the process proceeds to step 22 with the entire document A as a sub-document. On the other hand, when there is a break Xn whose similarity s is equal to or smaller than the threshold T1 (step 17; Y), a sub-document group (from sub-documents A1 to An) up to the break Xn and a sub-document group (from sub-document A1 to An) As shown in FIG. 6F, the document vectors B to n and Bn + 1 of the sub-documents An + 1 to AP are obtained by the document vector creation processing described with reference to FIG. 9 (step 19). If there are a plurality (m) of breaks obtained from the similarity s, a document vector is created for each sub-document group m + 1 of each break unit.
In the present embodiment, for simplicity of description, a case where there is one break will be described as an example.

Next, the CPU 111 calculates the similarity S between the adjacent sub-document groups according to the above-described formula 1 as in step 16 (step 20, FIG. 7 (G)). If the similarity S is larger than the predetermined threshold value T2, the break Xn provisionally determined in step 17 happens to be adjacent to the sub-documents An and An + 1 because they are divided into small sub-documents.
It can be determined that the two document vectors bn and bn + t are separated from each other, so that it is determined that the break Xn is not a change of topic. On the other hand, if the similarity S is equal to or smaller than the predetermined threshold T2, the sub-document group (A1 to An) and the sub-document group (An +
1 to AP) are described for different contents and can be determined not to be similar to each other. Therefore, it is finally determined that the break Xn is a change of topic and stored in the break position storage area 1133 of the RAM 113 (step 21). .

Next, as shown in FIG. 7H, the CPU 111 sets each sub-document group (A1 to A
(n, An + 1 to AP) are created (step 22). FIG. 10 is a flowchart showing the operation of the summary creation processing. The CPU 111 first extracts independent words included in the sub-document group by performing morphological analysis (step 221), and also extracts candidate words (phrases) including noun phrases, compound noun phrases, and the like from the document A to be summarized. RAM 113
(Step 222). Then, based on the summary parameters stored in the summary parameter storage area 1132 of the RAM 16, the frequency of appearance of the extracted candidate words (phrases) in the sub-document group, the evaluation function, etc., each candidate word (phrase) importance f (y) Is determined (step 223).
Here, as the evaluation function, for example, when a predetermined important word is specified in advance, weighting for the important word, weighting according to the type of a candidate word (phrase) such as a word, a noun phrase, a compound noun phrase, and the like are used. used.

Further, the CPU 111 determines the determined importance f
The importance F (z) for each sentence included in the sub-document group is determined from (y) and the summary parameters stored in the summary parameter storage area relay (step 22).
4). Then, the sentences that are within the summary ratio of the summary parameter (for example, the top 25% of the total number of sentences in the sub-document group) are listed from the top of the sentence having the high importance of the determined sentence F (z). (Step 225). Then, the CPU 111 arranges the listed sentences in the order of appearance in the sub-document group to form a sub-summary of the sub-document group, stores this in the summary storage area of the RAM 113 (step 226), and Return to the automatic summarization processing routine.

When the creation of the sub-summary for each sub-document group is completed, the CPU 111 synthesizes all the sub-summaries stored in the summary storage area as shown in FIG. This is stored in a predetermined area of the summary storage area 1136 (step 23), and the automatic summarization process according to the present embodiment ends. As described above, according to the automatic summarization process according to the present embodiment, it is determined whether or not one document includes a plurality of topics, and a subsummary for each topic is synthesized to create an abstract. In addition, it is possible to create a summary from which the contents of each topic can be accurately grasped.

When the above automatic summarization processing is completed, the CPU
Reference numeral 111 performs storage processing of each data stored in the RAM 113 according to a user's instruction. That is, the document A to be summarized is read from the document storage area 1131 to be summarized, and is stored in the document database 164 of the storage device 16. Further, the created summary is read from the summary storage area 1135 and stored in the summary database 165 of the storage device 16 with the relevance to the summary target document A stored in the document database 164 attached. Further, the document vector V obtained in the document vector creation processing (step 13 in FIG. 3, FIG. 9) is read out from the document vector storage area 1135, and is associated with the summarization target document A stored in the document database 164 and added to the storage device. It is stored in 16 document vector databases 166.

Although the configuration and the automatic summarization process of the present embodiment have been described above, the present invention is not limited to these embodiments, and various modifications are possible within the scope of the invention described in each claim. It is possible to For example, in the embodiment, for morphological analysis and extraction of candidate words (phrases), a document vector creation process (steps 131 and 132 in FIG. 9) and a summary creation process (step 2 in FIG. 10)
21 and step 222), the same processing is performed independently. However, in the present invention, the candidate words (phrases) extracted in the document vector generation processing are stored in a predetermined area of the RAM 16, and the summary generation processing is performed. May be used.

In the embodiment described above, only the document A to be summarized, the summary, and the document vector V are stored and stored in the respective databases 164, 165, and 166 of the storage device 16 in the storage process after the automatic summarization process is completed. However, in the present invention, in step 132 of the document vector creation processing (FIG. 9), the extracted
The candidate words (phrases) stored in the predetermined work area 13 may be stored in the document database 164 or a dedicated candidate word (phrase) database in association with the document A to be summarized. The summary parameter may be read from the summary parameter storage area 1132 and stored in the summary database 166 or a dedicated summary parameter database in association with the summary. In step 20 (FIG. 3), the break Xn finally determined to be a topic change is stored in the break position storage area 113.
3, and may be stored in the document database 164 or a dedicated topic section database in association with the document A to be summarized.

Further, in the described embodiment, the morphological analysis (steps 131 and 221) and the candidate word (steps 131 and 221) are performed in both the document vector creation processing (step 13 and FIG. 9) and the digest creation processing (step 22 and FIG. 10). Phrase) (steps 132 and 222). However, since the processing is for the same sentence, the extracted candidate words (phrases) are the same. Therefore, in the present invention, the candidate words (phrases) extracted in the document vector creation processing are stored in a predetermined area of the RAM 113, and the steps 221 and 222 are performed by using the candidate words (phrases) stored in the summarization processing. It may be omitted. This candidate word (phrase)
May be stored in the document database 164 or a dedicated candidate word (phrase) database as a candidate word (phrase) for the document A to be summarized.

In the above-described embodiment, predetermined fixed values are used as the threshold values T1 and T2 for determining a topic change. However, in the present invention, the threshold values can be changed by the user. It may be. Alternatively, threshold functions T1 (u) and T2 (u) in which the expected number of topics u (calculated from a fixed-value threshold function or a history of similar documents in the past) are used as parameters may be used.

In the embodiment described above, the document A to be summarized is divided into P sub-documents, and the temporary judgment of the break Xn, which is expected to be a topic change, is performed only once. In step 18; N), a case has been described in which it is determined that a plurality of topics do not exist and a summary for the entire document A to be summarized is created. However, if an actual topic change exists at the center of a sub-document An, the document vector bn of that sub-document has an intermediate value, and a significant difference between adjacent sub-documents bn−1 and bn + 1 is found. In other words, there is a possibility that the similarities sn-1n and snn + 1 with the adjacent sub-documents before and after are not lower than the threshold T1. Therefore, if it is determined in step 18 that there is no punctuation (step 18; N), the size to be divided into sub-documents is set to a random number or a disjoint numerical value (for example, 5
(Meaning 4 or 6 instead of 10) may be slightly changed, and a retry may be performed a plurality of times to obtain a significant difference.

In the embodiment described above, a case has been described where it is determined whether or not a plurality of topics are included in the document A to be summarized, and the result is applied to the process of creating a summary.
In the present invention, the result of the topic determination may be applied to another. For example, when establishing a link in WEB SGML, a specific pointer may be pointed to each determined topic. Alternatively, the jump destination of the hyperlink may be set as a topic unit determined and fine-grained.

In the embodiment described above, a method according to the flowchart of FIG.
Although described as an example, the present invention is not limited to this method. For example, a method of extracting a keyword from A to be summarized and a method of determining importance (= element value of document vector) for the extracted keyword are described. Can be replaced by various known methods. Similarly, the process of creating an abstract for each sub-document group is not limited to the method shown in the flowchart of FIG. 10, but various known methods of summarizing and preparing an abstract can be used. Furthermore, the method of calculating the similarity between two document vectors is calculated by Equation 1, but is not limited to this equation, and the similarity between vectors can be expressed. If so, it is also possible to calculate the similarity by using other mathematical expressions.

The embodiments described above are not limited to documents created in Japanese, but can be applied to documents created in any language. In this case, it is only necessary to change a portion that does not affect the configuration of the present invention, such as using a morphological analysis algorithm for the language in which the target document is created.

Each device, each unit, each operation, each process, and the like described in the above embodiment can be constituted by each unit as a high-level concept including these units. It is. For example, in response to the description "determining whether or not there is a break XC whose similarity s is equal to or less than the threshold value T1 (step 18)", a "break break determination means" is configured, and "the determined importance f The keywords a, b,... Of the document A to be summarized are determined from the value of (x) (step 13).
"4)", a "keyword determination means" is configured, and a summary ratio of summary parameters (for example, a sub-document group's summarization ratio from a sentence having a higher importance of the determined sentence F (z)). Top 25 of all sentences
%), "Sentence list-up means" may be configured. Similarly, the embodiment may be configured with a higher concept such as “「 (operation) means ”for various other operations.

As a first modification, as shown in FIG. 11, a document acquisition unit 101 for acquiring a document composed of a plurality of sentences in a predetermined format, and a document acquired by the Document dividing means 1 for dividing into documents
02, similarity calculating means 103 for calculating the similarity between two adjacent sub-documents for each sub-document divided by the document dividing means 102, and each sub-document calculated by the similarity calculating means 103 The document processing apparatus is provided with determining means 104 for examining a topic change based on the similarity between them and determining whether or not the document includes a plurality of topics, so as to achieve the first object. Is also good. As a second modification, as shown in FIG. 12, in the document processing apparatus described in the first modification,
And a document vector determining unit for determining a document vector characterizing the sub-document divided by 2. The similarity calculating unit 103 includes two adjacent document vectors determined by the document vector determining unit 105. Calculate the similarity between two sub-documents. By calculating the similarity between two adjacent sub-documents, the CP
It is possible to reduce the processing amount (calculation amount) by U111,
Further, a more accurate multiple topic search can be performed based on the continuity (connectivity) of text in one document.

However, in the present invention, two adjacent two
It is not necessary to calculate the similarity between two sub-documents. For example, the similarity between three or more adjacent sub-documents may be calculated. In other words, the sub-document of interest is compared with the sub-documents before and after it, and the result is (simultaneously)
In consideration of this, it is determined at a time whether the change of the topic is 1) within the noted sub-document, 2) the sub-document boundary immediately after the topic, or 3) the sub-document boundary immediately before the sub-document.
You may do so. Also, the similarity between sub-documents that are not necessarily adjacent may be determined. In this case, the processing includes processing one or two line feed codes and blank codes between paragraphs as a sub-document, and calculating the similarity between sub-documents that jumps over the sub-document due to the line feed or the like.

[0042]

According to the present invention, a document in a predetermined format composed of a plurality of sentences is obtained, the obtained document is divided into a plurality of sub-documents, and each divided sub-document is divided into two adjacent sub-documents. Since the similarity between documents was calculated, the topic change was checked from the calculated similarity between sub-documents, and it was determined whether or not the document contained multiple topics. Can be determined. Therefore, it is possible to create a summary for each topic and to associate other documents and data for each topic.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a document processing apparatus according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram conceptually showing the contents of a document vector database in the embodiment.

FIG. 3 is a flowchart showing a main operation of an automatic summarization process in the embodiment.

FIG. 4 is a part of an explanatory diagram conceptually showing processing corresponding to each step of the automatic summarization processing shown in FIG. 3 in the embodiment.

FIG. 5 is another part of the explanatory view conceptually showing processing corresponding to each step of the automatic summarization processing shown in FIG. 3 in the embodiment.

FIG. 6 is another part of the explanatory view conceptually showing processing corresponding to each step of the automatic summarization processing shown in FIG. 3 in the embodiment.

FIG. 7 is another part of the explanatory view conceptually showing processing corresponding to each step of the automatic summarization processing shown in FIG. 3 in the embodiment.

FIG. 8 is another part of the explanatory view conceptually showing processing corresponding to each step of the automatic summarization processing shown in FIG. 3 in the embodiment.

FIG. 9 is a flowchart illustrating an operation of a document vector creation process according to the embodiment.

FIG. 10 is a flowchart illustrating an operation of a summary creation process in the embodiment.

FIG. 11 is a diagram corresponding to claims of the invention described in claim 1;

FIG. 12 is a diagram corresponding to claims of the invention described in claim 2;

FIG. 13 is an example of a claim correspondence diagram of the invention described in claim 4;

FIG. 14 is an example of a claim correspondence diagram of the invention described in claim 5;

FIG. 15 is a diagram corresponding to claims of the invention described in claim 7;

FIG. 16 is a diagram corresponding to claims of the invention described in claim 8;

FIG. 17 is an example of a claim correspondence diagram of the invention described in claim 10;

FIG. 18 is an example of a claim correspondence diagram of the invention described in claim 11;

FIG. 19 is a diagram corresponding to claims of the invention described in claim 13;

FIG. 20 is a view corresponding to claims of the invention described in claim 14;

FIG. 21 is a view corresponding to claims of the invention described in claim 15;

[Explanation of symbols]

 11 Controller 112 ROM 113 RAM 1131 Summarization target document storage area 1132 Summarization parameter storage area 1133 Break position storage area 1134 Document vector storage area 1135 Summary storage area 12 Keyboard 13 Mouse 14 Display device 15 Printer 16 Storage device 161 Kana-kanji conversion Dictionary 162 Program storage 163 Data storage 164 Document database 165 Summary database 166 Document vector database 17 Storage medium drive 18 Communication controller 19 Input / output I / F 101 Document acquisition unit 102 Document division unit 103 Similarity calculation unit 104 Judgment unit Reference Signs List 105 Document vector determination means 106 Abstract creation means 107 Association means 201 Document acquisition function 202 Document division function 203 Similarity calculation function 204 Judgment Function 205 document vector decision section 206 summarization function 207 associated function

Claims (15)

[Claims] 1. A document acquisition unit for acquiring a document of a predetermined format composed of a plurality of sentences, a document division unit for dividing a document acquired by the document acquisition unit into a plurality of sub-documents, and the document division unit A similarity calculating unit for calculating a similarity between the sub-documents divided according to: and a similarity between the sub-documents calculated by the similarity calculating unit, and determining whether the document includes a plurality of topics. A document processing apparatus comprising: 2. A document vector determining means for determining a document vector characterizing a sub-document divided by the document dividing means, wherein the similarity calculating means determines a document vector of the sub-document determined by the document vector determining means. 2. The document processing apparatus according to claim 1, wherein the similarity between the sub-documents is calculated by the following. 3. The determining means tentatively determines a topic change from the similarity between the sub-documents calculated by the similarity calculating means, and the similarity calculating means tentatively determines the topic determined by the determining means. Further, the similarity between the sub-document groups re-divided at the transition point is calculated, and the determination unit includes a plurality of topics in the document based on the similarity between the sub-document groups calculated by the similarity calculation unit. The document processing apparatus according to claim 1, wherein it is determined whether or not the document processing is performed. 4. A summary creating means for automatically creating a summary of a document composed of a plurality of documents, wherein the summary creating means is determined by the determining means to include a plurality of topics in the document. 4. The document processing apparatus according to claim 1, wherein an abstract is created in a unit constituting a topic. 5. An associating means for associating predetermined data with other data, wherein the associating means associates with other data in units constituting a topic determined by the determining means. The document processing apparatus according to any one of claims 1 to 4, wherein 6. When the determining unit determines that a plurality of topics are not included, the dividing unit divides the document into sub-documents having different sizes, and the similarity calculating unit calculates the adjacent sub-documents after the re-dividing. Recalculating the similarity between the documents, wherein the determination unit examines a topic change from the recalculated similarity and re-determines whether the document includes a plurality of topics. The document processing device according to any one of claims 1 to 5. 7. A document acquisition function for acquiring a document of a predetermined format composed of a plurality of sentences, a document division function for dividing a document acquired by the document acquisition function into a plurality of sub-documents, and a document division function. A similarity calculation function for calculating the similarity between the sub-documents divided by the sub-document, and determining whether the document includes a plurality of topics from the similarity between the sub-documents calculated by the similarity calculation function And a computer-readable document processing program for causing a computer to implement the determination function. 8. A document vector determining function for determining a document vector characterizing a sub-document divided by the document dividing function, wherein the similarity calculating function is a document vector of the sub-document determined by the document vector determining function. A storage medium storing the document processing program according to claim 7, wherein the similarity between the sub-documents is calculated by the following. 9. The determination function tentatively determines a topic change based on the similarity between sub-documents calculated by the similarity calculation function, and the similarity calculation function determines the topic tentatively determined by the determination function. Further, the similarity between the sub-document groups re-divided at the transition of is calculated, and the determining function includes a plurality of topics in the document from the similarity between the sub-document groups calculated by the similarity calculating function. 9. A storage medium storing the document processing program according to claim 7, wherein it is determined whether the document processing program is executed. 10. A summary creation function for automatically creating a summary of a document composed of a plurality of documents, wherein the summary creation function is determined by the determination function to include a plurality of topics in the document. 10. The storage medium storing the document processing program according to claim 7, wherein an abstract is created in a unit constituting a topic. 11. An associating function for associating predetermined data with other data, wherein the associating function is to associate with other data in units constituting a topic determined by the determining function. A storage medium storing the document processing program according to any one of claims 7 to 10. 12. When the determining function determines that a plurality of topics are not included, the dividing function re-divides the document into sub-documents of different sizes, and the similarity calculating function performs the re-dividing of the adjacent sub-documents. Recalculating the similarity between the documents, the determining function examines a topic change from the similarity after the recalculation, and redetermines whether or not the document includes a plurality of topics. A storage medium storing the document processing program according to any one of claims 7 to 11. 13. A document in a predetermined format composed of a plurality of sentences is obtained, the obtained document is divided into a plurality of sub-documents, a similarity between the divided sub-documents is calculated, and the calculated A document processing method comprising: determining whether or not a plurality of topics are included in the document based on similarity. 14. The document according to claim 13, wherein the similarity between adjacent sub-documents is determined by determining a document vector characterizing the divided sub-document and using the determined document vector of the sub-document. Processing method. 15. The document processing method according to claim 13, wherein when it is determined that a plurality of topics are included in the document, an abstract is created in units of the topics.