JP6059271B2 - Image processing apparatus and image processing method - Google Patents
Image processing apparatus and image processing method Download PDFInfo
- Publication number
- JP6059271B2 JP6059271B2 JP2015036771A JP2015036771A JP6059271B2 JP 6059271 B2 JP6059271 B2 JP 6059271B2 JP 2015036771 A JP2015036771 A JP 2015036771A JP 2015036771 A JP2015036771 A JP 2015036771A JP 6059271 B2 JP6059271 B2 JP 6059271B2
- Authority
- JP
- Japan
- Prior art keywords
- image
- image processing
- endoscopic
- enlarging
- processing apparatus
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims description 24
- 238000003672 processing method Methods 0.000 title claims description 10
- 210000003855 cell nucleus Anatomy 0.000 claims description 49
- 206010028980 Neoplasm Diseases 0.000 claims description 42
- 238000000034 method Methods 0.000 claims description 31
- 238000004458 analytical method Methods 0.000 claims description 29
- 208000027121 wild type ATTR amyloidosis Diseases 0.000 claims description 20
- 208000003200 Adenoma Diseases 0.000 claims description 18
- 206010001233 Adenoma benign Diseases 0.000 claims description 16
- 201000011510 cancer Diseases 0.000 claims description 16
- 210000004027 cell Anatomy 0.000 claims description 15
- 238000003384 imaging method Methods 0.000 claims description 15
- 238000003745 diagnosis Methods 0.000 claims description 14
- 210000001035 gastrointestinal tract Anatomy 0.000 claims description 7
- 230000003287 optical effect Effects 0.000 claims description 4
- 208000037062 Polyps Diseases 0.000 claims description 3
- 206010048832 Colon adenoma Diseases 0.000 claims description 2
- 239000000284 extract Substances 0.000 claims description 2
- 210000004877 mucosa Anatomy 0.000 claims 2
- 230000002496 gastric effect Effects 0.000 claims 1
- 238000010827 pathological analysis Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 7
- 238000012706 support-vector machine Methods 0.000 description 7
- 210000001519 tissue Anatomy 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 239000008186 active pharmaceutical agent Substances 0.000 description 3
- 238000001839 endoscopy Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 210000004400 mucous membrane Anatomy 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000000762 glandular Effects 0.000 description 2
- 230000003902 lesion Effects 0.000 description 2
- 210000004940 nucleus Anatomy 0.000 description 2
- RBTBFTRPCNLSDE-UHFFFAOYSA-N 3,7-bis(dimethylamino)phenothiazin-5-ium Chemical compound C1=CC(N(C)C)=CC2=[S+]C3=CC(N(C)C)=CC=C3N=C21 RBTBFTRPCNLSDE-UHFFFAOYSA-N 0.000 description 1
- ZDZXXOJETOMNEI-UHFFFAOYSA-N 3-n,3-n,7-n,7-n-tetramethyl-10h-phenothiazine-3,7-diamine;hydrochloride Chemical compound Cl.C1=C(N(C)C)C=C2SC3=CC(N(C)C)=CC=C3NC2=C1 ZDZXXOJETOMNEI-UHFFFAOYSA-N 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 229910002056 binary alloy Inorganic materials 0.000 description 1
- 238000007621 cluster analysis Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 239000013078 crystal Substances 0.000 description 1
- ZXJXZNDDNMQXFV-UHFFFAOYSA-M crystal violet Chemical compound [Cl-].C1=CC(N(C)C)=CC=C1[C+](C=1C=CC(=CC=1)N(C)C)C1=CC=C(N(C)C)C=C1 ZXJXZNDDNMQXFV-UHFFFAOYSA-M 0.000 description 1
- 210000000805 cytoplasm Anatomy 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000012137 double-staining Methods 0.000 description 1
- 210000003238 esophagus Anatomy 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 210000002429 large intestine Anatomy 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 229960000907 methylthioninium chloride Drugs 0.000 description 1
- 238000001000 micrograph Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 210000000813 small intestine Anatomy 0.000 description 1
- 239000012192 staining solution Substances 0.000 description 1
- 210000002784 stomach Anatomy 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Landscapes
- Investigating Or Analysing Biological Materials (AREA)
- Instruments For Viewing The Inside Of Hollow Bodies (AREA)
- Endoscopes (AREA)
Description
本発明は、画像処理装置及び画像処理方法に関する。 The present invention relates to an image processing apparatus and an image processing method.
画像処理を行うことによって、病理診断を行う発明が提案されている（例えば、特許文献１〜３参照。）。しかし、食道、胃、小腸、大腸などの消化管の病理診断は、専門的な知識と経験を要するため、消化管の一部組織を切除し、体外にて顕微鏡による診断を行っている。 Inventions for performing pathological diagnosis by performing image processing have been proposed (see, for example, Patent Documents 1 to 3). However, since pathological diagnosis of the digestive tract such as the esophagus, stomach, small intestine, and large intestine requires specialized knowledge and experience, some tissues of the digestive tract are excised and diagnosed by a microscope outside the body.
一方で、近年、超拡大機能（３８０倍〜４５０倍）の高倍率機能により、リアルタイムに体内（消化管内）で病理組織を予測できる内視鏡ｅｎｄｏｃｙｔｏｓｃｏｐｙが開発されており、病理診断の代替になることが期待されているが、画像解読に専門的トレーニングが必要である。（非特許文献１）。そのため、先行研究によりｅｎｄｏｃｙｔｏｓｃｏｐｙ画像に対するコンピュータ診断支援システムが開発され（非特許文献２）、非腫瘍・腺腫・癌の鑑別に有用性があることがわかった。 On the other hand, in recent years, endoscopic endoscopies that can predict pathological tissues in the body (in the digestive tract) in real time with a high magnification function of a super-magnification function (380 to 450 times) have been developed, which is an alternative to pathological diagnosis Although it is expected, specialized training is required for image decoding. (Non-Patent Document 1). For this reason, a computer diagnosis support system for endocytoscopy images has been developed by prior research (Non-Patent Document 2), and it has been found that the system is useful for differentiating non-tumors, adenomas, and cancers.
本発明の課題は、ｓｅｓｓｉｌｅ ｓｅｒｒａｔｅｄ ａｄｅｎｏｍａ／ｐｏｌｙｐ （ＳＳＡ／Ｐ）という病変の診断をコンピュータ診断支援システムを用いて客観的に行うことである。ＳＳＡ／Ｐはこれまで非腫瘍に分類されてきたが、近年腫瘍になる可能性のあるということがわかってきた重要病変である。ＳＳＡ／Ｐは、細胞核の形状は非腫瘍のものと同じであるため、これまでの細胞核の形状のみに基づいた既存のコンピュータ診断支援システム（非特許文献２）では、非腫瘍であるのか、ＳＳＡ／Ｐであるのかを判断することはできなかった。 An object of the present invention is to objectively perform a diagnosis of a lesion called sessile serrated adenoma / polyp (SSA / P) using a computer diagnosis support system. SSA / P has been classified as a non-tumor so far, but is an important lesion that has recently been found to be a potential tumor. In SSA / P, the shape of the cell nucleus is the same as that of a non-tumor, so in the existing computer diagnosis support system (Non-patent Document 2) based only on the shape of the cell nucleus, SSA / P is a non-tumor. / P could not be determined.
本発明は、専門的な知識と経験を有する者でなくとも、非腫瘍、腺腫、癌及びＳＳＡ／Ｐのうちのいずれであるのかを判断可能にすることを目的とする。また、腺腫・癌の診断についても、より精度の高い診断を取得することを目的とする。 An object of the present invention is to make it possible to determine whether a tumor is a non-tumor, an adenoma, cancer, or SSA / P , even if it is not a person with specialized knowledge and experience. It is also intended to obtain a more accurate diagnosis for adenoma / cancer diagnosis.
発明者らは、内視鏡画像に含まれるすべての細胞核の特徴解析の結果と、同じ内視鏡画像から得られるテクスチャ解析の結果と、をパラメータに用いた分類を行うことで、非腫瘍、腺腫及び癌の診断精度が向上し、さらにＳＳＡ／Ｐが分類可能になることを発見した。 The inventors performed classification using the results of the characteristic analysis of all cell nuclei contained in the endoscopic image and the results of texture analysis obtained from the same endoscopic image as parameters, so that non-tumor, It was discovered that the diagnostic accuracy of adenomas and cancers was improved, and that SSA / P could be classified.
具体的には、本発明に係る画像処理装置は、内視鏡を介して光を照射した消化管の粘膜の細胞を拡大撮像して得られた内視鏡画像に含まれる細胞核を抽出し、抽出したすべての細胞核について、細胞核の特徴を計測する機能と、前記内視鏡画像の全体の濃淡について、テクスチャ解析を行う機能と、前記細胞核の特徴及び前記テクスチャ解析の結果を用いて、非腫瘍、腺腫、癌及びＳＳＡ／Ｐのいずれであるのかの識別を行い、非腫瘍、腺腫、癌及びＳＳＡ／Ｐのうちのいずれかを診断結果として出力する機能と、を備える。 Specifically, the image processing apparatus according to the present invention extracts cell nuclei contained in an endoscopic image obtained by enlarging and imaging cells of the mucous membrane of the digestive tract irradiated with light through an endoscope, Using all the extracted cell nuclei, the function of measuring the characteristics of the cell nuclei, the function of performing texture analysis on the overall density of the endoscopic image, and the characteristics of the cell nuclei and the results of the texture analysis, non-tumor And a function of identifying any one of adenoma, cancer and SSA / P and outputting any one of non-tumor, adenoma, cancer and SSA / P as a diagnostic result.
具体的には、本発明に係る画像処理方法は、画像処理装置が、内視鏡を介して光を照射した消化管の粘膜の細胞を拡大撮像して得られた内視鏡画像に含まれる細胞核を抽出し、抽出したすべての細胞核について、細胞核の特徴を計測する手順と、画像処理装置が、前記内視鏡画像の全体の濃淡について、テクスチャ解析を行う手順と、画像処理装置が、前記細胞核の特徴及び前記テクスチャ解析の結果を用いて、非腫瘍、腺腫、癌及びＳＳＡ／Ｐのいずれであるのかの識別を行い、非腫瘍、腺腫、癌及びＳＳＡ／Ｐのうちのいずれかを診断結果として出力する手順と、を行う。 Specifically, the image processing method according to the present invention is included in the endoscopic image obtained by the image processing apparatus by enlarging and imaging the cells of the mucous membrane of the digestive tract irradiated with light through the endoscope. A procedure for extracting cell nuclei, measuring the characteristics of the cell nuclei for all the extracted cell nuclei, a procedure for performing image analysis on the overall density of the endoscopic image, and an image processing device for Using the characteristics of the cell nucleus and the result of the texture analysis, it is identified whether it is non-tumor, adenoma, cancer, or SSA / P, and any of non-tumor, adenoma, cancer, or SSA / P is diagnosed And a procedure for outputting as a result.
前記内視鏡画像は、光学顕微鏡を搭載した内視鏡を用いて細胞を拡大撮像して得られた画像であってもよい。 The endoscopic image may be an image obtained by enlarging and imaging a cell using an endoscope equipped with an optical microscope.
前記内視鏡画像は、共焦点内視鏡を用いて細胞を拡大撮像して得られた画像であってもよい。 The endoscopic image may be an image obtained by enlarging and imaging a cell using a confocal endoscope.
前記内視鏡画像は、染色された細胞を拡大撮像して得られた画像であってもよい。 The endoscopic image may be an image obtained by enlarging and imaging stained cells.
本発明によれば、専門的な知識と経験を有する者でなくとも、非腫瘍、腺腫、癌及びＳＳＡ／Ｐのうちのいずれであるのかを判断可能にすることができる。また、腺腫・癌の診断についても既存の診断支援システムに比べ高い精度で診断が可能となる。 According to the present invention, it is possible to determine whether the tumor is non-tumor, adenoma, cancer, or SSA / P , even if the person has no specialized knowledge and experience. In addition, the diagnosis of adenoma / cancer can be performed with higher accuracy than existing diagnosis support systems.
以下、本発明の実施形態について、図面を参照しながら詳細に説明する。なお、本発明は、以下に示す実施形態に限定されるものではない。これらの実施の例は例示に過ぎず、本発明は当業者の知識に基づいて種々の変更、改良を施した形態で実施することができる。なお、本明細書及び図面において符号が同じ構成要素は、相互に同一のものを示すものとする。 Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited to embodiment shown below. These embodiments are merely examples, and the present invention can be implemented in various modifications and improvements based on the knowledge of those skilled in the art. In the present specification and drawings, the same reference numerals denote the same components.
図１に、本実施形態に係る画像処理方法の一例を示す。本実施形態に係る画像処理方法は、画像処理装置が、特徴量算出手順Ｓ１０１と、識別手順Ｓ１０２と、診断結果出力手順Ｓ１０３と、を順に実行する。本実施形態に係る画像処理装置は、ＣＰＵ（Ｃｅｎｔｒａｌ Ｐｒｏｃｅｓｓｉｎｇ Ｕｎｉｔ）と、メモリと、を備えるコンピュータである。本実施形態に係る画像処理装置は、メモリに記憶されたコンピュータプログラムを実行することで、各手順を行う。コンピュータプログラムは、コンピュータから読み出し可能な任意の記録媒体に記憶され得る。 FIG. 1 shows an example of an image processing method according to this embodiment. In the image processing method according to the present embodiment, the image processing apparatus sequentially executes the feature amount calculation procedure S101, the identification procedure S102, and the diagnosis result output procedure S103. The image processing apparatus according to the present embodiment is a computer including a CPU (Central Processing Unit) and a memory. The image processing apparatus according to the present embodiment performs each procedure by executing a computer program stored in a memory. The computer program can be stored in any recording medium that can be read from the computer.
特徴量算出手順Ｓ１０１では、画像処理装置が、細胞核計測手順及びテクスチャ解析手順を実行し、内視鏡画像に含まれる細胞核の特徴を算出する。細胞核計測手順では、内視鏡画像に含まれる細胞核を抽出し、抽出したすべての細胞核について、細胞核の特徴を計測する。テクスチャ解析手順では、内視鏡画像の全体について、テクスチャ解析を行う。
識別手順Ｓ１０２では、画像処理装置が、特徴量算出手順Ｓ１０１で得られた特徴量を用いて、病理診断に対応した分類を行う。
診断結果出力手順Ｓ１０３では、画像処理装置が、識別手順Ｓ１０２の分類結果を出力する。
In the feature amount calculation procedure S101, the image processing apparatus executes the cell nucleus measurement procedure and the texture analysis procedure, and calculates the feature of the cell nucleus included in the endoscopic image. In the cell nucleus measurement procedure, cell nuclei included in the endoscopic image are extracted, and the characteristics of the cell nuclei are measured for all the extracted cell nuclei. In the texture analysis procedure, texture analysis is performed on the entire endoscope image.
In the identification procedure S102, the image processing apparatus performs classification corresponding to the pathological diagnosis using the feature value obtained in the feature value calculation procedure S101.
In the diagnostic result output procedure S103, the image processing apparatus outputs the classification result of the identification procedure S102.
本発明では、内視鏡を介して光を照射した細胞を拡大撮像して得られた内視鏡画像を用いる。拡大撮像して得られた内視鏡画像は、細胞核の形状が特定可能な画像であり、例えば、ｅｎｄｏｃｙｔｏｓｃｏｐｙ系又は共焦点系の内視鏡を用いて拡大撮像された画像を用いることができる。共焦点系は、対物レンズの焦点位置と共役な位置（像位置）に開口を配置したものである。本実施形態では、内視鏡に光学顕微鏡を搭載したｅｎｄｏｃｙｔｏｓｃｏｐｙを用いることが好ましく、これにより共焦点系に比べて鮮明なカラー画像を取得することができる。 In the present invention, an endoscopic image obtained by enlarging and imaging a cell irradiated with light through an endoscope is used. The endoscopic image obtained by magnifying imaging is an image in which the shape of the cell nucleus can be specified, and for example, an image that is magnifying and imaged using an endocytoscope-type or confocal-type endoscope can be used. In the confocal system, an aperture is arranged at a position (image position) conjugate with the focal position of the objective lens. In the present embodiment, it is preferable to use endocytoscopy in which an optical microscope is mounted on an endoscope, whereby a clear color image can be obtained as compared with a confocal system.
内視鏡画像は、染色された細胞を拡大撮像して得られた画像であることが好ましい。染色液は、組織像の獲得が可能でありかつ安全性が高いものが好ましく、例えば、メチレンブルー（３，７−ビス（ジメチルアミノ）フェノチアジニウムクロリド）若しくはクリスタルバイオレット（［４‐｛ビス（４‐ジメチルアミノフェニル）メチレン｝‐２，５‐シクロヘキサジエン‐１‐イリデン］ジメチルアンモニウムクロリド）又はこれらの組み合わせを用いることができる。特にこれらの２重染色により、細胞核は青色に細胞質はピンク色に染め分けられるため、安全性を維持しつつ鮮明な組織像を獲得することが可能となる。 The endoscopic image is preferably an image obtained by enlarging and imaging the stained cells. The staining solution is preferably one that can acquire a tissue image and has high safety. For example, methylene blue (3,7-bis (dimethylamino) phenothiazinium chloride) or crystal violet ([4- {bis ( 4-dimethylaminophenyl) methylene} -2,5-cyclohexadiene-1-ylidene] dimethylammonium chloride) or combinations thereof. In particular, these double staining allows the cell nucleus to be dyed in blue and the cytoplasm into pink, so that a clear tissue image can be obtained while maintaining safety.
図２に、本実施形態に係る内視鏡画像の一例を示す。内視鏡画像には、複数の細胞核が含まれる。本実施形態に係る発明は、内視鏡画像の全体を用いるため、病理診断に有用な領域を切り出すことが好ましい。 FIG. 2 shows an example of an endoscopic image according to this embodiment. An endoscopic image includes a plurality of cell nuclei. Since the invention according to this embodiment uses the entire endoscopic image, it is preferable to cut out a region useful for pathological diagnosis.
また、本実施形態における内視鏡画像は、病理診断を行いたい組織を拡大したものの画像である。拡大の倍率は、細胞核の形状を測定可能な任意の倍率であり、例えば、３８０倍である。内視鏡画像の組織は、生体の一部を切除したものの画像であってもよいし、生体の表面を撮影したものの画像であってもよい。生体の表面は、例えば、内視鏡で観測可能な任意の部位であり、例えば、口腔内、咽頭、食道、胃、小腸、大腸、肛門などの消化管の粘膜や胆管・膵管および肺・気管支である。 Further, the endoscopic image in the present embodiment is an image obtained by enlarging a tissue to be pathologically diagnosed. The magnification of enlargement is an arbitrary magnification capable of measuring the shape of the cell nucleus, for example, 380 times. The tissue of the endoscopic image may be an image of a part of the living body excised or an image of the surface of the living body. The surface of a living body is, for example, an arbitrary part that can be observed with an endoscope. It is.
特徴量算出手順Ｓ１０１における細胞核計測手順について説明する。
まず、内視鏡画像に含まれるすべての細胞核を抽出する。細胞核の抽出は、細胞核の領域のセグメンテーションを行い、アーチファクト除去を行う。細胞核の領域のセグメンテーションは、例えば、Ｒ成分での大津の２値化手法を用いる。アーチファクト除去は、たとえば、２値化画像の白い画素が連続した画素を１つの領域とし、各領域に対し面積と長径、真円度を算出する。面積が設定した範囲（例えば３０μｍ２から５００μｍ２）、かつ長径が設定した値（例えば３０μｍ以下）、かつ真円度が設定した値（例えば０．３以上）のものを解析対象として残し、それ以外の領域を除去する。長径と真円度は、例えば、領域を楕円近似して算出する。抽出された核の個数が予め設定した個数（たとえば３０個）以下の場合は、画像の信頼性が低いと考え、診断不能例としてもよい。図３に、抽出した細胞核の一例を示す。
The cell nucleus measurement procedure in the feature amount calculation procedure S101 will be described.
First, all cell nuclei contained in the endoscopic image are extracted. In the extraction of cell nuclei, the region of the cell nuclei is segmented to remove artifacts. The segmentation of the cell nucleus region uses, for example, Otsu's binarization method with an R component. In the artifact removal, for example, a pixel in which white pixels of a binarized image are continuous is defined as one region, and the area, the major axis, and the roundness are calculated for each region. The area set in the range (for example, 30 μm 2 to 500 μm 2 ), the major axis set in the value (for example 30 μm or less), and the roundness set in the value (for example 0.3 or more) is left as the analysis target, Remove other areas. The major axis and the roundness are calculated, for example, by approximating the region to an ellipse. When the number of extracted nuclei is equal to or less than a preset number (for example, 30), it is considered that the reliability of the image is low, and an example in which diagnosis is not possible may be used. FIG. 3 shows an example of the extracted cell nucleus.
次に、抽出した全ての細胞核の特徴を計測する。細胞核の特徴は、例えば、細胞核の長径ＤＬ、細胞核の短径ＤＳ、細胞核の周囲長、細胞核の面積、細胞核の真円度及び細胞核の色である。面積及び真円度は、平均値及び標準偏差を用いることが好ましい。 Next, the characteristics of all extracted cell nuclei are measured. The characteristics of the cell nucleus are, for example, the cell nucleus major axis DL, the cell nucleus minor axis DS, the cell nucleus perimeter, the cell nucleus area, the cell nucleus roundness, and the cell nucleus color. For the area and roundness, an average value and a standard deviation are preferably used.
特徴量算出手順Ｓ１０１におけるテクスチャ解析手順について説明する。
まず、内視鏡画像を予め定められた画素に分割する。例えば、図４に示すように、画素Ｐ１１からＰ８９までの画素に分割する。
そして、各画素について、テクスチャ解析を行う。腺腔や核の配列によって、各画素の濃淡は変化する。特に、隣接する画素との濃淡を比較することで、腺腔や細胞核の配列を数値化することが可能である。そこで、テクスチャ解析では、濃淡変化によって表わす模様（＝テクスチャ）を解析する。テクスチャ解析の手法は任意であり、例えば、Ｌｏｃａｌ Ｂｉｎａｒｙ Ｐａｔｔｅｒｎ（ＬＢＰ）を用いることができる。
The texture analysis procedure in the feature amount calculation procedure S101 will be described.
First, an endoscopic image is divided into predetermined pixels. For example, as shown in FIG. 4, the pixel is divided into pixels P11 to P89.
Then, texture analysis is performed for each pixel. The shade of each pixel changes depending on the arrangement of glandular cavities and nuclei. In particular, the arrangement of glandular cavities and cell nuclei can be quantified by comparing the density of adjacent pixels. Therefore, in the texture analysis, a pattern (= texture) represented by shading change is analyzed. The method of texture analysis is arbitrary, and for example, Local Binary Pattern (LBP) can be used.
テクスチャ解析の一例であるＬＢＰについて、図５を参照しながら説明する。ＬＢＰは、３×３画素領域内での中央に位置する画素ＰＸと隣接する画素Ｐ１〜Ｐ８の濃淡の大小を２進法で数値化する手法である。 LBP, which is an example of texture analysis, will be described with reference to FIG. The LBP is a method of quantifying the magnitude of light and shade of the pixel PX located at the center in the 3 × 3 pixel region and the adjacent pixels P1 to P8 by a binary system.
例えば、画素ＰＸが図４に示す画素Ｐ２３である場合、画素Ｐ１〜Ｐ８は、それぞれ、Ｐ１２、Ｐ１３、Ｐ１４、Ｐ２４、Ｐ３４、Ｐ３３、Ｐ３２、Ｐ２２である。画素Ｐ２３の濃度と画素Ｐ１２の濃度を比較し、画素Ｐ２３の濃度が画素Ｐ１２の濃度よりも高ければ「１」とし、画素Ｐ２３の濃度が画素Ｐ１２の濃度よりも低ければ「０」とする。画素Ｐ２〜Ｐ８についても、画素Ｐ１と同様に「１」又は「０」のいずれであるかを判定する。これにより、画素Ｐ１〜Ｐ８の濃淡の大小を数値化した、例えば、００１１１００１というバイナリーデータを得ることができる。この場合、「００１１１００１」を１０進法に変換すると、２５６階調における５７という値が得られる。 For example, when the pixel PX is the pixel P23 shown in FIG. 4, the pixels P1 to P8 are P12, P13, P14, P24, P34, P33, P32, and P22, respectively. The density of the pixel P23 is compared with the density of the pixel P12, and is set to “1” if the density of the pixel P23 is higher than the density of the pixel P12, and is set to “0” if the density of the pixel P23 is lower than the density of the pixel P12. Whether the pixels P2 to P8 are “1” or “0” is determined similarly to the pixel P1. As a result, binary data such as 00111001, for example, in which the shades of the pixels P1 to P8 are digitized can be obtained. In this case, when “00111001” is converted to decimal, a value of 57 in 256 gradations is obtained.
図６に、テクスチャ解析の結果の一例を示す。テクスチャ解析を全画素に行い、画素値の分布を示すヒストグラムを作成する。 FIG. 6 shows an example of the result of texture analysis. Texture analysis is performed on all pixels, and a histogram showing the distribution of pixel values is created.
識別手順Ｓ１０２における分類について説明する。本実施形態では、特徴量算出手順Ｓ１０１で得られた特徴量を用いて、病理診断に対応した分類を行う。病理診断に対応した分類は、例えば、非腫瘍、腺腫、癌、ＳＳＡ／Ｐである。分類は、例えば、ＳＶＭ（Ｓｕｐｐｏｒｔ Ｖｅｃｔｏｒ Ｍａｃｈｉｎｅ）を用いる。 The classification in the identification procedure S102 will be described. In the present embodiment, classification corresponding to pathological diagnosis is performed using the feature amount obtained in the feature amount calculation step S101. The classification corresponding to the pathological diagnosis is, for example, non-tumor, adenoma, cancer, SSA / P. For classification, for example, SVM (Support Vector Machine) is used.
ＳＶＭの各データ点は、例えば、内視鏡画像から抽出した全ての細胞核の各特徴に加え、テクスチャ解析で得られたヒストグラムを用いる。細胞核の各特徴は、長径ＤＬ、短径ＤＳ、周囲長、面積、真円度及び色である。テクスチャ解析で得られたヒストグラムは、例えば、テクスチャ解析によって算出される全ての要素であり、例えば、２５６階調とした場合は２５６個の要素となる。 For each data point of SVM, for example, in addition to each feature of all cell nuclei extracted from the endoscopic image, a histogram obtained by texture analysis is used. Each feature of the cell nucleus is a major axis DL, a minor axis DS, a perimeter, an area, a roundness, and a color. The histogram obtained by texture analysis is, for example, all elements calculated by texture analysis. For example, when 256 gradations are used, there are 256 elements.
ここで、分類を行うに際し、非腫瘍、腺腫、癌、ＳＳＡ／Ｐのそれぞれについて、学習用サンプルとなるデータを画像処理装置に与える。本実施形態に係る画像処理装置は、学習用サンプルとして、内視鏡画像から抽出した全ての細胞核の各特徴に加え、テクスチャ解析で得られたヒストグラムを与えることで、非腫瘍とＳＳＡ／Ｐとを識別することができる。 Here, at the time of classification, data serving as learning samples is given to the image processing apparatus for each of non-tumor, adenoma, cancer, and SSA / P. The image processing apparatus according to the present embodiment provides a histogram obtained by texture analysis in addition to each feature of all cell nuclei extracted from an endoscopic image as a learning sample, so that non-tumor and SSA / P Can be identified.
ＳＶＭの各データ点は、任意であるが、例えば、細胞核の全特徴パラメータの平均、標準偏差及び変動係数の要素全てと、テクスチャ解析によって算出される２５６個の要素全てとする。これにより、１つの内視鏡画像から合計約３００次元の要素をもつ複合ベクトルをデータ点として生成し、ＳＶＭに適用する。 The data points of the SVM are arbitrary, but are, for example, all elements of the average, standard deviation, and coefficient of variation of all feature parameters of the cell nucleus and all 256 elements calculated by texture analysis. As a result, a composite vector having a total of about 300 dimensional elements is generated as a data point from one endoscopic image, and applied to the SVM.
診断結果出力手順Ｓ１０３では、画像処理装置が、識別手順Ｓ１０２の分類結果を出力する。例えば、画像処理装置が、非腫瘍、腺腫、癌、ＳＳＡ／Ｐのうちのいずれかを診断結果として出力する。出力は、例えば、コンピュータに備わるモニタに表示する。また、分類結果を内視鏡画像とともにメモリに格納する。 In the diagnostic result output procedure S103, the image processing apparatus outputs the classification result of the identification procedure S102. For example, the image processing apparatus outputs any of non-tumor, adenoma, cancer, and SSA / P as a diagnostic result. The output is displayed on, for example, a monitor provided in the computer. The classification result is stored in the memory together with the endoscopic image.
以上の特徴量算出手順Ｓ１０１、識別手順Ｓ１０２及び診断結果出力手順Ｓ１０３を、内視鏡画像ごとに行う。これによって、専門的な知識と経験を有する者でなくとも、病理診断を行いたい各部位が、非腫瘍、腺腫、癌、ＳＳＡ／Ｐのいずれであるか判断することができる。特に、消化管などの体内での粘膜の内視鏡画像を内視鏡を用いて撮像することで、リアルタイム自動診断が可能となり、内視鏡検査のクオリティを向上することができる。 The above-described feature amount calculation procedure S101, identification procedure S102, and diagnosis result output procedure S103 are performed for each endoscope image. Thereby, even if it is not a person with specialized knowledge and experience, it can be judged whether each site | part which wants to perform pathological diagnosis is a non-tumor, an adenoma, cancer, or SSA / P. In particular, by taking an endoscopic image of the mucous membrane in the body such as the digestive tract using an endoscope, real-time automatic diagnosis is possible, and the quality of endoscopy can be improved.
なお、本実施形態に係る発明は、内視鏡画像だけでなく、生体から採取した組織を顕微鏡で撮像した顕微鏡画像に対しても適用することができる。 The invention according to the present embodiment can be applied not only to an endoscopic image but also to a microscope image obtained by imaging a tissue collected from a living body with a microscope.
また、本実施形態では、細胞核の特徴が、長径ＤＬ、短径ＤＳ、周囲長、面積、真円度及び色である場合について説明するが、本発明はこれに限定されない。例えば、離心率・弦節比・凹凸形状・フラクタル次元・線の集中度・濃度コントラストを含めてもよい。 In the present embodiment, the case where the features of the cell nucleus are the major axis DL, the minor axis DS, the perimeter, the area, the roundness, and the color will be described, but the present invention is not limited to this. For example, eccentricity, chordal ratio, uneven shape, fractal dimension, line concentration, and density contrast may be included.
また、本実施形態では、ＳＶＭを用いて分類を行う例を示したが、ＳＶＭ以外の任意のアルゴリズムを用いることができる。そのようなアルゴリズムとしては、例えば、ニューラルネットワーク、単純ベイズ分類器、決定木、クラスター解析、線形回帰分析及びロジスティック回帰分析がある。 In the present embodiment, an example in which classification is performed using the SVM is shown, but any algorithm other than the SVM can be used. Such algorithms include, for example, neural networks, naive Bayes classifiers, decision trees, cluster analysis, linear regression analysis and logistic regression analysis.
Claims (8)
前記内視鏡画像の全体の濃淡について、テクスチャ解析を行う機能と、
前記細胞核の特徴及び前記テクスチャ解析の結果を用いて、非腫瘍、腺腫、癌及びＳＳＡ／Ｐ（ｓｅｓｓｉｌｅ ｓｅｒｒａｔｅｄ ａｄｅｎｏｍａ／ｐｏｌｙｐ）のいずれであるのかの識別を行い、非腫瘍、腺腫、癌及びＳＳＡ／Ｐのうちのいずれかを診断結果として出力する機能と、
を備える画像処理装置。 Function to extract cell nuclei contained in endoscopic images obtained by magnifying cells of gastrointestinal mucosa irradiated with light through an endoscope, and measure the characteristics of the cell nuclei for all extracted cell nuclei When,
A function of performing texture analysis on the overall shade of the endoscopic image;
Using the characteristics of the cell nucleus and the result of the texture analysis, it is identified whether the tumor is a non-tumor, adenoma, cancer or SSA / P (sessile serrated adenoma / polyp), and the non-tumor, adenoma, cancer and SSA / P A function of outputting any of P as a diagnosis result;
An image processing apparatus comprising:
請求項１に記載の画像処理装置。 The endoscopic image is an image obtained by enlarging and imaging cells using an endoscope equipped with an optical microscope.
The image processing apparatus according to claim 1.
請求項１に記載の画像処理装置。 The endoscopic image is an image obtained by enlarging and imaging a cell using a confocal endoscope,
The image processing apparatus according to claim 1.
請求項１から３のいずれかに記載の画像処理装置。 The endoscopic image is an image obtained by enlarging and imaging stained cells.
The image processing apparatus according to claim 1.
画像処理装置が、前記内視鏡画像の全体の濃淡について、テクスチャ解析を行う手順と、
画像処理装置が、前記細胞核の特徴及び前記テクスチャ解析の結果を用いて、非腫瘍、腺腫、癌及びＳＳＡ／Ｐ（ｓｅｓｓｉｌｅ ｓｅｒｒａｔｅｄ ａｄｅｎｏｍａ／ｐｏｌｙｐ）のいずれであるのかの識別を行い、非腫瘍、腺腫、癌及びＳＳＡ／Ｐのうちのいずれかを診断結果として出力する手順と、
を行う画像処理方法。 The image processing device extracts the cell nuclei contained in the endoscopic image obtained by enlarging the cells of the digestive tract mucosa irradiated with light through the endoscope, and for all the extracted cell nuclei, A procedure for measuring features;
The image processing apparatus performs a texture analysis for the entire shade of the endoscopic image, and
The image processing apparatus identifies whether the tumor is a non-tumor, adenoma, cancer, or SSA / P (sessellated adenoma / polyp) using the characteristics of the cell nucleus and the result of the texture analysis. A procedure for outputting any of cancer and SSA / P as a diagnostic result;
An image processing method.
請求項５に記載の画像処理方法。 The endoscopic image is an image obtained by enlarging and imaging cells using an endoscope equipped with an optical microscope.
The image processing method according to claim 5.
請求項５に記載の画像処理方法。 The endoscopic image is an image obtained by enlarging and imaging a cell using a confocal endoscope,
The image processing method according to claim 5.
請求項５から７のいずれかに記載の画像処理方法。 The endoscopic image is an image obtained by enlarging and imaging stained cells.
The image processing method according to claim 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2015036771A JP6059271B2 (en) | 2015-02-26 | 2015-02-26 | Image processing apparatus and image processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2015036771A JP6059271B2 (en) | 2015-02-26 | 2015-02-26 | Image processing apparatus and image processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| JP2016154810A JP2016154810A (en) | 2016-09-01 |
| JP6059271B2 true JP6059271B2 (en) | 2017-01-11 |
Family
ID=56824370
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP2015036771A Active JP6059271B2 (en) | 2015-02-26 | 2015-02-26 | Image processing apparatus and image processing method |
Country Status (1)
| Country | Link |
|---|---|
| JP (1) | JP6059271B2 (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6580446B2 (en) * | 2015-10-09 | 2019-09-25 | サイバネットシステム株式会社 | Image processing apparatus and image processing method |
| EP3529591B1 (en) | 2016-10-20 | 2021-08-04 | Optina Diagnostics, Inc. | Method and system for detecting an anomaly within a biological tissue |
| WO2018158817A1 (en) | 2017-02-28 | 2018-09-07 | 日本電気株式会社 | Image diagnosis device, image diagnosis method, and program |
| JP7343394B2 (en) | 2017-12-05 | 2023-09-12 | ソニー・オリンパスメディカルソリューションズ株式会社 | Medical information processing device, information processing method, and medical information processing system |
| EP4024324A4 (en) | 2019-08-28 | 2022-10-26 | NEC Corporation | Image processing device, image processing method, and computer-readable recording medium |
| CN111067473A (en) * | 2019-11-20 | 2020-04-28 | 于天成 | Gastric precancerous resection boundary marking system and method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2001022741A2 (en) * | 1999-09-23 | 2001-03-29 | Nadeau Richard G | Medical applications of orthogonal polarization spectral imaging |
| JP2004317437A (en) * | 2003-04-18 | 2004-11-11 | Olympus Corp | Optical imaging apparatus |
| JP2006320367A (en) * | 2005-05-17 | 2006-11-30 | Olympus Medical Systems Corp | Distal end of endoscope |
-
2015
- 2015-02-26 JP JP2015036771A patent/JP6059271B2/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| JP2016154810A (en) | 2016-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6059271B2 (en) | Image processing apparatus and image processing method | |
| Sánchez-Montes et al. | Computer-aided prediction of polyp histology on white light colonoscopy using surface pattern analysis | |
| EP2405265B1 (en) | Diagnostic imaging support device, diagnostic imaging support method, and storage medium | |
| JP6265588B2 (en) | Image processing apparatus, operation method of image processing apparatus, and image processing program | |
| JP2004286666A (en) | Pathological diagnosis support apparatus and pathological diagnosis support program | |
| Ali et al. | Automatic label‐free detection of breast cancer using nonlinear multimodal imaging and the convolutional neural network ResNet50 | |
| Takemura et al. | Quantitative analysis and development of a computer-aided system for identification of regular pit patterns of colorectal lesions | |
| JP6580446B2 (en) | Image processing apparatus and image processing method | |
| US20230125377A1 (en) | Label-free real-time hyperspectral endoscopy for molecular-guided cancer surgery | |
| JP6743066B2 (en) | Tissue sample analysis technology | |
| Harris et al. | A pulse coupled neural network segmentation algorithm for reflectance confocal images of epithelial tissue | |
| Wang et al. | Optical coherence tomography and computer-aided diagnosis of a murine model of chronic kidney disease | |
| Guo et al. | Multimodal image analysis in tissue diagnostics for skin melanoma | |
| US12131461B2 (en) | Systems, methods, and media for automatically transforming a digital image into a simulated pathology image | |
| US11948296B2 (en) | Method and system for assessing fibrosis in a tissue sample | |
| JP6824868B2 (en) | Image analysis device and image analysis method | |
| Gupta et al. | Large-scale extraction of interpretable features provides new insights into kidney histopathology–A proof-of-concept study | |
| Bower et al. | A quantitative framework for the analysis of multimodal optical microscopy images | |
| JP6879520B2 (en) | Image processing device and image processing method | |
| JP2019013461A (en) | Probe type confocal laser microscopic endoscope image diagnosis support device | |
| Alshawwa et al. | [Retracted] Segmentation of Oral Leukoplakia (OL) and Proliferative Verrucous Leukoplakia (PVL) Using Artificial Intelligence Techniques | |
| Rana et al. | High accuracy tumor diagnoses and benchmarking of hematoxylin and eosin stained prostate core biopsy images generated by explainable deep neural networks | |
| US20230386660A1 (en) | System and method for detecting gastrointestinal disorders | |
| Saroja et al. | Colon Cancer Detection Methods–A Review | |
| Thekkek et al. | Quantitative evaluation of in vivo vital-dye fluorescence endoscopic imaging for the detection of Barrett’s-associated neoplasia |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A02 | Decision of refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A02 Effective date: 20151208 |
|
| A912 | Re-examination (zenchi) completed and case transferred to appeal board |
Free format text: JAPANESE INTERMEDIATE CODE: A912 Effective date: 20160624 |
|
| A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20161013 |
|
| A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20161208 |
|
| R150 | Certificate of patent or registration of utility model |
Ref document number: 6059271 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 |
|
| R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
| R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
| R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
| R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
| R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
| R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |