Skip to content

๐Ÿ‘€ ๋„ˆ์˜ ์•Œ์•ฝ์ด ๋ณด์—ฌ ๐Ÿ’Š : ์•Œ์•ฝ ์ด๋ฏธ์ง€ ๋ถ„๋ฅ˜ ํ”„๋กœ์ ํŠธ

Notifications You must be signed in to change notification settings

boostcampaitech3/final-project-level3-cv-16

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

93 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

๋„ˆ์˜ ์•Œ์•ฝ์ด ๋ณด์—ฌ - by. Team Medic(CV-16)


๐Ÿ“šย Project Overview

๐Ÿ‘€ย ๋„ˆ์˜ ์•Œ์•ฝ์ด ๋ณด์—ฌ ๐Ÿ’Š

  • Metric Learning์„ ํ™œ์šฉํ•œ Reverse Pill Image Search
  • streamlit ์‹คํ–‰ ์˜ˆ์‹œ

แ„‘แ…ณแ„…แ…ฉแ„Œแ…ฆแ†จแ„แ…ณ แ„‰แ…ตแ„‹แ…งแ†ซ.gif

๐Ÿ˜Žย Members

๊ถŒ์ˆœํ˜ธ ์„œ๋‹ค๋นˆ ์„œ์˜ˆํ˜„ ์ด์ƒ์œค ์ „๊ฒฝ๋ฏผ
๊ถŒ์ˆœํ˜ธ ์„œ๋‹ค๋นˆ ์„œ์˜ˆํ˜„ ์ด์ƒ์œค ์ „๊ฒฝ๋ฏผ

๐Ÿค—ย Contribution

  • ๊ถŒ์ˆœํ˜ธ: FastAPI, BentoML, streamlit, GCP, OCR, Text Recognition
  • ์„œ๋‹ค๋นˆ: FastAPI, streamlit, OCR, Text Recognition
  • ์„œ์˜ˆํ˜„: Data EDA, Data Pre-processing, Image Classification, Custom Dataset Production
  • ์ด์ƒ์œค: Metric learning, Segmentation, Database, Docker
  • ์ „๊ฒฝ๋ฏผ: Data EDA, Data Pre-processing, Data Annotation, OCR, Text Recognition

โ“ย About This Project

Purpose

  • ์‚ฌ์šฉ์ž์˜ ์•Œ์•ฝ ์ด๋ฏธ์ง€๋กœ๋ถ€ํ„ฐ ์•Œ์•ฝ์„ ์‹๋ณ„ํ•˜๋Š” ์ธ๊ณต์ง€๋Šฅ ์„œ๋น„์Šค

Objective

  • ์‚ฌ์šฉ์ž์˜ ์•Œ์•ฝ ์ด๋ฏธ์ง€๋กœ๋ถ€ํ„ฐ ์„ฑ์ƒ, ์ œํ˜•, ์ƒ‰์ƒ์„ ์‹๋ณ„ ํ›„ ์กฐ๊ฑด์— ๋งž๋Š” ์•Œ์•ฝ์„ ๊ฒ€์ƒ‰ํ•˜์—ฌ ์•Œ์•ฝ์˜ ์ข…๋ฅ˜๋ฅผ ์‹๋ณ„ํ•œ๋‹ค.

Target Audience

  • ์ง€๋ฆฌ์ , ๋ฌผ๋ฆฌ์  ํ•œ๊ณ„๋กœ ์•ฝ๊ตญ์ด๋‚˜ ๋ณ‘์›์„ ๋ฐฉ๋ฌธํ•˜๊ธฐ ์–ด๋ ค์šด ์‚ฌ๋žŒ
  • ์•Œ์•ฝ์€ ์žˆ์ง€๋งŒ ์•Œ์•ฝ์„ ๊ตฌ๋ถ„ํ•  ์ˆ˜ ์—†๋Š” ์‚ฌ๋žŒ

Background Information

  • ์ข…์ข… ์ผ์–ด๋‚˜๋Š” ์ฒ˜๋ฐฉ ์‹ค์ˆ˜, ๋ฐ ์ฐฉ๊ฐ์œผ๋กœ ์ธํ•œ ์•ฝ๋ฌผ์‚ฌ๊ณ ๋ฅผ ์˜ˆ๋ฐฉํ•˜๊ณ ์ž ํ•˜์˜€๋‹ค.
  • ์‹ค์ œ ๋ณด๊ฑด ๊ณ„์—ด ์ข…์‚ฌ์ž์— ๋”ฐ๋ฅด๋ฉด ๋…ธ๋…„ ์ธต์˜ ๊ฒฝ์šฐ ์–ด๋–ค ์•Œ์•ฝ์ธ์ง€ ๋ณ‘์›์— ๋ฐฉ๋ฌธํ•˜์—ฌ ์•Œ์•ฝ์„ ์ฐพ๋Š” ๊ฒฝ์šฐ๊ฐ€ ์กด์žฌํ•œ๋‹ค๊ณ  ํ•˜๋ฉฐ, ํ•œ๊ตญ์˜ ํ†ต๊ณ„๋ฅผ ๋ณด์•˜์„ ๋•Œ๋„ ์•ฝ๋ฌผ ์˜ค๋ณต์šฉ์— ์˜ํ•œ ์‚ฌ๊ณ ๋Š” ์ค„์ง€ ์•Š๊ณ  ๊ณ„์† ์œ ์ง€๋˜๊ณ  ์žˆ๋Š” ์ถ”์„ธ์ด๋‹ค.

๐Ÿ—บย Service Architecture

Service Architecture

๐Ÿ’พย Datasets

  • ์˜์•ฝํ’ˆ ์•ˆ์ „๋‚˜๋ผ ๋ฐ์ดํ„ฐ (Link)
  • ePillID Benchmark (Link)
  • ๊ธฐํƒ€ ์ด๋ฏธ์ง€ ๋ฐ์ดํ„ฐ (Link)
    • Classification ๋ฐ Object Detection์„ ์œ„ํ•ด ์ง์ ‘ ์ดฌ์˜ ๋ฐ ์ˆ˜์ง‘ํ•œ, ๋ผ์ด์„ผ์Šค๊ฐ€ ์—†๋Š” ์ด๋ฏธ์ง€๋“ค

๐Ÿ’ปย Development Environment

  • GPU: Tesla V100
  • OS: Ubuntu 18.04.5LTS
  • CPU: Intel Xeon
  • Python : 3.8.5 / 3.9.13

๐Ÿ“ย Project Structure (Main branch)

final-project-level3-cv-16
โ”œโ”€ api_folder
โ”‚   โ”œโ”€ .streamlit
|   |   โ””โ”€ config.toml
โ”‚   โ”œโ”€ backend
|   |   โ”œโ”€ epillid_benchmark(cloned from Link)
|   |   โ”œโ”€ Dockerfile
|   |   โ”œโ”€ Backend.py
|   |   โ””โ”€ requirements.txt
โ”‚   โ”œโ”€ frontend
|   |   โ”œโ”€ Dockerfile
|   |   โ”œโ”€ frontend.py
|   |   โ””โ”€ requirements.txt
โ”‚   โ””โ”€ Docker
|       โ””โ”€ docker-compose.yml  
โ””โ”€ image_classification
    โ”œโ”€ data_preprocessing
    |   โ”œโ”€ download_pill_data.py 
    |   โ””โ”€ normalize_pill_data.py
    โ”œโ”€ image_concatenation
    |   โ””โ”€ concatenation_images.py
    โ”œโ”€ kaggle_pill_data_preprocessing
    |   โ”œโ”€ 1_annotation_file_name_to_txt.py
    |   โ”œโ”€ 2_edit_xml_path.py
    |   โ””โ”€ 3_xml_to_json.py
    โ”œโ”€ pill_excel_data
    |   โ””โ”€ README.md
    โ”œโ”€ .gitignore
    โ”œโ”€ data.py
    โ”œโ”€ dataset.py
    โ”œโ”€ log.py
    โ””โ”€ train.py

โœ๏ธย Evaluation

  • Top-1 accuracy: 43%
  • Top-5 accuracy: 80%

๐Ÿš€ย How to Start

  1. Image Classification: Link

  2. OCR: Link

  3. Object Detection (yolov5): Link

  4. Metric learning: Link

๐Ÿ”Žย Future Research

  • ๋ชจ๋ธ์˜ ์ •ํ™•๋„ ํ–ฅ์ƒ ๋ฐ inference time ๋‹จ์ถ•
  • Mobile Application ์ œ์ž‘
  • ์‹ค์šฉ์„ฑ ํ–ฅ์ƒ
  • OCR ์ ์šฉ

๐Ÿ“Žย Appendix

๐Ÿ“„ย Experiments & Submission Report

๐Ÿ“œย Reference

  • ePillID Dataset: A Low-Shot Fine-Grained Benchmark for Pill Identification (Link)
  • YOLACT: Real-time Instance Segmentation (Link)
  • How to make deep-text-recognition-benchmark model to recognize both Korean and English (Link)
  • OCR Model (Link)
  • Jaccard Similarity (Link)
  • Text-Recognition Model (Link)
  • Background-Removal program (Link)
  • Object Detection model YOLOv5 (Link)
  • timm (Link)

About

๐Ÿ‘€ ๋„ˆ์˜ ์•Œ์•ฝ์ด ๋ณด์—ฌ ๐Ÿ’Š : ์•Œ์•ฝ ์ด๋ฏธ์ง€ ๋ถ„๋ฅ˜ ํ”„๋กœ์ ํŠธ

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •