This project leverages deep learning for the classification of kidney disease using a combination of MLflow and DVC for experiment tracking, versioning, and orchestration. The project encompasses a structured workflow, facilitating both experimentation and production-grade deployment.
- config.yaml: General configuration settings.
- secrets.yaml: Optional file for sensitive information.
- params.yaml: Parameters for model training and evaluation.
- Update the entity definitions.
- Enhance the configuration manager in the
src
config directory. - Revise the components for improved functionality.
- Update the pipeline to reflect changes.
- Modify
main.py
to incorporate new logic. - Amend
dvc.yaml
for Data Version Control.
app.py
: Main application file to run the project.
git clone https://github.com/themihirmathur/Kiddis.git
cd Kiddis
conda create -n cnncls python=3.8 -y
conda activate cnncls
pip install -r requirements.txt
python app.py
Now, open your local host and port to access the application.
MLflow is used to track experiments, log models, and facilitate model deployment.
mlflow ui
export MLFLOW_TRACKING_URI=https://dagshub.com/entbappy/Kiddis.mlflow
export MLFLOW_TRACKING_USERNAME=entbappy
export MLFLOW_TRACKING_PASSWORD=6824692c47a369aa6f9eac5b10041d5c8edbcef0
python script.py
DVC is utilized for lightweight experiment tracking and pipeline orchestration.
dvc init
dvc repro
dvc dag
- Production Grade: Suitable for production environments.
- Experiment Tracking: Logs and tags models and experiments.
- Lightweight: Ideal for proof-of-concept projects.
- Experiment Tracker: Manages and tracks experiments.
- Pipeline Orchestration: Enables creation and management of pipelines.
This section describes the steps to deploy the application using AWS services and GitHub Actions for continuous integration and continuous deployment.
- Login to AWS Console: Access your AWS management console.
- Create IAM User for Deployment: Assign specific access permissions:
- EC2 Access
- ECR Access
- Build Docker Image: Create a Docker image of the source code.
- Push Docker Image to ECR: Store the Docker image in AWS Elastic Container Registry (ECR).
- Launch EC2 Instance: Create an EC2 instance to host the application.
- Pull Docker Image from ECR: Retrieve the Docker image from ECR in the EC2 instance.
- Run Docker Image in EC2: Deploy and run the Docker container in the EC2 instance.
AmazonEC2ContainerRegistryFullAccess
AmazonEC2FullAccess
aws ecr create-repository --repository-name kidney-disease-classification
- Launch EC2 Instance (Ubuntu)
- Install Docker on EC2
sudo apt-get update -y sudo apt-get upgrade curl -fsSL https://get.docker.com -o get-docker.sh sudo sh get-docker.sh sudo usermod -aG docker ubuntu newgrp docker
- Go to
Settings > Actions > Runners
in GitHub. - Add a new self-hosted runner and follow the instructions to configure it.
AWS_ACCESS_KEY_ID=<your_aws_access_key_id>
AWS_SECRET_ACCESS_KEY=<your_aws_secret_access_key>
AWS_REGION=us-east-1
AWS_ECR_LOGIN_URI=566373416292.dkr.ecr.us-east-1.amazonaws.com
ECR_REPOSITORY_NAME=kidney-disease-classification
This project demonstrates a comprehensive approach to developing, experimenting, and deploying a deep learning model for kidney disease classification using modern MLOps tools and practices.