This project aims to build a robust, scalable, and distributed face recognition system that leverages the power of Kubernetes and Docker. Designed to efficiently handle face recognition demands across various applications such as security and identity verification.
- Scalability: Handles an increasing number of requests without performance degradation.
- Distributed Architecture: Distributes computational load across multiple nodes.
- High Availability: Ensures minimal downtime and continuous operation.
- Modularity: Containerizes each component, simplifying deployment and scaling.
- User-friendly Interface: Features an intuitive interface for easy user interaction.
- Kubernetes: Manages and orchestrates the containerized applications.
- Docker: Provides containerization for each system component.
- Face Recognition Algorithms: Utilizes state-of-the-art algorithms for high accuracy.
- AWS: Utilized for deploying the application in Amazon Elastic Kubernetes Service (EKS), facilitating scalable and publicly accessible services.
This guide provides detailed instructions on how to set up and run the project both locally and on Kubernetes.
Click to expand!
First, clone the repository to your local machine:
git clone https://github.com/yourusername/your-repo.git
cd your-repo
Navigate to the Auth, MTCNN-detector, Recognition and Training Folders Create and activate a virtual environment to isolate the project dependencies:
python3 -m venv env
# On Windows
env\Scripts\activate
# On macOS and Linux
source env/bin/activate
Install the required packages using pip:
pip install -r requirements.txt
4. create a mongodDB free tier account and get the MongoDB URI, Database Name and Collections named as below
Training_Data # required as it is
users # required as it is
openssl rand -hex 32 #provides a random hex string
Add the following variables and their respective values to an .env
file:
SECRET_KEY='your_secret_key_here'
MONGO_URI='your_mongodb_uri_here'
DATABASE_NAME='your_database_name_here'
Now there are main.py and client.py files in Auth MTCNN-detector Recognition and Training folders you can run thoes files by activating the virtual environments. First run the Auth/main.py and perform SignUp and Login for a valid 1 hour token. Then only you can acess the other services.
The Password is full encrypted using the SECRECT_KEY we generated. Make sure it is not public.
Start the application locally:
python main.py
IMP: Stay on Camera Display Window
key c to capture the crop face
key s to save the captured Data
key ESC to exit the program
Click to expand!
First create a secret using the .env which configured in the step 1
# navigate to the folder where .env exists
kubectl create secret generic my-env-secret --from-env-file=.env
use the below yaml file to deploy the services in local cluster
---
apiVersion: v1
kind: Namespace
metadata:
name: facerec
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: crop-face-deployment
namespace: facerec
spec:
replicas: 3
selector:
matchLabels:
app: crop-face
template:
metadata:
labels:
app: crop-face
spec:
containers:
- name: crop-face
image: santoshburada/crop_face:latest_multiArch
ports:
- containerPort: 8001
envFrom: # Use all keys in the secret as environment variables
- secretRef:
name: my-env-secret
---
apiVersion: v1
kind: Service
metadata:
name: crop-face-service
namespace: facerec
spec:
type: NodePort
ports:
- port: 8001
targetPort: 8001
nodePort: 30001
selector:
app: crop-face
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: train-face-deployment
namespace: facerec
spec:
replicas: 3
selector:
matchLabels:
app: train-face
template:
metadata:
labels:
app: train-face
spec:
containers:
- name: train-face
image: santoshburada/train_face:latest-multiArch
ports:
- containerPort: 8003
envFrom: # Use all keys in the secret as environment variables
- secretRef:
name: my-env-secret
---
apiVersion: v1
kind: Service
metadata:
name: train-face-service
namespace: facerec
spec:
type: NodePort
ports:
- port: 8003
targetPort: 8003
nodePort: 30003
selector:
app: train-face
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: face-rec-deployment
namespace: facerec
spec:
replicas: 3
selector:
matchLabels:
app: face-rec
template:
metadata:
labels:
app: face-rec
spec:
containers:
- name: face-rec
image: santoshburada/face_rec:amd64
ports:
- containerPort: 8005
envFrom: # Use all keys in the secret as environment variables
- secretRef:
name: my-env-secret
---
apiVersion: v1
kind: Service
metadata:
name: face-rec-service
namespace: facerec
spec:
type: NodePort
ports:
- port: 8005
targetPort: 8005
nodePort: 30005
selector:
app: face-rec
kubectl apply -f <your-file-name>
Now before running the client files please change the url's of each client file with the respective services in the k8s cluster
The url must be in the below format
http:https://<your-laptop-ip-address>:<NodePort of the services>/path
# below is the example change the IP address wih your Ip
1) For Mtcnn which listens on NodePort 30001 in the k8s cluster
send_url = 'http:https://<Ip-address>:30001/crop_face_mtcnn'
2) For traning which listens on NodePort 30003 in the k8s cluster
send_url = 'http:https://<Ip-address>:30003/process_images'
3) For Recognition which listens on NodePort 30005 in the k8s cluster
send_url = 'http:https://<Ip-address>:30003/recognize'
Click to expand!
Setting up on Cloud Providers. Below is example for AWS, same can be followed in any other Cloud Providers with few minor changes in the commands**
First configure the EKS cluster. Navigate to AWS Cli and paste below command
eksctl create cluster --name=final-cluster --region=us-east-1 --nodegroup-name=final-cluster-nodegroup --node-type=t3.2xlarge --nodes=3 --nodes-min=3 --nodes-max=5 --managed
The above command will create a 3 managed worker nodes EKS Cluster. I used a t3.2xlarge for better performace and speed. I suggest minumum 2 cores of cpu. It will take atleast 15-30 minitues
# for ARM systems, set ARCH to: `arm64`, `armv6` or `armv7`
ARCH=amd64
PLATFORM=$(uname -s)_$ARCH
curl -sLO "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_$PLATFORM.tar.gz"
# (Optional) Verify checksum
curl -sL "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_checksums.txt" | grep $PLATFORM | sha256sum --check
tar -xzf eksctl_$PLATFORM.tar.gz -C /tmp && rm eksctl_$PLATFORM.tar.gz
sudo mv /tmp/eksctl /usr/local/bin
Execute below command in the AWS Cli
kubectl apply -f https://github.com/santosh-burada/FaceRecog-API/blob/master/Deploy.yaml
aws iam list-open-id-connect-providers | grep $oidc_id | cut -d "/" -f4\n
If not, run the below command
eksctl utils associate-iam-oidc-provider --cluster final-cluster --approve
curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/install/iam_policy.json
aws iam create-policy \
--policy-name AWSLoadBalancerControllerIAMPolicy \
--policy-document file:https://iam_policy.json
eksctl create iamserviceaccount \
--cluster=final-cluster \
--namespace=kube-system \
--name=aws-load-balancer-controller \
--role-name AmazonEKSLoadBalancerControllerRole \
--attach-policy-arn=arn:aws:iam::<your-aws-account-id>:policy/AWSLoadBalancerControllerIAMPolicy \
--approve
Add helm repo
helm repo add eks https://aws.github.io/eks-charts
Update the repo
helm repo update eks
Install aws alb in the cluster. Add you vpcId
of the cluster which is under Networking section of the cluster.
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
-n kube-system \
--set clusterName=final-cluster \
--set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controller \
--set region=us-east-1 \
--set vpcId=<your-vpc-id>
kubectl get deployment -n kube-system aws-load-balancer-controller
wait until the deployments are running. Once done execute below command to get the exposed domain to access the services inside the cluster.
kubectl get ingress ingress-facerec -n facerec
Now copy the value under Address
column and replace your IP's
and No need of ports
in the client.py files
# example format
http:https://<AWS-Address>/crop_face_mtcnn
http:https://<AWS-Address>/process_image
http:https://<AWS-Address>/recognize
Distributed under the MIT License. See LICENSE
for more information.
For support, collaboration, or issues, please email at [email protected].
Thank you for visiting the project!