Skip to content

A project to make trash detection and reporting using aerial images a simpler task

Notifications You must be signed in to change notification settings

srikharshashi/trash-detection-docs

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 

Repository files navigation

Trash Detection and Reporting using Yolov5,ServiceNow and Drone Kit

Problem Statement

We aim to provide the higher authorities responsible for sanitation and cleaning of large areas or intuitions a method for evaluation the work of their subordinates without manually inspecting the areas which might be kilometers apart.

Solution

  • Aerial Images taken using a drone will be inferred via a customyolov5 model which was trained on images of trash
  • If the number of detections are more than a set threshold an alert is sent to the authorities in the form of an Incident in ServiceNow
  • The inferred images are also stored in Amazon S3 with a reference of them in the database for reference
  • The images corresponding to a given flight can be viewed via our Web Client

Workflow

Untitled Diagram drawio

Drone Command App

  • The Drone Command app can do the following tasks
    1. Add new missions :- It uses GoogleMaps API to facilitate the user to add new missions for the drone to perform and also stores them in MongoDB
    2. Command the drone :- The app uses WebSocket API to command the ground station to perform a flight based on the mission chosen by the user
    3. Live Tracking :- The user can track the drone on a map as it completes it's missions to track it's progress live using the same Web Socket connection.
  • Tech Stack :- Flutter,Websockets API,REST API
  • Repository :- https://github.com/srikharshashi/drone-control dronecomp

Dronekit

  • DroneKit is a python API for communicating with the ground station and UAV's which wraps over the ArduPilot API for flight control leveraging the MAVLINK protocol
  • The Flight Controller we used (PixHawk) can communicate with the Onboard Computer as well as the Companion Computer on the drone.
  • The Onboard Computer enables the drone to click pictures programmatically when a way point is hit
  • Camera used: GoPro Hero 9 Black
  • The Ground station connects to the main server with a WebSocket connection
    1. Its listens for LAUNCH command
    2. It broadcasts it's location and it's update it's arming status to the status
  • Once a mission is completed it creates a flight in the server for the given mission id
  • The flight_id returned is then used for future purposes in Trash Detection Client
  • Simulator used : Ardupilot Simulator (for local development)
  • Tech Stack :- Ardupilot SITL,Dronekit SDK,Websockets using asyncio
  • Repository :- https://github.com/srikharshashi/dronekit_websockets simulator

Trash Detection Client

  • Trash Detection Client is a GPU enabled client for detecting trash from the images obtained by the drone
  • The script which is run on a folder of images accepts a flight id and performs inference on those images and updates the flight details in the database based on number of inference
  • The custom yolov5 trained model has an accuracy for 56%
  • It also uploads these images to Amazon S3 and preserves a reference URL to them
  • And finally after inference is done it creates an incident in Service Now image

image

Backend

  • The backend is an expressjs server with a MongoDB isntance hosted on Atlas .
  • The hosting used is Amazon EC2 Free Tier paired with Amazon S3 as an iage storage service
  • It acts a CRUD server as well as a WebSocket Server
  • Resources
    1. Missions
    2. Flights
    3. Images
  • It's responsible for the fetching the above resources ,location tracking as well as drone command
  • Repository : https://github.com/srikharshashi/Mission-Store

Web Application

About

A project to make trash detection and reporting using aerial images a simpler task

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published