Skip to content

Aim of this repository is to create software that enables humans to interact with computer without mouse

License

Notifications You must be signed in to change notification settings

datonefaridze/mouse_controller_from_screen

Repository files navigation

# Mouse Controller:

Aim of this repository is to create software that enables humans to interact with computer without mouse.

Repo contains script to run training and script which is responsible for user to interact with computer without mouse

model was trained on 60 epochs and It showed very high accuracy so I decided to not continue training after 60 epochs

How to interact with computer:

once you run the mosue_from_screen.py script you have to select finger which you want to track, after that you can take your finger and slide it across the air, at that time mouse will follow your finger directions. when you want to click on something you just have to show 5 fingers to the camera, if you show nothing or your fist it doesn't generate click, if you show 5 fingers it will generate click.

Future Plans:

Add action recognition to the code, currently it clicks if you show your five fingers, it would be more convinient if it would generate click on action like you do on smartphone screen. also this repo is tested on Windows, so in future i want to use Docker to make it accessible to all users

Here is demo how it performs:

grab-landing-page

About

Aim of this repository is to create software that enables humans to interact with computer without mouse

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages