This was a project I worked on in my third year of engineering ( June 2019 ) I used Arduino along with 2 Ultrasonic sensors. using a combination of these I was able to perform 4 actions without touching the computer ( by using the position of my hand ).The application was based on the relative position of my hand with respect to the 2 sensors located on the left and right side of the computerscreen.
Process.
-
The sensors track the obstacles ( Hand ) in front of it at regular intervals of time. each sensor calculate the distance of my hand if its beyond a certain limit ( 50 - 70CM ) its considered far, if its between 0-25 cm, its considered near.
-
depending on the distance of each hand a total of 8 actions can be performed ( L(left hand) - near & R- near , L - far & R - far , L - near & R - far, L - far & R - near, only L near, only R near, only L far, only R far ) .
-
Every combination can correspond to a different action on the computer. ( Each combination simulates a particular set of keys pressed on the keyboard , ex if left hand and right hand both are far, we can map it to ( ctrl + A ). This is done using the pyautogui module.
-
Using these gestures I programmed VLC media player and Photos app to perform various functions ( next photo, prev photo, zoom in, zoom out, increase/decrease volume, ext track , previous track , adjust brightness, rewind , forward )
-
The application cound also track if the hand is moving far or near to the computer. this is then used for applications such as increasing/decreasing the volume if the hand comes near / goes away from the screen respectively.
Tech and software packages used :
-
Aurdrino
-
Python
-
2 - Ultrasonic sensors
-
Python
-
pyautogui ( To simulate key presses ) and pyserial module ( to get the data from the sensors ) ( python modules )