I also like to tackle solutions that use both hardware and software. In this case, we created HandEEEman, a gesture-based controller!

Introduction

Back in my Wireless Sensor Networks class, my team wanted to create something simple yet useful. We wanted to create something that can help people with everyday operations. Later on, we identified that we wanted to help Persons with Disabilities (PWDs) who had a limited range of movement. Narrowing down this scenario, gesture control using hand movements was chosen. We wanted to create a device that can control other devices with simple movements. This will help people with limited movement to still use their hands for control. Thus, HandEEEman was born!

While this post is not meant to be a full recipe on how to rebuild HandEEEman, I hope it still gives insights on gesture detection using off-the-shelf components and perhaps help/inspire others to think of various exciting applications with such parts.

Parts

The main idea was to use an Arduino and other parts to sense gestures and transmit them to another device. To sense the gestures, we used the MPU-6050 module. It contains an accelerometer and gyroscope, which we could use to detect specific hand gestures. Due to its relatively low cost (and that we had a spare MPU-6050 on hand at the time), experimenting with it became more attractive for my group.

As for transmitting the gesture data, we used the NRF24L01 module because of its low cost. One finding we learned is that we could place a 10uF capacitor between the VCC and Ground pins of the module. This improves the stability of the supplied voltage to the module, reducing the chances of randomly restarting or freezing.

Setup

Setup of system. Photo courtesy of HandEEEman

Our setup used a pair of Arduino boards. One board serves as the transmitter. It collects orientation data and transmits commands based on the orientation to the receiver board. The other board serves as a receiver. It receives the transmitted command to set the brightness of a "bulb": an LED.

Transmitter

The transmitter consists of an Arduino board, an MPU-6050 module, an NRF24L01 module, and a glove. The orientation of the glove is measured with the MPU-6050 module. The time difference between orientations is also recorded. Based on the measured orientations and time differences, a command is formed.

Our transmitter was attached to a glove, with tape 😅 The main idea was to use the glove to control the light bulb ala-Iron Man 🤖

Iron Man glove-err.. HandEEEman transmitter 😆 Photo courtesy of HandEEEman

Receiver

The receiver consists of an Arduino board, an NRF24L01 module, and an LED and resistor as the "appliance".

The "appliance" to be controlled. Photo courtesy of HandEEEman

Commands

With the transmitter and receiver set up, now it's time to define commands. For testing, the HandEEEman can process two commands: lock and brightness. These commands were used to control the test "appliance" (an LED connected to the receiver)

Brightness

To set the brightness of the appliance, the user can rotate their hand clockwise or counter-clockwise. It's like turning an imaginary dial around.

Lock

To lock the brightness of the appliance, we needed another simple gesture that was different from rotating one's hand. In this case, we added a downwards gesture as a "flick" to lock the brightness.

In hindsight, perhaps a feature for saving a custom gesture for locking (and other actions) can be added. After all, different people have different needs and ways of physically expressing things.

Demo

We recorded a video of our prototype (we needed documentation for class). We held our breaths a bit for this one (and the live demo that same day 😅). We also recorded the gestures in action.

HandEEEman in action! Video courtesy of HandEEEman

Wrapping Up

I hope you had fun reading this post and perhaps even get some inspiration for your next project. As always, stay tuned to my blog for new posts and updates 🍷

Acknowledgements

This project would not have seen the light of day without my brilliant teammates: Norman, Marlo, Gab, and Jann Marc. Great engineers and I wish them the best in their careers and endeavours 🥂 😎

Images and videos came from the official HandEEEman website. It also contains a bit more info on the project itself 🤓

# hardware

Remote-controlled "Nerf" targets

AutoFaucet - The Contactless Water Dispenser

HandEEEman