Gesture Controlled Robotics- the Future of Haptic Communication Has Arrived

Alter comes back home from office after spending tiring hours. Standing at the door, he uses his fingers to draw an invisible pattern in the air to pass the security and enter his apartment. He leaves his bag by the bedside and flicks his wrist to turn on the heating, and then settles down to watch his favourite web series, pointing to pause, tapping to rewind and waving to scroll. A little while later, the lights of the seating area are turned off by a nod of his head, and Alter heads for a well-deserved rest.

Am I talking about a scene from the latest Science-Fiction movie? No, it's the future of communication of the Haptic Technology controlled by Gesture Controlled Robotics.

The Future of Communication is Here

Gesture control has been around for quite some-time holding a significant impact on the computing industry where audiences can interact with presentations; users can input text into their devices waving mid-air and service robots deploy image processing to be gesture control enabled.

The proliferation of touch interfaces is not new; of late there has been an increasing interest of the human mind to understand the consumer appeal of the improved User Interfaces, enabled through a variety of methods like 3D sensors, wearable devices and cameras. Gesture control is driving emergent technologies like wearable computing and Augmented Reality (AR) and is currently the most intuitive human-machine interface available to users, developers and manufacturers.

Gesture control is not restricted to waving controls in mid-air, with the passage of time; it has spread its wings to cover Vision, Touch and 3D Cameras.

Touch-Based Gesture Recognition

Wearable devices create the most viable opportunity for the industry adoption of gesture control as individuals are no longer restricted by the need to be near a stationary computing device for their work or business. Examples of Touch Based Recognition include, surgeons who can use gesture control to study and diagnosis details to flick through X-ray reports and patient profiles through during procedures.

Another instance may include engineers who can control various moving parts of a machine during repair and maintenance. Combined with augmented reality, gesture control also offers immense opportunities for education and training. Gesture control has revolutionized education too, where students can now model, manipulate and design 3D objects onscreen under the guidance of their tutor leveraging educators to combine theory with practice.

Vision-Based Gesture Recognition

Deploying a generic camera and/or range camera to capture and derive the hand gesture is termed as Vision-based gesture recognition. This technology requires higher processing power compared to a wired glove. There are multiple methods for camera-based gesture recognition, like that includes using a conventional 2D camera, simple gesture recognition can be implemented with functions provided by open source computer vision libraries, like OpenCV library or commercial computer vision libraries. The vision-based gesture recognition uses skin tone detection to detect hands in a constrained area and can then detect convex and defect points of hands.

Voice-Based and Robot/Drone Control

Voice control is one way for Haptic Communication but it may not be desirable in certain situations like when you have a bad sore throat one day, and cannot speak!!

Drone manufacturers like DJI are developing photo-taking drones that can fly autonomously from the user's hand and return back without using any remote control. In the coming years, hand gestures will be the viable way to guide drone operations outdoor, like summoning the drone back by waving hands.

3D Cameras

3D cameras which can measure depth have been broadly available and cheaper in recent years. In 2010, Kinect V1 motion controller was released by Microsoft, which used technology from PrimeSense. Kinect V1 provides strong three-dimensional body and hand motion capture capabilities in real-time thus freeing game players from physical input devices like joysticks and keyboards. In addition, Kinect supports multiple users in a small room setting and engages non-gamers of different ages to actively participate in fun games like dancing and sports.

The potential of gesture control technology has been recognized by Industry leaders across verticals which are reflected in recent developments including Apple's patent for a gesture-based UI designed to control 3D objects and Google's acquisition of Y-combinator backed gesture control startup, Flutter.

The future of Haptic Communication has indeed arrived.