Wristband Enables Wearers to Control Robotic Hand
This wearable device uses ultrasound technology to control robotic movements. Photo courtesy Massachusetts Institute of Technology
CAMBRIDGE, MA—Engineers at the Massachusetts Institute of Technology here have developed a wearable device that can control robotic movements. By moving their hands and fingers, users can direct a robot to play piano or shoot a basketball, or they can manipulate objects in a virtual environment.
The ultrasound-based wristband precisely tracks a wearer’s hand movements in real-time. It produces images of the wrist’s muscles, tendons and ligaments as the hand moves, and is paired with an artificial intelligence algorithm that continuously translates the images into the corresponding positions of the five fingers and palm.
A person wearing the wristband can wirelessly control a robotic hand. As the person gestures or points, the robot does the same.
“We think this work has immediate impact in potentially replacing hand tracking techniques with wearable ultrasound bands in virtual and augmented reality,” says Xuanhe Zhao, Ph.D., a mechanical engineering professor at MIT who is heading up the R&D project. “It could also provide huge amounts of training data for dexterous humanoid robots.”
According to Zhao, human hands are incredibly complex devices. The seemingly mundane task of scrolling through a smartphone screen, for example, requires the coordination of 34 muscles, 27 joints, and more than 100 tendons and ligaments. Mimicking their many nuanced gestures has been a longstanding challenge in robotics and virtual reality.
There are currently a number of approaches to capturing and mimicking human hand dexterity in robots. Some approaches use cameras to record a person’s hand movements as they manipulate objects or perform tasks.
Others involve having a person wear a glove with sensors, which records the person’s hand movements and transmits the data to a receiving robot. But, erecting a complex camera system for different applications is impractical and prone to visual obstacles. And sensor-laden gloves could limit a person’s natural hand motions and sensations.
Looking for quick answers on assembly and manufacturing topics? Try Ask ASM, our new smart AI search tool. Ask ASM
To address those challenges, Zhao and his colleagues experimented with various types of ultrasound stickers— miniaturized versions of the transducers used in doctor’s offices that are paired with hydrogel material that can safely stick to skin. They designed a wristband with an ultrasound sticker that is the size of a smartwatch, and added onboard electronics that are about as small as a cellphone.
The engineers are using the wristband to gather hand motion data from multiple users with different hand sizes, finger shapes and gestures. They envision building a large dataset of hand motions that can be plumbed, for instance, to train humanoid robots in dexterity tasks.
“We believe this is the most advanced way to track dexterous hand motion, through wearable imaging of the wrist,” says Zhao. “We think these wearable ultrasound bands can provide intuitive and versatile controls for virtual reality and robotic hands.”
Looking for a reprint of this article?
From high-res PDFs to custom plaques, order your copy today!
