With the whirl of a thumb, Georgia Tech researchers have created a simple device that lets people trace letters and numbers on their fingers and see those alphanumerics appear on a computer screen. The computer is triggered by a thumb ring outfitted with a gyroscope and tiny microphone. As wearers strum their thumb across the fingers, the hardware detects the movement and converts it to keyboard inputs.
In the future, the device, called Fingersound, could be used to send phone calls, voicemail or answer text messages, or write the great American novel—all without the wearer reaching for a phone or a keyboard. It could also be a boon for users with certain disabilities.
“When a person grabs their phone during a meeting, even if trying to silence it, it can be distracting,” says Thad Starner, a professor at Georgia Tech’s School of Interactive Computing who’s leading the project. “But if they can simply send the call to voicemail, perhaps by writing an ‘x’ on their hand below the table, there isn’t an interruption.” Starner also says the technology could be used in virtual reality, replacing the need to take off a head-mounted device to input commands via a mouse or keyboard.
Here are some of the motions FingerSound recognizes as numbers or directions.
Although other gesture-based devices require users to perform gestures in the air, Fingersound uses the fingers as a canvas. This lets the ring clearly recognize the beginning and end of intended gestures by using the microphone and gyroscope to detect the signal. In addition to helping recognize the start and end of a gesture, gesturing or brushing against the fingers provides tactile feedback while performing the gestures. Feedback is crucial for users’ experience and is missing in other gesture-based approaches.
The ring uses sound and movement to identify intended gestures, which improves the accuracy compared to devices just tracking movements. For instance, to a gyroscope, random finger movements during walking may look very similar to the thumb gestures. But based on our investigation, the sounds caused by these daily activities are quite different from each other.”
Fingersound sends the sound captured by the contact microphone, along with motion data captured by the gyroscope sensor, through several filters. The signals are analyzed to determine whether a gesture was performed or whether it was simply some other finger-related activity.
The research was part of a larger effort based on ring-based gesture devices. FingOrbits, for example, lets users control apps on a smartwatch or head-mounted display by rubbing their thumb on their hand. And with SoundTrak, users can write words or 3D doodles in the air by localizing the absolute position of the finger in 3D space, then see results immediately on a computer screen.
The research team’s next goal is to make Fingersound easier to use and available to users all of the time.
