Giving Artificial Hands a Sense of Touch

Authored by:
Stephen J. Mraz
Senior Editor
[email protected]

Resources:
Deka Research and Development Corp., www.dekaresearch.com
Johns Hopkins University Applied Physics Laboratory, www.jhuapl.edu
kinea design llc, www.kineadesign.com

Advancements in artificial arms and hands rarely make it out of the laboratory due to costs or impracticality. And there often seems to be more research than developments in marketable prosthetics. But the Defense Department, dedicated to ensuring wounded service members get the best possible treatment, is using its clout and its own Skunk Works, the Defense Advanced Research Projects Agency (Darpa), to remedy this situation. It assembled some of the world’s best engineers and scientists, along with a host of innovative companies and universities, to push the boundaries in prosthetics.

One of the project’s biggest challenges is a method that would let prosthetic hands transmit haptic information — the sense of touch — to patients.

Tactors and touch
The current state of the art in commercial haptics is extremely rudimentary. The person wearing a prosthetic uses various muscles to make the artificial hand’s fingers close in a grip. Once the fingers grasp an object, the person feels a small motor vibrating against his skin, usually on the upper arm or chest. The more force the artificial hand exerts, the faster the motor vibrates, giving the wearer an idea of how much force he is putting on the object.

As part of the Darpa project, engineers at kinea design llc, Evanston, Ill., were tasked with devising an approach that provides wearers with more sensory information, including contact pressure, friction, texture, and temperature.

The first step was designing the sensors.

Kinea engineers upgraded haptic sensors developed for Darpa’s Deka arm project, an earlier program at Deka Research and Development Corp., Manchester, N. H., a company founded by inventor Dean Kamen. Those sensors were designed to go on the prosthetic’s fingertips. They each contained two strain gages to detect force. One sensed side-to-side forces, and the other detected perpendicular forces. An accelerometer picked up vibrations and textures.

Kinea added another axis of force, giving it three, along with a three-axis accelerometer and a thermocouple to sense temperature. And because their mandate was to come up with the most-capable haptic interface possible, kinea put four distinct contact sensors in the fingertip unit, hoping the added resolution would let patients read Braille. (It didn’t.) They also planned to line three contact sensors along the bottoms (or bellies) of the prosthetic’s fingers to aid in gripping and grabbing, as well as four more on the palm. The engineering challenge was packaging all the sensors and electronics into the smallest device possible.

The sensors connect to tactors, devices that convey haptic data to the wearer. kinea’s most advanced tactor, about 30 × 54 × 12 mm, contains two motors controlling a plunger. The motors can make the plunger move up and down by 17 mm, slide back and forth 24 mm, and vibrate either up and down or sideways at about 200 Hz.

“It behaves as you would expect, “says Julio Santos-Munné, director of operations at kinea. “If a patient taps on a desk with a finger, the plunger from that finger’s tactor would tap on his skin at the same rate. And if he pressed that finger into the desk with increasing amounts of force, the plunger would press into his skin with more and more force, until it reached a safety limit.”

“We did not add a temperature output on our tactor due to project constraints,” says Santos-Munné. “But we had planned for one, a Peltier thermoelectric device. It’s made of P/N junctions, essentially diodes, sandwiched between two ceramic plates and connected to a dc power source. When powered, one side of the device gets hot while the other gets cold. Naturally, if they were used, they would be adjusted so as not to burn or harm the wearer.”

Some of the problems with this sensor/tactor approach is that it takes at least one tactor just to replicate haptic inputs from one fingertip sensor. It would take quite an array of tactors, and an equal amount of usable, sensitive skin space, to complete the haptic subsystem for a hand. Even then, the patient might have difficulty interpreting pressures and vibrations from several different locations as coming from his or her hand.

Another issue is with cosmetics. Patients want prosthetics that look as natural as possible. One approach is to cover the prosthetic with a skinlike glove made of flesh-tone rubber or polymer. But covering the sensors with a layer of material, no matter how thin, degrades their sensitivity. (Strangely enough, covering the temperature sensor would actually make it more lifelike since biological temperature receptors suffer from a lag as the flesh around the receptor must heat up or cool down before it sends a signal to the brain.) And cutting holes in the glove to let sensors poke through makes it susceptible to tearing and takes away from the natural look.

The next step
There is another scheme that might simplify the patient’s task of deciphering what “feelings” the tactors are conveying and from which sites. A revolutionary surgical procedure pioneered at the Rehabilitation Center of Chicago by Dr. Todd Kuiken transplants nerves that once served the patient’s missing hand and fingers to an area on the pectoral muscle (after removing nerves already there). About six months after the targeted muscle reinnervation (TMR), the new nerves spread into an area about 4 to 5 in. in diameter. Now, when the patient tries to flex a missing finger, the chest muscle twitches. Similarly, when the area of reinnervation is touched, the patient perceives it as if being touched on a missing fingers.

TMR presents biomedical engineers with several possibilities. From a haptic standpoint, they could take the sensors-and-tactor approach from kinea and let the tactors press and vibrate on reinnervated skin. The patient would then perceive the force and vibrations as coming from a missing finger or perhaps the palm. One downside to this, and common to all TMR approaches, is that results differ from patient to patient. So health-care technicians would need to correlate or map reinnervated skin surfaces to the missing digits and hand areas they now represent.

“The technicians would look for places that correspond to important areas of the hand,” says Santos-Munné. “So they might search for an area that corresponds to the thumb and put a tactor there, or an area that maps to the forefinger. And with the current size of our tactors, we could only get one or two of them over the reinnervated region the patients perceive as their thumb or a finger. But if you were lucky, you might get sensation for the thumb and forefinger close to each other and have one tactor serve both.”

There are some researchers developing single-modality tactors, making them as small as possible, perhaps 5 mm in diameter, notes Santos-Munné. One might only push, another might vibrate, another might handle temperature, and a fourth would slide back and forth. “This could let them place several tactors on the reinnervated area.”

But the likelihood a patient will have feelings in spatially different areas corresponding to each finger and thumb is almost nil. So the haptic subsystem on each prosthetic will need to be highly customized, which adds cost and complexity.

Second, researchers looking for ways to intuitively control prosthetics, especially ones with up to 18 degrees of freedom, hope to use TMR to create a patch of muscle(s) the patient controls by trying to move his missing fingers. While the actual nerve impulses are small and hard to detect with surface electrodes, the resulting muscle twitches generate stronger electromyographic signals that skin electrodes can pick up more readily. So when the patient tries to move his index finger, a muscle on his chest moves, an electrode picks it up, and the electromechanical finger on the prosthetic moves. But again, it’s unlikely TMR will give any patients a one-to-one mapping between reinnervated areas and control of individual prosthetic joints.

The ultimate solution for haptic and truly intuitively controlled prosthetics is to hook fingertip sensors directly to the patients nerves. But this is several years, if not decades away, if it’s possible at all. Biomedical engineers must first develop a long-term method of interfacing directly with individual nerves and nerve bundles. They must uncover how bioreceptors encode the sensory data sent to the brain via nerves and replicate it. And they need to discover how the nerve impulses to muscles tell muscles to contract, by how much, and how fast.

With luck, some of the discoveries made developing state-of-the-art prosthetics will be applicable (and affordable) for commercial devices and trickle down to patients who need them.

One motor for four fingers, a thumb, and a wrist
The ideal prosthetic hand would let patients individually control each joint on each digit, as well as twist, turn, raise, and lower the wrist. But controlling all joints and degrees of freedom would mean cramming about 20 motors or other actuators into the prosthetic, along with transmissions for each. Using current technology, this would make the prosthetic heavy, a drain on the batteries, and perhaps a bit too complex to be durable and affordable.

So when engineers at kinea design were tasked with coming up with a backup design for an actuator in Darpa’s Revolutionary Prosthetic 2009 program, they decided to go with a single 40-W dc motor to power the prosthetic’s movements. They would include lithium-ion batteries and a stack of continuously variable transmissions (CVTs) that could tap power from a single 6-in.-long rotating shaft for each joint. The company had already developed the CVT. The challenge was adopting it to the prosthetic.

“We combined three CVTs into a small assembly dubbed a puck, with five pucks stacked on the shaft coming out of the motor,” says Julio Santos-Munné, director of operations at kinea design. “And each CVT has a small motor that only shifts or steers power from the shaft.”

Each CVT controls a line or artificial tendon made of Spectra, a gel-spun polyethylene fiber. The Honeywell-made fiber has 15 times the strength of steel and is extremely lightweight.

The main motor, which fits in the forearm section of the prosthetic, spins at up to 7,000 rpm. Each CVT determines the speed and how much torque it needs, and adjusts accordingly.

In tests, a prototype Spectra tendon powered by the CVT assembly and motor could lift 70 lb without overloading the motor.

“Of course, you couldn’t do this with all the finger tendons at the same time,” says Santos-Munné. “But it turns out that people don’t need the same amount of power for each joint, according to the Activities of Daily Living table. Instead, some fingers use more power than others, so this approached would’ve worked well.”

The Activities of Daily Living table, which was developed at Johns Hopkins University, tries to tabulate how often people use their arms, for what activities, and what power these activities require.

In the end, project leaders went with a design that packaged all of the prosthetic’s hand actuators in the hand itself. Though they liked the elegance of the kinea design approach, it took up valuable space in the patient’s forearm and therefore would not benefit as many amputees.