Feeling Through a Bionic Hand

Advances in battlefield trauma care and in body armor have led to an increase in the survival rates following injuries in the battlefield. Although body armor protects the torso and internal organs, soldiers remain vulnerable to limb loss and upper spinal cord injury. [1] Developments in prosthetics could further improve the quality of life of military and civilian amputees.

One way to restore sensorimotor function in amputees and tetraplegic patients is to equip them with robotic arms and devise ways for them to control these arms. In the last 20 years, powerful algorithms to decode intended movements from signals recorded from muscles or from the nervous system have been developed, as have robotic limbs that reproduce most of the functionality of native human arms. [2] However, our ability to manipulate objects relies heavily on our sense of touch. Without these sensory signals, we would struggle to perform even the most basic activities of daily living, like turning a door knob or picking up a coffee cup. For upper-limb neuroprostheses to be clinically viable, then, it is necessary to develop ways not only to convert neural signals about motor intent into movements of the prosthesis, but also to convey tactile feedback about the consequences of those movements [3] for example about how hard we are grasping an object.

State-of-the-art prosthetic arms can move in most of the ways a native human arm can move, and include a variety of different sensors that can, in principle, replace the biological sensors in the skin (so-called mechanoreceptors). [4] Researchers are developing algorithms that convert the output of these sensors into patterns of electrical stimulation of the nervous system that evoke meaningful and intuitive touch sensations. Some of our work focuses on restoring the sense of touch by stimulating the somatosensory nerves, which carry touch signals from the hand to the brain. [5] While this approach is promising for amputees, it cannot be applied to tetraplegic patients, for whom the nerves are dysfunctional or are no longer connected to the brain. For these patients, we are working on ways to convey sensory feedback by stimulating the somatosensory cortex, the part of the brain that receives and processes touch signals from the hand. [3, 6-8]

Restoring Touch Through a Peripheral Nerve Interface

Three types of nerve fibers relay tactile information from the hand to the brain, each of which responds differently to skin deformations. When we grasp and manipulate objects, all three populations of nerve fibers respond and convey overlapping information about the size, shape and texture of the objects and about their movements across the skin. [9] Importantly, if the skin is stimulated in exactly the same way, these fibers produce almost exactly the same response. [10] Exploiting the near deterministic nature of these neural responses, we have developed models that can predict with millisecond precision the responses of nerve fibers to any spatio-temporal deformations of the skin. [11-13] With these models, we can predict how the nerve would respond to anything the hand might come into contact with. [14, 15]

A variety of technologies are being developed to interface directly with the peripheral nerve. [5] The idea is to use the models of nerve fiber responses described above to convert the output of the touch sensors on the prosthetic hand into desired patterns of nerve activation— those that would be observed in the native hand—and electrically stimulate the nerve to evoke these desired patterns using an electrical interface with the nerve. If the density and capabilities of the sensors on the prosthesis matched those of native mechanoreceptors and if we could concurrently stimulate individual nerve fibers, tens of thousands of them, with patterns of stimulation that were specific to that fiber, we could reproduce natural tactile sensations with perfect verisimilitude. Of course, the current state-of-the-art in hand sensorization only comprises tens of sensors and peripheral nerve interface technologies tens to hundreds of electrodes. The development effort consists in determining how we can create naturalistic patterns of nerve activation given the relatively small number of independent channels at our disposal. To this end, we seek to assess which aspects of nerve activation are critical to elicit meaningful tactile sensations and which are not.

The advantage of restoring touch through a peripheral nerve interface is that we have an advanced understanding of the relevant neural structures so the main challenge is a technological one. However, peripheral nerve interfaces cannot be applied to tetraplegic patients, whose nerves are either non-functional or no longer connected to the brain. For these patients, it is necessary to interface directly with the brain.

Restoring Touch Through a Brain Interface

Neural signals originating in the skin of the hand are processed in a part of the brain called primary somatosensory cortex, S1.[16] In a series of experiments, we have been investigating whether we can evoke meaningful tactile sensations by electrically stimulating neurons in S1. [3, 6, 7, 17, 18] We focused on trying to convey the most basic information necessary to grasp an object, namely information about contact location and contact pressure. Indeed, when we grasp an object, we need to know which parts of the hand are touching the object. To pick it up, the thumb and at least one of the fingers need to make contact with it. We also need to know how much pressure we are exerting on it. We want to exert enough pressure so as to not drop it but not so much pressure that we might crush it. In a series of experiments with monkeys, we sought to develop ways to stimulate S1 and convey information about contact location and pressure intuitively. To this end, we leveraged what we know about how contact location and timing are encoded in S1 of intact individuals and attempted to reproduce these natural patterns of neuronal activation through electrical stimulation.

In these experiments, we stimulated the brain with arrays of electrodes chronically implanted in S1. Our strategy consisted of training the animals to discriminate touches applied to their native hands. When the animals were trained on the task, we replaced some of the touches with small electrical pulse trains delivered to S1. We then assessed whether the animals responded to the electrical stimuli as they did to the touches they replaced. Importantly, the patterns of electrical stimulation were inspired by our knowledge about how location and pressure are represented in the brain.

Contact location has long been thought to be represented by a so-called place code. Touching different patches of skin activates different populations of neurons, and nearby neurons tend to respond to nearby patches of skin. Our ability to identify where we are touched is thus thought to be mediated by the location of the activated neurons within S1. We tested this by training animals to do a location discrimination task: We touched them sequentially at two locations of the skin and asked them whether the second touch was to the left or the right of the first. [8] Once the animals were trained, we replaced one of the touches with an electrical stimulus applied to an electrode that responded to the corresponding patch of skin. So, for example, we might touch the index fingertip, then the middle fingertip of the left hand, and the animal would correctly respond ‘right.’ Then, we might touch the middle fingertip of the left hand and electrically stimulate through an electrode located in a different region of S1 corresponding to the index fingertip. Lo and behold, the animal responded ‘left,’ as if we had actually touched its index finger. Across a wide variety of stimulation conditions, the animals responded as if they had been a touched in the location corresponding to that receptive field of the stimulating electrode. In other words, stimulating through individual electrodes evokes highly localized touch sensations. This phenomenon can then be exploited to convey information about contact location.

The magnitude of touch sensations tracks the pressure exerted on the skin: When we press weakly on the skin, we experience a weak sensation. When we press strongly on the skin, we experience a strong sensation. In S1, increasing pressure results in an increased response in active neurons and in the recruitment of nearby neurons. Increasing the amplitude of electrical stimulation has an analogous effect: As we increase the stimulation current, neurons near the electrodes become more active and neurons further away from the electrode become activated. We carried out a series of experiments to test whether information about pressure could be conveyed by varying the stimulation amplitude. These experiments culminated in a sensory encoding algorithm that converted skin pressure into stimulation current such that the magnitude of the electrically induced sensation matched that of the corresponding mechanical pressure sensation. We implemented this function to convert, in real time, the output of pressure sensors on prosthetic fingers into an appropriate electrical stimulus. We then demonstrated that animals could discriminate changes in pressure just as well when the touches were delivered to their native finger or to a prosthetic finger coupled with our algorithm.

To restore touch in a tetraplegic patient, then, the trick is to connect the electrodes implanted in S1 to the appropriate sensors on the hand. To this end, the prosthetist can stimulate through each electrode in turn and ask the patient where he or she experienced the sensation. Indeed, we have shown that stimulation through individual electrodes evokes highly localized tactile sensations. For example, if the patient reports that stimulation of electrode 23 evokes a sensation on the thumb tip, then electrode 23 should be connected to the sensor on the thumb tip. Anytime the thumb touches something, electrical stimulation will be delivered through electrode 23. To determine how much current to deliver through the electrode given a level of sensor output, we can use the psychometric equivalence function, which converts pressure into current. That way, the magnitude of the artificial tactile sensation experienced on the thumb will be appropriate to the amount of pressure that is exerted on the prosthetic thumb. Different electrodes will be connected to different sensors at different locations on the hand (using the approach sketched out above) to produce sensations that are experienced on the corresponding parts of the hand.

Conclusion

Contact force and location are two of the most basic aspects of touch so we initially focused on those. We are currently working on expanding the repertoire of artificially induced tactile sensations to include, for example, the sensation of motion across the skin or of texture. The key, we think, is to figure out how the brain represents this information and attempt to reproduce these natural patterns through electrical stimulation. When this technology comes to fruition, we anticipate that even the rudimentary tactile feedback described above will considerably improve the ability of patients to use robotic arms. Furthermore, it will lead patients to embody these robotic limbs, experience them as part of their own bodies.

As a result, wounded warriors outfitted with this advanced prosthesis may regain some of the independence that they lost as a consequence of their injuries.

Acknowledgments

This material is based on work supported by the Defense Advanced Research Projects Agency under Contract N66001-10-C-4056.

References

[1] Burck, J.M., Bigelow, J.D. & Harshbarger, S.D. 2011 Revolutionizing Prosthetics: systems engineering challenges and opportunities. Johns Hopkins APL Technical Digest 30, 186-197.
[2] Bensmaia, S.J. & Miller, L.E. 2014 Restoring sensorimotor function through intracortical interfaces: progress and looming challenges. Nature reviews. Neuroscience 15, 313-325. (doi:10.1038/nrn3724).
[3] Tabot, G.A., Kim, S.S., Winberry, J.E. & Bensmaia, S.J. 2014 Restoring tactile and proprioceptive sensation through a brain interface. Neurobiology of disease. (doi:10.1016/j.nbd.2014.08.029).
[4] ohannes, M.S., Bigelow, J.D., Burck, J.M., Harshbarger, S.D., Kozlowski, M.V. & Van Doren, T. 2011 An overview of the developmental process for the Modular Prosthetic Limb. The Johns Hopkins University Applied Physics Laboratory Technical Digest 30.
[5] Saal, H.P. & Bensmaia, S.J. in review Biomimetic approaches to bionic touch through a peripheral nerve interface. Neuropsychologia.
[6] Kim, S., Callier, T., Tabot, G.A., Tenore, F.V. & Bensmaia, S.J. 2015 Sensitivity to microstimulation of somatosensory cortex distributed over multiple electrodes. Frontiers in systems neuroscience 9, 47. (doi:10.3389/fnsys.2015.00047).
[7] Berg, J.A., Dammann III, J.F., Tenore, F.V., Tabot, G.A., Boback, J.L., Manfredi, L.R., Peterson, M.L., Katyal, K.D., Johannes, M.S., Makhlin, A., et al. 2013 Behavioral demonstration of a somatosensory neuroprosthesis. IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society 21, 500-507. (doi:10.1109/TNSRE.2013.2244616).
[8] Tabot, G.A., Dammann, J.F., Berg, J.A., Tenore, F.V., Boback, J.L., Vogelstein, R.J. & Bensmaia, S.J. 2013 Restoring the sense of touch with a prosthetic hand through a brain interface. Proceedings of the National Academy of Sciences of the United States of America 110, 18279-18284. (doi:10.1073/pnas.1221113110).
[9] Saal, H.P. & Bensmaia, S.J. 2014 Touch is a team effort: interplay of submodalities in cutaneous sensibility. Trends in neurosciences. (doi:10.1016/j.tins.2014.08.012).
[10] Mackevicius, E.L., Best, M.D., Saal, H.P. & Bensmaia, S.J. 2012 Millisecond precision spike timing shapes tactile perception. The Journal of neuroscience : the official journal of