The relatively recent developments in swarm robotics have found a variety of applications in areas ranging from military to environmental fields, where the superiority of swarm based distributed systems on certain tasks such as exploration, mapping and large-area sensing is indisputable. In these applications, the success of distributed robotic systems highly depends on the robotic agents’ capability to cope with uncertain and dynamic environmental conditions. Present day technology falls short in offering mobile robotic agents that function effectively under complex environmental conditions, such as those emerging after a natural disaster. On the other hand, insects exhibit an unmatched ability to navigate through a wide variety of environments and overcome perturbations by successfully maintaining control and stability.
Stimulation of neural system for remote control of navigation in insects
Neuromechanical response of cockroaches’ musculoskeletal structures provides a system level locomotive stability during their natural locomotion. Through the tactile guidance of antennae, cockroaches formulate their escape responses to avoid obstacles, where the optical cues may not be processed fast enough. [9] Among the many structures of the peripheral nervous system, the antennae was selected as the targeted stimulation location for this study. Gromphadorhina portentosa, Madagascar hissing cockroach, was selected as the terrestrial model animal for this study because of its easy rearing and maintenance conditions and commercial availability in the United States. The relatively large sizes and slower speed of these insects also enable larger payload capacities and easier biobotic manipulation of their locomotion.
Demonstrations of biobotic control
Fine wire electrodes were surgically implanted into the antenna to supply stimulation pulses through a radio frequency link. Insects followed an S-shaped line drawn on the floor through the right and left turn commands sent by the human operator via a manual remote controller. [3,4] To provide an objective test platform and assess the capability of each insect biobot, the control was automated using a Microsoft Kinect sensor. [1,4] In this platform, the PC uses the video feed from the Kinect to locate the insects and send neurostimulation pulses via a radio transmitter to automatically steer the insects along a predetermined test path. [Fig. 1] The infrared-based depth image provided by the Kinect also allows experiments to be conducted in the dark, when the insects are more active. Once an insect is implanted with the backpack
technologies, the successful steering along a semicircular path [Fig.1] within the test arena was used to assess the biobotic control capability. [1,3] The biobots that passed the Kinect test (success rate ~70 percent) were then used for navigation experiments. The unsuccessful ones were further tested to characterize the failure modes of biobotic implantation of electronics.
Figure 1. (A) The Kinect camera and insect biobot backpack. (B) Kinect based automated biobot evaluation platform. (C) The closed feedback loop. (D) representative trajectories that insect biobots followed. (Courtesy of A. Bozkurt, E. Lobaton, M. Sichitiu/Released)
The insect biobots were introduced into a maze environment to further test their biobotic capability. Walls were erected in the test arena, and a path was defined through the maze. The additional walls and corners distract the insect from completing the task, as cockroaches are naturally attracted to cool, dark areas and tend to stay close to wall corners. The insect biobots were placed in a designated start location and remotely navigated through this maze to the end location with a success rate of 75 percent. [Fig. 2]
Another biobotic capability test was to hold the position of a roachbot within a particular region of interest. Much like a modern invisible fence designed to keep pets within a backyard, the virtual fences in the test platform were designed to maintain the position of the roach biobot within a particular region. Moving outside the region results in automated stimulation pulses initiated by the PC to turn the biobot around. This was accomplished at a success rate of 85 percent. [Fig. 2]
Figure 2 (A) The Kinect view of a roach biobot in the maze and targeted trajectory. (B) A typical trajectory followed. (C) Invisible fence’s zone 1 and the buffer area around it (zone 2). (D) A representative trajectory in virtual fence. (Courtesy of A. Bozkurt, E. Lobaton, M. Sichitiu/Released)
Backpack technologies for biobots
The insect backpacks previously proposed in the literature were mostly used for telemetry applications, rather than stimulation. Their goal was to record neural potentials or electromyography signals and transmit these signals to remote computers to further understanding of insects’ neural system functions. In this study, an electronic backpack was developed for wireless control and communication. [Fig.1] To minimize the cost, a commercial-off-the-shelf system-on-chip solution from Texas Instruments that combines analog, digital and mixed-signal with radio-frequency functions on a single substrate level was used. Being tailored for IEEE 802.15.4/ZigBee and ZigBee RF4CE applications, CC2530 combines an 8051 microcontroller with a high performance RF transceiver, while providing 8 KB of RAM and up to 256 KB of flash memory. This single-chip system is an ideal solution for the insect backpack with its 21 general-purpose I/O pins and 8 channel 12-Bit ADC. The assembled backpack weighs around 300 mg without the batteries, and lithium-polymer batteries were added according to mission durations (20 mAh at 400 mg or 100 mAh at 2 g). Roach biobots were able to carry payloads on the order of 5 g in total.
To extend the propagation range of the sensor signal and minimize the transmission power, an ad hoc communications network was established by using the Zigbee-based connectivity provided by the Texas Instruments’ solution on the biobot backpack. This allows the biobotic swarm to cover a target area with their sensors located on the backpack. Once the targeted signal is detected, the information is passed from one node to another until it reaches a delivery point.
Autonomous sound localization
A miniature custom microphone array was mounted on the backpack to allow for detection and localization of a sound source with the aim of localizing victims under rubble. [Fig. 3] For this, three directional microphones were connected to three dedicated preamplifiers on a thin printed circuit board where conditioned audio signals were sampled by CC2530 at a rate of nearly 2 kHz. In order to evaluate the performance of this system, all audio data was streamed over the ZigBee network to a nearby PC. Initially, the PC is being used for audio processing, but the processing will eventually be implemented on the system-on-chip solution.
Wireless communication optimization and localization of biobots
The efficiency of biobot communication and using it for localization is the bottleneck for aiding earthquake or other disaster victims. Power efficient communications can and have to be addressed at many layers in the communication stack: each layer has to be power conscious and each layer can contribute to the overall power savings by careful design and tuning for the proposed application and resulting traffic patterns. Applications can employ compression as well as aggregation of sensed data to reduce traffic. Transport protocols can be tuned to minimize power consumption (at the cost of a potentially reduced throughput). Routing protocols can find routes that do not deplete the energy of a few key nodes leading to premature network disconnection. However, medium access control is the layer that has the largest potential for power savings as it has fine control over the radio transceiver, which is one of the major consumers in any wireless sensor system.
In the current state of the art, the most power-efficient medium access control layer solutions do not support a broadcast service, as broadcast tends to increase overhearing, which is a major source of power inefficiency. [10]
Contrariwise, most localization systems, especially those for mobile nodes, rely on broadcast for neighbor discovery and efficient transfer of information as a single transmission is capable of reaching any number of listening neighbors. Most routing protocols also rely on broadcast for exchanges of “Hello” messages to detect the availability of new links and degradation of existing links. Many time synchronization protocols (including Reference Broadcast Time Synchronization, a particularly accurate time synchronization protocol [11]) also rely on broadcast. The challenge is to design and implement a power efficient medium access control that supports broadcast while remaining power efficient over a large range of traffic patterns and network topologies.
The other problem to consider is localization, both for the biobots as well as the victims. Using the time of arrival of sound beacons has been considered for both tasks. Using the omnidirectional microphone array [Fig. 3] and one buzzer on each biobot has also been considered. The microphones are used in two different ways: on one hand, since three microphones forming a local array are available on each biobot, an approximate angle of arrival can be approximated from each biobot. The achievable resolution is, however, limited by the relatively small distance between microphones, the sampling rate of the microcontroller and available buffer size. On the other hand, microphones on different biobots are potentially spaced out one to two orders of magnitude more than those collocated on the same biobot. However, since the audio samples are now collected on different microcontrollers, accurate time synchronization and efficient data communication are required and compression is likely key in reducing the amount of data transferred.
Figure 3 (A) The sound localization system. (B) The backpack with the microphone array. (Courtesy of A. Bozkurt, E. Lobaton, M. Sichitiu/Released)
The relevant research questions for the localization of nodes and sound sources are first related to the localization precision achievable from a single biobot considering the relatively low sampling rate and buffer size; the second interesting research question is to explore the trade-off between power efficiency in running the time synchronization algorithm between neighboring biobots and corresponding precision in localization. Finally, fusing the information obtained from the local microphone array and the information from neighboring biobots is expected to result in higher precision than either of the two approaches alone, potentially at the cost of an increase in energy consumption.
Mapping of the under-rubble environment
Another essential task in a variety of applications, including search and rescue for emergency response, is the mapping of an unknown environment. This task becomes extremely challenging when localization information is not available (e.g., agents are indoor, underground, or do not have the necessary hardware or power requirements to implement traditional localization schemes). We have explored [12] how stochastic motion models and weak encounter information can be exploited to learn topological information about an unknown environment. A probabilistic motion model is assumed for the biobots [13], and tools from algebraic topology can be employed to extract spatial information of the environment based on inter-agent neighbor to neighbor interactions with no need for localization data.
A robust approach to obtain a topological map of an unknown environment using the coordinate free sensory data obtained from the biobot network is proposed.
The natural behavior of insects is used to an advantage to estimate a topological model of the environment based only on neighbor-to-neighbor interactions to minimize control input. The random walk of the agents is modeled as piecewise linear movements with fixed orientation, interrupted by isotropic changes in direction. Figure 4 illustrates a snapshot of agents moving under this motion model. Each agent is distinguished by its unique ID and assumed to have a limited sensing capability. It is assumed the agents are able to record their encounters with each other and the corresponding times as encounter events. The set of encounter nodes, together with the corresponding estimated metric, can be used to construct a filtration of simplicial complexes [14], denoted as encounter complexes. This filtration is obtained by computing the Vietoris-Rips complexes [15] from the encounter metric on G as a function of distance. From this filtration, the so-called persistence diagram can be computed [14]. This diagram quantifies the persistence of topological features extracted from the set of sample points corresponding to the encounter events. Figure 4 illustrates an example of such diagrams. In this illustration, density of the
encounters observed is estimated from interactions of agents. The persistence diagrams are extracted by thresholding the density estimates and keeping track of when topological features are born and die. Features that are most robust (shown outside the gray uncertainty region) correspond to some geometric information of the space. The blue circles represent connected components and the red triangles holes in the environment. In this scenario, the diagram identifies one persistent connected component and a hole in the domain. Nevertheless, estimating the density of events requires knowledge of location, so the density computation is bypassed to go directly to the persistence diagram [12].
Figure 4. (A) Random motion of a single agent and a snapshot of a swarm in an environment. (B) Density estimation of encounter events in the space. (C) Persistence diagram showing one persistent connected component (blue circle) and a persistent hole (red triangle). (Courtesy of A. Bozkurt, E. Lobaton, M. Sichitiu/Released)
Besides testing the approach on simulations, the algorithms were implemented on an educational and research swarm robotic platform, WolfBot. [16] This platform is open-source and open hardware with an accessible software interface. The robots are programmed to mimic the probabilistic motion model of the biobots and collect the encounter data to construct a map of the environment in a distributed manner relying only on local interactions.
Conclusions
We have demonstrated novel neural prosthetics methodologies and tools that can be used to control the locomotion of insects [1-8]. In these, we applied proprioceptive inputs in the form of electrical excitation directly to the central or peripheral nervous system through implanted payloads and produced guided behavioral responses. These demonstrations have set the basis to control individual insect’s behavior and benefit from its muscle power in a biobotic manner. In the meantime, we performed research to establish the fundamental physical and algorithmic building blocks of a biobotic swarm and produce a cyber-physical sensor network among the individuals of non-eusocial insects such as cockroaches. The areas of applicability for biobotic swarms are extremely diverse. In this case, the focus was on the problem of signal source localization and environment mapping, such as finding a surviving earthquake victim in a rubble pile.
Acknowledgement: The students who contributed to this work include Tahmid Latif, Alireza Dirafzoon, Eric Whitmire, Alexander Verderber, Juan Marin and Hong Xiong from the Department of Electrical and Computer Engineering at North Carolina State University in Raleigh, NC. This research was fully funded by NSF
CPS-1239243.
References:
[1] Whitmire, E., Latif, T. & Bozkurt A. (2013). Kinect-based System for Automated Control of Terrestrial Insect Biobots. Proc. IEEE EMBC, pp. 1470-1473.
[2] Bozkurt, A.. (2012). Latest Advances in Biobotic Agents for Environmental Sensor Networks. Proc. IEEE EMBC.
[3] Latif, T. & Bozkurt, A. (2012). Line Following Terrestrial Insect Biobots. Proc. IEEE EMBC, pp. 972-975.
[4] iBionicSLab. https://www.youtube.com/user/iBionicSLab
[5] Bozkurt, A., Gilmour, R. & Lal, A. (2011). In vivo Electrochemical Characterization of Tissue-Electrode Interface during Metamorphic Growth. IEEE Trans. Biomed. Eng, 58(8), pp. 2401-2406.
[6] Bozkurt, A., Gilmour, R. & Lal, A. (2009). Balloon Assisted Flight of Radio Controlled Insect Biobots. IEEE Trans. Biomed. Eng., 56
(9), pp. 2304-2307.
[7] Bozkurt, A., Gilmour, R., Sinha, A., Stern, D. & Lal, A. Insect Machine Interface Based Neuro Cybernetics. IEEE Trans. Biomed. Eng, 56(6), pp. 1727-1733.
[8] Paul. A., Bozkurt, A., Ewer, J., Blossey, B. & Lal, A. (2006). Surgically Implanted Micro-Platforms in Manduca-Sexta. Solid-State Sensors and Actuator Workshop, pp. 209-211.
[9] Camhi, J. & Johnson, E. (1999). High-frequency steering maneuvers in the cockroach. J. Exp. Biol., 202(5), pp. 189-202.
[10] Ye, W., Silva, F. & Heidemann, J. (2006). Ultra-low duty cycle MAC with scheduled channel polling. Proc. ACM SenSys., pp. 321-
334.
[11] Sun, Y., Gurewitz, O. & Johnson, D. B. (2008). RI-MAC: a receiver-initiated asynchronous duty cycle MAC protocol for dynamic traffic loads in wireless sensor networks. Proc. ACM SenSys, pp. 1-14.
[12] Dirafzoon, A. & Lobaton, E. (2013). Topological mapping of unknown environments using an unlocalized robotic swarm. Proc. IEEE/RSJ IROS, pp. 5545–5551.
[13] Jeanson, R., Blanco, S., Fournier, R., Deneubourg, J.-L., Fourcassi´e, V. & Theraulaz, G. (2003). A model of animal movements in a bounded space. J. Theoretical Biol., 225, pp. 443–451.
[14] Edelsbrunner H., Letscher, D. & Zomorodian, A. (2002). Topological persistence and simplification. Discrete and Computational Geometry, 28(4), pp. 511-533.
[15] Chambers, E., Silva, V., Erickson, J. & Ghrist, R. (2010). Vietoris-Rips complexes of planar point sets. Discrete and Computational Geometry, 44(1), pp. 75–90.
[16] Betthauser, J., Benavides, D., Schornick, J., O’Hara, N., Patel, J., Cole, J. & Lobaton, E. (2014). WolfBot: A distributed mobile sensing platform for research and education. Proc. ASEE Zone 1 Conf.
About the Authors:
Alper Bozkurt is an assistant professor in the Department of Electrical and Computer Engineering at North Carolina State University. He received a doctorate degree from Cornell University in Ithaca, N.Y. and master’s degree in biomedical engineering from Drexel University in Philadelphia, Penn. Bozkurt is the founder and the director of Integrated Bionic MicroSystems Laboratory at North Carolina State University. He is a recipient of the Calhoun Fellowship from Drexel University, Donald Kerr Award at Cornell University, Chancellor’s Innovation Award and William F. Lane Outstanding Teacher Award at North Carolina State University and the best paper award from the U.S. Government Microcircuit Applications & Critical Technology Conference.
Edgar Lobaton received his Bachelor of Science degrees in mathematics and electrical engineering from Seattle University in 2004. He completed his doctorate in electrical engineering and computer sciences from the University of California, Berkeley in 2009. He is an assistant professor in the Department of Electrical and Computer Engineering at North Carolina State University. His areas of research include robotics, sensor networks and computer vision. He was awarded the 2009 Computer Innovation Fellows post-doctoral fellowship award and conducted research in the Department of Computer Science at the University of North Carolina at Chapel Hill.
Mihail L. Sichitiu is employed as a professor in the Department of Electrical and Computer Engineering at North Carolina State University. In May 2001, he received a doctorate in electrical engineering from the University of Notre Dame. He received a Bachelor of Engineering and a Master of Science in electrical engineering from the Polytechnic University of Bucharest in 1995 and 1996 respectively. His primary research interest is in wireless networking with emphasis on multi-hop networking and wireless local area networks.