Spiking Central Pattern Generators through Reverse Engineering of Locomotion Patterns Spiking Central Pattern Generators through Reverse Engineering of Locomotion Patterns

In robotics, there have been proposed methods for locomotion of nonwheeled robots based on artificial neural networks; those built with plausible neurons are called spiking central pattern generators (SCPGs). In this chapter, we present a generalization of reported deterministic and stochastic reverse engineering methods for automatically designing SCPG for legged robots locomotion systems; such methods create a spiking neural network capable of endogenously and periodically replicating one or several rhythmic signal sets, when a spiking neuron model and one or more locomotion gaits are given as inputs. Designed SCPGs have been implemented in different robotic controllers for a variety of robotic platforms. Finally, some aspects to improve and/or complement these SCPG-based locomotion systems are pointed out.


Introduction
Since its beginning, robotics has been a research field strongly influenced by nature. For robotic platforms where wheels to displace themselves are not used, researchers have taken inspiration not only in the physical form of living beings as archetypes of their designs (e.g., legged, finned or winged robotic platforms), but also in the mechanisms to allow their locomotion (e.g., walking, swimming or flying), mainly known as central pattern generators (CPGs) [1]. In biology, the basis of CPGs was settled down in Brown's studies about how the rhythmic movements of locomotion in living beings are created [2]. In [3], Brown experimentally discovered that neurons, which inhibiting each other, generate periodically rhythmic activities that control bending and tension of muscles involved in locomotion. Moreover, GPGs have been linked to other unconscious activities besides locomotion such as swallowing, digestion, heart beating and breathing [4,5].
Furthermore, CPGs have become a suitable alternative to nonbiologically inspired methods for locomotion systems of nonwheeled robotic platforms [6]; this is due to several and interesting features of CPGs such as adaptability, rhythmicity, stability and variety [7]. The CPG-based locomotion systems have been successfully designed and implemented at software and/or hardware levels for different nonwheeled robotic platforms [8] such as walking robots (biped [9], quadruped [10], hexapod [11] and octopod [12]), swimming robots [13], flying robots [14]), among others (i.e., snake robots [15] and salamander robots [16]). Although vast amount of works made and reported in the state of the art about CPG-based locomotion systems, there is not a general and standard methodology to build CPGs [6]; however, working with CPGs commonly involves the following three phases [7]: (1) choosing the processing unit model, the coupling type and the connectivity topology (modeling and analysis), (2) dealing with parameter tuning, usually solved by optimization methods and gait transition, to handle variation on gaits as type or frequency (modulation) and (3) executing the designed CPG at the software and/or hardware level (implementation).
In this chapter, we focus on locomotion systems for legged robots, which are based on spiking central pattern generators (SCPGs) and reverse engineering methods for automatically design them. The study and implementation of SCPGs as locomotion systems have been barely explored and compared with other CPGs, which are built with oscillators or low-plausible neuron models (see [6,7,17] as reference). The SCPGs are built with plausible neuron models known as spiking neurons, models that define the third generation of artificial neural networks [18]; these neuron models naturally receive and send spatio-temporal information as generating rhythmic patterns are required for CPGs. The SCPGs have been designed and implemented as locomotion systems for robotic platforms such as bipeds [19][20][21]25], quadrupeds [23,25] and hexapods [22,[24][25][26][27], where the design methodologies used in [19][20][21]27] tend to follow the phases proposed in [7], while in [22][23][24][25][26] reverse engineering methods are used. Basically, a reverse engineering method to design SCPG-based locomotion systems for robotic platforms uses either deterministic or stochastic optimization methods, which, given an input set of discretized rhythmic signals and a fixed spiking neuron model, are capable of defining a spiking neural network (SNN), including both synaptic connections and weights, that endogenously and periodically replicates the input set of discretized rhythmic signals, which contribute to locomotion of a robotic platform. Herein, we present a generalization of reverse engineering methods to design SCPG-based locomotion systems for robotic platforms based on details of implementations of reviewed works.

Robotic platforms and controllers
Nowadays, there are a variety of robotic platforms, and each of them has particular technical specifications in design, displacement ways and so on. In reviewed works, for real implementations of SCPG-based locomotion systems, three types of legged robotic platforms have been particularly used such as hexapod, quadruped and biped robots. Particularly, the used robotic platforms have 3 degrees of freedom (DOFs) or servomotors per leg, that is, the hexapod has 18 DOFs, the quadruped has 12 DOFs and the biped has 6 DOFs. Although for the hexapod and the quadruped, just two DOFs per leg were used; the two are closer to the body of robots and directly related to the movement of the robot. Figure 1 shows the robotic platforms with a specific label for identifying the position of their respective servomotors.
Servomotors in the robotic platforms are handled by a processing unit, which in reviewed works, a SCPG is embedded into them to provide a locomotion mechanism to the robots. In Figure 2, we present the electronic boards, which have been used as processing units such as  Basically, the integration of the processing boards and platforms works as follows: an embedded SCPG into a processing board generates rhythmic signals, which are sent to the legs through a servo controller. This converts the spiking activity generated by the SCPGs into a control signal (voltage). The transmission process is carried out by using the RS-232 communication protocol. Figure 3 shows a block diagram of this integration.
Thus, an important aspect to achieve locomotion in robotic platforms is to design a SCPG according to the capabilities of the processing board, which, for reviewed works, must exactly replicate and periodically generate specific rhythmic patterns. In Section 3, we describe in detail the functionality of SCPGs and methods used for designing them.

SCPG-based locomotion system
The SCPGs, reviewed in this chapter, can generate endogenously discrete rhythmic signals; in other words, each periodical signal of a locomotion gait is represented by means of a spike train with spike times occurring periodically. This idea was firstly presented by Rostro-Gonzalez et al. in [22], where SCPGs built with discrete spiking neurons (Section 3.2.1) were automatically designed by using a deterministic reverse engineering method (Section 4.1) to imitate walking forms of a stick insect; based on steps sketches of walking forms of stick insect reported in [28], Rostro proposed three sets of discrete rhythmic signals as locomotion gaits (Section 3.1) to achieve hexapod robot locomotion by means of designing one SCPG for each of them.
In Figure 4, Rostro-Gonzalez's approach to achieve locomotion for six-legged robots through discrete events over time by means of SNNs is schematized. In Figure 4a, a walking form of stick insect reported in [28] is presented; black rectangles represent a leg on ground, while white ones represent a leg in the air. The L1 row is marked with dotted rectangle to exemplify how a step of real hexapod insect is interpreted and extended to a step of robot hexapod. Notice in Figure 4b that leg sketch coincides with a black rectangle in Figure 4a as leg is on the ground, and at this point the leg displacement that contributes to whole walking action occurs. Figure 4c shows the rhythmic signals over time to move a leg according to locomotion gait in Figure 4a; the coxa moves to front with spike events and to back without spike events, while the femur moves to down with spike events and to up in the absence of spike events. Finally, the combination of such movements according to the presence and absence of spikes of all legs provokes locomotion of legged robot.
Later, Espinal et al. generated SCPG-based locomotion systems for quadruped and hexapod robots based on Rostro-Gonzalez's idea by using the same discrete spiking neuron model (Section 3.2.1) and a stochastic reverse engineering (Section 4.2) for designing them. There were designed and implemented SCPG-based locomotion systems for quadruped robots in [23] and hexapod robots in [24]; the difference with the Rostro's work is that more compact SNN topologies were achieved, and it was achieved to design a single SCPG capable of generating the three original locomotion gaits for hexapod robots and was extended for quadruped ones as well.
Lately, SCPG-based locomotion systems for hexapods, quadrupeds and bipeds were designed by Guerra-Hernandez et al. in [25]. In his work, there was proposed a locomotion gait for biped robots following the Rostro's idea, and SNNs were designed by using the discrete spiking neuron model and a variant of stochastic reverse engineering (Section 4.2) proposed by Espinal et al. Lately, in [26], Perez-Trujillo et al. designed SCPG-based locomotion systems for hexapod robots based on Rostro-Gonzalez's, Espinal's and Guerra-Hernandez's works. The contribution of Perez-Trujillo's work was to create SCPG-based locomotion systems built with a nondiscrete spiking neuron model (Section 3.2.2), and the reverse engineering method was a variant stochastic one (Section 4.2).
The reviewed works are summarized in Table 1, including robotic platforms and processing boards as well as reverse engineering method and spiking neuron model used.
Following subsections complement the description of SCPG-based locomotion systems. In Section 3.1, the different rhythmic signal sets reported for each locomotion gait to robotic platforms are shown. Section 3.2 describes the spiking neuron models that have been used to define SNN as SCPGs on robot locomotion.

Locomotion patterns
The locomotion patterns contain the discrete rhythmic signals for each servomotor of a robotic platform. Each of them serves to define a specific locomotion gait that a SCPG must replicate endogenously and periodically. Besides, they are used for engineering reverse methods (Section 4) to design SCPGs. In Figures 5-7, show the different designed discrete rhythmic signals for hexapod, quadruped and biped robots, respectively; for each row corresponds a servomotor with the same label according to the robotic platform.

Beslon-Mazet-Soula neuron model
The Belson-Mazet-Soula (BMS) neuron [29] is a discrete-time model derived from a wellknown spiking neuron, that is, the integrate-and-fire [30] model. The BMS neuron model is defined by Eqs. (1) and (2), and they describe the behavior of the i-th neuron over time k; former equation models its membrane potential V i , and the last equation defines its firing state Z i .
In Eq. (1), γ ∈ 0; 1 ½ represents the leaky factor. The number of spiking neurons into the neural network is given by N. The synaptic strength weights are given by W ij . The I ext i is either a varying or constant external stimuli; due to that, CPGs endogenously produce periodic patterns I ext i ¼ 0: For Eq. (2), the fixed firing threshold is given by θ. Eq. (2) is used in Eq. (1) for tracking spike occurrence (Z i k ½ Þ and resetting the membrane potential of i-th neuron 1 À Z i k ½ ð Þ .

Integrate-and-fire neuron model
The integrate-and-fire (I&F) neuron [30], basically, models the evolution of its membrane potential's state over time as a resistor-capacitor (RC) circuit. In particular, the current-based leaky integrate-and-fire (LIF) model, or "if_curr_exp" model in the PyNN library [31], is a LIF neuron with a fixed firing threshold and exponentially decaying postsynaptic conductance given in Eq. (3); besides, the model requires of tau_refrac to define the refractory value and v_thresh to set the fixed firing threshold.
In Eq. (3), the membrane potential is represented with v. The excitatory and inhibitory current injections ie and ii are modelled by differential equations in Eqs. (4) and (5), respectively. The i_offset stands for a base input current, and i_inj is an external current injection; both added each timestep, but i_inj ¼ 0 due to the nature of CPGs.

Reverse engineering methods for designing SCPGs
In this section, we describe reverse engineering methods for automatically design SNNs by defining both their topology and synaptic weights. The reviewed methods can be generalized in diagram shown in Figure 8.
Basically, the generalization defines an input-process-output system where its inputs are a fixed spiking neuron model (Section 3.2) and one or more sets of discrete rhythmic signals (Section 3.1), and the process is defined according to an optimization method, which can be deterministic (Section 4.1) or stochastic (Section 4.2). Finally, the output is generally a partially connected, directed and weighted graph that defines all aspects of a SNN to behave as a SCPG.
Next, the optimization is briefly described, and their strengths and weaknesses are pointed out.

Deterministic method: Simplex method
This method, originally proposed by Rostro in [34], was created to build SNNs to replicate recorded biological neural dynamics. It was developed to work with the BMS spiking neuron model (Section 3.2.1). It is described in [22] as follows: First, Eq. (1) must be rewritten in form expressed in Eq. (6). where g ; see [34] for derivation details. With Eqs. (1) and (2), a linear programming system can be formulated to determine the synaptic connections and weights of SNNs, which replicates locomotion gaits represented as rhythmic spiking dynamics through the evolution of all V i k ½ (the membrane potential of each spiking neuron into the network), which are not known. Now, we define the expression: where θ ¼ 1 for simplification. Last expression can be written as next inequality: By substituting Eq. (6) in the last expression, we can get a linear programming system [35], given the expression in Eq. (7). Spiking Central Pattern Generators through Reverse Engineering of Locomotion Patterns http://dx.doi.org/10.5772/intechopen.72348 where Solving the aforementioned formulated linear programming problem in Eq. (7), by using a simple method (or any existing linear programming solver), we obtain the synaptic weights of all neurons and, indirectly, a SNN topology.
Next, the features of this method are listed as follows: • Strengths: • The definition of whole SNN is made by executing the method once.
• It has been successfully used as a reverse engineering method for designing SNNs, which replicate recorded biological neural dynamics.
• Weaknesses: • It can design SNN by using only BMS spiking neuron models.
• It generates one SNN for replicating just one neural dynamic pattern.

Stochastic method: Grammar-based genetic programming
For stochastic reverse engineering methods, evolutionary algorithms have been used; particularly, a variant of well-known genetic programming [36] called grammatical evolution (GE) [37]. Practically, GE is an optimization tool that searches approximated optimal solutions by representing them indirectly for a given problem; thus, working with GE to solve a specific problem requires four components: problem instance(s), representation of solutions, a fitness function to evaluate solution's quality and a search engine.
For the SCPG design problem, two types of representations have been proposed: one as a Context-Free Grammar (CFG) in Backus-Naur Form (BNF) and another as a Christiansen Grammar (CG) to use GE (in [25,26]) and a variant called Christiansen Grammar Evolution (CGE) [38] (in [23,24]), respectively. In general, both representations describe languages that define the presynaptic connectivity (including weights) of a specific spiking neuron; the common structure of expected words of two grammars is as follows: id 1 , weight 1 zfflfflfflfflfflfflffl}|fflfflfflfflfflfflffl{ 1st configured synapse |⋯| id n , weight n zfflfflfflfflfflfflffl ffl}|fflfflfflfflfflfflffl ffl{ nÀth configured synapse . The connectivity defined by a word has at least one connection and a maximum of connections according to number of neurons into the SNNs. The main difference between both representations is that words of CG representation are syntactically and semantically correct, this means that any generated word is valid and there are not repeated indexes, while words of CFG BNF are just syntactically correct or any generated word is just valid.
The fitness function is usually defined by the problem; to solve this, three fitness functions based on SPIKE [39] (used in [23,24,26]) and Victor-Purpura distances [40] (used in [25]) to compare similarity between generated spike train and target spike train to guide the search process have been proposed. Basically, first and second distance-based fitness functions are for generating one SCPG per locomotion gait; the difference is that the second one looks for minimal presynaptic connectivity and the first one does not care about number of presynaptic connections. The third distance-based fitness function allows to generate a single SCPG, which can replicate different locomotion gaits.
The search engine in GE is usually a metaheuristic algorithm, which tries to improve the quality of solutions. For reviewed works, three different algorithms have been used: Univariate Marginal Distribution Algorithm [40] (used in 23), (1 + 1)-Evolution Strategy [41] (used in [24]) and Differential Evolution [42] (used in [25,26]).

•
It can design SNNs which use either BMS spiking neuron model or LIF neuron model.

•
It can handle design criteria to design compact SNN topologies or SNN, which can replicate different neural dynamics or locomotion gaits.
• Weaknesses: • The process must be executed several times to build a single SNN; due to that, it defines synaptic connection and weights one neuron at time.
• It has not been tested on other design problems than design SCPGs.

Discussion and conclusion
Nowadays, autonomous robot locomotion is still a valid problem that has been partially solved in robotics. Particularly, locomotion of nonwheeled robotic platforms is a problem highly susceptible for trying to be solved by means of bioinspired algorithms known as CPGs. However, sometimes working with CPGs may represent a problem itself since its design; this is due to the different choices that must be made before implement the CPG according to [7]. In this chapter, we have explored researches made on SCPG field, a particular type of CPGs, which have been barely explored to date. The SCPGs are built with spiking neurons, a plausible neuron model, which handle similar information as such observed in biological neural systems. We specifically focus on SCPG designed by approaches that allow to dispense with human experts for explicitly define each CPG design phase. These kinds of works use reverse engineering methods to solve de SCPG design problem as an optimization one. By means of these methods, there are generated weighted and directed graphs as SNNs, which endogenously generate rhythmic discrete signals to allow locomotion of legged robots.
Biological CPGs do not work in isolation; they depend on the information interaction with other parts of the central nervous system [32]; even, external afferent inputs are used to shape their outputs [33]. Based on the aforementioned, the next step of SCPG-based locomotion systems could be their integration in navigation systems to endow them with sensors and can build more robust and plausible bioinspired algorithms.
Finally, there are other reasons to keep studying and implementing SCPGs, which go far beyond the locomotion of nonwheeled robots, their possible application in other areas, like medicine in developing prosthetic robotic devices for patients with spinal damage or amputated limbs [20].