Spiking Central Pattern Generators through Reverse Engineering of Locomotion Patterns

Abstract

In robotics, there have been proposed methods for locomotion of nonwheeled robots based on artificial neural networks; those built with plausible neurons are called spiking central pattern generators (SCPGs). In this chapter, we present a generalization of reported deterministic and stochastic reverse engineering methods for automatically designing SCPG for legged robots locomotion systems; such methods create a spiking neural network capable of endogenously and periodically replicating one or several rhythmic signal sets, when a spiking neuron model and one or more locomotion gaits are given as inputs. Designed SCPGs have been implemented in different robotic controllers for a variety of robotic platforms. Finally, some aspects to improve and/or complement these SCPG-based locomotion systems are pointed out.

1. Introduction

Since its beginning, robotics has been a research field strongly influenced by nature. For robotic platforms where wheels to displace themselves are not used, researchers have taken inspiration not only in the physical form of living beings as archetypes of their designs (e.g., legged, finned or winged robotic platforms), but also in the mechanisms to allow their locomotion (e.g., walking, swimming or flying), mainly known as central pattern generators (CPGs) [1]. In biology, the basis of CPGs was settled down in Brown’s studies about how the rhythmic movements of locomotion in living beings are created [2]. In [3], Brown experimentally discovered that neurons, which inhibiting each other, generate periodically rhythmic activities that control bending and tension of muscles involved in locomotion. Moreover, GPGs have been linked to other unconscious activities besides locomotion such as swallowing, digestion, heart beating and breathing [4, 5].

Furthermore, CPGs have become a suitable alternative to nonbiologically inspired methods for locomotion systems of nonwheeled robotic platforms [6]; this is due to several and interesting features of CPGs such as adaptability, rhythmicity, stability and variety [7]. The CPG-based locomotion systems have been successfully designed and implemented at software and/or hardware levels for different nonwheeled robotic platforms [8] such as walking robots (biped [9], quadruped [10], hexapod [11] and octopod [12]), swimming robots [13], flying robots [14]), among others (i.e., snake robots [15] and salamander robots [16]). Although vast amount of works made and reported in the state of the art about CPG-based locomotion systems, there is not a general and standard methodology to build CPGs [6]; however, working with CPGs commonly involves the following three phases [7]: (1) choosing the processing unit model, the coupling type and the connectivity topology (modeling and analysis), (2) dealing with parameter tuning, usually solved by optimization methods and gait transition, to handle variation on gaits as type or frequency (modulation) and (3) executing the designed CPG at the software and/or hardware level (implementation).

In this chapter, we focus on locomotion systems for legged robots, which are based on spiking central pattern generators (SCPGs) and reverse engineering methods for automatically design them. The study and implementation of SCPGs as locomotion systems have been barely explored and compared with other CPGs, which are built with oscillators or low-plausible neuron models (see [6, 7, 17] as reference). The SCPGs are built with plausible neuron models known as spiking neurons, models that define the third generation of artificial neural networks [18]; these neuron models naturally receive and send spatio-temporal information as generating rhythmic patterns are required for CPGs. The SCPGs have been designed and implemented as locomotion systems for robotic platforms such as bipeds [19–21, 25], quadrupeds [23, 25] and hexapods [22, 24, 25, 26, 27], where the design methodologies used in [19, 20, 21, 27] tend to follow the phases proposed in [7], while in [22, 23, 24, 25, 26] reverse engineering methods are used. Basically, a reverse engineering method to design SCPG-based locomotion systems for robotic platforms uses either deterministic or stochastic optimization methods, which, given an input set of discretized rhythmic signals and a fixed spiking neuron model, are capable of defining a spiking neural network (SNN), including both synaptic connections and weights, that endogenously and periodically replicates the input set of discretized rhythmic signals, which contribute to locomotion of a robotic platform. Herein, we present a generalization of reverse engineering methods to design SCPG-based locomotion systems for robotic platforms based on details of implementations of reviewed works.

2. Robotic platforms and controllers

Nowadays, there are a variety of robotic platforms, and each of them has particular technical specifications in design, displacement ways and so on. In reviewed works, for real implementations of SCPG-based locomotion systems, three types of legged robotic platforms have been particularly used such as hexapod, quadruped and biped robots. Particularly, the used robotic platforms have 3 degrees of freedom (DOFs) or servomotors per leg, that is, the hexapod has 18 DOFs, the quadruped has 12 DOFs and the biped has 6 DOFs. Although for the hexapod and the quadruped, just two DOFs per leg were used; the two are closer to the body of robots and directly related to the movement of the robot. Figure 1 shows the robotic platforms with a specific label for identifying the position of their respective servomotors.

Figure 1.

Robotic platforms from Lynxmotion company. (a) Phoenix hexapod robot model, (b) symmetric quadruped robot model and (c) Brat biped robot model. In ovals are marked and labeled the servomotors used in their locomotion where letters C, F and a stand for Coxa, femur and ankle, respectively, letters L and R represent left and right sides, and numbers mark the number according to their position (taken from [25]).

Servomotors in the robotic platforms are handled by a processing unit, which in reviewed works, a SCPG is embedded into them to provide a locomotion mechanism to the robots. In Figure 2, we present the electronic boards, which have been used as processing units such as Arduino (Figure 2a), Field Programmable Gate Array (FPGA) (Figure 2b) and a Raspberry Pi 3 Model B (Figure 2c) boards. Also, a servo controller board (Figure 2d) is required to send the output of the processing units to the servomotors of the robotic platforms.

Figure 2.

Boards for robot control. Processing units: (a) Arduino board, BotBoarduino for Lynxmotion robots, (b) FPGA Spartan 6 XEM6310-LX45 board from OpalKelly, and (c) Raspberry Pi 3 Model B board. Servo controller: (d) SSC-32 board.

Basically, the integration of the processing boards and platforms works as follows: an embedded SCPG into a processing board generates rhythmic signals, which are sent to the legs through a servo controller. This converts the spiking activity generated by the SCPGs into a control signal (voltage). The transmission process is carried out by using the RS-232 communication protocol. Figure 3 shows a block diagram of this integration.

Figure 3.

System block diagram of robotic controller coupled with servomotors of robotic platforms through a servo controller.

Thus, an important aspect to achieve locomotion in robotic platforms is to design a SCPG according to the capabilities of the processing board, which, for reviewed works, must exactly replicate and periodically generate specific rhythmic patterns. In Section 3, we describe in detail the functionality of SCPGs and methods used for designing them.

3. SCPG-based locomotion system

The SCPGs, reviewed in this chapter, can generate endogenously discrete rhythmic signals; in other words, each periodical signal of a locomotion gait is represented by means of a spike train with spike times occurring periodically. This idea was firstly presented by Rostro-Gonzalez et al. in [22], where SCPGs built with discrete spiking neurons (Section 3.2.1) were automatically designed by using a deterministic reverse engineering method (Section 4.1) to imitate walking forms of a stick insect; based on steps sketches of walking forms of stick insect reported in [28], Rostro proposed three sets of discrete rhythmic signals as locomotion gaits (Section 3.1) to achieve hexapod robot locomotion by means of designing one SCPG for each of them.

In Figure 4, Rostro-Gonzalez’s approach to achieve locomotion for six-legged robots through discrete events over time by means of SNNs is schematized. In Figure 4a, a walking form of stick insect reported in [28] is presented; black rectangles represent a leg on ground, while white ones represent a leg in the air. The L1 row is marked with dotted rectangle to exemplify how a step of real hexapod insect is interpreted and extended to a step of robot hexapod. Notice in Figure 4b that leg sketch coincides with a black rectangle in Figure 4a as leg is on the ground, and at this point the leg displacement that contributes to whole walking action occurs. Figure 4c shows the rhythmic signals over time to move a leg according to locomotion gait in Figure 4a; the coxa moves to front with spike events and to back without spike events, while the femur moves to down with spike events and to up in the absence of spike events. Finally, the combination of such movements according to the presence and absence of spikes of all legs provokes locomotion of legged robot.

Figure 4.

Representation of extrapolation of steps observed of hexapod insect into discrete rhythmic signals for SCPG-based locomotion design. (a) Walking form of stick insect reported in [28], (b) extrapolation of a step into position of a leg over time and (c) proposal of a step (femur row) with additional information (coxa row) represented as spike trains, in each row, darker rectangles represent a spike and lighter ones indicate the absence of spike.

Later, Espinal et al. generated SCPG-based locomotion systems for quadruped and hexapod robots based on Rostro-Gonzalez’s idea by using the same discrete spiking neuron model (Section 3.2.1) and a stochastic reverse engineering (Section 4.2) for designing them. There were designed and implemented SCPG-based locomotion systems for quadruped robots in [23] and hexapod robots in [24]; the difference with the Rostro’s work is that more compact SNN topologies were achieved, and it was achieved to design a single SCPG capable of generating the three original locomotion gaits for hexapod robots and was extended for quadruped ones as well.

Lately, SCPG-based locomotion systems for hexapods, quadrupeds and bipeds were designed by Guerra-Hernandez et al. in [25]. In his work, there was proposed a locomotion gait for biped robots following the Rostro’s idea, and SNNs were designed by using the discrete spiking neuron model and a variant of stochastic reverse engineering (Section 4.2) proposed by Espinal et al.

Lately, in [26], Perez-Trujillo et al. designed SCPG-based locomotion systems for hexapod robots based on Rostro-Gonzalez’s, Espinal’s and Guerra-Hernandez’s works. The contribution of Perez-Trujillo’s work was to create SCPG-based locomotion systems built with a nondiscrete spiking neuron model (Section 3.2.2), and the reverse engineering method was a variant stochastic one (Section 4.2).

The reviewed works are summarized in Table 1, including robotic platforms and processing boards as well as reverse engineering method and spiking neuron model used.

Author/Work	Robotic platform	Robotic controller	Spiking neuron model	Reverse engineering method
Rostro et al. [22]	Hexapod	FPGA	BMS Neuron	Deterministic
Espinal et al. [23]	Quadruped	Arduino	BMS Neuron	Stochastic
	Hexapod	FPGA
Espinal et al. [24]	Hexapod	FPGA	BMS Neuron	Stochastic
Guerra et al. [25]	Hexapod	FPGA	BMS Neuron	Variant stochastic
	Quadruped	FPGA
	Biped	FPGA
Perez et al. [26]	Hexapod	Raspberry Pi 3	LIF Neuron	Variant stochastic

Table 1.

Summary of Legged Robot Locomotion System Configurations and reverse engineering methods.

Following subsections complement the description of SCPG-based locomotion systems. In Section 3.1, the different rhythmic signal sets reported for each locomotion gait to robotic platforms are shown. Section 3.2 describes the spiking neuron models that have been used to define SNN as SCPGs on robot locomotion.

3.1. Locomotion patterns

The locomotion patterns contain the discrete rhythmic signals for each servomotor of a robotic platform. Each of them serves to define a specific locomotion gait that a SCPG must replicate endogenously and periodically. Besides, they are used for engineering reverse methods (Section 4) to design SCPGs. In Figures 5–7, show the different designed discrete rhythmic signals for hexapod, quadruped and biped robots, respectively; for each row corresponds a servomotor with the same label according to the robotic platform.

Figure 5.

Hexapod robot locomotion gaits proposed in [22] (figures taken from [26]). (a) Walk gait, (b) jog gait and (c) run gait.

Figure 6.

Quadruped robot locomotion gaits proposed in [23] (figures taken from [26]). (a) Walk gait, (b) jog gait and (c) run gait.

Figure 7.

Biped walking locomotion gait proposed in [26].

3.2. Spiking neuron models

3.2.1. Beslon-Mazet-Soula neuron model

The Belson-Mazet-Soula (BMS) neuron [29] is a discrete-time model derived from a well-known spiking neuron, that is, the integrate-and-fire [30] model. The BMS neuron model is defined by Eqs. (1) and (2), and they describe the behavior of the $i$-th neuron over time$k$; former equation models its membrane potential $V_i$, and the last equation defines its firing state $Z_i$.

$$V_i\left[k\right]=\mathrm{\gamma }{V}_i\left[k-1\right]\left(1-Z_i\left[k-1\right]\right)+\sum _{j=1}^N{W}_{\mathit{ij}}{Z}_j\left[k-1\right]+I_i^{\mathit{ext}}$$E1

In Eq. (1), $\mathrm{\gamma }\in \left[0,1\right]$represents the leaky factor. The number of spiking neurons into the neural network is given by $N$. The synaptic strength weights are given by $W_{\mathit{ij}}$. The $I_i^{\mathit{ext}}$is either a varying or constant external stimuli; due to that, CPGs endogenously produce periodic patterns $I_i^{\mathit{ext}}=0.$

$$Z_i\left[k\right]=\left\{\begin{array}{c}1\mathit{\text{if}}{V}_i\left[k\right]\ge \theta \\ 0\mathit{\text{otherwise}}\end{array}\right.$$E2

For Eq. (2), the fixed firing threshold is given by $\theta$. Eq. (2) is used in Eq. (1) for tracking spike occurrence ($Z_i\left[k\right])$and resetting the membrane potential of $i$-th neuron $\left(1-Z_i\left[k\right]\right)$.

3.2.2. Integrate-and-fire neuron model

The integrate-and-fire (I&F) neuron [30], basically, models the evolution of its membrane potential’s state over time as a resistor-capacitor (RC) circuit. In particular, the current-based leaky integrate-and-fire (LIF) model, or “if_curr_exp” model in the PyNN library [31], is a LIF neuron with a fixed firing threshold and exponentially decaying postsynaptic conductance given in Eq. (3); besides, the model requires of $\mathit{tau}\_\mathit{\text{refrac}}$to define the refractory value and $v\_\mathit{\text{thresh}}$to set the fixed firing threshold.

$$\frac{\mathit{dv}}{\mathit{dt}}=\frac{\mathit{ie}+\mathit{ii}+i\_\mathit{\text{offset}}+i\_\mathit{inj}}{c\_m}+\frac{v\_\mathit{\text{rest}}-v}{\mathit{tau}\_m}$$E3

In Eq. (3), the membrane potential is represented with $v$. The excitatory and inhibitory current injections $\mathit{ie}$and $\mathit{ii}$are modelled by differential equations in Eqs. (4) and (5), respectively. The $i\_\mathit{\text{offset}}$stands for a base input current, and $i\_\mathit{inj}$is an external current injection; both added each timestep, but $i\_\mathit{inj}=0$due to the nature of CPGs.

$$\frac{\mathit{die}}{\mathit{dt}}=-\frac{\mathit{ie}}{\mathit{tau}\_\mathit{syn}\_E}$$E4

$$\frac{\mathit{dii}}{\mathit{dt}}=-\frac{\mathit{ii}}{\mathit{tau}\_\mathit{syn}\_I}$$E5

Finally, $\mathrm{tau}\_\mathrm{syn}\_\mathrm{E}$and $\mathrm{tau}\_\mathrm{syn}\_\mathrm{I}$, in Eqs. (5) and (6), are excitatory and inhibitory input current decay time constant.

4. Reverse engineering methods for designing SCPGs

In this section, we describe reverse engineering methods for automatically design SNNs by defining both their topology and synaptic weights. The reviewed methods can be generalized in diagram shown in Figure 8.

Figure 8.

Diagram of reverse engineering method for designing SCPG.

Basically, the generalization defines an input-process-output system where its inputs are a fixed spiking neuron model (Section 3.2) and one or more sets of discrete rhythmic signals (Section 3.1), and the process is defined according to an optimization method, which can be deterministic (Section 4.1) or stochastic (Section 4.2). Finally, the output is generally a partially connected, directed and weighted graph that defines all aspects of a SNN to behave as a SCPG.

Next, the optimization is briefly described, and their strengths and weaknesses are pointed out.

4.1. Deterministic method: Simplex method

This method, originally proposed by Rostro in [34], was created to build SNNs to replicate recorded biological neural dynamics. It was developed to work with the BMS spiking neuron model (Section 3.2.1). It is described in [22] as follows:

First, Eq. (1) must be rewritten in form expressed in Eq. (6).

$$V_i\left[k\right]=\sum _{j=1}^N{W}_{\mathit{ij}}\sum _{\tau =0}^{{\tau }_{\mathit{ik}}}{\gamma }^{\tau }{Z}_j\left[k-\tau -1\right]+I_{\mathit{ik\tau }}$$E6

where $I_{\mathit{ik\tau }}={\sum }_{\tau =0}^{{\tau }_{\mathit{ik}}}{\gamma }^{\tau }{I}_i^{\left(\mathit{ext}\right)}\left[k-\tau \right]$with: ${\tau }_{\mathit{ik}}=k-$arg $\underset{l>0}{\mathit{min}}\left\{Z_i\left[l-1\right]=1\right\}$; see [34] for derivation details. With Eqs. (1) and (2), a linear programming system can be formulated to determine the synaptic connections and weights of SNNs, which replicates locomotion gaits represented as rhythmic spiking dynamics through the evolution of all $V_i\left[k\right]$(the membrane potential of each spiking neuron into the network), which are not known. Now, we define the expression: $Z_i\left[k\right]=0\iff V_i\left[k\right]<\theta$and $Z_i\left[k\right]=1\iff V_i\left[k\right]\ge \theta$; where $\theta =1$for simplification. Last expression can be written as next inequality: $Z_i\left[k\right]=0\iff V_i\left[k\right]<\theta$.

By substituting Eq. (6) in the last expression, we can get a linear programming system [35], given the expression in Eq. (7).

$$e_i=A_i{w}_i+b_i\ge 0$$E7

where

• $A_i=\left(\begin{array}{ccc}\cdots & \cdots & \cdots \\ \cdots & \left(2Z_i\left[k\right]-1\right)\sum _{\tau =0}^{{\tau }_{\mathit{ik}}}{\gamma }^{\tau }{Z}_j\left[k-\tau -1\right]& \cdots \\ \cdots & \cdots & \cdots \end{array}\right)$

• $b_i={\left(\cdots \left(2Z_i\left[k\right]-1\right)\left(I_{\mathit{ik\tau }}-1\right)\cdots \right)}^t$

• $w_i={\left(\begin{array}{ccc}\cdots & W_{\mathit{ij}}& \cdots \end{array}\right)}^t$

• $e_i={\left(\begin{array}{ccc}\cdots & \left(2Z_i\left[k\right]-1\right)\left(V_i\left[k\right]-1\right)& \cdots \end{array}\right)}^t$

Solving the aforementioned formulated linear programming problem in Eq. (7), by using a simple method (or any existing linear programming solver), we obtain the synaptic weights of all neurons and, indirectly, a SNN topology.

Next, the features of this method are listed as follows:

• Strengths:

  • The definition of whole SNN is made by executing the method once.

  • It has been successfully used as a reverse engineering method for designing SNNs, which replicate recorded biological neural dynamics.

• Weaknesses:

  • It can design SNN by using only BMS spiking neuron models.

  • It generates one SNN for replicating just one neural dynamic pattern.

4.2. Stochastic method: Grammar-based genetic programming

For stochastic reverse engineering methods, evolutionary algorithms have been used; particularly, a variant of well-known genetic programming [36] called grammatical evolution (GE) [37]. Practically, GE is an optimization tool that searches approximated optimal solutions by representing them indirectly for a given problem; thus, working with GE to solve a specific problem requires four components: problem instance(s), representation of solutions, a fitness function to evaluate solution’s quality and a search engine.

For the SCPG design problem, two types of representations have been proposed: one as a Context-Free Grammar (CFG) in Backus-Naur Form (BNF) and another as a Christiansen Grammar (CG) to use GE (in [25, 26]) and a variant called Christiansen Grammar Evolution (CGE) [38] (in [23, 24]), respectively. In general, both representations describe languages that define the presynaptic connectivity (including weights) of a specific spiking neuron; the common structure of expected words of two grammars is as follows: ${\displaystyle \stackrel{1\mathit{st}\mathit{\text{configured synapse}}}{\overbrace{{\mathit{id}}_1,{\mathit{\text{weight}}}_1}}}\mid \cdots \mid \stackrel{n-\mathit{th}\mathit{\text{configur}}\mathit{ed}\mathit{\text{synapse}}}{\overbrace{{\mathit{id}}_n,{\mathit{\text{weight}}}_n}}$. The connectivity defined by a word has at least one connection and a maximum of connections according to number of neurons into the SNNs. The main difference between both representations is that words of CG representation are syntactically and semantically correct, this means that any generated word is valid and there are not repeated indexes, while words of CFG BNF are just syntactically correct or any generated word is just valid.

The fitness function is usually defined by the problem; to solve this, three fitness functions based on SPIKE [39] (used in [23, 24, 26]) and Victor-Purpura distances [40] (used in [25]) to compare similarity between generated spike train and target spike train to guide the search process have been proposed. Basically, first and second distance-based fitness functions are for generating one SCPG per locomotion gait; the difference is that the second one looks for minimal presynaptic connectivity and the first one does not care about number of presynaptic connections. The third distance-based fitness function allows to generate a single SCPG, which can replicate different locomotion gaits.

The search engine in GE is usually a metaheuristic algorithm, which tries to improve the quality of solutions. For reviewed works, three different algorithms have been used: Univariate Marginal Distribution Algorithm [40] (used in 23), (1 + 1)-Evolution Strategy [41] (used in [24]) and Differential Evolution [42] (used in [25, 26]).

For implementation details, see [23, 24, 25]. Next, the features of this method are listed:

• Strengths:

  • It can design SNNs which use either BMS spiking neuron model or LIF neuron model.

  • It can handle design criteria to design compact SNN topologies or SNN, which can replicate different neural dynamics or locomotion gaits.

• Weaknesses:

  • The process must be executed several times to build a single SNN; due to that, it defines synaptic connection and weights one neuron at time.

  • It has not been tested on other design problems than design SCPGs.

5. Discussion and conclusion

Nowadays, autonomous robot locomotion is still a valid problem that has been partially solved in robotics. Particularly, locomotion of nonwheeled robotic platforms is a problem highly susceptible for trying to be solved by means of bioinspired algorithms known as CPGs. However, sometimes working with CPGs may represent a problem itself since its design; this is due to the different choices that must be made before implement the CPG according to [7]. In this chapter, we have explored researches made on SCPG field, a particular type of CPGs, which have been barely explored to date. The SCPGs are built with spiking neurons, a plausible neuron model, which handle similar information as such observed in biological neural systems. We specifically focus on SCPG designed by approaches that allow to dispense with human experts for explicitly define each CPG design phase. These kinds of works use reverse engineering methods to solve de SCPG design problem as an optimization one. By means of these methods, there are generated weighted and directed graphs as SNNs, which endogenously generate rhythmic discrete signals to allow locomotion of legged robots.

Biological CPGs do not work in isolation; they depend on the information interaction with other parts of the central nervous system [32]; even, external afferent inputs are used to shape their outputs [33]. Based on the aforementioned, the next step of SCPG-based locomotion systems could be their integration in navigation systems to endow them with sensors and can build more robust and plausible bioinspired algorithms.

Finally, there are other reasons to keep studying and implementing SCPGs, which go far beyond the locomotion of nonwheeled robots, their possible application in other areas, like medicine in developing prosthetic robotic devices for patients with spinal damage or amputated limbs [20].

Acknowledgments

The authors wish to thank the Consejo Nacional de Ciencia y Tecnología (CONACyT) for the support through the Computational Neuroscience: Theory of Neuromorphic Systems Development project, N. 1961, the University of Guanajuato and National Technology of Mexico. Horacio Rostro-Gonzalez wishes to thank the University of Guanajuato for the support provided through a sabbatical year.