## 1. Introduction

Underactuation, impulsive nature of the impact with the environment, the existence of feet structure and the large number of degrees of freedom are the basic problems in control of the biped robots. Underactuation is naturally associated with dexterity [1]. For example, headstands are considered dexterous. In this case, the contact point between the body and the ground is acting as a pivot without actuation. The nature of the impact between the lower limbs of the biped walker and the environment makes the dynamic of the system to be impulsive. The foot-ground impact is one of the main difficulties one has to face in design of robust control laws for biped walkers [2]. Unlike robotic manipulators, biped robots are always free to detach from the walking surface and this leads to various types of motions [2]. Finally, the existence of many degrees of freedom in the mechanism of biped robots makes the coordination of the links difficult. According to these facts, designing practical controller for biped robots remains to be a challenging problem [3]. Also, these features make applying traditional stability margins difficult.

In fully actuated biped walkers where the stance foot remains flat on the ground during single support phase, well known algorithms such as the Zero Moment Point (ZMP) principle guarantees the stability of the biped robot [4]. The ZMP is defined as the point on the ground where the net moment generated from ground reaction forces has zero moment about two axes that lie in the plane of ground. Takanishi [5], Shin [6], Hirai [7] and Dasgupta [8] have proposed methods of walking patterns synthesis based on ZMP. In this kind of stability, as long as the ZMP lies strictly inside the support polygon of the foot, then the desired trajectories are dynamically feasible. If the ZMP lies on the edge of the support polygon, then the trajectories may not be dynamically feasible. The Foot Rotation Indicator (FRI) [9] is a more general form of the ZMP. FRI is the point on the ground where the net ground reaction force would have to act to keep the foot stationary. In this kind of stability, if FRI is within the convex hull of the stance foot, the robot is possible to walk and it does not roll over the toe or the heel. This kind of walking is named as fully actuated walking. If FRI is out of the foot projection on the ground, the stance foot rotates about the toe or the heel. This is also named as underactuated walking. For bipeds with point feet [10] and Passive Dynamic walkers (PDW) [11] with curved feet in single support phase, the ZMP heuristic is not applicable. Westervelt in [12] has used the Hybrid Zero Dynamics (HZD) [13], [14] and Poincaré mapping method [15]-[18] for stability of RABBIT using underactuated phase. The controller proposed in this approach is organized around the hybrid zero dynamics so that the stability analysis of the closed loop system may be reduced to a one dimensional Poincaré mapping problem. HZD involves the judicious choice of a set of holonomic constraints that were imposed on the robot via feedback control [19]. Extracting the eigenvalues of Poincaré return map is commonly used for analyzing PDW robots. But using of eigenvalues of Poincaré return maps assumes periodicity and is valid only for small deviation from limit cycle [20].

The ZMP criterion has become a very powerful tool for trajectory generation in walking of biped robots. However, it needs a stiff joint control of the prerecorded trajectories and this leads to poor robustness in unknown rough terrain [20] while humans and animals show marvelous robustness in walking on irregular terrains. It is well known in biology that there are Central Pattern Generators (CPG) in spinal cord coupling with musculoskeletal system [21]-[23]. The CPG and the feedback networks can coordinate the body links of the vertebrates during locomotion. There are several mathematical models which have been proposed for a CPG. Among them, Matsuoka's model [24]-[26] has been studied more. In this model, a CPG is modeled by a Neural Oscillator (NO) consisting of two mutually inhibiting neurons. Each neuron in this model is represented by a nonlinear differential equation. This model has been used by Taga [22], [23] and Miyakoshi [27] in biped robots. Kimura [28], [29] has used this model at the hip joints of quadruped robots.

The robot studied in this chapter is a 5-link planar biped walker in the sagittal plane with point feet. The model for such robot is hybrid [30] and it consists of single support phase and a discrete map to model the frictionless impact and the instantaneous double support phase. In this chapter, the goal is to coordinate and control the body links of the robot by CPG and feedback network. The outputs of CPG are the target angles in the joint space, where P controllers at joints have been used as servo controllers. For tuning the parameters of the CPG network, the control problem of the biped walker has been defined as an optimization problem. It has been shown that such a control system can produce a stable limit cycle (i.e. stride). The structure of this chapter is as follows. Section 2 models the walking motion consisting of single support phase and impact model. Section 3 describes the CPG model and tuning of its parameters. In Section 4, a new feedback network is proposed. In Section 5, for tuning the weights of the CPG network, the problem of walking control of the biped robot is defined as an optimization problem. Also the structure of the Genetic algorithm for solving this problem is described. Section 6 includes simulation results in MATLAB environment. Finally, Section 7 contains some concluding remarks.

## 2. Robot model

The overall motion of the biped involves continuous phases separated by abrupt changes resulting from impact of the lower limbs with the ground. In single support phase and double support phase, the biped is a mechanical system that is subject to unilateral constraints [31]-[33]. In this section, the biped robot has been assumed as a planar robot consisting of

where

## A. Single support phase

Figure 1 depicts the single support phase and configuration variables of a 5-link biped robot (

where

(3) |

where

where

where

With setting

(7) |

where

where

## B. Frictionless impact model

In this section, following assumptions are done for modeling the impact [40]:

A1. the impact is frictionless (i.e.

A2. the impact is instantaneous;

A3. the reaction forces due to the impact at impact point can be modeled as impulses;

A4. the actuators at joints are not impulsive;

A5. the impulsive forces due to the impact may result in instantaneous change in the velocities, but there is no instantaneous change in the positions;

A6. impact results in no slipping and no rebound of the swing leg; and

A7. stance foot lifts from the ground without interaction.

With these assumptions, impact equation can be expressed by the following equation

where

This equation is solvable if the coefficient matrix has full rank. The determinant of the coefficient matrix is equal to

where

and also

After solving these equations, it is necessary to change the coordinates since the former swing leg must now become the stance leg. Switching due to the transfer of pivot to the point of contact is done by relabeling matrix [39], [40]

The final result is an expression for

In equation (16),

where

where

where

the hybrid model of the mechanical system can be given by

where

## 3. Control system

Neural control of human locomotion is not yet fully understood, but there are many evidences suggesting that the main control of vertebrates is done by neural circuits called central pattern generators (CPG) in spinal cord which have been coupled with musculoskeletal system. These central pattern generators with reflexes can produce rhythmic movements such as walking, running and swimming.

## A. Central pattern generator model

There are several mathematical models proposed for CPG. In this section, neural oscillator model proposed by Matsuoka has been used [24], [25]. In this model, each neural oscillator consists of two mutually inhibiting neurons (i.e. extensor neuron and flexor neuron). Each neuron is represented by the following nonlinear differential equations

(22) |

where suffixes

The positive or negative value of

## B. Tuning of the CPG parameters

The walking period is a very important factor since it much influences stability, maximum speed and energy consumption. The walking mechanism has its own natural frequency determined mainly by the length of the links of the legs. It appears that humans exploit the natural frequencies of their arms, swinging pendulums at comfortable frequencies equal to the natural frequencies [26]. Human arms can be thought of as masses connected by springs, whose frequency response makes the energy and the control required to move the arm vary with frequency [26]. Humans certainly learn to exploit the dynamics of their limbs for rhythmic tasks [42], [43]. Robotic examples of this idea include open-loop stable systems where the dynamics are exploited giving systems which require little or no active control for stable operation (e.g. PDW [11]). At the resonant frequency, the control need only inject a small amount of energy to maintain the vibration of the mass of the arm segment on the spring of the muscles and tendons. Extracting and using the natural frequency of the links of the robots is a desirable property of the robot controllers. According to these facts, we match the endogenous frequency of each neural oscillator with the resonant frequency of the corresponding link. On the other hand, when swinging or supporting motions of the legs are closer to the free motion, there will not be any additional acceleration and deceleration and the motion will be effective [44]. When no input is applied to the CPG, the frequency of it is called endogenous frequency. Endogenous frequency of the CPG is mainly determined

by

Table I specifies the lengths, masses and inertias of each link of the robot studied in this chapter [3]. By these data and extracting and using resonant frequencies of the links, we match the endogenous frequency of the CPG with the resonant frequency of each link. In this case,

## 4. Feedback network

It is well known in biology that the CPG network with feedback signals from body can coordinate the members of the body, but there is not yet a suitable biological model for feedback network. The control loop used in this section is shown in the Fig. 2 where

In animals, the stretch reflexes act as feedback loop [44]. In this section, the feedback signals to the CPG neurons of the hip joints are the tonic stretch reflex as follows [22], [23]

where

One of important factors in control of walking is the coordination of the knee and the hip joints in each leg. For tuning the phase difference between the oscillators of the knee and the hip joints in each leg, we propose the following feedback structure which is applied only at oscillators of the knee joints

(26) |

where

## 5. Tuning of the weights in the CPG network

The coordination and the phase difference among the links of the biped robot in the discussed control loop are done by the synaptic weights of connections in the CPG network. There are two kinds of connections in the CPG network. One of them is the connections among the flexor neurons and the other one is the connections among the extensor neurons. The neural oscillators in the CPG network can be relabeled as shown in the Fig. 3. According to this relabeling law,

NO1, NO2, NO3 and NO4 correspond to the right knee, the right hip, the left hip and the left knee neural oscillators, respectively. We show the weight matrix among the flexor and extensor neurons by

(27) |

This symmetry can be given by the following equations

In this chapter, we assume

where

and

where

The second sub cost function in (29) can be defined as the least value of the normalized height of the CoM of the mechanical system during simulation and it can be given by

where

The regulation of the rate change of the angular momentum about the CoM is not a good indicator of whether a biped will fall but the reserve in angular momentum that can be utilized to help recover from push or other disturbance is important. We use the rate change of the angular momentum about the CoM for defining the third sub cost function. With

setting

where

where

Hence, the third sub cost function is defined as following

In this chapter,

By using Genetic algorithm, the optimal solution can be determined. Genetic algorithm is one of the evolutionary algorithms based on the natural selection. In this section, the size of each generation in this algorithm is equal to 400, and at the end of each generation, 50% of chromosomes are preserved and the others are discarded. The roulette strategy is employed for selection and 100 selections are done by this strategy. With applying one-point crossover, 200 new chromosomes are produced. The mutation is done for all of the chromosomes with the probability of 10% except the elite chromosome which has the most fitness. Also, each parameter is expressed in 8 bits.

## 6. Simulation results

In this section, the simulation of a 5 link planar biped robot is done in MATLAB environment. Table I specifies the lengths, masses and inertias of each link of the robot. This is the model of RABBIT [3]. RABBIT has

The period of the neural oscillators in the biped robot with the best fitness is equal to

Control signals of the servo controllers during

For evaluating the robustness of the limit cycle of the closed loop system, an external force as disturbance is applied to the body of the biped robot. We assume that the external force is applied at the center of mass of the torso and it can be given by

## 7. Conclusion

In this chapter, the hybrid model was used for modeling the underactuated biped walker. This model consisted of single support phase and the instantaneous impact phase. The double support phase was also assumed to be instantaneous. For controlling the robot in underactuated walking, a CPG network and a new feedback network were used. It is shown that the period of the CPG is the most important factor influencing the stability of the biped walker. Biological experiments show that humans exploit the natural frequencies of their arms, swinging pendulums at comfortable frequencies equal to the natural frequencies. Extracting and using the natural frequency of the links of the robots is a desirable property of the robot controller. According to this fact, we match the endogenous frequency of each neural oscillator with the resonant frequency of the corresponding link. In this way, swinging motion or supporting motion of legs is closer to free motion of the pendulum or the inverted pendulum in each case and the motion is more effective.

It is well known in biology that the CPG network with feedback signals from body can coordinate the members of the body, but there is not yet a suitable biological model for feedback network. In this chapter, we use tonic stretch reflex model as the feedback signal at the hip joints of the biped walker as studied before. But one of the most important factors in control of walking is the coordination or phase difference between the knee and the hip joints in each leg. We overcome this difficulty by introducing a new feedback structure for the knee joints oscillators. This new feedback structure forces the mechanical system to fix the stance knee at a constant value during the single support phase. Also, it forces the swing knee oscillator to increase its output at the beginning of swinging phase and to decrease its output at the end of swinging phase.

The coordination of the links of the biped robot is done by the weights of the connections in the CPG network. For tuning the synaptic weight matrix in CPG network, we define the control problem of the biped walker as an optimization problem. The total cost function in this problem is defined as a summation of the sub cost functions where each of them evaluates different criterions of walking such as distance travelled by the biped robot in the sagittal plane, the height of the CoM and the regulation of the angular momentum about the CoM. By using Genetic algorithm, this problem is solved and the synaptic weight matrix in CPG network for the biped walker with the best fitness is determined. Simulation results show that such a control loop can produce a stable and robust limit cycle in walking of the biped walker. Also these results show the ability of the proposed feedback network in correction of the CPG outputs. This chapter also shows that by using the resonant frequencies of the links, the number of unknown parameters in the CPG network is reduced and hence applying Genetic algorithm is easier.