Cooperative Step Climbing Using Connected Wheeled Robots and Evaluation of Remote Operability

Abstract

The present study evaluates the remote operability of step climbing using two connected robots that are teleoperated by individual operators. In general, a teleoperated robot is manipulated by an operator who is viewing moving images from a camera, which is one of the greatest advantages of such a system. However, robot teleoperation is not easy when a teleoperated robot is affected by the force from another robot or object. We constructed a step climbing system using two connected teleoperated robots. A theoretical analysis and the results of simulations clarified the correlations among the robot velocity, the manipulation time of the robots, and the height of the front wheels when climbing a step. The experimental results demonstrate the step climbing ability of the teleoperated robot system.

1. Introduction

A wheeled mechanism can be easily controlled on a flat road and excels in energy efficiency. Many wheelchairs, carts, and robots that are used in offices or houses have wheeled mechanisms. On the other hand, they face problems with the steps that are commonly found in living spaces. Wheeled mechanisms that can navigate steps or stairs have been widely researched. Such studies have examined additional legs [1, 2], a combination of an adjustable center of gravity and multiple wheels [3], special wheels [4, 5], hopping robots [6], additional driving wheel systems [7], and multiple robots that have forklift mechanisms for climbing steps [8].

We have previously reported cooperative step climbing [9] and descending [10] using two wheeled robots, and we studied step climbing using a wheelchair and a wheeled robot connected by a passive link [11]. We also investigated wheelchair step climbing support by a partner robot equipped with dual manipulators [12, 13]. The above studies were conducted using autonomous or teleoperated robots, both of which have merits and demerits. It is therefore necessary to construct a robot system that is most appropriate for the desired purpose.

The purpose of the present paper is to evaluate the performance of two connected wheeled robots that are teleoperated by individual operators. Teleoperated robots can have cameras and manipulators on their bodies that allow them to be controlled by an operator. They may also be controlled by viewing images obtained from an external camera. This ability to be controlled by humans is one of their advantages.

However, the operators need to pay close attention when robots must be operated with pinpoint precision. For example, when teleoperated robots cooperate with each other to transport an object, interactive forces caused by delays in operation act on the robots. The robots must incline to climb a step, and the interactive forces between robots are always changing. This causes errors in movement or step climbing. Thus, step climbing under control by two operators is not simple. This is therefore the subject of the present study.

The remainder of this paper is organized as follows. Section 2 describes the cooperative system, and Section 3 describes the process of climbing a step. Section 4 presents a theoretical analysis and the results of simulations. Section 5 describes the experiment and results, and Section 6 presents a discussion of the experiments. Finally, Section 7 presents the conclusion of the present paper.

2. Cooperative step climbing robot system

The two robots used in the present study were wheeled robots that were developed in our laboratory (Figure 1). Each robot has a pair of front wheels and a pair of rear wheels. The front wheels are casters, and the rear wheels are driving wheels. The robots are connected by a link mechanism, referred to herein as a passive link, and the connecting link positions have free joints. Both robots have mechanisms to change the heights of the connecting positions of the passive link. This step climbing method is affected by the positions of the passive link (see Section 3), and we determined the suitable link positions to overcome a step [14]. The robots are deployed in a forward-and-aft configuration using the link. In the present paper, we refer to the front robot as Robot A and the rear robot as Robot B. Figure 2 shows a schematic of cooperative step climbing (see Appendix, Tables A1 and A2).

Figure 1.

Robots connected by passive link.

Figure 2.

Schematic of cooperative step climbing and descending robots.

Figure 3(a) and (b) shows the configuration of Robots A and B. The motors mounted on the robots are connected to a microcontroller (PIC16F873) through a motor driver circuit. The microcontrollers are connected to the robot’s PC using a ZigBee module for wireless communications.

Figure 3.

Configuration of (a) Robot A and Operator A and (b) Robot B and Operator B.

The robots have ball screw mechanisms to climb a step and are able to change the link height (see Section 3, Figure 4). Robot A has touch sensors on its back. The sensors detect the stopping position of the passive link when the operator of Robot A controls the link height (Figure 4). In the present study, the ball screw mechanism of Robot A was only used, and the operator of Robot A was able to control the link position using a joystick (Figure 5).

Figure 4.

Passive link mechanism.

Figure 5.

Joystick for manipulating Robot A.

Each robot was teleoperated by one operator. In the present paper, we refer to the operator of Robot A as Operator A and the operator of Robot B as Operator B. The operators controlled each robot using a joystick (Figure 5). The robots did not have a system to communicate the information for step climbing; however the operators were able to talk to each other.

Both robots had a camera (ELECOM UCAM-E130HWH, maximum resolution: 1280 × 1024, frame rate: 30 fps (640 × 480 pixels), 10 fps (1280 × 1024 pixels)) on their front (Figure 6). The cameras are connected to the PC using USB cables, and moving images from both cameras (Figure 7(a) and (b)) were displayed on the PC screens used by the operators (Figure 8).

Figure 6.

Cameras on Robot A and Robot B.

Figure 7.

Moving images from (a) Robot A and (b) Robot B.

Figure 8.

PC screen for robot operators.

The angles ${\theta }_A$ and ${\theta }_B$ in Figure 6 are the angles of the cameras on Robot A and B, respectively. These angles were set to ${\theta }_A=$ 10° and ${\theta }_B=$ 30°, which are the angles between the robot and the step that the robots are able to see when Robot A or B inclines to climb a step. Operators A and B teleoperated each robot using only video data from the cameras.

3. Cooperative step climbing method

The proposed method uses the equilibrium of the robots during step climbing. The two connected robots climb a step sequentially. In the present study, stages 1 and 2 indicate the processes in which the front and rear wheels, respectively, of Robot A climb the step. Similarly, stages 3 and 4 signify the processes in which the front and rear wheels, respectively, of Robot B climb the step (Figure 9). The ascent process, as shown in <1>−<16> in Figure 9, is described below. The velocities of the robots are constant in the forward direction.

Figure 9.

Entire process of step climbing.

3.1 Stage 1

<1> Both operators perceive the step using the moving images from the cameras on the robots (Figure 7). The link height of Robot A is set at a high position (Figure 4). <2> Robot B stops, and Robot A moves forward. As a result, the front wheels of Robot A are lifted. <3> The operators make both robots move forward while the front wheels of Robot A are lifted. <4> When the operators recognize that the front wheels of Robot A have passed over the step edge, the operators manipulate the joysticks to adjust the difference in speed between the robots, so that the front wheels of Robot A are placed on the upper level of the step. Here, in stages 1 and 2, if Robot A is faster than Robot B, then the tilt of Robot A increases. If Robot B is faster than Robot A, then the tilt of Robot A decreases.

3.2 Stage 2

<5> The operators make the robots continue to move forward. The back wheels of Robot A come into contact with the step. <6> Robot B pushes Robot A. Robot B supports the climbing of Robot A, and Robot B prevents Robot A from tipping over backward. <7> Robot A climbs up onto the step. <8> Once the rear wheels of Robot A have reached the upper level of the step, the operators stop each robot.

3.3 Stage 3

<9> After stage 2, the link height of Robot A is set at a low position (Figure 6). <10> Operator A makes Robot A stop, and Operator B makes Robot B move forward. As a result, the front wheels of Robot B are lifted. <11> Both robots move forward. <12> The operators recognize that the front wheels of Robot B have passed over the step edge. The operators manipulate each joystick to adjust the difference between the speeds of the robots, so that the front wheels of Robot B are placed on the upper level of the step. Here, in stages 3 and 4, if Robot B is faster than Robot A, the tilt of Robot B increases. If Robot A is faster than Robot B, then the tilt of Robot B decreases.

3.4 Stage 4

<13> The operators make the robots continue to move forward. The back wheels of Robot B come into contact with the step. <14> Robot A pulls Robot B. Robot A supports the climbing of Robot B and Robot A prevents Robot B from tipping over backward. <15> Robot B climbs up onto the step. <16> Once the rear wheels of Robot B have reached the upper level of the step, the operators stop each robot.

4. Theoretical analysis

When Robot A climbs a step, the body of the robot inclines, and its front wheels are lifted due to the difference in velocity between the two connected robots. When the robots are manipulated by the operators, the step climbing ability greatly influences the manipulation time.

In this section, we clarify the relationships among the robot incline, the velocity, and the manipulation time.

4.1 Relationships among the manipulation time, the velocity, and the height of the front wheels required to climb the step

In Figure 10, ${\mathrm{\Sigma }}_B$ is the basic coordinate system for the robots, where $p_0$ is the origin as well as the contact position between the rear wheels of Robot B and the ground. In addition, $p_i$(i = 1–5) are the joints ($p_1$, axis of the rear wheels of Robot B; $p_2$, link position of Robot B; $p_3$, link position of Robot A; $p_4$, axis of the rear wheels of Robot A; $p_5$, axis of the front wheels of Robot A; $p_6$, tread position of the front wheels of Robot A).

Figure 10.

Lifting the front wheels of Robot A (stage 1).

The position vectors for the joints in the coordinate system ${\mathrm{\Sigma }}_B$ are expressed as ${}^B\mathit{p}$_i$={\left[x_i{y}_i\right]}^T$ (i = 1–6). In the local coordinate system, for the case in which ${\mathrm{\Sigma }}_i$ is parallel to ${\mathrm{\Sigma }}_0$, ${}^0\mathit{p}$₁$={\left[\begin{array}{cc}0& R_B\end{array}\right]}^T$, ${}^1\mathit{p}$₂$={\left[\begin{array}{cc}{l}_B+l_{\mathit{LB}}& h_{\mathit{LB}}\end{array}\right]}^T$, ${}^2\mathit{p}$₃$={\left[\begin{array}{cc}d& 0\end{array}\right]}^T$, ${}^3\mathit{p}$₄$={\left[\begin{array}{cc}{l}_{\mathit{LA}}& -h_{\mathit{LA}}\end{array}\right]}^T$, ${}^4\mathit{p}$₅$={\left[\begin{array}{cc}{l}_A& -R_A+r_A\end{array}\right]}^T$, and ${}^5\mathit{p}$₆$={\left[\begin{array}{cc}{r}_A& 0\end{array}\right]}^T$ (Figure 10).

Then, ${\varphi }_i$ is the angle between ${\mathrm{\Sigma }}_i$ and ${\mathrm{\Sigma }}_{i-1}$, and ${\mathrm{\Sigma }}_1$ is parallel to ${\mathrm{\Sigma }}_0$ in stage 1.

Thus,

$${\varphi }_1=0$$E1

The incline of Robot A, ${\sum }_{k=1}^3{\varphi }_i$, is

$$\sum _{k=1}^3{\varphi }_i={\varphi }_2+{\varphi }_3$$E2

Here, ${\mathrm{\Sigma }}_4$ is always parallel to ${\mathrm{\Sigma }}_3$:

$${\varphi }_4=0$$E3

In the basic coordinate system ${\mathrm{\Sigma }}_B$, the homogeneous transformation matrix ${}^B\mathit{T}$₄ is as follows:

$${{}^B\mathit{T}}_4:\left[\begin{array}{ccc}cos{\varphi }_{23}& -sin{\varphi }_{23}& x_4\\ sin{\varphi }_{23}& cos{\varphi }_{23}& y_4\\ 0& 0& 1\end{array}\right]$$E4

Here, ${\varphi }_{23}={\varphi }_2+{\varphi }_3$:

$$x_4=l_{\mathit{LA}}cos{\varphi }_{23}+h_{\mathit{LA}}sin{\varphi }_{23}+l_B+l_{\mathit{LB}}+dcos{\varphi }_2$$E5

$$y_4=l_{\mathit{LA}}sin{\varphi }_{23}-h_{\mathit{LA}}cos{\varphi }_{23}+h_{\mathit{LB}}+R_B+dsin{\varphi }_2$$E6

Then, $y_4$ is equal to $R_A$ (the radius of the rear wheels of Robot A) in stage 1 (Figure 10), and we obtain the following equation from (6):

$$R_A=l_{\mathit{LA}}sin{\varphi }_{23}-h_{\mathit{LA}}cos{\varphi }_{23}+h_{\mathit{LB}}+R_B+dsin{\varphi }_2$$E7

In this system

$$R_A=R_B.$$E8

Thus, we have:

$$sin{\varphi }_2=\frac{h_{\mathit{LA}}cos{\varphi }_{23}-l_{\mathit{LA}}sin{\varphi }_{23}-h_{\mathit{LB}}}{d}$$E9

Here

$$cos{\varphi }_2=\sqrt{1-{sin}^2{\varphi }_2}.$$E10

The homogeneous transformation matrix, ${}^B\mathit{T}$₆, is given by

$${{}^B\mathit{T}}_6:\left[\begin{array}{ccc}cos{\varphi }_{2356}& -sin{\varphi }_{2356}& x_6\\ sin{\varphi }_{2356}& cos{\varphi }_{2356}& y_6\\ 0& 0& 1\end{array}\right]$$E11

Here, ${\varphi }_{2356}={\varphi }_2+{\varphi }_3+{\varphi }_5+{\varphi }_6$.

From (1) and (3), when $p_6$ (the tread position of the front wheels of Robot A, ${}^B\mathit{p}$₆$={\left[\begin{array}{cc}{x}_6& y_6\end{array}\right]}^T$, Figure 10) is at the bottom of the front wheels, ${\sum }_{k=1}^5{\varphi }_i={\varphi }_2+{\varphi }_3+{\varphi }_5={\varphi }_{235}=-90$°, where

$$cos{\varphi }_{235}=0$$E12

and

$$sin{\varphi }_{235}=-1$$E13

Thus, we have:

$$\begin{array}{ll}{x}_6=& l_{A}cos{\varphi }_{23}-\left(-R_A+r_A\right)sin{\varphi }_{23}+l_{\mathit{LA}}cos{\varphi }_{23}\\ & +h_{\mathit{LA}}sin{\varphi }_{23}+l_B+l_{\mathit{LB}}+dcos{\varphi }_2\end{array}$$E14

$$\begin{array}{ll}{y}_6=& -r_A+l_{A}sin{\varphi }_{23}+\left(-R_A+r_A\right)cos{\varphi }_{23}+l_{\mathit{LA}}sin{\varphi }_{23}\\ & -h_{\mathit{LA}}cos{\varphi }_{23}+h_{\mathit{LB}}+R_B+dsin{\varphi }_2.\end{array}$$E15

By substituting (9) for (15), we have:

$$y_6=l_{A}sin{\varphi }_{23}+\left(-R_A+r_A\right)cos{\varphi }_{23}+R_B-r_A$$E16

where

$$sin{\varphi }_{23}=\sqrt{1-{cos}^2{\varphi }_{23}}$$E17

From (8), (16), and (17), we obtain:

$$cos{\varphi }_{23}=\frac{e_1·\left(e_1-y_6\right)+l_A\sqrt{{l_A}^2+{e_1}^2-{\left(e_1-y_6\right)}^2}}{{l_A}^2+{e_1}^2}$$E18

Here, $e_1=R_A-r_A$.

Then, $p_4$ (the position of the axis of the rear wheels of Robot A) moves forward in lifting the front wheels of Robot A (Figure 10). Time $t=\mathit{tm}$is the time for the operators to lift the wheels.

$p_{4\left(t=0\right)}={\left[\begin{array}{cc}{x}_{4\left(t=0\right)}& y_{4\left(t=0\right)}\end{array}\right]}^T$ is the first position of $p_4$. When the robot manipulation time is $t=0$, the tilt of Robot A is zero (${\varphi }_{23}=0$). Then, $p_{4\left(t=0\right)}$ moves to $p_{4\left(t=\mathit{tm}\right)}={\left[\begin{array}{cc}{x}_{4\left(t=\mathit{tm}\right)}& y_{4\left(t=\mathit{tm}\right)}\end{array}\right]}^T$ after Robot A moves at a constant velocity of $v_A$ in $t$ s, and $∆x$ is the distance between $p_{4\left(t=0\right)}$ and $p_{4\left(t=\mathit{tm}\right)}$.

When the operator begins to teleoperate the robots $\left(t=0\right)$, the position of the axis of the rear wheels of Robot A, $x_{4\left(t=0\right)}$, is obtained from (5), as follows:

$$x_{4\left(t=0\right)}=l_{\mathit{LA}}+l_B+l_{\mathit{LB}}+dcos{\varphi }_{2\left(t=0\right)}$$E19

Here, $cos{\varphi }_{2\left(t=0\right)}$ is the value of $cos{\varphi }_2$ at $t=0$ s.

After manipulating the robots for $t=\mathit{tm}$, the position of the axis of the rear wheels of Robot A ($x_{4\left(t=\mathit{tm}\right)}$) is given by

$$x_{4\left(t=\mathit{tm}\right)}=l_{\mathit{LA}}cos{\varphi }_{23\left(t=\mathit{tm}\right)}+h_{\mathit{LA}}sin{\varphi }_{23\left(t=\mathit{tm}\right)}+l_B+l_{\mathit{LB}}+dcos{\varphi }_{2\left(t=\mathit{tm}\right)}$$E20

Here, $cos{\varphi }_{23\left(t=\mathit{tm}\right)}$ is the value of $cos{\varphi }_{23}$ at$t=\mathit{tm}$.

Based on (19) and (20), the movement distance of Robot A while lifting the front wheels, $∆x$ (Figure 10), is given as

$$\begin{array}{ll}∆x=& v_{A}t=x_{4\left(t=\mathit{tm}\right)}-x_{4\left(t=0\right)}\\ & =l_{\mathit{LA}}\left\{cos{\varphi }_{23\left(t=\mathit{tm}\right)}-1\right\}+h_{\mathit{LA}}sin{\varphi }_{23\left(t=\mathit{tm}\right)}+d\left\{cos{\varphi }_{2\left(t=\mathit{tm}\right)}-cos{\varphi }_{2\left(t=0\right)}\right\}\end{array}$$E21

Here, $v_A$ is the constant velocity of Robot A. When the front wheels begin to be lifted ($t=0$, Figure 10), the incline of Robot A is zero (${\varphi }_2+{\varphi }_3=0$). In this case, from (9) and (10), we obtain:

$$sin{\varphi }_{2\left(t=0\right)}=\frac{h_{\mathit{LA}}-h_{\mathit{LB}}}{d}$$E22

and

$$cos{\varphi }_{2\left(t=0\right)}=\sqrt{\frac{d^2-{\left(h_{\mathit{LA}}-h_{\mathit{LB}}\right)}^2}{d}}$$E23

After manipulating the robots for a time $\mathit{tm}$, the front wheels start to lift. Then, from (9) and (10), we obtain the following:

$$sin{\varphi }_{2\left(t=\mathit{tm}\right)}=\frac{h_{\mathit{LA}}cos{\varphi }_{23\left(t=\mathit{tm}\right)}-l_{\mathit{LA}}sin{\varphi }_{23\left(t=\mathit{tm}\right)}-h_{\mathit{LB}}}{d}$$E24

and

$$cos{\varphi }_{2\left(t=\mathit{tm}\right)}=\sqrt{\frac{d^2-{e_2}^2}{d}}$$E25

Here,

$$e_2=h_{\mathit{LA}}cos{\varphi }_{23\left(t=\mathit{tm}\right)}-l_{\mathit{LA}}sin{\varphi }_{23\left(t=\mathit{tm}\right)}-h_{\mathit{LB}}.$$E26

From (17) and (18), we obtain $sin{\varphi }_{2\left(t=\mathit{tm}\right)}$ and $cos{\varphi }_{2\left(t=\mathit{tm}\right)}$, as follows:

$$sin{\varphi }_{23\left(t=\mathit{tm}\right)}=\sqrt{1-{cos}^2{\varphi }_{23\left(t=\mathit{tm}\right)}}$$E27

and

$$cos{\varphi }_{23\left(t=\mathit{tm}\right)}=\frac{e_1·\left(e_1-y_6\right)+l_A\sqrt{{l_A}^2+{e_1}^2-{\left(e_1-y_6\right)}^2}}{{l_A}^2+{e_1}^2}$$E28

Substituting (22) and (23) for (21), we obtain:

$$t=\frac{1}{v_A}\left[l_{\mathit{LA}}\left(cos{\varphi }_{23\left(t=\mathit{tm}\right)}-1\right)-\sqrt{d^2-{\left(h_{\mathit{LA}}-h_{\mathit{LB}}\right)}^2}+\sqrt{d^2-{e_2}^2}+h_{\mathit{LA}}sin{\varphi }_{23\left(t=\mathit{tm}\right)}\right].$$E29

From (26)–(29), the relationships among the velocity of Robot A ($v_A$), the height of the bottom of the front wheels ($y_6$) (Figure 11(a)), and the manipulation time for lifting the front wheels ($t=\mathit{tm}$) are clarified.

Figure 11.

Height range of the front wheels of Robot A required to climb a step: (a) situation in which the bottom of the front wheels is equal to the step height and (b) situation in which the horizontal position of the center of gravity is at the contact position between the rear wheels and the road.

4.2 Range of front wheel height within which operators must teleoperate for step climbing

In stage 1, the operators lift the bottom of the front wheels of Robot A above the step height in order to place the wheels on the step (Figure 11(a)). Next, the operators maintain a suitable inclination for Robot A in order to prevent it from tipping over backward (Figure 11(b)). When Robot B stops and Robot A continues to move forward, Robot A tips over backward (Figure 12). Hence, the operators must maintain the height of the front wheels within the range between the step height and the height at which Robot A tips over backward.

Figure 12.

Tipping over backward of Robot A.

When the horizontal position of the center of gravity is at the contact position between the rear wheels and the road (Figure 11(b)), the height of the bottom of the front wheels is $y_6=$ 0.1526 m. On the other hand, Robot B does not tip over backward during stage 3 because the link touches its body when the incline of Robot B grows large (Figure 13), so stopping further inclination. The operators are able to teleoperate Robot B without tipping the robot over backward.

Figure 13.

Link touching the front part of Robot B, which acts to prevent incline of Robot A.

4.3 Simulation

We performed a numerical calculation to clarify the correlations among the manipulation time to lift the front wheels, the robot velocity, and the height of the front wheels using (29).

The horizontal axes in Figure 14(a)–(c) show the manipulation time for lifting of the front wheels, $t=\mathit{tm}$, and the vertical axes show the height of the bottom of the front wheels ($y_6$), indicating the correlations when the velocity of Robot A is $v_A=$ 0.3, 0.4, and 0.5 km/h, respectively.

Figure 14.

Relationships among the manipulation time ($\mathit{t}=\mathit{tm}$), the velocity of Robot A (${\mathit{v}}_{\mathit{A}}$), and the height of the front wheels of Robot A (${\mathit{y}}_{\mathit{6}}$) for (a) ${\mathit{v}}_{\mathit{A}}=$ 0.3 km, (b) ${\mathit{v}}_{\mathit{A}}=$ 0.4 km, and (c) ${\mathit{v}}_{\mathit{A}}=$ 0.5 km/h.

When Robot A moves at 0.3 km/h and climbs a 0.05-m-high step, 0.228 s is required to lift the bottom of the front wheels to the step height (Figure 11(a)), and 0.559 s is the time when the horizontal position of the center of gravity of Robot A is the contact position between the rear wheels and the road surface (Figure 11(b)). Therefore, Operator A must complete the lifting operation in a time between 0.228 and 0.559 s. If tm is less than 0.228 s, the bottom of the front wheels ($y_6$) will not reach a height of 0.05 m, and if tm is greater than 0.559 s, Robot A will tip over backward. In Figure 14(a), $t_3$, which 0.331 s, is the time between these two events. Thus, operators teleoperate the robots to lift the front wheels of Robot A and must stop the incline after $0.331$ s.

Similarly, $t_4$ and $t_5$ are the times at which Robot A moves at 0.4 and 0.5 km/h, respectively, where $t_4=$ 0.248 s and $t_4=$ 0.199 s. The results for $t_3$ and $t_5$ reveal that $t_3$ is more than 66.33% of $t_5$.

In Sections 5 and 6, we discuss the influence of the velocity difference for manipulation of the robots.

5. Experiment

The two robots were teleoperated by individual operators (Figure 15). Using joysticks (Figure 5), the robots were moved at speeds that were set by the program. Six subjects (five adult males and an adult female) participated as robot operators in the experiments.

Figure 15.

Step climbing experiment.

The six subjects were labeled $s_1$ to $s_6$ and were divided into three groups, $\alpha$, $\beta$, and $\gamma$. Subjects $s_1$ and $s_2$ were the operators in group $\alpha$, subjects $s_3$ and $s_4$ were the operators in group $\beta$, and subjects $s_5$ and $s_6$ were the operators in group $\gamma$. Subjects $s_1$, $s_3$, and $s_5$ operated Robot A, and subjects $s_2$, $s_4$, and $s_6$ operated Robot B.

Three experiments were conducted, in which the robot velocities were 0.3, 0.4, and 0.5 km/h. The step height and the friction coefficient were constant at $h=$ 0.05 m and $\mu =$ 0.72.

The subjects understood the step climbing process and learned how to teleoperate the robots before the experiments. The subjects repeated the test 20 times for each experiment, as was explained before the experiments. When either robot was unable to climb the step, the reason for the failure was recorded. The postures of the robots were then corrected, and the operators restarted the test.

The case in which either of the robots was not able to climb the step was taken to be a step climbing failure. The case in which the both robots were able to climb the step was taken to be a step climbing success.

Tables 1–3 show the results of the experiments for the cases in which the moving robot velocities were 0.3, 0.4, and 0.5 km/h, respectively. The numbers listed in the tables are the test numbers when the robots failed to climb the step, and Col.AS, Tip.A, Col.BS, and Tip.B are the reasons for failure. Here, Col.AS, Tip.A, Col.BS, and Tip.B indicate a collision between the front wheels of Robot A and the step wall (Figure 16), tipping over backward of Robot A (Figure 12), a collision between the front wheels of Robot B and the step wall, and tipping over backward of Robot B, respectively. However, as a result of the link touching its body, tipping over backward of Robot B did not occur in the experiments (see Figure 13).

Figure 16.

Collision between the front wheels and the step wall.

Group	Success rate (%)	Reason for failure and test number
		[Col. AS]	[Tip.A]	[Col.BS]	[Tip.B]
$\alpha$	85	3; 11; 15	—; —; —	—; —; —	—; —; —
$\beta$	85	1; 8; 11	—; —; —	—; —; —	—; —; —
$\gamma$	90	3; 4	—; —	—; —	—; —

Table 1.

Success rate for step climbing (0.3 km/h), reason for failure, and test number.

Group	Success rate (%)	Reason for failure and test number
		[Col. AS]	[Tip.A]	[Col.BS]	[Tip.B]
$\alpha$	100	—	—	—	—
$\beta$	75	2; 3; 7; —; 20	—; —; —; —; —	—; —; —; 11; —	—; —; —; —; —
$\gamma$	100	—	—	—	—

Table 2.

Success rate for step climbing (0.4 km/h), reason for failure, and test number.

Group	Success rate (%)	Reason for failure and test number
		[Col. AS]	[Tip.A]	[Col.BS]	[Tip.B]
$\alpha$	75	1; —; —; —; —	—; 2; —; —; —	—; —; 4; 7; 8	—; —; —; —; —
$\beta$	80	—; —; —; —	—; 18; 19; 20	14; —; —; —	—; —; —; —
$\gamma$	100	—	—	—	—

Table 3.

Success rate for step climbing (0.5 km/h), reason for failure, and test number.

In the first experiment (Table 1: velocity, 0.3 km/h; success, 52 times; failure, eight times), the success rates for groups $\alpha$, $\beta$, and $\gamma$ were 85, 85, and 90%, respectively. The reason for failure for all of the groups was a collision between the front wheels of Robot A and the step wall.

In the second experiment (Table 2: velocity, 0.4 km/h; success, 55 times; failure, five times), the success rates for groups $\alpha$, $\beta$, and $\gamma$ were 100, 75, and 100%, respectively. The reason for failure for all of the groups was collision between the front wheels and the step wall (Robot A, four times; Robot B, one time). In the third experiment (Table 3: velocity, 0.5 km/h; success, 51 times; failure, nine times), the success rates for groups $\alpha$, $\beta$, and $\gamma$ were 75, 80, and 100%, respectively.

The reasons for failure for the groups were collision between the front wheels of Robot A and the step wall (one time), tipping over of Robot A (four times), and collision between the front wheels of Robot B and the step wall (four times).

Table 4 lists the ratios for the reasons for failure of the robots to climb the step. The total number of failures for group $\mathrm{\alpha }$ was 8 (out of 60 tests), and the total number of failures for groups $\beta$ and $\gamma$ were 12 and 2, respectively (Tables 1–3).

Group (total number of failures)	Ratio of reason for failure
	[Col. AS] (%)	[Tip.A] (%)	[Col.BS] (%)	[Tip.B]
$\alpha$ (8)	50	12.5	37.5	0
$\beta$ (12)	58.33	25	16.67	0
$\gamma$ (2)	100	0	0	0
Three groups	59.09	18.18	22.73	0

Table 4.

Ratio of reason for failure of the robots to climb the step (0.3–0.5 km/h).

The most common reason for failure is collision between the front wheels of Robot A and the step (59.09%). The second most common reason is collision between the front wheels of Robot B and the step (22.73%). Therefore, approximately 82% of failures arise from collisions between the front wheels and the step. In other words, if a robot is fitted with an assistance system that is able to detect the distance between the robot and the step, the capabilities of teleoperated robots should be greatly improved.

6. Discussion

In this section, we discuss the remote operability of the proposed system based on the results of experiments.

6.1 Correlation between the robot velocity and the success rate

Figure 17(a) shows the correlation between the robot velocity and the success rate for step climbing for the three experiments by group (Tables 1–3). The success rate for group γ was high as a whole and increased to 100% at 0.4 and 0.5 km/h. In contrast, the success rate for group $\alpha$ increased at 0.4 km/h but decreased at 0.5 km/h. The success rate for group β decreased at 0.4 km/h and increased at 0.5 km/h. Therefore, the trends in the experimental results are not consistent.

Figure 17.

Ratio of successful step climbing: (a) success rate for each group and (b) total success rate for the three groups.

Figure 17(b) shows the total success rate at 0.3–0.5 km/h for the three groups (Tables 1–3). The results of the experiments were different from what we had expected, and the remote operability of the step climbing system did not depend on the velocity of the robots. If the velocity is sufficiently higher than 0.5 km/h, the success rate is likely to be reduced. The operators had to manipulate the robots hurriedly in the experiment in which the velocity was 0.5 km/h. However, the step climbing method was based on the premise that the robots move slowly and in balance with each other and using fast robots is beyond the scope of the proposed method.

6.2 Teleoperation skill

As mentioned above, all subjects knew the step climbing procedure before the experiments and became sufficiently proficient at teleoperating the robots. The experiments were carried out at velocities of 0.3, 0.4, and 0.5 km/h, and 20 tests were performed for each velocity. Thus, each subject performed a total of 60 tests. Tables 1–3 show that the failures in the experiments by the three groups occurred not only in the early stages of each experiment (1st–7th test) but also in the final stages (14th–20th test). If the subjects did not have sufficient skill to operate the robots, failures should frequently have occurred in the early stages of the experiments at 0.3 km/h. The skill of the operators would be expected to improve as the tests progressed, thus reducing the failure rate in the later tests at each velocity. Also, if the subjects did not have enough skill to react to the speed of the robot, failures should frequently have occurred in the early stages of each experiment (1st–7th test). However, from the results (Tables 1–3), failures also occurred in the final stages (14th–20th test), and the success rate was not improved except for group γ (Figure 17(a)). Thus, as far as these experiments are concerned, it is clear that the reason for failure was not lack of operator skill.

6.3 Lack of information during teleoperation

Based on interviews with subjects after the experiments, one reason for failure was a lack of information while teleoperating the robots. In these experiments, each robot had one camera on its body, and each subject teleoperated a robot while viewing moving images. The inclines of the robot cameras were set such that the subjects were always able to view the moving images of Robot A or the step (Figure 7(a) and (b)). The subjects had to piece together the status of the climbing robots based on information from the moving images.

We conducted an additional experiment in which a fixed external camera was installed that could view both robots at the same time (see Appendix B, Figures B1 and B2). This experiment was performed using only a single group. However, based on the results, we are fairly certain that using an external camera is effective for teleoperation.

6.4 Losing concentration during teleoperation and conversation between operators

Table 5 lists the total success rate for step climbing for each group (0.3–0.5 km/h). The results for group $\gamma$ were the best (96.67%), and the results for group $\beta$ were the worst (80%). There is a difference of approximately 17% for the total success rate between groups $\gamma$ and $\beta$, which is not small.

Group	Total success rate for step climbing (%)
$\alpha$; $\beta$; $\gamma$	86.67; 80; 96.67

Table 5.

Total success rate for step climbing for each group (0.3–0.5 km/h).

In the experiment in which the velocity of the robots was 0.5 km/h, the operators had to teleoperate the robots hurriedly, and as a whole did not have enough time to converse while manipulating the robots.

The subjects in group $\gamma$ ($s_5$ and $s_6$) were continuously conversing with each other throughout the experiments and frequently talked about their actions and what they intended to do next. As such, conversation was judged to make up for the lack of information during teleoperating the robots.

In contrast, the subjects of group $\alpha$ ($s_1$ and $s_2$) conversed little with their partners after the second experiment ($v_A=$ 0.4 km/h) because they felt that they were skillful operators and were able to manipulate the robots without conversation. In addition, these subjects were tired from repeating the experiments. The subjects of group $\beta$ ($s_3$ and $s_4$) had few conversations throughout the experiments.

6.5 Results of the experiments

Based on the results of the experiments, it is clear that the cooperative step climbing method can be performed using teleoperated robots. However, it seems reasonable to assume that the ability of the robot system was greatly influenced by loss of concentration and conversation between the robot operators. Even if the operators had sufficient skill to manipulate the robot, they sometimes became tired, and did not converse. The construction of an assist system for manipulation should improve the step climbing ability.

7. Conclusion

The present paper described cooperative step climbing using two wheeled robots connected by a passive link. We constructed a teleoperated robot system and carried out experiments. Our conclusions are as follows:

1. The cooperative step climbing method is practical even if the robots are controlled by teleoperation.

2. A theoretical analysis and the results of simulations clarified the correlations among the manipulation time for the robot, the velocity of the robot, and the height of the front wheels in climbing a step.

3. The ability of operators who reached sufficient proficiency in teleoperating the robots does not depend on the velocity of the robots.

4. It was difficult to perform teleoperation using only moving images from the cameras on the robots because the operators were not able to recognize the overall status of the robots during step climbing.

5. Approximately 82% of the step climbing failures were due to collisions between the front wheels and the step. If the robots have an assistance system that can detect the distance between the robot and a step, the capabilities of such teleoperated robots should improve greatly.

6. Loss of concentration by the operators greatly influenced the operation. Even if the operators had sufficient skill to manipulate the robot, when they became tired, the success rate for step climbing decreased. The robots are connected by a link and are affected by the force exerted by each other. Therefore, when one or both operators lose concentration, the robots are not able to ascend the step.

7. It is reasonable to assume that conversation between the operators made up for lack of information during teleoperation.

In the future, we intend to construct an augmented reality system to improve remote operability and to perform experiments to confirm its validity. In addition, we will construct an autonomous robot that has sensors and stereo cameras.

Overall length	315 mm
Overall height	395 mm
Radius of front wheels ($r_A$)	45 mm
Radius of rear wheels ($R_A$)	80 mm
Wheelbase ($l_A$)	190 mm
Position of gravity center ($l_{\mathit{rA}}$)	82 mm
Height of gravity center ($h_{\mathit{mA}}$)	74 mm
Link position from rear axle ($l_{\mathit{LA}}$)	65 mm
Height of link position ($h_{\mathit{LA}}$)	40–240 mm
Camera height ($h_{\mathit{CA}}$)	40 mm
Camera position ($l_{\mathit{CA}}$)	95 mm
Mass ($M_A$)	11.2 kg
Length of link ($d$)	200 mm

Table A1.

Specifications of Robot A.

Overall length	315 mm
Overall height	365 mm
Radius of front wheels ($r_B$)	45 mm
Radius of rear wheels ($R_B$)	80 mm
Wheelbase ($l_B$)	190 mm
Position of gravity center ($l_{\mathit{rB}}$)	92 mm
Height of gravity center ($h_{\mathit{mB}}$)	75 mm
Link position from front axle ($l_{\mathit{LB}}$)	50 mm
Height of link position ($h_{\mathit{LB}}$)	160 mm
Camera height ($h_{\mathit{CB}}$)	300 mm
Camera position ($l_{\mathit{CB}}$)	75 mm
Mass ($M_B$)	9 kg

Table A2.

Specifications of Robot B.

Group $\alpha$ conducted 20 tests (maximum velocity 0.3 km/h). Table B1 lists the experimental results. It can be seen that the success rate was improved from 85 (Table 1) to 95%. Based on interviews with subjects $s_1$ and $s_2$ (operators in group $\alpha$), teleoperation in this manner was easier than using cameras on the robots. Although the experimental results were obtained using only one group, we are fairly certain that using an external camera is effective for teleoperation. If the environment does not allow for the placement of an external camera, the robots should have a sensor system that can show the status of the robots and the step for assistance in teleoperation.

Figure B1.

A schematic of the setup used in an experiment in which an external camera (iBuffalo BSW3KMW01, maximum frame rate: 30 fps) was used to view both the robots and the step together.

Figure B2.

The robots did not have a camera on their body in this case, but the operators were able to determine the status of the robots using the external camera.

Group	Success rate	Reason for failure and test number
		[Col. AS]	[Tip.A]	[Col.BS]	[Tip.B]
$\alpha$	95%	18th	—	—	—

Table B1.

Ratio of reason for failure of the robots to climb the step (0.3–0.5 km/h).