Dynamic Optimized Bandwidth Management for Teleoperation of Collaborative Robots

Abstract

A real-time dynamic and optimized bandwidth management algorithm is proposed and used in teleoperated collaborative swarms of robots. This method is effective in complex teleoperation tasks, where several robots rather than one are utilized and where an extensive amount of exchanged information between operators and robots is inevitable. The importance of the proposed algorithm is that it accounts for Interesting Events (IEs) occurring in the system's environment and for the change in the Quality of Collaboration (QoC) of the swarm of robots in order to allocate communication bandwidth in an optimized manner. A general dynamic optimized bandwidth management system for teleoperation of collaborative robots is formulated in this paper. The suggested algorithm is evaluated against two static algorithms applied to a swarm of two humanoid robots. The results demonstrate the advantages of dynamic optimization algorithm in terms of task and network performance. The developed algorithm outperforms two static bandwidth management algorithms, against which it was tested, for all performance parameters in 80% of the performed trials. Accordingly, it was demonstrated that the proposed dynamic bandwidth optimization and allocation algorithm forms the basis of a framework for algorithms applied to real-time highly complex systems.

1. Introduction

Teleoperation, or remote tele-manipulation of robots in inaccessible environments such as deep sea and outer space [1, 2], includes a wide variety of applications such as micro- and nano-teleoperation [3], tele-surgery [4], etc. A network-based teleoperation system involves distant interactions between human operators and remote robotic systems [5, 6]. In addition, swarms of robots are widely employed in complex tasks that cannot be performed by a single robot or in tasks that are better achieved by cooperation of robots such as localization in formations [7], target tracking [8], mapping and localization [9], object pushing [10], area exploration for search and rescue [11], etc. When swarms of mobile robots are teleoperated, specific network requirements should be satisfied in order to guarantee a minimum quality of control, which results in efficient task execution. Research done on teleoperated systems showed that constraints such as bandwidth and CPU processing cause the Quality of Service (QoS) to degrade to an extent that may severely affect performance [12, 13]. To address this problem, various bandwidth management algorithms have been presented for distributed multimedia systems in order to maintain a performance that guarantees an adequate QoS [6]. However, the literature rarely tackled the problem of managing bandwidth based on sensory feedback and the quality of collaboration among robots. Accordingly, a real-time dynamic optimized bandwidth management for teleoperation of collaborative robots is introduced in this paper. The proposed method accounts for interesting events (IEs) and the change in the quality of collaboration (QoC) between robots in order to optimize the allocation of bandwidth between acting agents, where necessary, in a given environment. The developed optimization technique showed outstanding performances when implemented on a system of two collaborating humanoid robots, and thus could be considered a basis for a framework for highly complex algorithms implemented in systems involving real-time bandwidth optimization, where multiple users control multiple collaborating robots.

Different types of resource management algorithms are used to solve the bandwidth allocation problem in robotics systems. Such applications in networked control systems fall into two main categories: static [14] and dynamic [15] bandwidth allocation. Static methods cannot adapt to changes in the system state (surrounding environment, collaboration quality, etc.). Alternatively, dynamic bandwidth allocation algorithms increase performance at the cost of increased computation. Mourikis et al. [7] address the problem of resource allocation in formations of mobile robots localizing as a group. The goal is to determine the frequency at which each individual sensor should be used in order to attain the highest possible localization accuracy. The set of frequencies mentioned is obtained by solving an optimization problem that maximizes the accuracy matrix expressed in terms of the sensors' frequencies. However, the problem is solved offline and the algorithm does not account for any dynamic events that might occur. Sugiyama et al. [8] propose a bandwidth reservation algorithm for multi-robot systems in a target tracking mission. The interesting information, corresponding to a survivor’s detection, is sent to the base station with wideband signals such as dynamic picture images. The final call is left to operators to decide whether the received images indicate a real victim, by allowing/preventing the corresponding robot to reserve the bandwidth affecting the flow of various signals from other robots to the base station. In this approach, the operator's intervention is crucial in allocating bandwidth and thus the allocation process is not fully automated. Xi et al. [5] developed a bandwidth allocation mechanism based on online measured task dexterity index of dynamic tasks so that operators can control remote manipulators efficiently and smoothly even under poor network quality. However, the executed task is simple and does not require the collaboration of multiple robots to be performed. Thus, the quality of collaboration factor is not considered in the bandwidth allocation. Finally, in [10], a bandwidth management algorithm is introduced and the rate of feedback is regulated based on the amount of activities occurring in the environment. The work shows that during complex tasks, the operator's performance is affected by the rate of feedback of information. It is also confirmed that a higher sampling rate is required to maintain the same level of performance obtained when the environment is less dynamic. Yet, the implemented algorithm does not impose any constraint on the total bandwidth of the system. In addition, the notion of monitoring changes in QoC to allocate bandwidth is not mentioned since the task execution only requires the use of a single robot. To the best of the author's knowledge, there was limited research addressing bandwidth management for the specific application of collaborative robots teleoperation. In 2015, Ricardo and Guilherme designed a Dynamic Bandwidth Management Library to control the frequency of individual sensors present in a robotic environment performing a certain task [16]. This work is seeking a universal Dynamic Bandwidth Management Library designed to be used on a system with a variable number of heterogeneous robots performing any collaborative task that requires communication transactions such as the exchange of sensor data between involved agents.

Accordingly, the main contribution of the work presented in this paper is in accounting for (a) IEs occurring in the robotic swarm's environment and (b) changes in QoC among the swarm of robots in real-time optimized bandwidth management of teleoperated collaborative robots. Consequently, assessing the multi-robot swarm dynamics, the stability of the robotic swarm and the effects of packet loss and transmission delay under the proposed algorithm falls out of the scope of this paper. Factoring the latter into the proposed algorithm is possible but will alter the emphasis from the main contribution of dynamic optimized bandwidth management.

A literature review of the most relevant work in bandwidth management was presented in Section 1. The general problem is formulated in Section 2. The formulation is then implemented on an application in which an operator drives two collaborating robots. Section 3 describes the experimental set-up and the corresponding results. Finally, conclusions are presented in Section 4.

2. Problem formulation

The focus of the work presented is on real-time dynamic managing of the ‘User’ to ‘Robot’ and ‘Robot’ to ‘User’ (U2R/R2U) and ‘Robot’ to ‘Robot’ (R2R) communication channels, where actuation commands, system state and sensory data including video frames are exchanged. A general formulation of the problem that can be applied to swarms with a variable number of collaborating robots is presented first and is then implemented on a formation of two humanoid robots performing a collaborative task. The formulation is concluded by presenting the solution of the optimization problem.

2.1. The general formulation

The goal of the optimized dynamic bandwidth allocation algorithm is to optimize communication at each time event based on information related to the occurrence of IEsin the robot's surroundings and to changes in QoCamong robots. The idea comes from the fact that when sudden changes occur in the robot's environment, the teleoperator needs to be updated more frequently in order to retain the same level of performance. In other words, the rates of information exchanged between operator(s) and robots are updated based on changes in task conditions that may affect the performance severely. Similarly, the increase in R2Rcommunication compensates for any decay in the performance of the collaborative task of the robotic swarm. Hence, if the environment is less dynamic and the robots are collaborating well, the corresponding communication rates will be decreased. On the other hand, if the environment is more dynamic and the robots are collaborating poorly, the corresponding communication rates will be increased.

Let the collaborative task be executed by nrobots and let x_i, where $i\in \{1,2,\dots ,r\}$, be the communication rates to be optimized. Then, we define X as an r-dimensional vector of communication rates such that

$X={\left[\begin{array}{cccc}\hfill x_1\hfill & \hfill x_2\hfill & \hfill \cdots \hfill & \hfill x_r\hfill \end{array}\right]}^T$

Where the elements of X are classified into 3 sets of rates:

• Feedback Rates: R2U communication rates

• Collaboration Rates: R2R communication rates

• Command Rates: U2R communication rates

The rates x_iare subject to practical constraints that bound each of them with a minimum and maximum value x_iminand x_imax, respectively. In addition, the sum of bandwidth consumed by all channels is bounded by the total bandwidth of the system B_max. Hence, for all the rates, we have the following constraints:

$x_{imin}\le x_i\le x_{imax},\text{where}i\in \{1,2,\dots ,r\}$E1

and

$x_{imin}\le x_i\le x_{imax},\text{where}i\in \{1, 2, \dots , r\}$E2

where w_i's are weights associated with each rate corresponding to the rate of information (bps) sent at each time event on each channel. We define the vector $W={\left[\begin{array}{cccc}\hfill w_1\hfill & \hfill w_2\hfill & \hfill \dots \hfill & \hfill w_r\hfill \end{array}\right]}^T$as an r-dimensional vector of weights corresponding to each channel. We also define Pas an m-dimensional observation vector that is composed of two main sets of components such that

$P={\left[\begin{array}{ccccccc}\hfill p_1\hfill & \hfill p_2\hfill & \hfill \dots \hfill & \hfill p_t\hfill & \hfill p_{t+1}\hfill & \hfill \dots \hfill & \hfill p_m\hfill \end{array}\right]}^T=\left[\begin{array}{c}\hfill I{E}_{tx1}\hfill \\ \hfill Qo{C}_{(m-t)x1}\hfill \end{array}\right],\text{where}t < m$

each element in Pis an observation related to interesting events occurring in the robot environment or to the quality of the collaborative task executed by the robotic swarm. The elements of IE,{p₁,p₂,…,p_t}, and of QoC,{p_{t+ 1}, p_{t+ 2}, …, p_m}, are variables that dictate the choice of the communication rates. All the elements of Pare normalized in the interval [0: 1]. If the i^thobservation is reflecting a slightly changing environment or a high collaborative performance, then p_iwould be close to 0. On the contrary, if the i^thobservation reflects a highly changing environment or significant degradation in the collaborative performance, then p_iwould be closer to 1.

Distance to obstacles, speed and displacement of robot are potential examples of IEsthat could be monitored in order to allocate bandwidth. Moreover, any observation that tracks error in a collaborative task could also contribute to the bandwidth management algorithm.

In the formulation presented in this work, a mapping equivalent to the one presented in our previous work [17] is applied, however, with additional constraints that transform the problem from a simple matrix multiplication to a linear optimization problem. The new constraints bound the set of feasible communication rates x_idepending on the choice of the minimum and maximum rate for each channel x_iminand x_imaxand the maximum bandwidth of the system B_max.

In this mathematical formulation, s_iis defined in the interval [0: 1] for all $i\in \{1,2,\mathrm{...},r\}$to be as follows:

$x_i=a_i{s}_i+b_i$E3

where

$a_i=x_{imax}-x_{imin}$E4

and

$b_i=x_{imin}.$E5

Then, an r-dimensional vector Sis defined as follows:

$S_{r*1}={\left[\begin{array}{cccc}\hfill s_1\hfill & \hfill s_2\hfill & \hfill \dots \hfill & \hfill s_r\hfill \end{array}\right]}^T\text{.}$

Hence, at each time instant, the algorithm solves for the s_i's and then uses the mapping in (3) to get the rates x_i's.

$X_{r*1}=A_{r*r}\text{.}{S}_{r*1}+B_{r*1}$E6

Sis related to Pusing the mapping matrix Mas shown in (7):

$S_{r*1}=M_{r*m}\text{.}{P}_{m*1}$E7

where

$A_{r*r}=\left[\begin{array}{ccc}{a}_1& \dots & 0\\ ⋮& \ddots & ⋮\\ 0& \dots & a_r\end{array}\right];Br*1=\left[\begin{array}{c}{b}_1\\ ⋮\\ b_r\end{array}\right];S=\left[\begin{array}{c}{s}_1\\ s_2\\ ⋮\\ s_r\end{array}\right];$

$M=\left[\begin{array}{cccc}{m}_{11}& m_{12}& \cdots & m_{1m}\\ m_{21}& m_{22}& \cdots & m_{2m}\\ ⋮& ⋮& \ddots & ⋮\\ m_{r1}& m_{r2}& \cdots & m_{rm}\end{array}\right];P=\left[\begin{array}{c}{p}_1\\ p_2\\ ⋮\\ p_m\end{array}\right]$

In (7), the elements of the matrix Mare selected based on the relation between the observations and the rates. Each row of the matrix Mcan be interpreted as the weights of the observations in Paffecting the corresponding rate in X. Since s_i's are selected in the range [0: 1], then choosing the sum of the coefficients in each row of Mto be equal to 1 ensures that the result of multiplying any row of Mby the vector Prepresents a weighted average of the observations that results in a value in the [0: 1] range. Thus, for a specific environment, Mcan be initialized once at the beginning of the set of trials. However, in order to improve the performance, Mcould also be updated dynamically based on the quality of the task previously executed. Thus, an improvement in the overall performance is achieved while maintaining an equivalent level of bandwidth consumption.

Since the sum of bandwidth consumed on all channels is bounded by the maximum bandwidth of the system, B_max, the allocation of the rates on different channels will be formulated as a linear optimization problem. Thus, the problem formulation becomes:

Minimize:

${(S-M.P)}_1$E8

subject to

${\sum }_{i=1}^r{w}_i\text{.}{x}_i\le B_{\max }$E9

and

$0\le s_i\le 1\forall i\in \{1,2,\dots ,r\}.$E10

Since x_i= a_is_i+ b_i, then Eq. (9) can be written as follows:

${\sum }_{i=1}^r(w_i\text{.}{a}_i\text{.}{s}_i+w_i\text{.}{b}_i)\le B_{\max }$E11

which is equivalent to

${\sum }_{i=1}^r{w}_i\text{.}{a}_i\text{.}{s}_i\le B_{\max }-{\sum }_{i=1}^r{w}_i\text{.}{b}_i$E12

The constraint in (12) can be expressed in terms of matrix multiplication as follows:

$W^T\text{.}A\text{.}S\le B_{\max }-W^T\text{.}B$E13

Therefore, for any collaborative task, it is sufficient to set the parameters in Eq. (13) in order to define the linear optimization problem. Thus, optimization techniques can be applied to solve the defined problem.

2.2. Mathematical formulation

In order to solve the aforementioned problem, a simple change of variable is first performed. We let:

$Z=S-M\text{.}P$E14

Therefore, the problem formulation becomes minimize:

$Z_1.$E15

subject to

$W^T\text{.}A\text{.}Z\le B_{\max }-W^T\text{.}B-W^T\text{.}A\text{.}M\text{.}P$E16

and

$-M.P(i)\le z_i\le 1-M.P(i)\forall i\in [1:r].$E17

Since the problem is an L1 norm problem, it needs to be slightly modified in order to get rid of the absolute value that complicates the solution of the problem. Thus, the problem can be translated to minimize:

${\sum }_{i=1}^r{t}_i$E18

subject to

$W^T\text{.}A\text{.}Z\le B_{\max }-W^T\text{.}B-W^T\text{.}A\text{.}M\text{.}P$E19

and

$-M.P(i)\le z_i\le 1-M.P(i)\forall i\in [1:r]$E20

and

$-T\le Z\le T$E21

where T is an r-dimensional vector containing all the t_i's, which are dummy variables that are introduced to avoid the use of absolute value in the formulation and replace it by a simple minimization of a summation. Hence, at each time instant, the constraint matrix is formed, and then the optimization problem is solved. Since the problem is linear, it is solved efficiently using interior point method. In the experiments performed in this work, the optimization problem was solved in 10–15 iterations for an average time period nearly equal to 100 ms. Thus, by scheduling the rates' update at every 5 s, each communication rate is computed by averaging the values calculated in the last 50 iterations. On the other hand, it is worth noting here that the internal model matrix Mgrows quickly with the increase in the number of robots used, which would affect the complexity of the problem and the speed of convergence. Therefore, in the following section, the developed optimization technique was tested on a system of two collaborating humanoid robots. The proposed method could be considered a basis for a framework for developing highly complex algorithms for systems involving real-time bandwidth optimization, where multiple users control multiple collaborating robots in various scenarios.

2.3. Dynamic optimized bandwidth algorithm experimental verification

In order to illustrate the use of the formulation, we apply it for the case, where an operator drives two collaborating robots. The mission consists of navigating a delimited path to reach a predetermined destination while avoiding obstacles and preserving a formation. The formation is characterized by a distance of 60 cm that separates the two robots, while keeping alignment nearly zero as in Figure 1.

Figure 1.

Two robots in a formation requiring a fixed distance D = 60 cm. (a) Without error (b) with vertical and horizontal error.

The robots' feedback includes visual and haptic data reflecting the environmental conditions. Ultrasonic sensors mounted on each robot allow the detection of obstacles in the navigation path. Each returns an integer value, indicating the distance to the nearest detected obstacle, which is fed back to the operator in the form of haptic feedback. In addition, the camera mounted on top of each robot provides visual feedback of the area in front of the formation. In order to allocate bandwidth based on changes in task conditions, IEs such as the distance to obstacles with respect to each robot and the speed of the swarm are monitored. Also, to maintain a high performance at high speeds, the speed of the swarm is one of the dynamic events that are monitored. Moreover, QoCfactors are measures of how well the robots are collaborating together to efficiently accomplish the predetermined task. Specifically, the errors in position (Δxand Δy) between the robots indicating the deviation of the robots from the required formation are the quantities reflecting the change in the quality of the collaborative task, which need to be monitored. During the task execution, the visual and haptic feedback rates, the rates of commands and the rates of R2Rcommunication are allocated based on real-time observations related to occurrence of IEsin the environment and to changes in QoCbetween robots. Hence, the elements of Xare defined as follows:

• x₁: Rate of visual feedback from R1

• x₂: Rate of haptic feedback from R1

• x₃: Rate of visual feedback from R2

• x₄: Rate of haptic feedback from R2

• x₅: Rate of R2R collaboration information exchange

• x₆: Rate of commands generation

The observations in p_i's are also defined below:

• p₁: Distance to obstacle from R1

• p₂: Distance to obstacle from R2

• p₃: Speed of formation (Combined from R1 and R2)

• p₄: Positioning error in the horizontal direction Δx

• p₅: Positioning error in the vertical direction Δy.

Each element of Pis normalized with respect to the maximum value that the sensors could measure (in case of p₁ and p₂), to the maximum value that the system can reach (in case of p₃) or to the maximum allowed error beyond which the task built upon the formation would start being affected (in case of p₄ and p₅). It is worth mentioning that p₃ is introduced in order to detect any sudden changes in the system dynamics that could be captured by monitoring the speed of the formation, which would also require significant changes in video feedback.

Knowing that the maximum distance detected by the robots' sensors is 2.5 m, p₁ and p₂ are defined as follows:

$p_1=\frac{2.5-DistancefromR1}{2.5}{p}_2=\frac{2.5-DistancefromR2}{2.5}$

Given that v₁ and v₂ are the average forward/backward and sideway speed of the formation, respectively, and knowing the maximum forward and sideway speed that the robots could reach is 6 and 4 cm/s, respectively, p₃ is defined as follows:

$p_3=\frac{\sqrt{v_1{2}^{}+v_2{2}^{}}}{\sqrt{v_{1\max }{2}^{}+v_{2\max }{2}^{}}}$

where the exact values used in this case are $v_{1\max }=6\text{cm}/\text{s}$and $v_{2\max }=4\text{cm/s}$.

Finally, p₄ and p₅ are defined with respect to positioning error in the horizontal (Δx) and vertical (Δy) directions, respectively, as follows:

$p_4=minn\left(\frac{\left|e_x\right|}{e_{x_{\max }}},1\right);p_5=\min \left(\frac{\left|e_y\right|}{e_{y_{\max }}},1\right).$

Additionally, the maximum allowed positioning error in the horizontal and vertical directions $e_{x_{\max }}$and $e_{y_{\max }}$for the application are estimated to be 2.5 and 5 cm, respectively.

Furthermore, the experiment is performed for a maximum bandwidth B_maxequal to 1.4 Mbps. This value is deemed to be convenient for such small swarm with very few sensory data and video frame to exchange. Moreover, the imaging resolution of the robot's cameras is 160 × 120 × 3, which implies that each frame is formed of 3 matrices and thus, w₁ = w₃ = 160 × 120 × 3 × 8 bits= 460,800 bits. As for the four remaining rates, the size of the data packets sent to/by each agent is considered to be 1500 Bytes. Hence, w₂ = w₄ = 1500 × 8 = 12000 bits. But since the collaboration information exchanged requires both robots to send one packet, and since the operator generates a packet for each robot as command, then w₅ = w₆ = 2 × 12000 bits= 24000 bits. Consequently, Wis represented as follows:

$W=\left[\begin{array}{llllll}460800\hfill & 12000\hfill & 460800\hfill & 12000\hfill & 24000\hfill & 24000\hfill \end{array}\right]$

In addition, the matrices Aand Bare set based on the allowed minimum and maximum of these rates. During experimentation, a frequency of 1 Hz is allocated as minimum for all rates. As for the maximum rates of cameras, it is equal to the total bandwidth from which the minimum consumption of all other sensors is removed. For this application, $x_{1\max }$and $x_{3\max }$are chosen to be equal to 2 Hz. The force feedback ($x_{2\max }$and $x_{4\max }$), collaboration information exchange ($x_{5\max }$), and commands rates ($x_{6\max }$) are accorded a maximum rate of 5 Hz. Thus, using Eqs. (4) and (5), matrices Aand Bare calculated to be as follows:

$A=\text{diag}\left(\left[\begin{array}{llllll}1\hfill & 4\hfill & 1\hfill & 4\hfill & 4\hfill & 4\hfill \end{array}\right]\right);B=\left[\begin{array}{llllll}1\hfill & 1\hfill & 1\hfill & 1\hfill & 1\hfill & 1\hfill \end{array}\right]$

Finally, a fixed matrix Mis adopted in the implementation as described earlier. Each row of Mcontains the weights of the observations that affect the corresponding rate. The weights for all set of observations corresponding to each rate are allocated in a way to have the sum of each row remain equal to 1. It is worth mentioning that for the speed of the formation, weights of 25% are allocated in the rows of Mcorresponding to all the R2Uand U2Rrates (x₁ to x₄), since it was found to be the least dynamic observation. However, the rate of commands was deemed to be mostly dependent on the speed of the formation p₃. Thus, Mwould be as follows:

$M=\left[\begin{array}{ccccc}\hfill .75\hfill & \hfill 0\hfill & \hfill .25\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill .75\hfill & \hfill 0\hfill & \hfill .25\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill .75\hfill & \hfill .25\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill .75\hfill & \hfill .25\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill .5\hfill & \hfill .5\hfill \\ \hfill .25\hfill & \hfill .25\hfill & \hfill .5\hfill & \hfill 0\hfill & \hfill 0\hfill \end{array}\right]$

In the experiments, the aforementioned values of A, B, M, Wand B_maxare adopted while solving the optimization problem and allocating rates.

3. Experimental set-up and results

In this section, the suggested algorithm is evaluated against two static allocation algorithms: ‘Equal Bandwidth’ and ‘Equal Rates’. The equal bandwidth method allocates bandwidth equally among the different streams, while the equal rates method divides the available bandwidth such that the different streams would have equal rates of transmission.

The experimental set-up is composed of two NAO humanoid robots driven in a certain formation. An operator drives the two robots using a force feedback joystick (Microsoft SideWinder Force Feedback 2) and communicates with the robot through a router connected to a PC as shown in Figure 2. The operator sends real-time commands to the agents that return back haptic feedback as proximity measures and visual feedback from cameras mounted on each robot. The signal flows within the different components of the system as illustrated in Figure 3. The collaborative task that is to be accomplished by the robots is to maintain a certain formation while traversing an environment and avoiding collision with potential obstacles. The formation is characterized by a fixed distance (D = 60 cm) separating the two robots, while maintaining the error in the vertical direction nearly zero. Accurate position of the humanoids is calculated using the aid of the Inertial Unit built in the robots, which is made of 2-axis gyrometers with 5% precision (angular speed∼500°/s) and a 3-axis accelerometer with 1% precision (acceleration∼2G).

Figure 2.

Experimental set-up.

Figure 3.

Teleoperation experimental set-up of humanoid robots.

3.1. Testing scenarios

3.1.1. Scenario 1: path with no obstacles

The first scenario consists of driving the swarm in the space delimited by dashed lines, shown in Figure 4, and reaching the final destination (dashed rectangular region on the right side) with no obstacles in the path. Obviously, with the absence of obstacles, the shortest path in this case is moving from ‘Start’ to ‘End’ in a straight line. The operators tend to adopt the shortest path, the straight line in this case, to reach the final destination.

Figure 4.

Scenario 1—path with no obstacles.

3.1.2. Scenario 2: obstacle in front of robot 1

In the second scenario, an obstacle is detected in front of Robot 1 (dashed square) when driving the formation in the delimited area as shown in Figure 5. The operator should steer to the left in order to avoid the collision with the obstacle. The swarm is forced to steer toward the left since by steering in the opposite direction Robot 2 would then exit the delimited path.

Figure 5.

Scenario 2—obstacle in front of R1.

3.1.3. Scenario 3: obstacle in front of robot 2

The third scenario features an obstacle in front of Robot 2 (dashed square) when driving the formation in the delimited area. In this case, the operator steers to the left in order to avoid the collision of Robot 2 with the obstacle as shown in Figure 6.

Figure 6.

Scenario 3—obstacle in front of R2.

3.2. Testing and results

In order to evaluate the suggested algorithms, for each scenario, teleoperators drove the swarm under the Equal Bandwidth, Equal Rates and Optimized Bandwidth method. In each trial, the following performance parameters are collected: the completion time in seconds, the average speed of each robot in cm/s, the average deviation of each robot from the shortest path (Esp1 and Esp2) in , the average error in the formation in the horizontal and vertical directions (Δxand Δy) in cm, the maximum errors in both directions as well as the average bandwidth in Mbps consumed. The first static algorithm divides the available bandwidth equally among the 6 communication channels, whereas the second applied algorithm allocates equal rates to all communication channels. The computed rates for each channel for the static algorithms are reported in Table 1. It is worth noting that since the image frame size is much greater than the other data exchanged, allocating that equal bandwidth to all communication channels reduces the cameras' frame rate to around 1 Hz, while allocating equal rates to all communication channels increases frame rates to 1.5 fps; however, it drops the rates of all other data exchanged by a factor of five.

	x1	x2	x3	x4	x5	x6	Total bandwidth (Mbps)
Equal bandwidth	1	8	1	8	8	8	1.43
Equal rates	1.5	1.5	1.5	1.5	1.5	1.5	1.42

Table 1.

Computed rates (in Hz) of both static algorithms.

The path in front of the formation can be visualized by the cameras located on the forehead of each robot. Robots R₁ and R₂ navigate inside a delimited path while avoiding obstacles to reach the final destination. Moreover, a force feedback that corresponds to the distance to obstacles in front of the formation is calculated based on values measured by the ultrasonic sensors mounted on each robot. In order to evaluate the suggested algorithm, four teleoperators drove the formation under the three allocation methods in the three defined scenarios for a total number of runs equal to 36. Under each scenario, the bandwidth methods were randomlyselected for each driver. Additionally, two training runs were performed by each user in order to get familiar with the task performed and experiment the haptic and visual feedback before executing the official runs. Results for the three mentioned scenarios are recorded in Tables 2–4. Runs, which included visible slippage by the robots, were repeated in order not to bias the results.

Referring to Table 2, at an average bandwidth consumption less than that of both static algorithms, dynamic optimized bandwidth allocation method results in a better performance in the first scenario. With a reduction in bandwidth consumption of around 70 Kbps, the operator performs better when using the proposed dynamic algorithm than when applying the static ones. With the dynamic bandwidth algorithm, the average trial duration is 1.9 s less than the best static allocation method. Moreover, the driving performance improved significantly, since the parameters measuring the average error with respect to the shortest path improved in addition to the average speed of both robots. The instantaneous error to the shortest path has decreased by around 0.15 cm for both robots, while the average speed of both robots is around 0.35 cm/s higher. As for the parameters reflecting the quality of the executed collaborative task, we remark that the dynamic algorithm performs better than both static methods. The average errors in the horizontal and vertical directions are smaller as well as the maximum error is in both directions. For instance, the average horizontal error decreased by 10% for around 0.05 cm, while the average vertical error decreased by 40% (0.19 cm). It is worth noting that for most parameters the proposed method has a lower standard deviation indicating a more consistent performance.

		Duration(s)	SpeedR1(cm/s)	SpeedR2(cm/s)	Esp1(cm)	Esp2(cm)	Avghorizerror(cm)	Avgverterror(cm)	Maxhorizerror(cm)	Maxverterror(cm)	Averagebandwidth(Mbps)
Equal bandwidth	Average	35.2	4.53	4.73	0.52	0.53	0.75	0.48	2.63	3.95	1.40
	Std Dev	1.13	0.17	0.23	0.15	0.19	0.56	0.23	1.28	1.30	0.00
Equal rates	Average	34.2	4.71	4.91	0.43	0.47	0.54	0.59	1.99	2.92	1.40
	Std Dev	5.26	0.77	0.77	0.09	0.10	0.37	0.18	1.25	1.21	0.00
Optimized rates	Average	32.3	5.04	5.27	0.37	0.41	0.49	0.29	1.32	1.33	1.33
	Std Dev	4.39	0.45	0.45	0.05	0.05	0.34	0.18	0.79	0.69	0.0015

Table 2.

Results of Scenario 1 at 1.4 Mbps.

In Scenario 2, the advantage of dynamic bandwidth allocation is also demonstrated by the collected results in Table 3. With a reduction in bandwidth consumption of around 70 Kbps, the operators perform better when using the proposed dynamic algorithm than when applying the static ones. With the dynamic bandwidth algorithm, the average trial duration is 3.7 s less than the best static allocation. Moreover, the driving performance improves significantly, since the parameters measuring the average error with respect to the shortest path improve in addition to the average speed of both robots. The instantaneous error to the shortest path decreased by around 0.4 cm, while the average speed of both robots is around 0.35 cm/s higher. Additionally, the maximum errors in the horizontal and vertical directions decreased by around 0.35 cm. It is worth noting here that the runs performed with ‘Equal Rates’ method lead to a better average horizontal and vertical error; however, with this method, high peaks of errors are reached in both directions that almost reach the tolerated bounds of 2.5 and 5 cm.

The rates of visual feedback of robots R₁ and R₂ during Scenario 3 for the different adopted bandwidth algorithms are presented in Figure 7. Additionally, the rates of haptic feedback of robots R₁ and R₂ and the collaboration and commands rate during Scenario 3 for the different bandwidth algorithms are presented in Figures 8 and 9, respectively.

Furthermore, we examine the percentage of runs in which the suggested algorithm outperforms the two static algorithms for each performance parameter in all scenarios. In other words, we count the number of times a user performed better according to a parameter when adopting the dynamic algorithm versus when driving with each of the static algorithms. Percentage of best performance for task duration, average speed of each robot (Speed R1, Speed R2), error of each robot with respect to the shortest path (Esp1 and Esp2) and maximum alignment and separation errors in the formation are recorded in Table 5. From the collected results, it can be seen that operators perform better with the dynamic allocation algorithm than the static algorithms at a minimum of 67% of the runs (Esp1 and Esp2 with Equal Bandwidth and Max horizontal/vertical error with Equal Rates). The suggested algorithm reaches a success rate of 92% for the speed R1 with respect to ‘Equal Rates’ static allocation.

		Duration(s)	SpeedR1(cm/s)	SpeedR2(cm/s)	Esp1(cm)	Esp2(cm)	Avghorizerror(cm)	Avgverterror(cm)	Maxhorizerror(cm)	Maxverterror(cm)	Averagebandwidth(Mbps)
Equal bandwidth	Average	45.3	4.01	4.17	7.49	7.51	0.54	0.51	2.60	3.39	1.40
	Std Dev	1.49	0.34	0.32	2.10	2.02	0.24	0.14	0.72	0.61	0.00
Equal rates	Average	46.1	4.15	4.34	5.92	5.98	0.37	0.42	2.25	4.66	1.40
	Std Dev	3.83	0.03	0.07	0.31	0.33	0.20	0.37	1.64	4.88	0.00
Optimized rates	Average	41.6	4.49	4.66	5.62	5.46	0.65	0.49	2.23	2.61	1.33
	Std Dev	2.21	0.21	0.24	0.82	0.89	0.15	0.23	0.53	1.96	0.0002

Table 3.

Once more, the results collected during the experiments performed in Scenario 3 have shown the advantages of the suggested dynamic allocation method as depicted in Table 4. The parameters measuring the quality of the collaborative task and the driving performance show real improvements. It is worth noting that in Scenario 3, the average task duration with the dynamic bandwidth method is equal to the average duration with equal bandwidth method. However, three of the four drivers have performed better with the suggested algorithm than with static bandwidth allocation. Only one user performed the trial with a total duration of 53 s. This trial biased the calculated average, which was reflected by the standard deviation value.

Results of Scenario 2 at 1.4 Mbps.

		Duration(s)	SpeedR1(cm/s)	SpeedR2(cm/s)	Esp1(cm)	Esp2(cm)	Avghorizerror(cm)	Avgverterror(cm)	Maxhorizerror(cm)	Maxverterror(cm)	Averagebandwidth(Mbps)
Equal bandwidth	Average	47.5	4.09	4.23	7.55	7.52	0.66	0.63	7.66	4.93	1.4
	Std Dev	2.31	0.38	0.45	2.31	2.39	0.64	0.27	7.62	1.95	0.00
Equal rates	Average	49.3	3.99	4.00	9.03	9.09	0.66	0.88	2.35	6.32	1.4
	Std Dev	2.47	0.16	0.21	1.27	1.16	0.25	0.54	0.98	6.04	0.00
Optimized rates	Average	47.5	4.15	4.27	6.78	6.81	0.36	0.23	1.43	2.90	1.33
	Std Dev	4.40	0.23	0.28	1.16	1.15	0.24	0.15	0.76	2.52	0.0010

Table 4.

Results of Scenario 3 at 1.4 Mbps.

Figure 7.

Visual rates of R1 & R2 in Scenario 3.

Figure 8.

Haptic rates of R1 & R2 in Scenario 3.

Finally, the advantages of the proposed dynamic bandwidth optimization and management scheme over legacy bandwidth management schemes are clearly expressed in the results in terms of performance improvement and conserving network resources. Since the proposed algorithm is scalable and not limited to a single task, the improvement in performance is greatly realized in critical situations, where the collaborative task requires high levels of accuracy especially in cases involving human safety.

Figure 9.

Collaboration & commands rates in Scenario 3.

	Duration; (%)	Speed; R1 (%)	Speed; R2 (%)	Esp1 (%)	Esp2 (%)	Max; horizontal; error (%)	Max; vertical; error (%)
Dynamic vs. equal bandwidth	83	75	83	67	67	83	83
Dynamic vs. equal rates	75	92	83	83	83	67	67

Table 5.

Percentage of best performance.

4. Conclusions

In this work, dynamic optimized bandwidth management in teleoperated collaborative robotics is tackled. The focus was on managing all communication channels, where actuation commands, system state and sensory data are exchanged. This was achieved by monitoring the interesting events occurring in the robots' environment and the changes in quality of collaboration among them. Effective completion of the collaborative task with lower bandwidth consumption and better performance was accomplished proving that the proposed method could be the basis of a framework for developing more complex algorithms applied to highly complex systems.

Acknowledgments

This research was funded and supported by AUB University Research Board and the Lebanese National Council for Scientific Research.