A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions

Abstract

Research in the area of cooperative multi-agent robot systems has received wide attention among researchers in recent years. The main concern is to find the effective coordination among autonomous agents to perform the task in order to achieve a high quality of overall performance. Therefore, this paper reviewed various selected literatures primarily from recent conference proceedings and journals related to cooperation and coordination of multi-agent robot systems (MARS). The problems, issues, and directions of MARS research have been investigated in the literature reviews. Three main elements of MARS which are the type of agents, control architectures, and communications were discussed thoroughly in the beginning of this paper. A series of problems together with the issues were analyzed and reviewed, which included centralized and decentralized control, consensus, containment, formation, task allocation, intelligences, optimization and communications of multi-agent robots. Since the research in the field of multi-agent robot research is expanding, some issues and future challenges in MARS are recalled, discussed and clarified with future directions. Finally, the paper is concluded with some recommendations with respect to multi-agent systems.

1. Introduction

Research on the multi-agent robot systems has been conducted since late 80s as it provides a more efficient and robust system compared to a single robot. ALLIANCE [1] and ACTRESS [2] robot are among of the earliest heterogeneous multi-agent robots developed by previous researchers. The benefits received from information sharing among agents, data fusion, distribution of task, time and energy consumption have made the multi-agents research still relevant until present.

There were many researchers who focused on cooperative multi-agent research. The most challenging part was to provide a robust and intelligent control system so that the agents can communicate and coordinate among them to complete the task. Hence, it has been found that designing the control architecture, communication, and planning system were the major issues discussed and solved among researchers. Other than that, improvement to the existing coordination techniques, optimal control architectures, and communication were also the main highlights in the previous research. A few examples of cooperative multi-agent robots applications are soccer robot [3], unmanned guided vehicles (UGV’s) and unmanned aerial vehicles (UAV’s) [4], micro chain [5], and paralyzed robot [6].

There were two main reviewed papers proposed by Cao and Zhi Yan which were related to cooperative multi-agent research. Cao et al. [7] proposed a paper that represents the antecedents and direction of the cooperative mobile robot in the mid-1990s (most of the reviewed papers were published from 1990 to 1995). There were several issues discussed such as group architecture, resource conflict, the origin of cooperation, learning, and geometric problem. The applications and critical survey of the issues and direction of cooperative robots based on existing motivation have been indicated. Besides that, there were also a survey and an analysis of multi-robot coordination proposed by Yan et al. [8] in 2013 (most of the reviewed papers were published from 2000 to 2013). They presented a systematic survey and analysis of multiple mobile robot systems coordination. Related problems such as communication mechanism, a planning strategy, and a decision-making structure have been reviewed. In addition, various additional issues of cooperative MARS have been highlighted in these reviewed papers. Most of the papers were published from 2010 to 2015 which the recent research papers on cooperative multi-agent systems have been reviewed.

The main contributions of this paper are (i) the most reflected and affected key elements and current issues in cooperative mobile robots and (ii) directions and future challenges for the multi-agents robot, with recommendations and related suggestions. The remain sections of the paper are structured as follows: the first section discusses three main categories of multi-agent robot systems, the second section focuses on discussion of problems and some current issues of multi-agent systems and the final section is the conclusions with some challenges and recommendations for future research direction in the field of cooperative multi-agent systems.

2. Key elements of cooperative multi-agent robot systems

A wide means of research in cooperative multi-agent robots systems have focused on the three main elements which are (1) types of agents; homogeneous and heterogeneous, (2) control architectures; reactive, deliberative and hybrid, and (3) communication; implicit and explicit. In order to provide efficient coordination among multi-agent robots, the selections and designs of the control architecture and communication must possess a coherent behavior with the agents. Therefore, this paper thoroughly explains each of the key elements with related examples from previous research and followed by the issues and directions of the multi-agent robot systems.

2.1. Types of agents: homogeneous and heterogeneous

Multi-agent robots can be divided into two categories which are homogeneous and heterogeneous. The agents become homogeneous when the physical structures or capabilities of the agents/individuals are identical (Figure 1). The capabilities for heterogeneous agents are not identical and they are different among robots, where each robot has its own specialization or specific task to complete [8]. Besides that, the physical structures of heterogeneous agents are also not identical among them (Figures 2 and 3).

Figure 1.

Multi-agent (homogeneous agents) control with broadcast [33].

Figure 2.

Examples of Heterogeneous agent (chained micro robots) [5].

Figure 3.

The heterogeneous team includes a single UAV (Pelican Quadrotor) controlling many UGV’s (Khepera II robots) [4].

Research carried out by Sugawara and Sano [9] and Hackwood and Beni [10] have proven that their homogeneous agents that have identical structures and identical capabilities can perform the task efficiently. However, for Li and Li [11], the heterogeneous agents are more applicable than homogeneous agents in the real world. Therefore, instead of focusing on homogeneous agents, current researchers are also concerned about heterogeneous agent’s issues [1, 2, 3, 4, 5, 6, 11, 12, 13, 14, 15]. The agent’s physical structures and capabilities which are not identical have made the agents fall into these heterogeneous agents categories [16, 17].

There are two researchers known as Parker [18] and Goldberg [19] who compared the task coverage and interference between homogeneous and heterogeneous agents. Parker discovered that the task coverage for homogeneous agents is maximum compared to heterogeneous. This is because the homogeneous agents execute the same task at one time, while the heterogeneous agents need to distribute their task to another agent during the execution. Due to the task distributions among heterogeneous agents, the interference becomes higher compared to homogeneous agents, as proven in Goldberg’s research [19]. As a result, we can summarize that the selection of a homogeneous or heterogeneous agent depends on the research application. Since the capability of heterogeneous agents is not identical, it becomes a challenging issue especially in finding consensus among agent during execution of the task. Table 1 shows the research conducted by previous researchers using their heterogeneous agents.

Robot task	Type of robots	Reason of heterogeneous
The pusher robots work among them and push the paralyzed robot to a certain point. The paralyzed robot is driven by the global system [6]	One paralyzed robot with multiple numbers of pusher robots	Different robot, different task
The ROBOCUP robot team plays a soccer ball [3]	Group of agents (robots) acts as a goal keeper, middle field player, striker, and defender	Same robot, different task for each group of agents since each agent has different capabilities and characteristics
Unmanned aerial vehicles (UAV) acts as a supervisor to control unmanned ground vehicles (UGV) robots from any danger and collide with obstacles [4]	Single UAV flies to control and allocate several UGV’S	Different robot, different task
Coordination of heterogeneous multi-agents systems. Second order dynamics is the state of the leader while first order dynamics is the state of the followers [11]	Consists of leader and few followers	Same agent, different state dimension among the leader and follower (not identical)
Coordination of the micro robots chain [5]	Consists of different modules (active and passive) such as rotation, support, extension and helicoidally modules	Different modules, different task/function
Multi-agent robots construct four different blocks [12]	Multiple agents	Same agent, different task
ACTRESS robot pushes the objects [2]	3 different robotors act as interface human operator, image processor, and global environment manager	Different agent, different task
ALLIANCE robot executes few tasks. The tasks are box pushing, puck gathering, marching, information, marching, hazardous and waste cleanup [1]	Small to a medium size of heterogeneous teams	Different agent, different task

Table 1.

Examples of heterogeneous agent’s research.

2.2. Control architectures: reactive, deliberative, and hybrid

The selection of control architectures for multi-agent robots is based on the capabilities of each agent to work in the groups and it also depends on how the overall systems work. The control architectures can be classified into three categories which are (i) reactive, (ii) deliberative and (iii) hybrid (reactive and deliberative).

Reactive control is also known as decentralized control. Reactive control relies on the concept of perception-reaction where the agents will cooperate between agents based on direct perception, signal broadcast or indirect communication via environmental changes. It does not require a high-level of communication to interact with agents. There are a few approaches which are related to reactive control for multi-agent robots. Glorennec [20] coordinated multi-agent robots by using fuzzy logic techniques to avoid obstacles and robots in the environment, whereas, research done by Lope et al. [12] coordinated their multi-agent robots by using the reinforcement learning algorithm based on the learning automata and ant colony optimization theory. Their multi-agent robots can organize the task by themselves to choose any task to be executed. It was proven that without interference from the central controller, the robots are capable of selecting their own task independently.

Despite these approaches, Chen and Sun [21] proposed a new optimal control law as distributed control for multi-agent robots in finding consensus to avoid obstacles in the environment. Local information from neighbors is required in this research. It is proven that this approach is capable of solving consensus problem under obstacle avoidance scenarios. In terms of local information context, Vatankhah et al. [22] developed a unique adaptive controller to move the leader and follower to a specific path. Finally, the decentralized control for stabilizing nonlinear multi-agent systems by using neural inverse optimal paper is carried out by Franco et al. [23].

Deliberative approach relied on the high-level communication, rich sensor and complete representation of the environment which allow the planning action. This approach is also known as centralized approach. The input data (usually from the static environment) that represents the global map can be planned to drive the agents efficiently to the target point [6, 24, 25]. The hybrid approach represents the integration control between reactive and deliberative control. Both controls complement each other to find the robust control system in controlling multi-agents robot. In deliberative control, all of the planning processes are involved with the calculation of a global target. As for reactive control, it is more towards a local plan for the robot to avoid the obstacles. There are examples of hybrid approaches related to multi-agents research studies as shown in Table 2 [5, 6, 24, 25].

Task	Deliberative (D)	Reactive (R)	Communication of D and R
Pusher robots push the paralyzed robot to a specified target point [6]	Emit an attractive signal to move paralyzed robot to a specific target and to recruit another pusher robot to push the paralyzed robots (broadcast simple signal). It has a vision of the environment to determine the path	A force field approach used to define the pushers robots motion	D broadcast emitted signal to R controller
Solving dynamic problem for multi-agents by proposing a novel control scheme [25]	Introduce supervisor that assists a group of agents with centralized coverage control law and global trajectory tracking control law	Introduce control laws for coverage agents to avoid a collision and maintain proximity to a supervisor	Using a control law. Each law is active at a given time
Movement of chain micro-robots [5]	High layer for central control	Low-level embedded layer based on behavior function	D and R communicate using command exchange protocol

Table 2.

Hybrid control architectures of multi-agent robots.

Every researcher has used different types of control architecture that are suitable for their system. They have come out with their own idea about the control architectures. Based on [26], hybrid architectures offer the most widespread solution in controlling intelligent mobile robots. Besides that, in a real world, agents also require acting in a dynamic and uncertain environment [6]. Subsequently, the hybrid approach allows the robot to navigate the target as well as avoiding the obstacles successfully within that environment [24].

The researchers who have focused on reactive architectures or known as decentralized approach [27] have claimed that decentralization will provide flexibility and robustness. However, Franco et al. [23] have different views where they agreed with the deliberative approach (centralized) is obviously good for their system although it is hard to control in a complex and large system due to technical and economic reasons. Sometimes, centralized control design totally depends on the system structure and it cannot handle structural changes. Once removed, it needs to be designed all over again. It is also costly and complex in terms of online computation and its control design.

2.3. Communications: implicit and explicit

Cooperation is usually based on some forms of communication. Communication is a mode of interactions between multi-agent robots. With an efficient communication system, the robot is capable of interacting, sharing and exchanging information. Communication also determines the success in mobile robots cooperation [28, 29]. Based on research by Cao et al. [7], there are three types of communication structures which are (i) interaction via the environment, (ii) interaction via sensing, and (iii) interaction via communications. However, this section will only focus on the two main types of interaction (ii) and (iii) which are important in the communication of mobile robots.

Implicit communication or also known as interaction via sensing refers to the local interactions between agents (agent to agent) as shown in Table 3. The agents will sense other agents by embedding a different kind of sensors among them. They will react to avoid obstacles among themselves if they sense signals from other agents [4, 10, 30, 31, 32]. However, due to limitation of hardware parts, the interaction via sensing has been replaced by using a radio or infrared communication.

Task	Sensors
Multi-robots work together to avoid other robots, remove obstacles and pass objects [30]	Real mobile robots equipped with CCD cameras
Follower robots follow the leader while avoiding the obstacles [31]	4 robots equipped with ultrasound sensors
Robot teams will track the target and push the box cooperatively [32]	4 robots are equipped with side sensors. Different signals are emitted to differentiate between the robots (3 robots) and target robot (1 robot)
UAV allocate UGV’s [4]	UAV equipped with a video camera with onboard Inertial Measurement Unit (IMU). UGV equipped with onboard laser range finder sensor
Multi-robot cooperatively collects the pucks in the field [9]	Robots are equipped with a pair of photo sensors and a pair of IR sensors

Table 3.

Implicit communication researches.

Explicit communication refers to the direct exchange of information between agents or via broadcast messages. This often requires onboard communication modules. Issues on designing the network topologies and communication protocol arise because these types of communication are similar to the communication network [3, 5, 6, 33, 34, 35, 36, 37, 38, 39]. Table 4 shows an example of explicit communications being used in the robot systems.

Task	Communication devices/network	Interaction
Coordination of the ROBOCUP teams (middle size league) [3]	Robots equipped with communicating devices (off-the-shelf) radio modems and wireless Ethernet cards. Communication is based on underlying IP protocol either TCP-IP or UDP-IP	Agent to agent
Developing the control architectures for chain micro robots [5]	Command exchange protocol is used for communication between modules and PC by sending a message. The name of the protocol is $I^{2}C$ protocol	One to many agents (modules) for global. Agent to agents for local
The pusher robots work cooperatively to push the paralyzed robot to a specific point [6]	PC will send messages to the Mindstorm robot (paralyzed) to control Mirosot robots (pusher) by using infrared serial communications interface/transceiver	One way communication. One to all agents (broadcast) for global
Broadcast control framework for multi-agent coordination [33]	The broadcast signal sent from computer to all agents via Bluetooth	One way communication. One to all agents (broadcast)
Sign board based inter-robot communication in distributes robotic system [34]	Communication-based on conceptual mechanism of “sign-board” being used in inter-robot system	Agent to agent
Cooperative multi-robot system using Hello-Call Communication [35]	Each agent communicates together (chains) using “hello-call” protocol to extend their effective communication ranges	Agent to agent
Swarm robots control mobile robot using wireless sensor networks [36]	Using Wifi and three communication channels to interact between swarms for cooperation	One to many agents (broadcast)
Effect of grouping in local communication system of multiple mobile robots [37]	Information spread by the effect of random walk and local communication known as information diffusion (equation of acquisition probability)	Agent to groups of agents
A design method of local communication area in multiple mobile robots systems [38, 39]	Communication by information probability by infinite series	Agent to agent

Table 4.

Explicit communication researches.

3. Problems and issues of cooperative multi-agent robot systems

Although researchers in recent years have addressed the issues of multi-agent robot systems (MARS), the current robot technology is still far from achieving many real world applications. Some real world MARS applications can be found in unmanned aerial vehicles (UAV’s), unmanned ground vehicles (UGV’s), unmanned underwater vehicles (UUV’s), multi-robot surveillance, planetary exploration, search and rescue missions, service robots in smart homes and offices, warehouse management, as well as transportation. Therefore, in this paper, problems and issues related to cooperative multi-agent systems are discussed to improve the current approaches and to further expand the applications of MARS.

3.1. Centralized and distributed control

Based on Section 2.2, the differences between two types of control approaches have been highlighted. However, some problems and issues of both control system in coordinating multi agents will be discussed. By having centralized control, the global information of the environment has been used to calculate the path, trajectory or position of the agents before all [5, 6, 24, 25, 26, 40]. The information then can be sent directly to the agents by using a suitable communication medium. This is one advantage of this control where the agents can obtained the information directly from its central. Research by Azuma [33] shows that the central will sent the updated location directly to the agents by using a WIFI continuously until the agents reach the target point. The quadratic equation is used to calculate agent performances while Simultaneous Perturbation Stochastic Approximation is the algorithm used for the control design [41]. Besides that, A* algorithm, Dijkstra, Genetic Algorithm [42, 43, 44] and Ant Colony Optimization algorithm [45, 46, 47], are example of another algorithms have been used in multi agent centralized control. Oleiwi et al. [24] used a modified GA with A* algorithm for its global motion controller while Atinc et al. [25] proposed a novel control scheme that has a centralized coverage control law.

The main issue in centralized control exists when the number of agents is expanding. The computation will become high since there is only one centralized processor that control over all of the system. Effect of this high computation, the time as well as the energy consumption will be effected at some point. Therefore, to solve this problem, hybrid control approach [5, 6, 24, 25] has been proposed with objective to balance between centralized control and distributed control [23, 26, 48, 49, 50, 51, 52]. Besides that, alternative towards optimizing or minimizing the trajectory length, time and energy consumption [24] as well as adding the intelligences [11, 20, 22, 27, 31, 32, 53] has taken into consideration to reduce the computation time. In terms of scalability, adaptability and flexibility of the controller can be claimed lesser as compared to distributed control. Any changes especially dealing with dynamics will cause the repetition in the computing and sometimes will effect overall of the system with only a limited number of controllers. Thus, centralized control sometimes does not fit with the dynamic environment.

Distributed control had proven scalable, adaptive, flexible and robust for multi agents system not only in static but also in a dynamic environment [54]. Many researchers had proven that their distributed controller can work efficiently for their multi agent robot systems [12, 26, 31, 32, 34, 48, 49, 50, 53, 55, 56, 57, 58]. Innocenti et al. [27] have proven that their ActivMedia Pioneer 2DX mobile robots can reach its target by using their fuzzy logic controller. Same goes to Chen and Sun [21], Vatankhah et al. [22] and Glorennec [20] where they develop the distributed controller purposely for obstacles avoidance for their multi agents by using a fuzzy, neuro fuzzy, and a new optimal control protocol.

In distributed, the main issue is the task has to be distributed in a robust an efficient manner to ensure that every agent is able to perform its individual task cooperatively with another agents to achieve certain target. Distributing task among heterogeneous agents [11, 15] is more crucial and complex comparing with homogeneous agents which are identical [20, 21, 22, 59]. Limited sensing range and low bandwidth are also among physical constraints in distributed approach. With a limited local information, the agent cannot predict and cannot control the group behavior effectively in some sense. Another issues in distributed such as consensus, formation, containment, task allocation, optimization and intelligence will also discussed thoroughly in below section.

3.2. Consensus

Since multi-agent robots need to interact and communicate together to work cooperatively, issue on finding consensus for the homogeneous and heterogeneous robot has attracted researchers’ attention over the past few years. Consensus refers to the degree of agreement among multi-agents to reach certain quantities of interest. The main problem of consensus control in multi-agent robots is to design a distributed protocol by using local information which can guarantee the agreements between robots to reach certain tasks or certain states. Therefore, a large number of interest concerning on developing the consensus control distributed protocol for homogeneous and heterogeneous robots which can be classified into a leader following consensus [60] and leaderless consensus [61, 62, 63, 64, 65, 66], (to name a few), have been intensively studied by researchers recently [22, 67].

Each of heterogeneity agents is not identical and the states between agents are different which will cause difficulties in finding consensus. This is known as cooperative output consensus problem. This is a challenging issue for heterogeneous robots and there are a number of researchers who focused on the leaderless output consensus problem [13, 15, 68] and leader-follower output consensus problem [13, 15, 69, 70, 71]. Wieland et al. [68] proposed an internal model principle to solve the leaderless output consensus problem for heterogeneous linear multi-agent systems. Wang et al. [69] discussed the classes of multi-agent system by switching topologies via static and dynamic feedback.

Research on finding consensus in the broadcasting area has also been carried out by few researchers. Li and Yan [72] solved the consensus in both fixing and switching type topology based on the spectrum radius of stochastic matrices. Azuma et al. [73] studied the consensus problem with a limited communication range and unlimited broadcast range by proposing its own controller. They introduced a concept of connected agent groups. This is to reduce consensus for “group to group” relation and for “agent to agent” relation in the groups by proposing two groups of consensus controller which are local and global. They proved that their controller can work efficiently in a mixed environment with communication and broadcast.

Besides that, research carried out by Das and Ghose [74, 75] solved the positional consensus problem for multi-agents. Das and Ghose [74] proposed a novel linear programming formulation and random perturbation input in the control command to achieve consensus at the pre-specified location. The results showed that novel linear programming that is less intensive computation and perfect consensus can be obtained from random perturbation. They also proposed a novel linear programming formulation for their research [75]. Overall, it can be summarized that consensus problem is a vital issue which has been solved by many researchers. They have identified solutions to consensus problems for either homogeneous agents or heterogeneous agents which focus on finding an agreement among agents based on agent states (linear, nonlinear, static or dynamics topology) although there is an existence of leader in the environment or leaderless. Other than that, finding consensus in broadcasting topology/communication, broadcast mixed environment [73] and positioning agents [74] are also another recent issues focused by previous researchers.

3.3. Containment

Containment control is another problem investigated by many researchers. Containment problem refers to introducing more than one leader among the agents to ensure the groups are not ventured by the hazardous environment. If the agents are faced with this situation, they will move the robots to the safe region spanned by a group of leaders. The agents can either be homogeneous agents that have identical dynamics or heterogeneous agents that have different dynamics.

There are several issues investigated by previous researchers to solve the containment control problem for multi-agent robots such as (i) containment problem for a different dynamic level of the leaders and followers [70, 71, 14], (ii) containment problem for a linear and nonlinear systems [50, 76, 77, 78, 79], (iii) containment problem for first order and second order systems [43, 44, 72]. By assuming the follower and the leader of heterogeneous agents have different dynamics but the dynamics between each follower are similar, Youcheng and Yiguang [80] and Yuanshi and Long [81] had carried out their research studies. Besides that, Haghshenas et al. [14] solved the containment problem for two followers when the dynamic level is not identical.

A research on solving the containment problem for linear and nonlinear systems has been carried out by few researchers. Ping and Wen [76] investigated the linear first order systems for their multiple leaders. The distributed finite time containment problem for linear systems was also being explored by the authors [77]. For nonlinear systems, Liu et al. [50] investigated the distributed containment control problem for second order nonlinear multi-agents with dynamic leaders. The issues of containment problem for the first order and second order systems have been investigated by Ping and Wen [76], Bo et al. [51] and Rong et al. [52]. Ping and Wen [76] studied the first order multi-agent systems while Bo et al. [51] proposed the control protocol for first order discrete-time systems with fixed time delays.

3.4. Formation

Formation control is an important issue to coordinate and control a group of multi-agent robots [49, 82, 83, 84]. The robots must be able to control their relative position and orientation among the robots in a group to move to a specific point. The motivations that drive the most attention among researchers to this problem are the biological inspirations, challenging control problems and the demand of multi-robot systems. There are many issues needed to be considered in designing a controller for mobile robot formation such as the stability of the formation, controllability of different formation patterns, safety and uncertainties in formations [82]. Other than that, issues of formation shape generation, formation reconfiguration and selection, formation tracking as well as role assignments in formation is discussed by Kiattisin.

There are three main control strategies for formation control proposed by previous researchers [82, 85, 86] such as (i) behavior based [87], (ii) virtual structure [88], (iii) leader-follower [89]. Each formation control method has its advantages and disadvantages. Balch and Arkin [87] proposed behavior-based formation control for their multi-robot teams. Behavior-based approach refers to several desired behavior of the agents such as goal seeking, obstacles avoidance, collision avoidance, etc. The final robot decision to choose which behavior comes first is based on the average weight of the behavior. The advantage of this approach is it can be used to guide the multi-agent robots in the unknown or dynamic environment by using the local information that the robot has. However, the drawback is where it cannot guarantee to converge easily during the process.

Virtual structure is a formation control that considers the entire formation as a rigid body which was pioneered by Lewis and Tan [88]. The main advantage of this approach is easy coordination of the group’s behavior and the formation is well maintain during maneuvers. However, the limitation of the virtual structures is it has to maintain the same virtual structure at all times especially when the formation shape needs to be frequently reconfigured. If not, the possible applications are limited. Leader and followers approach is another formation control for multi-agent robots proposed by previous researchers [85, 86, 89]. In this strategy, some robots are considered as leaders while others will act as followers. The leaders will lead the followers to the target path while the followers will position and orientate by themselves while following the leaders. The main advantage of this approach is it can reduce the tracking error while the disadvantages are it will lead to a poor disturbance rejection property and the leader’s motion will not depend on the followers. In addition, the formation does not tolerate to the leader’s faults.

The networking system in formation control is another challenging issue highlighted by Chen and Wang [82] and Kiattisin in their reviewed papers. The communication delay in inter-robot information flow and communication loss problem will affect the performance of formation control and can even make the formation control system unstable. Therefore, a suitable communication protocol and network control system need to be implemented correctly into the robot system. In order to get more realistic formation control design for multi-agent robots coordination, the formation control needs to come together with an effective communication system design (either for local or global information via sensing or wireless network). Lastly, an alternative of implementing a hybrid control framework for multi-agent robots formation control has also become an issue to let the robots work in real world applications.

3.5. Task allocation

The problem of task allocation among multi-agent robots has attracted researcher’s attention. Once the computer assigns the task, the task needs to be sent to the robots for execution. Thus, a suitable approach needs to be applied in the system to ensure that the task is successfully allocated to the robots. Sarker et al. [90] used attractive field model to self-organize their robots while allocating its task. On the other side, Tolmidis and Petrou [91] proposed multi-objective optimization for their dynamic task allocation. The experiment results show a scalability, a generic solution and a better utilization of time as well as energy. Nagarajan and Thondiyath [92] also provided their own algorithm for task allocation which had proven better performances and better in minimizing the turnaround time, makespan and also cost.

3.6. Intelligences

The intelligence of multiagent robots to work cooperatively or coordinate its task is based on its controller. The design of the controller will determine agent’s performances. The evolution of MARS shows that the level of intelligence is increasing in proportional with the technology. Since the beginning of artificial intelligences has been introduced, many researchers have started to design their controller by using this artificial intelligences approaches.

There are several approaches of artificial intelligences have been used by researchers in their multi agent controller development [11, 20, 22, 27, 31, 32, 53]. Fuzzy logic and neural network are approach have been used in multi agent robot control design which already proven its robustness and effectiveness [93]. Al-Jarrah et al. [31] used 2 fuzzy levels, which consisted of a fuzzy probabilistic control and adaptive neuro-fuzzy inference system, ANFIS. Vatankhah et al. [22] proposed a neuro-fuzzy structure with critic based learning structure and [11] proposed iterative learning control (ILC) scheme for their control system. Another researchers [20, 27, 32, 53, 94, 95, 96, 97] were also using fuzzy control as one of the artificial intelligence approaches to develop their robots controller.

Other than artificial intelligence, there is another kind of intelligence proposed by previous researchers that had proven their multi-agent robots work effectively and successfully. Instead of focusing to a basic learning method, Tosic and Vilalta [98] proposed a unified framework for their multi-agent coordination by adopting the reinforcement learning, co-learning, and meta-learning in their system. Leader and follower concept also has been applied by few researchers to coordinate and plan their agent path [11, 22, 31]. Broadcast concept and framework for multi agent coordination can also be considered as an alternative towards intelligence [33, 61, 73, 99]. Azuma et al. [33, 73] developed the controller and broadcasted the signal from “agent to agent” or “agent to all agents”. They also proposed integral-type broadcast controllers and provide a sufficient condition for the controller gain to stabilize the broadcast for their group of Markov Chains [99]. Seyboth et al. [61] proposed the novel control strategy known as event-based broadcast control. They proved that their controller is more effective as compared to the time-based broadcast control. Finally, by having the intelligence, multi agent robot control is ready to be apply for an advance and complex multi-agents applications [3, 36, 49, 59]. As an example, Jolly et al. [53] and Candea et al. [3] have proposed their own controller to let the soccer robots coordinate and play successfully.

3.7. Optimization

Optimization is one of the important issue in designing a control system for multi agent robots. The objective is to find an optimal strategy under a given cost function either to find optimum trajectory/path, time, speed an as well as energy consumption. For example, by minimizing the path, less time is taken by the agent to move to its target point and the energy consumption will become less also.

Kumar and Kothare [100] have discovered the optimal strategy and optimal control architectures for their swarm agents. Their aim was to stabilize a swarm of stochastic agents by proposing the novel broadcast stochastic receding horizon controller. In order to search for an optimal path trajectory by minimizing the trajectory, time and energy consumption, Oleiwi et al. [24] had proposed the optimal motion planner. They combined the modified genetic algorithm with A* algorithm to find a path from the start point to the goal point, fuzzy to avoid obstacles and cubic spline interpolation curve to reduce energy consumption.

However, Chen and Sun [21] had a different approach, where they had proposed a new optimal control protocol to find an optimal control for their multi-agent consensus. Nagarajan and Thondiyath [92] proposed an algorithm that can minimize the turnaround time and cost during the agent’s allocation task. The result showed that the algorithm performed better than the existing algorithm.

3.8. Communications

Issues on communications either implicit or explicit type of communication has been tackled since it will give effect to the multi agent controller performances. There are researchers who have focused on implicit communication where their robots interact based on sensor signal embedded to the robots [4, 30, 31]. However, there are also some drawbacks of implicit communication such as (1) limitations of the hardware and the sensors i.e. the hardware cannot support too many sensors, and the sensors can only work at certain conditions and distances, and (2) time delay if too many agents need to pass the information from one to another. Therefore, explicit communication come in to place where the information (messages) can be sent via broadcast (one to all) [5, 6, 33] or one to one agent [3, 5, 34].

However, other challenging parts of explicit communication are (1) to design a control framework to send the messages efficiently [33], (2) to design a suitable protocol that can guarantee all agents communicate effectively in the environment [3, 5, 34, 35], (3) to solve consensus problem which occurs during the interaction process either for homogeneous or heterogeneous agents [3], and (4) to design optimal controller that can optimize the speed and energy of the robots [33]. With the aim of providing an effective communication system for the robot coordination, researchers have tried to fix these problems by designing a suitable communication control that is relevant to the systems. There are also researchers who complement both communications implicitly and explicitly for their cooperative multi-agents research.

4. Conclusion

This paper has provided a review of cooperative multi-agent robots system (MARS). It shows that this research is leading to the creation of a robust cooperation and coordination of multi-agent robots in various real applications. In order to produce high performance among agents, improvement in controller and communication part is the most crucial issues highlighted by researchers. Thus we strongly believe that this research has a potential to be expanded as the technology develops and the cooperative agents are foreseen to produce a big contribution towards the applications. Improvement on the controller design and communications either by adding intelligences or optimize certain cost function is in parallel with the technologies development which will then produce a multi agents which are mobile, scalable, flexible, global, dynamic and persistent connectivity. Regardingly, the following are other future challenges and recommendations that could be explored by our future researchers in expanding the area of MARS.

4.1. Future challenges

There are many challenges for future cooperative multi-agent systems and, among them, the most crucial challenge lies in controller design, which should be robust and intelligent enough to support overall system. Besides that, communication among agents is also important since it will determine the success of the system. Therefore, there are several future challenges that should be taken into consideration:

1. The need of more powerful coordination among homogeneous and heterogeneous agents. This is especially for advance and complex multi-agent robots application such as soccer robots [3], swarm robots [36], UGV’s [4], UAV’s [4] or any other robots.

2. Since the physical identity and capability among heterogeneous agents are not identical, issues in coordinating the agents will become more challenging compared to homogeneous agents [2, 3, 4, 5, 6, 9, 12]. Attention should be given more to these agents.

3. Adapting various artificial intelligence approaches in solving control and communication problems of MARS either consensus [13, 15, 69, 70, 71], containment [70, 71, 14, 49, 82, 83, 84], position [45, 58] or any other problems should be considered as long as there is an improvement towards the robot performances.

4. Issues in reducing the energy consumption and time travel will produce an optimal controller for the agents. Thus, an appropriate design of controller should be applied together with the suitable communication system that can support the MARS [3, 5].

5. By broadcasting the information to agents, the information can be sent directly to agents, to avoid losses and time delay during transmission of the information [33, 45, 47, 48, 51, 52]. Thus, this research should be expanded since the communication among agents can be improved from time to time.

4.2. Recommendations

Some recommendations for cooperative MARS are as follows:

1. The reactive and deliberative control architectures have their own strengths and weaknesses. In the future, an effective way is to implement hybrid approach into MARS which consists of both reactive and deliberative control that leads to a more efficient system.

2. An effective interaction between multi-agent robots can be achieved by integrating the implicit and explicit communications especially when the number of agents is increasing.

3. A suitable communication protocol and network control system should be implemented into MARS to avoid time delay during transmission of information among agents.

Acknowledgments

The financial support by Malaysian Government and University Teknologi Malaysia for this research is gratefully acknowledged.

Conflicts of interest

Some works in this paper are based on study review from selected journals and proceedings regarding the cooperative multi-agent robot systems. All works from other researchers have been cited carefully.