Autonomous Role Assignment Using Contact Stimuli in Swarm Robotic Systems

Abstract

This study proposes a novel autonomous role assignment method for swarm robotic systems using the response threshold model based on local interactions in a dynamic environment. Ants are social insects with high and low pheromone sensitivity. The pheromone sensitivity of ants is related to autonomous role assignment. The response threshold model was proposed to describe the pheromone sensitivity of ants. The conventional response threshold model assumes that an ant knows the number of workers in an ant colony. However, it is difficult for an ant to contact all workers because its functions are very limited. Therefore, our proposed method adopts a response threshold model based on contact stimuli with foraging ants instead of the worker ratio in an ant colony. In this study, to evaluate the proposed method’s robustness in dynamic environments, we apply it to ant foraging problems in environments with varying amounts and distributions of feeds.

1. Introduction

Swarm robotics is an approach that applies the smart swarm behaviour [1] observed in flocks of birds, schools of fish and swarms of social insects to engineering problems [2, 3, 4]. This study focuses on the excellent functions of swarms of ants, which are social insects. Ants sustain large colonies through caste systems, with the queen at the top, which assign different roles to each caste member. The perception functions and action rules of ants are limited, and communication between them can only be conducted through different pheromones. The queen ant cannot monitor everything that happens in a colony and cannot give instructions to each ant directly. Nevertheless, ants are successfully assigned different roles, such as colony protection, food exploration and foraging, without any centralised management system [5]. The autonomous role assignment mechanism of ants may be useful for transport automation by using several autonomous mobile robots in large warehouses and for search-and-rescue operations using several autonomous drones in disaster relief.

As one of the autonomous role assignments in termites, a worker ant specialises as a soldier ant [6]. However, an appropriate role assignment system is required because an excessively increasing number of soldier ants reduce the amount of collected feeds. Therefore, the specialisation of a worker ant to a soldier ant is impeded by a soldier pheromone, the concentration of which rises with the number of soldier ants. In addition, a colony’s autonomous role assignment allows it to adapt to changing circumstances. For example, when food becomes scarce, a colony needs to increase the number of ants exploring new food sources as well as the number of ants foraging for food once a new food source has been discovered. Rather than assigning roles in a top-down manner, ants assign roles appropriately through local communication using pheromones.

Bonabeau et al. [7, 8, 9] modelled this role assignment using a response threshold model. The response threshold model is an equation that describes the sensitivity of ants to pheromones. There are two types of ants [10]: one with high sensitivity to pheromones and the other with low sensitivity. These different sensitivities are thought to contribute to an autonomous role assignment. However, the conventional response threshold model uses the ratio of workers in an ant colony as an external stimulus, ignoring the crucial factor that social insects can assign roles through local communication.

In contrast, Gordon et al. [11, 12, 13] revealed that an ant’s tendency to perform midden work¹ or foraging work is related to the recent history of its contact with an ant engaged in those works based on the observation of red harvester ants. Our research group has proposed an autonomous role assignment and task allocation method with local interactions in scalable swarm robotic systems [14, 15]. The method used a response threshold model using the ratio of encountered foraging ants in the short term as an external stimulus and mimicking the action rules of real ants. We applied the proposed method to ant foraging problems in a dynamic environment with a varying number of ants [15]. Through simulation results, we confirmed that, during internal environment fluctuations, the proposed method using local interactions outperformed the conventional method using global information.

In this study, we propose a simple autonomous role assignment method using contact stimuli with foraging ants, rather than the ratio of encountered foraging ants in the short term. To evaluate the proposed method’s effectiveness, we apply the method to ant foraging problems in a dynamic environment with fluctuating amounts and distributions of feeds. Through simulation results, we demonstrate that, during external environment fluctuations, the proposed method using local interaction outperforms the conventional method using global information. In addition, we demonstrate that the method can successfully perform role assignment in an ant colony by switching between exploring and foraging behaviours through contact stimuli with foraging ants.

The rest of this chapter is organised as follows. Section 2 explains how to model an ant foraging problem. Section 3 shows the new response threshold model. Section 4 demonstrates the proposed method’s effectiveness through simulations. Conclusions and future work are discussed in Section 5.

2. Foraging problem

This section models an ant foraging problem as a multi-agent simulation following previous studies [16]. In this model, an ant is modelled as an agent. An agent has the following three functions:

• Exploring/foraging behaviour

An agent exhibits either exploring or foraging behaviour. When an agent discovers food, the agent exhibits foraging behaviour and carries food to its nest. Otherwise, the agent explores food sources.

• Homing/trail pheromone

An agent possesses chemical substances termed homing and trail pheromones. The agent secretes the homing pheromone while exploring for food and the trail pheromone while carrying food to the nest. Both pheromones are volatile substances that diffuse and evaporate quickly.

• Worker/non-worker

An agent can be either a worker or a non-worker. While exploring food sources, an agent can perceive trail pheromones if it is a worker but cannot if it is a non-worker. While carrying food, the agent can perceive homing pheromones whether it is a worker or a non-worker.

Table 1 shows the relationship among exploring/foraging behaviour, worker/non-worker and homing/trail pheromone.

Figure 1.

Pheromone-following action rules.

	Worker	Non-worker
Exploring behaviour	able to perceive trail pheromones	unable to perceive trail pheromones
	laying homing pheromones
Foraging behaviour	able to perceive homing pheromones
	laying trail pheromones

Table 1.

Relationship among behaviour, pheromones and worker/non-worker.

Next, we describe the modelling of perception and action. As shown in Figure 1, an agent can perceive the difference in pheromone level between three front cells and the current cell and can select one of three action rules before moving to the next cell. The three action rules are as follows:

1. Random walk

   When the difference in pheromone level is negative, an agent randomly selects one cell from three cells and moves to that cell.

2. Pheromone trail

   When the difference in pheromone level is positive and an agent detects the pheromone in the front cell (Figure 1a), the agent moves forward. On the other hand, when the agent detects pheromones in the right and left front cells, it moves to the cell with the highest pheromone level (Figure 1b).

3. Turn around

   When an agent discovers food or exhausts its homing pheromone, the agent turns to the nest. However, the agent also turns to the nest if the angle between the moving direction and the direction of the nest is greater than 90° [17].

   An agent can exhibit exploring or foraging behaviour by changing the combination of rules (Table 2). In exploring behaviour, an agent moves according to Rule 2 if it is a worker and can detect a trail pheromone, but an agent moves randomly according to Rule 1 if it is a non-worker. In foraging behaviour, both worker and non-worker agents follow Rule 3. If an agent detects a homing pheromone, it moves according to Rule 2. If it detects no homing pheromone, it moves according to Rule 1.

	Worker	Non-worker
Exploring behaviour	Rule 2 $\to$ Rule 1	Rule 1
Foraging behaviour	Rule 3 $\to$ Rule 2 $\to$ Rule 1

Table 2.

Relationship between worker/non-worker and behaviour rules.

3. Response threshold model

This section proposes a new response threshold model using contact stimuli with foraging agents. Firstly, we describe the role that a response threshold model plays in autonomous role assignment and introduce the conventional response threshold model. Next, we describe the proposed response threshold model that uses contact stimuli with foraging agents as external stimuli. There are two types of ants: those sensitive to external stimuli and those insensitive to external stimuli. Sensitivity to external stimuli can be modelled using a parameter called a response threshold. An agent with a low response threshold is likely to become a worker even if its sensitivity to external stimuli is weak; however, an agent with a high response threshold is unlikely to become a worker even if its sensitivity to external stimuli is high. Thus, a response threshold can prevent outcomes in which all agents are workers or non-workers. In the conventional response threshold model, an agent changes from a worker to a non-worker with probability $p$ and changes from a non-worker to a worker with the probability described using the following equation:

$$q=\frac{s{\left(t\right)}^2}{s{\left(t\right)}^2+\theta {\left(t\right)}^2},$$E1

where $\theta$ and $s$ represent a response threshold and an external stimulus at time $t$, respectively. The response threshold is updated using Eq. (2) if the agent is a worker and using Eq. (3) if it is a non-worker. If the agent is a worker, the response threshold decreases and its sensitivity to external stimuli increases. If the agent is a non-worker, the response threshold increases and its sensitivity to external stimuli decreases.

$$\theta \left(t+1\right)=\theta \left(t\right)-\xi .$$E2

$$\theta \left(t+1\right)=\theta \left(t\right)+\psi .$$E3

In the conventional model, a stimulus $s$ is updated by the ratio of the number of workers $N_w\left(t\right)$ to the total number of ants $N_t\left(t\right)$ in an ant colony, as described by the following equation:

$$s\left(t+1\right)=s\left(t\right)+\delta -\alpha \frac{N_w\left(t\right)}{N_t\left(t\right)},$$E4

where $\delta$ represents an increase in loads per unit time if no ant forages. The third term on the right side of Eq. (4) represents a decrease in loads per ant to the ratio of the number of workers in the ant colony, and $\alpha$ represents a scale factor. That is, if the worker ratio in the ant colony decreases, the stimulus $s$ increases with increasing loads per ant and the probability of changing from a non-worker to a worker increases. However, ants cannot know the state of all other ants. Therefore, the above equation cannot represent the mechanism by which ants can form orderly swarms through local interactions. We, therefore, propose a novel equation as follows:

$$s\left(t+1\right)=\mathit{\beta cs}\left(t\right),$$E5

$$\mathit{cs}\left(t\right)=c\left(t\right)+\mathit{\gamma cs}\left(t-1\right),$$E6

$$c\left(t\right)=\left\{\begin{array}{ll}1& \text{if\hspace{0.17em}an\hspace{0.17em}agent\hspace{0.17em}contacts\hspace{0.17em}with}\mathrm{a}\text{foraging\hspace{0.17em}agent},\\ 0& \text{otherwise}.\end{array}\right.$$E7

In the proposed model, a stimulus $s\left(t+1\right)$ is updated by multiplying a contact stimulus $\mathit{cs}\left(t\right)$ by a scale factor $\beta$. The contact stimulus $\mathit{cs}\left(t\right)$ decreases by the attenuation rate $\gamma$ over time if an agent does not contact a foraging agent. If an agent contacts a foraging agent, $c\left(t\right)$ is 1, otherwise 0.

4. Simulations

We applied the proposed method to an ant foraging problem and evaluated its robustness in a fluctuating external environment. The simulation results show mechanisms that can flexibly assign foraging and exploring agents in an environment with fluctuating amounts and distributions of feeds through contact stimuli with foraging agents as the local interaction.

4.1 Simulation setting

Figure 2 depicts the simulator. The simulator was constructed with reference to the following previous studies [9, 15, 16, 18], with the best simulation parameters selected through preliminary experiments. Table 3 shows the parameters used in simulation experiments. The experimental environment comprised a two-dimensional grid space of 150 × 150 cells. The nest was placed in the centre of the environment. The simulation halted after 10,000 steps in one trial, and we conducted 50 trials in each experimentation setting. A red cell represents a worker agent, and a purple cell represents a non-worker agent. When an agent touches the feed, it carries the feed to the nest; it is represented as an orange cell. A green cell contains a trail pheromone, and a blue cell contains a homing pheromone. As the pheromone evaporates, the pheromone level decreases, and the colour of the cell becomes lighter. The homing and trail pheromones do not mix. In the initial state, food sources were randomly placed in the food source area. When the feed in one source is exhausted, the next food source is placed randomly in the area. In the simulation, we fluctuated the amount and distribution of feeds to evaluate the proposed method’s robustness in a dynamic environment. The simulation alternated between three different types of environments as follows:

1. Type-A: In Figure 3a, the environment is dotted with four small food sources. Each food source includes one feed.

2. Type-B: In Figure 3b, the environment is dotted with four medium food sources. Each food source includes nine feeds.

3. Type-C: In Figure 3c, the environment has one large food source. The food source contains one hundred feeds.

Figure 2.

Simulator.

Notation	Description	Values
Simulation
step	the maximum number of steps in one trial	10,000
trial	the number of trials in each experiment	50
Response threshold
$p$	probability with which an agent changes from a worker to a non-worker	0.001
$\theta$	initial response threshold (Eq. 1)	500
	response thresholds range (Eq. 1)	0–1000
$\xi$	coefficient for updating a response threshold when an agent is a non-worker (Eq. 2)	1
$\psi$	coefficient for updating a response threshold when an agent is a worker (Eq. 3)	10
Conventional method
$\left(\delta ,\alpha \right)$	combinations of (1,3), (3,5), (5,7), (7,9) and (9,11) were tested in simulation experiments (Eq. 4)
Proposed method
$\beta$	coefficient for scaling a contact stimulus with a foraging agent (Eq. 5)	1000
$\gamma$	attenuation rate for decreasing the contact stimulus over time (Eq. 6)	0.99

Table 3.

Parameters of simulation experiments.

Figure 3.

Three types of environments, with varying amounts and distributions of feeds.

4.2 Evaporation and diffusion of pheromones

An agent secretes a trail or homing pheromone while moving. The initial value of each pheromone is 1.0, and each pheromone decreases at a rate of 0.99. The agent returns to the colony when the residual quantity of each pheromone is less than 0.01. These parameters were set through preliminary experiments so that an agent can sufficiently explore an environment. Here, an agent can explore an environment in approximately 450 steps.

Pheromones are spread out by evaporation and diffusion, diluting their density. The equations for these evaporation and diffusion phenomena are defined as follows.

$$F_p\left(x,y,t\right)=\mu F_p\left(x,y,t-1\right)+\mathrm{\Delta }{F}_p\left(x,y,t\right)$$E8

$$\mathrm{\Delta }{F}_p\left(x,y,t\right)=\left\{\begin{array}{ll}{Q}_p& \text{if}\text{an}\text{agent}\text{is}\text{in}\text{the}\text{grid}\left(x,y\right)\\ 0& \text{otherwise}\end{array}\right.$$E9

$$a_p\left(x,y,t\right)=\begin{array}{l}{a}_p\left(x,y,t-1\right)+\lambda \{a_p\left(x+1,y-1,t-1\right)+a_p\left(x+1,y,t-1\right)\\ +a_p\left(x+1,y+1,t-1\right)+a_p\left(x,y-1,t-1\right)+a_p\left(x,y+1,t-1\right)\\ +a_p\left(x-1,y-1,t-1\right)+a_p\left(x-1,y,t-1\right)+a_p\left(x-1,y+1,t-1\right)\\ -9a_p\left(x,y,t-1\right)\}+\left(1-\mu \right)F_p\left(x,y,t\right),\end{array}$$E10

where $Q_p$ denotes the addition quantity of the pheromone, and $F_p\left(x,y,t\right)$ represents the quantity of the pheromone in a grid $\left(x,y\right)$ at a time, $t$. $a_p\left(x,y,t\right)$ represents the quantity of the pheromone above a grid $\left(x,y\right)$ at a time, $t$. The second term on the right side of Eq. (10) represents the quantity of the pheromone that inflows, outflows and disappears from neighbouring grids, and the third term on the right side represents the quantity of the pheromone that evaporates. An agent detects the pheromone quantity, $a_p$, in the three forward cells. In the simulation, the initial pheromone density was set as $Q_p=1.0$. $\gamma$ and $\lambda$ represent the rates of evaporation and diffusion, respectively, with values set as $\mu =0.99$ and $\lambda =0.01$. These parameters were set with reference to a previous study [19]. However, if the rates of evaporation and diffusion are very high, agents cannot arrive at a food source by following a pheromone trail because pheromones will disappear rapidly. Conversely, if the rates are very low, agents cannot discover pheromone trails leading to food sources because pheromones will fill the environment. Thus, we made appropriate adjustments to fit the simulation environment through preliminary experiments.

4.3 Simulation results

4.3.1 Appropriate worker ratio in each environment

Firstly, we reveal the appropriate worker ratio in the three types of environments with varying amounts and distributions of feeds. Figure 4 depicts the means and standard deviations of collected feeds by a swarm of agents with different worker ratios in the three types of environments. The swarm of agents needed to increase the number of exploring agents because there was only a small amount of feed in the type-A environment, with four small food sources. Therefore, the mean of collected feeds was higher as the worker ratio was smaller. In the type-B environment, with four medium food sources, both foraging and food exploration were important for the swarm of agents. Therefore, the maximum mean of collected feeds was obtained when the worker ratio was 60%. In the type-C environment, with only one large food source, a swarm of agents could easily discover the large food source. Therefore, the swarm of agents needed to mobilise several ants to collect feeds efficiently. However, if all agents attended to the foraging call, it may take a long time to discover a new food source. Thus, the maximum mean of collected feeds was obtained when the worker ratio was 80%. According to the above results, the swarm of agents uses the appropriate worker ratio in each environment with varying amounts and distributions of feeds.

Figure 4.

The means and standard deviations of collected feeds in different worker ratios.

4.3.2 Adaptability of the proposed method

We studied the proposed method’s adaptability in environments with varying amounts and distributions of feeds. In addition, we compared the proposed method with the conventional method in terms of the mean of collected feeds by a swarm of agents.

Figure 5 illustrates the means and standard deviations of collected feeds in the type-A environment. For example, C(1–3) denotes the conventional method, with the load parameter, $\delta$ and the scale factor, $\alpha$, set as 1 and 3, respectively. On the other hand, P denotes the proposed method. Here, the maximum mean of collected feeds of the conventional method was 126. Similarly, the mean of collected feeds of the proposed method was 131. The conventional and proposed methods had the same foraging ability.

Figure 5.

Means and standard deviations of collected feeds in type-A environment.

Figure 6 shows the relationship among the worker ratio, the foraging agent ratio and the amount of existing feeds in an environment. The horizontal axis represents the number of steps. The vertical axis represents the ratio of workers and foraging agents, and the secondary vertical axis represents the amount of existing feeds in the environment. In the simulation, when the feed in one source is exhausted, the next food source is placed randomly in the environment. Therefore, the amount of feeds fluctuated between 3 and 4. That is, the number of vertical blue lines represents the amount of collected feeds, and its slits represent the time spent discovering a new food source. Figure 6 depicts the results of the proposed and conventional methods in terms of the amount of feeds collected. The worker ratio of the conventional method remained constant at approximately 30%, whereas that of the proposed method fluctuated with the amount of existing feeds in the environment. In addition, the mean worker ratio of the proposed method was 28%. As a result, in both methods, the swarm of agents could discover a new food source quickly by increasing the number of exploring agents.

Figure 6.

The transition of the worker ratio and the amount of existing feeds in type-A environment.

Figure 7 displays the means and standard deviations of collected feeds in the type-B environment. The maximum mean of collected feeds of the conventional method was 604. Here, the load parameter, $\delta$ and the scale factor, $\alpha$, were set as 3 and 5, respectively. The mean of collected feeds of the proposed method was 622. As shown in Figure 8, the worker ratio of the conventional method remained constant at approximately 60%. On the other hand, the worker ratio of the proposed method increases when a new food source is placed in the environment, whereas its worker ratio decreases when the amount of existing feeds in the environment reduces. That is, with the proposed method, a swarm of agents could collect large amounts of feeds by adjusting the number of foraging and exploring agents according to the amount of existing feeds in the environment. Furthermore, the proposed method’s mean worker ratio was 60%, which is mostly identical to that of the conventional method.

Figure 7.

Means and standard deviations of collected feeds in type-B environment.

Figure 8.

The transition of the worker ratio and the amount of existing feeds in type-B environment.

Figure 9 displays the means and standard deviations of collected feeds in the type-C environment. The maximum mean of collected feeds of the conventional method was 997. Here, the load parameter, $\delta$ and the scale factor, $\alpha$, were set as 9 and 11, respectively. The proposed method’s mean of collected feeds was 943. As shown in Figure 10a, the conventional method’s mean worker ratio is 80% because many workers must collect feed from one large food source effectively. On the other hand, the proposed method’s mean worker ratio is 64%, as shown in Figure 10b. The reason for this was that, with the proposed method, the swarm of agents exhausted the feed in a food source before its worker ratio reached 80%.

Figure 9.

Means and standard deviations of collected feeds in type-C environment.

Figure 10.

The transition of the worker ratio and the amount of existing feeds in type-C environment.

4.3.3 Role assignment process

We explain the role assignment process in the proposed method in each environment with varying amounts and distributions of feeds. Figure 11 depicts the relationship between the contact stimuli and the worker/non-worker state for a certain agent. In this graph, the horizontal axis represents the number of steps. The vertical axis represents the response threshold and the probability of changing from a non-worker to a worker. However, the value of the response threshold was normalised from 0.0 to 1.0. The secondary vertical axis represents the strength of the contact stimulus. The red line indicates the strength of contact stimuli with foraging agents. The green line indicates the transition of the response threshold. The blue line indicates the transition of the probability of changing from a non-worker to a worker. The light blue line indicates that an agent is in a worker or non-worker state. In addition, a convex shape indicates a worker state and a concave shape indicates a non-worker state. The probability of changing from a non-worker to a worker varied with the contact stimuli and the response threshold. On the other hand, the probability of changing from a worker to a non-worker remained constant at 0.1%.

Figure 11.

Relationship between contact stimuli and worker state in the proposed method.

As shown in Figure 11a, an agent remains in a non-worker state until approximately 5,000 steps, and the duration of the non-worker state is very long. The reason for this was that the frequency of contact stimuli with foraging agents was low because the type-A environment had only a small amount of feed. On the other hand, as shown in Figure 11b, an agent contacts foraging agents frequently because there is much feed in the environment, increasing the duration of the worker state. The agent contacted two foraging agents continuously at approximately 2,800 steps. Furthermore, the strength of the contact stimulus reached 1.99, and the agent changed to a worker. As shown in Figure 11c, an agent contacts foraging agents frequently after 4,000 steps and changes to a worker quickly even if it had changed to a non-worker. According to the aforesaid results, the proposed method can perform role assignment automatically under different environmental conditions through contact stimuli with foraging agents according to the amounts of feeds.

4.3.4 Robustness in a dynamic environment

To evaluate the proposed method’s effectiveness in a dynamic environment, we compared the proposed method with the conventional method in terms of the mean of collected feeds in an environment with varying amounts and distributions of feeds. Figure 12 depicts the means and standard deviations of collected feeds in a dynamic environment. The conventional method’s maximum mean of collected feeds was 1102, and its standard deviation was 414. Here, the load parameter, $\delta$ and the scale factor, $\alpha$, were set as 5 and 7, respectively. Conversely, the proposed method’s mean of collected feeds was 1176, and its standard deviation was 61. Based on the difference between standard deviations, the amount of collected feeds of the conventional method was unstable, whereas that of the proposed method was stable in the dynamic environment.

Figure 12.

Means and standard deviations of collected feeds in a dynamic environment.

Figure 13 depicts the proposed method’s results. The red line indicates the transition of the worker ratio, and the blue line indicates the amount of existing feeds in the environment. As shown in Figure 13, the worker ratio is low in the type-A environment with a small amount of feed. This means that the swarm of agents could discover new food sources by increasing the number of exploring agents. Conversely, the worker ratio was higher in the type-B and C environments, with more feeds. This means that the swarm of agents could collect much feed effectively by increasing the number of foraging agents. As a result, the proposed method could perform autonomous role assignment effectively through contact stimuli with foraging agents according to the external environment fluctuations. On the other hand, to change the worker/non-worker state, the conventional method used the stimulus updated by the previous stimulus, the load parameter and the worker ratio in the colony. This means that the strength of stimuli did not reflect the external environment fluctuations. Therefore, the conventional method did not allow agents to be assigned roles according to external environment fluctuations.

Figure 13.

The transition of worker ratio and the amount of collected feeds of the proposed method. (A), (B) and (C) in the graph denote an environment with four small food sources, an environment with four medium food sources and an environment with one large food source, respectively. The maximum number of steps in this simulation is 50,000.

5. Conclusions

This study proposed a novel autonomous role assignment method using the response threshold model through local interactions in swarm robotic systems. In this study, to study the proposed method’s robustness in a dynamic environment, we applied the proposed method to ant foraging problems, where the amount and distribution of feeds fluctuate in an environment. The conventional response threshold model, which mimics autonomous role assignment mechanisms of ants, uses the worker ratio in an ant colony as external stimuli. Conversely, the proposed method uses contact stimuli with foraging agents as external stimuli. Our simulations confirmed that, with the proposed method, agents can maintain an appropriate worker ratio and can effectively collect feeds in a dynamic environment compared with the conventional method. In addition, through the analysis of autonomous role assignment processes of ants, we revealed the mechanisms by which agents can specialise as a worker or a non-worker appropriately according to the external environment fluctuations through the frequency of contact stimuli with foraging agents. In future work, we will apply other mechanisms of cooperative behaviours of ants and bees, which are social insects, to swarm robotics systems. In addition, we will create a rescue robot team composed of many autonomous drones.