Agent based model

This is anOpe Abstract – In the CONFIDENCE project, we developed an agent based model (ABM) to simulate the decision making process involving stakeholders of different interests. Our model aims to support decisions on the most suitable protection strategies in different accident phases. The intelligent agents and the models of the negotiation/voting process are described in the paper. Given five scenarios, the numerical results from the computational implementation of the ABM are visualized and analysed in order to better understand the negotiation and voting processes. Our ABM can be expanded in order to support the decision making processes of many different stakeholders of various types of risk management apart from nuclear and radiological emergency management.


Introduction
Agent-based modelling (Gilbert, 2008) is a microscopic computational method that reproduces and predicts complex phenomena by simulating simultaneous actions and interactions of autonomous agents (independent individuals or collective entities such as organizations, teams). It integrates several other ideas such as game theory, complex systems, evolutionary computations, and generates randomness with Monte Carlo methods. One of the generally accepted principles is that the whole is greater than the sum of its parts. The behavior of the whole agent-based system is not simply a linear superposition of individual behaviors, since the effect of the interaction between agents is not a simple addition of the individual effects, but a kind of organic connection, which is always interdependent and interacting. The relationship of agents is usually not totally parallel but more likely causal, complementary, or reciprocal. That is, agents can change their behavior according to the behaviors of their neighborhoods. For example, two people, Tom and Peter, want to go for dinner. There are two restaurants A and B available. Tom selects Restaurant B but Peter selects Restaurant A. In negotiation, Tom explains that he doesn't want to go to Restaurant A only because it is too far away from his apartment. He can accept food in Restaurant A and B. Peter really wants to go to Restaurant A and therefore he offers Tom that he can pick him up on the way to Restaurant A. Tom can accept this offer and then both agree to go to Restaurant A. Roughly speaking, the actions of agents determine those of the whole system in a "zero-sum" game or in other ways, for example, mutual benefit and cooperation, in which ultimately the agents' interests as a whole can be maximized as much as possible, where it can be frequently observed in physical systems as resonance phenomenon.
In the CONFIDENCE project, the decision process between stakeholders with different backgrounds has been well studied Raskob et al., 2020) taking into account outputs and recommendations from the previous research projects (Dubreuil et al., 2010;Duranova, 2016;Bohunova et al., 2016;Gallego and Montero, 2016). In this decision process that takes place mostly in later phases of an emergency, the stakeholders decide about most suitable protection strategies, some of them might be stored on a knowledge database (Bai et al., 2018a), while each stakeholder may seek to exert his own influence on the decision process. Considering that multiple stakeholders have different viewpoints, preferences and aspirations, the decision processes with multi stakeholders can be more challenging compared to those in which only one is involved.
In CONFIDENCE ABM, the intelligent agents are realized by three computer modules for agent groups, preferences with uncertainty, and evaluation of protection strategies. The interactions of agents are implemented as two processes, negotiation and voting. Details are given in the following sections.
personal gain, such as for reproduction, for positive interests or for social status, and can make decisions only by preliminary or simple decision rules. However, in CONFIDENCE ABM we focus on modelling intelligent agents in cooperation, which means the agents are designed to be able to learn from each other and be adapted to the society, so that the system is able to be cooperative. During negotiations happened in real life, there may exist stakeholders that stick to their own interests and act non-cooperative. In this ABM, the degree of compromises of agents is introduced as an indicator to illustrate the extent of cooperation. When the degrees of compromises of most agents who attended in the negotiation are very low, it may lead to the negotiation deadlock. In the CONFIDENCE ABM, the voting process is active if the negotiation fails. It can be also alternative methods to avoid the deadlock, for example, to set a leader making the final decision if no consensus can be found. Further study on such a topic could be continued in follow-up projects.

Agent groups
According to the questionnaire distributed to organizations that participated in CONFIDENCE, we classified 12 possible agent groups: experts from nuclear regulatory, representatives of environmental protection associations, representatives of economic organizations, representatives of trade unions, representatives of medical unions, representatives from justice, members of parliament elected in the area, representatives of general councils and regional councils concerned, representatives of governmental bodies, representatives of site operator, NGO, and police.

Degree of compromises of the agents in each negotiation
To describe how much the agents can compromise in each negotiation, we introduce the indicator, Deg (A, B) = Deg (B, A) ∈ [0, 1] where A, B are agents. If A and B are from the same group, then the degree will be very high, e.g. up to 1. If A and B are from different groups, the degree may be very low, e.g. down to 0.1. When the value of this factor is too low, it can lead to low efficiency in the negotiation and therefore probably the process lasts for a long time but without forming a unified opinion.

Boundary conditions on the preference of the agent group
The boundary condition of the preference value of each agent to be introduced can be seen as one implementation of uncertainty. In fact, we denote the preference values of Agent A as a vector, P (A) = (P 1 (A) , P 2 (A) , ...). The preference value of agents P i ðAÞ∈½a; bð0 a < b 10; i ¼ 1; 2; . . .Þ should be chosen randomly from the range [a, b] which can be regarded as uncertainty. Currently, we assume the randomness obeys the standard normal distribution, but in future, when the empirical data are available, skewed distributions can also be taken into account during data analysis. Furthermore, the boundary condition also affects the negotiation process because it describes actually the bottom line for the agent, i.e. no negotiation process will break these constraints, and in any case the agent will not unconditionally compromise. In short, the boundary condition influences agents on the extent to which they make concessions in negotiations.

Criteria used for decision-making
In this section, we list the criteria implemented in CONFI-DENCE ABM application: health effects, economic aspects (cost), environment impact, effectiveness (approximate amount of contamination removed), available resources, maintenance of reputation, duration (for people to return, food ban), number of people affected, public acceptability, ethical considerations.

Negotiation process 3.1 Outline of multilateral negotiation
In this paper, we consider multilateral automated negotiation, which means that more than two agents are involved in the negotiation process. Figure 1 shows that the negotiation process can be realized in a concise and efficient way by deploying the following five parts, i.e. agent, protection strategies, negotiation protocol, negotiation strategies and meeting room.

Scoring based on MCDA
Multiple-Criteria Decision Analysis (MCDA) can be applied in the negotiation process to support the negotiating agents in eliciting their preferences and scoring the protection strategies. It reflects the structure in agents' preferences over multi-attribute strategies spaces, builds the utility functions and applies them as the scores of the specific protection strategy for each agent.

Ranking based on the scores
The strategy considered by an agent to be of the highest rank receives the highest score from that agent. Negotiation cut-down mechanisms can be described as follows.
If one of the following conditions is satisfied, most agents select the same strategy as the best (i.e. with the highest score from the agents), the simulation runs more than a pre-defined number of times,
The ranking system is also used in the voting process.

Setting for the agents
In order to test, we assume 12 agents in negotiation and the composition of agents is that two are experts from nuclear regulatory; none is police and one from each of the remaining groups given in Section 2.1.

Setting of preferences
Preference values can range between 0 and 10 and the stakeholders can have different preference values on every criterion, even if they come from the same stakeholder group. Figure 2 shows the box plot of one test data. Such a box plot can give us some detailed information. When we compare Agent Group (AG) 1 and AG10, the lowest value of 2 in AG10 is greater than the lowest score of 1 in AG1. The same applies to the highest value and first/third quartile. But when considering the median, the preference value of AG1 is the same as AG10.

Description of scenarios
In our example we model the agents to be fully cooperative again assuming their willingness to come to consensus. The protection strategies in the long-term accident phase are selected from a knowledge database (KDB) by Cased-based reasoning (CBR) (Moehrle et al., 2018). The generic knowledge base for emergency management comprises the comprehensive description of events, including the information about the cause, time, place, weather, environment, a number of indicators or measurements, etc. One application of the KDB is, given a description of nuclear event, the protection target category, the level of contamination and time for implementation, the strategies for remediation options in inhabited areas and food production systems can be retrieved, reused and revised by CBR. In CONFIDENCE project, the strategies for long term management are characterised as: a limited number of surfaces covered; all surfaces, low waste; all surfaces, high waste; all surfaces, low waste, late start of measures; do nothing.

Numerical results of negotiation process
In this section, we show numerical results that describe the negotiation process of the 12 agents related to the five strategies. Finally, they successfully agree on the rank of one preferred strategy. In the left subfigure in Figure 3, different agents have different ranking of the five strategies before negotiation. For example, Agent 1 considers Strategy 2 ≻ Strategy 3 ≻ Strategy 4 ≻ Strategy 5 ≻ Strategy 1, while Agent 2 prefers Strategy 4 ≻ Strategy 5 ≻ Strategy 3 ≻ Strategy 2 ≻ Strategy 1. After negotiation, the agents get the same ranking shown in the right subfigure in Figure 3. Figure 4 shows more details of the process. From the left subfigure, we can see the agent selects Strategy 4 as the best before negotiation but after several iterations, he concludes that Strategy 3 is the best. This clearly demonstrates the effect of negotiation. In the right subfigure, the scores of one specific strategy for all the agents become close and stable during negotiation, that is, the simulation is to some extent convergent.

Voting process
When agents can't agree in negotiation, they may decide to vote. In the design of CONFIDENCE ABM, agents vote for protection strategies according to the ranking based on the scores of these strategies obtained by the MCDA. They are all treated as equal leading for all the votes to have the same impact. Other strategies of considering different influences of agents are possible but have not been further investigated. As mentioned before, five protection strategies are pre-selected for negotiation. Based on this number each agent has 15 (= 1 þ 2 þ 3 þ 4 þ 5) votes, which refers to the fact that all the agents are treated equally. The more votes received from the agents, the higher is the rank of that strategy. The decision tree is shown in Figure 5. The voting can proceed in limited rounds. If the best strategy is not selected in one round of voting, the number of protection strategies will be reduced in the next voting, so that the votes can be more concentrated, thereby increasing the possibility of selecting the best strategy in the next round.
One result is presented in Figures 6a and 6b: Strategy 3 receives 55 votes and Strategy 1 receives 48 votes. 4 agents select Strategy 1 as the best strategy, while 8 agents select Strategy 3 as the best strategy. Therefore Strategy 3 is presented as solution.

Discussion and conclusions
The developed under CONFIDENCE project ABM toolkit can simulate decision-making and demonstrate how results are affected by changes in the composition of stakeholders, the preference values, the degree of compromises, and the boundary conditions on the preferences (Bai et al., 2018b). Users can design different inputs and observe how the agents negotiate in this setting, if they can come to an agreement or not, which strategy is to be selected if the negotiation is successful, and what the results of the voting are. By the aid of the toolkit, users may get a better understanding of the negotiation process and might transfer the knowledge to those in real life. S. Bai et al.: Radioprotection 2020, 55(HS1), S187-S191 S189    S. Bai et al.: Radioprotection 2020, 55(HS1), S187-S191 Regarding the ABM software itself, there are two main improvements envisaged in the future. The first is about the negotiation process. More negotiation protocols and negotiation strategies can be realized specifically tailored to different projects. Further uncertainty studies can be carried out based on the statistical analysis of hundreds of numerical negotiation simulations (Bai and Raskob, 2019). The second is related to the voting process. The voting process can be used in combination with the negotiation process in the case that there is no single strategy agreed on by most agents as the best strategy but those, e.g., which get high total scores (the sum of the scores over all agents) can be used for voting. It is worth mentioning that the voting process can also be applied directly after scoring the strategies by agents before negotiation. In the future, more types of voting, e.g. runoff voting, can be taken into consideration to be implemented in the toolkit.