MCDA handling uncertainties

This is anOpe Abstract – Work package 6 (WP) of the European project CONFIDENCE focussed on decision support for stakeholders in nuclear emergencies especially considering uncertainties in such scenarios. A well-suited method for collaborative decision support is the Multi Criteria Decision Analysis (MCDA). It provides a transparent approach for choosing a suitable strategy from a pool of strategies taking into account the different preferences of the involved stakeholders. One goal of WP 6 was to provide this method as a software tool to the nuclear emergency management community. However a default MCDA is not capable of handling uncertainties as input parameters. We embedded an existing MCDA tool in an ensemble evaluation framework to overcome this limitation and to process probabilistic input parameters. Within this framework random deterministic MCDAs are generated from the probabilistic MCDA and their results are combined into a single result reflecting this uncertainty. The enhanced MCDA tool provides user interfaces for defining probabilistic input parameters for impact values and criteria weights, as well as graphical components to communicate the results to the stakeholders. The tool was presented in several workshops to the potential stakeholders where it was applied to fictitious scenarios. The feedback was used to improve the tool in several iterations.


Introduction
Multi Criteria Decision Analysis (MCDA) is a well known method used for decision support in many fields (Triantaphyllou, 2000;Hiete et al., 2010) and could be a part of tools for informing discussions, debates and deliberations (TIDDDs) (see Dubreuil et al., 2010). It is of low computational complexity in comparison to other methods and very generic which is why it can be applied not only in crisis management but in many different fields.
The goal of a main class of MCDA methods is to establish a ranking on a set of alternatives thus providing decision support by determining a comparable value A 1 , ..., A n for each alternative through integrating a set of predefined criteria C 1 , ..., C m . Since the criteria are typically of different units and scales, the values C 1 , ..., C m have to be normalized before combining them into an overall result. For this purpose, normalization functions N 1 , ..., N m , such as e.g. min-max normalization, have to be defined for every single criterion (Vafaei et al., 2016). Trade-offs between criteria are represented through weights. Their relative importance is reflected in a specific normalized weight w 1 , ..., w m for each criterion. Using the normalized values and weights of criteria a ranking value is determined by an aggregation method. One of the most popular aggregation methods is the computation of the weighted sum, which for each alternative requires the following computation: The results A 1 , ..., A n are sorted according to their value. The top alternative having the highest value is assumed to be the most preferable one (assuming all criteria have to be maximised, which can be part of the normalization). Figure 1 shows the matrix-like structure of the previously mentioned formula.
The results can be presented in multiple ways like e.g. charts, graphs, textual report, and others depending on the specific requirements of the decision makers.
The MCDA as described above will process deterministic parameters, yet many (if not all) scenarios of decision making are affected by uncertainties, which boil down to processing probabilistic parameters. The following section describes how this limitation can be overcome.
criterion. Both types are deterministic in the default MCDA.
As both values and weights can be affected by uncertainty, both have to be defined as probabilistic.

Defining uncertainties
For a practical application there are two possibilities to define probabilistic values and weights. If the characteristics of a criterion are well know, it is possible to provide a distribution function with the according parameters for each criterionalternative pair for example a normal distribution with parameters mean and standard deviation. If such specific knowledge is not available but numerical measurements are on hand, the actual distribution can be estimated by use of a histogram. Both representations allow for random sampling of values which is required for the ensemble evaluation (see example below). Figure 2 shows an example of the parametrization of a criterion value as normal distribution function as it was implemented in the MCDA tool for CONFIDENCE.

Ensembles
As the probabilistic MCDA cannot be evaluated directly, an ensemble of deterministic MCDAs is generated by random sampling of the probabilistic MCDA. The members of the ensemble are evaluated one by one and the results are combined into a single result, which inherently contains uncertainty. As the number of criterion-alternative pairs together with weights can easily become large, the number of generated ensemble members should take this into account and be chosen sufficiently large. Fortunately, the evaluation of one MCDA instance is very simple, so even evaluating hundreds of thousands of ensemble members is a matter of seconds.

Aggregating ensemble results
As MCDA provides a value for each alternative, which is used to establish a ranking on the set of alternatives, a natural approach to aggregate the ensemble results is to determine statistical attributes like mean, median, quantiles, minimum, maximum, etc. For practical reasons, as the ensemble set may be huge in size, applying methods like moving average, moving median, and moving quantiles is recommendable. Even if not always exact, these methods significantly reduce the amount of data to be stored during the evaluation.
However, displaying these attributes as box-plot shows that they are not very suitable for decision support as shown in an example in Figure 3. While the means appear different in general, boxes frequently overlap and it is not clear which alternative would be better. Instead of aggregating the ranking values aggregating the actual ranking might be preferred as shown in the following.
Several methods to aggregate on the ranking itself are possible, for example to determine rank winners by counting how many times an alternative ranked first place, how many times it ranked second place, etc., or to determine outranking by counting how often an alternative was better in rank compared to another one. For visualization of these results bar charts or bubble charts are suitable. Figures 4 and 5 show examples of such charts as they were implemented in the MCDA tool for CONFIDENCE. It is much clearer from these charts compared to the box charts that alternative "LWþRelocation" dominates the others. The shortcoming of this second approach is that it does not show the actual performance of the different alternatives. For this reason, the use of both types of information may be most helpful.

Evaluation
The enhanced MCDA tool capable to deal with uncertainties was presented in several workshops during the CONFIDENCE project where the stakeholders provided suggestions to further improve the visualization of input and results (Müller et al., 2019). The suggestions were followed and re-evaluated in the following workshops. The evaluations were performed by means of moderated exercises using different scenarios, and involving stakeholders with different backgrounds, who may be involved in decision making during a real nuclear emergency. The scenarios, data used for evaluation and evaluation results from the final workshop of CONFIDENCE are presented in other articles of this special issue (Charnock et al., 2020;Duranova et al., 2020aDuranova et al., , 2020b.

Conclusion
Within the framework of the CONFIDENCE project an existing MCDA tool was enhanced to cope with uncertainties in the input parameters. For this purpose, probabilistic MCDA and ensemble evaluations were developed. The uncertainties were defined as distribution functions or histograms and graphically displayed to stakeholders. The results were communicated as charts and textual report. The method was presented in stakeholder workshops where the stakeholders assessed the MCDA tool to be helpful in decision making under some boundary conditions. These may imply, for example, that there should be enough time to discuss preferences (which should be the case in the transition phase), and that the tool be operated by trained personal. T. Müller et al.: Radioprotection 2020, 55(HS1), S181-S185 S183