Different types of uncertainty in nuclear emergency management

This is anOpe Abstract – The objective of the CONFIDENCE project has been to identify, assess and communicate the uncertainties that arise in managing and recovering from a nuclear accident. To do that, it is important to be clear on what uncertainty means, how it arises and how we might analyse it. In fact, there are many forms of uncertainty, some of which we can assess with current tools, but some of which are more nebulous and difficult to deal with. Nonetheless, all need to be communicated to the emergency managers. Starting with a review of different categories of uncertainties, and using examples from the CONFIDENCE project, this paper discusses how the various uncertainties are manifested in nuclear emergency management. It concludes with a simple framework for categorising and analysing them. The overall intention is that such a framework could contribute to a broader discussion on identifying and handling uncertainties with nuclear emergency management actors


Introduction
Emergencies inevitably involve uncertainties; threatened or actual offsite releases of radiation from nuclear facilities are no exceptions. In the threat and early release phases, the source term, its strength, time profile and composition are hugely uncertain (Mathieu et al., 2018a). Meteorological and hydrological uncertainties further confound predictions of the dispersion of the contamination (Wellings et al., 2018). Additional uncertainties enter into dose predictions from external dose or via food chains, and the effectiveness of countermeasures and their implementation including public compliance increases the uncertainty further (Howard et al., 2005;Nisbet et al., 2005). However, these are only some of the uncertainties faced by emergency managers and recovery teams in responding to a nuclear accident. The modelling and analysis of dose, while reducing some of the uncertainties, introduces further ones related to the model choices and the computation (Haywood, 2010;Haywood et al., 2010). Then there are ambiguities and value uncertainties that arise when the managers try to contextualise emergency plans with their imperatives to minimise the risk to "human health" or some such objective to the specifics of the accident and those affected (French et al., 2016). In short, many uncertainties need to be addressed in the decision making. This paper builds on existing literature on uncertainty types (French, 1997;Snowden, 2002;Walker et al., 2003), and discusses various forms of uncertainty, how they arise and how we might analyse them in the context of emergency management. Examples from CONFIDENCE work are included as to contextualise the challenges and possible approaches to addressing different uncertainty types. We believe that we need to be clearer in our understanding and communication of uncertainties if we are to support emergency managers in their tasks and decision making.

Uncertainties in nuclear emergency management
The range of uncertainties that need be considered has been in focus for decades (French, 1997), but until recently in developing tools and procedures for nuclear emergencies, they have not been addressed in a comprehensive sense. Indeed, current decision procedures tend to ignore uncertainties and focus on an expected or reasonable worst case (French et al., 2016). The CONFIDENCE project had the overriding aim of identifying, addressing and communicating the myriad of uncertainties in a much more comprehensive and effective way.
There are many ways of categorising uncertainty (see, e.g., Berkeley and Humphreys, 1982;French, 1995;Snowden, 2002), and no real agreement on how to do so. This was also apparent during early discussions within the CONFIDENCE project. Some proposed that as scientists, we should use the term more tightly, restricting it to contexts in which probability modelling is appropriate. Recognising the many dimensions of uncertainty, as well as the fact that emergency managers and stakeholders use the term very widely (Perko et al., 2019), we adopted a broader but pragmatic approach. Drawing from existing literature on uncertainty categorisation, we briefly describe 9 types of uncertainty, recognising that neither is the list exhaustive nor the categories mutually exclusive.

Stochastic or aleatory uncertainty
Many uncertainties arise from randomness within many physical behaviours or natural variations in populations and are referred to as stochastic or aleatory uncertainties. In such situation, we have some knowledge about the potential outcomes, but we are not certain about which of these will be realised. Whether the world is truly random or whether it is so complex that the slightest variation in conditions can dramatically affect the outcome of a deterministic behaviour does not matter. What matters is that we cannot predict an outcome with certainty: we need probability. There is general agreement across the scientific and lay communities that probability models are the appropriate means of describing uncertain behaviours in physical systems (Barnett, 1999). Similarly, probability is the accepted way of modelling natural variation in a population, for instance when estimating cancer risk from radiation exposure (Keil and Richardson, 2018). Examples of such types of uncertainties can be found throughout the CONFIDENCE project, including meteorological conditions and dispersion models, impact of soil parameters on food-chain transfer, as well as the use of probability models within scenario based assessments (Duranova et al., 2020a(Duranova et al., , 2020bBeresford et al., 2020).

Actor uncertainty
Decision makers need to consider how other actors will behave. For instance, after a nuclear accident many uncertainties relate to the behaviour and the compliance of the local public. Within the CONFIDENCE project, examples included whether or not affected communities would follow the advice of authorities on countermeasure such as stable iodine intake or sheltering , as well as consumer trust in food from affected areas (Duranova et al., 2020a). It is possible to model human behaviour using probability models, which assume that over a population, variations in how people act can be described stochastically. Simulation models, including agent-based modelling, do this. So do many countermeasure models, perhaps forgetting the variation and just using averages. However, humans think, and their behaviour is not random. It is driven by their wants and desires. There are models of human behaviour that may be more useful than simply using a probability distribution to predict the actions that people take: e.g. prospect theory (Kahneman and Tversky, 1979;Mercer, 2005;Barberis, 2013). Currently, there are developments in adversarial risk analysis, which model interactions between individuals, allowing that each may adopt different levels of "rationality" (Banks et al., 2015), though such models need more investigation before they might be applied. The mental model approach applied within the CONFIDENCE project is another example . Ultimately, stakeholder engagement processes, both in the preparedness and in the emergency and recovery phase are essential to understand each actor's perspective, improve emergency and recovery plans and reach effective decisions.

Epistemic uncertainties
Some uncertainties relate to our lack of knowledge and have a different character to stochastic uncertainty. For instance, we may have a number of possible source term models but not know which represents the actual release better. Another example relates to health effects of low radiation doses: at present we do not have sufficient knowledge to estimate who among the exposed population will be affected and in which way. Such uncertainties are called epistemic. Statistical theory, which articulates the process of scientific inference or induction, has considered how epistemic uncertainty should be introduced and dealt with in analyses. Frequentist approaches, which once dominated statistical methods, eschew full quantification of epistemic uncertainty leaving the scientist to learn intuitively from the evidence displayed to them in the analyses through p values, confidence intervals and significance levels (Barnett, 1999). Bayesian approaches, based on quantifying epistemic uncertainty through probabilities (French and Rios Insua, 2000;Gelman et al., 2013), now dominate statistical thinking. Probability is taken as representing the uncertainty that an idealised rational person beginning with an agreed body of knowledge would hold in the light of the available empirical evidence. Bayesian methods now provide a coherent foundation to machine learning, decision modelling and artificial intelligence.
Sensitivity and uncertainty analysis have a role in exploring and assessing the implications of epistemic uncertainty for the support of specific decisions. If all plausible explanations and models predict similar outcomes of potential actions, any epistemic uncertainty will not be significant for that decision (French, 2003).
Some uncertainties, particularly epistemic ones, may be deep (French, 2015) or severe (Comes et al., 2011) in that we know too little to assess them or build a probability model of our uncertainty convincingly in the time available. Relevant data may be sparse and there may be little agreement among experts about neither what is happening nor how to model the behaviour. The types of uncertainty prevalent during the early phase of a nuclear emergency, especially related to the source term, timing of release and future meteorological conditions are well-recognised examples of epistemic uncertainties (Mathieu et al., 2018b), but also include behavioural factors (e.g. how many people will self-evacuate and which routes will they take; or will all people from the affected area have access to iodine tablets timely) or factors impacting on long-term consequences (deposition and dispersion, and health impacts) of accidents are other examples epistemic uncertainties (Duranova et al., 2020a).

Judgemental uncertainties
In any analysis, there are many judgements to be made: which model to use, what parameter values should be set, among other. There may be many candidates (e.g. Galmarini et al., 2008). For example, there are many atmospheric dispersion models with many parameters, and little agreement on which should be used. So choices have to be made, leading to judgemental uncertainty. In some cases, this may be assessed using Monte Carlo methods (Evans and Olson, 2002) or a variety of more deterministic sensitivity analyses (Saltelli et al., 2000a(Saltelli et al., , 2000bFrench, 2003;Saltelli et al., 2004). Model intercomparison studies, such as the scenario comparisons carried out in WP1 of the CONFIDENCE project are very useful too, providing examples of the impact of parameter choice on the predicted consequences during early release phases such as the evolution of plumes and differences in prognosis frequency maps for countermeasures in affected regions (Hamburger et al., 2020;Duranova et al., 2020a).

Computational uncertainties
The algorithms that are used to analyse models and evaluate their predictions are not perfect. Computer codes are developed using approximations, iterations and enormous numbers of arithmetical calculations based on finite mathematics, inevitably introducing errors: e.g. computational uncertainties. We may find that the computations are intractable in feasible time, so further approximations may be introduced to increase speed and hence computational uncertainty. Statistical emulation may be used to fit a complex model with a much simpler one, a sort of functional regression (Craig et al., 2001;O'Hagan, 2006;Conti et al., 2009;Goldstein, 2011), again increasing computational uncertainty. Numerical analysis provides bounds on computational errors in specific calculations; emulation algorithms provide some assessment of their own accuracy. Recently Hennig et al. (2015) have promoted probabilistic techniques for representing overall computational uncertainty.

Model uncertainty
However good the model and computations, the outputs will not fit the real world perfectly: the only true model of reality is reality itself. For instance, many radioecological models use empirical transfer factors to estimate soil-to-plant transfer of radionuclides and do not fully account for the variation in root uptake caused by variation in soil properties (Almahayni et al., 2019). Over the years, attempts have been made to model the gap between a model and reality (Draper, 1995;Goldstein and Rougier, 2009;O'Hagan, 2012). But the task, though informative in understanding the process of modelling, is fruitless, creating an infinite regress of models modelling errors of modelling error models. In some cases, modelling error may be significant, the model only giving broad indications behaviour. The cited papers provide some techniques to allow for modelling error. However, in using models for prediction, one has to rely on the user's experience and tests made on existing data to allow for "how good the model is" (Kuhn, 1961).
There are considerable overlaps between judgemental, computational and modelling uncertainty. We did warn that our categories were not mutually exclusive. The important point is that modelling and analysis introduce uncertainties over and above stochastic, actor and epistemic uncertainties.

Uncertainties resulting from ambiguity and lack of clarity
While judgemental, computational, model, actor and even stochastic uncertainties can be considered specific cases of epistemic uncertainty, since they relate to a lack of knowledge, ambiguity and lack of clarity are entirely different. They relate to a lack of clear understanding about what is meant by some wording or certain visual information. For instance, what is the meaning of maximal permitted levels of radiation in food: does this mean that food with radioactivity below these levels is safe while food above is unsafe? (Charron et al., 2016). Other uncertainties of this type relate to the description of a consequence (French et al., 2017). Some researchers have suggested modelling such uncertainty with fuzzy concepts (Kacprzyk and Zadrozny, 2010), especially in natural language processing. For decision-making, however, fuzzy methods are not an appropriate way forward (French, 1995). When making decisions, we do not need a model of some ambiguity or lack of clarity; rather we need to think more deeply about what we mean and resolve any lack of clarity by conscious deliberation. A common approach to this is via facilitated workshops in which the facilitator continually challenges participants to explore and define much more clearly what is meant by phrases such as "health effects" (Eden and Ackermann, 1998;O'Brian and Dyson, 2007;French et al., 2009). Resolving ambiguity and lack of clarity in modelling and analysis invariably requires value judgements. Indeed, it requires value-focused thinking Keeney (1992) in that we need to think about why we need to undertake these. What do we want to learn from the modelling and analysis that we are about to undertake? Work within CONFIDENCE on different ways of calculating and presenting radiation induced health effects provide a good example of such ambiguities (Walsh et al., 2019), also highlighted in discussions at the CONFIDENCE workshop of how this might be communicated to the public (Duranova et al., 2020a).

Value, social and ethical uncertainties
Some uncertainties relate to questions that require value judgements in their resolution: e.g. how much to trade-off a reduction in radiation exposure for an increase in cost, or at what dose levels countermeasures should be initiated. These differ from the judgmental uncertainties discussed above in that they do not relate to the choice of parameters used in models, but the decisions and choices made based on the results of models and predictions. Such value uncertainties stem mainly from value pluralism: an emergency affects different stakeholders in different ways, and these stakeholders may have different priorities. They are, however, important enough to consider separately from ambiguity and lack of clarity. Even if the science base were undisputable, the values underlying the priorities of different stakeholders may be conflict with each other. For instance, the authorities may set their priority reducing the doses to population, below a certain threshold, while for affected population the socio-economic revitalisation of the area may be at least as important. Again, modelling such uncertainties is unhelpful; they need to be resolved by thoughtful deliberation, perhaps supported by sensitivity analysis since precision is irrelevant when there is no effect on the ultimate choice. Value uncertainties introduce social responsibilities and ethical concerns, particularly when acting on behalf of stakeholders. Social uncertainties in how expert recommendations are implemented in society may refer to public acceptance and compliance with protective actions advice; social and economic consequences of the recommendation and actions, and uncertainties in those consequences; and the level of stakeholder and public engagement used or planned (Turcanu et al., 2020b). Ethical uncertainties also refer to whether members of a population feel that they have control over or have given consent to being exposed to a particular level of risk and the need to be sensitive to inequalities in the distribution of risk (Oughton et al., 2004;Tomkiv et al., 2020). Again debate and deliberation, when possible with the engagement of affected stakeholders, are the only useful way forward (Nisbet et al., 2010).

Uncertainty about the depth of modelling
When decisions are taken, a further uncertainty arises: are the analyses sufficient to justify the actions being taken? Is it requisite, e.g. "good enough"? Such uncertainty again can only be resolved by judgement and deliberation (Phillips, 1984;French et al., 2009); although in the emergency itself, the need to make timely decisions may supersede this. Discussions on the robustness of different food-chain models carried out within CONFIDENCE (Beresford et al., 2020) illustrate the importance of having this deliberation prior to an actual emergency.
A related issue here is how confident the decision makers feel when taking the decision. The analyses and results presented to emergency managers and recovery teams are based on long and complex model chains. Considerable judgemental, computational and model uncertainties may have accumulated in developing the analysis and results. Such uncertainties, as we have indicated, are extremely difficult to quantify and will not be fully represented in any uncertainty bounds and plots produced. Yet they need to be communicated to the decision makers so that they have as full as possible understanding of the total uncertainty that they face. Within CONFIDENCE, robustness indicators have been developed which reflect the uncertainty level of different components and inputs to the model chain (Nagy et al., 2020).

Conclusions and broader groupings
We emphasise again that the list of uncertainties above is only one possible categorisation and that we make no claim that it is comprehensive nor that the categories are nonoverlapping. We have, however, found it useful in discussing how to handle uncertainties in nuclear emergency management. In Table 1, we organise these types of uncertainty into three groups. The first group relates to our knowledge of the external world; it might be called scientific uncertainty. The second relate to uncertainties and errors that are introduced by the models and techniques that are used to analyse the risks and possible interventions in an emergency. The third relates to uncertainties that the emergency managers, experts and stakeholders hold about themselves, their values and responsibilities and how to account for these differences in decision-making. In our experience, in the past most attention and discussion has focused on the first and some of the third group of uncertainties, but much less attention has been given to the second. The CONFIDENCE project made an important contribution to increasing reflection on, and communication of, the many different types of uncertainty inherent within nuclear emergency management, that to date had been poorly communicated to the decision-making process. However, this is only a start and there is a need for further discussions of the different types of uncertainties, and how to handle them within decision-making, perhaps with the aid of MCDA within stakeholder engagement workshops (Hiete et al., 2010). Both the categories and the groupings can act as a check-list for the analysts to be sure that they have analysed and communicated all major uncertainties involved in predicting what might happen. Finally, extending the list of uncertainties beyond the purely technical is an attempt to adopt rather than correct the language used by actual emergency managers, as well as to recognise the real challenges faced by decision-makers. Acknowledgement. The work described in this paper was conducted within the CONFIDENCE project, which was part of the CONCERT project. This project has received funding from the Euratom research and training programme 2014-2018 under grant agreement No 662287. Disclaimer (Art. 29.5 GA). This publication reflects only the author's view. Responsibility for the information and views expressed therein lies entirely with the authors. The European Commission is not responsible or any use that may be made of the information it contains.