List of methods for data analysis

For organizing the different data analysis techniques, we make use of a Venn diagram. The different categories we use are based on the four aspects of the data, studied in the exploratory data analysis step: The requirement for multivariate methods, the presence of spatial structure, the presence of trends, and the requirement for robust methods. The methods that are able to handle two, three or all aspects, are listed in the corresponding intersections. It is also possible none of these aspects apply, in which case we list the method outside of the diagram. In case it would not be clear which method to use, or if multiple methods are mentioned in the relevant section of the diagram, it is recommended to go through the brief introductions first. If it would still be unclear what method to use, further reading is recommended, or the consultation of a more experienced person.

Validation techniques for assessing the results obtained

Cross-validation can be used to assessing how the results of a statistical analysis will generalize to an independent data set. The model is initially fit on a training dataset, which is a set of values used to fit the parameters of the model. Successively, the fitted model is used to predict the responses for the observations in a second dataset called the validation dataset. This hold back sample of the full data should be used to give an unbiased estimate of the skill of the final tuned model by comparing or selecting between final models. A dataset can be repeatedly split into a training dataset and a validation dataset. Cross-validation combines (averages) measures of fitness in prediction to derive a more accurate estimate of model prediction performance. We distinguish two types of cross-validation: exhaustive and non-exhaustive cross-validation. Exhaustive cross-validation methods are cross-validation methods which learn and test on all possible ways to divide the original sample into a training and a validation set. Non-exhaustive cross validation methods do not compute all ways of splitting the original sample.

Attention should be paid to the re-estimation of extreme values as they can significantly bias the cross validation in the case of skewed distribution. As a consequence, a model can be preferred globally from cross-validation by better honouring the extreme values whereas the characterisation objective focuses on a low threshold where the model is not satisfactory.

Uncertainty & sensitivity analysis

The final outcome is anyhow affected by a certain degree of uncertainty, which often strongly impacts the decision-making. Quantifying the uncertainty on the variables of interest is usually a multi-dimensional and therefore complex task. Ideally, a Bayesian inference approach would be preferential compared to Monte Carlo error propagation or an approach based on first-order Taylor expansion. The latter might suffer from important drawbacks related to non-Gaussian distributions and non-linear expressions, or the omission of systematic uncertainties. Although applying a Bayesian inference approach or even a Monte Carlo error propagation would be the best technical choice, it might not be particularly suitable due to its complexity and the individual character of each model and case. For a one-time case, the following alternative options could be considered:

• A qualitative description, examining the most important and uncertain assumptions without performing a quantitative analysis.
• A scenario comparison, comparing two or more alternative scenarios (i.e. best case & worst case scenario).
• A once-at-a-time approach, introducing one at a time changes (i.e. minimum and maximum values for certain parameters) and comparing with a given baseline.

In addition, sensitivity analysis is a valuable instrument, which enriches the quantitative analysis of impact with a deeper investigation and identification of the sources of uncertainty. Among the most relevant techniques:

• Screening methods (Morris, 1991; Campolongo et al., 2011).
• Non-parametric or regression-based approaches (Saltelli & Marivoet, 1990; Helton, 1993).
• Variance-based methods (Sobol’, 1993; Saltelli et al., 2008; Iman & Hora, 1990; Sacks et al., 1989).
• Spectral methods (Cukier et al., 1973; Saltelli et al., 1999; Sudret, 2008; Shao et al., 2017), and
• Moment-independent importance measures (Park & Ahn, 1994; Plischke et al.,2013).

In particular, variance- based methods related to Sobol’s sensitivity indices are the most popular methods among practitioners due to their versatility and easiness of interpretation.

Uncertainty and sensitivity analysis are crucial as they help identifying the factors (assumptions, variables, data, and uncertainties) at play and provide information on their influence in quantitatively driving the impacts of the various decision options. In particular in view of nuclear site characterization:

• The impact of the sampling design is most fundamental (see section on sampling design).
• Using advanced statistical methods for dealing with samples below detection limit can have an important (beneficial) impact on estimated values and should be considered in case the data set contains a considerable amount of results below detection limit (see section on overall strategy). Obviously, the potential impact depends on the objective (estimated quantity).
• The following observed effects could be occasionally generalised as well, but might be specifically related to of specific use cases that have been tested for the validation of this guide (see section 6; Implementation and validation of the strategy in the following four use cases).
  • In the case of small data sets, the presence of outliers could clearly increase the uncertainty. Additional verification (process driven or error) might be necessary when estimations are becoming close to a decision threshold.
  • Sensitivity analysis shows that:
    • Most important uncertainties for the physical parameters estimated (e.g. volume categorisation, total activity) are due to spatial uncertainty (geostatistical simulations) and heteroscedasticity. Heteroscedasticity occurs more often in datasets that have a large range between the largest and smallest observed values, which is typically the case for radiological data.
    • The impact of sample measurement uncertainty on the volume categorisation is quite limited (if this uncertainty does not come from a systematic bias).
    • The effect on the estimated quantity (e.g. total activity or threshold) is not always the same as on the uncertainty.
    • The integration of non-destructive measurements in a multivariate approach significantly reduces uncertainties when sampling density is reduced.

Next