Uncertainty analysis methods for multi-criteria decision analysis.

Abstract

Planning, design and operational decisions are made under complex circumstances of multiple objectives, conflicting interests and participation of multiple stakeholders. Selection of alternatives can be performed by means of traditional economics-based methods, such as benefit-cost analysis. Alternatively, analyses of decision problems, including water resource allocation problems, which involve trade-offs among multiple criteria, can be undertaken using multi-criteria decision analysis (MCDA). MCDA is used to assist decision makers (DMs) in prioritising or selecting one or more alternatives from a finite set of available alternatives with respect to multiple, usually conflicting, criteria. In the majority of decision problems, MCDA is complicated by input parameters that are uncertain and evaluation methods that involve different assumptions. Consequently, one of the main difficulties in applying MCDA and analysing the resultant ranking of the alternatives is the uncertainty in the input parameter values (i.e. criteria weights (CWs) and criteria performance values (PVs)). Analysing the sensitivity of decisions to various input parameter values is, therefore, an integral requirement of the decision analysis process. However, existing sensitivity analysis methods have numerous limitations when applied to MCDA, including only incorporating the uncertainty in the CWs, only varying one input parameter at a time and only being applicable to specific MCDA techniques. As part of this research, two novel uncertainty analysis approaches for MCDA are developed, including a distance-based method and a reliability based approach, which enable the DM to examine the robustness of the ranking of the alternatives. Both of the proposed methods require deterministic MCDA to be undertaken in the first instance to obtain an initial ranking of the alternatives. The purpose of the distance-based uncertainty analysis method is to determine the minimum modification of the input parameters that is required to alter the total values of two selected alternatives such that rank equivalence occurs. The most critical criteria for rank reversal to occur are also able to be identified based on the results of the distance-based approach. The proposed stochastic method involves defining the uncertainty in the input values using probability distributions, performing a reliability analysis by Monte Carlo Simulation and undertaking a significance analysis using the Spearman Rank Correlation Coefficient. The outcomes of the stochastic uncertainty analysis approach include a distribution of the total values of each alternative based upon the expected range of input parameter values. The uncertainty analysis methods are implemented using a software program developed as part of this research, which may assist in negotiating sustainable decisions while fostering a collaborative learning process between DMs, experts and the community. The two uncertainty analysis approaches overcome the limitations of the existing sensitivity analysis methods by being applicable to multiple MCDA techniques, incorporating uncertainty in all of the input parameters simultaneously, identifying the most critical criteria to the ranking of the alternatives and enabling all actors preference values to be incorporated in the analysis. Five publications in refereed international journals have emerged from this research, which constitute the core of the thesis (i.e. PhD by Publication). The publications highlight how uncertainty in all of the input parameters can be adequately considered in the MCDA process using the proposed uncertainty analysis approaches. The methodologies presented in the publications are demonstrated using a range of case studies from the literature, which illustrate the additional information that is able to be provided to the DM by utilising these techniques. Publications 1 and 2 (Journal of Environmental Management and European Journal of Operational Research) demonstrate the benefits of the distance-based uncertainty analysis approach compared to the existing deterministic sensitivity analysis methods. In addition, the benefits of incorporating all of the input parameters in the uncertainty analysis, as opposed to only the CWs, are illustrated. The differences between global and non-global optimisation methods are also discussed. Publications 3 and 4 (Journal of Water Resources Planning and Management and Journal of Multi-Criteria Decision Analysis) present the stochastic uncertainty analysis approach and illustrate its use with two MCDA techniques (WSM and PROMETHEE). Publication 5 (Environmental Modelling & Software) introduces the software program developed as part of this research, which implements the uncertainty analysis approaches presented in the previous publications. Despite the benefits of the approaches presented in the publications, some limitations have been identified and are discussed in the thesis. Based on these limitations, it is recommended that the focus for further research be on developing the uncertainty analysis methods proposed (and in particular the program, and extension of the program) so that it includes additional MCDA techniques and optimisation methods. More work is also required to be undertaken on the Genetic Algorithm optimisation method in the distance-based uncertainty analysis approach, in order to simplify the specification of input parameters by decision analysts and DMs.