A comparison of sensitivity analysis techniques for complex models for environment management

Abstract

Computer based modelling methods are being used increasingly to replicate natural systems in order to review both large and small scale policy measures prior to their implementation. Integrated Assessment Modelling (IAM) incorporates knowledge from several different disciplines into one model in order to provide an overarching assessment of the impact of different management decisions. The importance of IAM is that the environmental, social and economic impacts of management choices can be assessed within a single model, further allowing assessment in relation to sustainability criteria. The considerable detail facilitated by these models often requires the inclusion of a large number of parameters and model inputs, many of whose values may not be known with certainty. For this reason and because models do not always behave intuitively (in particular when there are nonlinearities involved), sensitivity analysis (SA) of the model to changes in its parameters and inputs is an important stage of model development. Current SA methods have not kept pace with rapid increases in computing power and availability and more importantly the resultant increases in model size and complexity. Also related to the complexity is increased difficulty in finding and fitting distributions to all parameters. Further, the complex nature of integrated models requires SA that is flexible and can be implemented regardless of model structure. This research aims to establish new criteria for SA used in the context of integrated models for environmental management and decision-making. These criteria are believed to reflect the current requirements specific to this type of modelling. Desirable criteria are identified as: high computational efficiency; ability to take into account higher order parameter interactions; ability to account for model non-linearities; not requiring knowledge of parameter probability distributions; and use in decision making. SA of an integrated model of the Namoi River catchment is performed using the Fourier Amplitude Sensitivity Testing (FAST) method, Morris method, method of Sobol’, and regression and correlation coefficients. The results from these analyses are used as a basis for comparing the SA methods by the new criteria outlined above. The Namoi model is a combination of a flow model with a non-linear component, a policy model, an economic model and an extraction model. It can be used for assessing management options for the river. SA of two different potential management options for the catchment is undertaken to facilitate comparison of sensitivity between two slightly different models. Comparison of the different SA methods shows that none of the methods meet all of the criteria and, in particular, there are no methods that are effective for use when comparing management options. This lack of an adequate SA method for integrated models indicates that development of a new method of SA specifically for integrated models for environmental management is desirable. The FAST method is shown to meet the criteria most effectively, being able to account for model non-linearity and non-monotonicity, requiring only parameter ranges (not distributions), and being relatively computationally efficient (although this does come at a loss of some resolution). Results from the FAST SA of the Namoi model show the model to be sensitive to several parameters within the non-linear loss module. Further, one management option shows sensitivity to the decision variables within the model while the other does not. This means that the first management option clearly corresponds to the more controllable form of the model.