New insights on model evaluation inspired by the stochastic simulation paradigm

A. Efstratiadis, New insights on model evaluation inspired by the stochastic simulation paradigm, European Geosciences Union General Assembly 2011, Geophysical Research Abstracts, Vol. 13, Vienna, 1852, European Geosciences Union, 2011.



The working paradigm for evaluating the performance of practically any kind of mathematical model is based on metrics that assess an “average” departure between modelled outputs and observations (i.e. residuals). Yet, the outputs of hydrological, hydrogeological and climatic models are not deterministic responses against known or predictable inputs; they are stochastic variables, the interpretation of which should, consequently, be implemented in statistical terms. In addition, these processes exhibit multiple peculiarities (seasonality, long-term persistence, intermittency, skewness, spatial variability), which are rather impossible to be accounted for within a single measure (typically efficiency or other least square error expression). In this context, a comprehensive statistical framework is discussed for the evaluation of such models, seeking for the reproduction of a number of statistical characteristics of the observed data, instead of focusing to optimize an “overall” distance measure. This is inspired by the requirements of advanced stochastic simulation schemes, which are by definition built to preserve the essential statistics of the parent (i.e. historical) time series (marginal and joint statistics). This is a key concept, ensuring the generation of synthetic data that are statistically equivalent to the historical ones. The proposed framework emphasises the following issues: (a) the statistical comparison of computed and observed data at multiple time scales, to account for the variability of the modelled processes in both the short and the long term; (b) the preservation of the observed cross-correlations in multi-response calibration, to represent the interrelationship of the physical processes under study, and (c) the investigation of the model response under different stress conditions, preferably using synthetic data of appropriate length; this allows recognising structural deficiencies and irregular behaviours, which are hard to identify within the, typically short, period of observations. The above issues are analysed using examples from a number of modelling works, where initial calibration approaches, following typical hydrological practices, may result in misleading conclusions.

PDF Full text:

Our works that reference this work:

1. A. Efstratiadis, I. Nalbantis, and D. Koutsoyiannis, Hydrological modelling of temporally-varying catchments: Facets of change and the value of information, Hydrological Sciences Journal, 60 (7-8), 1438–1461, doi:10.1080/02626667.2014.982123, 2015.
2. A. Efstratiadis, I. Tsoukalas, and P. Kossieris, Improving hydrological model identifiability by driving calibration with stochastic inputs, Advances in Hydroinformatics: Machine Learning and Optimization for Water Resources, edited by G. A. Corzo Perez and D. P. Solomatine, doi:10.1002/9781119639268.ch2, American Geophysical Union, 2024.