AMSEEAS: Adaptive hybrid scheme of machine learning, evolutionary algorithms and annealing-simplex for objective function optimization

The purpose of this thesis is to develop global optimization algorithms, designed for problems of high computational requirements. At first, the theory of nonlinear optimization is reviewed and its most widely used methods are briefly described. This is followed by the extensive review of two specific algorithms, the Evolutionary Annealing-Simplex (EAS) and its upgrade version, namely the Surrogate-Enhanced Evolutionary Annealing-Simplex (SEEAS). The latter incorporates a metamodel in the core of EAS, resulting in speeding up the surface response exploration procedure. Afterwards, an introduction to the machine learning theory is made and its major elements are described, which can be integrated into the optimization procedure, thus taking the role of surrogate models. Key component of the study is the development of an original code, namely the Adaptive Multi-Surrogate Enhanced Evolutionary Annealing-Simplex (AMSEEAS), an improved version of SEEAS. The basic idea introduced lies in the use of multiple metamodels, simultaneously integrated in the same algorithm, that manage to coexist harmoniously and cooperate within a group. The new code is extensively compared to other popular algorithms, via testing them on many challenging mathematical functions, in which the global optima estimation procedure is significantly complicated. The performance of AMSEEAS is also evaluated against a particularly demanding problem of the water resources field, namely the stochastic calibration of a hydrological model. Long synthetic time series are used in the problem instead of historical ones, thus significantly increasing the computational burden of the simulation. The results of the above analysis confirm the success of the idea proposed and highlight the potential for its further development. The three main algorithms (EAS, SEEAS, AMSEEAS) have been developed in Python programming language and are publicly available.