Data-Driven Machine Learning Aided Fine Scale Reconstruction for Hierarchical Parameter Systems and the Geometry of Adaptive Learners

Abstract

While statistical uncertainty is observable in many real-world systems and data, its incorporation into numerical prediction and optimization algorithms presents numerous challenges in terms of accuracy, efficiency and storage costs. In this dissertation, we make two contributions to numerical algorithms that account for uncertainties.

In the first part, we discuss the statistical simulation of a known physical system with spatially varying random model parameters. The resulting ensembles of model outputs can be used to determine statistical properties, such as means and confidence bounds, of related quantities of interest. In these simulations, the complexity (dimensionality) of the system parameter's statistical description, which often reflects the spatial scale at which the parameters are resolved, may affect both the statistical accuracy of resulting outputs and the spatial accuracy of the model solution, especially when the parameter fields are rough. In particular, very low-complexity models, while amenable to efficient sampling methods, may introduce a statistical modeling error by underestimating the true variation in the output. On the other hand, high-dimensional parameter representations, while more detailed, may result in model responses that require a higher resolution computational mesh, thereby increasing the computational cost. We propose a method to generate high-complexity samples conditional from low-complexity ones by means of a conditional variational autoencoder.

In the second part of the dissertation, we propose GeoAdaler, a new stochastic optimization algorithm, whose step length selection is based on a scalar measure of steepness of the objective function's sample gradient, based on the angle of its normal vector with the horizontal plane. We show how the resulting family of algorithms relate to existing adaptive steplength selection techniques and establish regret bounds in the case of convex functions. We also demonstrate numerically that GeoAdaLer is capable of being extended to nonconvex settings.