Carolin Benjamins
PhD at Leibniz University Hannover
Our goal is to progress on Dynamic Algorithm Configuration (DAC) for Bayesian Optimization (BO). BO is a widely-used and sample-efficient framework for optimizing black-box problems, which are often expensive to evaluate. Dynamically configuring BO enables to adapt to the optimization progress and to any problem landscape without prior knowledge, which is especially interesting for application domains. An inherent question to BO is how to determine the exploration-exploitation trade-off which needs to be adapted to the problem at hand. Prior work showed that steering this trade-off (dynamically) via the acquisition function improves performance and sample-efficiency. However, the exploration-exploitation trade-off begins right at the start of BO with the initial design which covers the search space and is explorative by nature. So far, only heuristics for setting the size of the initial design exist. We hypothesize the optimal size depends on the problem and thus propose to smartly switch from the initial design to the surrogate-based optimization on a per-problem basis. This research project aims to shed light onto the optimization dynamics and to provide principled methods to configure BO. This will eventually lead to better optimization performance that can be applied to a wide range of applications, including hyperparameter optimization, robotics, material design and logistics.
Keywords: Hyperparameter Optimization, Bayesian Optimization, Dynamic Algorithm Configuration
Scientific area: Automated Machine Learning
Bio: PhD Student @ Leibniz University Hannover. I love automation and making complex algorithms more accessible!
Visiting period: March 2024 – April 2024 at RWTH Aachen