Bayesian optimization is a powerful tool used to optimize complex systems, such as machine learning models. It involves using Bayesian methods to model the objective function and then use this model to guide the search for the optimal solution. In this article, we review recent advances in Bayesian optimization and how it can be used to automate the process of optimizing complex systems.
One challenge with Bayesian optimization is that it requires a lot of expertise and computational resources to implement correctly. To address this challenge, we propose a new method called SnAKe (Sequential Approximate Bayesian Optimization with Kernel Evaluation). SnAKe is an efficient and flexible algorithm that can be used to optimize complex systems in a variety of fields, including machine learning and energy applications.
SnAKe works by creating a batch of points using Thompson Sampling, which is a way of modeling the objective function using a probability distribution. The points are then evaluated for their cost, and the ones with the lowest cost are selected for the next batch. This process continues until the optimal solution is found or a stopping criterion is reached.
To make SnAKe more efficient, we propose several improvements over traditional Bayesian optimization methods. These include:
- Using random scalarization to reduce the computational complexity of Thompson Sampling
- Implementing multi-objective optimization to consider multiple objectives simultaneously
- Using active exploration to focus the search on the most promising areas of the objective function
We demonstrate the effectiveness of SnAKe using several examples, including energy applications and machine learning. Our results show that SnAKe can find the optimal solution more efficiently than traditional Bayesian optimization methods while also providing a good trade-off between exploration and exploitation.
In summary, this article proposes a new efficient and flexible algorithm called SnAKe for Bayesian optimization. SnAKe can be used to automate the process of optimizing complex systems in various fields, including machine learning and energy applications. By using random scalarization, multi-objective optimization, and active exploration, SnAKe can find the optimal solution more efficiently than traditional methods while also providing a good trade-off between exploration and exploitation.
