# Questions tagged [bayesian-optimization]

Bayesian optimization is a family of global optimization methods which use information about previously-computed values of the function to make inference about which function values are plausibly optima. Its applications include computer experiments and hyper-parameter optimization in some machine learning models.

192 questions
Filter by
Sorted by
Tagged with
15 views

### Understanding Bayesian Optimal Experiment Design

I read this tutorial on Bayesian Experimentation Design (https://pyro.ai/examples/working_memory.html) and I'm trying to wrap my head around it. Suppose you have data (X,y). You're thinking about ...
12 views

• 9,421
1 vote
58 views

### How to apply Bayesian Optimization to a function depending only on categorical variables?

I would like to apply Bayesian Optimization (BO) to a black-box function depending only on multiple categorical variables. In my application, each categorical variable has 3 possible categories. I ...
182 views

### Expected Improvement (EI) in Bayesian Optimization failed to converge

I am not sure if "converge" is a proper description of my problem. I'll state it in detail below. I'm working on a CFD problem which means each sample is expensive. The framework I choose to ...
• 43
1 vote
111 views

### Minimum sampling for maximising the prediction accuracy [closed]

Suppose that I'm training a machine learning model to predict people's age by a picture of their faces. Lets say that I have a dataset of people from 1 year olds to 100 year olds. But I want to choose ...
• 73
1 vote
44 views

### How to solve this type of multi-task Bayesian optimization problem?

Let us consider a collection of local Bayesian optimization tasks, each employs a Gaussian Process model to find the local optimum (i.e. global optimum of that task). The goal is to design a ...
• 183
23 views

### How to evaluate likelihood in MCMC for arbitrarily shaped distributions? [closed]

I'm very confused with the use of MCMC to estimate distributions that have a complex shape, like multiple peaks, or that aren't generated from a known distribution. In particular, when calculating the ...
196 views

### What is global concavity of the (log-)likelihood worth in Bayesian estimation?

In maximum likelihood estimation there is a big emphasis on finding the global maximum, which is why likelihood functions that are provably globally (log-)concave are desirable (despite often being ...
• 1,352
57 views

### Is it correct to replace the oldest data when using Gaussian Process in blackbox hyper-parameter optimization?

I try to apply GPR in a blackbox HPO question. My input will have 6 dimensions like X=[x1,...x6]. The implementation is quite straightforward with sklearn with a ...
• 145
2k views

### How to run Bayesian optimization experiments in parallel?

Suppose I have the following hyperparameters for tuning: learning_rate: [0.00001, 0.1] epochs: [200,300,400,....,1000] batch_size: [16,32,64,128] If I want to run experiments using 4 parallel jobs, ...
• 1,007
47 views

### How much reduction of hyperparameter experiments can I get using Bayesian optimization vs Grid search?

I have 5 hyperparameters for tuning and the number of combinations of all possible values is 9,360. This means if I want to find the optimal parameter setting using Grid Search, I need to do 9,360 ...
• 1,007
321 views

### Why is it desirable to standardize the inputs of a Gaussian Process Regression?

I read this question (Should we standardize the data while doing Gaussian process regression?) and wondered, why do we need to normalize the inputs of a Gaussian Process? In my case, I want to use ...
24 views

### Clarification on Bayesian ensembling

I was reading a paper https://arxiv.org/pdf/2007.06823.pdf and at the end of page 3 the author presents the technique called "ensembling" for the estimation of the expected outputs and the ...
• 325
426 views

### Is it possible to pick more than 1 sample point in each iteration of bayesian optimisation?

I want to use Bayesian optimisation for my project and I plan to build a closed-loop system, such that there is a model, robot to conduct experiments, measurement of experimental data which updates ...
• 81
165 views

### How does removal of symmetry (e.g. via constraints) in a Bayesian optimization search space affect search efficiency?

There are many examples of search space symmetry in real-world optimization problems in the physical sciences. To motivate this, here are some that come to mind: When optimizing a formulation such as ...
• 69
666 views

### Package for hyperparameter optimization with categorical values

Context I'm trying to solve a black-box optimization problem, and I can "reformulate" parts of the problem is different ways that may lead to lower or higher costs, and which can interact ...
• 89
1 vote
73 views

### Use of Monte Carlo Tree Search

I was talking with someone much more experienced in stats than I am and they suggested the use of Monte Carlo Tree Search for a problem I am facing. Problem Statement: I am collecting jitter ...
199 views

### What is the difference between Bayesian Optimizaiton for hyperparameters and using validation set when training?

While studying hyperparameter tuning in Machine Learning, I have come to read Bayesian Optimization for Hyperparameter Tuning and Using validation set when training the model but it is kind of ...
• 3,535
497 views

### Bayesian optimization resulting in overfitting despite split test train

I am using XGBRegressor on a time series to predict the next periods value. I use sklearns TimeSeriesSplit to split the data into test and train. I use skopts BayesSearchCV to search through a ...
346 views

### Is the result of Bayesian Inference with MCMC reliable since there maybe a big variance?

I'm new to Bayesian inference and I came across a, maybe simple, question. That is, in many cases, we don't know how the posterior distribution really looks like, and we adapt the result of MCMC, or ...
• 131