Questions tagged [bayesian-optimization]

Bayesian optimization is a family of global optimization methods which use information about previously-computed values of the function to make inference about which function values are plausibly optima. Its applications include computer experiments and hyper-parameter optimization in some machine learning models.

0
votes
1answer
17 views

How to Derive SISO Kalman Filter Update Equation Using only Probability Density Functions

I'm trying to prove to myself that a single state/single measurement kalman update can be derived using bayes theorem (as proof of concept for a more complicated task) only using the PDF. I am able ...
0
votes
0answers
14 views

Multi-armed Bandit Algorithm selection and Optimization

I have 2 channels that I can sent my products, the A channel cost 0.10\$ per product and the B costs 0.01\$ and I am trying dynamically to optimize the channel selection by minimize the cost. ...
0
votes
0answers
9 views

Method to optimise portfolio WITHOUT mean or variance

Can anyone suggest a method to optimise a portfolio of stocks with high volatility. I would like to use the expected return, traded volume and delta from Mid point. Have looked at MPT and the would ...
0
votes
1answer
65 views

Bayesian A/B testing with parameters other than success rate

If I have certain number of clicks and conversions for a group of ads, I can do Bayesian A/B testing following this method http://ucanalytics.com/blogs/bayesian-statistics-to-improve-ab-testing-...
4
votes
1answer
44 views

What is the relation between a surrogate function and an acquisition function?

A surrogate function is a simpler function than the objective function to evaluate. An acquisition function is used to propose sampling points. In the context of Bayesian optimisation and Gaussian ...
0
votes
0answers
17 views

How can I determine what values of alpha and kappa to use for Bayesian Optimization?

I'm using the pretty great Bayesian Optimization package for python. I have a very noisy function I'd like to optimize for a given hyperparameter. I've read a little on this, and it seems like if ...
0
votes
1answer
19 views

Bayesian optimization integer set constraint

Given an ordered input set of boolean values $S$ of length $|S|$, I want to minimize a function $f(S)$ while maximizing the number of $True \in S$ constrained by $|True \in S|$ $\ge$ $threshold$ ...
0
votes
0answers
18 views

BNLearn: How to merge the estimating parameters of a Gaussian Bayesian network with its conditional structure?

I define the structure of a gaussian baesian network usind " iamb" function and then estimated the coeficients of the nodes using "bn.fit". ...
0
votes
0answers
26 views

How does Gaussian process update after adding a new point to the model?

I am using a Gaussian Process model in the Bayesian Optimization setting. Concretely, a gaussian process is built on some initial $N$ points and the model is updated sequentially by evaluating a new ...
2
votes
1answer
21 views

Computing Sampling Cost Tradeoff

I'm working with a Bayesian Optimization problem where the accuracy of sampled data varies based on sampling cost (low-uncertainty data is expensive, high-uncertainty data is cheap). What kinds of ...
0
votes
0answers
27 views

Bayes Theorem in call center routing problem?

I want to route the call to the agent to maximize customer satisfaction. I have following data: Probability that customer is currently happy p(C+), we have this from sentiment analysis of the reviews/...
1
vote
1answer
22 views

How to tune hyperparameters/architecture of networks that are expensive to train?

What are some recommended ways to tune hyperparameters and/or develop domain-specific architectures for a large neural network model? That is, how to further tune a large neural network that already ...
2
votes
1answer
110 views

Learning prior distribution from data

Suppose I have a dataset. How can I learn the prior distributions of the parameters of a model from this data? I want to learn the prior from this data in order to use them in a Bayesian model. Sorry ...
0
votes
0answers
9 views

parameter boundary in LaplacesDemon (R library)

In LaplacesDemon (R library), I was wondering if there is a way to restrict the samples to come from a certain range. The model is set to have a prior calculated, but I don't think it is being ...
2
votes
1answer
131 views

If Bayesian optimization is used for hyperparameter tuning, do we still need to perform cross validation?

Given the hyperparameter tuning is sampled from the probability distribution, is it still necessary to perform cross validation?
0
votes
0answers
19 views

Are there kernel functions available for categorical variables where matches between different variables would also raise the similarity?

For my master thesis I have to apply bayesian optimization on the development of modular endolysins. This endolysin consists of 3 building blocks that are linked together (variables). Each of these ...
2
votes
0answers
89 views

Can Bayesian Optimization solve this problem?

Suppose ${\bf{x}} = (x_1,\ldots,x_n)$ and $f({\bf{x}})\propto 1_A({\bf{x}}) \prod_{i=1}^n {x_i}^{\alpha_i-1} e^{-\beta_i x_i}$ , i.e. $f$ is proportional to the product of independent gamma ...
1
vote
0answers
82 views

Personalize Recommendations for small dataset

I'm working on a recommender system for a set of niche products. These are products that don't have a large number of customers. Does anyone have any tips on algorithms or approaches that work well ...
1
vote
0answers
48 views

How to analytically solve the probability of improvement acquisition function in Bayesian Optimization with Vector inputs?

I have been using the probability of improvement acquisition function in my Bayesian Optimization program, but I've run into a problem because I am not optimizing the acquisition function that quickly....
0
votes
0answers
9 views

Does increasing the number of users improve BPR

I'm trying to improve recommendations for my recommender system using Bayesian personalized ranking. I've tried increasing the amount of seen and unseen items I'm training my model on for a small ...
1
vote
0answers
45 views

How do you use the predictive distribution with noise in Bayesian Optimization?

I have been reading a paper on Bayesian Optimization, and I was reading the section on adding Gaussian noise to your Gaussian process. The article is: Brochu, Cora and de Freitas (2010). A Tutorial ...
1
vote
1answer
1k views

How to use the squared exponential kernel with multidimensional vector inputs?

I'm constructing an optimization (Bayesian optimization) algorithm using Java code. I have created the program, but the similarity values between inputted vectors in the kernel equation does not ...
1
vote
1answer
194 views

MCMC vs Bayesian Optimization Efficiency for MAP estimate

I believe MCMC could be utilized to estimate the MAP. At least there is an option in packages like PyMC. I just started reading about Bayesian Optimization, but the first thing that hit me was that ...
0
votes
0answers
29 views

Very high/low values of threshold in Logistic Regression

I am training a few logistic regression models in pyspark. Since pyspark accepts threshold as an input in the fit method I ...
1
vote
0answers
60 views

Optimization: Approximate function - Which points to evaluate next?

I am looking for a statistical method (and a link to a nice R package would be cool too!) which allows me to find which point to evaluate next for a given function. I have a non-stochastic function z ...
2
votes
0answers
431 views

Gaussian Process Hyperparameter Tuning

I'm planning to use Gaussian Process (GP) to model my case. However, while learning the GP I found out that we have to tuning the hyperparameters to give us the best solution. I have checked several ...
2
votes
1answer
163 views

How to find the “right” hyperparameters for the Gaussian process used Bayesian optimization

Suppose we are using Gaussian process as a surrogate model in Bayesian optimization.To compute the acquisition function using Gaussian process we need to know the right hyperparameter of the Gaussian ...
6
votes
1answer
528 views

Why does Bayesian optimization work?

Bayesian optimization is used to optimize costly black-box functions. The idea is to use a surrogate model to model the black-box function and then an acquisition function is used to find the next ...
8
votes
1answer
167 views

Bayesian optimization for non-Gaussian noise

A black box function $f: \mathbb{R}^n \rightarrow \mathbb{R}$, which is evaluated pointwise subject to Gaussian noise, i.e., $f(x) + \mathcal{N}(\mu(x),\sigma(x)^2)$, can be minimized using Bayesian ...
1
vote
0answers
170 views

How to calculate RKHS norm of a function under given kernel transformation

This was a question asked before in mathoverflow but not yet got answered. I have the same problem when reading Srinivas et al (2010) [appendix B]'s paper. Here are my problems: Definitions: ...
1
vote
0answers
254 views

Gaussian Process UCB acquisition function: where do the constants come from?

I am trying to understand the GP-UCB acquisition function (Srinivas et al.) when applied to compact, convex spaces. The GP-UCB acquisition function is: $x_t = \mathrm{argmax}_{x \in D} \quad \mu_{t-...
0
votes
0answers
236 views

Do Bayesian Optimization GP-UCB algorithm always converged for any continuous function in theory or practice?

Recently,I am studying the paper of Gaussian Process Optimization in the Bandit Setting, Srinivas. In theorem 3, they state: Let $\delta\in(0,1)$. Assume that the true underlying f lies in the RKHS ...
0
votes
0answers
389 views

Tuning skopt to minimise correctly

I have a problem where I need to fit a pair of parameters in a model to best fit observed data for hundreds of instances. I need to determine these parameters to high accuracy, rather than just ...
3
votes
1answer
115 views

Are Matérn class kernels universal kernels or not?

This is a question that I can't find the solution. I don't know it is a open question or it is a well-known result that can be attained from several lemmas. Here are the definition of Matérn class ...
2
votes
1answer
39 views

How to infer the eigenvalue distribution from matrix where each entry has a known Gaussian distribution?

Problem Given $X \in \mathbb{R}^{n \times n}$ where $X_{ij} \sim \mathcal{N}(\mu_{ij}, \sigma_{ij}^2 I)$ Find the eigenvalue distribution using whatever you can. Background In my field, I have a ...
3
votes
1answer
187 views

Neural Network Tuning with Bayesian Optimisation - Number of layers and size of layers

I'd like to use Bayesian Optimization to tune the hyper parameters of a feed-forward neural network. Among these hyper parameters, there is the number of hidden layers in the network, as well as the ...
0
votes
1answer
41 views

best practices for baysian optimization of hyper parameters DNN [closed]

Many people suggest fine-tuning a network using Bayesian optimization ( or grid search or what every other black box optimization method you like ) so I tried it for my self. I am not sure about the ...
0
votes
0answers
109 views

Equivalent of using a Poisson prior in terms of a penalized regression?

I know that most penalized regressions have also a Bayesian interpretation, e.g. ridge least squares regression corresponds to the MAP estimate obtained under a Gaussian prior in a Bayesian regression,...
2
votes
1answer
137 views

Bayesian Inference for More Than Linear Regression

I learned about MCMC and variational inference for Bayesian inference, and I would like to try it out in some regression problem. However, all existing related models I know falls within either of the ...
2
votes
1answer
608 views

Bayesian Hyperparameter Tuning

I would like to implement Bayesian hyperparameter tuning in Python, but I am struggling to find a complete explanation of how it is exactly working. As anyone links, books or papers I could read about ...
0
votes
1answer
48 views

Simplification of delta mutual information formula

I'm calculating the difference between two mutual information's, to see if adding a new parameter is worth. My goal is to simplify the formula which might be a little CPU intensive, but I can't see ...
1
vote
1answer
482 views

Bayesian optimization neural network

When tuning my neural net with Bayesian optimization I want to determine the optimal number of hidden layers and the corresponding number of neurons in each hidden layer. The problem is that with an ...
2
votes
1answer
1k views

How to calculate the standard deviation for a Gaussian Process?

I am quite new to Gaussian processes. A Gaussian Process looks like the following: Where the dark blue line denotes the mean, and the filled-area denotes the mean+std and mean-std respectively. ...
0
votes
0answers
33 views

Accelerate the optimizing process of Deep Learning Model

Bayesian Optimization is currently the best optimization method for deep learning model. Although it already offer a good optimizing time but I want to accelerate by Instead of training every ...
1
vote
1answer
160 views

Is it posible to obtain the next point without bruteforcing in Bayesian Optimization?

The implementations I have found till now of Bayesian optimization tipically find the optimal input point X for next step, which minimizes the output Y of the acquisition function which is being ...
2
votes
1answer
44 views

Bayesian Optimization: How do we compute the maximum increase in information for the next step

If I understand correctly, Bayesian Optimization works by having a prior and choosing the next step by finding the direction that will maximize the mutual information between the prior and posterior. ...
4
votes
1answer
111 views

Bayesian optimizations of hyper-parameters - magical curve based on two data points?

I am trying to understand how Bayesian optimizations of hyper-parameters works in practice. Question 1: How shape of this curve (solid blue line in the middle) is calculated given Gaussian kernel. ...
2
votes
1answer
272 views

Problem with building Optimizer using Random Forest as Surrogate Model

I am trying to use Random Forest as Surrogate Model instead of Gaussian Process for my Bayesian Optimizer Framework and already studied the concept of it through SMAC and mlrMBO Papers. I use the ...
4
votes
1answer
573 views

multivariate gaussian processing | bayesian optimization with multiple features (parameter optimization)

Best My goal is very simple, I would like to optimize 3 parameters in my algorithm. And I would like to do this via, multivariate gaussian processes. But the problem which I've is, I do understand ...
1
vote
0answers
58 views

Bayesian estimation of an econometric State Space Model [Need suggestions or feedback]

I'm struggling to estimate the parameters $\theta_{t},\beta,\gamma,\sigma^{2}_{\epsilon},\sigma^{2}_{\eta}$ of the following model: $y_{t}=\theta_{t}+\beta (y_{t-1}-\theta_{t-1})+\gamma' x_{t} + \...