1
$\begingroup$

What is meant by the dimension in bayesian optimization? In many papers it is stated that the dimension should be lower than 20 (then there are papers which are solving high-dimensional problem). But, is this the dimension of data set, or the number of variables to be optimized? For example, I have a data set with 100 features and I want to train a neural network and want to find the best learning rate and momentum term (i.e. 2 parameters to optimize).

edit: Here it is written in second sentence: https://www.groundai.com/project/a-tutorial-on-bayesian-optimization/1

$\endgroup$

2 Answers 2

1
$\begingroup$

I suppose 20 dimensions just means optimization problem for some parameter vector $\theta$ that has 20 coordinates such that $\theta=(\theta_1,...,\theta_{20}) \in \mathbb R^{20}$. They are considering problems of dimension less that 20 so they are solving for less than $20$ values. As the article you refer to say the solution is in $\mathbb R^d$ with $d<20$.

$\endgroup$
2
$\begingroup$

In this case, it is parameters. Parameters are random in subjective Bayesian thinking and are uncertain in objective Bayesian thinking. For a bivariate normal distribution, you would have five dimensions. You would have both means, the variances and the one covariance which is identical to the other covariance and so does not get separate treatment. If your posterior would be a Polya distribution for classification purposes, then you would have one parameter per variable.

With one hundred features you would want to use some other algorithm if this one is not recommended for more than nineteen.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.