lacerbi
• Member for 6 years, 7 months
• Last seen this week

There are many wrong or slightly off assumptions in this question, I will try to work through them. Minor point: Neural networks may or may not be statistical models (many of them are not). I would ...

If re-rolls are allowed (which is unclear from your question, but nothing seems to prevent it from the problem setup), there is a fairly easy solution. For simplicity (but it's not required), suppose ...

If they are (conditionally) independent, as you say, then the joint entropy (conditioned on the same thing as above) is the sum of the marginal entropies. This follows immediately from the ...

The more general formulation for the log marginal likelihood (not marginal log likelihood, as you originally wrote - I edited it in your post) of a GP is $$\log p(y|X) = -\frac{1}{2}(y - m(X))^T(K+\... View answer 14 votes Edits: Added reference to this paper in the body, as requested by the OP. I am giving an answer as a naive empirical Bayesian here. First, the posterior distribution allows you to do computations ... View answer 10 votes It means that your chain most likely did not converge. By this I mean you should be wary of the entire chain, not just worry about the dimensions with low effective number of samples. Solutions Burn-... View answer Accepted answer 22 votes Genetic algorithms (GA) are a family of heuristics which are empirically good at providing a decent answer in many cases, although they are rarely the best option for a given domain. You mention ... View answer Accepted answer 2 votes NoS means "numbers of" (that is, count the occurrences of the substring in the individual's chromosome). F(x) is the fitness you are computing (a scalar number). In particular, mNoS("110") where m is ... View answer Accepted answer 12 votes I would like to correct an erroneous assumption in the original post, a mistake which is relatively common. The OP says: From what I have read and from answers to other questions I have asked here, ... View answer Accepted answer 6 votes A linear transformation of the position variables is equivalent to the inverse linear transformation of the momentum variables. Ideally, you want to sample from a (transformed) distribution whose ... View answer Accepted answer 15 votes In loose terms, a kernel or covariance function k(x, x^\prime) specifies the statistical relationship between two points x, x^\prime in your input space; that is, how markedly a change in the ... View answer Accepted answer 29 votes Yes. Unlike what other answers state, 'typical' machine-learning methods such as nonparametrics and (deep) neural networks can help create better MCMC samplers. The goal of MCMC is to draw samples ... View answer 11 votes A1. This sounds like a sensible plan to me. Just to mention a couple of points. You'll want to test with different error metrics (L^p, K-L divergence, etc.) since methods will perform differently ... View answer 2 votes This is more of an extended comment, so take it as such. Define:$$ f(x) \equiv \frac{1}{n} \sum_{i = 1}^n \mathcal{N}\left(x | x_i, \sigma_i^2 \right)  (I am using the standard notation for ...

Recapping and expanding on my discussion in the comments. If you had $(lb_i, ub_i)$ for every $i$, one approach would be the one I recommended to this other question, that is to have a single GP with ...

This is an attempt which I didn't completely work through, but too long for the comments section. It might be useful to put it here as another basic alternative for very low $k$. It does not require ...

(Expanding my comment to a proper answer.) As I mentioned, it depends on your goal. The expected value $\mathbb{E}[f(x)]$ is only one of many possible choices for the optimization target. For ...

After some online searching, I have come under the impression that a comprehensive benchmark of established MCMC methods, analogous to what one can find in the optimization literature, does not exist. ...

There is another simple answer: a bug in the code. Whops. After a long debugging session, I found a subtle indexing mistake in the script that was reading the results of the 10-fold CV. I am writing ...

For the record, I posted a similar question to the Stan users mailing list, which you can find here. I was answered by one of the authors of the original PSIS-LOO paper and by other contributors of ...

The norm is to use any global optimizer you like. The problem is that the EI surface is highly multi-modal and disconnected; optimizing this acquisition function is a nontrivial problem in itself. A ...

It seems that you want to do is to train a Gaussian process (GP) for regression with observation-dependent noise so as to approximate $f(\textbf{x})$. The idea is that your $y_i$ are noisy (uncertain) ...

The state-of-the-art for non-convex optimization in a complex, ill-conditioned and multi-modal landscape is Covariance Matrix Adaptation - Evolution Strategies, aka CMA-ES, which in various versions (...

(I am the OP.) For example, my answers would be: Slice sampling as per Neal (2003). It requires very little tuning; you only need to very vaguely know the length scales of your problem (and you can ...