Questions tagged [laplace-approximation]
The laplace-approximation tag has no usage guidance.
29
questions
0
votes
0
answers
45
views
Laplace approximation from a log-posterior in R
I would like to perform a Laplace approximation of a log-posterior.
The evolution of a cancer cell at given time $t_j$, $j = 1,\cdots,n$ for an experiment $i$ follows the following Poisson ...
- 111
2
votes
0
answers
142
views
Gaussian Processes: multi-class Laplace approximation
In Chapter 3.5 of Gaussian Process for Machine Learning book by Rasmussen and Williams (R&W 2006), authors present a Laplace approximation for a multi-class Gaussian Process (GP) classifier. ...
- 51
2
votes
2
answers
337
views
Inference: How is the Laplace approximation actually useful to us compared with MLE and MAP?
I was reading a few different sources (including the "Machine Learning and Pattern Recognition" book by Bishop) about the Laplace integral approximation method for inference. However, I am ...
- 131
1
vote
0
answers
263
views
Random Fourier Features approximating a kernel inverse?
There is a method I have been studying called Spectral Normalised Neural Gaussian Processes which leaves me with a question I cannot answer. In this method, they utilize Random Fourier Features but ...
- 720
0
votes
0
answers
220
views
Why is the Hessian in the Laplace approximation negative
The Laplace approximation builds from the Taylor expansion of the MAP estimate, where the first derivative is 0. The second order Taylor series goes...
$$
f(a) + \frac{f'(a)}{1!}(x - a) + \frac{f''(a)}...
- 720
2
votes
2
answers
188
views
On the P-value of the variance of random intercept in glmer model
I'm using a logistic mixed-effect model with random intercept through glmer function from lme4 package. I want to test the ...
- 155
3
votes
2
answers
253
views
Laplace approximation in high-dimensions
Obviously computing the inverse Hessian is hard when a probability distribution is fitted on high-dimensional datapoints. One idea to reduce computational cost would be to approximate the distribution ...
- 924
7
votes
0
answers
312
views
Why does Quadratic (Normal/Laplace) Approximation fail on multilevel models?
In Statistical Rethinking, 2nd Edition, section 13.1, Richard McElreath says:
Why doesn’t simple quadratic approximation, using for example quap, work with multilevel models? When a prior is itself a ...
1
vote
0
answers
70
views
Laplace approximation for small number of data
When a large number of data points is available, according to the central limit theorem, Laplace method can give an efficient, good approximation of posterior as a Gaussian distribution centered at ...
- 924
16
votes
1
answer
2k
views
How are PQL, REML, ML, Laplace, Gauss-Hermite related to each other?
While learning about the Generalized Linear Mixed Models, I often see the above terms. Sometimes it seems to me these are separate methods of estimation of (fixed? random? both?) effects, but when I ...
- 287
1
vote
1
answer
559
views
What is the difference between approximate bayesian computation vs approximate bayesian inference?
What are the main differences between approximate bayesian computation vs approximate bayesian inference?
Are they essentially the same?
Do they refer to the same of different family of models?
My ...
- 202
1
vote
0
answers
152
views
Why does this improper prior = constant?
MacKay has an exercise on using Laplace's method for a Poisson model:
$$
p(r \mid \lambda ) = \frac{e^{-\lambda} \lambda^r}{r!}, \qquad p(\lambda) = \frac{1}{\lambda}
$$
And he asks the reader to ...
- 1,594
1
vote
0
answers
228
views
Approximating the Kullback-Leibler Divergence with a Laplace approximation
Suppose I wish to compute the (asymptotic) Kullback-Leibler Divergence (KLD) between the exact Bayesian posterior
$$q_{n}(\theta|x_{1:n}) \propto \pi(\theta)\prod_{i=1}^n p(x_i|\theta)$$
and the ...
- 1,511
1
vote
0
answers
105
views
Equation (3.23) GP for ML book
This is the computation of the variance when we do Laplace Approximation for inference in binary classification. I do not understand why the variance is decomposed into these two terms.
- 140
8
votes
2
answers
1k
views
Bayesian inverse modeling with non-identifiable parameters?
If I have a physical model
\begin{equation}
y = \frac{1}{\beta_0} (\beta_1 x_1 + \beta_2 x_2)
\end{equation}
and want to estimate coefficients $\beta_0$, $\beta_1$, and $\beta_2$ from given data ...
- 799
3
votes
2
answers
371
views
Lower-bound on covariance estimated via Laplace approximation?
I think when a posterior is approximated to be multivariate normal as in Laplace approximation, the covariance matrix is taken to be the negative inverse Hessian evaluated at the log-posterior maximum,...
- 603
3
votes
0
answers
241
views
Which optimizer use for laplace approximation
I have been trying to estimate the marginal posterior for D variable using Laplace approximation:
$p(\theta_i) \approx \left[\frac{\det{H}}{2\pi\det{H(\theta_i)}}\right]^{1/2} \exp\left[-L(\theta_i, \...
- 538
5
votes
1
answer
306
views
Constant of Laplace approximation
I'm reading Example 3.16 of Robert & Casella's Monte Carlo Statistical Methods. It uses a Laplace approximation for approximating an integral related with the Gamma distribution namely $$\int_a^b\...
- 535
16
votes
2
answers
2k
views
Comparing Laplace Approximation and Variational Inference
Does anyone know of any references that look at the relationship between the Laplace approximation and variational inference (with normal approximating distributions)? Namely I'm looking for something ...
- 1,548
3
votes
0
answers
74
views
Simple approximation of joint posterior
Consider the (hierarchical) Bayesian inference problem with two unknowns $(x,\theta)$ and data $y$. I'm using a very simple ("independence"?) approximation
$$ p(x,\theta|y) \approx p(x|\theta_\star,y) ...
- 832
3
votes
0
answers
187
views
Is BIC useful for a multimodal distribution?
Given a dataset $D$ and a model $M$ with parameters $\theta$, the Bayesian Information Criterion can be used to approximate the model's marginal likelihood $\int p(D|\theta,M)p(\theta|M) d\,\theta$. ...
- 143
6
votes
1
answer
493
views
What's the relationship between Laplace approximation and Variational Bayes methods?
To be precise, I'm checking this presentation https://kaybrodersen.github.io/talks/Brodersen_2013_03_22.pdf, but I don't understand what is the connection between Laplace method and variational bayes? ...
2
votes
2
answers
831
views
Modelling random effects as an autoregressive-autoregressive process
Has anyone ever come across an autoregressive-autogressive process.
I am modelling positively correlated time-series of random effects in a fisheries model. They are integrated out via the Laplace ...
- 616
2
votes
1
answer
424
views
Hessian for Laplace Approximation in Uncertainty Propagation
This is possibly a silly conceptual question, ... but anyway:
Imagine I have a function:
$f = F(\mathbf{x}) = F(x_1,x_2) = ax_1^2 + bx_2^3,$
where $x_1,x_2 \sim N(0,1)$ for example.
For a naive ...
- 420
0
votes
0
answers
241
views
Calculating variance using Laplace approximation for GP classification
I'm having some trouble implementing Algorithm 3.2 from Rasmussen and Williams.
Namely, sometimes when I evaluate step 6, I obtain a negative variance, which I believe is impossible (and makes line ...
- 802
8
votes
2
answers
2k
views
Marginalization of GP regression hyperparameters with Laplace approximation
I am using Gaussian Processes (GP) for regression (via the gpml package for MATLAB).
So far, I was optimizing the hyper-parameters by maximizing the log likelihood, but I would like to try a more ...
- 5,096
3
votes
0
answers
657
views
Laplace approximation for binomial distribution in matlab
i using bionrnd() function to generate a random vector and Laplace approximation formula to approximate the binomial distribution. but Laplace histogram dose not ...
- 131
4
votes
0
answers
926
views
How to derive the Fisher information in the laplace approximation of a generalized linear mixed model?
I am currently using the Laplace approximation to fit some geostatistical models for binomial data. Regarding parameters estimation I do not have any problem. I can easily implement the Laplace ...
7
votes
1
answer
1k
views
Reference for generalized linear mixed models using Laplace approximation
I'm fitting a generalized linear mixed model in R using the Laplace approximation. I'm looking for a reference for the Laplace approximation used for that, or a reference regarding the comparison ...
- 71