I'm trying to find the MAP estimate for a model by gradient descent. My prior is multivariate Gaussian with a known covariance matrix.
On a conceptual level, I think I know how to do this, but I was hoping for some help with the details. In particular, if there is an easier way to approach the problem, then that would be especially useful.
Here's what I think I need to do:
- For each dimension, find the conditional distribution, given my current position in the other dimensions.
- This gives me a local univariate Gaussian in each dimension, with the correct mean and standard deviation.
- I think that the gradient should just be a vector of derivatives for each of these univariate distributions.
My question has two parts:
- Is this the best approach to take, or is there an easier way?
- Assuming I need to go this route, what's the best way to go about finding these conditional distributions?