Is there a way around the numerical problem of calculating the denominator of the Multivariate Gaussian distribution of high dimensional vectors?
Given the following formula $$f_{\mathbf X}(x_1,\ldots,x_k) = \frac{\exp\left(-\frac 1 2 ({\mathbf x}-{\boldsymbol\mu})^\mathrm{T}{\boldsymbol\Sigma}^{-1}({\mathbf x}-{\boldsymbol\mu})\right)}{\sqrt{(2\pi)^k|\boldsymbol\Sigma|}}$$
With a feature vector of size, $>128$ the denominator cannot be computed with normal precision(64 bit) frameworks.
My feature vectors are $1024$ long, and I haven't found a way to come across the problem.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
It's better to operate on log domain in such cases. There are implementations (employing factorizations like Cholesky, LU) available in common languages:
- python: numpy, tensorflow
- R: msos, lme4
