# Tag Info

Accepted

### Maximum Likelihood Estimators - Multivariate Gaussian

Deriving the Maximum Likelihood Estimators Assume that we have $m$ random vectors, each of size $p$: $\mathbf{X^{(1)}, X^{(2)}, \dotsc, X^{(m)}}$ where each random vectors can be interpreted as an ...

### Deriving the conditional distributions of a multivariate normal distribution

The answer by Macro is great, but here is an even simpler way that does not require you to use any outside theorem asserting the conditional distribution. It involves writing the Mahalanobis distance ...
• 129k

### Understanding degenerate multivariate normal distribution

A joint density function, say of two random variables $X$ and $Y$, is $f_{X,Y}(x,y)$ is an ordinary function of two real variables and the meaning that we ascribe to it is that if $\mathcal B$ is a ...
• 47.2k
Accepted

### Deriving the normalizing constant for the multivariate Gaussian

The reason that $\int_{\mathcal{R}^d} e^{-\frac{1}{2} \sum_i \lambda_i y_i^2} dy$ is easier to integrate is that it can be expressed as a product of univariate integrals: \begin{align} \int_{\mathcal{...
Accepted

### Quadratic form and Chi-squared distribution

In general, the quadratic form is a weighted sum of $\chi_1^2$ It is not true in general that $\mathbf{z}^\text{T} \mathbf{\Sigma} \mathbf{z} \sim \chi^2_p$ for any symmetric positive-definite (...
• 129k
Accepted

### How to calculate the probability of a data point belonging to a multivariate normal distribution?

Yeah, that sounds right. If you have parameters $\mu$ and $\Sigma$ and data point $x$, then the set of all data points that are less likely than $x$ are the ones that have less density, or in other ...
• 21.2k

### Maximum Likelihood Estimators - Multivariate Gaussian

An alternate proof for $\widehat{\Sigma}$ that takes the derivative with respect to $\Sigma$ directly: Picking up with the log-likelihood as above: \begin{eqnarray} \ell(\mu, \Sigma) &=& ...
Accepted

### Why are contours of a multivariate Gaussian distribution elliptical?

You can understand the shape of the ellipsoid better if you look at the spectral/eigen decomposition of the precision matrix (inverse of the covariance matrix). You want to look at the eigenvalues of ...
• 21.2k

### Generalized Chi-Squared Distribution PDF

Your question is really a special case of https://math.stackexchange.com/questions/442472/sum-of-squares-of-dependent-gaussian-random-variables/442916#442916 (with $A=I$). But nevertheless: $X$ is ...
• 81.5k
Accepted

### What is the geometric relationship between the covariance matrix and the inverse of the covariance matrix?

Before I answer your questions, allow me to share how I think about covariance and precision matrices. Covariance matrices have a special structure: they are positive semi-definite (PSD), which means ...
• 605
Accepted

### Why does the resulting matrix from Cholesky decomposition of a covariance matrix when multiplied by its transpose not give back the covariance matrix?

As explained in my comment, the inconvenient truth is that the Cholesky decomposition while usually defined as $K=LL^T$ where $L$ is lower triangular, is equally valid as $K=U^TU$ where $U$ is upper ...
• 45k
Accepted

• 329k
Accepted

### Sufficient statistic for bivariate or multivariate normal

As W. Huber tried to lead you to conclude, the sufficiency is a simple consequence of looking at the likelihood: \begin{align*} f(\mathbf{x}_1,\ldots,\mathbf{x}_n|\boldsymbol{\mu},\boldsymbol{\Sigma}) ...
• 107k

### Generate normally distributed random numbers with non positive-definite covariance matrix

The question concerns how to generate random variates from a multivariate Normal distribution with a (possibly) singular covariance matrix $\mathbb{C}$. This answer explains one way that will work ...
• 329k
Accepted

### MLE of multivariate normal distribution with diagonal covariance matrix

This is not an answer to your question, but a hint that should make the answer fairly obvious. If $\Sigma$ is diagonal, then the likelihood function factors into a product of $p$ densities (one could ...
• 12.2k
Accepted

### If $X \sim N(0,1)$, does $(X, X)^\prime$ have a bivariate normal distribution?

Two copies of the same normal variable stacked up in a vector yield a degenerate bivariate normal. See wikipedia on the degenerate case of the multivariate normal While it is a special case of the ...
• 286k

### Why is Gaussian Copula's Tail Dependence Zero?

For a non-technical, intuitive view of what the tail index is telling you, we can look at simulation and compute sample estimates of the quantity $P[F(Y) > q | F(X) > q]$ as $q$ increases. Here ...
• 286k
Accepted

• 11.4k
Accepted

### Expected value of the outer product of normalized, non-centered Gaussian vector

After some work I found the solution to the problem. The solution is the following: Eq (1). \mathbb{E}\left[\frac{XX^T}{X^TX}\right] = \frac{1}{n}{}_{1}F_1\left(1; \frac{n}{2}+1; \...
• 1,268

### Variance Among Coordinates of Multivariate Normal

Based on your comment, you question can be clarified as follows: If $(X_1, \ldots, X_n) \sim N_n(0, \Sigma)$, what is the expected value of the sample variance (before bias correction) \begin{align} ...
• 20.8k

### Finding $\mathbb E(Y_1^2Y_2^2)$ when $(Y_1,Y_2)$ is normal

You can use the conditional distributions, which are univariate normal. For example, $Y_2$ given $Y_1$ is normal with mean $$E\left[Y_2\mid Y_1\right]=\frac{\sigma_{12}}{\sigma_{11}} Y_1$$ and ...
• 11.4k

### Finding $\mathbb E(Y_1^2Y_2^2)$ when $(Y_1,Y_2)$ is normal

Use the Characteristic function. It is well known that, $$\varphi_Y(s) := \mathbb E\left[e^{is^\intercal Y}\right] = e^{-\frac12s^\intercal \Sigma s}.$$ So \begin{align} \frac{\partial^2 \varphi_Y}{\...
• 280

### Is Gaussian Process just a Multivariate Gaussian Distribution?

The multivariate Gaussian distribution is a distribution that describes the behaviour of a finite (or at least countable) random vector. Contrarily, a Gaussian process is a stochastic process defined ...
• 129k
Accepted

### Does assumption of normality of each mixture components implies that each margins is normal

Multivariate Gaussian mixtures are not themselves multivariate Gaussian, their components are. Statements that apply to the components of a mixture don't generally apply to the mixture (this would ...
• 286k
Accepted

### Transform general multivariate normal to standard multivariate normal

With the help of the content here, I was able to work this out and thought I'd post the resulting code and verification: ...
• 13.9k
Accepted

### Multivariate normal distribution vs. sampling multiple times from univariate normal distribution

What you propose ignores correlations. When marginals have correlation (say positive for now), then when one is high, the others will tend to be high. Example Marginals are $X,Y\sim N(0,1)$. Draw from ...
• 65.1k

### Conceptual proof that conditional of a multivariate Gaussian is multivariate Gaussian

The device used in the answer that you cite will also get you the conditional distribution. Here is a self-contained derivation with a slight change in notation. Partition the column vector \$X:=(X_1, ...
• 2,862

Only top scored, non community-wiki answers of a minimum length are eligible