Questions tagged [wishart-distribution]
The Wishart distribution is a common matrix distribution on square symmetric semi-definite matrices.
107
questions
0
votes
0
answers
30
views
Sample covariance of non-independent Gaussian vectors
Problem definition
Consider the following dataset
\begin{equation*}
\{y_j=m+v_j\}_{j=1}^N
\end{equation*}
of bivariate Gaussian vectors, where
\begin{equation*}
m\sim\mathcal{N}(\hat{m}, P_m)
\qquad
...
- 131
0
votes
0
answers
42
views
Kronecker product representation of symmetric normal matrix specification
I'm looking for a generally-accepted way of specifying a symmetric random matrix with standard normal matrix elements, viz., $A_{ij} \sim N(0,1), \text{cov}(A_{ij},A_{kl})=\delta_{ik}\delta_{jl}+\...
- 2,164
2
votes
2
answers
64
views
Questions on the Wishart distribution
If $X$ is an $n\times p$ matrix where each row is iid multivariate normal, then $X^TX$ has a Wishart distribution.
What is known about the limiting distribution of $X^TX$ for large $n$ when the rows ...
- 253
2
votes
0
answers
119
views
What is the distribution of the Frobenius distance between two covariance matrices?
I am computing the Frobenius norm of the difference between two covariance matrices,
\begin{align}
|\mathbf{C}-\mathbf{C}'|_F=\sqrt{\sum_{i,j}\left(c_{ij}-c'_{ij}\right)^2}.
\end{align}
Each of these ...
- 61
5
votes
2
answers
101
views
Random variate of a singular Wishart distribution with non-integral degrees of freedom
Let's say that I have a random variable $X$ that follows a singular Wishart distribution with $\nu$ degrees of freedom and a shape matrix $\Sigma$ that is $p\times p$. We can write "symbolically&...
- 205
0
votes
0
answers
39
views
From GMM to Wishart MM
Given a random vector $X$ distributed as a vector Gaussian mixture model in $\mathbb R^n$ mixing $K$ centered multivariate Gaussian distributions as follows
$$
\textstyle X \sim \sum_{i=k}^K \alpha_k \...
1
vote
0
answers
101
views
Partition of a Multivariate Normal Distribution with Inverse-Wishart Covariance Prior
I'm trying to partition a multivariate normal with the following structure:
$$\bf{X} = \left. \begin{bmatrix} \bf{X}_1 \\ \bf{X}_2 \end{bmatrix}\right|\Sigma \sim N_p \left(\begin{bmatrix} \mu_1 \\ \...
- 111
5
votes
1
answer
114
views
What is the expectation of the Cholesky factor of a Wishart distributed random matrix?
Let a $d-\text{dimensional}$ Wishart random variable with $\nu$ degrees of freedom $\Sigma$ be distributed according to $\mathcal{W}(\Sigma|\Sigma_0, \nu) \propto |\Sigma|^\frac{\nu-d-1}2\exp{(-\...
- 595
2
votes
0
answers
62
views
On the Wishart distribution
Definitions
Consider the following bivariate model
\begin{equation*}y\triangleq S(p)h+v\end{equation*}
where
$p\triangleq \begin{bmatrix}\theta & \ell_1 & \ell_2\end{bmatrix}'$ and $\theta \...
- 227
2
votes
0
answers
451
views
Clustering with gaussian mixtures: choice of hyperparameters
Question: I am interested in general in understanding how to choose the hyperparameters if we are interested in clustering bivariate vectors assuming a mixture of Gaussian mixture with conjugate ...
- 501
1
vote
0
answers
53
views
Is there a closed form solution for the Hellinger Distance between two Wishart distributions?
Given two Wishart distributions $X_0 \sim W_{p_0}(V_0,n_0)$, $X_1 \sim W_{p_1}(V_1,n_1)$, what is the Hellinger Distance between them? Can it be obtained in closed form assuming $p_0 = p_1$?
For ...
- 16.9k
1
vote
0
answers
118
views
Marginal likelihood of bivariate Gaussian model
I assume the following model for a sample $y_1 \in \mathbb{R}^2$ of size $1$ with bivariate Gaussian likelihood and independent bivariate Gaussian and inverse-Wishart prior for the mean and variance ...
- 501
1
vote
1
answer
63
views
Variance of $\operatorname{tr}(W^2)$ with $W \sim \text{Wishart}(n, \Sigma)$
Suppose $W \sim \text{Wishart}(n, \Sigma)$, where $\Sigma \in \mathbb R^{p\times p}$, the expectation of $\operatorname{tr}(W^2)$ is
$$E[\operatorname{tr}(W^2)] =n(n+1)\operatorname{tr}(\Sigma^2) + n\...
- 21
1
vote
0
answers
33
views
Finite integral vs Finite probability Density
I was reading the Bayesian Data Analysis book from Gelman et al. I was going through Appendix A which describes probability distributions, and realize the book talks about something I don't really ...
- 117
3
votes
1
answer
432
views
How to impose restrictions on a random matrix via its prior distribution?
I am reading the paper Factor analysis and outliers: A Bayesian approach. The author starts with a factor analysis model given by
$${\bf y}_i = {\bf \Lambda} {\bf z}_i + {\bf e}_i, \quad i = 1, \ldots,...
- 636
1
vote
0
answers
180
views
Kullback–Leibler divergence between two inverse Wishart distributions
Is there a closed form formula for the KL divergence between two inverse Wishart distributions?
- 924
2
votes
0
answers
130
views
Marginal distribution for the off-diagonal elements of a Matrix following an Inverse-Wishart
I've found on CV the marginal distribution for the main-diagonal elements of a matrix which follows an Inverse-Wishart distribution.
However, what will happen to the off-diagonal elements? what ...
- 5,330
1
vote
0
answers
206
views
How to Estimate Covariance Matrix using Wishart Distribution?
I learned that Wishart distribution can help estimate covariance matrix of MVN without sampling directly from MVN. But there are some points that are still unclear for me:
Why we bother to estimate ...
- 63
3
votes
2
answers
1k
views
How to calculate the Jacobian of the transformation ( for covariance matrix)
I'm reading this Paper about a separation strategy for modeling covariance matrices with focus on Bayesian analysis. Direct decomposition of covariance matrix is as follows: $\Sigma = \text{diag}(S)\,...
- 51
3
votes
1
answer
510
views
Kullback Leibler Divergence between two Normal Whishart Distributions
I'm having trouble to compute the KL Divergence between two normal-Wishart distributions. KL divergence from $Q$ to $P$ is defined as:
$$D_{\mathrm{KL}}(P \Vert Q) = \int p(x) \log \frac{p(x)}{q(x)}...
- 1,381
2
votes
0
answers
249
views
Sum of Log Chi-Squared Asymptotic Distribution
I'd like to find the asymptotic distribution of
$$\sqrt{n}\left(\log|\mathbf{S}| - \log|\boldsymbol{\Sigma}|\right), ~~~~~n \rightarrow \infty$$
where $\mathbf{S} \sim W_j\left(n, \frac{\boldsymbol{\...
- 33
3
votes
1
answer
166
views
Sampling prior covariance matrices - nested sampling
I am trying to fit a multivariate Gaussian with a non-diagonal covariance matrix $\Sigma$ using nested sampling.
Usually, in other Bayesian analyses, we would use a Inverse Wishart or LKJ prior on ...
- 497
3
votes
1
answer
666
views
Semi-conjugate inverse Wishart posterior, can we obtain the marginal?
In Hoff's text (A First Course in Bayesian Statistical Methods), he uses a semi-conjugate inverse-Wishart prior for the covariance matrix of a multivariate normal process. In equation 7.9, he has the ...
- 145
2
votes
0
answers
50
views
Showing a useful result for Wisharts and Multivariate Beta random matrices
Let $\mathbf{A} \sim \text{Wishart}_m\left(k_a,\mathbf{V} \right)$ and $\mathbf{B} \sim \text{Wishart}_m\left(k_b,\mathbf{V} \right)$ be two full rank Wishart random matrices. Define
$$
\mathbf{S} = \...
- 19.4k
2
votes
1
answer
533
views
Inverse Wishart Prior for linear model
I know some bayesian methods employ an inverse wishart distribution for the prior distribution of the covariance matrix in a linear regression. I.e. for the model:
$$Y=X\beta+\epsilon$$
Where $\...
- 576
1
vote
0
answers
28
views
Estimate parameters of inv-Wishart distribution using Bayesian
$\Sigma \sim Inv-Wishart_{v}(\Lambda^{-1})$
Suppose that I have a set of observations of $\Sigma$s, I wonder if there is a conjugate way to estimate the value of $v$ and $\Lambda$ (especially $\...
- 433
4
votes
1
answer
1k
views
Integrate out (covariance) matrix in Normal-Wishart distribution
In Gelman's Bayesian Data Analysis Chapter 3.6, he introduces the multivariate normal with unknown mean and variance, with the priors
$\Sigma\sim \text{Inv-Wishart}_{\nu_0}(\Lambda_0^{-1})$
$\mu\...
- 145
1
vote
0
answers
216
views
Is assigning an inverse-Wishart distribution to a diagonal matrix problematic?
I'm reading the paper Bayesian Vector Autoregressions by Thomas Wozniak. He considers the model
$$y_t = \mu + A_1 y_{t-1} + \cdots A_k y_{t-k} + u_t$$ where each $y_i$ is a $N$-vector, each $A_j$ is a ...
user235125
5
votes
0
answers
156
views
Integrating the inverse-Wishart density
It is alleged in this question and in the Wikipedia article and elsewhere that the density function for the inverse-Wishart distribution with $n$ degrees of freedom on $p\times p$ positive-definite ...
- 7,574
3
votes
0
answers
135
views
What's the role of the scale matrix for the Inverse-Wishart and Wishart distributions?
What's the role of the scale matrix for the Inverse-Wishart and Wishart distributions?
The purpose of finding this information is to enlighten me on how should one decide on a prior for a positive-...
- 5,330
2
votes
1
answer
640
views
Deriving the sampling distribution of MLE for Normal distribution
Let $X_1,\ldots,X_n$ be an observed random sample from $N_p(\mu, \Sigma)$.
I know that the MLE of $\Sigma$ is $\frac{1}{n} \sum_i^n(X_i -\bar X)(X_i -\bar X)^T$, which is biased.
We define $S = \...
- 31
4
votes
0
answers
136
views
Marginal Distribution of Matrix Normal with Two Inverse Wisharts
Say I have a Matrix-Normal distribution and two Inverse Wishart Distributions
$$X \sim MN_{p\times n}(0, \Sigma, \Omega)$$
$$ \Sigma \sim IW(a, A) $$
$$ \Omega \sim IW(b, B)$$
where $a$ and $b$ are ...
- 958
3
votes
0
answers
691
views
Appropriate Distribution for Diagonal Covariance Matrices
Let's say I have a model like:
\begin{align}
X\mid\mu,\Sigma_X &\sim \mathcal{N}(\mu,\Sigma_X)\\
\mu\mid m, \Sigma_\mu &\sim \mathcal{N}(m,\Sigma_\mu) \\
\Sigma_X\mid \Psi, c &\sim \...
- 2,084
1
vote
1
answer
958
views
Intuitive explanation of Inverse Wishart prior for covariance estimation
I am trying to understand what is going on in the use of an Inverse Wishart prior for (Gaussian) covariance, and what is the motivation for it. I am seeing this posed as a solution for when the ...
- 591
2
votes
0
answers
396
views
How do I find the "elliptical confidence region" from columns of a matrix that follows the Wishart distribution?
The subject is about the sample mean and the sample covariance estimators and their respective confidence regions for the estimated parameters.
Suppose that $n$ samples are taken from a $p$-variate ...
5
votes
1
answer
389
views
Transformation of Inverse Wishart
Let $\Sigma$ be an $p\times p$ dimensional covariance matrix that is distributed Inverse Wishart with degrees of freedom $\nu$ and Prior scale matrix $\Psi$ such that we write $\Sigma \sim W^{-1}(\nu, ...
- 958
4
votes
0
answers
210
views
Expected eigenvalues of a Wishart Matrix
I consider a $n\times n$ Wishart Matrix with expected value $p \cdot I_n$, i.e. a matrix of the form
$$W = XX'$$
with $X$ a $n\times p$ matrix with independent standard normal entries.
It is easy ...
- 12.2k
4
votes
1
answer
220
views
What is the relationship between the normalization constants of the normal distribution and the (Inverse-)Wishart distribution?
I was looking at the probability density function of the multivariate normal distribution and that of the inverse-Wishart distribution:
$$
\begin{array}{rccll}
p_{\mathcal{N}}\left(\mathbf{x}; \...
- 378
4
votes
1
answer
452
views
Scaling for Wishart distribution
If $X$ has a Wishart distribution $W_p(n,\Sigma)$ , what's the distribution for $cX$ where $c>0$ ?
I know that for a $\chi^2_n (x)$ distribution with $n$ degrees of freedom, $c\chi^2 $ follows $\...
- 407
0
votes
0
answers
466
views
Expectation of the inverse of $\textbf{z} \textbf{z}^{H}$ where $\textbf{z}$ is a complex Gaussian vector
Considering the vector $\textbf{z} \sim \mathcal{CN}(\textbf{0}_{M},\Theta_{M \times M})$, what would be the expectation of $\frac{1}{\textbf{z} \textbf{z}^{H}}$, i.e.,
$\mathbb{E} \left\lbrace \frac{...
0
votes
0
answers
441
views
Property of Wishart distribution
I am solving the next exercise about a property of a Wishart Distribution:
$$M_1\sim W_p(\Sigma,n_1)$$
$$M_2\sim W_p(\Sigma,n_2)$$
are independent, then $M_1+M_2\sim W_p(\Sigma,n_1+n_2)$
I have ...
- 965
5
votes
1
answer
987
views
Result for covariance between elements of the sample covariance matrix
Given data matrix $X_{n,p}$ where $n$ is the sample size, $p$ is the dimension, the sample covariance is
$$\hat{\Sigma}=\frac{1}{n-1}X^\top X=\{\hat{\sigma}_{ij}\}_{1\le i,j\le p}$$
Is there any ...
- 468
3
votes
0
answers
1k
views
Maximum Likelihood of Wishart parameters
I'm having some difficulty in deriving the ML estimation of the parameters of a Wishart distribution.
Given a set of matrices $\{W_1, W_2,\dots,W_N\} \in \mathbb{C}^{k\times k}$ for which $W_i \sim ...
- 151
2
votes
1
answer
454
views
Is Wishart distance always positive?
From what is told on page 268 of the Polarimetric Radar Imaging: From Basics To Applications:
I have written the following code for wishart distance calculation in Matlab:
...
- 269
2
votes
0
answers
100
views
expectation associated with Wishart distribution
Suppose that $W$ follows the Wishart distribution with parameter $\Sigma$ and d.f. $n$.
Then, I would like to know the result of the following expectations.
$E[tr(WAWB)tr(WC)]$
$E[tr(WAWB)tr(WCWD)]$...
- 183
1
vote
3
answers
369
views
Unsolvable Integral?
Is the following integral solvable?
$$P(X) = \int^{\infty}_{-\infty} \int^{\infty}_{-\infty} P(X|\mu,K)P(\mu|K)P(K) d\mu dK$$
with
$$P(K) = \frac{|K| ^{(v-d-1)/2}}{2^{vd/2}|V|^{v/2}\Gamma_d|\frac{...
- 91
1
vote
1
answer
282
views
use inverse Wishart for variance in MCMC
When you have a posterior that looks like as one step in Gibber Sampler
$P(\xi | \Sigma_\xi, \theta) ∝ exp\{-1/2 \xi\Sigma_\xi^{-1}\xi\}P(data | \xi, \theta)$
Do you always assume inverse Wishart ...
- 547
3
votes
0
answers
310
views
Distribution for sum of Wishart Matrices with scale matrices proportional to identity
Consider the independent $n\times p$ dimensional matrices $\mathbf{X}_i, i=1,...,G$, from the matrix variate normal distribution, $N_{n,p}\left(0,\mathbf{I}_n, \mathbf{\Sigma}_i\right)$:
$$P_i\left(\...
- 161
1
vote
1
answer
708
views
What is the correct form of Metropolis Hasting step in scaled Inverse Wishart prior for covariance matrix?
I was going through the paper of O'Malley and Zaslavsky (2008) for the scaled inverse Wishart priors for a covariance matrix, in order to write an R-code for hierarchical Bayesian estimation of mixed ...
- 11
0
votes
0
answers
37
views
When are $X^tC_1 X$ and $X^tC_2X$ independent?
Let $X$ be a $n\times p$ data matrix from $N_p(0,\Sigma)$. Let $C_1$ and $C_2$ be two symmmetric idempotent matrices. When are $X^tC_1 X$ and $X^tC_2X$ independent?
- 1,237