Questions tagged [wishart-distribution]

The Wishart distribution is a common matrix distribution on square symmetric semi-definite matrices.

Filter by
Sorted by
Tagged with
0 votes
0 answers
30 views

Sample covariance of non-independent Gaussian vectors

Problem definition Consider the following dataset \begin{equation*} \{y_j=m+v_j\}_{j=1}^N \end{equation*} of bivariate Gaussian vectors, where \begin{equation*} m\sim\mathcal{N}(\hat{m}, P_m) \qquad ...
0 votes
0 answers
42 views

Kronecker product representation of symmetric normal matrix specification

I'm looking for a generally-accepted way of specifying a symmetric random matrix with standard normal matrix elements, viz., $A_{ij} \sim N(0,1), \text{cov}(A_{ij},A_{kl})=\delta_{ik}\delta_{jl}+\...
2 votes
2 answers
64 views

Questions on the Wishart distribution

If $X$ is an $n\times p$ matrix where each row is iid multivariate normal, then $X^TX$ has a Wishart distribution. What is known about the limiting distribution of $X^TX$ for large $n$ when the rows ...
  • 253
2 votes
0 answers
119 views

What is the distribution of the Frobenius distance between two covariance matrices?

I am computing the Frobenius norm of the difference between two covariance matrices, \begin{align} |\mathbf{C}-\mathbf{C}'|_F=\sqrt{\sum_{i,j}\left(c_{ij}-c'_{ij}\right)^2}. \end{align} Each of these ...
  • 61
5 votes
2 answers
101 views

Random variate of a singular Wishart distribution with non-integral degrees of freedom

Let's say that I have a random variable $X$ that follows a singular Wishart distribution with $\nu$ degrees of freedom and a shape matrix $\Sigma$ that is $p\times p$. We can write "symbolically&...
  • 205
0 votes
0 answers
39 views

From GMM to Wishart MM

Given a random vector $X$ distributed as a vector Gaussian mixture model in $\mathbb R^n$ mixing $K$ centered multivariate Gaussian distributions as follows $$ \textstyle X \sim \sum_{i=k}^K \alpha_k \...
1 vote
0 answers
101 views

Partition of a Multivariate Normal Distribution with Inverse-Wishart Covariance Prior

I'm trying to partition a multivariate normal with the following structure: $$\bf{X} = \left. \begin{bmatrix} \bf{X}_1 \\ \bf{X}_2 \end{bmatrix}\right|\Sigma \sim N_p \left(\begin{bmatrix} \mu_1 \\ \...
  • 111
5 votes
1 answer
114 views

What is the expectation of the Cholesky factor of a Wishart distributed random matrix?

Let a $d-\text{dimensional}$ Wishart random variable with $\nu$ degrees of freedom $\Sigma$ be distributed according to $\mathcal{W}(\Sigma|\Sigma_0, \nu) \propto |\Sigma|^\frac{\nu-d-1}2\exp{(-\...
  • 595
2 votes
0 answers
62 views

On the Wishart distribution

Definitions Consider the following bivariate model \begin{equation*}y\triangleq S(p)h+v\end{equation*} where $p\triangleq \begin{bmatrix}\theta & \ell_1 & \ell_2\end{bmatrix}'$ and $\theta \...
  • 227
2 votes
0 answers
451 views

Clustering with gaussian mixtures: choice of hyperparameters

Question: I am interested in general in understanding how to choose the hyperparameters if we are interested in clustering bivariate vectors assuming a mixture of Gaussian mixture with conjugate ...
  • 501
1 vote
0 answers
53 views

Is there a closed form solution for the Hellinger Distance between two Wishart distributions?

Given two Wishart distributions $X_0 \sim W_{p_0}(V_0,n_0)$, $X_1 \sim W_{p_1}(V_1,n_1)$, what is the Hellinger Distance between them? Can it be obtained in closed form assuming $p_0 = p_1$? For ...
  • 16.9k
1 vote
0 answers
118 views

Marginal likelihood of bivariate Gaussian model

I assume the following model for a sample $y_1 \in \mathbb{R}^2$ of size $1$ with bivariate Gaussian likelihood and independent bivariate Gaussian and inverse-Wishart prior for the mean and variance ...
  • 501
1 vote
1 answer
63 views

Variance of $\operatorname{tr}(W^2)$ with $W \sim \text{Wishart}(n, \Sigma)$

Suppose $W \sim \text{Wishart}(n, \Sigma)$, where $\Sigma \in \mathbb R^{p\times p}$, the expectation of $\operatorname{tr}(W^2)$ is $$E[\operatorname{tr}(W^2)] =n(n+1)\operatorname{tr}(\Sigma^2) + n\...
  • 21
1 vote
0 answers
33 views

Finite integral vs Finite probability Density

I was reading the Bayesian Data Analysis book from Gelman et al. I was going through Appendix A which describes probability distributions, and realize the book talks about something I don't really ...
  • 117
3 votes
1 answer
432 views

How to impose restrictions on a random matrix via its prior distribution?

I am reading the paper Factor analysis and outliers: A Bayesian approach. The author starts with a factor analysis model given by $${\bf y}_i = {\bf \Lambda} {\bf z}_i + {\bf e}_i, \quad i = 1, \ldots,...
1 vote
0 answers
180 views

Kullback–Leibler divergence between two inverse Wishart distributions

Is there a closed form formula for the KL divergence between two inverse Wishart distributions?
  • 924
2 votes
0 answers
130 views

Marginal distribution for the off-diagonal elements of a Matrix following an Inverse-Wishart

I've found on CV the marginal distribution for the main-diagonal elements of a matrix which follows an Inverse-Wishart distribution. However, what will happen to the off-diagonal elements? what ...
1 vote
0 answers
206 views

How to Estimate Covariance Matrix using Wishart Distribution?

I learned that Wishart distribution can help estimate covariance matrix of MVN without sampling directly from MVN. But there are some points that are still unclear for me: Why we bother to estimate ...
3 votes
2 answers
1k views

How to calculate the Jacobian of the transformation ( for covariance matrix)

I'm reading this Paper about a separation strategy for modeling covariance matrices with focus on Bayesian analysis. Direct decomposition of covariance matrix is as follows: $\Sigma = \text{diag}(S)\,...
3 votes
1 answer
510 views

Kullback Leibler Divergence between two Normal Whishart Distributions

I'm having trouble to compute the KL Divergence between two normal-Wishart distributions. KL divergence from $Q$ to $P$ is defined as: $$D_{\mathrm{KL}}(P \Vert Q) = \int p(x) \log \frac{p(x)}{q(x)}...
  • 1,381
2 votes
0 answers
249 views

Sum of Log Chi-Squared Asymptotic Distribution

I'd like to find the asymptotic distribution of $$\sqrt{n}\left(\log|\mathbf{S}| - \log|\boldsymbol{\Sigma}|\right), ~~~~~n \rightarrow \infty$$ where $\mathbf{S} \sim W_j\left(n, \frac{\boldsymbol{\...
3 votes
1 answer
166 views

Sampling prior covariance matrices - nested sampling

I am trying to fit a multivariate Gaussian with a non-diagonal covariance matrix $\Sigma$ using nested sampling. Usually, in other Bayesian analyses, we would use a Inverse Wishart or LKJ prior on ...
3 votes
1 answer
666 views

Semi-conjugate inverse Wishart posterior, can we obtain the marginal?

In Hoff's text (A First Course in Bayesian Statistical Methods), he uses a semi-conjugate inverse-Wishart prior for the covariance matrix of a multivariate normal process. In equation 7.9, he has the ...
  • 145
2 votes
0 answers
50 views

Showing a useful result for Wisharts and Multivariate Beta random matrices

Let $\mathbf{A} \sim \text{Wishart}_m\left(k_a,\mathbf{V} \right)$ and $\mathbf{B} \sim \text{Wishart}_m\left(k_b,\mathbf{V} \right)$ be two full rank Wishart random matrices. Define $$ \mathbf{S} = \...
  • 19.4k
2 votes
1 answer
533 views

Inverse Wishart Prior for linear model

I know some bayesian methods employ an inverse wishart distribution for the prior distribution of the covariance matrix in a linear regression. I.e. for the model: $$Y=X\beta+\epsilon$$ Where $\...
  • 576
1 vote
0 answers
28 views

Estimate parameters of inv-Wishart distribution using Bayesian

$\Sigma \sim Inv-Wishart_{v}(\Lambda^{-1})$ Suppose that I have a set of observations of $\Sigma$s, I wonder if there is a conjugate way to estimate the value of $v$ and $\Lambda$ (especially $\...
  • 433
4 votes
1 answer
1k views

Integrate out (covariance) matrix in Normal-Wishart distribution

In Gelman's Bayesian Data Analysis Chapter 3.6, he introduces the multivariate normal with unknown mean and variance, with the priors $\Sigma\sim \text{Inv-Wishart}_{\nu_0}(\Lambda_0^{-1})$ $\mu\...
  • 145
1 vote
0 answers
216 views

Is assigning an inverse-Wishart distribution to a diagonal matrix problematic?

I'm reading the paper Bayesian Vector Autoregressions by Thomas Wozniak. He considers the model $$y_t = \mu + A_1 y_{t-1} + \cdots A_k y_{t-k} + u_t$$ where each $y_i$ is a $N$-vector, each $A_j$ is a ...
user avatar
5 votes
0 answers
156 views

Integrating the inverse-Wishart density

It is alleged in this question and in the Wikipedia article and elsewhere that the density function for the inverse-Wishart distribution with $n$ degrees of freedom on $p\times p$ positive-definite ...
3 votes
0 answers
135 views

What's the role of the scale matrix for the Inverse-Wishart and Wishart distributions?

What's the role of the scale matrix for the Inverse-Wishart and Wishart distributions? The purpose of finding this information is to enlighten me on how should one decide on a prior for a positive-...
2 votes
1 answer
640 views

Deriving the sampling distribution of MLE for Normal distribution

Let $X_1,\ldots,X_n$ be an observed random sample from $N_p(\mu, \Sigma)$. I know that the MLE of $\Sigma$ is $\frac{1}{n} \sum_i^n(X_i -\bar X)(X_i -\bar X)^T$, which is biased. We define $S = \...
  • 31
4 votes
0 answers
136 views

Marginal Distribution of Matrix Normal with Two Inverse Wisharts

Say I have a Matrix-Normal distribution and two Inverse Wishart Distributions $$X \sim MN_{p\times n}(0, \Sigma, \Omega)$$ $$ \Sigma \sim IW(a, A) $$ $$ \Omega \sim IW(b, B)$$ where $a$ and $b$ are ...
  • 958
3 votes
0 answers
691 views

Appropriate Distribution for Diagonal Covariance Matrices

Let's say I have a model like: \begin{align} X\mid\mu,\Sigma_X &\sim \mathcal{N}(\mu,\Sigma_X)\\ \mu\mid m, \Sigma_\mu &\sim \mathcal{N}(m,\Sigma_\mu) \\ \Sigma_X\mid \Psi, c &\sim \...
  • 2,084
1 vote
1 answer
958 views

Intuitive explanation of Inverse Wishart prior for covariance estimation

I am trying to understand what is going on in the use of an Inverse Wishart prior for (Gaussian) covariance, and what is the motivation for it. I am seeing this posed as a solution for when the ...
2 votes
0 answers
396 views

How do I find the "elliptical confidence region" from columns of a matrix that follows the Wishart distribution?

The subject is about the sample mean and the sample covariance estimators and their respective confidence regions for the estimated parameters. Suppose that $n$ samples are taken from a $p$-variate ...
5 votes
1 answer
389 views

Transformation of Inverse Wishart

Let $\Sigma$ be an $p\times p$ dimensional covariance matrix that is distributed Inverse Wishart with degrees of freedom $\nu$ and Prior scale matrix $\Psi$ such that we write $\Sigma \sim W^{-1}(\nu, ...
  • 958
4 votes
0 answers
210 views

Expected eigenvalues of a Wishart Matrix

I consider a $n\times n$ Wishart Matrix with expected value $p \cdot I_n$, i.e. a matrix of the form $$W = XX'$$ with $X$ a $n\times p$ matrix with independent standard normal entries. It is easy ...
  • 12.2k
4 votes
1 answer
220 views

What is the relationship between the normalization constants of the normal distribution and the (Inverse-)Wishart distribution?

I was looking at the probability density function of the multivariate normal distribution and that of the inverse-Wishart distribution: $$ \begin{array}{rccll} p_{\mathcal{N}}\left(\mathbf{x}; \...
4 votes
1 answer
452 views

Scaling for Wishart distribution

If $X$ has a Wishart distribution $W_p(n,\Sigma)$ , what's the distribution for $cX$ where $c>0$ ? I know that for a $\chi^2_n (x)$ distribution with $n$ degrees of freedom, $c\chi^2 $ follows $\...
  • 407
0 votes
0 answers
466 views

Expectation of the inverse of $\textbf{z} \textbf{z}^{H}$ where $\textbf{z}$ is a complex Gaussian vector

Considering the vector $\textbf{z} \sim \mathcal{CN}(\textbf{0}_{M},\Theta_{M \times M})$, what would be the expectation of $\frac{1}{\textbf{z} \textbf{z}^{H}}$, i.e., $\mathbb{E} \left\lbrace \frac{...
0 votes
0 answers
441 views

Property of Wishart distribution

I am solving the next exercise about a property of a Wishart Distribution: $$M_1\sim W_p(\Sigma,n_1)$$ $$M_2\sim W_p(\Sigma,n_2)$$ are independent, then $M_1+M_2\sim W_p(\Sigma,n_1+n_2)$ I have ...
  • 965
5 votes
1 answer
987 views

Result for covariance between elements of the sample covariance matrix

Given data matrix $X_{n,p}$ where $n$ is the sample size, $p$ is the dimension, the sample covariance is $$\hat{\Sigma}=\frac{1}{n-1}X^\top X=\{\hat{\sigma}_{ij}\}_{1\le i,j\le p}$$ Is there any ...
3 votes
0 answers
1k views

Maximum Likelihood of Wishart parameters

I'm having some difficulty in deriving the ML estimation of the parameters of a Wishart distribution. Given a set of matrices $\{W_1, W_2,\dots,W_N\} \in \mathbb{C}^{k\times k}$ for which $W_i \sim ...
  • 151
2 votes
1 answer
454 views

Is Wishart distance always positive?

From what is told on page 268 of the Polarimetric Radar Imaging: From Basics To Applications: I have written the following code for wishart distance calculation in Matlab: ...
2 votes
0 answers
100 views

expectation associated with Wishart distribution

Suppose that $W$ follows the Wishart distribution with parameter $\Sigma$ and d.f. $n$. Then, I would like to know the result of the following expectations. $E[tr(WAWB)tr(WC)]$ $E[tr(WAWB)tr(WCWD)]$...
  • 183
1 vote
3 answers
369 views

Unsolvable Integral?

Is the following integral solvable? $$P(X) = \int^{\infty}_{-\infty} \int^{\infty}_{-\infty} P(X|\mu,K)P(\mu|K)P(K) d\mu dK$$ with $$P(K) = \frac{|K| ^{(v-d-1)/2}}{2^{vd/2}|V|^{v/2}\Gamma_d|\frac{...
  • 91
1 vote
1 answer
282 views

use inverse Wishart for variance in MCMC

When you have a posterior that looks like as one step in Gibber Sampler $P(\xi | \Sigma_\xi, \theta) ∝ exp\{-1/2 \xi\Sigma_\xi^{-1}\xi\}P(data | \xi, \theta)$ Do you always assume inverse Wishart ...
  • 547
3 votes
0 answers
310 views

Distribution for sum of Wishart Matrices with scale matrices proportional to identity

Consider the independent $n\times p$ dimensional matrices $\mathbf{X}_i, i=1,...,G$, from the matrix variate normal distribution, $N_{n,p}\left(0,\mathbf{I}_n, \mathbf{\Sigma}_i\right)$: $$P_i\left(\...
1 vote
1 answer
708 views

What is the correct form of Metropolis Hasting step in scaled Inverse Wishart prior for covariance matrix?

I was going through the paper of O'Malley and Zaslavsky (2008) for the scaled inverse Wishart priors for a covariance matrix, in order to write an R-code for hierarchical Bayesian estimation of mixed ...
  • 11
0 votes
0 answers
37 views

When are $X^tC_1 X$ and $X^tC_2X$ independent?

Let $X$ be a $n\times p$ data matrix from $N_p(0,\Sigma)$. Let $C_1$ and $C_2$ be two symmmetric idempotent matrices. When are $X^tC_1 X$ and $X^tC_2X$ independent?
  • 1,237