8
votes
Distribution of the sum of Wishart distributed random matrices
Depending on what you mean by closed-form, the answer is to some extent, yes.
As per my comment, if $\Psi_1=\Psi_2$, then it can be shown that $A = A_1+A_2$ follows a Wishart distribution with scale ...
- 12.1k
6
votes
Unsolvable Integral?
It is a well-known problem. A complete set of solutions can be found at https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf. If speed is of the essence then this is an acceptable solution, but ...
- 7,810
6
votes
Is Inverse-Wishart a conjugate prior for Wishart likelihood?
This is actually quite a well-known result in Bayesian statistics (see e.g., Evans 1965, Chen 1979, Dickey, Lindley and Press 1985 and Leonard and Hsu 1992). In most of the literature on Bayesian ...
- 133k
5
votes
Accepted
Result for covariance between elements of the sample covariance matrix
If you're assuming Normality, you want to have a look at the Wishart distribution.
If $\mathbf{X}_{n,p}$ ($n \ge p$) has rows that are iid multivariate normal with mean $0$ and variance $\Sigma$, ...
- 21.6k
5
votes
Accepted
Integrate out (covariance) matrix in Normal-Wishart distribution
First, notice that, if you use some properties of the trace operator,
\begin{align*}
p(\mu, \Sigma) &\propto \lvert \Sigma\rvert^{-((\nu_0+d)/2+1)}\exp\Big(-\frac{1}{2}\text{tr}(\Lambda_0\Sigma^{-...
- 21.6k
5
votes
Accepted
What is the Fisher's information matrix for the Wishart distribution?
$\newcommand{\D}{\textsf{D}}$
$\newcommand{\tr}{\text{tr}}$
$\newcommand{\vec}{\text{vec}}$
$\newcommand{\vech}{\text{vech}}$
I've derived it with the second order differential.
The log-likelihood is
$...
- 19.7k
5
votes
Derivation of Normal-Wishart posterior
The likelihood $\times$ prior is
$$|\boldsymbol{\Lambda}|^{N/2} \exp\left\{-\frac{1}{2}\left(\sum_{i=1}^N \boldsymbol{x}_i^T \boldsymbol{\Lambda} \boldsymbol{x}_i - N\boldsymbol{\bar{x}}^T \boldsymbol{...
- 722
5
votes
Eigenvectors of a Wishart matrix
Assume $\Sigma = I_p$. The eigenvectors of $X^TX$ are the right singular vectors of $X$, e.g., $v_1,...,v_r$ with $r=\min(n,p)$ if the SVD of $X$ is $X=\sum_{i=1}^r s_i u_i v_i^\top$.
You wish to ...
- 596
5
votes
Accepted
What is the expectation of the Cholesky factor of a Wishart distributed random matrix?
In 1933 Bartlett described the Wishart distribution in terms of the distribution of the factors after Cholesky decomposition. I can not read the original source (On the theory of statistical ...
- 86.6k
5
votes
Random variate of a singular Wishart distribution with non-integral degrees of freedom
The Wishart Distribution is defined on the manifold $\mathcal{M}(p)$ of all positive-definite symmetric (psd) $p\times p$ matrices. In the coordinate system $(x_{ij}, 1\le j \le i \le p)$ (which ...
- 334k
4
votes
Accepted
The multivariate normal distribution has the same relationship with the Wishart distribution as the multivariate t-distribution with the ...?
Sutradhar and Ali (1989) - A Generalization of the Wishart Distribution for the Elliptical Model and Its Moments for the Multivariate t Model
They show:
Let the p-dimensional random vectors $X_1, \...
- 842
4
votes
Prior distribution importance in Bayesian inference
By Bayes theorem
$$ \text{posterior} \propto \text{prior} \times \text{likelihood} $$
so posterior combines information that comes from your data (through likelihood) and information that comes from ...
- 141k
4
votes
Accepted
Semi-conjugate inverse Wishart posterior, can we obtain the marginal?
Simple answer
No, you cannot. Since in your problem setting, $\Sigma | \vec{\mathbf{y}}$ and $\theta | \vec{\mathbf{y}}$ are by no means independent. Thus we cannot simply marginalize it.
Your try
...
- 1,565
4
votes
How to calculate the Jacobian of the transformation ( for covariance matrix)
The algebra of differential forms makes short work of this.
I take it that $\Sigma = (\sigma_{ij})$ is coordinatized by means of one of its triangles, say the upper triangle with components $\sigma_{...
- 334k
4
votes
Accepted
$E[W\otimes W]$ for Wishart R.V. $W$
I believe that there is an error in Seber's formula and the correct form is
$$
\mathbf{E}(WAW)
= m[\Sigma A \Sigma + \mathrm{Tr}(A\Sigma) \Sigma] + m^2 \Sigma A \Sigma
$$
(try this and the orgiginal ...
- 664
3
votes
Accepted
Integrating the inverse-Wishart density
In complete analogy to the (non-singular) Wishart distribution with $n$ degrees of freedom, for $n \in \mathbb N_{\geq p},$ the function $f$ can be interpreted as a joint probability density of the $p(...
- 6,650
3
votes
Intuitive explanation of Inverse Wishart prior for covariance estimation
I assume you are aware that Wishart matrices can be generated as the outer product $X^TX$ of a matrix where each row is an independent observation of a multivariate normal distribution, yes? So a ...
- 1,079
3
votes
Prior distribution importance in Bayesian inference
The posterior already incorporates the prior through Bayes rule; that's the whole point of it being a posterior.
"Uninformative" has many different mathematical definitions (see their reference 12 ...
- 2,611
3
votes
What is the correct form of Metropolis Hasting step in scaled Inverse Wishart prior for covariance matrix?
I found this question when I was working on inverse-Wishart in this question. I wrote a piece of code in Python. To do MH, I think we only need to compare old/new likelihood and prior.
...
- 387
3
votes
Accepted
Intuitive explanation for Marchenko-Pastur law
You are asking several different questions here. you are asking 1) what is the marchenko-pastur law when there is some correlation between the columns of the Wishart matrix, 2) how do we find $\lambda^...
- 408
3
votes
Accepted
Kullback Leibler Divergence between two Normal Whishart Distributions
You are getting closed. To ease the derivation, let's redefine some notations:
\begin{cases}
p(\pmb{\mu}, \pmb{\Lambda}) = p(\pmb{\mu} \vert \pmb{\Lambda}) p(\pmb{\Lambda}) & = \mathcal{N}\left( \...
- 451
3
votes
Bounds of integration for the Wishart density
Here is the requested theorem
You would probably be interested in the proof, too. It runs over a few pages.
Multiplying differentials is done in the skew-symmetric sense, not in the typical ...
- 21.6k
2
votes
Normal-inverse-Wishart distribution
One application is the Gibbs Sampling from a Dirichlet Process mixture model, where a conjugate prior is required. See page 33 of the pdf below
https://www.cs.cmu.edu/~kbe/dp_tutorial.pdf
- 21
2
votes
Accepted
Reconciling Wishart mean and Inverse-Wishart mean
Let's look at the simpler case of positive univariate random variables. I assume it's already clear that in general for a positive random variable whose variance is not $0$, that $\text{Cov}(X,1/X)<...
- 291k
2
votes
Prior distribution importance in Bayesian inference
No, you cannot skip it. Uninformative priors contain information and for multivariate regression assure that the sum of the probabilities will be unity. In fact, you cannot use a uniform prior on a ...
- 7,810
2
votes
Generating correlation matrices using Wishart distribution
Let $S$ be from Wishart$(\Sigma,r)$ and let $D$ be the diagonal matrix such that $D_{ii} = \sqrt{S_{ii}}$. Then $R=D^{-1}SD^{-1}$ will be a random correlation matrix. Actually one can easily check ...
2
votes
use inverse Wishart for variance in MCMC
There is no "default" priors for anything. Wishart distribution is commonly used for variance because it is a conjugate prior for multivariate normal distribution, but that does not make it anyhow "...
- 141k
2
votes
Is Wishart distance always positive?
Can any sensible distance measure be negative? No.
Consider this. Can you have a distance that is closer than zero? If you do then what is the meaning of it?
The beginning of the page 268 from your ...
- 62.3k
2
votes
Unsolvable Integral?
The answer can be derived from the following result.
If $\Sigma \sim {\cal IW}_\nu(V)$ (inverse-Wishart) and $(G \mid \Sigma) \sim {\cal N}(\theta, \lambda\Sigma)$, then $G \sim {\cal T}_{\nu-d+1}\...
- 19.7k
2
votes
Accepted
Unsolvable Integral?
Yeah, it's a multivariate t density.
Multiplying the three densities together and integrating (also using some properties of determinants) gives
\begin{align*}
&\iint \frac{|K|^{1/2}}{(2\pi)^{d/2}...
- 21.6k
Only top scored, non community-wiki answers of a minimum length are eligible