Questions tagged [multivariate-normal-distribution]
The multivariate normal distribution, is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. (Also called, multivariate Gaussian)
695
questions
1
vote
1
answer
15
views
How can I quantify uncertainty for a least squares estimator in a multivariate linear regression with covariance structure?
Suppose that we have
$$\mathbf{y}\sim\text{N}(\mathbf{X}\boldsymbol{\beta},\sigma^2\mathbf{M}\mathbf{M}'),$$ and let $\boldsymbol{\hat{\beta}}$ be the least squares estimator for $\boldsymbol{\beta}$. ...
- 1,829
1
vote
1
answer
58
views
Using linear combination of multivariate normal $X$
I have a question and its answer but I don't understand it. would you please explain it to me?
QUESTION:
Let $X$ follow a $N(0, 1)$ and $Z$ a random variable independent of $X$ following a $U(-1, 1)$. ...
- 13
0
votes
0
answers
65
views
MLE of multivariate normal distribution when the VCV matrix is full of equations
Short Version: Given a variance covariance matrix for my multivariate normal distribution where the entries are equations of other parameters, how do I find which of those parameter values maximizes ...
1
vote
0
answers
20
views
How to Handle Non-Multinormality in the Context of Exploratory Factor Analysis for Logistic Regression
I'm trying to follow the book A Step-by-Step Guide to Exploratory Factor Analysis with R and Rstudio, by Marley W. Watkins, and apply the principles in the book to a real-world data set. Ultimately, ...
- 5,808
0
votes
0
answers
23
views
meaning of empirical argument in mvrnorm
I'm using the mvrnorm function from MASS library, and I have a question regarding the ...
- 1,305
0
votes
0
answers
39
views
Is this the formula for the conditional covariance of normally distributed random variables? [duplicate]
Assuming $X$, $Y$, and $Z$ are normally distributed random variables, is it true that:
$Cov[X, Y | Z] = Cov[X, Y] - Cov[X, Z]Cov[Y, Z] / Var[Z]$
Could you provide a simple derivation?
- 47
0
votes
0
answers
23
views
How to compute a posterior for the parameter of a model?
I have quite the specific question over a model for which I am asked to compute a posterior. Here are all the details : Bayesian model .
For clarity purposes, let
$X=[x_1^{\top}, \ldots, x_T^{\top}]^{\...
11
votes
2
answers
337
views
Formulas, approximations, or bounds for $\mathbb{E}\left( \frac{X}{\lVert X \rVert} \right)$, $X\sim N(\mu, \Sigma)$?
In another question, I asked for $\mathbb{E}\left( \frac{X}{\lVert X \rVert} \right)$, in the case where $X \in \mathbb{R}^d \sim N(\mu, I_{d})$. Somebody posted an exact formula based on the symmetry ...
- 1,133
3
votes
1
answer
162
views
Bivariate normal covering circles and ellipses
I am looking at covering circles for cartesian coordinates given by independent bivariate random variables $X, Y \sim N(0, \sigma)$.
The radius of a circle that will cover proportion p of these ...
- 1,024
0
votes
0
answers
12
views
How do I adequately calculate the distribution of function of the multivariate normal distribution using the R mvtnorm package?
I am currently working on an analysis project involving simultaneous testing of several hypotheses. I intend to use the multcomp R package, which calculates the family-wise error rate for multiple ...
0
votes
0
answers
34
views
How to find $\mathbb{P}(\textbf{Y}^T\mathbf{A}\textbf{Y}>0)$ where $\mathbf{Y}$ is a vector of independent normal distributions? [duplicate]
Say $\mathbf{Y} \sim \mathcal{N}(\mathbf{\mu}, \mathbf{D})$ where $\mathbf{D}$ is diagonal and $A$ is any real matrix, how could you calculate $\mathbb{P}(\mathbf{Y}^T\mathbf{A}\mathbf{Y}>0)$?
In ...
2
votes
1
answer
126
views
Using Partial Correlations and Correlations to calculate each other's values
EDIT:
I have a correlation matrix with some known values and some unknown values, and i have a partial correlation matrix with the exact opposite known and unknown values. For example, my correlation ...
0
votes
0
answers
12
views
Can I sample from a multivariate normal when I can only compute matrix-vector products?
I want to sample from a distribution $\mathcal{N}(0, \Sigma)$ where all I have is the ability to calculate $\Sigma v$ for all $v$.
Is there any algorithm such that I can compute $Lu$ for any $u$ such ...
- 53
0
votes
0
answers
63
views
Expectation Maximization on Multivariate Gaussian Mixture Model for clustering
I have a dataset with 1000 observations and two features that define those N=1000 data points. Hence it is 1000*2 input matrix. I need to cluster them into k clusters.
I am not understanding the E-M ...
- 1
0
votes
0
answers
11
views
Calculating Mean Vector and Covariance Matrix of Mixture of Multivariate Normal Distributions [duplicate]
In an effort to better understand multivariate normal distributions, I am attempting to derive the mean vector and covariance matrix of multivariate random vector defined by a mixture distribution. ...
0
votes
0
answers
37
views
Find the conditional PDF of a multivariate normal distribution given a constraint [duplicate]
Problem to solve
We have a vector of random variables $\textbf{X}=(X_1,X_2)$ issued from a bivariate normal distribution. In particular, $\mu = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$, $\Sigma = \begin{...
0
votes
0
answers
28
views
corr(xz, yz) given all pairwise correlations
Suppose we have random variables $x,y,z$ where each random variable has mean 0 and standard deviation of 1. $Corr(x,y) = a$, $corr(x,z)=b$ and $corr(y,z)=c$
How do I go about calculating $corr(xz, yz)$...
- 1
0
votes
0
answers
7
views
Combining factors, represented as normal distributions, to one combined factor, normally distributed
I'm trying to combine the different factors that may affect running pace, such as GPS-measured distance, grade, terrain, heat and other factors (such as wind etc.). Each factor is represented as a ...
1
vote
0
answers
48
views
Using Copulas to find mutual information
I have two multidimensional datasets $X, Y$ of dimensions $m \times n$. Here $m$ is the successive measurements and $n$ is the data collected during each measurement. We can say each of $m$ are ...
- 117
3
votes
1
answer
515
views
Approximate the Fisher information matrix of a multivariate normal distribution
For $d\geq 2$, consider the d-dimensional multivariate normal distribution $\mathcal N(x|\mu,\Sigma)$ whose the log of density is given by
$$
l(x;\mu,\Sigma)=-\frac{d}{2}\log(2\pi)-\frac{1}{2}\log|\...
- 153
2
votes
1
answer
79
views
Covariance of quadratic form and a random vector of type $\mathbf{G}\,\mathbf{y}$
Assume that the $p \times 1$ vector $\mathbf{y}$ has multivariate Normal distribution with $\mathbb{E}[\mathbf{y}] = \boldsymbol{\mu}$ and $\mathrm{V}[\mathbf{y}] = \boldsymbol{\Sigma}$. Let $\mathbf{...
- 321
1
vote
1
answer
46
views
Deriving covariance of joint distributions of MVN [Linear Gaussian systems?]
Let $z$ ∈ R^L be an unknown vector of values, and $y$ ∈ R^D be some noisy measurement of z. We assume these variables are related by the following joint distribution
$p(z) \sim N(z|\mu_{z}, \Sigma_{z})...
- 111
0
votes
0
answers
11
views
Product of normal multivariate distributions [duplicate]
I read a book about statistics and machine learning, and can't understand assertion that: let $P(y) \sim N(y|\mu^*, \Sigma^*)$, i.e. multivariate normal distribution $p(y_1|\mu, \Sigma)$, where $\mu_1 ...
- 11
0
votes
0
answers
31
views
Multivariate normal imputation model Jomo R package
I have three questions about multivariate normal imputation using R package jomo. I modified the example in jomo to illustrate ...
- 172
1
vote
0
answers
43
views
Bayesian inference on model parameters from summary statistics alone
Consider quantities $y_1,y_2,\dots,y_p$, for the $j$th of which we have $n_j$ measurements $y_{1j},y_{2j},\dots,y_{n_jj}$. Unfortunately, I do not have access to the raw data $y_{ij}$ -- only to the ...
- 11
1
vote
0
answers
40
views
rjags - implementing a weighted multivariate normal distribution
I am trying to implement a weighted multivariate normal distribution in JAGS 4.3.0
Here is the code I used:
...
- 11
0
votes
0
answers
33
views
Coefficient of variation from independent estimates and multivariate normal
I'm estimating the coefficients of variation (CV) for features in a dataset. Each of the features is well represented by a normal distribution. I've done the estimate in two ways, both using bayesian ...
- 101
1
vote
0
answers
19
views
With mult-dimensional input vectors, what are the dimensions of the covariance matrix elements? (Gaussian Process)
I am trying to create a Bayesian Optimisation code with a Gaussian Process. My input data, $\vec{X}_i$ is 8-dimensional, where each dimension corresponds to a feature of my data,
$\vec{X}_i = [\...
- 111
5
votes
0
answers
105
views
Covariance as a function of explanatory variable
Suppose we have a collection of bivariate random variables $X_{1i}$ and $X_{2i}$ indexed by a continuous variable $t$ such that, for the vector ${\bf{X}} = (X_{1i} \ X_{2i})^T$ we can assume
\begin{...
- 61
0
votes
0
answers
20
views
What is meaning/geometric interpretation of the Expectation parameters of a multivariate gaussian (exponential family)?
According to page 17 of Statistical exponential families the expectation parameters for a multivariate gaussian are given by
$$
\boldsymbol{H}=\left(\mu,-(\Sigma+\mu\mu^T)\right)=(\eta,H)
$$
I am ...
- 1
2
votes
1
answer
93
views
Is there an analytical equation for the conditional entropy of a Gaussian random variable?
I am looking for a way to simulate ground-truth conditional entropy. Say I have $\mathbf{X}$ is a 3-dimensional multivariate Gaussian random variable. I am interested in computing the ground-truth of ...
- 255
3
votes
1
answer
74
views
Can I speed up the dmvnorm() function if I have already computed the inverse of the covariance matrix?
I am running code in R where I repeatedly evaluate the multivariate normal density of a high-dimensional object for thousands of sequential iterations. I am using dmvnorm() to do this. My ...
- 31
1
vote
0
answers
136
views
Multivariate Normal Bayesian Updating with Conjugate Priors but Non-Standard Likelihood
I am trying to solve for the posterior of two parameters $\theta_1$ and $\theta_2$. I have priors $N(\mu_1, \sigma_1^2)$ and $N(\mu_2, \sigma_2^2)$ for $\theta_1$ and $\theta_2$ respectively, where $\...
- 11
6
votes
5
answers
226
views
Finding $\mathbb E(Y_1^2Y_2^2)$ when $(Y_1,Y_2)$ is normal
Let $$Y =\begin{pmatrix} Y_1 \\ Y_2 \end{pmatrix}
\sim N(0, \Sigma) \quad \Sigma=\begin{pmatrix} \sigma_{11} & \sigma_{12}\\ \sigma_{21} & \sigma_{22} \end{pmatrix}$$
Show that $$\mathbb E(Y_1^...
- 255
0
votes
1
answer
60
views
Squared norm of linear system proportional to Multivariate Gaussian log-density?
I am reading https://epubs.siam.org/doi/10.1137/140964023, and I got confused by this part:
In the above, it is assumed that $m \geq n$.
If $m = n$, I can see how the above works.
$|| J \theta - y||^...
- 31
1
vote
0
answers
57
views
Completing the square and marginalizing a multivariate Gaussian [closed]
Edit: This question has been closed for being unrelated although I see similar questions posted here with the same objective, yet not with enough detailed answers or not exactly what I am looking for (...
- 31
3
votes
1
answer
144
views
Gaussian Markov Random Fields - Conditional distribution from jointly gaussian with given precision matrix
Suppose I have jointly normal random vectors $[\bf{v_1}, \ldots, \bf{v_K}]$' with mean $ \bf{M}$ and joint block tridiagonal precision matrix $ \bf{P}$:
$$ \bf{M}= \begin{bmatrix}\bf{\mu_1} \\\ldots \\...
- 189
2
votes
1
answer
111
views
How can I sample a multivariate normal vector that satisfies a linear equality constraint?
Let $X \sim N_n(\mu, \Sigma)$, such that $AX=b$ where $A$ is a ($p \times n$) matrix, with $p \ll n$. How can I efficiently sample from this distribution?
I've seen techniques using elliptical slice ...
- 53
1
vote
2
answers
101
views
First derivative of multivariate normal density with exchangeable correlation structure
As part of a proof, I need to take the first derivative of the log of the following multivariate normal density: $(2\pi)^{-k/2} |\Sigma|^{-1/2} \exp\left(\frac{-1}{2} x'\Sigma^{-1}x\right)$.
In this ...
- 714
0
votes
0
answers
25
views
Is there an alternate estimator for a sample covariance matrix when n < p such that the estimator is not singular
Let's say I have $n$ samples which are vectors of length $p$. I know that the $p \times p$ sample covariance matrix is singular if $n \leq p$. Is there another estimator for the covariance that ...
- 97
2
votes
2
answers
69
views
Calculate joint distribution from marginal distributions
I am struggling with the following problem: $X_1, X_2 \sim N(0, 1)$ are independent random variables. Let $Y_1 = \frac{1}{\sqrt{2}}(X_1 + X_2)$ and $Y_2 = \frac{1}{\sqrt{2}}(X_1 - X_2)$. Show that $...
- 123
2
votes
1
answer
38
views
Pointwise Confidence Regions in Gaussian Process
In Rasmussen's Gaussian Processes for Machine Learning, the joint distribution of noisy function observations, $y=f(x)+\epsilon$, at $x$ and noiseless function evaluations, $f^\star$, at unseen points,...
- 313
1
vote
0
answers
21
views
Bayesian reparametrization are they equivalent?
Suppose that we are in a Bayesian context, we we have the following matrix $n,$ $K\times K,$ as parameter, and we assume that
$$n_{ij}\sim Pois(w*w_{ij})$$
where $w\sim Gamma(N+1,1)$ and $w_{ij}$ is ...
- 2,152
0
votes
2
answers
52
views
Calculating the distribution of the sum of the squares of the predictors in linear regression
I'm calculating the distribution of the sum of the squares of the components of the MLE $hat{\beta}$ in linear regression with normal errors. We are assuming that $\beta = 0$. The distribution of the ...
- 206
5
votes
1
answer
108
views
Non-Uniform Spherical Distributions
Suppose $X_i\overset{\text{iid}}{\sim} N(0,1)$, and define the random vector $\mathbf{X}=(X_1,\ldots,X_n)$. Then the normalized vector $\mathbf{Z}:=\frac{\mathbf{X}}{\|\mathbf{X}\|_2}$ is uniformly ...
- 153
0
votes
1
answer
60
views
Conditional Expectation of Multivariate Normal
I'm trying to calculate a good mean shrinkage parameter for a custom quadratic discriminant analysis (QDA), and I ran into a math problem. Suppose $X=(X_1, X_2, \ldots, X_k)^T\sim{\mathcal{N}(\textbf{...
- 45
0
votes
0
answers
51
views
Find the conditional distribution from joint normal distribution with vec operators
I have two random matrices one on the top of the other:
$ \begin{bmatrix}\boldsymbol{B_1} \\ \boldsymbol{B_2} \end{bmatrix}$.
and they are both of dimension $k \times N$. I have that:
$ vec\begin{...
- 189
1
vote
3
answers
66
views
Distribution of specific column of a random variable after repeated sampling
Suppose $\mathbf X$ is normally distributed with mean $\mu$ and standard deviation $\sigma$. After sampling $n$ samples, we repeat the sampling process $m$ times and the sampling data is stored in an $...
- 498
0
votes
0
answers
51
views
Kullback-Leibler divergence between product of independent gaussians and a multivariate normal distribution
what's the correct way to quantify the loss of information we have when we approximate the likelihood from multivariate normal distribution with a full covariance matrix to a product of univariate ...
- 325
0
votes
0
answers
133
views
Scipy.stats.mvn.mvndst function for 3d inputs
I have some code that I'm trying to convert from working with floats to numpy arrays for performance reasons. I have it down to one last function, which is below and requires the ...
- 143