Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [covariance-matrix]

A $k\times k$ matrix of covariances between all pairs of $k$ random variables. It is also called variance-covariance matrix or simply the covariance matrix.

1
vote
0answers
17 views

Modelling latent variables: lavaan warning: covariance matrix of latent variables is not positive definite [on hold]

I have 3 independent variables (linguistic, onk2_sc3 and dsk2_sc3) and one dependent variable (Reading) in SEM multigroups (3 groups) comparison: ...
1
vote
0answers
17 views

Apply K-means to the columns of the covariance matrix

In Section 5.3 of the paper distilling the knowledge in a neural network, it says we apply a clustering algorithm to the covariance matrix of the predictions of our generalist model, so that a set ...
0
votes
0answers
13 views

Covariances and correlations in curve fitting

I have a set of data that I am trying to curve fit and I'm ultimately interested in the errors on my fit coefficients. I take my errors on each fit coefficient as the on-diagonal elements of the ...
2
votes
1answer
22 views

How to show sample correlation is sample covariance for standardized values?

Given a matrix $X$ and the resulting sample correlation matrix $R$, consider the standardized observations: $$\frac{(x_{jk} - \bar x)} {\sqrt{S_{kk}}} \quad k=1,2,...,p \quad j=1,2,...,n$$ Show that ...
4
votes
2answers
404 views

Inverse of the covariance matrix of a multivariate normal distribution

Is the covariance matrix of a multivariate normal distribution always invertible?
5
votes
1answer
91 views

Condition on the covariance matrix of a gaussian process needed to have the Markov property

Let suppose to have a realization $\mathbf{X}=(\mathbf{X}_1,\dots, \mathbf{X}_n)$, where $\mathbf{X}_i \in \mathcal{R}^d$, from a $d-$variate Gaussian process. Let also suppose that $E(\mathbf{X}_i)= ...
0
votes
0answers
23 views

Common covariance matrix explanation (LDA and QDA)

I'm looking for a layman's explanation of the "common covariance matrix" assumption in LDA because I don't think I understand it. I understand that a common covariance matrix (as assumed in LDA for ...
0
votes
0answers
12 views

Represent Mean-Squared-Prediction error as function of covariance (or Fisher) matrix

Given a simple linear model: $$ y_i = x_i^T \beta + \epsilon_i $$ For simplicity, $\epsilon_i$ is Gaussian iid with variance $\sigma_e^2$, then the solution for $\hat{\beta}$ is given via Ordinary ...
0
votes
0answers
8 views

Create covariance with blocks structure (artificially)

I would like to create some artificial variables that their covariance matrix will have blocks structure. Any idea how to do this? By saying "their covariance matrix will have blocks structure" I ...
0
votes
0answers
15 views

When the elements of first basis are always positive for PCA?

I am computing the PCA projection matrix of some data. I notice that the elements of first basis vector (corresponding to the highest eigenvalue) are always positive. My data is real and contain both ...
1
vote
0answers
29 views

Residual variance-covariance matrix in vector autoregression

It's my understanding that the general form of a variance-covariance matrix has variance terms on the diagonal and covariance terms on the off-diagonal. I have seen in multiple references (for ...
0
votes
0answers
14 views

Generating correlated data using numpy while controlling multicollinearity

I am using the following code (adopted from the code in this post). I have no problems with the code. My question is that if with this code I can create or prevent multicollinearity among the ...
1
vote
1answer
25 views

Calculating partitioned covariance matrices using R

Let's say my data consists of 25 observations of four features, which are in groups of 2. So we'll call my variables $x_1, x_2, y_1, y_2$. We have a partitioned sample mean vector given by $\begin{...
0
votes
1answer
28 views

Calculating variance of process with time-varying variance

This is a question stemming off a previous post I had regarding calculating portfolio volatility. For a portfolio consisting of multiple assets, I understand that there are multiple ways to calculate ...
2
votes
0answers
14 views

Causality in variance with a BEKK model

I am using a BEKK model in the following form, $$H_t=C^\ast{C^\ast}^\prime+\sum_{i=1}^{m}{A_i\varepsilon_{t-i}\varepsilon_{t-i}A_i^\prime+\sum_{j=1}^{s}{B_jH_{t-j}B_j^\prime}}$$ I first start with a ...
2
votes
1answer
34 views

Calculating portfolio volatility from portfolio returns vs. from covariance matrix

I'm having trouble understanding the difference in calculating portfolio volatility via the portfolio returns vs. via the covariance matrix. To be more specific: I understand that on the individual ...
2
votes
0answers
53 views

Covariance structures in glmmTMB for temporal autocorrelation

I'm running a zero-inflated, mixed-effects negative binomial model with the glmmTMB package in R. My current format: ...
0
votes
0answers
25 views

Explanation of covariance matrix of polynomial parameters [duplicate]

I'm asked to find the covariance matrix of $\alpha$, $\beta$, and $\gamma$ for: $$y_i=\alpha+\beta(x_i-\bar{x})+\gamma[(x_i-\bar{x})^2-\zeta^2]+\epsilon_i$$ where all the errors have equal variance $...
7
votes
0answers
131 views

Rao-Blackwellization in variational inference

The Black box VI paper introduces Rao-Blackwellization as a method to reduce the variance of the gradient estimator using score function, in section 3.1. However I don't quite get the basic idea ...
2
votes
0answers
26 views

Covariance of a covariance matrix [closed]

Given that covariance matrix, why is the covariance of Y and Z in this case "(-1 0)" or what would be the covariance of X and Y?
2
votes
1answer
48 views

Simplification of an expectation

While attempting to simplify a combination of expectations, I'm stuck at a particular term whose simplification I'm unable to deduce. The term to be simplified is: $\mathbb{E}[X^{T}F^{T}FX]$ where $...
0
votes
0answers
14 views

Checking equality of covariance matrices using Box's M test in multifactor MANOVA

With only one factor (independent variable) in multivariate analysis of variance (MANOVA), Box's M test can be used to check the equality of covariance matrices. ...
2
votes
1answer
24 views

Eigenvectors for correlation and covariance matrix PCA

I know the generally reasons of using correlation matrix vs a covariance matrix when doing PCA (and visa versa) however when thinking about the eigenvectors (principal components of the data) of each ...
0
votes
1answer
36 views

Sum of product as product of sums

Assuming I have two random non-independent vectors $A,B$ which are within [-1,1]. I want to approximate their sum of product by product of sums (everything is a dot product), i.e. $\sum_{i=1}^NA_iB_i ...
1
vote
1answer
19 views

relationship between correlation, covariiance and conditional distribution

What are the relationships between correlation and conditional distribution. For instance, given three dependent variables, X1, X2 and ...
0
votes
0answers
23 views

What type of transformation for Latent Growth Curve analysis with lavaan when observed variances are over 40,000,000?

Edit: I believe a linear transformation is just fine - divide each number by 100 for example. Just make sure you change back your units when you are interpreting the output! In fact, when I do this, ...
0
votes
0answers
22 views

Is Cov(X,Y|Z).x always positive? (with X,Y,Z, normal random vectors and x>0)

Let x be a vector of positive values, we know that for multivariate normal distributions of X, Y and Z, $Cov(X,Y|Z)x=(\Sigma_{XZ}-\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YZ})x$ does not depend on the given ...
0
votes
0answers
21 views

Confidence interval of fit parameter

First of all: I am quite new to statistics, so please don't tear me apart. ;-) I am currently trying to calculate a confidence interval for a fit of a curve to measured data points. I have done this ...
0
votes
1answer
30 views

Explain coefficients in a multiple regression are the same as in simple regressions

Given the matrix of covariances, $M$ (below), three variables $X, Y, Z$, and a multiple regression $\hat{Z} = \frac{5}{4}X + \frac{4}{5}Y$: $$M=\begin{bmatrix}16 & 0 & 20\\0 & 25 &...
0
votes
0answers
28 views

Does the variance of a sum equal a double sum of covariances? [duplicate]

Say I have a collection of $n$ random variables $X_i$ that don't necessarily have any special properties like independence or identical distribution. Is it true in general that $\text{Var}\left(\sum_{...
0
votes
1answer
17 views

Correlation between two multivariate measures

I'm reading a paper, but I'm with a problem. The authors say: Let $\boldsymbol{X} = (X_1, \ldots, X_p)^T$ be a vector $m \times 1$ whose the estimative of variance is proportional to $\boldsymbol{\hat{...
0
votes
0answers
16 views

What is the difference between Residual Covariance Matrix in Linear Weighted Least Square and Non-Linear Weighted Least Square?

We have these two models : z = h(x) + e, r1 = z - z_hat = h(x) - h(x_hat) z = Bx + e, r2 = z - z_hat = Bx - Bx_hat in the first equation x is estimated by nonlinear weighted least square and second ...
0
votes
1answer
16 views

Covariance of sums of pairs of correlated variables

Take two vectors of normally-distributed random variables $\mathbf{x} = (x_1, x_2, \ldots x_n)$ $\mathbf{y} = (y_1, y_2, \ldots y_n)$ where the covariance of each pair $(x_i, y_i)$ is known, $\...
1
vote
0answers
27 views

Simplifying a covariance expression

Let $I=\begin{pmatrix}I_1\\\vdots\\ I_n \end{pmatrix}$ be a random vector, and $\Omega$ and $\Omega_I$ two random variables. I am trying to simplify the following equation (which worth $\frac{\rho_{I\...
0
votes
0answers
13 views

How to apply Lagrange Multiplier to a matrix?

I am learning about the Lagrange Multiplier and I see how to apply it to a set of equations but I don't know how to apply it to matrices. Suppose I have: ...
0
votes
0answers
28 views

Valid covariance matrix?

I am trying to replicate some results from this well-cited paper evaluating various methods to determine the rank of a matrix. They give several covariance matrices, and then sample from a Gaussian ...
0
votes
0answers
8 views

how can I interpret a long-run covariance matrix?

can I use the long-run covariance matrix to examine multicollinearity between my independent variables? and if yes, what how can I interpret matrix for determining Multicollinearity?
1
vote
1answer
31 views

Conditional covariance of a multivariate normal vector

We know that the conditional variance of a multivariate normal vector $(X,Y)$ is the Schur complement: $$V(X|Y=(y_1,...,y_n))=\Sigma_{XX}-\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX}$$ I have the intuition ...
1
vote
1answer
64 views

Use the RBF kernel to construct a positive definite covariance matrix

A commonly used kernel in Gaussian processes is the RBF kernel: $$ \kappa(x,x') = \exp\left(-\frac{|| x-x'||^2}{2\sigma^2}\right) $$ In the context of a Gaussian process, a kernel is used to ...
6
votes
1answer
177 views

Are all symmetric matrices with diagonal elements 1 and other values between -1 and 1 correlation matrices?

A question for the statisticians and other math lovers: Are all symmetric matrices with diagonal elements 1 and other values between $-1$ and 1 correlation matrices?
1
vote
0answers
45 views

Why that distribution is normal with those parameters?

Mi question is the following. I have two independent 2-dimensional normal distributions with the same mean vector and different covariance matrixes, lets say $X_1 \sim N_2( \mu, C)$ and $X_2 \sim N_2(...
1
vote
1answer
29 views

Analytical solution to the covariance between a continuous and a categorical variable

Let $X$ be a continuous variable with mean $\mu$ and $Y$ be a categorical variable with event probability vector $\mathbf{p}$. I am trying to calculate $\operatorname{Cov}(X, Y)$. I have the solution ...
0
votes
1answer
20 views

How to interpret a given 2D co-variance matrix?

I am trying to solve a problem regarding revision for my Big Data module. I have two main questions. 1) Given a predefined co-variance matrix: A cluster of points is distributed in a two-...
2
votes
0answers
34 views

Why mean in Gaussian Process is not so important? [duplicate]

Source of my doubt is the section 2.7 of GPML book by Rasmussen, an screenshot of the book is attached below. Much of my confusion is clarified by this discussion. If mean of GP is not estimated and ...
2
votes
1answer
55 views

How to calculate the covariance matrix for a categorized variable?

Let $X$ and $Y$ be jointly distributed as a multivariate normal with the following parameters: $$ \mu_{XY} = \begin{bmatrix} 0 \\ 0.2 \end{bmatrix} \qquad \Sigma_{XY} = \begin{bmatrix} 1 & 0.05 \...
0
votes
0answers
41 views

Comparing species correlations between species in two habitats

I have two community data sets (samples as rows, species as columns, populated with abundance). This data comes from two habitats/sites, with differing numbers of samples at each site. What I want ...
1
vote
1answer
82 views

Deriving the sampling distribution of MLE for Normal distribution

Let $X_1,\ldots,X_n$ be an observed random sample from $N_p(\mu, \Sigma)$. I know that the MLE of $\Sigma$ is $\frac{1}{n} \sum_i^n(X_i -\bar X)(X_i -\bar X)^T$, which is biased. We define $S = \...
1
vote
1answer
25 views
0
votes
0answers
15 views

How to model multivariate Gaussians with prior knowledge on covariances?

When modelling multiple variables with multivariate Gaussian, we often assume that the covariance matrix is diagonal to reduce the complexity and computation cost of the problem. In such cases, if we'...
0
votes
0answers
21 views

Autocorrelated time series analysis

I have a set of observations $X_1, ...,X_n$. They can be generated with a simple Markov chain with $k$ states - in theory, in practice there are covariances between observations from different time ...