A $k\times k$ matrix of covariances between all pairs of $k$ random variables. It is also called variance-covariance matrix or simply variance matrix.

learn more… | top users | synonyms (1)

1
vote
0answers
52 views

What does it mean when all eigenvalues of a correlation matrix are zero?

I am applying PCA to a dataset. After calculating eigenvalues of the covariance matrix I noticed that all eigenvalues are equal to zero. Can anyone please explain to me what does it mean when all ...
0
votes
3answers
68 views

Express correlation matrix of $X$ in terms of $X^{T}X$ (in the OLS context)

In least squares estimation where $Y = \beta X$, how can we find the correlation matrix of $X$ in terms of $X^{T}X$? It seems that $X^{T}X$ is very close in structure to the correlation matrix, but ...
4
votes
2answers
361 views

Why does the correlation function in R, cor() return a matrix with fewer rows that you started with?

I have a matrix $G$ in which I would like to compute the correlation matrix of. The matrix in R looks like: ...
0
votes
1answer
28 views

How to get the determinant of a covariance matrix from its diagonal elements

I am trying to implement a speaker recognition system in MATLAB. I am using Gaussian Mixture Models (GMM) for speaker modelling and maximizing the posterior probabilities for classification. The ...
0
votes
0answers
11 views

SSCP vs correlation matrix [duplicate]

Lets say I have a matrix X, where each row is a subject and each column is a variable. I have a few questions about manipulating this matrix, and what the result is: If I calculate X'X, am I correct ...
1
vote
1answer
53 views

Variance of $a^TX$ for MVN X

How do you show that the variance of $a^TX$ for multivariate normal X is $a^T\Sigma a$? I have $V(a'X)=E(a'X-E[a'X])^2$, but it seems like the dimensions get messed up or something after that. So I'm ...
0
votes
0answers
11 views

Distribution of a transformation of a inverse Wishart distributed covariance matrix

Suppose to have a covariance matrix $\boldsymbol{\Sigma}$ of dimension $n \times n$ with $\boldsymbol{\Sigma} \sim IW(\nu, \mathbf{I}_{n} )$. Then $$ f(\boldsymbol{\Sigma}) \propto ...
1
vote
0answers
22 views

Do I distort information stored in covariance matrix if I normalize the matrix to range of $[-1 , 1]$

I am using covariance matrix to maximize mutual information between samples that I selected and sample that I did not selected. The optimization algorithm that I use is sfo_greedy_lazy. It computes ...
1
vote
0answers
30 views

Is the covariance matrix the equivalent of standard deviation for a 2d matrix?

I want to measure the scale of a 2d matrix that contains the coordinates (x,y) of a set of point. I know that the standard deviation of the x coordinates of these points is the scale of the x ...
1
vote
0answers
20 views

Practicality of sparse inverse covariance matrix assumptions

For a set of $p$ datapoints in $m$ dimensional space, if the features are packed in a $p\times m$ matrix $X$, then $C = XX^T$ is the covariance matrix and $K = C^{-1}$ is the inverse covariance ...
3
votes
1answer
39 views

How to compute the variances of PCA-transformed variables?

In principal component analysis (PCA), what is the covariance matrix in the new space of reduced dimensionality, i.e. the covariance matrix after multiplying the data by the eigenvector matrix? I ...
3
votes
1answer
68 views

Correlated features in dataset

I undestand in general that it is important to take correlational structure into account while applying almost any statistical techniques. First question - could you help with the examples why it is ...
0
votes
0answers
18 views

Estimating residual variance with covariance matrix and mean vector

I'm currently working on estimating a multiple regression solely with the covariance matrix and the mean vector. The example I'm working with gives the following formulas for the bivariate case of ...
1
vote
1answer
58 views

Find covariance if given mean and variance

I have a signal x that I want to classify in one of the classes A and B in which the means are Ma=[0.5,0.6] and Mb=[2,2] and with variances ...
2
votes
1answer
43 views

Antisymmetric components in Gaussian kernel

In Bishop's Pattern Recognition and Machine Learning, he states on page 80 in reference to the squared Mahalanobis distance $(\mathbf{x} - \boldsymbol{\mu})^\top\boldsymbol{\Sigma}^{-1}(\mathbf{x} - ...
3
votes
2answers
40 views

Converting several t-statistics to a single F-statistic?

I have dummy coded a categorical regression, and ran OLS to get parameter estimates, along the lines of: $$ y= \left( \begin{array}{ccc} 1 & 0 &0\\ 1 & 0 & 0 \\ 1 & 1 & 0 ...
4
votes
1answer
57 views

Ill-conditioned covariance matrix in GP regression for Bayesian optimization

Background and problem I am using Gaussian Processes (GP) for regression and subsequent Bayesian optimization (BO). For regression I use the gpml package for MATLAB with several custom-made ...
7
votes
0answers
100 views

How to test if a cross-covariance matrix is non-zero

The background of my study: In a Gibbs sampling where we sample $X$ (the variable of interests) and $Y$ from $P(X|Y)$ and $P(Y|X)$ respectively, where $X$ and $Y$ are $k$-dimensional random vectors. ...
2
votes
1answer
39 views

Dispersion of points on 2D or 3D

Suppose that we have a set of points on a line. The amount of dispersion can be measured by standard deviation in this case. My question is, is there something similar for higher dimensions? For ...
2
votes
1answer
59 views

What is the difference between Covariance matrix and the variance-covariance matrix

I am bit unsure whether there exist any difference at all.. Google tells me that variance-covariance matrix is the matrix where the variance is written in the diagonal of the matrix, and the other ...
2
votes
0answers
34 views

Ill-conditioned covariance matrices in EM

I am currently working with the Expectation-Maximization algorithm. I have some pre-clustered sets of 3D points and am trying to run the algorithm. However I've seen that most of my covariance ...
0
votes
0answers
17 views

Estimation of covariance matrix using 2D power spectral density

1. What I need : A full Covariance matrix, $\Sigma$ $(N^2 \times N^2)$, in order to apply my algorithm. PSD - Power Spectral Density (and not positive semi-definite) 2. What I have: a) An image N ...
0
votes
0answers
20 views

Inference for multivariate normal when the sample covariance matrix is not invertible

Let $X_1, \ldots, X_n$ be a random sample from $N_d(\mu, \Sigma)$ and let $S=\frac{1}{n}\sum_{i=1}^n(X_i-\bar{X})(X_i-\bar{X})'$ denote the sample covariance matrix. If $S$ is invertible, it can be ...
0
votes
1answer
31 views

Deriving $\text{Var}[\boldsymbol{\hat{\beta}}]$

I have already read How to derive variance-covariance matrix of coefficients in linear regression. Assume we're working with the usual simple regression model, $\mathbf{Y} \in \mathbb{R}^N$, $X \in ...
1
vote
0answers
5 views

Generate sample process with CrossSectional Correlation and autoregressive structure

How can you generate a sample multidimensional time series $X_{t,i}$ for $t \in \{1,2,...,T\}$ and $i \in \{1,2,...,M\}$ where: $E[X_{t,i}] = 0$ $E[X_{t,i}X_{t,j}] = \Sigma_{i,j}$ ...
1
vote
2answers
23 views

Stationary covariance function three times continuously differentiable with finit support?

Suppose we have an Euclidiean space $\mathbb{R}^p$ and Gaussian process with covariance function of the form $k(x_1, x_2) = k(\|x_1 - x_2\|), x_1, x_2 \in \mathbb{R}^p$. I am looking for a covariance ...
0
votes
0answers
17 views

Log-likelihood with time series covariance matrix

I'm doing time series modeling, where I need to calculate the log likelihood as a function of $\sigma_v^2$ and $\rho$ for the model $Y = X\beta + e$. I'm using the generalized least square estimates ...
1
vote
1answer
70 views

Q-mode vs. R-mode PCA

I have some doubts on Q-mode and R-mode principal component analysis (PCA). I've read from different sources that: Q-mode PCA is equivalent to R-mode PCA of the transposed data matrix! Q-mode PCA ...
1
vote
1answer
53 views

Mahalanobis distance with LDL decomposition

I've got an extended Kalman filter with innovation covariance defined as $\mathbf{W}=\mathbf{H}\mathbf{P}\mathbf{H}^\textrm{T} + \mathbf{R}$. I want to know the squared Mahalanobis distance $\|z\|^2$ ...
0
votes
0answers
13 views

Computing design matrix from covariance matrix

Suppose I have the following regression model: $Y = b_1 \times T + b_2 \times Z + b_3 \times T\times Z + \epsilon$ where T is a randomly assigned treatment condition and Z is some covariate. I want ...
6
votes
3answers
397 views

Is every correlation matrix positive definite?

I'm talking here about matrices of Pearson correlations. I've often heard it said that all correlation matrices must be positive semidefinite. My understanding is that positive definite matrices must ...
2
votes
0answers
80 views

Interpreting numerator of an F-statistic

"the numerator expresses a measure of squared distance standardized by the covariance matrix" That is the author's interpretation on page 97 in Introduction to Linear Regression Analysis by ...
0
votes
0answers
67 views

How can I calculate the root mean square error (RMSE) for two covariance matrices?

I want to compare different methods of estimating the covariance matrices on the basis of RMSE and will recommend having the minimum RMSE. I have a sample of, say, 356 weekly observations of 10 ...
0
votes
0answers
29 views

Variance estimation of GLM coefficients

I'm having trouble understanding the relationship between the variance of the GLM coefficients and the estimated observed Hessian. In the textbook I'm using it's stated "An obvious and suitable choice ...
0
votes
0answers
31 views

error covariance matrices of (vector) MMSE & LMMSE

Consider estimating $X$ based on observation of $Y$, with an estimator $\hat{X}$. The estimation error is $Err\triangleq\hat{X}-X$. If $X$ is a scalar R.V., the expectation of the squared estimation ...
0
votes
0answers
20 views

Scaling a covariance matrix for different time periods?

I have data where covariance matrices are calculated based on daily data. With these covariance matrices, we want to scale so they work for weekly/monthly/yearly time periods. The formula that I have ...
1
vote
0answers
39 views

How to calculate covariance matrix weighted by elements, not by rows or columns?

I want to calculate weighted covariance matrix where the weights are given to each element (entry, cell) of the original data matix, and not to rows (as done by cov.wt in ...
2
votes
1answer
26 views

On the definition of covariance matrix

I am having a misunderstanding with the definition of the covariance matrix from the wikipedia page on the covariance matrix. It says the covariance matrix $\sum$ is equivalent to where It ...
1
vote
1answer
45 views

Why does this covariance matrix have a rank of $n$?

I'm reading this paper, which contains the following covariance matrix: In the example there are six forecasters who estimate some quantity, and then we look at the covariances of those estimates. ...
3
votes
1answer
79 views

Computing covariance matrix from the given variances?

How can I obtain the covariance matrix when all the variances of the variables are known? This is from the paper http://www.cs.berkeley.edu/~jordan/papers/CSD-04-1330.pdf $$V_1 \sim ...
1
vote
0answers
35 views

Units of measure of the first principal component

I performed a PCA on credit default swaps spreads and I want to predict the first principal component scores. My two doubts are: If I use the covariance matrix in my PCA I'll have a predict first ...
7
votes
1answer
180 views

Why is the rank of covariance matrix at most $n-1$?

As stated in this question, the maximum rank of covariance matrix is $n-1$ where $n$ is sample size and so if the dimension of covariance matrix is equal to the sample size, it would be singular. I ...
0
votes
0answers
20 views

Kernel and Mercer Kernel

Kernel is a function that takes vectors as input and returns the dot product of the vectors in the feature space (possibility a higher-dimensional space). Mercer's Theorem tells us whether or not a ...
5
votes
1answer
578 views

Ethics of publishing covariate matrix

I am planning to submit a paper with a Structural Equation Model applied into a dataset built with measurements from schizophrenic patients. In the spirit of promoting an open research culture I've ...
0
votes
0answers
18 views

How to Properly use Variance Stabilizing Transformations (With PCA)

Is there a general etiquette to using Variance Stabilizing Transformations? Is the response variable transformed along with the predictors or is it just the response? Once the transformations have ...
3
votes
0answers
65 views

Distributions of eigenvalues of random matrices: what can they be used for in data mining?

I've accidentally come across some papers discussing distributions of principal components of the sample covariance matrices. An example of such a paper is Johnstone, 2001, On the distribution of the ...
0
votes
0answers
58 views

How can I use PCA to estimate the variance-covariance matrix?

I am working on alternative ways for the estimation of variance-covariance matrices. For this I have already estimated the sample variance-covariance matrix, single index covariance matrix. I also ...
1
vote
1answer
17 views

What is mean by single numerical measure of variability in multivariate observations?

Suppose there is matrix $X_{nxp}$ representing n p-variate observations. We know that the variance-covariance matrix $S$ for the given observations $X$ is way of quantifying the variability in $p$ ...
0
votes
1answer
41 views

Mean Squared Error (MSE) equivalence to trace + bias on multivariate case

I'm trying to show that: $$ MSE(\theta)\equiv E[(\hat{\theta}_n-\theta)'(\hat{\theta}_n-\theta)]=tr(V(\hat{\theta}_n))+Bias(\hat{\theta}_n)'Bias(\hat{\theta}_n) $$ where ...
0
votes
0answers
11 views

Do I use GLM or GEE. Determine the relationship of a continuous covariate on a 2x2 (within Subjects) interaction?

I have a 2x2 within subjects design with a continuous covariate. This covariate significantly alters the 2x2's interaction. I need to Determine the slope of the covariate's influence on the ...