A $k\times k$ matrix of covariances between all pairs of $k$ random variables. It is also called variance-covariance matrix or simply variance matrix.

learn more… | top users | synonyms (1)

2
votes
1answer
28 views

A measure of “variance” from the covariance matrix?

If the data is 1d, the variance shows the extent to which the data are different from each other. If the data is multi-dimensional, we'll get a covariance matrix. Is there a measure that gives a ...
0
votes
0answers
21 views

The covariance matrix is not positive definite in a classification task in Matlab

I am trying to perform a simple classification task in Matlab. I have an NxM matrix F with rows representing the samples and column representing the features (that is my training set). This is fed to ...
0
votes
0answers
13 views

Recursive least squares with forgetting factor - parameter covariance

The recursive least-squares algorithm equipped with forgetting factor is summarized as \begin{array}{l} \hat \theta \left( t \right) = \hat \theta \left( {t - 1} \right) + L\left( t \right)\left[ {y\...
0
votes
0answers
8 views

Whitening transform for the case of two stochastic processes

The classic whitening transform allows us to find a linear transformation for a given random process X, yielding a new process Xw with a unity Covariance matrix. Given an extended problem with two ...
11
votes
6answers
463 views

Why are symmetric positive definite (SPD) matrices so important?

I know the definition of symmetric positive definite (SPD) matrix, but want to understand more. Why are they so important, intuitively? Here is what I know. What else? For a given data, Co-...
-1
votes
0answers
11 views

How to apply different values for input noises in GPML toolbox?

As you probably know, GPML toolbox accepts only one value for noise in both white noise covariance function and likelihood. Actually in my case, each input data has its own value for noise (16 ...
2
votes
0answers
15 views

Algorithm for mixed-effects models with 100 random effects

I am wondering if there is any algorithm can estimate a mixed-effects model with 100 random effects, i.e., the covariance matrix $\boldsymbol D$ for random effects is 100$\times$100. I tried the ...
2
votes
0answers
26 views

Is autocorrelation not worth addressing with small N?

Consider a simple regression context in which there is a small set of response values, $Y$, and corresponding dates, $X$. (For simplicity, we can assume the dates are equally spaced.) We would like ...
0
votes
0answers
13 views

How to propagate scaling uncertainties into a covariance matrix?

Say I have a covariance matrix $C$ given in a certain unit (say, kg^2), corresponding to measurements made in this unit. All measurements are done in this unit. Say, further, that I want to convert ...
0
votes
0answers
17 views

How do I calculate (or approximate) the covariance matrix of a multivariate Gaussian distribution with only the variances of the components?

With the constraint that all components sum to a given specific real number; The Mean vector is also known; No sample available; correlations between any two of the components is unknown.
0
votes
2answers
48 views

Covariance Matrix for Time Series

I'm trying to investigate how events affect the stock market through econo-physics and I came across a paper that uses the co-variance matrix. What I don't understand is how such a matrix can be ...
0
votes
0answers
13 views

Estimating correlation(covariance) matrix when fitting a copula using R copula package [migrated]

I have a question about the R package copula. When using fitCopula to fit a copula to data, ...
9
votes
2answers
342 views

Are a sum and a product of two covariance matrices also a covariance matrix?

Suppose I have covariance matrices $X$ and $Y$. Which of these options are then also covariance matrices? $X+Y$ $X^2$ $XY$ I have a bit of trouble understanding what exactly is needed for ...
0
votes
0answers
19 views

RSS of correlated variables

Suppose that I have the variance-covariance matrix for a 3D point $$ c = \begin{bmatrix} S_{X} & S_{XY} & S_{XZ} \\ S_{XY} & S_{Y} & S_{YZ} \\ S_{XZ} & S_{YZ} & S_{Z} \end{...
0
votes
0answers
21 views

Expectation of infinite sum of random wishart matrices

I have problem figuring out how to find the expectation for an infinite sum of random matrices. More explicitly, my problem is: Let $\mathbf{S}_i$ be the maximum likelihood estimator of the sample ...
1
vote
2answers
64 views

Why the inverse of the covariance matrix is equal to the original covariance matrix in Seemingly Unrelated Regression?

In Zellner 1962 p350-351, you may see (2.4) & (2.6), and you can also verify that the Sigma(c)=Sigma(c)^-1. Why the inverse of the covariance matrix is equal to the original covariance matrix in ...
0
votes
0answers
13 views

Existence of data when mean vector and covariance matrix (p.s.d.) is fixed? [duplicate]

I would like to prove that for any positive semidefinite matrix $\Sigma_{n\times n}$ and any vector $\mu_{n\times1}$, are there always data points (no matter how many, as long as finite) whose ...
1
vote
0answers
40 views

The effect of non-positive-definite covariance matrix (in $p>n$ case) on PCA

Gene data has large number of dimensions as compared to samples. This leads to a non-positive-definite covariance matrix. In R when I try to use princomp which does ...
-6
votes
1answer
210 views

How to define the covariance for a finite set of vectors in an inner product space space V? What object is it? [closed]

Motivation: This question is motivated by a type of problems in medical imaging and computer vision as follows: suppose we've a set $A$ of points ("shapes") $\{x_1, ...x_d\} $in a Riemannian ...
1
vote
1answer
75 views

Do Stata and SPSS give conflicting versions of Anti-Image matrices?

I read on p1 of the Stata Manual glossary that: The image of a variable is defined as that part which is predictable by regressing each variable on all the other variables; hence, the anti-...
2
votes
0answers
31 views

Fisher information matrix with a general covariance structure

For the linear model, general linear models which allow for a more general covariance structure $V(\theta)_{N\times N}=(I_{N}+\theta A_{N\times N})(I_{N}+\theta A_{N\times N})^{'}$ ,where $A_{N\times ...
4
votes
2answers
51 views

Mahalanobis distance: What if S is not invertible? [duplicate]

The Mahalanobis distance is a distance metric used to measure the distance between two points in some feature space. Unlike the Euclidean distance, it uses the covariance matrix to "adjust" for ...
0
votes
0answers
13 views

How to make a covariance matrix from multiple observations of different objects?

I have $N$ objects. From each object, I sample $M$ values $(x,y)$ like so: ...
0
votes
0answers
79 views

Volume representation of ring distribution (cylindrical coordinates)

If we have a normal random distribution of points in N-space, we often use $\sqrt{det(COV)}$ to represent the volume, where COV is the NxN covariance matrix. In 2D Cartesian coordinates ($x$,$y$), $...
0
votes
1answer
17 views

Covariates and degrees of freedom

Consider a problem, when one is interested in finding influence of variable set {X} on Y (assuming reverse causal relation is infeasible), when confounding influences from a set of variables {Z} are ...
0
votes
1answer
23 views

Reference for incremental sandwich covariance from biglm?

I am working on some similar methods to Lumley's biglm wrapper around Miller's AS274 algorithm, and I can't seem to find a reference for his incremental Huber/White ...
0
votes
0answers
23 views

Covariance matrix of complex random variables

Let's say we have datapoints $x_i \in \mathbb{C}^N$, like this: $x_1 = \begin{pmatrix}z_1 & z_2 & ... & z_N \end{pmatrix}^T$ and $x_2=\begin{pmatrix}z_1 & z_2 & ... & z_N \end{...
7
votes
2answers
139 views

How to create an arbitrary covariance matrix

For example, in R, the MASS::mvrnorm() function is useful for generating data to demonstrate various things in statistics. It ...
1
vote
1answer
20 views

Use case for Anomaly Detection using the multivariate Gaussian distribution

We have 5000 vehicles of different classes (trucks, small cars, large cars) with 100 sensors in each car measuring fuel consumption, distance traveled, average speed etc for some time period $t$ that ...
1
vote
0answers
11 views

DCC models in R: how is the first starting value chosen?

The DCC model is defined through the proxy $Q_t$ as $$Q_t=(1-\alpha-\beta) \overline{Q} +\alpha\epsilon_{t-1}\epsilon_{t-1}' + \beta Q_{t-1}$$which is then normalized to find the correlation matrix $...
0
votes
0answers
17 views

Chosing an covariance matrix or 'repeated covariance type' spss

I'm struggeling with my choice for a covariance matrix or 'repeated covariance type' for my lineair mixed effects analysis in SPSS. I have a continuous measurement for experienced emotion(0-100), ...
0
votes
1answer
34 views

Inverse of block covariance matrix

I have a positive definite symmetric covariance matrix which looks like this: A, B, C, D and E, F, G are MATRICES, also positive definite symmetric covariance What is the inverse of such a matrix? ...
0
votes
1answer
24 views

Calculating the Covariance matrix for the mean of variables

Assume that we have 3 3D points to make this questions simpler. $$ pt_1 = \begin{bmatrix} x_1 & y_1 & z_1 \end{bmatrix} \\ pt_2 = \begin{bmatrix} x_2 & y_2 & z_2 \end{bmatrix} \\ pt_3 ...
0
votes
0answers
4 views

Covariance between non symmetrical matrices with a unidirectional composition

I am trying to explain variation in why some researchers go to certain places and not others by analyzing how many articles scientists from one publish about another location. My matrix is ...
0
votes
0answers
22 views

Covariance Matrix and Correlation Matrix - Singularity

If a covariance matrix is non-singular, does this implies that correlation matrix is also non-singular. My guess is it depends on mean vector in $K_{X} = R_{X} - m_X.{m_X}^H$ Not sure though.
0
votes
0answers
34 views

Common covariance matrix in linear discriminant analysis

Say I want to perform LDA classification involving three classes with within-class covariance matrices $$ \hat{\Sigma}_1 \,, \hat{\Sigma}_2 \, , \hat{\Sigma}_3$$ and that these matrices are calculated ...
1
vote
1answer
19 views

Evaluate high-dimensional Gaussian with variance matrix $\sigma^{2}I_{n_{t}\times n_{t}}+\boldsymbol{\Sigma}_{t}\boldsymbol{\Sigma}_{t}^{'}$

I need to compute the log-likelihood function in a high-dimensional Gaussian time-series. I have the following model: $\mathbf{y}_{t}\left|\mathcal{F}_{t-1}\sim\mathcal{N}\left(\mathbf{\boldsymbol{\...
4
votes
1answer
39 views

Decomposing a locally stationary covariance matrix

Say I have a non-stationary Gaussian Process with a square exponential covariance whose shape varies throughout space. The covariance entries are: $$ K_{ij} = N(|x_i-x_j|,\sigma_i^2+\sigma_j^2) $$ ...
2
votes
1answer
112 views

Covariance matrix of parameters in logistic regression

Let $$f(y)=\prod_i \binom{n_i}{y_i}\pi(x_i)^{y_i}(1-\pi(x_i))^{n_i-y_i}$$ where $$\pi(x_i)=\frac{e^{\sum_j x_{ij}B_j}}{1+e^{\sum_j x_{ij}B_j}}$$ then the likelihood is $$L\propto \prod_{i}\pi(x_i)^{...
1
vote
0answers
70 views

When can I use $XX^\top/n$ as covariance matrix for PCA?

Given a data matrix $\mathbf X$, can I always obtain its covariance matrix (to use in PCA) by centering (subtracting the column means) and then computing $\mathbf X \mathbf X^\top/n$? Is this always ...
1
vote
0answers
38 views

Is there any point to Reverse Engineering the Fisher Information Matrix from an Inverse Covariance Matrix?

Would there be any advantage in deriving a Fisher Information Matrix backwards from an inverse covariance matrix? I've discovered that this is much easier to do on the SQL Server platform I use than ...
0
votes
1answer
16 views

Why is multinomial variance different from covariance between the same two random variables?

We know that if $\big(X_1,X_2...X_k) \sim multinomial(n;p_1,p_2...p_k)$ then $X_i \sim bin(n;p_i) $ Then, $var(X_i) = np_i(1-p_i)$. But we have $cov(X_i,X_j) = -np_ip_j$. So doesnt that imply ...
1
vote
0answers
56 views

Covariance matrix of multivariate multiple regression coefficients

I would like to perform a regression analysis on a dataset comprising one independent variable (X) and two dependent variables (Y1 and Y2) which may be affected by correlated errors. R's stats::lm ...
1
vote
0answers
35 views

Why estimated covariance matrix by glasso is always zero?

I am using glasso function from glasso package, as follow: ...
4
votes
1answer
43 views

Intuition for near-decorrelation through centering

Consider a $p\times 1$ random vector $\mathbf u = (u_1,...,u_p)'$ with zero mean vector and variance-covariance matrix $$E(\mathbf u \mathbf u')\equiv \mathbf{\Sigma}=\sigma^2((1-\rho)I_p+\rho\...
0
votes
1answer
24 views

$LDL^T$ decomposition from Cholesky decomposition

Suppose we have a covariance matrix $\Sigma$. I know that the Cholesky decomposition $A^T A$ can be found from the LDL decomposition using $$ \Sigma = LDL^T = (LD^{\frac 1 2})(LD^{ \frac 1 2 })^T = A^...
1
vote
3answers
158 views

Decrease of $(X'X)^{-1}$ as n increases

Let $X$ be a $n \times p$ matrix ($n \geq p$ like a conventional data matrix), with each column j filled by iid draws from a variable $\mathcal{X}_j$. I would like ...
0
votes
1answer
29 views

Perform a t-test in multiple regression matrix

I have the following matrices: $$ X'X $$ $$X'y$$$$ y'y $$ I know that the B matrix can be computed as follows : $$ B = (X'X)^{-1}X'y $$ If I want to perform a t test for a specific B, say $$B_{1}$$, ...
1
vote
0answers
18 views

Calculation of vector norm uncertainty using covariance matrix

I have a vector $\mathbf{v} = [ v_x , v_y ] ^T$ whose PDF is Gaussian, thus it has an associated covariance matrix to represent its uncertainty (which is zero mean): $P = \begin{bmatrix} \sigma_x^2 &...
7
votes
1answer
62 views

If an inverse covariance matrix is sparse, what can I say about the covariance matrix?

How does the sparsity condition on an inverse covariance matrix affect the actual covariance matrix?