Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Make your voice heard. Take the 2019 Developer Survey now

Questions tagged [covariance-matrix]

A $k\times k$ matrix of covariances between all pairs of $k$ random variables. It is also called variance-covariance matrix or simply the covariance matrix.

0
votes
1answer
15 views

Covariance of sums of pairs of correlated variables

Take two vectors of normally-distributed random variables $\mathbf{x} = (x_1, x_2, \ldots x_n)$ $\mathbf{y} = (y_1, y_2, \ldots y_n)$ where the covariance of each pair $(x_i, y_i)$ is known, $\...
1
vote
0answers
17 views

Simplifying a covariance expression

Let $I=\begin{pmatrix}I_1\\\vdots\\ I_n \end{pmatrix}$ be a random vector, and $\Omega$ and $\Omega_I$ two random variables. I am trying to simplify the following equation (which worth $\frac{\rho_{I\...
0
votes
0answers
10 views

How to apply Lagrange Multiplier to a matrix?

I am learning about the Lagrange Multiplier and I see how to apply it to a set of equations but I don't know how to apply it to matrices. Suppose I have: ...
0
votes
0answers
21 views

Valid covariance matrix?

I am trying to replicate some results from this well-cited paper evaluating various methods to determine the rank of a matrix. They give several covariance matrices, and then sample from a Gaussian ...
0
votes
0answers
8 views

how can I interpret a long-run covariance matrix?

can I use the long-run covariance matrix to examine multicollinearity between my independent variables? and if yes, what how can I interpret matrix for determining Multicollinearity?
1
vote
1answer
24 views

Conditional covariance of a multivariate normal vector

We know that the conditional variance of a multivariate normal vector $(X,Y)$ is the Schur complement: $$V(X|Y=(y_1,...,y_n))=\Sigma_{XX}-\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX}$$ I have the intuition ...
1
vote
1answer
25 views

Use the RBF kernel to construct a positive definite covariance matrix

A commonly used kernel in Gaussian processes is the RBF kernel: $$ \kappa(x,x') = \exp\left(-\frac{|| x-x'||^2}{2\sigma^2}\right) $$ In the context of a Gaussian process, a kernel is used to ...
6
votes
1answer
175 views

Are all symmetric matrices with diagonal elements 1 and other values between -1 and 1 correlation matrices?

A question for the statisticians and other math lovers: Are all symmetric matrices with diagonal elements 1 and other values between $-1$ and 1 correlation matrices?
1
vote
0answers
44 views

Why that distribution is normal with those parameters?

Mi question is the following. I have two independent 2-dimensional normal distributions with the same mean vector and different covariance matrixes, lets say $X_1 \sim N_2( \mu, C)$ and $X_2 \sim N_2(...
0
votes
1answer
21 views

Analytical solution to the covariance between a continuous and a categorical variable

Let $X$ be a continuous variable with mean $\mu$ and $Y$ be a categorical variable with event probability vector $\mathbf{p}$. I am trying to calculate $Cov(X, Y)$. I have the solution if $\mathbf{p} ...
0
votes
1answer
18 views

How to interpret a given 2D co-variance matrix?

I am trying to solve a problem regarding revision for my Big Data module. I have two main questions. 1) Given a predefined co-variance matrix: A cluster of points is distributed in a two-...
2
votes
0answers
26 views

Why mean in Gaussian Process is not so important? [duplicate]

Source of my doubt is the section 2.7 of GPML book by Rasmussen, an screenshot of the book is attached below. Much of my confusion is clarified by this discussion. If mean of GP is not estimated and ...
2
votes
1answer
46 views

How to calculate the covariance matrix for a categorized variable?

Let $X$ and $Y$ be jointly distributed as a multivariate normal with the following parameters: $$ \mu_{XY} = \begin{bmatrix} 0 \\ 0.2 \end{bmatrix} \qquad \Sigma_{XY} = \begin{bmatrix} 1 & 0.05 \...
0
votes
0answers
34 views

Comparing species correlations between species in two habitats

I have two community data sets (samples as rows, species as columns, populated with abundance). This data comes from two habitats/sites, with differing numbers of samples at each site. What I want ...
1
vote
1answer
61 views

Deriving the sampling distribution of MLE for Normal distribution

Let $X_1,\ldots,X_n$ be an observed random sample from $N_p(\mu, \Sigma)$. I know that the MLE of $\Sigma$ is $\frac{1}{n} \sum_i^n(X_i -\bar X)(X_i -\bar X)^T$, which is biased. We define $S = \...
1
vote
1answer
20 views
0
votes
0answers
17 views

Autocorrelated time series analysis

I have a set of observations $X_1, ...,X_n$. They can be generated with a simple Markov chain with $k$ states - in theory, in practice there are covariances between observations from different time ...
0
votes
0answers
10 views

Testing for Causality in variance

I have read a couple of papers which mention causality in variance. For example Cheung and Ng (1996, Journal of Econometrics), "A causality-in-variance test ands its applications to financial markets"....
0
votes
1answer
29 views

Covariance kernel of a Gaussian process

I just started studying the theory of Gaussian processes. I'm mainly interested in studying functional data and I haven't found the answer to my doubt. Let's say I have some curves that I consider as ...
3
votes
1answer
43 views

Why Does Second Order Weak Stationarity include Statement on Covariances in addition to Statement on Mean and Variance?

A stochastic process is second order weakly stationary if all random variables have same mean (first moment), and same variance (second moment?), and covariances that are time-invariant (second moment ...
3
votes
0answers
48 views

What is the origin of the Winer's compound symmetry test of the variance-covariance matrix?

In his famous book, Winer (Statistical principles in experimental design 1971; reedited Winer, Brown, Michels, 1991, p. 517) introduced a test of compound symmetry. The test expands on the likelihood ...
1
vote
0answers
34 views

VAR(p) Model Covariances and Moment Equation

I'm currently going through the book Analysis of Financial Time Series by Ruey S. Tsay and reached the following statement (The book can be found here, with VAR(1) included in the preview): Where: $...
-1
votes
0answers
61 views

Using full covariance matrix of a linear fit reduces the errors?

I am currently studying linear fitting and error propagation. The model to fit is this: $$B ( N , Z ) = a _ { v } A - a _ { s } A ^ { 2 / 3 } - a _ { c } \frac { Z ( Z - 1 ) } { A ^ { 1 / 3 } } - a ...
1
vote
0answers
8 views

Which control chart can use for monitoring of covariance matrix of cofficient?

I have a series of panel data, which includes p cross section and t time. I estimated the covariance matrix of coefficients for periods for periods. Now my data includes the individual period and the ...
1
vote
0answers
23 views

Likelihood of a linear model in matrix form

I have difficulty finding the likelihood of the data represented in the matrix form. The mapping between target variable $\mathbf{T}$ and observed variable $\mathbf{X}$ is given as $f:\mathbf{X}\...
0
votes
0answers
37 views

Spherical covariance matrix, what does it mean?

I've recently started with machine-learning but I feel like I've missed important concepts, especially when it comes to the covariance. As far as I understand it would mean that we have a matrix of ...
1
vote
0answers
48 views

Limiting results for non-unique eigenvalues and eigenvectors for a sample covariance matrix

I am working on the limiting behavior for the eigenvalue and the corresponding eigenvectors, especially the minimum eigenvalues. For instance, let $S_X=\frac{1} {T} \sum_t X_t X_t ^\prime$ be a $p \...
1
vote
0answers
27 views

Higher order extensions of gaussian distribution?

I anticipate that this question may be predicated on some misconceptions or confusion. Please have patience. Through Bishop's PR&ML book, as well as a little bit of exposure to statistical ...
0
votes
0answers
20 views

residualized covariance matrix from pca/eigenvalue decomposition

I understand that given N dimensional data you can use PCA to construct an N dimensional orthonormal basis that explains 100% of the variance of the original data. However, you can also construct ...
0
votes
0answers
17 views

Understanding MCD

I have recently stumpled upon the robust MCD (Minimal Covariance Determinant) Estimator. If I have $n$ datapoints of dimension $p$. Let`s say we want to obtain a robust estimate for the Covariance ...
1
vote
0answers
30 views

How can I perform bivariate random-effects meta-regression if the implied between-study covariance matrix is not positive definite?

I have described my problem here: How can one get consistent (i.e. direct+indirect=total) effects in a Meta-Analytic SEM model with latent variables? Unfortunately, the solution I've found to this ...
-1
votes
0answers
42 views

Can Mahalanobis distance not be applied reliably after performing one-hot encoding with certain data-sets?

I am working with a data-set of patient performance-data and patient demographics for people with a medical condition. I am trying to assess the effect of a treatment on the patients, and as I am ...
1
vote
1answer
86 views

Can the covariance matrix in a Gaussian Process be non-symmetric?

I was watching a lecture on Gaussian Process and when the covariance matrix was introduced, the tutor explained that the matrix is $(n \times n)$ because every point is covered twice - we include the ...
1
vote
2answers
51 views

Covariance between a normal variable (x1) and a sum including it (x1 + x2)

I'm interested in the joint distribution of two variables, $x_1$ and $x_2$. $$ x_1 \sim Normal(0, \sigma^2_1);\\ \epsilon \sim Normal(0, \sigma^2_{\epsilon});\\ x_2 = x_1 + \epsilon; $$ as a ...
0
votes
0answers
17 views

Bias of the eigenvalues of sample covariance matrix

Consider and i.i.d sample $X_1, \ldots X_n$ in $\mathbb{R}^p$ with covariance matrix $\Sigma \in \mathbb R^{p\times p}$ What is the expectation of the eigenvalues of the sample covariance matrix?. ...
0
votes
1answer
18 views

matrix inequality related to finance

I'm trying to show that, for certain investment strategies, it pays to have more precise estimates of the covariance matrix of your returns. I have always took this for granted, but I've been having ...
1
vote
0answers
16 views

Is there any background for constraining covariances on fitting GMM?

On clustering data using GMM model, I often see the option to constrain covariances of each clustered GMM. For example, http://scikit-learn.org/0.16/auto_examples/mixture/plot_gmm_classifier.html ...
1
vote
0answers
37 views

Mahalanobis distance between 2 points doesn't work when covariance matrix has values close to 0

I am working on a project where I am trying to replicate a randomized experiment from an observational study data, using Mahalanobis distance matching to ensure that the control and treated groups are ...
2
votes
2answers
66 views

Lower-bound on covariance estimated via Laplace approximation?

I think when a posterior is approximated to be multivariate normal as in Laplace approximation, the covariance matrix is taken to be the negative inverse Hessian evaluated at the log-posterior maximum,...
1
vote
0answers
25 views

Covariance Estimation of time series with mixed frequencies

I have three times series $x_{1,t}$, $x_{2,t}$ and $x_{3,t}$. For the first two time series ($x_{1,t}$ and $x_{2,t}$) I have daily data, but for the last one, I have monthly data (let us assume that ...
2
votes
4answers
117 views

Can we use covariance matrix to examine feature collinearity?

Consider using Multi-variate Gaussian to approximate $X = [X_1, X_2, ..., X_n]$ and $X_i = [x_{i1}, x_{i2}, x_{i3}, ..., x_{im}]$, so we have n data points and each data point has m features. Multi-...
0
votes
0answers
37 views

Covariance of a nxm matrix with n<m is not positive semidefinite?

Is it true that, for a matrix $A \in \mathbb{R}^{n \times m}$ with $n<m$ (so with more features than samples), its covariance matrix is (or might be?) not positive semidefinite? If that's the case, ...
1
vote
0answers
15 views

Partial least square different maximization programs

PLS regression is a regression method based based on latent variables in order to handle collinearity or violation of full rank assumption in linear regression. Latent variables called components are ...
1
vote
2answers
36 views

Probability of a measurement with uncertainty covariance being generated by a normal distribution

I have the following situation: A set of Kalman filters with the same model, each with its own current estimated state and state covariance. A measurement with a covariance matrix expressing its ...
0
votes
0answers
12 views

Constraints on covariance matrix's correlation, and covariance matrix decomposition. [duplicate]

I have a $3 \times 3$ covariance matrix $\boldsymbol{\Sigma}$. I have two questions 1) which constraints on the correlations $\rho_{ij}$ must be satisfied to have a positive definite matrix? 2) does ...
0
votes
0answers
20 views

bootstrapping covariance matrices with different sampling procedures

The regression model has heteroskedasticity. The variance of error term depends on regressors. From boostrapping analysis, I got two different covariance matrices of $\beta$. The difference results ...
1
vote
1answer
50 views

Can a covariance matrix be recalculated for dummy variables?

Say I have the following sample from a continuous variable $X$ and a categorical (dichotomous) variable $Y$: X Y 0.5 1 2.3 2 2.2 2 1.8 1 Moreover, the covariance matrix between $X$ and $Y$ is also ...
1
vote
1answer
46 views

mixed effects model in repeated measurements

My question is conceptual. Suppose $n$ patients, where each one is measured at 4 different time points. The outcome is continuous. The patients are randomly assigned to two groups, intervention yes/no....
2
votes
0answers
61 views

Appropriate Distribution for Diagonal Covariance Matrices

Let's say I have a model like: \begin{align} X\mid\mu,\Sigma_X &\sim \mathcal{N}(\mu,\Sigma_X)\\ \mu\mid m, \Sigma_\mu &\sim \mathcal{N}(m,\Sigma_\mu) \\ \Sigma_X\mid \Psi, c &\sim \...
0
votes
0answers
6 views

Allowing cluster-level residuals to covary in Proc Glimmix

I am modeling the probability of a child being retained in kindergarten based on individual and school-level factors. The model includes random intercepts and slopes, as follows: ...