The variance-covariance matrix, or sometimes just covariance matrix, is the matrix whose $(i,j)$ element is the covariance between the $i^\text{th}$ and $j^\text{th}$ variable (or the $i^\text{th}$ and $j^\text{th}$ parameter).

learn more… | top users | synonyms

2
votes
1answer
29 views

Automatically fixing ill-conditioning or collinearity

I'm backtesting a regression model, which entails running it on a bunch of bootstrap samples of a "rewound" version of our data set. Unfortunately, in some of these resamplings, I end up getting some ...
1
vote
0answers
21 views

Matlab: Help in mitigating the problem: Non-positive definiteness of a matrix

EKUKF toolbox contains a file ut_sigmas.m I am facing a frustrating problem with the covariance matrix $P$ of the measurement when using Cholesky decomposition. ...
0
votes
0answers
19 views

Estimating the variance of a sum of predictions

I have $N$ plots that were used to estimate a relationship between three predictor variables, $X_1$, $X_2$, $X_3$, and an outcome, $Y$, using a generalized linear (lognormal) model. The resulting ...
2
votes
0answers
33 views

Covariance of many simple regressions

Assume we have a true model of $$Y=X\beta+\varepsilon,$$ where $Y$ is some outcome , $X$ is a $1\times p$ vector of covariates which have a (non-diagonal) variance-covariance matrix $\Omega$, ...
0
votes
0answers
8 views

drawbacks of Soft and Hard thresholding operators

I am reading about Soft and Hard thresholding operators for the estimation of very large covariance matrices. I noticed that another operator such the "Smooth Clipped Absolute Deviation (SCAD)" is ...
1
vote
0answers
14 views

Matlab FACTORAN error on line 162: a covariance matrix is not positive definite [closed]

I have a data set called Z2 that consists of 717 observations (rows) which are described by 33 variables (columns). The data is standardized by using ZSCORES. Additionally, there is no case for which ...
1
vote
0answers
53 views

Asymptotic covariance matrix of the covariance parameters SAS versus lme

I am trying to obtain the asymptotic covariance matrix of the covariance parameters of a mixed-effects model using SAS and R In SAS, this matrix can easily be obtained by using the 'asycov' option in ...
3
votes
0answers
19 views

Importance of semi-positive definiteness of covariance matrix [duplicate]

Since Covariance matrix is symmetric it is Hermitian (self adjoint) and always diagonalizable. If the matrix has all non zero eigen-values its is a full rank matrix. But what is the importance of the ...
0
votes
1answer
54 views

Can someone explain the mechanics of the variance-covariance matrix in OLS?

I have read many of similar posts here already as well as other resources on the topic, but they all generally just show the steps that generate this equation: $\hat{\sigma^2}({X}'X)^{-1}$ What I ...
0
votes
0answers
13 views

Evaluate the multivariate normal using variance matrices $\boldsymbol{\Lambda}+\alpha_{i}\mathbf{a}\mathbf{a}^{T}+\beta_{j}\mathbf{b}\mathbf{b}^{T}$

I need to calculate a huge amount of inverses and determinants to evaluate the pdf of the multivariate Gaussian. Specifically I need to compute the inverses and determinants of the following ...
1
vote
0answers
22 views

Why is the covariance matrix symmetric [closed]

I am aware that for a 2-d multivariate variable, the cov(x,y)=cov(y,x). This brings about the symmetry of the covariance matrix, but is it possible to have a non--symmetric covariance matrix?
4
votes
1answer
25 views

covariance of squared terms

Assuming two variables $X1$ ~ $N(0,1)$, $X2$ ~ $N(0,1)$ with $Cov(X1,X2) = a$. Is it possible to derive analytically what the covariance between $X1^2$ and $X2^2$ would be? Empirically (I tried this ...
1
vote
1answer
13 views

Finding covariance matrix for MLE of correlated outputs

I generate data using the following model: $\begin{pmatrix}Y_1\\Y_2\end{pmatrix} \sim \mathcal{N}\left( \begin{pmatrix}\mathbf{X}\beta_1\\\mathbf{X}\beta_2\end{pmatrix}, \mathbf{\Sigma} \right)$ I ...
2
votes
1answer
41 views

Decomposition of inverse covariance matrix

Let $\Sigma$ be a covariance matrix and let $$x^T \Sigma^{-1} x = \|Ax\|_2^2.$$ What is the interpretation of matrix $A$? I tried solving for $A$ with an eigenvalue decomposition of $\Sigma$ as ...
3
votes
1answer
35 views

Why is pure sample covariance a bad metric to understand the degree of correlation between two variables?

Covariance helps you understand how variables are linearly related. Would it be possible to have two pairs of variables in a deterministic relationship (i.e. linearly correlated variables) that have ...
0
votes
0answers
10 views

How to obtain the covariance matrix for the parameters estimated by maximizing this log-likelihood of logit/probit

I have this linear latent model : $y^*=\beta_0+\beta_1 X+\epsilon,$ $y^*$ is latent (non observed) variable and $\epsilon\sim iid (0,\sigma^2 I)$ Consider : $y=1$ if $\beta_0+\beta_1 X>0$ and ...
1
vote
1answer
32 views

Variance / Covariance Matrix - mean of squared errors

I'm trying to build a stats library. I'm following along with the tutorial on multiple regression analysis here: http://reliawiki.org/index.php/Multiple_Linear_Regression_Analysis I have the ...
1
vote
1answer
80 views

Estimating VAR by GLS versus OLS: efficiency

Suppose I have a VAR model with different regressors in different equations (this could be due to restricting some coefficients of a full VAR($p$) model to zero or having some different exogenous ...
3
votes
1answer
53 views

Covariance of natural logarithms — how to estimate $\sigma (\mathbf X, \mathbf Y)$ from $\sigma (\ln{\mathbf {(X)}},\ln{\mathbf {(Y)}})$?

I have $$\operatorname{Cov} (f{\mathbf {(X)}},f{\mathbf {(Y)}})$$ where $\operatorname{Cov}$ denotes the covariance and $f(\mathbf X)$ is a nonlinear function, i.e. $f(\mathbf X) = \ln(\mathbf X)$. ...
2
votes
1answer
94 views

Variance estimation in a one-factor linear model

I was given a dataset (a mat file) of $100\: 000$ observations, each with $50$ dimensions (coordinates). Denote matrix $X$ a $50\times 100\:000$ matrix in which each column was generated according to: ...
2
votes
1answer
64 views

Can covariance be derived from means and variances?

In treatment studies it is common to report multiple outcome measures from the same subjects. The treatment effects on these outcomes are typically correlated so this should be taken into account ...
0
votes
0answers
64 views

Identification of Difference in Weighted Mean and its Standard Error in Clustered Data Using Stata

I already asked this question on the statalist but have not received any comments on this issue yet. So I am going to post it here too. I am concerned with the following problem. Suppose my data ...
2
votes
1answer
89 views

Best OLS estimators

Hi i am stuck on this one, the question is related to Gauss-Markov theorem: Consider a general alternative to the OLS estimator that is also a linear unbiased estimator, say ${\tilde \beta}$. ...
3
votes
0answers
74 views

Generate Symmetric Positive Definite Matrix with a pre-specified Sparsity

I am trying to generate a correlation matrix $p\times p$ (symmetric p.s.d) with a pre-specified sparsity structure (specified by a graph on $p$ nodes). The nodes that are connected in the graph have ...
1
vote
0answers
42 views

LS vs MLE for Gaussian Conditional Random Field estimation

Is there such a thing as Least Squares estimation for the conditional mean and covariance of a conditional gaussian random field? I'm looking at this paper by Wytock and Kolter 2013, in which they ...
1
vote
2answers
88 views

Is every correlation matrix positive semi-definite?

I am generating correlation matrix by some new algorithm. Generated matrix is non positive semi definite matrix. I'm getting few negative eigenvalues. Rest of eigenvalues are quite equal to ideal ...
0
votes
1answer
55 views

Multivariate normal with singular covariance

I'm an undergraduate student. I read about multivariate normal distribution in hogg and craig. And i wonder why the covariance is allowed to be positive SEMI-definite. I read this Normal distribution ...
10
votes
4answers
414 views

How to generate a large full-rank random correlation matrix with some strong correlations present?

I would like to generate a random correlation matrix $\mathbf C$ of $n \times n$ size such that there are some moderately strong correlations present: square real symmetric matrix of $n \times n$ ...
2
votes
0answers
85 views

Can the law of total covariance apply to variables from different sample spaces?

Wikipedia says this about the law of total covariance (http://en.wikipedia.org/wiki/Law_of_total_covariance): In probability theory, the law of total covariance,[1] covariance decomposition ...
8
votes
1answer
563 views

Confused about the visual explanation of eigenvectors: how can visually different datasets have the same eigenvectors?

A lot of statistics textbooks provide an intuitive illustration of what the eigenvectors of a covariance matrix are: The vectors u and z form the eigenvectors (well, eigenaxes). This makes sense. ...
2
votes
1answer
57 views

How to select a subsample of fixed size to maximize its total PCA variance?

I would like to use PCA to help design my genomics experiment. I can only afford to perform my experiment on a limited number of genotypes so would like to maximize the variation of the ones I ...
0
votes
0answers
22 views

Can any one give me inf how I can form X, Z, R, G, and A matrices using dummy variables using the posted info below?

Y=Xb +Zu + e, where y represents a vector of observed (measured) phenotype values, b is vector of unknown parameters for “fixed" effects, while X is corresponding design matrix, u is vector of ...
3
votes
1answer
381 views

What is the correct formula for between-class scatter matrix in LDA?

At one point in the process of applying linear discriminant analysis (LDA), one has to find the vector $v$ that maximizes the ratio $vBv'/vWv'$, where $B$ is the "between-class scatter" matrix, and ...
0
votes
1answer
36 views

covariance matrix of residuals from a fitted model to decorrelate residuals

I fit a geeglm model with clustered data and now I would like to decorrelate the residuals of the model in order to run model diagnostics. I read that if I can obtain the covariance matrix of the ...
3
votes
1answer
83 views

Variance of Beta IV

I'm trying to calculate the variance of the Instrumental Variables (IV) estimator $${\hat \beta _{IV}} = {\left( {{Z^T}X} \right)^{ - 1}}{Z^T}y = \beta + {\left( {{Z^T}X} \right)^{ - 1}}{Z^T}u$$ (or, ...
0
votes
1answer
61 views

Correlated random draws with graph structured correlation

I have a problem where I have a graph structure, such that some nodes are connected to other nodes i.e. we have an adjacency matrix of size n*n with a 1 corresponding to a connection and 0 otherwise. ...
1
vote
1answer
75 views

What are the implications of estimating a covariance matrix from a correlated sample?

Given a sample of $n$ independent observations $x_1,...,x_n$ (where $x_i$ are $p$-dimensional column vectors), the $p \times p$ sample covariance matrix is defined as ...
1
vote
1answer
83 views

Determinant of the covariance matrix in a normal distribution

Suppose a $p \times 1$ vector $x \sim N_p(\boldsymbol 0, \boldsymbol \Sigma_1)$. Now, There is another covariance matrix $\boldsymbol \Sigma_2$. We know that $|\boldsymbol \Sigma_2| < |\boldsymbol ...
4
votes
2answers
103 views

Generating data with a given sample covariance matrix

Given a covariance matrix $\boldsymbol \Sigma_s$, how to generate data such that it would have the sample covariance matrix $\hat{\boldsymbol \Sigma} = \boldsymbol \Sigma_s$? More generally: we are ...
0
votes
0answers
11 views

What if the variance-covariance matrix of a sum of two random vectors? [duplicate]

If X is a px1 random vector with mean Mu(x) and variance-covariance matrix sigma(x) and if Y is a qx1 random vector with mean Mu(y) and variance-covariance matrix sigma(y). If p=q, what would be the ...
1
vote
1answer
41 views

Comparing and Interpreting covariances

I had a discussion about covariance recently and it would be nice to hear your feedback about this. Let's say we have a dataset of $n$ samples with $d$ attributes. For simplicity, let's say 3 of ...
3
votes
1answer
87 views

Manual calculation of variance-covariance from published confidence intervals

I would like to obtain the variance-covariance matrix from a published set of regression outputs. These outputs provide a mean value and confidence intervals. I will convert the confidence intervals ...
0
votes
0answers
25 views

Estimating covariance for naive Bayes

I am a beginner in Pattern Recognition and started reading up Bayesian classifiers. I came across the case of naive Bayes with equal covariance in all dimensions. Given sufficient data, how does one ...
2
votes
0answers
316 views

How to interpret glmer results (variance, correlation and ICC)

I'm a beginner in statistics and I have to run multilevel logistic regressions. I am confused with the results as they differ from logistic regression with just one level. I don't know how to ...
1
vote
1answer
76 views

Correlation between two normally distributed variables

Let a~$\mathcal{N}(\mu_a,{\sigma_a}^2)$,b~$\mathcal{N}(\mu_b,{\sigma_b}^2)$ and c~$\mathcal{N}(\mu_c,{\sigma_c}^2)$. We construct two normal variables x~$a-b$ and y~$a-c$. Can we find the ...
0
votes
0answers
23 views

Can you combined two sources with difference variance to reduce error? [duplicate]

I have two samples of data each estimates of a position x, y with Gaussian noise. One source has a larger variance than the other. Is this source in any way useful ...
4
votes
0answers
261 views

Inverse covariance matrix, off-diagonal entries

Let $\Sigma$ be a covariance matrix. According to the material in this link, If the elements of $\Sigma$ are all positive, most of the off-diagonal elements in $\Sigma^{-1}$ will be negative ...
2
votes
0answers
47 views

Variance-covariance matrix as the sum of variance covariance matrices

I have a variance-covariance matrix, $\mathrm{V}$. This allows me to take a vector, $x$ of independent random variables drawn from a known distribution, and induce a required variance-covariance ...
2
votes
0answers
249 views

Negative variance from inverse Hessian matrix

I used nlm function in R to do the optimization. When I calculated the correlation between estimated parameters using the inverse of Hessian matrix, I got negative values on the diagonal. My questions ...
2
votes
1answer
157 views

Covariance matrix estimation in presence of missing data

I want to estimate a covariance matrix from data with some missing values. Ideally I'd like an R package but python could be ok. R has some built in ways of doing this. You can use ...