Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [covariance-matrix]

A $k\times k$ matrix of covariances between all pairs of $k$ random variables. It is also called variance-covariance matrix or simply the covariance matrix.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

Sampling prior covariance matrices - nested sampling

I am trying to fit a multivariate Gaussian with a non-diagonal covariance matrix $\Sigma$ using nested sampling. Usually, in other Bayesian analyses, we would use a Inverse Wishart or LKJ prior on ...
2
votes
1answer
78 views

Find $\delta$ such that sparse covariance matrix is positive definite

Good day, I was looking through some papers to help with my project assignment that wants me to implements 2 lasso approaches. I am having trouble simulating the samples from a MVN distribution. $\...
3
votes
1answer
340 views

Has anyone studied linear regression where the covariance matrix of the error is a function of the parameters being estimated?

Consider the multivariate linear model: $$y = X\beta + e$$ $y$ is the measured output, $X$ is the model matrix, $\beta$ is the parameter vector, and $e$ is the zero-mean error vector: $$E[e] = 0 \...
2
votes
1answer
43 views

Why is a sample covariance not even semi-positive definite with missing data?

I am trying to estimate a sample covariance when I have less observations $n$ than variables $p$ ($n<p$). This will serve later on as basis for a shrinkage estimator. We know (see this post) that ...
0
votes
0answers
22 views

What is the relation between $cov(X_1,X_2)$ of the Random variables and the $cov(F_{X_1}(X_1),F_{X_2}(X_2))$

I have a question about the covariance matrix. In copula literature and applications many covariance matrix estimations are being used. Let $X_1, X_2$ as two random variables. For producing samples ...
0
votes
0answers
19 views

Can the coefficients and standard errors for a nested model be derived from the variance-covariance matrix of the full model

If you have a generalised linear model like this:- $$ f(y) = \beta_0 + \beta_1x_1 + \beta_2x_2 $$ and you know the coefficients and variance covariance matrix for the betas, but don't have access to ...
0
votes
0answers
8 views

How can I compute (empirically) the covariance of the difference of two jointly Gaussian random vectors?

I have $N$ realizations each of zero-mean random vectors $\mathbf{x}$ and $\mathbf{y}$ that I assume are jointly Gaussian. I would like to estimate the covariance of the random variable $\mathbf{d}=\...
2
votes
1answer
32 views

Estimating two-parameter covariance matrix, with average variance and average covariance?

Which methods can I use if I want to estimate a highly-structured two-parameters covariance matrix, when I have few observations $N$ for many variables K? Motivation: given a dataset of K=3000 ...
1
vote
1answer
116 views

On the properties of covariance and kernel matrices

I'm stumbling upon an example of a mixed model or a Gaussian Process, say: $Z \in\mathbb{R}^{n \times m}, m \ge n$ ie random effect $X \in\mathbb{R}^{n \times p}, p \ge 1$ ie fixed effects $K \in\...
2
votes
1answer
22 views

covariance matrix derivation

How to derive from the first step to the second step, where y is k by 1 vector and A is k by p matrix. Thanks for any help in advance
0
votes
1answer
39 views

Upper bound trace of inverse of covariance matrix

Let C be the covariance matrix from any normal distribution. If the trace of C is upper-bounded by a constant k (i.e., tr(C)<=k), can I find an upper bound for the trace of the inverse of C (i.e., ...
0
votes
1answer
17 views

Finding the covariance of $u_{it} = \nu _{it} - \theta \nu _{i\left ( t-1 \right )}$ for $t>1$

Given $u_{it} = \nu _{it} - \theta \nu _{i\left ( t-1 \right )}$ for $t>1$ $u_{i1} = \nu _{i1}$ and the $\nu _{it}$ are white noise with variance equal to $\sigma^{2}$. I can find the expected ...
0
votes
0answers
7 views

Energy Vector and covariance matrix, intuition

I am reading about Gaussian processes and I am encountering a lot of terms like this: $r^T\Sigma r$ or $r^T\Sigma^{-1}r$ or $r^T\Sigma^{-1}y$ Where $\Sigma$ is a covariance matrix I ...
1
vote
0answers
29 views

Sum of Two Bivariate Normal Distributions

I'm just confused on how to set up and start this problem. I'm confident that once I start down the right path, I'll have little issue. Let $p_1$ denote a bivariate normal distribution $N(0, 0, 1, 1, ...
0
votes
0answers
28 views

Covariance matrix as a sum of two covariance matrices

Suppose that a random vector $\mathbf{n}$, $$\mathbf{n} = \mathbf{n}_A + \mathbf{n}_B \ , \tag{1}$$ can be written as a sum of two random vectors $\mathbf{n}_A$ and $\mathbf{n}_B$, that are ...
0
votes
0answers
29 views

How to partition a correlation matrix into two extreme groups? [closed]

I have a correlation matrix of 40 random variables: $C_{ij}$, $1 \leq i,j \leq 40$ I want to partition the 40 variables into two groups (of 20 each) such that the average correlation differs by most ...
0
votes
0answers
24 views

Relation between eigenvalues of original and transformed matrices

Let the matrix $X$ be some data arranged in rows. Consider the following eigenvalue decomposition $X^\top X = Q \Theta Q^\top=\sum_{i=1}^n \theta_iq_iq_i^\top$ where $q_i$ are the eigenvectors and $\...
2
votes
3answers
126 views

Why does the variance of the OLS estimator scale with $\frac{1}{n}$?

Let $\mathbf{X}$ be an $n\times p$ matrix. In multiple linear regression, we have $$\boldsymbol{\hat{\beta}}=\mathbf{[X^TX]^{-1}X^Ty}\sim\mathcal{N}\Big(\boldsymbol{\beta}, \sigma^2 \mathbf{[X^TX]^{-...
0
votes
0answers
13 views

Unbiasedness and Variance of Predictions

Here is the problem I'm working on: I'm not quite sure if I'm showing either unbiasedness property right, and am stuck on finding the expressions for the variances. Here's what I've done so far. (a) ...
0
votes
1answer
22 views

Properties of Covariance (Matrix and Vector Case)

In 1 dimensional case, we have $Cov(cX_1,X_1) = cCov(X_1,X_1).$ Here $X_1$ is just a random variable. I was wondering if we have an analogue for $Cov(AX_1,X_1)$, where $A$ is a matrix with ...
1
vote
0answers
14 views

How do I construct a variance-covariance matrix from a matrix formulation of a MLR? [duplicate]

I'm trying to calculate the SE for each coefficient given by a matrix formulation of MLR by the root of each diagonal in the so-called variance-covariance matrix, but I'm unsure how to construct this ...
0
votes
1answer
18 views

Mahalanobis distance - understanding the formula [duplicate]

I've read quite a few explanations on this topic, liking this one the most: https://mccormickml.com/2014/07/22/mahalanobis-distance/ But there is still one thing I don't understand. I understand ...
0
votes
0answers
20 views

Distribution of the squared norm of a vector with multivariate normal distribution and dependent components [duplicate]

Let x be a p-dimensional random vector with dependent components. Assume that x is distributed according to a multivariate normal distribution with mean vector m and variance/covariance matrix V which ...
0
votes
1answer
25 views

Why the first principal component is mostly negative while the second component is mostly positive?

I am running PCA for a fleet management data frame $X$, where each column is a city, each row is a date, there are 50 cities and 500 dates. I run PCA on $A=X^{T}X$. Then the first component $v_{1}$ ...
0
votes
1answer
55 views

How to use inverse information matrix and Delta method to find sample variation?

Explain how the inverse of the information matrix and the Delta Rule be used to generate an approximate sampling variance for the estimated proportion exp(beta 0 + beta 1)/(1+exp( beta 0 + beta 1))? ...
0
votes
0answers
12 views

CFA, Covariance matrix is not positive definite?

I have a 3-factor CFA model with 16 items. I conducted 8 groups comparison.For one group, Amos analytic result showed " The following covariance matrix is not positive definite." The sample size for ...
0
votes
0answers
25 views

How to create a variance-covariance matrix for forecasted fantasy basketball scores?

I have three basketball players who have played in games together and I want to find a Variance-Covariance matrix that will be as accurate as possible for their fantasy points in an upcoming game. My ...
0
votes
0answers
18 views

Covariance Matrix (Long/Short Portfolio; Different Weightings)

I am attempting to calculate the expected one-day standard deviation of a portfolio in dollars. In other words, I am looking for the following: "I expect my portfolio to move _______ dollars on ...
0
votes
0answers
22 views

Estimate variance of parameters in non-linear ridge regression

I am basically estimating the shape of an unknown function f(x) from multi-dimensional chemical reaction data by estimating the most likely function values $f$ on a grid $x$ with kernel regression. To ...
0
votes
1answer
30 views

How to decompose covariance matrice, multiplied by constant, to sample from multivariate normal? [closed]

I need to sample from multivariate normal distribution with mean vector $\mu$ and covariance matrix $\Sigma_1$. For that I want to use the decomposition of $\Sigma_1$ into $UΛ{U}^T$ and samle as $\...
5
votes
1answer
181 views

What is the quantile covariance?

Suppose that $X$ is a p-dimensional random vector and $Y$ is a random scalar. Then, Dodge and Whittaker (2009) indicate that the covariance of these two variables can be formulated as a minimization ...
1
vote
1answer
30 views

How to calculate individual covariances and residual covariances in a multivariate mixed model

I need enlightenment in calculating individual covariances and residual covariances in a multivariate mixed model. I'm going to use the dataset 'Owls', present in the glmmTMB package to replicate ...
0
votes
0answers
16 views

MSE for the kernel-based HAC long-run covariance matrix itself (in the Frobenius norm sense)

Consider the stationary multivariate time series $X_1, \ldots, X_T$ and the HAC-consistent long-run covariance matrix estimator $$\hat{\Gamma} = \hat{\Gamma}_0 + \sum_{l=1}^{T-1} K\left(\frac{l}{h_T}\...
1
vote
0answers
19 views

convergence and efficiency of mcmc chains and estimation of covariance matrix

I am doing some bayesian analysis and exploring posterior distribution with mcmc method. I would like some clarification with estimating the covariance matrix. I have a model with 6 parameters. ...
2
votes
0answers
34 views

Geometric interpretation of Cholesky Decomposition

I understand that a square matrix, say $A$, can be thought of as a linear transformation within the same space. I could be as simple as basis change or some other transformation. In this way of ...
0
votes
1answer
44 views

How to show that $X = LY$ where $Y\sim N(0,I)$?

Let $X\sim MVN(0,\Sigma)$ denote a random vector having the multivariate normal distribution with mean $0$ and covariance matrix $\Sigma$. Suppose we want to sample from $X\sim MVN(0,\Sigma)$. ...
0
votes
0answers
33 views

Inverse Covariance Matrix of a Gaussian Distribution: Relationship of Precision Matrix and Information Matrix

In the book "Probabilistic Robotics" (Thrun et al.), chapter 3.5.1 states that The canonical parameterization of a multivariate Gaussian is given by a matrix $\Omega$ and a vector $\xi$. The ...
2
votes
1answer
47 views

Doubt about proof of positive semi-definite matrix implies covariance matrix

I have a doubt about the proof of the fact that a positive semi-definite matrix is a covariance matrix. The professor do the following proof: Let $\Sigma$ be a positive semi-definite $p \times p$ ...
3
votes
0answers
26 views

What are the differences between HC estimators and their small sample properties?

I am currently using R to run regression with the following code: ...
1
vote
1answer
23 views

Variance explained by a set of variables (dimensionality reduction) [closed]

I am interested in estimating the amount of variance that can be accounted for by a set of variables. After reading this previous post where a similar question is answered but for only one variable: ...
1
vote
0answers
41 views

Long-run covariance matrix estimators with Ledoit-Wolf (2004) shrinkage; what methods exist?

Ledoit and Wolf ("A Well-Conditioned Estimator for Large-Dimensional Covariance Matrices", 2004) proposed an estimator for the covariance matrix of a data set, $S^* = p I_d + (1 - p) \hat{S}$ with $p \...
2
votes
1answer
23 views

Creating a covariance matrix for SEM

I would like to use lavaan for SEM. Specifically I want to use the paper: "Original Article Maximum Likelihood for Cross-lagged Panel Models with Fixed Effects", by Paul D. Allison, Richard Williams, ...
1
vote
0answers
14 views

Variance of Random Vector in the Circular Orthogonal Ensemble

Let $x$ be a (uniformly) randomly chosen column of a random orthogonal matrix (of size $K$ x $K$) distributed according to Haar measure. What is $\mathbb{E}[x]$, $\mathbb{E}[x x^T]$, $Cov(x, x)$, and $...
1
vote
0answers
35 views

data simulation from given covariance matrix from a regression model

I have a set of data. I ran a linear model fit to the data, let's just say to a general form: $af_1(x)+bf_2(x)+cf_3(x)+df_4(x)+e$ where $a,b,c,d,e$ are the fitted parameters. I am able to get the ...
1
vote
0answers
38 views

Correct error estimation for linear fit

This may be a simple problem, but I want to be thorough in setting up my problem as I'd like to know why I should proceed in one of two ways (or another if someone thinks it is suitable), so please ...
1
vote
1answer
59 views

Is the first principal component is the one with the largest eigenvalue and how to convert it to explained variance?

In PCA, after we calculate the eigenvalues of each variable, we need to get the explained variance, I read an article which suggests: ...
1
vote
1answer
25 views

What is a good measure of the similarity of 6 different time series?

Essentially, I have 6 different data time series that were each generated first using an industry standard methodology (call it method m.A) and then again using my technique (call it method m.B). ...
1
vote
0answers
17 views

Multiplying by vectors to assess covariance is zero

I want to prove the following but am unsure how. Show that if: For all fixed vectors $c$, $Cov(X,c'Yc) = 0$ Where $X$ and $Y$ are matrices of random variables, then it must be true that: $Cov(X,Y)=...
0
votes
0answers
6 views

Optimal weighted mean of multidimensional points with covariance estimates

I have multiple measurements of the position of an object in 2D/3D with normally distributed uncertainties. These uncertainties have corresponding covariance matrixes where the off-diagonal terms can ...
2
votes
1answer
55 views

Standard error of the intercept in Frisch-Waugh theorem (de-meaned regression)

I am applying in Frisch-Waugh Theorem to partial out a set of fixed effects D and get the OLS estimates and standard errors of the remaining regressors ...