Questions tagged [covariance-matrix]

A $k\times k$ matrix of covariances between all pairs of $k$ random variables. It is also called variance-covariance matrix or simply the covariance matrix.

Filter by
Sorted by
Tagged with
0
votes
1answer
21 views

Creating a Covariance Matrix

Lets say that you have the correlation of x,y and you have the standard deviations of x and y , how would you then find the covariance of x,y using the correlation of x,y and and the standard ...
0
votes
0answers
6 views

How to transform the covariance matrix for a GHK simulator?

I've used an MCMC Gibbs sampler to estimate parameters of the Multinomial Probit Model. In doing so, I've differenced all utilities with respect to the last alternative, such that I obtained (J-1) ...
0
votes
0answers
29 views

How $\Sigma = AA^T$ derives to $A = \Sigma^{1/2}$ [closed]

This is how I start, but I'm stuck on how to proceed. I think I'm missing some algebra properties. $\Sigma=AA^T$ $\Sigma^{-1} = (AA^T)^{-1}$
0
votes
0answers
14 views

Creating non-negative definite matrix for multivariate spatial random field?

I am trying to model a spatial multivariate (three variate) random filed. I have defined correlation matrix using LMC as function of distance lag of interested points $\mathbf{C(h}{{\mathbf{)}}_{3\...
2
votes
1answer
44 views

Find the covariance matrix of the random variables [duplicate]

What is the covariance matrix of $$f = 2x + 3y$$ if random variables x,y are independent and have a covariance matrices $\sum_{x}$ $\sum_{y}$? I know the covarince matrix of a random variable is ...
0
votes
0answers
18 views

Is there an alternative to Box's M test for data that is not multivariate normally distributed?

My research question is to test whether two groups have a difference in variance-covariance across multiple measures. However, the data do not follow the multivariate normality assumption required for ...
1
vote
0answers
18 views

Singular the contemporaneous covariance matrix of error terms in VAR

I'm interested in the case if we have i.e. VAR(1) model: $$ \mathbf{y}_t = \Phi \mathbf{y}_{t-1} + \mathbf{\epsilon}_t, \qquad \mathbb{E}[\mathbf{\epsilon}_t\mathbf{\epsilon}_t^T] = \Omega, $$ where ...
0
votes
0answers
7 views

Equivalence of gaussian process and bayesian linear regression by inspecting the covariance matrix

I'm aware that a gaussian process is equivalent to bayesian linear regression for the kernel $K(x_i,x_j) = x_i x_j$ (assume scalar $x$ here). However, the proof itself didn't lend much intuition to me....
0
votes
0answers
7 views

Covariance structure with two time points

I am looking at change in score of an outcome Y using mixed-effects Poisson regression.I noticed that in the software I am using (SAS with PROC GLIMMIX for fitting generalized linear mixed models) the ...
0
votes
0answers
16 views

Clarity on Covariance Matrix and it's relation to length

I have been trying hard since a month to get a clear intuition on the relation of the covariance matrix to length of projections(maximum or otherwise). While I understand conceptually that if I have a ...
4
votes
1answer
77 views

Estimate of $\text{Var}(\hat\beta)$ in a linear mixed model

Let $Y = X\beta + Zu + \sigma\epsilon$ be a Gaussian linear mixed model. Let $V = Var(Y)$ be the marginal variance matrix of $Y$. Define the matrix $$ \Phi = {(X'V^{-1}X)}^{-1}. $$ According to this ...
0
votes
0answers
18 views

The difference of Standard Error from vcov() and sigma() of poisson regression by glm in R

I got the SE of glm(y=x) by the following program. But I couldn't get the same SE as vcov() and sigma() by the following program glm(y ~ x, family=poisson(log)). I can't understand the difference of ...
0
votes
0answers
12 views

Scaled covariance for error calculation

I am trying to do some fits (linear fits) and I just discovered (I am new to statistics) that most fitting programs have a parameter that can turn on and off the covariance scaling (e.g. in Python ...
0
votes
0answers
20 views

GDA producing negative covariance determinant

I'm running Gaussian Discriminant Analysis across a large set of examples (~80k) in $\mathbb{R}^{8}$. I know that the covariance matrix $\Sigma$ is, by definition, positive semi-definite, which means ...
1
vote
1answer
11 views

Invertibility of covariance matrix when number of training examples are lesser than number of features

I was trying to study an outlier detection algorithm and realized that in case we use a multinomial Gaussian distribution to model data then the invertibility of Covariance matrix ($\sum$) is ...
3
votes
2answers
100 views

How to calculate the Jacobian of the transformation ( for covariance matrix)

I'm reading this Paper about a separation strategy for modeling covariance matrices with focus on Bayesian analysis. Direct decomposition of covariance matrix is as follows: $\Sigma = \text{diag}(S)\,...
1
vote
0answers
17 views

Generate Multivariate Log-Normal Variables with given Covariance and Mean

Let ${\bf X}=(X_1,...,X_n)$ be an $n$-dimensional log-normal random variable. I want to $force$ my random variables to be such that $Cov(X_i,X_j)=\Sigma_{i,j}$ and $E(X_i)=\mu_i$ where $\Sigma_{i,j}$ ...
0
votes
2answers
41 views

Relationship between sample size and parameter covariance matrix in OLS

I am dealing with a linear system of equations that I am solving by OLS: $$ \mathbf{y} = \mathbf{X} \mathbf{p} + \mathbf{e} $$ Where I have $n$ samples and $k$ parameters ($\mathbf{X}$ is an $n \...
0
votes
0answers
17 views

Shrink a covariance matrix in R

I have a covariance matrix estimate for which I want to shrink the eigenvalues of the matrix to obtain an better estimate. In R various packages offer covariance shrinkage like in Ledoit Wolf(2004) ...
2
votes
1answer
38 views

Singular covariance matrix in GMM?

I understand that typically the covariance matrix should not be singular (see e.g. this discussion here: Could the covariance matrix of the moment conditions in GMM be ill-conditioned?) But in the ...
0
votes
0answers
32 views

Importance of absolute values of the covariance matrix in the nonlinear mixed models

I am fitting a nonlinear mixed model (three-parameter logistic function) without any hierarchical structure. I have adopted an unstructured variance-covariance structure for the random effects. Is it ...
2
votes
1answer
56 views

Expectation of Quadratic Form with Two Random Vectors

Assume I have two independent $(N \times 1)$ random vectors, $\epsilon_{1} \sim N(0,\Sigma_1)$ and $\epsilon_{2} \sim N(0,\Sigma_2)$. We could assume $\Sigma_1=\Sigma_2$ for my purposes but a general ...
0
votes
0answers
18 views

R - factor model - R-squared calculation

I am trying to recalculate in R an example of a factor model presented in the Zivot, Wang book (Modeling Financial Time Series with S-PLUS) p.548 (link) I am looking for an explanation of the ...
0
votes
0answers
9 views

Sampling prior covariance matrices - nested sampling

I am trying to fit a multivariate Gaussian with a non-diagonal covariance matrix $\Sigma$ using nested sampling. Usually, in other Bayesian analyses, we would use a Inverse Wishart or LKJ prior on ...
2
votes
1answer
94 views

Find $\delta$ such that sparse covariance matrix is positive definite

Good day, I was looking through some papers to help with my project assignment that wants me to implements 2 lasso approaches. I am having trouble simulating the samples from a MVN distribution. $\...
4
votes
1answer
350 views

Has anyone studied linear regression where the covariance matrix of the error is a function of the parameters being estimated?

Consider the multivariate linear model: $$y = X\beta + e$$ $y$ is the measured output, $X$ is the model matrix, $\beta$ is the parameter vector, and $e$ is the zero-mean error vector: $$E[e] = 0 \...
2
votes
1answer
49 views

Why is a sample covariance not even semi-positive definite with missing data?

I am trying to estimate a sample covariance when I have less observations $n$ than variables $p$ ($n<p$). This will serve later on as basis for a shrinkage estimator. We know (see this post) that ...
0
votes
0answers
23 views

What is the relation between $cov(X_1,X_2)$ of the Random variables and the $cov(F_{X_1}(X_1),F_{X_2}(X_2))$

I have a question about the covariance matrix. In copula literature and applications many covariance matrix estimations are being used. Let $X_1, X_2$ as two random variables. For producing samples ...
0
votes
0answers
20 views

Can the coefficients and standard errors for a nested model be derived from the variance-covariance matrix of the full model

If you have a generalised linear model like this:- $$ f(y) = \beta_0 + \beta_1x_1 + \beta_2x_2 $$ and you know the coefficients and variance covariance matrix for the betas, but don't have access to ...
2
votes
1answer
58 views

Estimating two-parameter covariance matrix, with average variance and average covariance?

Which methods can I use if I want to estimate a highly-structured two-parameters covariance matrix, when I have few observations $N$ for many variables K? Motivation: given a dataset of K=3000 ...
1
vote
1answer
122 views

On the properties of covariance and kernel matrices

I'm stumbling upon an example of a mixed model or a Gaussian Process, say: $Z \in\mathbb{R}^{n \times m}, m \ge n$ ie random effect $X \in\mathbb{R}^{n \times p}, p \ge 1$ ie fixed effects $K \in\...
3
votes
1answer
28 views

covariance matrix derivation

How to derive from the first step to the second step, where y is k by 1 vector and A is k by p matrix. Thanks for any help in advance
0
votes
1answer
53 views

Upper bound trace of inverse of covariance matrix

Let C be the covariance matrix from any normal distribution. If the trace of C is upper-bounded by a constant k (i.e., tr(C)<=k), can I find an upper bound for the trace of the inverse of C (i.e., ...
0
votes
1answer
17 views

Finding the covariance of $u_{it} = \nu _{it} - \theta \nu _{i\left ( t-1 \right )}$ for $t>1$

Given $u_{it} = \nu _{it} - \theta \nu _{i\left ( t-1 \right )}$ for $t>1$ $u_{i1} = \nu _{i1}$ and the $\nu _{it}$ are white noise with variance equal to $\sigma^{2}$. I can find the expected ...
0
votes
0answers
8 views

Energy Vector and covariance matrix, intuition

I am reading about Gaussian processes and I am encountering a lot of terms like this: $r^T\Sigma r$ or $r^T\Sigma^{-1}r$ or $r^T\Sigma^{-1}y$ Where $\Sigma$ is a covariance matrix I ...
1
vote
0answers
36 views

Sum of Two Bivariate Normal Distributions

I'm just confused on how to set up and start this problem. I'm confident that once I start down the right path, I'll have little issue. Let $p_1$ denote a bivariate normal distribution $N(0, 0, 1, 1, ...
0
votes
0answers
38 views

Covariance matrix as a sum of two covariance matrices

Suppose that a random vector $\mathbf{n}$, $$\mathbf{n} = \mathbf{n}_A + \mathbf{n}_B \ , \tag{1}$$ can be written as a sum of two random vectors $\mathbf{n}_A$ and $\mathbf{n}_B$, that are ...
0
votes
0answers
24 views

Relation between eigenvalues of original and transformed matrices

Let the matrix $X$ be some data arranged in rows. Consider the following eigenvalue decomposition $X^\top X = Q \Theta Q^\top=\sum_{i=1}^n \theta_iq_iq_i^\top$ where $q_i$ are the eigenvectors and $\...
2
votes
3answers
134 views

Why does the variance of the OLS estimator scale with $\frac{1}{n}$?

Let $\mathbf{X}$ be an $n\times p$ matrix. In multiple linear regression, we have $$\boldsymbol{\hat{\beta}}=\mathbf{[X^TX]^{-1}X^Ty}\sim\mathcal{N}\Big(\boldsymbol{\beta}, \sigma^2 \mathbf{[X^TX]^{-...
0
votes
0answers
13 views

Unbiasedness and Variance of Predictions

Here is the problem I'm working on: I'm not quite sure if I'm showing either unbiasedness property right, and am stuck on finding the expressions for the variances. Here's what I've done so far. (a) ...
0
votes
1answer
24 views

Properties of Covariance (Matrix and Vector Case)

In 1 dimensional case, we have $Cov(cX_1,X_1) = cCov(X_1,X_1).$ Here $X_1$ is just a random variable. I was wondering if we have an analogue for $Cov(AX_1,X_1)$, where $A$ is a matrix with ...
1
vote
0answers
15 views

How do I construct a variance-covariance matrix from a matrix formulation of a MLR? [duplicate]

I'm trying to calculate the SE for each coefficient given by a matrix formulation of MLR by the root of each diagonal in the so-called variance-covariance matrix, but I'm unsure how to construct this ...
0
votes
1answer
27 views

Mahalanobis distance - understanding the formula [duplicate]

I've read quite a few explanations on this topic, liking this one the most: https://mccormickml.com/2014/07/22/mahalanobis-distance/ But there is still one thing I don't understand. I understand ...
0
votes
0answers
22 views

Distribution of the squared norm of a vector with multivariate normal distribution and dependent components [duplicate]

Let x be a p-dimensional random vector with dependent components. Assume that x is distributed according to a multivariate normal distribution with mean vector m and variance/covariance matrix V which ...
0
votes
1answer
27 views

Why the first principal component is mostly negative while the second component is mostly positive?

I am running PCA for a fleet management data frame $X$, where each column is a city, each row is a date, there are 50 cities and 500 dates. I run PCA on $A=X^{T}X$. Then the first component $v_{1}$ ...
0
votes
1answer
71 views

How to use inverse information matrix and Delta method to find sample variation?

Explain how the inverse of the information matrix and the Delta Rule be used to generate an approximate sampling variance for the estimated proportion exp(beta 0 + beta 1)/(1+exp( beta 0 + beta 1))? ...
0
votes
0answers
27 views

CFA, Covariance matrix is not positive definite?

I have a 3-factor CFA model with 16 items. I conducted 8 groups comparison.For one group, Amos analytic result showed " The following covariance matrix is not positive definite." The sample size for ...
0
votes
0answers
34 views

How to create a variance-covariance matrix for forecasted fantasy basketball scores?

I have three basketball players who have played in games together and I want to find a Variance-Covariance matrix that will be as accurate as possible for their fantasy points in an upcoming game. My ...
0
votes
0answers
20 views

Covariance Matrix (Long/Short Portfolio; Different Weightings)

I am attempting to calculate the expected one-day standard deviation of a portfolio in dollars. In other words, I am looking for the following: "I expect my portfolio to move _______ dollars on ...
0
votes
0answers
23 views

Estimate variance of parameters in non-linear ridge regression

I am basically estimating the shape of an unknown function f(x) from multi-dimensional chemical reaction data by estimating the most likely function values $f$ on a grid $x$ with kernel regression. To ...