Questions tagged [covariance-matrix]

A $k\times k$ matrix of covariances between all pairs of $k$ random variables. It is also called variance-covariance matrix or simply the covariance matrix.

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

Does invariance of PCA under orthogonal transformation hold for data that is not centered?

I read the proof in the top answer to this question, but that page assumes that $\overline{A} = 0$. If the data instead has some nonzero mean $\mu$, I'm not sure if the same logic applies: ...
4 votes
1 answer
62 views

Covariance matrix of multivariate normal when negative values are made zero

Let $x$ be $n$ dimensionally multivariate normally distributed with mean $\mu$ and covariance matrix $\Sigma$. Now let $y$ be random variables defined by \begin{equation} y_i= \begin{cases} 0, ...
0 votes
0 answers
55 views

Interpret Coefficient of Determination in matrix form

In matrix form, a linear regression can be represented in the following form: $$ Y \sim \mathbf{X} \beta + \epsilon; \\ \epsilon \sim N(0, \sigma^2 \mathbf{I}) $$ The definition of $R^2$ is the ...
2 votes
1 answer
21 views

How to interpret the endogenous factor matrix in the General Structural Equation Model?

Bollen (1989) introduces a general structural equation model of the (matrix) form: $$\eta = \beta \eta + \gamma \xi + \zeta$$ $$y = \lambda \eta + \epsilon$$ Most textbooks (e.g. Depaili 2021) ...
3 votes
2 answers
61 views

Creating a random sparse precision matrix?

In my current project, I want to create a random sparse precision matrix $\boldsymbol{P}=\boldsymbol{\Sigma}^{-1}$ (the inverse of a covariance matrix $\boldsymbol{\Sigma}$). My current procedure ...
  • 473
0 votes
0 answers
48 views

Interpretation of the elements of the error matrix as inverse of hessian matrix [duplicate]

In a report I am reading at work, the error matrix is calculated as the inverse of the hessian matrix and used to calculate the error ellipse angle and axes with a not theoretically correct formula. ...
0 votes
0 answers
29 views

Is it possible to know the actual covariance matrix, but fail to take-down heteroscedasticity and autocorrelation?

Is it possible to know the actual covariance matrix and estimate betas using $\Sigma$, but fail to defeat the problems caused by heteroscedasticity and autocorrelation and have biased or high-...
1 vote
0 answers
16 views

Addition of Covariate Matrices in Multi Normal Gaussian Addition

I want to merge two different distance functions for one set of observations $(x, y)$ to use in a gaussian process regression. I have the covariance matrix $\Sigma_1$ from distance function number 1 ...
1 vote
0 answers
47 views

Understanding the Covariance and Loading Matrices in Confirmatory Factor Analysis

I am currently stuck on how to correctly formulate the model for Confirmatory Factor Analysis(CFA). The general formula is $Y = \Lambda \xi + \epsilon$. According to Wikipedia $Y$ is a $p \times 1$ ...
0 votes
1 answer
30 views

Why is "design matrix of correlation parameters" a proxy for the "actual covariance matrix/working correlation matrix?

The example shows that knowing the design matrix of correlation parameters is sufficient to specify the working correlation. ...
0 votes
0 answers
25 views

How to obtain "normalized" model residuals for Generalized Estimating Equations?

How does one compute the "normalized" model residuals based via geepack's geeglm/gee in R? The nlme package in R allows one to compute the normalized model residuals: (standardized ...
1 vote
1 answer
37 views

$\mathrm{E}\left[u u^{\mathrm{T}}\right]=\sigma^{2} I_{n} \text { is untrue } \iff \text{heteroskedasticity?}$

$\mathrm{E}\left[u u^{\mathrm{T}}\right]=\sigma^{2} I_{n} \text { is untrue } \iff \text{heteroskedasticity?}$ I know heteroskedasticity $\implies \mathrm{E}\left[u u^{\mathrm{T}}\right]=\sigma^{2} I_{...
1 vote
0 answers
36 views

Why does PCA maximize variance between the standard deviations?

Consider an $n \times n$ covariance matrix $\Sigma$ (so semi positive-definite, symmetric and realvalued). We can find the $n$ principle components by $n$ times finding the direction of maximum ...
2 votes
0 answers
34 views

How to include an independent variable as an input to a statistical model

I want to create a statistical model of the following model inputs $v_{in}$ and outputs $v_{out}$ that consist of continuous random variables. The final aim is to calculate the conditional ...
1 vote
0 answers
52 views

Why PCA is invariant under rotation

Lets say that we have a matrix of variables (the columns are variables and rows are the observations) called X whenre X = [x1, x2, ...., xp] where ...
0 votes
0 answers
22 views

Sufficient conditions for positive, symetric, bivariate function to be a covariance

Let $f:\mathbb{R}_+^2\rightarrow \mathbb{R}$ be a symmetric function (i.e., $f(t_1, t_2) = f(t_2, t_1)$ for all $t_1, t_2 \in\mathbb{R}_+^2$). Are you aware of any extra condition that $f$ should ...
  • 11
3 votes
1 answer
67 views

Why is the expectation of a random vector still a vector?

In my previous post on derivation of covariance between y and random effect, for the following linear model: Frank's anwer proved cov(y, u) = ZD as below: Frank's proof involved E[u]. As far as I ...
  • 197
1 vote
0 answers
15 views

Updating the Posterior Psi parameter for an Inverse Wishart Distribution

I am fitting a Mixed Multivariate Normal Distribution where the mixing occurs over the mean $\mu_j$ and the covariance matrix $H$ with mixing parameter $B_j$. The number of mixing elements is denotes ...
3 votes
0 answers
39 views

Is it possible to build a confidence interval a covariance matrix or bootstrap samples of observed covariance matricies?

Is it possible to build a confidence interval for a covariance matrix? Matrices are the generalization of numbers. We are 95% confidence the true covariance matrix is in Point Estimate Covariance ...
1 vote
0 answers
25 views

CVXPY PSD constraint not working

I am using CVXPY to solve for a PSD matrix, example as follows: ...
0 votes
0 answers
12 views

Correlation of entries of the fundamental matrix of an absorbing Markov chain

Suppose I have an absorbing Markov chain with state space $S$, partitioned into $T$, the set of transient states, and $A$, the set of absorbing states. Let $N$ denote the fundamental matrix of this ...
  • 1,002
4 votes
1 answer
350 views

Can we construct a pair of random variables having any given covariance?

Consider a vector $r$ of $n$ random variables. Let $\mu = E[r]$ and $\Sigma$ denote the covariance matrix of $r$; that is, $$\Sigma_{ij} = \sigma_{ij} = E\Bigl[\bigl(r_i - E[r_i]\bigr)\bigl(r_j - E[...
  • 181
2 votes
0 answers
36 views

What is the distribution of the Frobenius distance between two covariance matrices?

I am computing the Frobenius norm of the difference between two covariance matrices, \begin{align} |\mathbf{C}-\mathbf{C}'|_F=\sqrt{\sum_{i,j}\left(c_{ij}-c'_{ij}\right)^2}. \end{align} Each of these ...
  • 61
1 vote
1 answer
31 views

Confusion about principal component and major axis of the ellipse corresponding to the covariance matrix

Based on my understanding, in PCA, we try to find a linear combination of axes such that the variance in that direction is maximized. If variables have the covariance matrix $\Sigma$, then, the first ...
  • 513
1 vote
0 answers
21 views

Estimation of the residual covariance matrix

Is the estimation of the residual covariance matrix under multivariate least squares the same as under maximum likelihood in a vector autoregression framework if we assume Gaussian white noise for the ...
  • 57
1 vote
1 answer
86 views

Calculating weighted covariance matrix of a weighted finite mixture of multivariate normal distributions

I am trying to calculate the weighted covariance matrix for a finite mixture of multivariate normal distributions. I read this post here and this one here, but the first post is focused on uniformly ...
1 vote
0 answers
23 views

Optimal combination of correlated estimations

Consider two random unbiased estimates $\hat X_1,$, $\hat X_2$ of a parameter (complex number) $x$, with estimation errors $E_1 = \hat X_1-x$, $E_2 = \hat X_2-x$. If the random variables $E_1$, $E_2$ ...
2 votes
0 answers
25 views

Rate of convergence of the eigenvalues of the samples covariance matrix

Assume $\{ x_i \}_{i=1}^n$ to be i.i.d. normally distributed with mean 0 and covariance matrix $\Sigma$. What can we say about the convergence of the eigenvalues of the samples covariance matrix $\...
  • 769
0 votes
0 answers
15 views

How to run a path analysis in a bayesian framework

I am trying to run a path analysis to understand how reproductive output (i.e., number of offspring that survive until adulthood) is directly versus indirectly related to exploration and parental ...
2 votes
1 answer
26 views

Determine if high dimensional data is multimodal

I have p-dimensional data and I need to determine if that data has significant modes or if it’s clustered in any way. Here p=50, (dense embedding), we have n samples and p <<< n. What are ...
  • 21
0 votes
1 answer
44 views

What does it mean to say that a covariance matrix is ​a positive definite matrix? [duplicate]

I'm doing a work on covariance matrix and this question came to me that was not very clear.
0 votes
0 answers
15 views

A fun example of unexplained variance: Ballantine diagrams in partialling out covariance

Please can you help with my understanding of this concept. If I was trying to calculate variance unexplained, in the example pictured, I assume this would be 1 - R2 for wine (X1) and Netflix (X2) ...
0 votes
0 answers
38 views

why estimator of covariance matrix will be very bad if samples smaller than random variables

I do not have strong math background but I am currently working on a project that requires me to use a covariance matrix. and it is my first time touching on this topic, I am reading a note, which ...
  • 1
0 votes
0 answers
25 views

The relationship between eigenvalues of a covariance matrix and the variances of the same data matrix after using eigenvectors as bases

Suppose we have a data matrix $\mathbf{X}\in \mathbb{R}^{M\times N}$ with $M$ features, $N$ samples and zero means ($M \lt N$). The covariance matrix of $\mathbf{X}$ is $\mathbf{C_x}=\frac{1}{N}\...
  • 1
0 votes
0 answers
23 views

Transformation of data with missing variables using first principal component in PCA

I have a set of N (normalized) metrics of length T. Unfortunately not all metrics start at the same point in time, so at the beginning of the window some metrics have missing values (NAs). I am trying ...
0 votes
0 answers
24 views

How to incorporate a variance-covariance matrix from abnormal returns and using this to weight the returns in computing the test statistic, using R?

I want to run GLS model in my research, using R. My general specification is the following (cross-sectional regression): Abnormal ~ Country_dummy + Industry_dummy + ESG + Cash + STD + LTD + Size + ...
  • 1
1 vote
0 answers
36 views

Generate a random covariance matrix with specified eigenspectra and diagonal elements and first off-diagonal?

I want to generate a random covariance matrix ($c \in \mathcal{R}^{n \times n} $) whose eigenspectra, i.e., $n$ eigenvalues $e_0 \in \mathcal{R}^{n\times 1}$ and diagonal elements $c_{ii} \,\, i=1 \,\,...
9 votes
3 answers
552 views

Is the sum of two singular covariance matrices also singular?

I have two sample covariance matrices, computed from $n$ samples, less than $p$ variables: they are singular then. I know that the sum of two covariance matrices is also a covariance matrix. My ...
1 vote
1 answer
97 views

Demonstration and Interpretation between a Fisher matrix and its dual space which is covariance matrix

I have a simple (maybe not) issue about the interpretation of the link between Fisher information matrix and its inverse which is the covariance matrix. How to formulate that a line of Covariance ...
  • 1
1 vote
1 answer
42 views

What is the cubic expectation (third-order moment) of a complex gaussian vector (say, E[$aa^{T}a$])?

Note: I also posted this question on MATHEMATICS. For a real gaussian vector, an explicit formula for the cubic expectation can be found in Matrix Reference Manual (search 'Cubic Expectations' in this ...
0 votes
0 answers
18 views

The Variance Covariance Matrix of an Estimator Stacking Two OLS Estimators

I am looking for how to derive the variance covariance matrix (henceforth, VCOV) of an estimator stacking two OLS estimators. Suppose that we have two OLS estimators: $$\hat{\alpha}\sim N(\alpha,\;\...
  • 661
0 votes
0 answers
25 views

Apply Shrinkage Coefficient to Exponentially Weighted Moving Covariance Matrices

Title says it all. I'm working on doing volatility forecasting and like the approach of exponentially weighted moving covariance matrices, but I also know that applying shrinkage coefficients further ...
0 votes
1 answer
50 views

Estimate the mean vector and the covariance matrix using the simple returns

I would appreciate help with how to to estimate the mean vector and the covariance matrix using the simple returns in R. I have historical (weekly) values of five stocks from a capital market for a ...
  • 23
0 votes
0 answers
45 views

Correlation matrix from pairwise correlations with specified structure

I need to simulate multivariate normal samples with a pre-specified correlation structure. The structure is such that the bigger the (GPS) distance between two points, the smaller the correlation (...
  • 184
0 votes
0 answers
57 views

Estimate 2D covariance from 3D matrix where 3rd column contains probability density values

I have an nx3 matrix where the first 2 columns contain uniformly distributed random (x, y) points and the 3rd column contains pdf values evaluated at each point. The pdf values are computed from a ...
  • 21
0 votes
0 answers
42 views

Variance-covariance structure for random-effects in `lme4` or `nlme` (covariance specification)

I am running a multilevel growth model with multiple random slopes. In the Mplus software, I can specify exactly which random effect covariances are estimated (and which are not). For example, I can ...
  • 1
0 votes
1 answer
34 views

Effect of using Cholesky transpose

I am generating random normal samples Y with covariance C using a well known procedure: Let L be the Cholesky decomposition of C, such that $C = LL^T$. Now given a matrix of random numbers $X, x_{ij}...
  • 709
2 votes
0 answers
33 views

Covariance matrix of beta coefficients for constrained multiple regression

I have a linear least-squares problem with constraints that two of the coefficients must be non-negative. For a typical (unconstrained) least squares estimation, I know that the variance-covariance ...
  • 21
2 votes
0 answers
51 views

Why do some poeple claim, that choice of the working correlation in GEE doesn't affect the marginal coefficients?

I found this discussion: GEE: choosing proper working correlation structure Cite: Correlation structure in GEE, unlike mixed models, does not affect the marginal parameter estimates (which you are ...
  • 51
0 votes
1 answer
47 views

What is the purpose to have the "independent" covariance structure in GEE or GLS?

The methods of estimation like GLS or GEE are especially helpful, when there are clusters of data, like repeated observations, many per cluster=subject. Such observations are naturally correlated in ...
  • 51

1
2 3 4 5
21