Skip to main content
14 votes

Is it possible to have a multivariate random distribution with all its random variables (pair-wise) reverse correlated?

If $X \mathrel{:=} \left(X_1, \ldots, X_k\right)^\top \sim \mathop{\mathrm{Multinomial}}\left(n, \left(p_1, \ldots, p_k\right)^\top\right)$ with $n \in \mathbb N_{\geq 1}, k \in \mathbb N_{\geq 2},$ ...
statmerkur's user avatar
  • 6,195
13 votes
Accepted

Beyond AR(1) as a covariance structure for mixed models with repeated measures

When we encounter models with repeated measures in time, we naturally want to account for the correlations within subjects and also the possibility of heterogeneous variances across time points. When ...
Robert Long's user avatar
  • 62.5k
13 votes
Accepted

Is it possible to have a multivariate random distribution with all its random variables (pair-wise) reverse correlated?

Mathematically, as I commented, you can use the equi-correlation matrix \begin{align*} \Sigma = \begin{bmatrix} 1 & \rho & \cdots & \rho \\ \rho & 1 & \cdots & \rho \\ \vdots ...
Zhanxiong's user avatar
  • 20.4k
9 votes
Accepted

Calculating Matérn covariance in R returns matrices that are not positive definite

Your problem is not in the Matérn part. It is already in the "distance" part, i.e. with your matrix $d$. In your code, $d$ is just a (slightly fudged) symmetric random matrix with zeros on ...
g g's user avatar
  • 2,708
7 votes

Origin of the term "spherical" in relation to covariance matrices?

This form of covariance matrix is actually more "elliptical" than "spherical" I have not heard of this form of matrix being called a "spherical" covariance matrix, and ...
Ben's user avatar
  • 127k
7 votes
Accepted

Measuring level of uncorrelation from correlation matrix?

It turns out that if you take the inverse of the correlation matrix, then take the reciprocal of the diagonal elements of the inverse, the result is one minus the $R^2$ values from the regression ...
Greg Snow's user avatar
  • 52.2k
7 votes

Measuring level of uncorrelation from correlation matrix?

Perhaps the tolerance statistic is something that could be helpful for you. It is defined as 1 - R^2 for a given variable, where R^2 is calculated based on a linear regression of that variable on all ...
Christian Geiser's user avatar
7 votes
Accepted

exclude random effects component for a repeated measure

If you want to capture the subject correlation in a multilevel model, I think you have to include it as a random effect, and this doesn't really depend on wanting to predict individual scores over ...
Peter Flom's user avatar
  • 124k
6 votes

Why would SAS proc mixed produce residual variance estimates under AR1 covariance structure, but none for unstructured covariance?

The AR(1) covariance matrix is: $$ \begin{bmatrix} \sigma^2 & \rho\sigma^2 & \rho^2\sigma^2 & \rho^3\sigma^2 \\ \rho\sigma^2 & \sigma^2 & \rho\sigma^2 & \rho^2\sigma^2 \...
Robert Long's user avatar
  • 62.5k
6 votes

Is it possible to have a multivariate random distribution with all its random variables (pair-wise) reverse correlated?

Let X,Y have a negative covariance/correlation, and define a third variable as a linear sum of those two plus independent noise $\epsilon$ $$Z = -X -aY + \epsilon$$ Now, this will be inverse ...
Sextus Empiricus's user avatar
6 votes

Beyond AR(1) as a covariance structure for mixed models with repeated measures

The choice of the correlation pattern to fit is a crucial one that affects the efficiency and bias of estimates of the main effects of interest. AR(1) tends to be a very competitive fit in terms of ...
Frank Harrell's user avatar
6 votes

exclude random effects component for a repeated measure

Although the (ICC) based on SubjectID suggests moderate correlation (0.5-0.7), I've opted not to include SubjectID as a random component in the model due to lack of interest in predicting individual ...
Christian Hennig's user avatar
5 votes
Accepted

difference between GLM covariance matrix from MLE vs. IRLS for non-canonical link

The likelihood score for a glm is $$U_i= x_i\frac{d\eta}{d\mu}V^{-1}(\mu)(y-\mu)=D^TV^{-1}S.$$ The derivative of this product has three components, two of which are multiples of $S$ and so are mean ...
Thomas Lumley's user avatar
5 votes

In SAS, does the using the GLM statement ABSORB invalidate the standard errors of the parameter estimates?

In SAS, if you ABSORB a variable in PROC GLM, it adjusts its effects but does not include it in the VCV matrix of the estimated ...
Robert Long's user avatar
  • 62.5k
5 votes
Accepted

How do we interpret the covariance matrices $\textbf{U}$ and $\textbf{V}$ in the Matrix Variate Normal Distribution?

In the context of the Matrix Normal Distribution, the entries $X_{ij}$ are samples from the Gaussian distribution with mean $M_{ij}$. To fully characterise their variance though, we need two matrices $...
usεr11852's user avatar
  • 44.7k
4 votes

Interpolate covariance matrix

Could you consider this an optimal transport problem in translation between Gaussians? For example, the 2-Wasserstein distance interpolation is given in e.g. Mallasto, Anton, Augusto Gerolin, and Hà ...
Klimaat's user avatar
  • 143
4 votes

Is there an intuitive interpretation of $A^TA$ for a data matrix $A$?

I think it is more intuitive to think of $A^\top A$ as a bilinear form than as a linear map. Say we have a linear transformation $\mathbb{R}^n \to \mathbb{R}^m$ with matrix $A$ (with respect to the ...
Steven Gubkin's user avatar
4 votes
Accepted

How to Implement Newey-West covariance matrix properly for MDE estimation

I found the solution using a heuristic way. As mentioned in the comments obove, the size of the sum in ($4$) seems a bit problematic. After analysing the results, I took the upper bound of the sum as $...
Erdem Şen's user avatar
4 votes
Accepted

GMM derivation for diagonal covariance matrices

TL;DR while the diagonal form of the covariance matrix indeed has some effect on the final estimates for the variance terms (which are not easily available in the general case), the formulae are still ...
Spätzle's user avatar
  • 4,012
3 votes

Origin of the term "spherical" in relation to covariance matrices?

I wil try to expand on @whuber's comment to the OP. I hope I'm getting this right ! A covariance matrix is expressed as $\Sigma$. It encapsulates how each pair of dimensions in a dataset co-varies. ...
underflow's user avatar
  • 313
3 votes

Is it possible to have a multivariate random distribution with all its random variables (pair-wise) reverse correlated?

YES (I find this fact surprising, too.) ...
Dave's user avatar
  • 64.2k
3 votes

Interpolate covariance matrix

The problem of interpolating covariance matrices is an active area of research. Let's imagine that in your problem the covariance matrices are changing smoothly as given by an unknown function $f$ of ...
dherrera's user avatar
  • 1,298
3 votes
Accepted

lavaan's estimated residuals output different than manually estimated residuals

I just came cross the same problem, and found out that lavaan standardized the sample first before compuation. Acutually you can access this standardized observed covariance matrix by the command <...
Sola Cong Mou's user avatar
3 votes

How do we interpret the covariance matrices $\textbf{U}$ and $\textbf{V}$ in the Matrix Variate Normal Distribution?

The $i$th column of $X$ has a covariance matrix $V_{ii}U$, and the $j$th row of $X$ has a covariance matrix $U_{jj}V$. Hence, you can think of $U$ as the covariance of the columns, and of $V$ as the ...
Car Loz's user avatar
  • 850
3 votes
Accepted

Distance matrix of actual dataset doesn't obey triangle inequality, leading to non-positive definite covariance matrix

Don't fudge the diagonal. Replace the final diagonal in m by sigma**2 to get rid of the (removable) singularity from 0 * Inf in the original expression. Your ...
g g's user avatar
  • 2,708
3 votes

Why does system GMM fail due to computationally singular system in my setup?

The main reason is that your moment matrix is badly conditioned. This is specific to your dataset. Using momentfit, we can try to see what happens if we compute the ...
Pierre Chausse's user avatar
2 votes

Is there a way to use the covariance matrix to find coefficients for multiple regression?

The answer above is complete and great. The following aims to complement the answer with a different perspective. I also want to bring attention to the fact that using covariances to estimate ...
Tomas da Nobrega's user avatar
2 votes

Why does correlation matrix need to be positive semi-definite and what does it mean to be or not to be positive semi-definite?

@Dilip Sarwate mentioned why a covariance matrix $\Sigma$ is positive semidefinite, namely because it does not make sense to think of the variance of a linear combination of random variables as ...
Taylor's user avatar
  • 20.9k
2 votes
Accepted

How to calculate covariance matrix properly using Bartlett's formula

A few days more research and I finally found the answer. With large sample size ($n$) this equation gives a good aproximation to $C_{m\times m}$ (with $m < n$ as an arbitrary value). There is a ...
Erdem Şen's user avatar
2 votes

Incrementally Computing $\Sigma_t v_t$ Without Storing $x_i$, $v_i$, or $\Sigma_t$

You ask how to perform a sequential update of the vector sequence $$\sigma_n = \sum_{t=1}^n \Sigma_t v_t = \sum_{t=1}^n \sum_{i=1}^t x_i x_i^\prime \, v_t.$$ To update $\sigma_n$ to $\sigma_{n+1}$ you ...
whuber's user avatar
  • 327k

Only top scored, non community-wiki answers of a minimum length are eligible