# Questions tagged [covariance-matrix]

A $k\times k$ matrix of covariances between all pairs of $k$ random variables. It is also called variance-covariance matrix or simply the covariance matrix.

1,141 questions
Filter by
Sorted by
Tagged with
10 views

### Issue with REML likelihood--logdet terms cancel

I'm trying to write an implementation of a linear mixed effects model using REML. I'm working with a simple model: $$y_{ij} = X_{ij}\beta + Z_{ij}b_i + \epsilon_{ij}$$ In my case, the covariate $X$ ...
• 190
11 views

64 views

### Why does system GMM fail due to computationally singular system in my setup?

I am estimating a system of seemingly unrelated regressions (SUR) with gmm::sysGmm in R. Each of the equations has one unique regressor and one common regressor. ...
• 68.6k
100 views

### System GMM yields identical results for any weighting matrix

I am estimating a system of seemingly unrelated regressions (SUR) in R. Each of the equations has one unique regressor and one common regressor. I am using ...
• 68.6k
56 views

### Why does systemfit yield different results for OLS and WLS under cross-equation restrictions?

I am following up on the question "Why does systemfit yield identical results for OLS and WLS?". It deals with estimating a system of linear equations ...
• 68.6k
38 views

### Covariance inversion for Gaussian process

Background Let $x=f(u_x)\in\mathbb{R}$ and let $y=[f(u_y^1)\cdots f(u_y^{N})]\in\mathbb{R}^N$ for some function $f:u \in \mathbb{R}\mapsto \mathbb{R}$. Given $y$, $u_x$, $u_{y}^1,\dots, u_{y}^{N}$, I ...
• 373
89 views

### How to test for equal spread in bivariate normal samples with equal means?

I'm working with samples taken from a bivariate normal distribution, where the differences in means is not relevant since all samples are scaled to mean (0,0) anyway, and I'm trying to remember how to ...
1 vote
45 views

### Heteroscedasticity-consistent (robust) standard errors complemented by i) confidence intervals for beta, ii) Tolerance and iii) VIF values in R?

In order to solve heteroscedasticity in my data, I ran a regression with heteroscedasticity-consistent ("robust") standard errors. I would also like to report i) standardized betas together ...
• 31
50 views

### Mixed models: equivalence between residual covariance structure and random effects?

Is there a way to specify the covariance structure of a within-subject repeated measures model (MMRM with no random effects) such that the model is mathematically equivalent to a mixed model with ...
• 225
206 views

### exclude random effects component for a repeated measure

I'm analyzing a dataset on the Nurse Licensure Exam, comprising 3000 participants. (n) These 3000 participants were randomly recruited from 13 Sites across the US. (group level variable) About 40% of ...
1 vote
40 views

### Mapping covariance matrices with the datasets

Can someone please help me with this exercise and explain clearly to me? For me personally, when I look at this exercise, the matrix (2) stand out obviously, as it is a isotropic matrix so, it must be ...
• 13
1 vote
17 views

### Joint Distribution Formulation of a Spatial X, Spatial Y, and Spatial Error Model

Introductory Problem: I have $n$ points in 3-D space, where I know their X and Y coordinates (not Z), and therefore the distances between points in those 2 dimensions. Each of the three dimensions has ...
88 views

### Distance matrix of actual dataset doesn't obey triangle inequality, leading to non-positive definite covariance matrix

Yesterday I asked a question about why my randomly generated distance matrices were leading to Matérn covariance matrices that were not positive definite. The answer there called my attention to the ...
• 63
157 views

### Calculating Matérn covariance in R returns matrices that are not positive definite

In R, I am trying to calculate Matérn covariance matrices whose inputs are randomly created distance matrices. However, I often end up getting covariance matrices that are not positive definite, ...
• 63
31 views

### Bootstrap method with 2 Fisher matrices in order to do the cross-correlations between both

I have 2 Fisher matrices where each colum/row represents the information (in Fisher's sense) of astrophysical parameters. These parameters are in the same order for both matrices. Now, I would like to ...
134 views

### GMM derivation for diagonal covariance matrices

I was trying to understand the derivation of M step in the EM algorithm for GMM. All the resources available consider only "full covariance" matrices. I wanted to implement GMM for "...
1 vote
50 views

### Sampling from a very high dimensional Gaussian

I would like to a sample from a Gaussian $N(0,K)$ where $K$ is a kernel gram matrix, so that $K=[K_{ij}]$ with $K_{ij} = k(x_i,x_j)$ for some positive definite function $k$. The first issue is that ...
• 1,501
139 views

### How do we interpret the covariance matrices $\textbf{U}$ and $\textbf{V}$ in the Matrix Variate Normal Distribution?

Consider the Matrix Normal Distribution. My first question is: how do we interpret the entries $\textbf{X}_{ij}$ of the random matrix $\textbf{X}(n\times p)$? My second question is: how do we ...
• 245
1 vote
63 views

### Understanding the multicollinearity issue in relation to linear regression

There are 2 issues that multicollinearity in linear regression leads to Interpretability goes for a toss Parameter confidence intervals are wide and it is difficult to find a parameter significant I ...
• 2,603
37 views

### Is the assumption of a diagonal covariance matrix on the latent space in a variational autoencoder in any way restrictive?

The covariance matrix in an autoencoder is assumed to be diagonal. And, I see it mentioned in good places that this is a fairly restrictive assumption. To quote However, in order to simplify the ...
• 2,603
15 views

### Clarification on covariance matrix for multidimensional Gaussian distributions

It is a well known property of Gaussian distributions that if $Y = (Y_1, \ldots, Y_n)$, where each $Y_i$ is a real Gaussian random variable, then the components of $Y$ are independent if and only if ...
• 311
1 vote
68 views

### Intuition behind between-group covariance matrix from MANOVA?

Suppose that we have samples from $m$ different $p$-dimensional normal multivariate distributions, where they share a common covariance matrix $\Sigma$ but the mean vectors may be different for each ...
• 69
68 views

### Passing a cholesky decomposition for a matrix with constrained variances to an objective function

I am trying to optimize an objective function $L(\theta)$ in which some parameters that I aim to recover belong to a covariance matrix, $\Sigma$. $\Sigma$ has a unique structure, which includes ones ...
• 33
36 views

### Decompose covariance matrix into uncorrelated and correlated part

I have an almost diagonal covariance matrix, I would like to decompose it in an uncorrelated and correlated part: $$\Sigma = \Sigma_U + \Sigma_C$$ where all the matrices above are covariance ...
184 views

### Prove that modified RBF function satisfies Mercer conditions

Suppose that I have a modified RBF kernel function. $k(\mathbf{x},\mathbf{y}) = \exp{(-||\mathbf{x}-P\mathbf{y}||^2 })$ where $\mathbf{x},\mathbf{y}$ represent $d$ dimensional inputs and $P$ is the ...
• 21
26 views

### Is this formula the calculation of covariance matrix

I came across this formula in a text that says $S$ is the sample covariance matrix where $$S = \sum_{j=1}^n(\mathbf{X}_j - \bar{\mathbf{X}})(\mathbf{X}_j-\bar{\mathbf{X}})'$$. What I am trying to ...
51 views

101 views

### Exponentially Weighted Covariance Matrix with Ledoit Wolf Shrinkage

The Ledoit Wolf paper "Honey, I Shrunk the Sample Covariance Matrix" presents the formulation for the shrinkage intensity parameter estimate in Appendix B. The formula for a weighted ...
• 49
1 vote
27 views

### Sample Random Effects from Mixed-effects Model

I have a mixed effects model that I'd like to simulate random effects from using a MVN approximation (sampling from the predictive distribution). My question is what is the advantage of simulating ...
• 675
179 views

### Origin of the term "spherical" in relation to covariance matrices?

I understand that a covariance matrix with all diagonal elements equal, and all off-diagonal elements also equal (but different to the diagonal elements) is called "spherical". I am curious ...
• 63.9k
19 views

### Distribution of covariance parameter resulting from sum of known- and unknown-covariance noise processes

Let's say I have a set $X$ of $N$ random $p$-dimensional vectors generated by $\mathbf{x}_i = \boldsymbol{\mu} + \Psi_i^{1/2} \boldsymbol{\xi}_i + \Sigma^{1/2} \boldsymbol{\zeta}_i$, where \$\...
• 11
93 views

### difference between GLM covariance matrix from MLE vs. IRLS for non-canonical link

Someone asked a question on Stack Overflow where they noted a difference between Minitab and R (glm) results for the variance-covariance matrix of the parameters, ...
• 44.5k
143 views

### Fast Cholesky decomposition of a Toepllitz matrix via embedding in a circulant & fft

As I understand it, the Cholesky decomposition of a Toeplitz matrix can be computed more efficiently by first embedding it in a circulant matrix then using FFT, but I'm having trouble finding any ...
• 13.9k