Skip to main content

Questions tagged [covariance-matrix]

A $k\times k$ matrix of covariances between all pairs of $k$ random variables. It is also called variance-covariance matrix or simply the covariance matrix.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Issue with REML likelihood--logdet terms cancel

I'm trying to write an implementation of a linear mixed effects model using REML. I'm working with a simple model: $$ y_{ij} = X_{ij}\beta + Z_{ij}b_i + \epsilon_{ij} $$ In my case, the covariate $X$ ...
Alex's user avatar
  • 190
0 votes
0 answers
11 views

Bound of operator norm for Gaussian ensemble (wainwright example 6.2)

Consider $W \in \mathbb{R}^{n \times d}$ generated with i.i.d. $N(0,1)$ entries, theorem 6.1 in the Martin Wainwright HDS implies that $$ \frac{\sigma_\max(W)}{\sqrt{n}} \leq 1 + \delta + \sqrt{\frac{...
Mondayisgood's user avatar
0 votes
0 answers
7 views

Calculate HC3 robust vcov matrix when knowing only of Gradient and Hessian matrices of a GLM model

Suppose that a Generalized Linear Model is fitted using Maximum-Likelihood Estimation, but we only have access to two results from it: the gradient matrix $G$ is a $n \times p$ matrix where each row $...
Parlare's user avatar
0 votes
0 answers
47 views

Error in Bayesian Derivation of Covariance Matrix in Least Squares

I know variants of this question have been asked a million times, but rather than just asking "how do I derive the covariance matrix" I ask you to check the error in my calculations, because ...
geo's user avatar
  • 372
0 votes
0 answers
31 views

Confusion regarding PCA, FA, and PCR?

I learned here: Is PCA followed by a rotation (such as varimax) still PCA? About the relationship between PCA and FA and how they each provide a perspective for looking at the same thing. However, at ...
Ozz's user avatar
  • 1
0 votes
0 answers
33 views

Relating covariances for (θ, Χ) and (cos(θ), Χ)

From basic error propagation rules, we have σ(cos(θ)) = |sin(θ)| σ(θ). Question: does something similar hold for the covariance cov(cos(θ),X) and cov(θ,Χ)?
I_need_answers's user avatar
0 votes
0 answers
33 views

Distribution of the sample covariance of a multivariate exponential family

I am wondering if there is a known form for the distribution of the sample covariance matrix of a random variable that follows a multivariate exponential family distribution. I guess it would be a ...
aurora_s's user avatar
4 votes
1 answer
49 views

What is the difference between a matrix normal distribution and the multivariate gaussian distribution?

$\newcommand{\vec}{\operatorname{vec}}$Consider a set of $N$ matrices $X_1, X_2, \ldots, X_N$. I want to estimate the distribution of these matrices represented by the mean and covariance. I address ...
Ralff's user avatar
  • 242
2 votes
2 answers
50 views

Bounds of integration for the Wishart density

I once took a course that included zillions of exercises concerning the Wishart distribution, but as far as I recall, never mentioned the Wishart density. I asked something about that in this question,...
Michael Hardy's user avatar
1 vote
0 answers
33 views

Fitting a model with multiple inputs, multiple outputs, multiple parameters, and covariance matrices for each data point

This question is the theoretical counterpart to another question posted on StackOverflow, where I asked about the implementation of the fitting algorithm using Scipy or lmfit libraries for Python. ...
Swike's user avatar
  • 139
0 votes
0 answers
46 views

Expectation of $u^\top(u+Ax)$, when $A$ and $u$ are nonlinear functions of $x$

Let $x\in\mathbb R^d$, and $s=\operatorname{softmax}(x)$. Let $y$ be a fixed one-hot vector such that $$u = s-y \\ v =(\operatorname{diag}(s) - ss^\top)x$$ I am interested in the inequality $u^\top (u ...
Phoenix's user avatar
  • 101
0 votes
0 answers
21 views

Reconstruct Kernel from sampling a Gaussian Process

I am generating 50K draws from Gaussian Process with GPy in python. The Gaussian Process has a RBF kernel with length scale = 10....
apt45's user avatar
  • 273
1 vote
0 answers
54 views

Covariance matrix of sliding windows of a time series

I have a non-stationary time series $\{X_t\}$ (it is a stock price) and from this set I collect sliding windows of length 400 timesteps $\{Y^i_\tau\}$ where $i$ labels the window and $\tau \in [1,400]$...
apt45's user avatar
  • 273
1 vote
0 answers
19 views

Large $N$, small $T$ in SUR: workaround using system GMM

Consider a system of linear equations as in seemingly unrelated regression (SUR). If the number of equations $N$ is large relative to the sample size $T$, the weighting matrix in SUR (i.e. the error ...
Richard Hardy's user avatar
0 votes
0 answers
17 views

Coefficient matrix in terms of covariance [duplicate]

I'm currently reading a paper (White et al 2001) on the regression calibration method for addressing measurement error in studies, but am getting stuck on the set up in section 3.1 We have that $A$ ...
Jessica F's user avatar
0 votes
1 answer
26 views

Does Partial Correlation Affect Likelihood of Multivariate Normal? [duplicate]

Suppose I have a 3-dimensional multivariate normal distribution characterized by the following variance-covariance matrix $$ \begin{bmatrix} V_{X} & C_{XY} & C_{XZ} \\ C_{XY} & V_{Y} & ...
A Friendly Fish's user avatar
2 votes
1 answer
64 views

Why does system GMM fail due to computationally singular system in my setup?

I am estimating a system of seemingly unrelated regressions (SUR) with gmm::sysGmm in R. Each of the equations has one unique regressor and one common regressor. ...
Richard Hardy's user avatar
2 votes
1 answer
100 views

System GMM yields identical results for any weighting matrix

I am estimating a system of seemingly unrelated regressions (SUR) in R. Each of the equations has one unique regressor and one common regressor. I am using ...
Richard Hardy's user avatar
2 votes
1 answer
56 views

Why does `systemfit` yield different results for OLS and WLS under cross-equation restrictions?

I am following up on the question "Why does systemfit yield identical results for OLS and WLS?". It deals with estimating a system of linear equations ...
Richard Hardy's user avatar
0 votes
0 answers
38 views

Covariance inversion for Gaussian process

Background Let $x=f(u_x)\in\mathbb{R}$ and let $y=[f(u_y^1)\cdots f(u_y^{N})]\in\mathbb{R}^N$ for some function $f:u \in \mathbb{R}\mapsto \mathbb{R}$. Given $y$, $u_x$, $u_{y}^1,\dots, u_{y}^{N}$, I ...
matteogost's user avatar
2 votes
2 answers
89 views

How to test for equal spread in bivariate normal samples with equal means?

I'm working with samples taken from a bivariate normal distribution, where the differences in means is not relevant since all samples are scaled to mean (0,0) anyway, and I'm trying to remember how to ...
epistemetrica's user avatar
1 vote
1 answer
45 views

Heteroscedasticity-consistent (robust) standard errors complemented by i) confidence intervals for beta, ii) Tolerance and iii) VIF values in R?

In order to solve heteroscedasticity in my data, I ran a regression with heteroscedasticity-consistent ("robust") standard errors. I would also like to report i) standardized betas together ...
mbp's user avatar
  • 31
0 votes
0 answers
50 views

Mixed models: equivalence between residual covariance structure and random effects?

Is there a way to specify the covariance structure of a within-subject repeated measures model (MMRM with no random effects) such that the model is mathematically equivalent to a mixed model with ...
Evan's user avatar
  • 225
6 votes
2 answers
206 views

exclude random effects component for a repeated measure

I'm analyzing a dataset on the Nurse Licensure Exam, comprising 3000 participants. (n) These 3000 participants were randomly recruited from 13 Sites across the US. (group level variable) About 40% of ...
Sundown Brownbear's user avatar
1 vote
1 answer
40 views

Mapping covariance matrices with the datasets

Can someone please help me with this exercise and explain clearly to me? For me personally, when I look at this exercise, the matrix (2) stand out obviously, as it is a isotropic matrix so, it must be ...
Thai Mai's user avatar
1 vote
0 answers
17 views

Joint Distribution Formulation of a Spatial X, Spatial Y, and Spatial Error Model

Introductory Problem: I have $n$ points in 3-D space, where I know their X and Y coordinates (not Z), and therefore the distances between points in those 2 dimensions. Each of the three dimensions has ...
A Friendly Fish's user avatar
2 votes
1 answer
88 views

Distance matrix of actual dataset doesn't obey triangle inequality, leading to non-positive definite covariance matrix

Yesterday I asked a question about why my randomly generated distance matrices were leading to Matérn covariance matrices that were not positive definite. The answer there called my attention to the ...
hannah's user avatar
  • 63
3 votes
1 answer
157 views

Calculating Matérn covariance in R returns matrices that are not positive definite

In R, I am trying to calculate Matérn covariance matrices whose inputs are randomly created distance matrices. However, I often end up getting covariance matrices that are not positive definite, ...
hannah's user avatar
  • 63
0 votes
0 answers
31 views

Bootstrap method with 2 Fisher matrices in order to do the cross-correlations between both

I have 2 Fisher matrices where each colum/row represents the information (in Fisher's sense) of astrophysical parameters. These parameters are in the same order for both matrices. Now, I would like to ...
foutou_10's user avatar
3 votes
2 answers
134 views

GMM derivation for diagonal covariance matrices

I was trying to understand the derivation of M step in the EM algorithm for GMM. All the resources available consider only "full covariance" matrices. I wanted to implement GMM for "...
Equation_Charmer's user avatar
1 vote
0 answers
50 views

Sampling from a very high dimensional Gaussian

I would like to a sample from a Gaussian $N(0,K)$ where $K$ is a kernel gram matrix, so that $K=[K_{ij}]$ with $K_{ij} = k(x_i,x_j)$ for some positive definite function $k$. The first issue is that ...
WeakLearner's user avatar
  • 1,501
4 votes
2 answers
139 views

How do we interpret the covariance matrices $\textbf{U}$ and $\textbf{V}$ in the Matrix Variate Normal Distribution?

Consider the Matrix Normal Distribution. My first question is: how do we interpret the entries $\textbf{X}_{ij}$ of the random matrix $\textbf{X}(n\times p)$? My second question is: how do we ...
user1234's user avatar
  • 245
1 vote
2 answers
63 views

Understanding the multicollinearity issue in relation to linear regression

There are 2 issues that multicollinearity in linear regression leads to Interpretability goes for a toss Parameter confidence intervals are wide and it is difficult to find a parameter significant I ...
figs_and_nuts's user avatar
2 votes
1 answer
37 views

Is the assumption of a diagonal covariance matrix on the latent space in a variational autoencoder in any way restrictive?

The covariance matrix in an autoencoder is assumed to be diagonal. And, I see it mentioned in good places that this is a fairly restrictive assumption. To quote However, in order to simplify the ...
figs_and_nuts's user avatar
0 votes
0 answers
15 views

Clarification on covariance matrix for multidimensional Gaussian distributions

It is a well known property of Gaussian distributions that if $Y = (Y_1, \ldots, Y_n)$, where each $Y_i$ is a real Gaussian random variable, then the components of $Y$ are independent if and only if ...
CBBAM's user avatar
  • 311
1 vote
1 answer
68 views

Intuition behind between-group covariance matrix from MANOVA?

Suppose that we have samples from $m$ different $p$-dimensional normal multivariate distributions, where they share a common covariance matrix $\Sigma$ but the mean vectors may be different for each ...
Bergson's user avatar
  • 69
2 votes
1 answer
68 views

Passing a cholesky decomposition for a matrix with constrained variances to an objective function

I am trying to optimize an objective function $L(\theta)$ in which some parameters that I aim to recover belong to a covariance matrix, $\Sigma$. $\Sigma$ has a unique structure, which includes ones ...
EB727's user avatar
  • 33
0 votes
0 answers
36 views

Decompose covariance matrix into uncorrelated and correlated part

I have an almost diagonal covariance matrix, I would like to decompose it in an uncorrelated and correlated part: $$ \Sigma = \Sigma_U + \Sigma_C $$ where all the matrices above are covariance ...
Ruggero Turra's user avatar
2 votes
0 answers
184 views

Prove that modified RBF function satisfies Mercer conditions

Suppose that I have a modified RBF kernel function. $k(\mathbf{x},\mathbf{y}) = \exp{(-||\mathbf{x}-P\mathbf{y}||^2 })$ where $\mathbf{x},\mathbf{y}$ represent $d$ dimensional inputs and $P$ is the ...
flammmes's user avatar
0 votes
1 answer
26 views

Is this formula the calculation of covariance matrix

I came across this formula in a text that says $S$ is the sample covariance matrix where $$S = \sum_{j=1}^n(\mathbf{X}_j - \bar{\mathbf{X}})(\mathbf{X}_j-\bar{\mathbf{X}})'$$. What I am trying to ...
John Smith's user avatar
0 votes
0 answers
51 views

Multivariate sample covariance

I have a set of $X_1,...,X_n$ samples with covariance $\Sigma_1,...,\Sigma_n$. The multivariate sample mean is then $$ \left(\sum_{i=1}^n \Sigma_i^{-1} \right)^{-1} \left(\sum_{i=1}^n \Sigma_i^{-1} ...
ThibautOphelia's user avatar
2 votes
0 answers
39 views

Interpreting eigenvalues of non-normalized covariance matrix of physical system

Cross-posted from physics stackexchange Summary: Eigenvalues of a "non-normalized" covariance matrix of time-series measurements from a linear system have units of Action (energy * time). ...
user3716267's user avatar
1 vote
1 answer
18 views

Efficient construction of correlation matrix—serial correlation

Given $\rho$, is there a way to efficiently construct this matrix (i.e., as a product of matrices, rather than using a for loop)? $$ \Sigma = \begin{pmatrix} 1 & \rho & \rho^2 &\cdots &...
veloskaraptor's user avatar
0 votes
0 answers
101 views

Exponentially Weighted Covariance Matrix with Ledoit Wolf Shrinkage

The Ledoit Wolf paper "Honey, I Shrunk the Sample Covariance Matrix" presents the formulation for the shrinkage intensity parameter estimate in Appendix B. The formula for a weighted ...
nka5we's user avatar
  • 49
1 vote
0 answers
27 views

Sample Random Effects from Mixed-effects Model

I have a mixed effects model that I'd like to simulate random effects from using a MVN approximation (sampling from the predictive distribution). My question is what is the advantage of simulating ...
user13317's user avatar
  • 675
9 votes
2 answers
179 views

Origin of the term "spherical" in relation to covariance matrices?

I understand that a covariance matrix with all diagonal elements equal, and all off-diagonal elements also equal (but different to the diagonal elements) is called "spherical". I am curious ...
Robert Long's user avatar
  • 63.9k
0 votes
0 answers
19 views

Distribution of covariance parameter resulting from sum of known- and unknown-covariance noise processes

Let's say I have a set $X$ of $N$ random $p$-dimensional vectors generated by $\mathbf{x}_i = \boldsymbol{\mu} + \Psi_i^{1/2} \boldsymbol{\xi}_i + \Sigma^{1/2} \boldsymbol{\zeta}_i$, where $\...
hank's user avatar
  • 11
5 votes
1 answer
93 views

difference between GLM covariance matrix from MLE vs. IRLS for non-canonical link

Someone asked a question on Stack Overflow where they noted a difference between Minitab and R (glm) results for the variance-covariance matrix of the parameters, ...
Ben Bolker's user avatar
  • 44.5k
4 votes
0 answers
143 views

Fast Cholesky decomposition of a Toepllitz matrix via embedding in a circulant & fft

As I understand it, the Cholesky decomposition of a Toeplitz matrix can be computed more efficiently by first embedding it in a circulant matrix then using FFT, but I'm having trouble finding any ...
Mike Lawrence's user avatar
0 votes
0 answers
19 views

Conditions of the covariance matrix between discrete and continuous variables

Does the covariance matrix for a discrete variable and a set of continuous variables have extra constraints beyond being positive semi-definite as in the case of a real-valued random vector? ...
Sergio's user avatar
  • 336

1
2 3 4 5
23