Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [covariance-matrix]

A $k\times k$ matrix of covariances between all pairs of $k$ random variables. It is also called variance-covariance matrix or simply the covariance matrix.

0
votes
0answers
15 views

Defining a covariance matrix [on hold]

I'm working with a Gaussian Process, and my covariance matrix is isotropic, i.e. it is defined just by the distance of the points locations. Suppose I'm working with Squared Exponential Covariance. ...
0
votes
0answers
10 views

Non-informative prior for the covariance matrix

I'm currently working on a project around the Bayesian approach to portfolio selection, and I can't manage to wrap my mind around the specification of the non-informative (diffuse) prior. Assuming ...
2
votes
1answer
50 views

Minimum variance of the mean for $n$ correlated random variables

If $X_1,\cdots,X_n$ all have the same variance equal to 1, then $0\leq \mbox{Var}[\bar{X}]\leq 1$ where $\bar{X}=(X_1 + \cdots + X_n)/n$. The upper bound is attained if $\mbox{Cov}[X_k,X_l]=1$ for all ...
3
votes
1answer
26 views

variance-covariance matrix with negative entries on mixed model fit

I am fitting a linear mixed effect model in R (function lme), and I get a Var-Cov matrix with negative entries (Log-Cholesky). This does not allow me to compute confidence intervals on the standard ...
0
votes
0answers
21 views

Notation problem with sparse regularized correlation matrix

I am trying to apply a specific method to obtain a sparse correlation matrix $R$ from a regularized correlation matrix $\Sigma^{\delta}$, which was computed from $N$ observations of a multivariate ...
0
votes
0answers
18 views

Interpretation of $det(X'X)$ in MLR

I would like to understand the interpretation of $det(X'X)$ in case of multiple regressors. $Var(x) = \sum_i^n(x_i-\bar{x})^2 = \frac{1}{n}\sum_i^nx_i^2 - \bar{x}^2 = \frac{1}{n}\sum_i^nx_i^2 - \frac{...
1
vote
0answers
29 views

Convergence of covariance matrix

I was looking for a simple way to find the number of samples $n$ needed to get a decent approximation to the covariance matrix $\boldsymbol{\Sigma}$. Given a random sample $\{ \mathbf{X}_1,\mathbf{X}...
0
votes
0answers
15 views

True covariance matrix in Monte Carlo simulation?

In a linear model, such that Population model is $y_{it}=\ {\beta_{i1}f}_1+{\beta_{i2}f}_2+\cdots{\beta_{ik}f}_k\ +\ \varepsilon_{it} , i = 1,2,3...,p$ and $t=1,2,3,...T$ and $\mathrm{\Sigma_y}\ =\ ...
0
votes
0answers
22 views

Finding inverse of $X'X$ in the case of two regressors [duplicate]

Variance of OLS etimator in matrix form look like this: $Var(\hat{\beta_j})=\sigma^2(X'X^{-1})$ I'm struggling to derive inverse matrix for the case with two independent variables. $X'X$ $=$ $\...
0
votes
0answers
18 views

Covariance matrix for multilevel data [closed]

I have multilevel data and I want to compute corresponding covariance matrix. Are there any methods or theory how can I do that?
1
vote
1answer
20 views

Computation of LDA in Elements of Statistical Learning 4.3.2

Elements of Statistical Learning 4.3.2 elaborates on computation for Linear Discriminant Analysis. https://web.stanford.edu/~hastie/Papers/ESLII.pdf Procedure is said to be • Sphere the data with ...
0
votes
1answer
37 views

Calculate first principal component direction and scores

Given that x1 = (9, 9, −18)^T and x2 = (18, 9, 9)^T with eigendecomposition of its sample covariance matrix Σ = cov(X) How do I calculate the first two principal component direction and the ...
0
votes
0answers
20 views

Should the predicted variance in Gaussian process regression include experimental error?

Suppose I have an experiment where I measure the temperature of water in a cup, $y$, as a function of time, $x$. My measurement is normally distributed with an experimental uncertainty given by $\...
0
votes
1answer
10 views

marchenko pastur for Correlation

It has been suggested to me that if I construct a covariance or correlation matrix using factor model then I can use the Marchenko-Pastur distribution to highlight significant correlations (or ...
0
votes
0answers
31 views

Non-positive variance-covariance in a linear mixed model

I have some problem with my lienar mixed model. When I do my model with the maximum likelihood, I have no error. But, when I want the confidence intervals, I get the Error : " cannot get confidence ...
0
votes
0answers
28 views

How do we prove this identity related to expectation and variance?

Prove that if $\textbf{a}$ is a vector of constants with the same dimension as the random vector $\textbf{X}$, then \begin{align*} \textbf{E}[(\textbf{X} - \textbf{a})(\textbf{X} - \textbf{a})^{\prime}...
0
votes
0answers
26 views

Multiple correlation coefficient from zero-order correlations (4 or more variables)

For my research, I am interested in calculating the multiple correlation coefficient(s) as a function of the "simple" (zero-order) correlations in a 4x4 (and larger) correlation matrix. For 3 ...
2
votes
1answer
83 views

How does one do a Wald test on estimates from two variables?

Given a dataset with two variables $X$ and $Y$, with each observation independent of the others, test the null hypothesis $$\mu_X = \mu_Y\\ \sigma^2_X = \sigma^2_Y$$ using a Wald test. This question ...
0
votes
0answers
36 views

Dimensionality in Gaussian Process regression

I have a hard time understanding what it means that in Gaussian Process (GP) regression, every point is a new dimension. I'm reading the distill article which usually does a very good job explaining ...
0
votes
0answers
7 views

Simultaneous non-significant variance parameters but significant co-variance parameters mixed models/random effects

I specified a linear mixed model in SPSS: TNA_HRQOL is the DV TIME_0 is a rep measures factor (0,1,2) which I specified as a covariate TK_Name (doctors) is LVL3 subjects, TNA_Name (individuals) is ...
0
votes
0answers
5 views

Given data and covariance matrix in Euclidean space, how to compute covariance matrix in Polar forms without computing the sample covariances?

For 2D, given (x,y) data points, and the covariance matrix C, how to compute the covariance matrix in polar form efficiently without transforming all data to polar form first then compute the sample ...
0
votes
0answers
25 views

Covariance Matrix: Unit Vectors

If you have a random vector $X = [X(1), X(2), ..., X(n)]$, dimension $d\times 1$. The probability of any vector to be like: $$X(1) = [1, 0, 0, 0..., 0]$$ is equal to $1/d$. I'm trying to calculate ...
1
vote
1answer
49 views

Checking error covariances between indicator variables in sem/cfa

I'm learning SEM/CFA, and am currently following Beaujean's (2014) book on using lavaan. In the chapter where he talked about CFA and the number of indicator variables to have to ensure the model ...
1
vote
0answers
13 views

numerically stable sparse gasussian process regression (matrix inversion)

In sparse approximations of GP for large data set $(X,\mathbf{y})$ with $n$ samples, usually $m$ inducing points are chosen such that the true covariance matrix is approximated by $K_{nn}\to K_{nm}K_{...
1
vote
0answers
11 views

Diffusion tensor as a covariance matrix

TLDR: In nuclear magnetic resonance (NMR), to study molecular diffusion we assume that molecules displace in 3D space according to a trivariate gaussian distribution. The variables are then the ...
1
vote
0answers
146 views

Modelling latent variables: lavaan warning: covariance matrix of latent variables is not positive definite

I have 3 independent variables (linguistic, onk2_sc3 and dsk2_sc3) and one dependent variable (Reading) in SEM multigroups (3 groups) comparison: ...
1
vote
0answers
21 views

Apply K-means to the columns of the covariance matrix

In Section 5.3 of the paper distilling the knowledge in a neural network, it says we apply a clustering algorithm to the covariance matrix of the predictions of our generalist model, so that a set ...
0
votes
0answers
25 views

Covariances and correlations in curve fitting

I have a set of data that I am trying to curve fit and I'm ultimately interested in the errors on my fit coefficients. I take my errors on each fit coefficient as the on-diagonal elements of the ...
2
votes
1answer
37 views

How to show sample correlation is sample covariance for standardized values?

Given a matrix $X$ and the resulting sample correlation matrix $R$, consider the standardized observations: $$\frac{(x_{jk} - \bar x)} {\sqrt{S_{kk}}} \quad k=1,2,...,p \quad j=1,2,...,n$$ Show that ...
4
votes
2answers
535 views

Inverse of the covariance matrix of a multivariate normal distribution

Is the covariance matrix of a multivariate normal distribution always invertible?
5
votes
1answer
95 views

Condition on the covariance matrix of a gaussian process needed to have the Markov property

Let suppose to have a realization $\mathbf{X}=(\mathbf{X}_1,\dots, \mathbf{X}_n)$, where $\mathbf{X}_i \in \mathcal{R}^d$, from a $d-$variate Gaussian process. Let also suppose that $E(\mathbf{X}_i)= ...
0
votes
0answers
64 views

Common covariance matrix explanation (LDA and QDA)

I'm looking for a layman's explanation of the "common covariance matrix" assumption in LDA because I don't think I understand it. I understand that a common covariance matrix (as assumed in LDA for ...
0
votes
0answers
32 views

Represent Mean-Squared-Prediction error as function of covariance (or Fisher) matrix

Given a simple linear model: $$ y_i = x_i^T \beta + \epsilon_i $$ For simplicity, $\epsilon_i$ is Gaussian iid with variance $\sigma_e^2$, then the solution for $\hat{\beta}$ is given via Ordinary ...
0
votes
0answers
10 views

Create covariance with blocks structure (artificially)

I would like to create some artificial variables that their covariance matrix will have blocks structure. Any idea how to do this? By saying "their covariance matrix will have blocks structure" I ...
0
votes
0answers
18 views

When the elements of first basis are always positive for PCA?

I am computing the PCA projection matrix of some data. I notice that the elements of first basis vector (corresponding to the highest eigenvalue) are always positive. My data is real and contain both ...
1
vote
0answers
70 views

Residual variance-covariance matrix in vector autoregression

It's my understanding that the general form of a variance-covariance matrix has variance terms on the diagonal and covariance terms on the off-diagonal. I have seen in multiple references (for ...
0
votes
0answers
15 views

Generating correlated data using numpy while controlling multicollinearity

I am using the following code (adopted from the code in this post). I have no problems with the code. My question is that if with this code I can create or prevent multicollinearity among the ...
1
vote
1answer
40 views

Calculating partitioned covariance matrices using R

Let's say my data consists of 25 observations of four features, which are in groups of 2. So we'll call my variables $x_1, x_2, y_1, y_2$. We have a partitioned sample mean vector given by $\begin{...
0
votes
1answer
41 views

Calculating variance of process with time-varying variance

This is a question stemming off a previous post I had regarding calculating portfolio volatility. For a portfolio consisting of multiple assets, I understand that there are multiple ways to calculate ...
2
votes
0answers
16 views

Causality in variance with a BEKK model

I am using a BEKK model in the following form, $$H_t=C^\ast{C^\ast}^\prime+\sum_{i=1}^{m}{A_i\varepsilon_{t-i}\varepsilon_{t-i}A_i^\prime+\sum_{j=1}^{s}{B_jH_{t-j}B_j^\prime}}$$ I first start with a ...
2
votes
1answer
35 views

Calculating portfolio volatility from portfolio returns vs. from covariance matrix

I'm having trouble understanding the difference in calculating portfolio volatility via the portfolio returns vs. via the covariance matrix. To be more specific: I understand that on the individual ...
2
votes
0answers
102 views

Covariance structures in glmmTMB for temporal autocorrelation

I'm running a zero-inflated, mixed-effects negative binomial model with the glmmTMB package in R. My current format: ...
0
votes
0answers
37 views

Explanation of covariance matrix of polynomial parameters [duplicate]

I'm asked to find the covariance matrix of $\alpha$, $\beta$, and $\gamma$ for: $$y_i=\alpha+\beta(x_i-\bar{x})+\gamma[(x_i-\bar{x})^2-\zeta^2]+\epsilon_i$$ where all the errors have equal variance $...
7
votes
0answers
148 views

Rao-Blackwellization in variational inference

The Black box VI paper introduces Rao-Blackwellization as a method to reduce the variance of the gradient estimator using score function, in section 3.1. However I don't quite get the basic idea ...
2
votes
0answers
30 views

Covariance of a covariance matrix [closed]

Given that covariance matrix, why is the covariance of Y and Z in this case "(-1 0)" or what would be the covariance of X and Y?
2
votes
1answer
51 views

Simplification of an expectation

While attempting to simplify a combination of expectations, I'm stuck at a particular term whose simplification I'm unable to deduce. The term to be simplified is: $\mathbb{E}[X^{T}F^{T}FX]$ where $...
0
votes
0answers
31 views

Checking equality of covariance matrices using Box's M test in multifactor MANOVA

With only one factor (independent variable) in multivariate analysis of variance (MANOVA), Box's M test can be used to check the equality of covariance matrices. ...
2
votes
1answer
84 views

Eigenvectors for correlation and covariance matrix PCA

I know the generally reasons of using correlation matrix vs a covariance matrix when doing PCA (and visa versa) however when thinking about the eigenvectors (principal components of the data) of each ...
0
votes
1answer
37 views

Sum of product as product of sums

Assuming I have two random non-independent vectors $A,B$ which are within [-1,1]. I want to approximate their sum of product by product of sums (everything is a dot product), i.e. $\sum_{i=1}^NA_iB_i ...
1
vote
1answer
20 views

relationship between correlation, covariiance and conditional distribution

What are the relationships between correlation and conditional distribution. For instance, given three dependent variables, X1, X2 and ...