Questions tagged [covariance-matrix]

A $k\times k$ matrix of covariances between all pairs of $k$ random variables. It is also called variance-covariance matrix or simply the covariance matrix.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

When to use k * (k+1)/2 (e.g. covariances and variances) vs. k * (k-1)/2 (e.g. variances only) for number of observed statistics in CFA/SEM?

Two formulas are often used for estimating the number of observed statistics in a CFA or SEM. Either: k * (k+1)/2 for the lower triangle of covariances and variances k * (k-1)/2 for the lower tiangle ...
1
vote
0answers
23 views

Matrix Covariance Algebra

In the structural equation modeling (SEM) context, one of the modeling frameworks is called the reticular action model (RAM). In RAM, the observed variables (y) and latent variables (η) are combined ...
0
votes
0answers
8 views

How to compute the covariance error term in an astrophysics context?

I have posted initially on physics exchange but don't get any answer, so I try hopefully here which seeems to be a more appropriate forum (I am going to delete the initial post on physics exchange). ...
0
votes
1answer
19 views

Does lack of data affect covariance matrix estimate?

I am building some experiments using the multivariate normal probability density function to estimate the likelihood of a given sample to come from a distribution. For that, the PDF is built using as ...
0
votes
1answer
16 views

Combining two covariance matrices — Multiplying two multi-variate Gaussian PDFs

I want to multiply two Normal probability density functions, $$ {\displaystyle f_{\mathbf {X} }(x_{1},\ldots ,x_{k})={\frac {\exp \left(-{\frac {1}{2}}({\mathbf {x} }-{\boldsymbol {\mu }})^{\mathrm {T}...
0
votes
1answer
50 views

convergence rate of sample covariance matrix

I have a question about deriving the rate of convergence of sample covariance matrix. For the sake of simplicity, we can assume that our sample $\{ X_i\}_{i=1}^{n}$ is i.i.d. (I known we can relax ...
0
votes
0answers
11 views

Defining an LKJ prior not centered at zero

I am working with a Hierarchical Bayesian Model. In each of its units, I need to define a covariance matrix between 2 variables. I am planning to sample the covariance matrix from the LKJ prior. ...
1
vote
1answer
31 views

Using eigendecomposition to transform state vector in linear Gaussian state space model

Paper: A Unifying Review of Linear Gaussian Models by Roweis & Ghahramani The generative model is the typical state space model written as \begin{align} \text{state transition equation: }{\bf x}_t ...
2
votes
1answer
29 views

Marginal Covariance of State Vector in a Linear Gaussian State Space Model

Paper: A Unifying Review of Linear Gaussian Models by Roweis & Ghahramani The generative model is the typical state space model written as \begin{align} \text{state transition equation: }{\bf x}_t ...
1
vote
0answers
7 views

Estimate Variance-Covariance matrix via Error Propagation of Weighted Least Squares Equation

Given a linear system $b_{obs} = Ax$, how can I derive the covariance of $x$ (i.e. $C$) from the weighted least square solution equation: $$x = (A^TS^{−1}A)^{-1}A^TS^{−1}b_{obs}$$ With $C$ being the ...
0
votes
1answer
33 views

Covariance matrix for ranking data

I am testing concordance between boys and girls on their rankings of 9 items. The literature says this can be done by comparing their mean rank vectors under the null hypothesis that the mean rank are ...
5
votes
1answer
104 views

Is there a formula for the determinant of the covariance matrix $\mathbf{X_n}^T \mathbf{X_n}$ in the case of multiple regression?

Consider the standard simple linear regression model: $$ Y_i = \beta_0 + \beta_1 X_i + \epsilon_i, $$ for $i=1,\dots,n$. In matrix-vector form this is $$ \mathbf{Y} = \mathbf{X_n}\beta + \epsilon, $$ ...
0
votes
1answer
26 views

What does “having excellent Likelihoods” mean ? (MCMC code) [closed]

I asked an astrophysicist about MontePython code (MCMC code). He told me that its team had excellent Likelihoods about a cosmological survey. What does "having excellent Likelihoods" mean ? ...
0
votes
1answer
42 views

Portfolio theory: confusion about variance-covariance matrix

I am taking an introductory course to finance in my Master's, and wanted to go further in the topic of portfolio theory (I am an engineering bachelor graduate, but as I just hinted, I am new to ...
1
vote
0answers
23 views

Does $E[\hat \Sigma^{-1}] \to \Sigma^{-1}$ still hold for samples drawn from a non-normal population?

For a sample of observations $\{x_i\}_{i=1}^n$ where $x_i=(x_{i1},\dots,x_{ik})^T$ of a population random vector $X=(X_1,\dots,X_k)^T$, the population covariance is $$ \Sigma = E[(X-E[X])(X-E[X])^T], $...
2
votes
1answer
29 views

Prove that the following matrix is positive definite

We define $K_{\mathbf{a}, \mathbf{b}}$ as the $n \times m$ matrix whose $ij^{th}$ entry is $\kappa(a_{i}, b_{j})$ Where, $\kappa$ is a (positive definite) kernel function. Here, $\mathbf{a}_{i}, \...
0
votes
0answers
13 views

Property of Covariance Matrices and Symmetric Matrices

I have a question about covariance matrices. I have read one interesting property that, all symmetric matrices are diagonalizable. Suppose we have a data matrix $X$ that has only $m$ independent ...
0
votes
0answers
13 views

Is the Karhunen–Loève transform optimal only for Gaussian data?

I read in [1, Sec. I] that "the Karhunen–Loève transform is the best to decorrelate discrete jointly gaussian random variables". Is that true? Is Gaussianity really required for optimality? ...
1
vote
1answer
22 views

Is there an intuition about the matrix operations in the exponent of the multivariate normal distribution?

In the exponent of the multivariate distribution, there are 2 vectors and a square matrix multiplied together to get a scalar result: $$(\mathbf{x} - \mu)^{\text{T}}\Gamma^{-1}(\mathbf{x} - \mu)$$ ...
0
votes
0answers
14 views

Signed circular-linear correlation or covariance?

In my recent project, I am implementing an adaptive MCMC procedure for a set of variables, some of which are linear (e.g., temperature in Celsius), some of which are circular (e.g., direction in ...
0
votes
0answers
5 views

Estimate the error on gaussian fit parameters as function of curve sampling points

My function is a Gaussian: $$ y(x_i,\mathbf{a})=a_1 e^{-(x_i-a_2)^2/(2a_3)^2} $$ which is sampled at discrete locations $x_i$. According to the definition the covariance matrix $C$, can be obtained ...
1
vote
0answers
15 views

Is it valid to assume that principal components follow Gaussian distribution?

My professor said that we can assume that principal components follow Gaussian distribution. If we consider the covariance matrix as a linear operator and all data points as linearly transformed data ...
0
votes
0answers
12 views

How to reduce a covariance matrix after clustering?

I have an N = 100 covariance matrix. I am clustering the covariance matrix say into 5 clusters. How can I compute the reduced ...
0
votes
1answer
56 views

The variance of variance-covariance matrix

Looking at answer on the standard error of the variances and this answer on the standard error of the covariances, and knowing that both are part of the variance-covariance matrix, $\Sigma$, I am ...
0
votes
0answers
20 views

What if zero mean assumption is relased in graphical LASSO?

I am working on a graphical LASSO (GLASSO) shrinkage of the variance-covariance matrix of financial log-returns data for 10 years. The objective of the graphical LASSO is: $$\ell(0,\Sigma) = {-\text{...
1
vote
0answers
22 views

The variance matrix of the unique solution to linear regression [duplicate]

By minimizing the mse, we get that the unique solution to this optimizing problem is $\beta = (X^TX)^{-1}X^Ty$. Why is its variance matrix $Var(\beta)=(X^TX)^{-1}\sigma^{2}$, where we assume that the ...
0
votes
0answers
39 views

Calculate similarity between two matrices

I have two matrices, $A$ and $B$, each of size $n\times m$, where $n$ is discrete time points, and $m$ are the variables measured (specifically, $n$ are dates and $m$ are investments measured in ...
0
votes
1answer
26 views

Covariance and Correlation Matrices

I have a somewhat dumb question. When determining the correlation or covariance (doesn't matter I suppose) amongst random vectors, is the covariance computed among features or among observations? For ...
1
vote
0answers
13 views

Finding Covariance from linear algebra projection chapter

I am trying to solid background of linear regression by using linear algebra. In linear algebra, there are some chapters that related to linear regression. (orthogonality, Projection) I learned some ...
1
vote
0answers
25 views

Proof that variance-covariance matrix of var(b|X) - var(b*|X) is positive-semidefinite [closed]

I'm having trouble finding the proof to show that the variance-covariance matrix of var (b|X) - var (b*|X) is positive-semidefinite. OLS estimator = GLS estimator = Hint: Note that A is the Cholesky ...
0
votes
0answers
13 views

PCA, correlation of the features of projected data [duplicate]

I have just got familiar with some methods of dimensionality reduction, and one of them was PCA. So we have data $X\in \mathbb{R}^{n\times n}$ and want to reduce its dimension to $k$. PCA just takes ...
0
votes
0answers
31 views

Covariance matrix as a linear transformation

I am trying to understand the general relationship between the covariance between two random variables and linear transformations. For example, consider the figure here: https://en.wikipedia.org/wiki/...
1
vote
1answer
21 views

What is the relationship betwen covariance, covariance matrix and cross-correlation?

I am not good at math and apologize if I made stupid mistakes in this question. Assume I have two random variables X = [1, 2, 7, 3, 0, 10 ...] and Y = [12, 1, 5, 7, 9, ...]. According to the ...
0
votes
0answers
12 views

Covariance matrix in multivariate is Wishart random variable

I'm working on sample variance distribution, on page 111 of Methods of Multivariate Analysis, it says "The joint distribution of these $p(p + 1)/2$ distinct variables in $W =(n−1)S = \sum_i(y_i − ...
0
votes
0answers
24 views

Modelling random effects covariance structure with lme - why this won´t work?

I have a few questions here. I am lacking some understanding regarding modelling the random effects covariance stucture and some specific parts of my model. First the covariance structure modelling ...
0
votes
1answer
16 views

Compute covariance matrix given probability density fonction [closed]

I'm given a probability function for 2 random variables and I have to compute the covariance matrix of the vector they form.
1
vote
1answer
52 views

Why is $E(ee')$ a matrix?

One of the assumptions of linear regression is that $E(ee') = \sigma^2 * I $, where I is the identity matrix and sigma squared is the variance of residuals. Why is $E(ee')$ a matrix, though? $ee'$ is ...
0
votes
0answers
5 views

Implication of a kronecker product that is Martignale Difference Sequence

I have only come across the Kronecker product of two matrices in the past, and as such I find this equation that I have encountered in an Econometric Theory paper (surrounding sign-based inference) ...
0
votes
1answer
27 views

Covariance matrices and independency [duplicate]

If we have a diagonal covariance matrix does that guarantee independency?
0
votes
0answers
16 views

Covariance matrix in Multivariate Gaussian Process

I am working on Gaussian Processes Regression and having some trouble understanding how to properly state the covariance matrix of 3 random vectors. Say I have an input space of 3 dimensions, $X$, $Y$ ...
0
votes
0answers
19 views

Sample Covariance of a Data Matrix

I am having a hard time interpreting the sample covariance of this data matrix. From what I understand, the rows of the data matrix X are x1, x2,...xn - given by the transpose. Now, I am slightly ...
0
votes
1answer
42 views

Optimal Portfolios with Skewed and Heavy-Tailed Distributions

I am learning about portfolio theory and been using Markowitz. The Markowitz problem is an optimization problem of a series of Gaussian distributions (symmetric) with a variance-covariance matrix, to ...
2
votes
0answers
27 views

Intuitive interpretation of the covariance matrix

I am trying to get a "feel" for the covariance matrix. I know that $v_iCov(X)v_j$ gives the covariance between $X$ projected along $v_i$ and $v_j$. I'm curious if there's a similar intuitive ...
0
votes
0answers
23 views

Aligning multivariate Gaussian distributions

I have two multivariate Gaussian variables $\mathbf x_0\sim\mathcal N(0,\Sigma_0)$ and $\mathbf x_1\sim\mathcal N(0,\Sigma_1)$, which are generated by almost the same process. Specifically $\mathbf ...
1
vote
0answers
17 views

Reference request: (spectral) convergence rate of sample covariance matrix with fixed dimension $p$

I am looking for a reference on convergence of sample covariance matrix (in some reasonable sense) when the dimension $p$ is fixed, but the number of samples $n$ goes to infinity. The ideal result I ...
0
votes
0answers
28 views

How to compute the covariance matrix of a multivariate bernoulli distribution?

Considering this toy example: Let $x$ be a random variable $x \sim \mathcal{N}(\mu_x, \Sigma_x)$ Where $\mu_x \in \mathbb{R}^2$ is the mean vector and $\Sigma_x \in \mathbb{R}^{2 \times 2}$ is the ...
0
votes
0answers
11 views

How is covariance matrix affected if each data points is multipled by some constant?

I have a 2D multivariate Normal distribution with some mean and a covariance matrix. While fitting the function I had normalized the data.so the mean and covariance I have are for the normalized data. ...
6
votes
1answer
65 views

Multivariate Normal Distribution: Divide each random variable by its standard deviation

If $X$~$Normal(\mu,\Sigma)$, and I divide each random variable in $X$ (the marginals) by its standard deviation, what will happen to the covariance matrix $\Sigma$?
0
votes
0answers
11 views

Why eigen vector of a covariance matrix is the largest principle components? [duplicate]

I am self studying principle component analysis using this tutorial, I got most of the reasoning behind PCA but I don't get the intuitive reason why eigen vectors of a covariance matrix is also its ...
0
votes
1answer
24 views

Covariance Linear Shrinkage Estimator : Implied Data

I have been using linear shrinkage to better estimate the covariance matrix when I do not have enough data. Let $\bf X$ be a $T\times n$ matrix representing the data (previously centered) and $${\bf S}...

1
2 3 4 5
18