Questions tagged [covariance-matrix]

A $k\times k$ matrix of covariances between all pairs of $k$ random variables. It is also called variance-covariance matrix or simply the covariance matrix.

Filter by
Sorted by
Tagged with
0
votes
1answer
17 views

how to get total Fisher matrix that makes cross synthesis of 2 Fisher matrix

I have initially posted on physics.stackexchange but I think my issue is more adapted on Cross-Validated (so I am going to delete the initial post on physics.stackexchange). I have 2 Fisher matrixes ...
2
votes
1answer
93 views

Why does the condition number of the covariance matrix explode as number of variables increases?

From asset returns of $N$ stocks, the symmetric covariance matrix sized $N\times N$ is constructed, which treats the asset returns as variables. When the number of variables $N$ is fairly low like $N=...
2
votes
0answers
12 views

How to remove the covariance of two measurement methods in order to separately estimate the variance of each

I wish to compare two different methods of measuring an underlying property, and wish to extract the variance of each method of measurement, independent of the other. The problem is how to correctly ...
4
votes
1answer
52 views

Prove that sample covariance matrix is positive definite [duplicate]

Consider the $p \times p$ sample covariance matrix: $$\mathbf{S} = \frac{1}{n-1} \cdot \mathbf{Y}_\mathbf{c}^\text{T} \mathbf{Y}_\mathbf{c} \quad \quad \quad \mathbf{Y}_\mathbf{c} = \mathbf{C} \mathbf{...
2
votes
1answer
36 views

Do we assume graphical LASSO explanatory variables to be normally distributed? And what if this assumption fails?

I am working on a graphical LASSO (GLASSO) shrinkage of the variance-covariance matrix of financial log-returns data for 10 years. I tested for normality and the Jarque-Bera test (but also other tests)...
0
votes
1answer
22 views

Why aren't the coskewness and cokurtosis matrices square like the covariance matrix?

The variance-covariance matrix is shaped $p\times p$, whereas the co-skewness matrix is shaped $p\times p^2$ and the co-kurtosis matrix is $p\times p^3$. Why is this, given that skewness and kurtosis ...
2
votes
1answer
51 views

Multivariate gaussian bivariate gaussian proof

I'm having trouble seeing how the multivariate gaussian formula evaluates to the bivariate gaussian. See multivariate PDF, source: http://cs229.stanford.edu/section/gaussians.pdf [![multivariate][1]][...
1
vote
0answers
28 views

Proof multivariate normal distribution

Definition: A random vector $X$ has a (multivariate) normal distribution if for every real vector $a$, the random variable $a^TX$ is normal. Theorem: If $X$ has p-variant normal distribution then $\mu=...
1
vote
0answers
16 views

Low rank plus diagonal approximation of covariance of model parameters

In this paper regarding Bayesian Deep Learning, in Section 3.4, the authors want to approximate the covariance of a distribution over the model parameters $\theta$. They first get an estimate of the ...
1
vote
1answer
59 views

Proof: If a matrix is semi-definite and symmetric positive then it is a covariance matrix

Anyone have the following proof? If a matrix is semi-definite positive and symmetric then it is a covariance matrix.
0
votes
0answers
8 views

Estimation of means and variance from 10 independent samples of varying sizes

Consider a stochastic computer simulation that outputs a unique integer value x and a unique float value y. I am trying to ...
0
votes
0answers
8 views

getting same AIC (or any other comparison criterion values) even after using different var-cov structures when comparing GLMM models

We are comparing models that are GLMM , in which for each one of them the fixed effects are exactly the same, but in the random effects portion, we used different variance-covariance structures (i.e ...
0
votes
0answers
33 views

Does the uncertainty of the ratio of two distributions suppose to decrease with correlation?

I need to calculate the variance of the ratio B/(A+B) between two correlated distributions A and B which are known with their variances. ...
1
vote
0answers
46 views

Projecting a covariance matrix to a lower dimensional space

I have a point $\mathbf{x}$ in 3-dimensional space, which is measured with a degree of uncertainty. The point falls within a unit cube, and the uncertainty is assumed to follow a multivariate normal ...
0
votes
0answers
25 views

Shrunken covariance matrix in the Sparse inverse covariance selection

The original version of the L1 regularization method uses sample covariance matrix ${\mathbf{S}}$ as follows: \begin{equation} \hat{\mathbf{\Omega}}= argmin_{\mathbf{\Theta}\succ 0} \bigg(tr(\...
1
vote
0answers
12 views

How to avoid matrices when designing models for multivariate problems? [closed]

Multivariate mathematical problems are often resolved classically with some form of a model that uses the covariance matrix, or some similar device, to collect statistics of multivariate data into one ...
1
vote
0answers
25 views

Estimating the random effect structure and variance-covariance matrix for RRs from observational studies

long time reader, first time poster. Have found this community to be extremely helpful but alas have not had luck finding a previous question relating to this: I am attempting to run a meta-regression ...
2
votes
0answers
72 views

Upperbound for Norm of Lagged Autocovariance Matrix

Suppose $X_t$ is a vector valued time series in $\mathbb{R}^d$. Assume for the moment that $X_t$ is stationary with $EX_t=0$, $E\|X_t\|^2<\infty$, and let $$ C_h = E[X_0 X_h^\top] $$ denote the ...
2
votes
0answers
19 views

Covariance matrix of regularized likelihood

My question is how to estimate the covariance matrix of parameters in a regularized likelihood maximization. Lets assume we have constructed some negative log-likelihood with a set of parameters and a ...
-1
votes
1answer
37 views

What came first, the covariance or the correlation matrix?

Covariance can be calculated from correlation and correlation can be calculated from covariance. Is it sufficient to refer to them as only transformations of one another? Is one considered more of a ...
3
votes
1answer
49 views

Correlation is to covariance, what mutual information is to --?

The information theory equivalent of the correlation matrix is the mutual information matrix, which has individual entropies along its diagonal, and mutual information estimates in the off-diagonals. ...
2
votes
0answers
16 views

Decomposition of a Gaussian Markov random field in independent subfields

A zero-mean GMRF (i.e., a multivariate normal distribution whose precision matrix is sparse) with precision $Q \in \mathbb{R}^{n \times n}$ and covariance $\Sigma = Q^{-1}$ is eigendecomposed as $Q = ...
1
vote
0answers
13 views

Variance and covariance of a measurement given a benchmark measurement

I have a set of 20 measurements $$(x_1, x_2, ..., x_{19}, x_{20}) $$ in which 18 of the measurements are a sort of benchmark, and two measurements are special, although obeying the same sources of ...
1
vote
0answers
23 views

Different sample covariance formulae (conventions) [duplicate]

Page 358 of Introduction to probability, second edition, by Blitzstein and Hwang, defines the sample covariance as $$r = \dfrac{1}{n} \sum_{i = 1}^n (x_i - \bar{x})(y_i - \bar{y}),$$ where $\bar{x} = \...
0
votes
1answer
41 views

Covariance matrix for p dimensional vector

I am working on making a conjecture about necessary and sufficient conditions for a singular covariance matrix of a p-dimensional random vector. To get to this conjecture I have to find the conditions ...
1
vote
1answer
48 views

How covariance matrix of the error term in linear regression can be NON-singular?

I don't understand linear regression. Assume the classic linear model: $$Y = X \beta + \epsilon,\\ \epsilon \sim \mathbb{N}(0, \sigma^2 I_n), $$ where $Y$ is a vector of length $n$, $X$ is a matrix of ...
0
votes
0answers
29 views

A question about the covariance matrix of a bivariate normal distribution

I am reading the paper Feature Screening via Distance Correlation Learning and I have some difficulties understanding Example 3 on page 15: Let $\mathbf{x}=(X_1,...,X_p)^T$ from normal distribution ...
1
vote
0answers
27 views

Maximum likelihood estimate of positive semidefinite matrix equivalent to MLE of its inverse

While reading on Gaussian Discriminant Analysis, I came across a derivation of the parameters (specifically, $\sum$) using Maximum Likelihood Estimation that claimed that the MLE of $\sum$ is ...
1
vote
1answer
28 views

requirements for simulating a covariance matrix

I was wondering, is any positive semidefinite matrix a valid covariance matrix? My problem is the following. I want to simulate a stochastic covariance matrix where the log-volatility (log of square ...
1
vote
0answers
26 views

Selection of covariance structures in SAS PROC GLIMMIX

Stroup and Claassen (2020) recently published an article titled Pseudo-Likelihood or Quadrature? What We Thought We Knew, What We Think We Know, and What We Are Still Trying to Figure Out in the ...
4
votes
1answer
48 views

covariance ,correlation, within subject and between subjects

My apologies if this has been asked earlier. I have been reading many textbooks and I am confused with the defination and meaning of covariance and correlation. I like to understand 1) The difference ...
1
vote
1answer
24 views

VAR equation-by-equation and variance-

It is written that the estimation of reduced form VAR is possible via equation-by-equation OLS. How does the reduced form errors variance-covariance matrix is estimated in this case?
0
votes
0answers
25 views

Multivariate Linear Regressions with heterogeneous covariance matrices

Let $\mathbf Y$ be $N\times K$ response matrix and $\mathbf X$ be $N\times (p+1)$ design matrix (including the intercept), consider the multivariate linear regression, $$ \mathbf Y = \mathbf{XB+E}\,, $...
1
vote
2answers
40 views

Generating correlated positive random numbers (given means, variances and degree of correlation)? [closed]

I have a vector of means and a covariance matrix and I'm trying to create a data set that would like to generate strictly positive numbers that would fit the parameters. I have seen quite a few ...
0
votes
0answers
5 views

What is the difference between a factorial analysis for mixed data (FAMD) and a PCA on a dataset where qualitative variable are dummy-encoded?

There are many variants of the principal component analysis (PCA) framework for discrete variables or a mixture of quantitative and discrete variables. Image from this book. However, I am not ...
3
votes
0answers
46 views

Why can't we accurately compute covariance matrix in high dimensions?

I am reading pg 651 of Elements of Statistical Learning,where is says: "The simplest form of regularization assumes that the features are independent within each class, that is, the within-class ...
3
votes
0answers
53 views

Examples of positive definite periodic covariance matrices

My aim is to find a few examples of positive definite covariance matrices $\pmb{R} = \{R(s,t)\}_{s,t=1}^n$ that satisfy $$R(s,t) = R(s+T, t+T),~~~1\leq s,t \leq n-T,$$ where $T$, $1\leq T\leq n-1$, is ...
0
votes
0answers
10 views

Combine Two Covariance Matrix [duplicate]

I have two sets of data and each data set has its own mean, standard deviation, and covariance matrix. Can I combine the covariance matrices from each data set to make the covariance matrix for the ...
1
vote
1answer
35 views

Question relating to joint PDFs

Here are my questions: Let $X$ ~ Unif$(0, 1)$, and $0<a<b<1$. Also, let \begin{cases} Y = 1 & \text{if $0<X<b$} \\ ...
1
vote
1answer
44 views

A linear process $x_{t}$ satisfies $\sum\limits_{j \in \mathbb Z}\lvert \gamma(j) \rvert < \infty$

A linear process $x_{t}$ is the weighted sum of white noise variates $(w_{t})_{t}$, i.e. $$x_{t}=\mu+\sum\limits_{k \in \mathbb Z}\psi_{k}w_{t-k}$$ such that $$ \sum\limits_{j \in \mathbb Z}\lvert \...
4
votes
1answer
45 views

Interpretation of covariance term in loss function

We have an $N\times 1$ vector containing some experimental values $y$, an $N\times 1$ vector $\hat{y}$ containing some predicted values, and an $N\times N$ covariance matrix $V_y$ for the experimental ...
1
vote
1answer
46 views

Rank of sample covariance matrix when $p = n$

Suppose we have a $p$-dimensional Gaussian distribution, and we take $n$ observations from that distribution. This answer states that when $p > n$, then the sample variance covariance matrix is ...
0
votes
0answers
22 views

generation of random covariane matrix

in the latest book by Marcos Lopez de Prado, he provides sample code for generating a random variance-covariance matrix. He starts by generating a rectangular dataset with fewer observations than ...
0
votes
0answers
51 views

Covariance matrices with exponential time decay

I am applying exponential time decay to financial time series to estimate their covariance matrices. The decay factor corresponds to a half-life equal to half of the estimation period. What I get is ...
0
votes
1answer
46 views

How can a covariance matrix for a normal distribution not be quadratic?

Currently Im reading this paper and in section 3.3., I came across the definition of a multi-dimensional standard normal distribution: \begin{align} q(\pmb{\epsilon}) = \mathcal{N}(\textbf{0}, \...
0
votes
2answers
39 views

Shouldn't all values of the covariance matrix under homoskedacity be zero?

The following is an excerpt from Greene's Econometric Analysis, 8th Edition. In homoskedacity, the covariance matrix has zero values for the expected errors of all pairs of observations $(i,e)$ ...
0
votes
1answer
26 views

How can one derive the original data from the correlation- or covariance matrix of that data?

How can one derive the original data from the correlation- or covariance matrix of that data? I know the way a new, reduced dataset can be calculated from the correlationmatrix and it’s eigenvectors ...
1
vote
1answer
37 views

Is the covariance matrix almost always postive definite?

I understand a covariance matrix is always positive semi-definite, but it seems that the covariance matrix would almost always be positive definite (although theoretically is only guaranteed to be ...
0
votes
0answers
22 views

Calculating a Weighted Standard Error of the Fit for Nonlinear Regression

I have a data set of $N$ points to which I have fit an equation of $n$ parameters $\theta_{1..n}$ such that $y_i \sim f(x_i; \theta_{1..n})$. These data $(x_{1..N},y_{1..N})$ have been provided with ...
3
votes
1answer
86 views

Finding a matrix $\mathbf{A}$ that projects a point to an eigenvector of $\mathbf{A}\mathbf{C}\mathbf{A}^T$

Suppose $\mathbf{b}=[b_1,b_2]'$ is $2\times 1$ and $\mathbf{C}$ is a full-rank symmetric $2\times 2$ matrix which both are real and given. Now, consider the problem of finding a $2\times 2$ matrix $\...

1
2 3 4 5
17