Questions tagged [covariance-matrix]

A $k\times k$ matrix of covariances between all pairs of $k$ random variables. It is also called variance-covariance matrix or simply the covariance matrix.

Filter by
Sorted by
Tagged with
0
votes
0answers
18 views

How to compute the covariance matrix of a multivariate bernoulli distribution?

Considering this toy example: Let $x$ be a random variable $x \sim \mathcal{N}(\mu_x, \Sigma_x)$ Where $\mu_x \in \mathbb{R}^2$ is the mean vector and $\Sigma_x \in \mathbb{R}^{2 \times 2}$ is the ...
0
votes
0answers
10 views

How is covariance matrix affected if each data points is multipled by some constant?

I have a 2D multivariate Normal distribution with some mean and a covariance matrix. While fitting the function I had normalized the data.so the mean and covariance I have are for the normalized data. ...
6
votes
1answer
55 views

Multivariate Normal Distribution: Divide each random variable by its standard deviation

If $X$~$Normal(\mu,\Sigma)$, and I divide each random variable in $X$ (the marginals) by its standard deviation, what will happen to the covariance matrix $\Sigma$?
0
votes
0answers
10 views

Why eigen vector of a covariance matrix is the largest principle components? [duplicate]

I am self studying principle component analysis using this tutorial, I got most of the reasoning behind PCA but I don't get the intuitive reason why eigen vectors of a covariance matrix is also its ...
0
votes
1answer
21 views

Covariance Linear Shrinkage Estimator : Implied Data

I have been using linear shrinkage to better estimate the covariance matrix when I do not have enough data. Let $\bf X$ be a $T\times n$ matrix representing the data (previously centered) and $${\bf S}...
0
votes
0answers
13 views

Using eigenvalues of the covariance matrix to reduce noise in my data

I have an idea to help reduce the noise in my signal but am stuck with a significant problem. I have a very noisy data set $y_n[t]; n\in\{0, N_{\text{samples}}-1\}; t\in\{0, T-1\}$ I am fitting this ...
0
votes
0answers
15 views

Use pairwise correlation matrix for clustering

Suppose I have random variables $X_1,\dots,X_d$. Suppose I collected some data, say $n$, and want to calculate the sample correlation matrix of the $d$ random variables. However due to missing data, ...
1
vote
1answer
12 views

Correlation matrix from VAR(1) model

I have implemented a simple VAR(1) model with gaussian noise and no bias to generate two dimensional data. When computinng the empirical covariance matrice (for lag 0) of of this signal, it is always ...
0
votes
0answers
5 views

Noise removal from a matrix when the noise covariance structure is known

I have a matrix made by the addition of the signal and the noise. I know the covariance structure of the noise but don't have the actual noise matrix. Is there a way to remove the noise/obtain an ...
0
votes
0answers
11 views

Finding a Subset of Observations that Minimizes the Corr bw Vectors

I have 5 vectors each with 100,000+ observations. Because the vectors are more similar than different, the correlations are high (0.8+). Using knowledge of the dataset, I have identified a subset of ...
1
vote
1answer
45 views

Projection of a multivariate normal into a univariate normal; calculating variance from covariance matrix

On the PDF of a multivariate gaussian, we have $e^{-\frac{1}{2}(x-\mu)^t\Sigma^{-1}(x-\mu)}$ This leads me to assume that if I want restrict my distribution to a given direction, I will have $(x-\mu)^...
1
vote
0answers
12 views

Can I decompose a signal in interval T between symmetric and anti-symmetric signals?

I have computed many samples of a signal $y_n[t]; t\in \{-T/2, T/2\}$ for $n=1,\ldots,N_{\text{samples}}$. I want to model this with an even function $f(t;\vec{\theta}) = f(-t; \vec{\theta})$ where $\...
0
votes
1answer
16 views

Eigenvectors of covariance matrix and inertia tensor

The moment of inertia tensor from physics looks very similar to the covariance matrix, used for PCA. How are their eigenvectors and eigenvalues related?
1
vote
0answers
5 views

Can't calculate np.cov() correctly [migrated]

This question might be silly, but i couldn't find an explanation to that. I am coding the multivariate probability density function from scratch (for study purposes), and one of the things that i need ...
1
vote
1answer
26 views

Covariance matrix of element-wise quotient of two sets of measurements with known covariance matrices

I have two sets of measurements, $x_i$ and $y_i$, both with the same number of elements, $N$. For each of these sets, I have known $N\times N$ covariance matrices, $\Sigma_x$ and $\Sigma_y$, ...
0
votes
0answers
14 views

Calculate Conditional Expectation using dataset and covariance matrices

So I'm having trouble understanding what I'm doing wrong here. For context, I have some velocity components in my dataset for turbulence (simplified). I have flattened them out so my 3 velocity ...
0
votes
0answers
8 views

factor analysis on covariance matrix and rotation

I have a matrix that I analyze with a PCA without standardization (the variables are on the same metric and size is important) Some suggest performing a FA to confirm the dimensionality of the PCA ...
2
votes
1answer
92 views

Ordinal logistic regression - How to handle NaNs

I'm trying to run an ordinal logistic regression model with 5 IV's and a DV with 4 levels. I'm using the function polr from the MASS package in R. My data consists of 46 observations and all the IV's ...
2
votes
2answers
25 views

Why is there covariance instead of simply variance in multidimensional normal distributions?

Maybe it is just because I'm only experimenting with 2 dimensional normal distributions, but multi-dimensional normal distributions for me seem like just multiple one dimensional normal distributions. ...
5
votes
0answers
60 views

What is the precise relation between the eigenvalues of a covariance *function* and the eigenvalues of a covariance *matrix*?

Assume we have a temporal Gaussian Process $\mathcal{GP}(t;\ m,k)$ (GP) with mean $m$ and covariance function (aka. kernel) $k$ on some compact time interval $[0,T]$. Then, the eigenvalues $\lambda$ ...
0
votes
0answers
7 views

Writing $E[f(\mathbf{X})]\approx f(\mu)+\frac{1}{2}E[(\mathbf{X}-\mu)^T Hf(\mu)(\mathbf{X}-\mu)]$ in terms of $\text{Cov}[\mathbf{X}]$?

Let $\mathbf{X}$ be a random vector with corresponding mean vector $\mu$. By a Taylor series expansion we get $$ E[f(\mathbf{X})] \approx f(\mu) + \frac{1}{2} E[(\mathbf{X}-\mu)^T H f(\mu)(\mathbf{X}-\...
1
vote
1answer
30 views

Is covariance matrix positive definite in complex space?

I have a covariance matrix $\Sigma$. I know $\Sigma$ is positive definite if we are working in real space because for any non-zero $x \in \mathbb{R}^n$, $x^T\Sigma x \gt 0$ (Edit: I am assuming non-...
1
vote
1answer
104 views

Validity of approximating a covariance matrix by making use of a probability limit?

I want to know can we approximate the covariance matrix of a random vector by making use of a probability limit. Define the linear regression model in matrix form as $$ \mathbf{Y} = \mathbf{X} \beta + ...
0
votes
0answers
15 views

Generating Low-Rank (correlated) Uniform Random Samples

I'm looking to generate $n$ uniform random samples that occupy an ambient dimension $D$ but live on a $d$-dimensional linear subspace. I know how to do this form normal samples. Specifically, given ...
-2
votes
1answer
33 views

How does the covariance matrix of the predictors in multiple regression relate to the matrix $(\mathbf{X}^T \mathbf{X})^{-1}$? [duplicate]

Note in advance, due to my question previously being marked as a duplicate, that the question I ask here is concerned with the relationship between $(\mathbf{X}^T \mathbf{X})^{-1}$ and $\text{Cov}[\...
0
votes
0answers
24 views

What is the relationship between $\text{Cov}[\mathbf{X},\mathbf{X}]$ and $(\mathbf{X}^T \mathbf{X})^{-1}$ in multiple regression? [duplicate]

The matrix formulation of multiple regression for $n$ observations is $$ \mathbf{Y} = \mathbf{X}^T \beta + \varepsilon, $$ where the error $\varepsilon$ has finite variance $\sigma^2$. Let $\mathbf{b}$...
1
vote
2answers
53 views

Determinant =1 constraint in PCA reconstruction Error

Let $q\leq p$. As in Tibshirani's statistical learning book, one can describe the PCA problem as optimizing the $q$-dimensional reconstruction error, given on a dataset $\{x_n\}_{n=1}^N$ in $\mathbb{...
1
vote
1answer
27 views

Question regarding Optimal Designs of experiments

I'm a bit unclear on the concept of optimal design of a data matrix $X$. I propose a small example to work through: Suppose $\epsilon_i \sim N(0, \sigma^2)$ are i.i.d., and I have some experiment ...
0
votes
1answer
40 views

Why are covariance matrices projected by both right and left multiply?

I've been doing a lot of Kalman filtering work recently. I've derived all the equations starting from a basic linear inverse problem, so strictly speaking I know where everything comes from. I also ...
0
votes
0answers
19 views

Conditional Density between three gaussians

Suppose we have two independent standard normal variables $X$ and $Y$ and lets construct $Z$ such as $Z = \rho X + \sqrt{1-\rho^2 Y}$. Therefore, $Z$ is dependent on $X$ and $Y$. I want to find the ...
1
vote
1answer
23 views

Covariance of an uncertain vector going through an uncertain transformation

Let's have two vectors $\mathbf \omega \in \mathbb R^3$, $\mathbf \theta \in \mathbb R^3$ and their associated covariance, $\Sigma_{\omega} \in \mathbb R^{3\times3}$ and $\Sigma_{\theta} \in \mathbb R^...
0
votes
0answers
19 views

Intuition behind sampling from a covariance matrix in gaussian processes

I am trying to grasp an intuition here. Say we have a prior covariance matrix defined by a RBF function, if we do a contour plot based on that, we have these characteristic diagonal matrix with unit ...
0
votes
1answer
58 views

Interpretation of non-independent errors in linear regression

I have problems to interpret dependencies in the covariance matrix $Cov(\epsilon)$ of errors in the linear Regression model $Y = X\beta + \epsilon$​, i.e. when there are non-zero entries apart from ...
0
votes
0answers
11 views

What is the maximum entropy distribution given the median (instead of the mean) and median variant deviation?

Following this question, I wonder if we add further condition that the median variant deviation is given, say $\delta$, is it possible to determine the distribution? For 1-d, the $\delta$ can be ...
1
vote
0answers
16 views

Notation question

In what follows $\Phi$ is the autoregressive polynomial obtained in estimating a (noncausal) VAR model, $Y_t - \Phi Y_{t-1}$ is the error term from the VAR process and $Y_{t-h} - \Phi Y_{t-h-1}$ is a ...
0
votes
0answers
32 views

Fisher's formalism : How to find a complementary matrix to respect the Maximum Likelihood Estimator (MLE)?

I make following a previous post : Bad attempt to do cross-correlations between 2 matrices Indeed, I say "Bad attempt" since the beginning of this study, I did a major error. By wanting to ...
2
votes
1answer
45 views

How to get the eigenvalue expansion of the covariance matrix?

Working through Bishops’s Pattern Recognition and Machine Learning and have the following question regarding the Eigenvalue expansion of a covariance matrix: “ Assume we have a symmetric real-valued ...
0
votes
0answers
9 views

How to quantify a prior or information into a Fisher matrix?

I have 2 Fisher matrices of the same dimension, each one representing parameters that I have to constain by combining these 2 matrices in Fisher's formlism. Firstly, I did a simple sum between both to ...
0
votes
0answers
25 views

Deriving a Wald Statistic from Scratch for a MLE Estimator in R?

Seems like a substantial undertaking Define the wald statistic as W=$(R\hat{\theta}-r)'[R\hat{V}(\hat{\theta})R']^{-1}(R\hat{\theta}-r)$ Where r is a vector of restrictions, R is imposes the ...
1
vote
1answer
34 views

Wald Statistic Simplification - 2 Restrictions?

(Specify the wald statistic as: W=$(R\hat{\theta}-r)'[R\hat{V}(\hat{\theta})R']^{-1}(R\hat{\theta}-r)$ Where r is a vector of restrictions, R imposes the restrictions on $\hat{\beta}$, $\hat{\theta}$ ...
0
votes
0answers
27 views

Eigenvectors and eigenvalues of covariance matrix or its inverse in drawing ellipsoid

I'm trying to draw an ellipsoid of the $3 \times 3$ covariance matrix. Usually, I see the sentence an ellipsoid corresponding to the eigenvectors and eigenvalues of covariance matrix. But from the ...
1
vote
1answer
72 views

Can off-diagonal elements in a covariance matrix ever be greater than the diagonal elements?

Can off-diagonal elements in a variance-covariance matrix ever be greater than the diagonal elements?
1
vote
0answers
56 views

Estimating sum of off-diagonals in covariance matrix with known diagonal

Assume we have $n$ samples from $\mathbf{X}\ \sim\ \mathcal{N}_k(\boldsymbol 0,\, \boldsymbol\Sigma)$ where elements of $\boldsymbol\Sigma_{k \times k} = [ \Sigma]_{ij}$ are the covariance matrix. Is ...
1
vote
1answer
44 views

Covariance of sum of multivariate normals

If I have a variable $a = v*b + c$ where $v$ is a vector of length n, $b$ is a normally distributed random variable with mean 0 and variance 1, and $c$ is normally distributed with mean 0 and ...
2
votes
0answers
26 views

Best approximation of the Mahalanobis distance by standardized Euclidean distance

I am looking for the best way to approximate the Mahalanobis distance by the standardized Euclidean distance, which would reduce the number of the required multiplications. The easiest way is the ...
0
votes
2answers
37 views

Variance matrix of a sum of random vectors

Let X and Y be random vectors of same dimension. Let var(X) be the covariance matrix of X; var(Y) the correspondent matrix; cov(X,Y) the matrix where the coordinate (i,j) is cov(x_i, y_j) I saw the ...
1
vote
1answer
37 views

Covariance scalar from matrix

I am trying to implement SSIM Structural similarity in Python. One of the necessary elements is the covariance between the two matrices. Using numpy.cov() We get a ...
1
vote
0answers
128 views

Cross-correlation synthesis for 2 Fisher matrices

I am trying to do a synthesis between 2 Fisher matrices, i.e without only considering a simple sum : I am looking for a final Fisher matrix that performs the XC (cross-correlations). One done that, I ...
0
votes
0answers
10 views

ordinal logistic regression random effects and specify correlation structure

I am dealing with an unbalanced repeated measure dataset. My outcome variable, y, is a 3 level ordinal variable : Hot, Moderate, Cold. My independent variable is a repeated time varying measure. Due ...
2
votes
0answers
33 views

Covariance Matrix and Gaussian Process

In a paper i'm reading they use gaussian processes but i'm a little bit confused about their use of the covariance matrix. The setup is as follows: the inputs are $x_i \in \mathbb{R}^Q$ and there are $...

1
2 3 4 5
18