Linked Questions
10 questions linked to/from Is a sample covariance matrix always symmetric and positive definite?
4
votes
1
answer
2k
views
Prove that sample covariance matrix is positive definite [duplicate]
Consider the $p \times p$ sample covariance matrix:
$$\mathbf{S} = \frac{1}{n-1} \cdot \mathbf{Y}_\mathbf{c}^\text{T} \mathbf{Y}_\mathbf{c}
\quad \quad \quad
\mathbf{Y}_\mathbf{c} = \mathbf{C} \mathbf{...
- 211
56
votes
3
answers
102k
views
Maximum Likelihood Estimators - Multivariate Gaussian
Context
The Multivariate Gaussian appears frequently in Machine Learning and the following results are used in many ML books and courses without the derivations.
Given data in form of a matrix $\...
- 9,986
4
votes
2
answers
513
views
Why is $\mathbf\Phi^{\top}\mathbf\Phi$ a positive definite matrix?
I had this question when reading section 3.5.3 on page 170 of "Pattern Recognition and Machine Learning" written by Christopher M. Bishop:
Here $\mathbf\Phi$ represents the design matrix ...
- 333
5
votes
1
answer
544
views
Must a matrix of sample pairwise covariances be PSD?
Consider a random vector $\mathbf{X}=(X)_{i=1}^n$. Then the covariance matrix $$C=\mathbb{E}[(\mathbf{X}-\mu(\mathbf{X}))(\mathbf{X}-\mu(\mathbf{X}))^\top]$$ is by definition positive-semidefinite. (...
- 153
3
votes
0
answers
2k
views
Simulating data from a given multivariate covariance matrix - workarounds for a non positive definite covariance matrix?
As part of a simulation study, I would like to create multivariate data that follow a specific covariance matrix. In this study I would like to be able to show that my algorithm is able to find highly ...
- 86
1
vote
0
answers
939
views
Why absolute value of eigenvalues are used in PCA or LDA?
In PCA and LDA techniques, eigenvectors with the $k$ largest eigenvalues give principal components. However, when selecting these eigenvalues, are they to be sorted by the absolute value (regardless ...
- 11
3
votes
2
answers
155
views
Granger Causality and positive semidefiniteness
Suppose that we have vector $y_t = (z_t, x_t)'$ for $t = 1, 2, \dots, T$. Let $\Omega_t$ be the information set available at time $t$ and $z_t(h|\Omega_t)$ be optimal $h$-step predictor. $\Sigma_z(h|\...
- 1,179
2
votes
0
answers
392
views
requirements for simulating a covariance matrix
I was wondering, is any positive semidefinite matrix a valid covariance matrix?
My problem is the following. I want to simulate a stochastic covariance matrix where the log-volatility (log of square ...
- 211
1
vote
0
answers
242
views
How to relate a covariance matrix to the generated length and angle of a elliptical distribution?
I am trying to generate a plot of points randomly sampled from a 2D elliptical distribution. I want to control the length and orientation of the ellipse this random sample creates. It seems like ...
- 400
1
vote
0
answers
75
views
Problems with SEM: Non-positive definite matrix [duplicate]
Link to picture: https://www.dropbox.com/s/bxzxy0y37npqc4o/SEM%20problems.PNG
Need a bit of help determining what I should do to fix this problem: The SEM wont run because it believes it's non-...
- 11