# Questions tagged [eigenvalues]

For questions involving calculation or interpretation of eigenvalues or eigenvectors.

349 questions
Filter by
Sorted by
Tagged with
684k views

### Making sense of principal component analysis, eigenvectors & eigenvalues

In today's pattern recognition class my professor talked about PCA, eigenvectors and eigenvalues. I understood the mathematics of it. If I'm asked to find eigenvalues etc. I'll do it correctly like ...
46k views

### Why does correlation matrix need to be positive semi-definite and what does it mean to be or not to be positive semi-definite?

I have been researching the meaning of positive semi-definite property of correlation or covariance matrices. I am looking for any information on Definition of positive semi-definiteness; Its ...
41k views

### How does centering make a difference in PCA (for SVD and eigen decomposition)?

What difference does centering (or de-meaning) your data make for PCA? I've heard that it makes the maths easier or that it prevents the first PC from being dominated by the variables' means, but I ...
18k views

### Why does Andrew Ng prefer to use SVD and not EIG of covariance matrix to do PCA?

I am studying PCA from Andrew Ng's Coursera course and other materials. In the Stanford NLP course cs224n's first assignment, and in the lecture video from Andrew Ng, they do singular value ...
2k views

### If I generate a random symmetric matrix, what's the chance it is positive definite?

I got a strange question when I was experimenting some convex optimizations. The question is: Suppose I randomly (say standard normal distribution) generate a $N \times N$ symmetric matrix, (for ...
11k views

### Why are there only $n-1$ principal components for $n$ data if the number of dimensions is $\ge n$?

In PCA, when the number of dimensions $d$ is greater than (or even equal to) the number of samples $N$, why is it that you will have at most $N-1$ non-zero eigenvectors? In other words, the rank of ...
1k views

### What is the meaning of the eigenvectors of a mutual information matrix?

When looking at the eigenvectors of the covariance matrix, we get the directions of maximum variance (the first eigenvector is the direction in which the data varies the most, etc.); this is called ...
369 views

### Explain how eigen helps inverting a matrix

My question relates to a computation technique exploited in geoR:::.negloglik.GRF or geoR:::solve.geoR. In a linear mixed model ...
10k views

### Is every correlation matrix positive definite?

I'm talking here about matrices of Pearson correlations. I've often heard it said that all correlation matrices must be positive semidefinite. My understanding is that positive definite matrices must ...
15k views

### Why does PCA maximize total variance of the projection?

Christopher Bishop writes in his book Pattern Recognition and Machine Learning a proof, that each consecutive principal component maximizes the variance of the projection to one dimension, after the ...
3k views

### Why are eigen and svd decompositions of a covariance matrix based on sparse data yielding different results?

I am trying to decompose a covariance matrix based on a sparse / gappy data set. I'm noticing that the sum of lambda (explained variance), as calculated with svd, ...
4k views

### What is principal subspace in probabilistic PCA?

if $X$ is observed data matrix and $Y$ is latent variable then $$X=WY+\mu+\epsilon$$ Where $\mu$ is the mean of observed data, and $\epsilon$ is the Gaussian error/noise in data, and $W$ is called ...
214 views

2k views

### Correlation between principal components

I have two matrices a, b of dimensions (100x500), (100x15000) and I am trying to find associations between sets of variables in both matrices. When I perform principal component analysis on matrix a,...
7k views

### Is centering a valid solution for multicollinearity?

Let's assume that $y = a + a_1x_1 + a_2x_2 + a_3x_3 + e$ where $x_1$ and $x_2$ both are indexes both range from $0-10$ where $0$ is the minimum and $10$ is the maximum. I found by applying VIF, CI and ...
3k views

### PCA: Eigenvectors of opposite sign and not being able to compute eigenvectors with solve in R

I'm learning PCA in R language. I met two problems right now that I don't understand. I am performing a PCA analysis in R on a 318×17 dataset using some custom code. I take eigen function in R to ...
708 views

### Interpretation of eigenvectors of Hessian inverse

I'm reading a paper in which they use the eigenvectors of the inverse Hessian of a continuous probability distribution to characterize dimensions along which the distribution is most and least ...
645 views

### In PCA, why do we assume that the covariance matrix is always diagonalizable?

In Principal Component Analysis, what is the justification for the assumption that the covariance matrix is always a diagonalizable matrix? What happens when the covariance matrix is not ...
6k views

### Confused about Cholesky and eigen decomposition

I'm looking to generate correlated random variables. I have a symmetric, positive definite matrix. So I know that you can use the Cholesky decomposition, however I keep being told that this only works ...
1k views

PCA is known to be quite sensitive to outlier noise (and this is why several Robust PCA techniques exists.) However, I am looking for a concrete example of sensitivity of PCA to adversarial noise that ...
890 views

### Why does the direction with highest eigenvalue have the largest semi-axis?

So, in PCA, we decompose the covariance matrix by its eigenvalues and eigenvectors. I understand that an ellipsoid is fully characterized by the eigenvalues and eigenvectors of a positive definite ...
8k views

### How to interpret variation explained by principal coordinates?

I have recently seen a couple of Principal Coordinates Analysis (PCoA) projection plots which show "percentage variation explained" by the respective principal coordinates. Given that the analysis is ...
560 views

### Why do we use the eigenvectors of the Laplacian and not the Affinity matrix in spectral clustering?

In spectral clustering, it's standard practice to solve the eigenvector problem $$L v = \lambda v$$ where $L$ is the graph Laplacian, $v$ is the eigenvector related to eigenvalue $\lambda$. My ...
2k views

### Appropriate negative eigenvalue correction for PCoA of genetic distances

I am trying to find the best way to represent genetic distances in a plane so that they may use them as response variables in canonical redundancy analysis (using ...
13k views

### How does eigenvalues measure variance along the principal components in PCA? [duplicate]

I understand that eigenvalues measure variance along the principal components. Questions How are eigenvalues and variance same for PCA? What is the intuition behind this being the same? What is ...
2k views

### Perform PCA. Extract PCs. Can one then tell what the most important _original_ features were, from the PCs? [duplicate]

Suppose that you have 1000 features, and a data set made up of say, 50,000 points. Suppose then that we perform PCA, and we extract the top 5 PCs, since they explain 99.99 percent of the variance, and ...
3k views

### Eigenvalues of correlation matrices exhibit exponential decay

I have a data-set of $P$ samples of size $N$, and noticed that the eigenvalues of the correlation matrices $A^TA$, when presented in descending order, can in many cases be described as an exponential ...
4k views

### Condition number of covariance matrix

I am interested in generating a covariance matrix of dimension say 100. I managed to get a correlation matrix with finite condition number. To construct a covariance matrix I need to have standard ...
384 views

### Interpretation of the cluster criterion $\operatorname{tr}(S_W^{-1}S_B)$

There is a cluster criterion defined as: $$\mathcal{C} = \operatorname{tr}(S_W^{-1}S_B) = \sum_{i=1}^d \lambda_i,$$ where $\operatorname{tr}$ is the trace, $S_W$ is the pooled within-group scatter ...
Consider a random vector $\mathbf{X}=(X)_{i=1}^n$. Then the covariance matrix $$C=\mathbb{E}[(\mathbf{X}-\mu(\mathbf{X}))(\mathbf{X}-\mu(\mathbf{X}))^\top]$$ is by definition positive-semidefinite. (...