# Questions tagged [eigenvalues]

For questions involving calculation or interpretation of eigenvalues or eigenvectors.

349 questions
Filter by
Sorted by
Tagged with
649 views

### In PCA, why do we assume that the covariance matrix is always diagonalizable?

In Principal Component Analysis, what is the justification for the assumption that the covariance matrix is always a diagonalizable matrix? What happens when the covariance matrix is not ...
45 views

### How do I find the eigenvalues and eigenvectors of this matrix?

I'm having a little trouble finding the eigenvectors and eigenvalues of this covariance matrix I have: $$\mathbf{B}\mathbf{B}'a + \mathbf{R}.$$ $\mathbf{R}$ is diagonal, $\mathbf{B}$ is some low-...
2k views

### How is PCA applied to new data?

I understand the basic intuition behind PCA: reducing the dimensionality of data by finding the eigenvectors along which there is most variance in the data, and projecting the data along these ...
701 views

### Calculating CCA “scores” by hand in R

I'm trying to compute "by hand" the output of some popular Canonical Correlation Analysis functions in R, in order to be sure I understand the underlying math. I can produce the "canonical ...
297 views

### How to “direct” sample from a 2D Gaussian distribution?

I checked the source code of MASS::mvrnorm function. Key parts are listed here ...
389 views

### Distribution of eigenvalues of a random matrix

Given a random symmetric matrix A whose entries are Poisson distributed, can anything be said about the distribution of A's eigenvalues? Would be great if someone could link a paper citing such a ...
2k views

### If I generate a random symmetric matrix, what's the chance it is positive definite?

I got a strange question when I was experimenting some convex optimizations. The question is: Suppose I randomly (say standard normal distribution) generate a $N \times N$ symmetric matrix, (for ...
2k views

### *Why* are eigenvectors the principal components in Principal Component Analysis? [duplicate]

I am confused as to why eigenvalues are the principal components. What is the intuition behind finding the eigenvectors of the covariance matrix for PCA?
196 views

### Oja's rule gives unit eigenvector

Does Oja's rule for normalized Hebbian learning always result in a unit eigenvector which corresponds to the largest eigenvalue? Or are there any specific conditions or assumptions under which this is ...
323 views

1k views

### How to use eigenvectors to identify which variables are involved in collinearity? [duplicate]

The question involves a regression of $Y$ on $11$ predictor variables $X_1$ through $X_{11}$. The problem asks me to identify the variables involved in the collinearity using the eigenvectors that ...
3k views

### Negative eigenvalues in principle component analysis in the presence of missing data

For the purpose of dimension reduction I have performed an eigen analysis (using Jacobi-iteration) on a correlation matrix R of 163 variables (based on 1500 cases). The scree plot is attached. The ...
135 views

### Textbook for AIC, BIC, ANOVA, Eigenvalues and some other topics

I am going through Casella's statistical inference in one-semester standard statistic course and have mathematical background from Sheldon Axler's linear algebra done right and Louis Brand's advanced ...
31 views

### In Singular value decomposition is there any way to relate singular values to the columns of the original matrix?

Given a Matrix $A$ where the SVD would be $$A= U \Sigma V^t$$ Where $\Sigma$ is a diagonal matrix with its singular values Assuming that I only had the $\Sigma$ values and I don't have the $U$ ...
18k views

### Why does Andrew Ng prefer to use SVD and not EIG of covariance matrix to do PCA?

I am studying PCA from Andrew Ng's Coursera course and other materials. In the Stanford NLP course cs224n's first assignment, and in the lecture video from Andrew Ng, they do singular value ...
294 views

### Do I use eigen values from common factor solution OR from the original matrix? EFA in R, polychoric

I'm currently running an EFA in r, using the cor = "poly" setting with principal axis factoring, and am trying to determine the eigenvalues of the analysis. I am getting two different sets of values:...
115 views

### What does it mean for all principal components to capture a small amount of variance?

I'm doing PCA to analyze a dataset. The dataset size is 10^7 rows, and I have about 2,000 features. My analysis shows that no single principal component captures more than 1% of the dataset's variance....
43 views

### Is it possible to get original variances from the PCA eigenvalues of covariance matrix?

I am trying to obtain the variance of each attribute given the eigenvalues for each attribute. I know the eigenvalues come from the covariance matrix and that the diagonals of the covariance matrix ...
194 views

### Why shouldn't we scale data before entering into KPCA?

In Kernel Principal Component Analysis (KPCA), data comes in as a $n\times d$ matrix $X$ where $n$ is the number of observations and $d$ is the number of features. The process has been explained in ...
4k views

### What are eigenvalues and eigenvectors in factor analysis?

I understand the idea behind factor analysis, but everything I read on the topic seems to very vaguely cover the topic of eigenvalues and eigenvectors Whats the correct way to understand eigenvalues ...
2k views

### Quadratic form and Chi-squared distribution

It's about the demostration of the quadratic forms and chi-squared distribution. Let's split the problem: We have a $n$ vector with n standardized normal distribution called $z={[z_1,z_2...z_n]}$. ...
525 views

### Compute covariance matrix via rank-1 update to $M^\top M$

I have a large, sparse matrix $M\in\mathbb{R}^{n\times p}$. Centering $M$ to compute the covariance matrix $\Sigma$ would, in general, destroy the "zeros aren't stored" property of sparse matrices. ...
203 views

### Data compression using either Singular Values or Eigenvalues

In many applications, an SVD of a matrix is used to determine which features are important and which ones less important. For example, in image compression, the smallest singular values are often ...
40 views

### why we need to select the smallest singular values in locally linear embedding (LLE)?

I'm learning about locally linear embedding. The cost function for finding embedded data is given by $\Phi(X) = \Sigma_{ij}M_{ij}(X_i.X_j^T$) Why we need to select the $2^{nd}$ to $(P+1)^{th}$ ...
136 views

### Size of PCA decorrelation matrix?

I have a $P\times K$ matrix $\mathbf X$ with $K$ random vectors as columns (with the respective means subtracted from each entry). My goal is to decorrelate the columns of $\mathbf X$ via PCA to ...
105 views

### Deriving PCA solution using infinitesimals

The problem of PCA basically boils down to finding the extremum of $$\psi(q) = q^TRq \tag 1$$ subject to constraint $$q^Tq = 1 \tag 2.$$ How I would go about solving it, is by differentiating the ...
307 views

### How to obtain covariance matrix eigenvalues from singular values? [duplicate]

I would like to implement closed form of PPCA (Bishop, Tipping, 1999, Appendix A). In this paper they calculate $W$ in formula (15): $W=U_q(K_q-\sigma^2I)^{1/2}R$ ...
293 views

### Any intuition for why eigenvector centrality relates to walks of infinite length?

Some centrality measures can be interpreted in terms of walks. Degree centrality relates to a walk of length one: The more walks of length one reach a node, the higher this centrality measure. ...
749 views

### Help needed with intuition of eigenvalue spectrum of correlation matrix

I wish to get a better understanding of the meaning of the eigenvalues of a correlation matrix I am studying. I have a correlation matrix of noise levels for 10 cells in a wireless network over time....
165 views

### What happens if I do principal components of the principal components?

Just a hypothetical question: What happens if I do a second principal component analysis over the principal components derived from the first principal component analysis? What will be the ...
150 views

### Interpretation of a low-rank correlation matrix

I have a set of data from which I have generated a 12x12 Pearson correlation matrix. The rank of the correlation matrix is 8 but the number of eigenvalues that are of reasonable magnitude (not close ...
586 views

### Unexpected eigenvalues in parallel analysis for factor analysis in SPSS

Would greatly appreciate if someone could clarify which eigenvalues I am supposed to compare when using parallel analysis to determine factor retention. I am running Principal Axis Factoring in SPSS ...
55 views

### Why is this expression so common in statistics and machine learning?

I am talking about the expression $x^{T}Ax$. And, this is usually followed up mostly with some relevance to eigen values and eigen vectors. I have seen it in several problems, theoretical discussions ...
337 views

I have a table with the factor loadings obtained from a PCA for each variable and each component. Is it possible only with these data to obtain the eigenvalues for each Principal Component?
63 views

### My PCA projected points don't look right

I checked my PCA code many times, and I can't find anything wrong. I do PCA with eigen-decomposition, by using the eigen function in R. My data is 2-dimensional, I want to reduce it to one dimension. ...
909 views

### Lower bound on smallest eigenvalue of covariance matrices

Assume that a class of $p\times p$ covariance matrices is characterized by a parameter $\theta$, i.e, $$\mathbb{F} = \left\{\Sigma(\theta), \theta\in R\right\}$$ Also suppose we know the following ...
2k views

### Geometric interpretation of covariance matrix

In the following graph, the diagonal elements in the covariance matrix are the same, which I suppose means the spread of data in either direction should be the same, but why the data points are still ...
299 views

### What is the fastest way to calculate the leading singular value of a very large matrix (10mln x 100k)?

I only know of the following power iteration. But it needs to create a huge matrix A'*A when both of rows and columns are pretty large. And A is a dense matrix as well. Is there any alternative to ...
317 views

### Generating a positive semi-definite covariance matrix (using KL transform)

I have a set of input data X consisting of S&P 500 returns, that provides me with a covariance matrix C that is non positive semi-definite. The reason for the non-semi definite nature of the ...
955 views

### How can eigenfaces (PCA eigenvectors on face image data) be displayed as images?

I am trying to clarify some concepts for face recognition. According to my understanding, given a training set of images with each image measuring 225 x 255 pixels, we will have a matrix of training ...
606 views

### How does the matrix being centered change the eigenvalues/eigenvectors?

I have this question and I think I can just use maths for (ii) X'Xa=lambda a XX'Xa=XX'(Xa)=lambda(Xa) And so if lambda and a are eigenvalues/eigenvectors of X'X then lambda and Xa will be eigenvalues/...
142 views

### Jackknife and matrix diagonalization

Suppose I have $M$ matrices $n\times n$ of the form: \begin{equation} C^{k} = \begin{bmatrix} x^{k}_{11} & x^{k}_{12} & x^{k}_{13} & \dots & x^{k}_{1n} \\ x^{k}_{21} ...
92 views

### Different eigenvalues in R and SPSS

I'm trying to understand some canonical correlation outputs, and I found differences between eigenvalues results for R and SPSS. Some code: ...
2k views

### spectral clustering/theory, is there any meaning for the magnitude of values in eigenvectors?

Basically, spectral clustering is an application of spectral graph theory, which utilizes the eigenvalues and eigenvectors of a Laplacian matrix or adjacency matrix to disclose the connected ...
3k views