Questions tagged [eigenvalues]

For questions involving calculation or interpretation of eigenvalues or eigenvectors.

Filter by
Sorted by
Tagged with
6
votes
1answer
649 views

In PCA, why do we assume that the covariance matrix is always diagonalizable?

In Principal Component Analysis, what is the justification for the assumption that the covariance matrix is always a diagonalizable matrix? What happens when the covariance matrix is not ...
1
vote
0answers
45 views

How do I find the eigenvalues and eigenvectors of this matrix?

I'm having a little trouble finding the eigenvectors and eigenvalues of this covariance matrix I have: $$ \mathbf{B}\mathbf{B}'a + \mathbf{R}. $$ $\mathbf{R}$ is diagonal, $\mathbf{B}$ is some low-...
7
votes
1answer
2k views

How is PCA applied to new data?

I understand the basic intuition behind PCA: reducing the dimensionality of data by finding the eigenvectors along which there is most variance in the data, and projecting the data along these ...
1
vote
1answer
701 views

Calculating CCA “scores” by hand in R

I'm trying to compute "by hand" the output of some popular Canonical Correlation Analysis functions in R, in order to be sure I understand the underlying math. I can produce the "canonical ...
0
votes
0answers
297 views

How to “direct” sample from a 2D Gaussian distribution?

I checked the source code of MASS::mvrnorm function. Key parts are listed here ...
3
votes
0answers
389 views

Distribution of eigenvalues of a random matrix

Given a random symmetric matrix A whose entries are Poisson distributed, can anything be said about the distribution of A's eigenvalues? Would be great if someone could link a paper citing such a ...
33
votes
1answer
2k views

If I generate a random symmetric matrix, what's the chance it is positive definite?

I got a strange question when I was experimenting some convex optimizations. The question is: Suppose I randomly (say standard normal distribution) generate a $N \times N$ symmetric matrix, (for ...
1
vote
0answers
2k views

*Why* are eigenvectors the principal components in Principal Component Analysis? [duplicate]

I am confused as to why eigenvalues are the principal components. What is the intuition behind finding the eigenvectors of the covariance matrix for PCA?
0
votes
1answer
196 views

Oja's rule gives unit eigenvector

Does Oja's rule for normalized Hebbian learning always result in a unit eigenvector which corresponds to the largest eigenvalue? Or are there any specific conditions or assumptions under which this is ...
5
votes
1answer
323 views

Second directional derivate and Hessian matrix

I am reading the following from the book Deep Learning, and I have the following questions. I don't quite understand second directional derivatives. The first directional derivative of a function $f:\...
5
votes
2answers
745 views

Why do the leading eigenvectors of $A$ maximize $\text{Tr}(D^TAD)$?

Given a matrix $X\in\mathbb{R}^{m\times n}$, I am trying to maximize $\text{Tr}(D^TX^TXD)$ over $D\in\mathbb{R}^{n\times l}$ ($n<l$) subject to $D^TD=I_l$, where $\text{Tr}$ denotes the trace, and $...
1
vote
1answer
1k views

How to use eigenvectors to identify which variables are involved in collinearity? [duplicate]

The question involves a regression of $Y$ on $11$ predictor variables $X_1$ through $X_{11}$. The problem asks me to identify the variables involved in the collinearity using the eigenvectors that ...
5
votes
2answers
3k views

Negative eigenvalues in principle component analysis in the presence of missing data

For the purpose of dimension reduction I have performed an eigen analysis (using Jacobi-iteration) on a correlation matrix R of 163 variables (based on 1500 cases). The scree plot is attached. The ...
2
votes
2answers
135 views

Textbook for AIC, BIC, ANOVA, Eigenvalues and some other topics

I am going through Casella's statistical inference in one-semester standard statistic course and have mathematical background from Sheldon Axler's linear algebra done right and Louis Brand's advanced ...
0
votes
0answers
31 views

In Singular value decomposition is there any way to relate singular values to the columns of the original matrix?

Given a Matrix $A$ where the SVD would be $$A= U \Sigma V^t$$ Where $\Sigma$ is a diagonal matrix with its singular values Assuming that I only had the $\Sigma$ values and I don't have the $U$ ...
38
votes
7answers
18k views

Why does Andrew Ng prefer to use SVD and not EIG of covariance matrix to do PCA?

I am studying PCA from Andrew Ng's Coursera course and other materials. In the Stanford NLP course cs224n's first assignment, and in the lecture video from Andrew Ng, they do singular value ...
0
votes
0answers
294 views

Do I use eigen values from common factor solution OR from the original matrix? EFA in R, polychoric

I'm currently running an EFA in r, using the cor = "poly" setting with principal axis factoring, and am trying to determine the eigenvalues of the analysis. I am getting two different sets of values:...
1
vote
0answers
115 views

What does it mean for all principal components to capture a small amount of variance?

I'm doing PCA to analyze a dataset. The dataset size is 10^7 rows, and I have about 2,000 features. My analysis shows that no single principal component captures more than 1% of the dataset's variance....
1
vote
0answers
43 views

Is it possible to get original variances from the PCA eigenvalues of covariance matrix?

I am trying to obtain the variance of each attribute given the eigenvalues for each attribute. I know the eigenvalues come from the covariance matrix and that the diagonals of the covariance matrix ...
0
votes
0answers
194 views

Why shouldn't we scale data before entering into KPCA?

In Kernel Principal Component Analysis (KPCA), data comes in as a $n\times d$ matrix $X$ where $n$ is the number of observations and $d$ is the number of features. The process has been explained in ...
0
votes
0answers
4k views

What are eigenvalues and eigenvectors in factor analysis?

I understand the idea behind factor analysis, but everything I read on the topic seems to very vaguely cover the topic of eigenvalues and eigenvectors Whats the correct way to understand eigenvalues ...
7
votes
2answers
2k views

Quadratic form and Chi-squared distribution

It's about the demostration of the quadratic forms and chi-squared distribution. Let's split the problem: We have a $n$ vector with n standardized normal distribution called $z={[z_1,z_2...z_n]}$. ...
3
votes
1answer
525 views

Compute covariance matrix via rank-1 update to $M^\top M$

I have a large, sparse matrix $M\in\mathbb{R}^{n\times p}$. Centering $M$ to compute the covariance matrix $\Sigma$ would, in general, destroy the "zeros aren't stored" property of sparse matrices. ...
2
votes
0answers
203 views

Data compression using either Singular Values or Eigenvalues

In many applications, an SVD of a matrix is used to determine which features are important and which ones less important. For example, in image compression, the smallest singular values are often ...
0
votes
0answers
40 views

why we need to select the smallest singular values in locally linear embedding (LLE)?

I'm learning about locally linear embedding. The cost function for finding embedded data is given by $\Phi(X) = \Sigma_{ij}M_{ij}(X_i.X_j^T$) Why we need to select the $2^{nd}$ to $(P+1)^{th}$ ...
0
votes
0answers
136 views

Size of PCA decorrelation matrix?

I have a $P\times K$ matrix $\mathbf X$ with $K$ random vectors as columns (with the respective means subtracted from each entry). My goal is to decorrelate the columns of $\mathbf X$ via PCA to ...
3
votes
1answer
105 views

Deriving PCA solution using infinitesimals

The problem of PCA basically boils down to finding the extremum of $$\psi(q) = q^TRq \tag 1$$ subject to constraint $$q^Tq = 1 \tag 2.$$ How I would go about solving it, is by differentiating the ...
0
votes
0answers
307 views

How to obtain covariance matrix eigenvalues from singular values? [duplicate]

I would like to implement closed form of PPCA (Bishop, Tipping, 1999, Appendix A). In this paper they calculate $W$ in formula (15): $W=U_q(K_q-\sigma^2I)^{1/2}R$ ...
2
votes
1answer
293 views

Any intuition for why eigenvector centrality relates to walks of infinite length?

Some centrality measures can be interpreted in terms of walks. Degree centrality relates to a walk of length one: The more walks of length one reach a node, the higher this centrality measure. ...
0
votes
1answer
749 views

Help needed with intuition of eigenvalue spectrum of correlation matrix

I wish to get a better understanding of the meaning of the eigenvalues of a correlation matrix I am studying. I have a correlation matrix of noise levels for 10 cells in a wireless network over time....
5
votes
1answer
165 views

What happens if I do principal components of the principal components?

Just a hypothetical question: What happens if I do a second principal component analysis over the principal components derived from the first principal component analysis? What will be the ...
1
vote
0answers
150 views

Interpretation of a low-rank correlation matrix

I have a set of data from which I have generated a 12x12 Pearson correlation matrix. The rank of the correlation matrix is 8 but the number of eigenvalues that are of reasonable magnitude (not close ...
1
vote
0answers
586 views

Unexpected eigenvalues in parallel analysis for factor analysis in SPSS

Would greatly appreciate if someone could clarify which eigenvalues I am supposed to compare when using parallel analysis to determine factor retention. I am running Principal Axis Factoring in SPSS ...
1
vote
0answers
55 views

Why is this expression so common in statistics and machine learning?

I am talking about the expression $x^{T}Ax$. And, this is usually followed up mostly with some relevance to eigen values and eigen vectors. I have seen it in several problems, theoretical discussions ...
0
votes
0answers
337 views

How to compute eigenvalues from loadings in a PCA?

I have a table with the factor loadings obtained from a PCA for each variable and each component. Is it possible only with these data to obtain the eigenvalues for each Principal Component?
0
votes
1answer
63 views

My PCA projected points don't look right

I checked my PCA code many times, and I can't find anything wrong. I do PCA with eigen-decomposition, by using the eigen function in R. My data is 2-dimensional, I want to reduce it to one dimension. ...
2
votes
1answer
909 views

Lower bound on smallest eigenvalue of covariance matrices

Assume that a class of $p\times p$ covariance matrices is characterized by a parameter $\theta$, i.e, $$\mathbb{F} = \left\{\Sigma(\theta), \theta\in R\right\}$$ Also suppose we know the following ...
2
votes
1answer
2k views

Geometric interpretation of covariance matrix

In the following graph, the diagonal elements in the covariance matrix are the same, which I suppose means the spread of data in either direction should be the same, but why the data points are still ...
1
vote
1answer
299 views

What is the fastest way to calculate the leading singular value of a very large matrix (10mln x 100k)?

I only know of the following power iteration. But it needs to create a huge matrix A'*A when both of rows and columns are pretty large. And A is a dense matrix as well. Is there any alternative to ...
0
votes
1answer
317 views

Generating a positive semi-definite covariance matrix (using KL transform)

I have a set of input data X consisting of S&P 500 returns, that provides me with a covariance matrix C that is non positive semi-definite. The reason for the non-semi definite nature of the ...
5
votes
1answer
955 views

How can eigenfaces (PCA eigenvectors on face image data) be displayed as images?

I am trying to clarify some concepts for face recognition. According to my understanding, given a training set of images with each image measuring 225 x 255 pixels, we will have a matrix of training ...
0
votes
1answer
606 views

How does the matrix being centered change the eigenvalues/eigenvectors?

I have this question and I think I can just use maths for (ii) X'Xa=lambda a XX'Xa=XX'(Xa)=lambda(Xa) And so if lambda and a are eigenvalues/eigenvectors of X'X then lambda and Xa will be eigenvalues/...
1
vote
1answer
142 views

Jackknife and matrix diagonalization

Suppose I have $M$ matrices $n\times n$ of the form: \begin{equation} C^{k} = \begin{bmatrix} x^{k}_{11} & x^{k}_{12} & x^{k}_{13} & \dots & x^{k}_{1n} \\ x^{k}_{21} ...
0
votes
0answers
92 views

Different eigenvalues in R and SPSS

I'm trying to understand some canonical correlation outputs, and I found differences between eigenvalues results for R and SPSS. Some code: ...
4
votes
1answer
2k views

spectral clustering/theory, is there any meaning for the magnitude of values in eigenvectors?

Basically, spectral clustering is an application of spectral graph theory, which utilizes the eigenvalues and eigenvectors of a Laplacian matrix or adjacency matrix to disclose the connected ...
4
votes
1answer
3k views

Covariance matrix decomposition and coregionalization

The original question (that can be seen at the bottom of this post) was replaced by this first edit (below) EDIT I I give more details about my problem. First of all let suppose to have K vectors $\...
4
votes
0answers
122 views

Determine largest magnitude given a covariance matrix

Let's say one has a univariate Gaussian distribution with mean $\mu = 0$ and standard deviation $\sigma$. It is easy to see that the distance from $\mu$ to +1$\sigma$ is...well...$\sigma$. Let's ...
2
votes
0answers
65 views

How do I apply AIC to test out how many eigenvalues are different from 1 in a diagonalised covariance matrix?

Given that we have $x_1$,...,$x_n$ with dimensions $p * 1$ each, and $X_i$ ~ N(0,Σ), we form the diagonalised covariance matrix Σ such that the first K eigenvalues are unknown and the ...
3
votes
0answers
2k views

Principal Component Analysis: how to interpret the total contribution of variables on several dimensions

When we calculate the total contribution of a variable for a single dimension, the sum of all single contributions is equal to 100%, which makes perfect sense. The http://www.sthda.com suggests to ...
8
votes
2answers
3k views

Why are PCA eigenvectors orthogonal and what is the relation to the PCA scores being uncorrelated?

I'm reading up on PCA, and I'm understanding most of what's going on in terms of the derivation apart from the assumption that eigenvectors need to be orthogonal and how it relates to the projections (...

1 2 3
4
5
7