Questions tagged [eigenvalues]
For questions involving calculation or interpretation of eigenvalues or eigenvectors.
415
questions
4
votes
1
answer
580
views
General PCA optimization problem
I was looking at the PCA optimization problem, which is finding a matrix $U \in \mathbb{R}^{d\times n}$, $n \le d$, that solves the problem
$$\max{tr(U^TCU)},\ \ \ s.t. U^TU = I, $$
where $C$ is the ...
- 71
2
votes
0
answers
33
views
What significance can be attached to the eigenmodes of a transition matrix?
I am studying a continuous dynamical system and am categorising preferred areas of the system's phase space using a 3 state hidden Markov model.
As the resulting transition matrix is row stochastic, ...
- 41
2
votes
1
answer
2k
views
Covariance matrix of image data is not positive definite matrix
I've really hit the wall here and need help with direction :).
I am trying to use mvnpdf as part of basic EM algorithm but the covariance matrix of data seems to be not positive definite. There are ...
- 21
1
vote
1
answer
187
views
Rank of N x D vs D x N matrices
If $X$ is a random $N \times D$ matrix where $N > D$, then why is the rank of X - mean(X, 1) $D$ while the rank of ...
- 2,963
2
votes
3
answers
4k
views
If the eigenvalues of a covariance matrix have very low variance, what does it mean?
If we have a covariance matrix $A$
$AP=PD$
where $D$ is a diagonal matrix which contains all the eigenvalues
then if the variance of the eigenvalues is very small, what does this tell us about $A$?
...
- 1,489
1
vote
0
answers
77
views
What should I check after I do PCA?
I have X in which each row is a sample and each column is a predictor.
I de-mean X first and then construct the co-variance matrix
$A=X^{T}X$
after that I do PCA
$AP=PD$
while each column in P is ...
- 1,489
1
vote
1
answer
214
views
Why is the result a set of factors instead of just only one?
I am trying to get into SEM and factor analysis. I understand a factor is a latent construct, say e.g. $intelligence$, user-defined by the (weighted) average of a set of indicators $x_1, x_2\dots x_n$....
- 661
1
vote
0
answers
135
views
Interpreting cointegration test
I am very new to interpreting cointegration tests and eigenvalues, so any help would be appreciated.
3
votes
1
answer
267
views
For an eigenvector v, is it always true that v^tv =1?
I am reading this link
PCA
which is a very insightful tutorial
however, in this tutorial, the author mentioned a constraint on PCA:
$C^{T}C=1$
when we look at eigenvalue/eigenvector definition
$...
- 1,489
0
votes
0
answers
363
views
Why does pnorm() change the correlation in my multivariate normal sample?
I came across 2 problems that puzzle me while simulating variables for a Monte Carlo simulation, using the rnormalcopula command from the rCopula example. The first one is the one from the title, the ...
- 21
2
votes
1
answer
262
views
Adding/removing variables to PCA
If I have a PCA that I ran on some set of variables, how (if at all) will it relate to the PCA results if I add or remove one variable?
Will the PCA components change in some well-defined way, or is ...
- 831
5
votes
1
answer
7k
views
Maximum number of principal components in PCA. Is sklearn wrong?
Recently I've been interested in applying PCA to a dataset I have and I wanted to develop a deep understanding of what I would actually be doing when I implement it.
Today I encountered two ...
1
vote
0
answers
16
views
I need to know how to explain this two table specially the principal components matrix
Principal component analysis matrix
!
2
votes
1
answer
3k
views
What does removing the first principal component from data signify?
The first principal component is the axis along which the data varies the most. So, what happens if I remove that while retaining all the remaining components? I am guessing that the data kind of ...
user198990
1
vote
1
answer
459
views
Why are my eigenvalues so huge?
I'm running a PCA using prcomp in R and I get a table like this from the summary function:
...
- 111
1
vote
0
answers
314
views
rate of exponential decay of autocovariance function
Let $\epsilon_n$ be a Markov switching GARCH process as defined by Haas et al.
I have a covariance function $\mathbb{E}(\epsilon_{n-\tau}^2 \epsilon_n^2)$ which depends only on the powers of 2 ...
- 265
0
votes
1
answer
753
views
Co-variance matrix, Eigen vector and Eigen values [duplicate]
What does eigen vector with largest eigen value mean and how it has effect on covariance matrix?
- 76
9
votes
1
answer
2k
views
In PCA, why do we assume that the covariance matrix is always diagonalizable?
In Principal Component Analysis, what is the justification for the assumption that the covariance matrix is always a diagonalizable matrix?
What happens when the covariance matrix is not ...
- 690
1
vote
0
answers
47
views
How do I find the eigenvalues and eigenvectors of this matrix?
I'm having a little trouble finding the eigenvectors and eigenvalues of this covariance matrix I have:
$$
\mathbf{B}\mathbf{B}'a + \mathbf{R}.
$$
$\mathbf{R}$ is diagonal, $\mathbf{B}$ is some low-...
- 19.5k
7
votes
1
answer
5k
views
How is PCA applied to new data?
I understand the basic intuition behind PCA: reducing the dimensionality of data by finding the eigenvectors along which there is most variance in the data, and projecting the data along these ...
- 1,234
1
vote
1
answer
1k
views
Calculating CCA "scores" by hand in R
I'm trying to compute "by hand" the output of some popular Canonical Correlation Analysis functions in R, in order to be sure I understand the underlying math.
I can produce the "canonical ...
- 131
0
votes
0
answers
476
views
How to "direct" sample from a 2D Gaussian distribution?
I checked the source code of MASS::mvrnorm function. Key parts are listed here
...
- 34.9k
3
votes
0
answers
452
views
Distribution of eigenvalues of a random matrix
Given a random symmetric matrix A whose entries are Poisson distributed, can anything be said about the distribution of A's eigenvalues? Would be great if someone could link a paper citing such a ...
- 157
33
votes
1
answer
3k
views
If I generate a random symmetric matrix, what's the chance it is positive definite?
I got a strange question when I was experimenting some convex optimizations. The question is:
Suppose I randomly (say standard normal distribution) generate a $N \times N$ symmetric matrix, (for ...
- 34.9k
1
vote
0
answers
3k
views
*Why* are eigenvectors the principal components in Principal Component Analysis? [duplicate]
I am confused as to why eigenvalues are the principal components. What is the intuition behind finding the eigenvectors of the covariance matrix for PCA?
- 216
1
vote
1
answer
287
views
Oja's rule gives unit eigenvector
Does Oja's rule for normalized Hebbian learning always result in a unit eigenvector which corresponds to the largest eigenvalue? Or are there any specific conditions or assumptions under which this is ...
- 111
6
votes
1
answer
713
views
Second directional derivate and Hessian matrix
I am reading the following from the book Deep Learning, and I have the following questions.
I don't quite understand second directional derivatives. The first directional derivative of a function $f:\...
- 1,189
6
votes
2
answers
2k
views
Why do the leading eigenvectors of $A$ maximize $\text{Tr}(D^TAD)$?
Given a matrix $X\in\mathbb{R}^{m\times n}$, I am trying to maximize $\text{Tr}(D^TX^TXD)$ over $D\in\mathbb{R}^{n\times l}$ ($n<l$) subject to $D^TD=I_l$, where $\text{Tr}$ denotes the trace, and $...
- 1,189
1
vote
1
answer
2k
views
How to use eigenvectors to identify which variables are involved in collinearity? [duplicate]
The question involves a regression of $Y$ on $11$ predictor variables $X_1$ through $X_{11}$.
The problem asks me to identify the variables involved in the collinearity using the eigenvectors that ...
- 245
5
votes
2
answers
7k
views
Negative eigenvalues in principle component analysis in the presence of missing data
For the purpose of dimension reduction I have performed an eigen analysis (using Jacobi-iteration) on a correlation matrix R of 163 variables (based on 1500 cases). The scree plot is attached.
The ...
2
votes
2
answers
264
views
Textbook for AIC, BIC, ANOVA, Eigenvalues and some other topics
I am going through Casella's statistical inference in one-semester standard statistic course and have mathematical background from Sheldon Axler's linear algebra done right and Louis Brand's advanced ...
- 205
0
votes
0
answers
45
views
In Singular value decomposition is there any way to relate singular values to the columns of the original matrix?
Given a Matrix $A$ where the SVD would be
$$A= U \Sigma V^t$$
Where $\Sigma$ is a diagonal matrix with its singular values
Assuming that I only had the $\Sigma$ values and I don't have the $U$ ...
- 101
47
votes
7
answers
29k
views
Why does Andrew Ng prefer to use SVD and not EIG of covariance matrix to do PCA?
I am studying PCA from Andrew Ng's Coursera course and other materials. In the Stanford NLP course cs224n's first assignment, and in the lecture video from Andrew Ng, they do singular value ...
- 613
0
votes
0
answers
621
views
Do I use eigen values from common factor solution OR from the original matrix? EFA in R, polychoric
I'm currently running an EFA in r, using the cor = "poly" setting with principal axis factoring, and am trying to determine the eigenvalues of the analysis.
I am getting two different sets of values:...
1
vote
0
answers
267
views
What does it mean for all principal components to capture a small amount of variance?
I'm doing PCA to analyze a dataset. The dataset size is 10^7 rows, and I have about 2,000 features. My analysis shows that no single principal component captures more than 1% of the dataset's variance....
- 11
1
vote
0
answers
44
views
Is it possible to get original variances from the PCA eigenvalues of covariance matrix?
I am trying to obtain the variance of each attribute given the eigenvalues for each attribute. I know the eigenvalues come from the covariance matrix and that the diagonals of the covariance matrix ...
- 111
1
vote
0
answers
462
views
Why shouldn't we scale data before entering into KPCA?
In Kernel Principal Component Analysis (KPCA), data comes in as a $n\times d$ matrix $X$ where $n$ is the number of observations and $d$ is the number of features. The process has been explained in ...
- 119
0
votes
0
answers
5k
views
What are eigenvalues and eigenvectors in factor analysis?
I understand the idea behind factor analysis, but everything I read on the topic seems to very vaguely cover the topic of eigenvalues and eigenvectors
Whats the correct way to understand eigenvalues ...
- 2,271
12
votes
2
answers
6k
views
Quadratic form and Chi-squared distribution
It's about the demostration of the quadratic forms and chi-squared distribution.
Let's split the problem:
We have a $n$ vector with n standardized normal distribution called $z={[z_1,z_2...z_n]}$.
...
- 395
3
votes
1
answer
926
views
Compute covariance matrix via rank-1 update to $M^\top M$
I have a large, sparse matrix $M\in\mathbb{R}^{n\times p}$. Centering $M$ to compute the covariance matrix $\Sigma$ would, in general, destroy the "zeros aren't stored" property of sparse matrices. ...
- 86.3k
3
votes
0
answers
315
views
Data compression using either Singular Values or Eigenvalues
In many applications, an SVD of a matrix is used to determine which features are important and which ones less important. For example, in image compression, the smallest singular values are often ...
- 544
0
votes
0
answers
61
views
why we need to select the smallest singular values in locally linear embedding (LLE)?
I'm learning about locally linear embedding. The cost function for finding embedded data is given by
$\Phi(X) = \Sigma_{ij}M_{ij}(X_i.X_j^T$)
Why we need to select the $2^{nd}$ to $(P+1)^{th}$ ...
- 203
0
votes
0
answers
161
views
Size of PCA decorrelation matrix?
I have a $P\times K$ matrix $\mathbf X$ with $K$ random vectors as columns (with the respective means subtracted from each entry). My goal is to decorrelate the columns of $\mathbf X$ via PCA to ...
- 41
3
votes
1
answer
120
views
Deriving PCA solution using infinitesimals
The problem of PCA basically boils down to finding the extremum of $$\psi(q) = q^TRq \tag 1$$ subject to constraint $$q^Tq = 1 \tag 2.$$
How I would go about solving it, is by differentiating the ...
- 2,231
0
votes
0
answers
345
views
How to obtain covariance matrix eigenvalues from singular values? [duplicate]
I would like to implement closed form of PPCA (Bishop, Tipping, 1999, Appendix A). In this paper they calculate $W$ in formula (15):
$W=U_q(K_q-\sigma^2I)^{1/2}R$
...
- 402
3
votes
1
answer
597
views
Any intuition for why eigenvector centrality relates to walks of infinite length?
Some centrality measures can be interpreted in terms of walks.
Degree centrality relates to a walk of length one: The more walks of length one reach a node, the higher this centrality measure.
...
1
vote
1
answer
1k
views
Help needed with intuition of eigenvalue spectrum of correlation matrix
I wish to get a better understanding of the meaning of the eigenvalues of a correlation matrix I am studying.
I have a correlation matrix of noise levels for 10 cells in a wireless network over time....
- 629
6
votes
1
answer
222
views
What happens if I do principal components of the principal components?
Just a hypothetical question: What happens if I do a second principal component analysis over the principal components derived from the first principal component analysis?
What will be the ...
- 539
1
vote
0
answers
159
views
Interpretation of a low-rank correlation matrix
I have a set of data from which I have generated a 12x12 Pearson correlation matrix. The rank of the correlation matrix is 8 but the number of eigenvalues that are of reasonable magnitude (not close ...
- 11
1
vote
0
answers
781
views
Unexpected eigenvalues in parallel analysis for factor analysis in SPSS
Would greatly appreciate if someone could clarify which eigenvalues I am supposed to compare when using parallel analysis to determine factor retention.
I am running Principal Axis Factoring in SPSS ...
- 11