Questions tagged [eigenvalues]

For questions involving calculation or interpretation of eigenvalues or eigenvectors.

Filter by
Sorted by
Tagged with
3
votes
0answers
370 views

General PCA optimization problem

I was looking at the PCA optimization problem, which is finding a matrix $U \in \mathbb{R}^{d\times n}$, $n \le d$, that solves the problem $$\max{tr(U^TCU)},\ \ \ s.t. U^TU = I, $$ where $C$ is the ...
1
vote
0answers
28 views

What significance can be attached to the eigenmodes of a transition matrix?

I am studying a continuous dynamical system and am categorising preferred areas of the system's phase space using a 3 state hidden Markov model. As the resulting transition matrix is row stochastic, ...
2
votes
1answer
1k views

Covariance matrix of image data is not positive definite matrix

I've really hit the wall here and need help with direction :). I am trying to use mvnpdf as part of basic EM algorithm but the covariance matrix of data seems to be not positive definite. There are ...
1
vote
1answer
47 views

Rank of N x D vs D x N matrices

If $X$ is a random $N \times D$ matrix where $N > D$, then why is the rank of X - mean(X, 1) $D$ while the rank of ...
1
vote
2answers
1k views

If the eigenvalues of a covariance matrix have very low variance, what does it mean?

If we have a covariance matrix $A$ $AP=PD$ where $D$ is a diagonal matrix which contains all the eigenvalues then if the variance of the eigenvalues is very small, what does this tell us about $A$? ...
1
vote
0answers
68 views

What should I check after I do PCA?

I have X in which each row is a sample and each column is a predictor. I de-mean X first and then construct the co-variance matrix $A=X^{T}X$ after that I do PCA $AP=PD$ while each column in P is ...
1
vote
1answer
73 views

Why is the result a set of factors instead of just only one?

I am trying to get into SEM and factor analysis. I understand a factor is a latent construct, say e.g. $intelligence$, user-defined by the (weighted) average of a set of indicators $x_1, x_2\dots x_n$....
1
vote
0answers
119 views

Interpreting cointegration test

I am very new to interpreting cointegration tests and eigenvalues, so any help would be appreciated.
2
votes
1answer
145 views

For an eigenvector v, is it always true that v^tv =1?

I am reading this link PCA which is a very insightful tutorial however, in this tutorial, the author mentioned a constraint on PCA: $C^{T}C=1$ when we look at eigenvalue/eigenvector definition $...
0
votes
0answers
271 views

Why does pnorm() change the correlation in my multivariate normal sample?

I came across 2 problems that puzzle me while simulating variables for a Monte Carlo simulation, using the rnormalcopula command from the rCopula example. The first one is the one from the title, the ...
0
votes
1answer
79 views

Adding/removing variables to PCA

If I have a PCA that I ran on some set of variables, how (if at all) will it relate to the PCA results if I add or remove one variable? Will the PCA components change in some well-defined way, or is ...
5
votes
1answer
4k views

Maximum number of principal components in PCA. Is sklearn wrong?

Recently I've been interested in applying PCA to a dataset I have and I wanted to develop a deep understanding of what I would actually be doing when I implement it. Today I encountered two ...
1
vote
0answers
15 views
2
votes
1answer
1k views

What does removing the first principal component from data signify?

The first principal component is the axis along which the data varies the most. So, what happens if I remove that while retaining all the remaining components? I am guessing that the data kind of ...
1
vote
1answer
94 views

Why are my eigenvalues so huge?

I'm running a PCA using prcomp in R and I get a table like this from the summary function: ...
1
vote
0answers
211 views

rate of exponential decay of autocovariance function

Let $\epsilon_n$ be a Markov switching GARCH process as defined by Haas et al. I have a covariance function $\mathbb{E}(\epsilon_{n-\tau}^2 \epsilon_n^2)$ which depends only on the powers of 2 ...
0
votes
1answer
509 views

Co-variance matrix, Eigen vector and Eigen values [duplicate]

What does eigen vector with largest eigen value mean and how it has effect on covariance matrix?
8
votes
1answer
728 views

In PCA, why do we assume that the covariance matrix is always diagonalizable?

In Principal Component Analysis, what is the justification for the assumption that the covariance matrix is always a diagonalizable matrix? What happens when the covariance matrix is not ...
1
vote
0answers
45 views

How do I find the eigenvalues and eigenvectors of this matrix?

I'm having a little trouble finding the eigenvectors and eigenvalues of this covariance matrix I have: $$ \mathbf{B}\mathbf{B}'a + \mathbf{R}. $$ $\mathbf{R}$ is diagonal, $\mathbf{B}$ is some low-...
7
votes
1answer
2k views

How is PCA applied to new data?

I understand the basic intuition behind PCA: reducing the dimensionality of data by finding the eigenvectors along which there is most variance in the data, and projecting the data along these ...
1
vote
1answer
755 views

Calculating CCA “scores” by hand in R

I'm trying to compute "by hand" the output of some popular Canonical Correlation Analysis functions in R, in order to be sure I understand the underlying math. I can produce the "canonical ...
0
votes
0answers
306 views

How to “direct” sample from a 2D Gaussian distribution?

I checked the source code of MASS::mvrnorm function. Key parts are listed here ...
3
votes
0answers
396 views

Distribution of eigenvalues of a random matrix

Given a random symmetric matrix A whose entries are Poisson distributed, can anything be said about the distribution of A's eigenvalues? Would be great if someone could link a paper citing such a ...
33
votes
1answer
2k views

If I generate a random symmetric matrix, what's the chance it is positive definite?

I got a strange question when I was experimenting some convex optimizations. The question is: Suppose I randomly (say standard normal distribution) generate a $N \times N$ symmetric matrix, (for ...
1
vote
0answers
2k views

*Why* are eigenvectors the principal components in Principal Component Analysis? [duplicate]

I am confused as to why eigenvalues are the principal components. What is the intuition behind finding the eigenvectors of the covariance matrix for PCA?
0
votes
1answer
204 views

Oja's rule gives unit eigenvector

Does Oja's rule for normalized Hebbian learning always result in a unit eigenvector which corresponds to the largest eigenvalue? Or are there any specific conditions or assumptions under which this is ...
5
votes
1answer
373 views

Second directional derivate and Hessian matrix

I am reading the following from the book Deep Learning, and I have the following questions. I don't quite understand second directional derivatives. The first directional derivative of a function $f:\...
5
votes
2answers
858 views

Why do the leading eigenvectors of $A$ maximize $\text{Tr}(D^TAD)$?

Given a matrix $X\in\mathbb{R}^{m\times n}$, I am trying to maximize $\text{Tr}(D^TX^TXD)$ over $D\in\mathbb{R}^{n\times l}$ ($n<l$) subject to $D^TD=I_l$, where $\text{Tr}$ denotes the trace, and $...
1
vote
1answer
1k views

How to use eigenvectors to identify which variables are involved in collinearity? [duplicate]

The question involves a regression of $Y$ on $11$ predictor variables $X_1$ through $X_{11}$. The problem asks me to identify the variables involved in the collinearity using the eigenvectors that ...
5
votes
2answers
4k views

Negative eigenvalues in principle component analysis in the presence of missing data

For the purpose of dimension reduction I have performed an eigen analysis (using Jacobi-iteration) on a correlation matrix R of 163 variables (based on 1500 cases). The scree plot is attached. The ...
2
votes
2answers
153 views

Textbook for AIC, BIC, ANOVA, Eigenvalues and some other topics

I am going through Casella's statistical inference in one-semester standard statistic course and have mathematical background from Sheldon Axler's linear algebra done right and Louis Brand's advanced ...
0
votes
0answers
35 views

In Singular value decomposition is there any way to relate singular values to the columns of the original matrix?

Given a Matrix $A$ where the SVD would be $$A= U \Sigma V^t$$ Where $\Sigma$ is a diagonal matrix with its singular values Assuming that I only had the $\Sigma$ values and I don't have the $U$ ...
39
votes
7answers
19k views

Why does Andrew Ng prefer to use SVD and not EIG of covariance matrix to do PCA?

I am studying PCA from Andrew Ng's Coursera course and other materials. In the Stanford NLP course cs224n's first assignment, and in the lecture video from Andrew Ng, they do singular value ...
0
votes
0answers
323 views

Do I use eigen values from common factor solution OR from the original matrix? EFA in R, polychoric

I'm currently running an EFA in r, using the cor = "poly" setting with principal axis factoring, and am trying to determine the eigenvalues of the analysis. I am getting two different sets of values:...
1
vote
0answers
126 views

What does it mean for all principal components to capture a small amount of variance?

I'm doing PCA to analyze a dataset. The dataset size is 10^7 rows, and I have about 2,000 features. My analysis shows that no single principal component captures more than 1% of the dataset's variance....
1
vote
0answers
43 views

Is it possible to get original variances from the PCA eigenvalues of covariance matrix?

I am trying to obtain the variance of each attribute given the eigenvalues for each attribute. I know the eigenvalues come from the covariance matrix and that the diagonals of the covariance matrix ...
0
votes
0answers
208 views

Why shouldn't we scale data before entering into KPCA?

In Kernel Principal Component Analysis (KPCA), data comes in as a $n\times d$ matrix $X$ where $n$ is the number of observations and $d$ is the number of features. The process has been explained in ...
0
votes
0answers
4k views

What are eigenvalues and eigenvectors in factor analysis?

I understand the idea behind factor analysis, but everything I read on the topic seems to very vaguely cover the topic of eigenvalues and eigenvectors Whats the correct way to understand eigenvalues ...
7
votes
2answers
2k views

Quadratic form and Chi-squared distribution

It's about the demostration of the quadratic forms and chi-squared distribution. Let's split the problem: We have a $n$ vector with n standardized normal distribution called $z={[z_1,z_2...z_n]}$. ...
3
votes
1answer
579 views

Compute covariance matrix via rank-1 update to $M^\top M$

I have a large, sparse matrix $M\in\mathbb{R}^{n\times p}$. Centering $M$ to compute the covariance matrix $\Sigma$ would, in general, destroy the "zeros aren't stored" property of sparse matrices. ...
2
votes
0answers
212 views

Data compression using either Singular Values or Eigenvalues

In many applications, an SVD of a matrix is used to determine which features are important and which ones less important. For example, in image compression, the smallest singular values are often ...
0
votes
0answers
41 views

why we need to select the smallest singular values in locally linear embedding (LLE)?

I'm learning about locally linear embedding. The cost function for finding embedded data is given by $\Phi(X) = \Sigma_{ij}M_{ij}(X_i.X_j^T$) Why we need to select the $2^{nd}$ to $(P+1)^{th}$ ...
0
votes
0answers
140 views

Size of PCA decorrelation matrix?

I have a $P\times K$ matrix $\mathbf X$ with $K$ random vectors as columns (with the respective means subtracted from each entry). My goal is to decorrelate the columns of $\mathbf X$ via PCA to ...
3
votes
1answer
108 views

Deriving PCA solution using infinitesimals

The problem of PCA basically boils down to finding the extremum of $$\psi(q) = q^TRq \tag 1$$ subject to constraint $$q^Tq = 1 \tag 2.$$ How I would go about solving it, is by differentiating the ...
0
votes
0answers
315 views

How to obtain covariance matrix eigenvalues from singular values? [duplicate]

I would like to implement closed form of PPCA (Bishop, Tipping, 1999, Appendix A). In this paper they calculate $W$ in formula (15): $W=U_q(K_q-\sigma^2I)^{1/2}R$ ...
2
votes
1answer
324 views

Any intuition for why eigenvector centrality relates to walks of infinite length?

Some centrality measures can be interpreted in terms of walks. Degree centrality relates to a walk of length one: The more walks of length one reach a node, the higher this centrality measure. ...
0
votes
1answer
799 views

Help needed with intuition of eigenvalue spectrum of correlation matrix

I wish to get a better understanding of the meaning of the eigenvalues of a correlation matrix I am studying. I have a correlation matrix of noise levels for 10 cells in a wireless network over time....
5
votes
1answer
167 views

What happens if I do principal components of the principal components?

Just a hypothetical question: What happens if I do a second principal component analysis over the principal components derived from the first principal component analysis? What will be the ...
1
vote
0answers
153 views

Interpretation of a low-rank correlation matrix

I have a set of data from which I have generated a 12x12 Pearson correlation matrix. The rank of the correlation matrix is 8 but the number of eigenvalues that are of reasonable magnitude (not close ...
1
vote
0answers
615 views

Unexpected eigenvalues in parallel analysis for factor analysis in SPSS

Would greatly appreciate if someone could clarify which eigenvalues I am supposed to compare when using parallel analysis to determine factor retention. I am running Principal Axis Factoring in SPSS ...

1 2 3
4
5
8