Questions tagged [eigenvalues]
For questions involving calculation or interpretation of eigenvalues or eigenvectors.
0
votes
0answers
36 views
Determining the Direction of Eigenvectors in PCA [duplicate]
I'm using R to get the principal components for several datasets.
An example result, using prcomp yields:
...
0
votes
0answers
27 views
Idea behind change of basis and how it relates to projecting your points onto principal components
I would like to clarify if my understanding is correct. In the traditional X-Y coordinate system, our choice of basis vectors are $\vec{i} = (1, 0)$ and $\vec{j} = (0, 1)$ and when you I have a point $...
1
vote
0answers
44 views
Change in eigenvalues due to perturbation to a correlation matrix
Let $A$ be a $m \times n$ matrix defined as
$ A = \Big[\frac{a_1}{\|a_1\|} \cdots \frac{a_n}{\|a_n\|}\Big]$ and $a_k \in \mathbb{R}^{m\times 1}$ where $k \in [1,\dots,n]$.
Now, we define a ...
3
votes
0answers
54 views
Maximum likelihood: Why is the number of non-zero eigenvalues equal to $x^T \hat{\Sigma}^{-1} x$
I've been reading this code (based on this R package) and I found that the number of non-zero eigenvalues of the estimated covariance is roughly equal to $x_i^T \hat{\Sigma}^{-1} x_i$. I want to know ...
0
votes
0answers
16 views
The miracle of the Lanczos/conjugate gradient algorithm
Lanczos/Arnoldi/Rietz/CG-like algorithm share the same core strategy... In each, a little miracle appears, most of the Gram-Schmidt inner products are zeroes ! In others words, new direction need only ...
1
vote
0answers
48 views
Limiting results for non-unique eigenvalues and eigenvectors for a sample covariance matrix
I am working on the limiting behavior for the eigenvalue and the corresponding eigenvectors, especially the minimum eigenvalues. For instance, let $S_X=\frac{1} {T} \sum_t X_t X_t ^\prime$ be a $p \...
0
votes
1answer
89 views
Are eigenfaces same as eigenvectors?
I'm trying to understand the difference between eigenvectors and eigenfaces, are they different names for same concepts?
I ask this because I got confused when I am trying to compute eigenvectors for ...
0
votes
1answer
19 views
PCA- creating a model with values obtained
Hoping somebody can help me. I cannot find an example that 'finishes' a problem. I run a proc princomp in SAS. I have hundreds of variables but used four for the purpose of an example. I ...
2
votes
1answer
30 views
Compute the $k$ largest eigenvector in spectral clustering
In Spectral Clustering, we need to compute the top $k$ largest eigenvector of normalized $L$.
$$L = D^{-\frac{1}{2}}SD^{-\frac{1}{2}}$$
In Andrew NG's paper, L is not positive definite (unless ...
0
votes
0answers
13 views
Approximation error on standard and sparse PCA
I am trying to understand the approximation error of pca explained in this tutorial.
$\sum_{n=1}^{N}||x_n-\tilde{x}_n||=N\sum_{i=M+1}^{D}\lambda_i$ where $||x_n-\tilde{x}_n||=\sum_{i=M+1}^{D}\left(\...
0
votes
0answers
16 views
Eigenvalue decomposition/SVD and the filtering perspective
I have been studying the SVD algorithm recently and I can understand how it might be used for compression but I am trying to figure out if there is a perspective of SVD where it can be seen as a low ...
-1
votes
0answers
23 views
Does it make sense to analyse an autocorrelation matrix with Random Matrix Theory?
I wonder whether you can gain valuable information about a time series by analysing its autocorrelation matrix using RMT.
I know that RMT can help to extract information about the collective ...
0
votes
0answers
13 views
Signs in SPSS's PCA with rotations with the FACTOR algorithm
I am trying to reproduce the results of the PCA with rotations from SPSS in python.
But there is some information I didn't find in their documentation.
I am trying to do the PCA like in the FACTOR ...
0
votes
0answers
42 views
Intuition behind finding number of clusters in spectral clustering according to Zelnik-Manor and Perona
I have a quick question on the intuition behind the estimation strategy for the number of clusters presented in the paper "Self-Tuning Spectral Clustering" by Lihi Zelnik-Manor and Pietro Perona. See ...
0
votes
1answer
146 views
PCA in psych package with more columns than rows
Why is it impossible to do a PCA in R using principal from psych package without warnings with a matrix, which has more columns ...
1
vote
0answers
26 views
PCA for three-dimensional linear fit on time-resolved trajectory
I study the behavior of organisms that are able of self-locomotion and that show directed motion toward one another. This directed motion occurs through the detection of chemical trails released by ...
3
votes
3answers
560 views
PCA basic: Must eigenvalues converge to zero at high dimension?
Recently, I obtained several PCA plots, and because I am unable to produce eigenvalues for higher dimensions, I tried to extrapolate them based on the available data. The reason why I want to do this ...
2
votes
1answer
47 views
PCA Basic: Do eigenvalues remain constant with increase in dimensions?
Say I have a set of genomic data to be analysed for population structure. I ran two different analyses using two different maximum principal components. For the first analysis, I modeled the data ...
0
votes
0answers
35 views
Weighting a Covariance Matrix via Eigendecomposition
Given a covariance matrix $\mathbf{\Sigma} \in \mathbb{R}^{N \times N}$ between $N$ variables of interest, generated by some kernel function $\mathbf{\kappa}(\mathbf{x}, \mathbf{y})$, and some vector $...
2
votes
3answers
142 views
PCA to choose variables based on its loadings on PC1 [duplicate]
I have a dataset of cave dimensions (and other variables related to their features).
The problem is that 3 of these variables are: Length, Area, and Volume.
These 3 are highly correlated as they ...
0
votes
1answer
108 views
Negative eigenvalues when computing MultiDimensional Scaling given nonnegative distance matrix
I am using the Smile MDS
https://github.com/haifengl/smile/blob/master/core/src/main/java/smile/mds/MDS.java and occasionally running into:
...
0
votes
0answers
60 views
Interpreting low Cronbach's alpha value in CATPCA
I ran categorical principal component analysis (CATPCA) on the data that I collected through a questionnaire. The purpose of the questionnaire was to understand pedestrians intentions and attitudes ...
3
votes
0answers
55 views
Why are the discriminant axes in linear discriminant analysis (LDA) not orthogonal?
This may be a quite silly question and please correct me if I'm wrong.
The discriminants (discriminant axes) are essentially eigenvectors of $\mathrm{Cov}_\mathrm{within}^{-1} \mathrm{Cov}_\mathrm{...
0
votes
0answers
33 views
Is max. Eigenvalue of k-sparse PCA always $\leq$ max. Eigenvalue of normal PCA on same dataset?
Is max. Eigenvalue of k-sparse PCA always less than or equal to the max. Eigenvalue of normal PCA on same dataset? K refers to the number of non zero eigenvalues when the dataset is of dimension n <...
2
votes
1answer
31 views
How to infer the eigenvalue distribution from matrix where each entry has a known Gaussian distribution?
Problem
Given $X \in \mathbb{R}^{n \times n}$ where $X_{ij} \sim \mathcal{N}(\mu_{ij}, \sigma_{ij}^2 I)$
Find the eigenvalue distribution using whatever you can.
Background
In my field, I have a ...
1
vote
1answer
81 views
Usage of the term “feature vector” in Lindsay I Smith's PCA tutorial
I'm currently following Lindsay Smith's (in large parts very well written) PCA tutorial. I'm a bit confused about the usage of the term "feature vector" in this paper though. A quote from this paper ...
2
votes
1answer
38 views
Can I use combination of eigenvectors as a single vector to explain most of variance?
I have a problem trying to find a combination (or weighted average) of variables (statistics) that best explains the sample statistics.
A – n x p matrix (n: observations p: variables, here are ...
5
votes
1answer
160 views
Must a matrix of sample pairwise covariances be PSD?
Consider a random vector $\mathbf{X}=(X)_{i=1}^n$. Then the covariance matrix $$C=\mathbb{E}[(\mathbf{X}-\mu(\mathbf{X}))(\mathbf{X}-\mu(\mathbf{X}))^\top]$$ is by definition positive-semidefinite. (...
1
vote
1answer
110 views
Orthogonality as found by the Gram-Schmidt process vs. uncorrelated basis vectors
I have a data matrix $Y$ of size $n \times p$, a basis vector in $\mathbb{R}^p$ $v_1$, and a potential basis vector in $\mathbb{R}^p$ $v_2'$. Then, if I use the Gram-Schmidt process on $[v_1, v_2']$ ...
0
votes
1answer
35 views
Exploratory factor analysis with 5 positively loaded and 1 negatively loaded factor. How to interpret?
I conducted an EFA with Maximum Likelihood and Direct Oblimin Rotation. Following the Kaiser Eigenvalue 1 rule, I identified 6 latent factors. 5 of them are positively loaded, while 1 is not.
Here ...
1
vote
1answer
61 views
Is it possible to have a basis for a covariance matrix such that the greatest variance is greater than the variance of the first eigenvector?
Suppose we have a covariance matrix $C$. Define the eigendecomposition, $C = Q^{-1} \Lambda Q$ and some other arbitrary basis $C = B^{-1} D B$.
Define $V_\text{PCA} = \text{diag}(QCQ^{-1})$ and $V_B =...
4
votes
2answers
2k views
How does eigenvalues measure variance along the principal components in PCA? [duplicate]
I understand that eigenvalues measure variance along the principal components.
Questions
How are eigenvalues and variance same for PCA?
What is the intuition behind this being the same?
What is ...
0
votes
0answers
46 views
Find the missing eigenvector values in PCA
A principal component analysis is carried out using the correlation matrix R of a
data set with n = 25 observations and p = 4 variables. The ordered eigenvalues of
R are given by
$$\lambda_1 = 2.25, \...
1
vote
0answers
67 views
Interpretation of Square of Covariance Matrix
I have a random variable, distributed as a sum of independent chi-squared random variables each with one degree of freedom.
$$ X = \sum_{i=1}^n \lambda_i \chi^2_{i(1)}$$
where $ \lambda_i$ are ...
2
votes
0answers
70 views
Expected eigenvalues of a Wishart Matrix
I consider a $n\times n$ Wishart Matrix with expected value $p \cdot I_n$, i.e. a matrix of the form
$$W = XX'$$
with $X$ a $n\times p$ matrix with independent standard normal entries.
It is easy ...
0
votes
0answers
40 views
Associated eigenvalue problem to spline penalization
In Natural Spline functions, their associated eigenvalue problem by F. Utreras they prove a result. I am interested to know if a more generalized form exists and where to find it.
Consider the space ...
-1
votes
2answers
199 views
Most important variable in a principal component
How do I tell what is the most important variable for a particular principal component? Do I calculate the absolute value for all the eigenvectors associated with that particular principal component ...
3
votes
0answers
147 views
General PCA optimization problem
I was looking at the PCA optimization problem, which is finding a matrix $U \in \mathbb{R}^{d\times n}$, $n \le d$, that solves the problem
$$\max{tr(U^TCU)},\ \ \ s.t. U^TU = I, $$
where $C$ is the ...
0
votes
0answers
14 views
What is the point of applying PCA to correlation matrix? [duplicate]
I have always applying PCA to covariance matrix . This is what most of text books and web based tutorials do :
$X^{T}X$
Is the covariance matrix when X is centered.
PCA tries to capture the big ...
1
vote
0answers
22 views
What significance can be attached to the eigenmodes of a transition matrix?
I am studying a continuous dynamical system and am categorising preferred areas of the system's phase space using a 3 state hidden Markov model.
As the resulting transition matrix is row stochastic, ...
1
vote
1answer
240 views
Covariance matrix of image data is not positive definite matrix
I've really hit the wall here and need help with direction :).
I am trying to use mvnpdf as part of basic EM algorithm but the covariance matrix of data seems to be not positive definite. There are ...
0
votes
1answer
19 views
Rank of N x D vs D x N matrices
If $X$ is a random $N \times D$ matrix where $N > D$, then why is the rank of X - mean(X, 1) $D$ while the rank of ...
1
vote
2answers
260 views
If the eigenvalues of a covariance matrix have very low variance, what does it mean?
If we have a covariance matrix $A$
$AP=PD$
where $D$ is a diagonal matrix which contains all the eigenvalues
then if the variance of the eigenvalues is very small, what does this tell us about $A$?
...
1
vote
0answers
46 views
What should I check after I do PCA?
I have X in which each row is a sample and each column is a predictor.
I de-mean X first and then construct the co-variance matrix
$A=X^{T}X$
after that I do PCA
$AP=PD$
while each column in P is ...
0
votes
1answer
46 views
Why is the result a set of factors instead of just only one?
I am trying to get into SEM and factor analysis. I understand a factor is a latent construct, say e.g. $intelligence$, user-defined by the (weighted) average of a set of indicators $x_1, x_2\dots x_n$....
1
vote
0answers
55 views
Interpreting cointegration test
I am very new to interpreting cointegration tests and eigenvalues, so any help would be appreciated.
1
vote
1answer
72 views
For an eigenvector v, is it always true that v^tv =1?
I am reading this link
PCA
which is a very insightful tutorial
however, in this tutorial, the author mentioned a constraint on PCA:
$C^{T}C=1$
when we look at eigenvalue/eigenvector definition
$...
0
votes
0answers
109 views
Why does pnorm() change the correlation in my multivariate normal sample?
I came across 2 problems that puzzle me while simulating variables for a Monte Carlo simulation, using the rnormalcopula command from the rCopula example. The first one is the one from the title, the ...
0
votes
1answer
52 views
Adding/removing variables to PCA
If I have a PCA that I ran on some set of variables, how (if at all) will it relate to the PCA results if I add or remove one variable?
Will the PCA components change in some well-defined way, or is ...
3
votes
1answer
777 views
Maximum number of principal components in PCA. Is sklearn wrong?
Recently I've been interested in applying PCA to a dataset I have and I wanted to develop a deep understanding of what I would actually be doing when I implement it.
Today I encountered two ...
