Questions tagged [eigenvalues]
For questions involving calculation or interpretation of eigenvalues or eigenvectors.
416
questions
0
votes
1
answer
474
views
Interpretation of Eigenvalue vs. Singular Value plot
I'm doing some preliminary analysis on the feature matrix for a certain dataset (rows are observations, columns are feature dimensions).
I have computed the SVD and PCA decompositions for this matrix ...
- 175
2
votes
1
answer
36
views
Determine missing eigenvalues given only correlations between variables and components
Given that I just have a correlation matrix ($X$ Variables vs. $Y$ Principal Components), and that I am trying to find 2 missing eigenvalues (e.g., missing $\lambda_1$ and $\lambda_5$) from the total $...
- 21
0
votes
1
answer
191
views
Gradient descent derivation in Eigenspace [duplicate]
I am trying to decode article on https://distill.pub/2017/momentum/
I was able to follow everything until the part with a change of basis x$^k=Q^T(w^k−w^⋆)$ to eigenspace... I conceptually understand ...
- 47
2
votes
0
answers
468
views
Eigenvalue decomposition of a covariance matrix using a fast Cholesky decomposition
Let $\mathbf{C}$ be a $n \times n$ covariance matrix and assume that
the LDL' Cholesky decomposition can be obtained efficiently. Can we
take advantage of this to obtain a fast eigenvalue ...
- 4,688
3
votes
1
answer
68
views
Does $\text{cov}(a_1' X, a_2' X) = 0$ imply $a_1 \cdot a_2 = 0$?
Let $X$ be a $p$-dimensional random vector with $p$ principal components $y_1, y_2, \dots, y_p$. By definition, a restriction put on the second principal component $y_2 = a_2'X$ is
$$
\text{cov}(y_1, ...
- 1,677
5
votes
1
answer
1k
views
Why do PCA and PCoA give the same components but different explained variances?
I'm quite familiar with Principal Component Analysisis, as I use it to study genetic structure. Lately, I was revisiting some of the functions I was using in R (...
- 51
0
votes
1
answer
3k
views
Calculate first principal component direction and scores
Given that x1 = (9, 9, −18)^T and x2 = (18, 9, 9)^T with eigendecomposition of its sample covariance matrix Σ = cov(X)
How do I calculate the first two principal component direction and the ...
- 101
3
votes
0
answers
333
views
Failure to replicate calculation of PCA residuals in linear regression with heteroscedasticity
In their preprint, Rocha et al. suggest a new type of residual for linear regression models with heteroscedasticity. They call their new residual PCA residuals. I have tried to replicate some of their ...
- 27.9k
1
vote
0
answers
14
views
Is eigenspace based classification possible
Imagine I would like to classify an image (e.g. into healthy and sick)
and have a lot of labeled data. Could I classify any image by comparing it to the eigenspaces of the two sets?
It sounds simple, ...
- 83
0
votes
0
answers
137
views
When the elements of first basis are always positive for PCA?
I am computing the PCA projection matrix of some data. I notice that the elements of first basis vector (corresponding to the highest eigenvalue) are always positive. My data is real and contain both ...
- 658
2
votes
0
answers
94
views
Is it better to interpret PCA components using the eigenvectors or the rescaled loadings?
I have a dataset to which I am applying PCA, and looking to each PCA component.
Initially I was using the eigenvectors as a way to understand what each component "means". When using the eigenvectors ...
- 133
0
votes
0
answers
228
views
eigen function in R
I want to ask about the eigen function of R.
I am currently doing a project of NBA team analysis. I am trying to figure out correlation effect of two players lineup ...
- 1
1
vote
0
answers
20
views
Basis vectors for categorical images
I have a sequence of categorical images. For a two category image, each image pixel can have one of two values. I would like to analyze these images using a technique like eigen images. The goal is to ...
- 11
0
votes
1
answer
4k
views
How to interpret eigenvectors in PCA analysis?
I'm trying to apply the output from PCA analysis I've run on some yield curve history and am getting a bit confused. I have followed the steps below,
From a history of the yield curve ($m \times n$ ...
1
vote
0
answers
88
views
Kaiser Criterion after Rotation
Within my PCA, I rotated the solution, so the output would make more sense given the complex structure using a Varimax rotation. However, when I am confirming the number of factors to extract based on ...
- 603
2
votes
1
answer
2k
views
Drawing 95% ellipse over scatter plot
The context is regression analysis using Eviews, but first I wanted to create a few scatter plots and overlay error ellipses on them. Eviews doesn't support that kind of graph ornamentation so I am ...
- 837
1
vote
2
answers
180
views
Significance of eigenvector components in PCA
Long time reader, first time poster. Hopefully I won't screw this up...
In the context of Principal Component Analysis, I have the sense that the components of an eigenvector are a measure of the ...
4
votes
2
answers
2k
views
adding a small constant to the diagonals of a matrix to stabilize
I have a large correlation matrix (110x110) with some small eigenvalues (about 20 < 0.1). It has been suggested that adding a constant (about 0.1) to the diagonals will help to stabilize the matrix....
- 41
0
votes
1
answer
44
views
Graphical understanding of PCA
I learned about PCA and how to find the principal components via eigenvectors/values. Now for the following problems my professor says that "Feature 2 is constant and can hence be ignored, so you can ...
- 161
1
vote
1
answer
92
views
importance of correlation between data for a PROC CLUSTER
i'm working on a clustering analysis on SAS.
I need to improve an actual code :
...
- 430
0
votes
0
answers
60
views
Determining the Direction of Eigenvectors in PCA [duplicate]
I'm using R to get the principal components for several datasets.
An example result, using prcomp yields:
...
- 153
0
votes
0
answers
215
views
Idea behind change of basis and how it relates to projecting your points onto principal components
I would like to clarify if my understanding is correct. In the traditional X-Y coordinate system, our choice of basis vectors are $\vec{i} = (1, 0)$ and $\vec{j} = (0, 1)$ and when you I have a point $...
- 123
1
vote
0
answers
187
views
Change in eigenvalues due to perturbation to a correlation matrix
Let $A$ be a $m \times n$ matrix defined as
$ A = \Big[\frac{a_1}{\|a_1\|} \cdots \frac{a_n}{\|a_n\|}\Big]$ and $a_k \in \mathbb{R}^{m\times 1}$ where $k \in [1,\dots,n]$.
Now, we define a ...
- 11
3
votes
0
answers
136
views
Maximum likelihood: Why is the number of non-zero eigenvalues equal to $x^T \hat{\Sigma}^{-1} x$
I've been reading this code (based on this R package) and I found that the number of non-zero eigenvalues of the estimated covariance is roughly equal to $x_i^T \hat{\Sigma}^{-1} x_i$. I want to know ...
0
votes
1
answer
137
views
The miracle of the Lanczos/conjugate gradient algorithm
Lanczos/Arnoldi/Rietz/CG-like algorithm share the same core strategy... In each, a little miracle appears, most of the Gram-Schmidt inner products are zeroes ! In others words, new direction need only ...
- 21
3
votes
1
answer
752
views
How to estimate the largest eigenvalue of a correlation matrix from one observation of underlying data matrix?
Suppose that I have $N$ time series $x_{1t},x_{2t},\dots,x_{Nt},$, that are correlated with each other. A $N\times N$ correlation matrix is $R=\rho_{ij}$. It can be represented with eigen value ...
- 58.5k
2
votes
0
answers
62
views
Limiting results for non-unique eigenvalues and eigenvectors for a sample covariance matrix
I am working on the limiting behavior for the eigenvalue and the corresponding eigenvectors, especially the minimum eigenvalues. For instance, let $S_X=\frac{1} {T} \sum_t X_t X_t ^\prime$ be a $p \...
- 121
0
votes
1
answer
1k
views
Are eigenfaces same as eigenvectors?
I'm trying to understand the difference between eigenvectors and eigenfaces, are they different names for same concepts?
I ask this because I got confused when I am trying to compute eigenvectors for ...
- 105
0
votes
1
answer
20
views
PCA- creating a model with values obtained
Hoping somebody can help me. I cannot find an example that 'finishes' a problem. I run a proc princomp in SAS. I have hundreds of variables but used four for the purpose of an example. I ...
- 21
3
votes
2
answers
813
views
Compute the $k$ largest eigenvector in spectral clustering
In Spectral Clustering, we need to compute the top $k$ largest eigenvector of normalized $L$.
$$L = D^{-\frac{1}{2}}SD^{-\frac{1}{2}}$$
In Andrew NG's paper, L is not positive definite (unless ...
- 133
3
votes
1
answer
2k
views
Eigenvalue decomposition/SVD and the filtering perspective
I have been studying the SVD algorithm recently and I can understand how it might be used for compression but I am trying to figure out if there is a perspective of SVD where it can be seen as a low ...
- 4,530
0
votes
1
answer
1k
views
PCA in psych package with more columns than rows
Why is it impossible to do a PCA in R using principal from psych package without warnings with a matrix, which has more columns ...
- 143
1
vote
0
answers
86
views
PCA for three-dimensional linear fit on time-resolved trajectory
I study the behavior of organisms that are able of self-locomotion and that show directed motion toward one another. This directed motion occurs through the detection of chemical trails released by ...
3
votes
3
answers
4k
views
PCA basic: Must eigenvalues converge to zero at high dimension?
Recently, I obtained several PCA plots, and because I am unable to produce eigenvalues for higher dimensions, I tried to extrapolate them based on the available data. The reason why I want to do this ...
- 65
2
votes
1
answer
272
views
PCA Basic: Do eigenvalues remain constant with increase in dimensions?
Say I have a set of genomic data to be analysed for population structure. I ran two different analyses using two different maximum principal components. For the first analysis, I modeled the data ...
- 65
1
vote
3
answers
2k
views
PCA to choose variables based on its loadings on PC1 [duplicate]
I have a dataset of cave dimensions (and other variables related to their features).
The problem is that 3 of these variables are: Length, Area, and Volume.
These 3 are highly correlated as they ...
- 23
2
votes
1
answer
1k
views
Negative eigenvalues when computing MultiDimensional Scaling given nonnegative distance matrix
I am using the Smile MDS
https://github.com/haifengl/smile/blob/master/core/src/main/java/smile/mds/MDS.java and occasionally running into:
...
0
votes
0
answers
984
views
Interpreting low Cronbach's alpha value in CATPCA
I ran categorical principal component analysis (CATPCA) on the data that I collected through a questionnaire. The purpose of the questionnaire was to understand pedestrians intentions and attitudes ...
9
votes
0
answers
676
views
Why are the discriminant axes in linear discriminant analysis (LDA) not orthogonal?
This may be a quite silly question and please correct me if I'm wrong.
The discriminants (discriminant axes) are essentially eigenvectors of $\mathrm{Cov}_\mathrm{within}^{-1} \mathrm{Cov}_\mathrm{...
- 485
1
vote
0
answers
45
views
Is max. Eigenvalue of k-sparse PCA always $\leq$ max. Eigenvalue of normal PCA on same dataset?
Is max. Eigenvalue of k-sparse PCA always less than or equal to the max. Eigenvalue of normal PCA on same dataset? K refers to the number of non zero eigenvalues when the dataset is of dimension n <...
- 2,425
3
votes
1
answer
126
views
How to infer the eigenvalue distribution from matrix where each entry has a known Gaussian distribution?
Problem
Given $X \in \mathbb{R}^{n \times n}$ where $X_{ij} \sim \mathcal{N}(\mu_{ij}, \sigma_{ij}^2 I)$
Find the eigenvalue distribution using whatever you can.
Background
In my field, I have a ...
2
votes
1
answer
1k
views
Usage of the term "feature vector" in Lindsay I Smith's PCA tutorial
I'm currently following Lindsay Smith's (in large parts very well written) PCA tutorial. I'm a bit confused about the usage of the term "feature vector" in this paper though. A quote from this paper ...
- 458
2
votes
1
answer
245
views
Can I use combination of eigenvectors as a single vector to explain most of variance?
I have a problem trying to find a combination (or weighted average) of variables (statistics) that best explains the sample statistics.
A – n x p matrix (n: observations p: variables, here are ...
- 29
5
votes
1
answer
481
views
Must a matrix of sample pairwise covariances be PSD?
Consider a random vector $\mathbf{X}=(X)_{i=1}^n$. Then the covariance matrix $$C=\mathbb{E}[(\mathbf{X}-\mu(\mathbf{X}))(\mathbf{X}-\mu(\mathbf{X}))^\top]$$ is by definition positive-semidefinite. (...
- 153
2
votes
1
answer
2k
views
Orthogonality as found by the Gram-Schmidt process vs. uncorrelated basis vectors
I have a data matrix $Y$ of size $n \times p$, a basis vector in $\mathbb{R}^p$ $v_1$, and a potential basis vector in $\mathbb{R}^p$ $v_2'$. Then, if I use the Gram-Schmidt process on $[v_1, v_2']$ ...
- 51
0
votes
1
answer
458
views
Exploratory factor analysis with 5 positively loaded and 1 negatively loaded factor. How to interpret?
I conducted an EFA with Maximum Likelihood and Direct Oblimin Rotation. Following the Kaiser Eigenvalue 1 rule, I identified 6 latent factors. 5 of them are positively loaded, while 1 is not.
Here ...
- 11
1
vote
1
answer
218
views
Is it possible to have a basis for a covariance matrix such that the greatest variance is greater than the variance of the first eigenvector?
Suppose we have a covariance matrix $C$. Define the eigendecomposition, $C = Q^{-1} \Lambda Q$ and some other arbitrary basis $C = B^{-1} D B$.
Define $V_\text{PCA} = \text{diag}(QCQ^{-1})$ and $V_B =...
- 51
6
votes
2
answers
32k
views
How does eigenvalues measure variance along the principal components in PCA? [duplicate]
I understand that eigenvalues measure variance along the principal components.
Questions
How are eigenvalues and variance same for PCA?
What is the intuition behind this being the same?
What is ...
- 5,493
0
votes
0
answers
725
views
Find the missing eigenvector values in PCA
A principal component analysis is carried out using the correlation matrix R of a
data set with n = 25 observations and p = 4 variables. The ordered eigenvalues of
R are given by
$$\lambda_1 = 2.25, \...
- 233
4
votes
0
answers
222
views
Expected eigenvalues of a Wishart Matrix
I consider a $n\times n$ Wishart Matrix with expected value $p \cdot I_n$, i.e. a matrix of the form
$$W = XX'$$
with $X$ a $n\times p$ matrix with independent standard normal entries.
It is easy ...
- 12.3k