Questions tagged [eigenvalues]

For questions involving calculation or interpretation of eigenvalues or eigenvectors.

Filter by
Sorted by
Tagged with
2
votes
1answer
236 views

How does eigenvalues work with binary data in redundancy analysis?

I am using the vegan package in R to do a redundancy analysis (RDA, a part of canonical correlation analysis). My response data is binary and my explanatory variables contains 0, 0.5 an 1. I get quite ...
1
vote
1answer
8k views

Understanding PCA - How to calculate scores [duplicate]

I'm looking for advice to whether or not the following method is good and is standard for calculating PCA of the data. So the examples that I will give will be small. Given a matrix of $A = [4, 6, ...
1
vote
1answer
138 views

Is there a way to “regularize” the Johansen cointegration test?

I'm using a Johansen cointegration test to check for cointegration among a large number of time series. I've found that while the eigenvectors look great in-sample, the cointegrating relationships ...
1
vote
0answers
727 views

Trying to use Cholesky decomposition of covariance matrix to sample error ellipsoid

I'm trying to construct an error ellipsoid from a covariance matrix (which exists for a 3D point) and then sample consistent xyz points in this region. In a previous question when I asked about this (...
12
votes
2answers
15k views

Why does PCA maximize total variance of the projection?

Christopher Bishop writes in his book Pattern Recognition and Machine Learning a proof, that each consecutive principal component maximizes the variance of the projection to one dimension, after the ...
1
vote
2answers
2k views

How to prove higher eigenvalues correspond to significant principal components?

How to prove higher eigenvalues correspond to significant principal components? Is there any explanation or at least intuition for that, if not a proof? I would like to convince myself with some ...
6
votes
0answers
2k views

Appropriate negative eigenvalue correction for PCoA of genetic distances

I am trying to find the best way to represent genetic distances in a plane so that they may use them as response variables in canonical redundancy analysis (using ...
0
votes
0answers
88 views

PCA: When using eigendecompostion instead of single value decompostion [duplicate]

If you want to perform a PCA, I guess that using SVD will always work. Eigendecomposition on the covariance matrix only works when your data is not high dimensional(so n > p). But I'm wonder if there ...
6
votes
3answers
7k views

Is centering a valid solution for multicollinearity?

Let's assume that $y = a + a_1x_1 + a_2x_2 + a_3x_3 + e$ where $x_1$ and $x_2$ both are indexes both range from $0-10$ where $0$ is the minimum and $10$ is the maximum. I found by applying VIF, CI and ...
3
votes
1answer
1k views

Tolerance for pseudoinverse

In calculating the pseudoinverse of a matrix $A$, of size (m,n), I need to choose a tolerance threshold for the eigenvalues. I'm trying to understand how I should pick this. Matlab default is to use ...
5
votes
2answers
929 views

Factors with only two variables in factor analysis

I am running a factor analysis and have a couple of questions. I have 10 variables, all of them come from a survey, with each answer is in the scale of 1 to 7. I have calculated a correlation matrix ...
5
votes
1answer
731 views

How to best define a “contrast” in a Principal Component Analysis (PCA)?

I have been studying how to interpret principal components. I recently came across an example of a particular eigenvector: $$e_j^T = \left[ \frac{\sqrt{2} }{2}, \frac{-\sqrt{2} }{2}, 0, \dots,0 \...
5
votes
4answers
4k views

Condition number of covariance matrix

I am interested in generating a covariance matrix of dimension say 100. I managed to get a correlation matrix with finite condition number. To construct a covariance matrix I need to have standard ...
2
votes
1answer
2k views

Plotting error ellipsoid from 3x3 covariance matrix in R?

I'm hoping to be able to take a 3x3 covariance matrix and turn this into an error ellipsoid but so far I haven't been able to achieve this. I'm very new to R (in fact turned to it to attempt to solve ...
1
vote
0answers
169 views

Principal Components, Canonical Correlation and Eigenvalue problems [duplicate]

It is well known that the solution to the optimization problems proposed in Principal Components and Canonical Correlation Analysis are given by the solution to eigenvalue problems and generalized ...
0
votes
0answers
586 views

Computing CDF of multivariate Gaussian from eigendecomposition

I have a multivariate Gaussian for a set of data, and I'd like to compute the confidence interval for that data sample. In hopes of finding an elegant solution, I did an eigendecomposition and ...
5
votes
1answer
3k views

Eigenvalues of correlation matrices exhibit exponential decay

I have a data-set of $P$ samples of size $N$, and noticed that the eigenvalues of the correlation matrices $A^TA$, when presented in descending order, can in many cases be described as an exponential ...
6
votes
1answer
6k views

Confused about Cholesky and eigen decomposition

I'm looking to generate correlated random variables. I have a symmetric, positive definite matrix. So I know that you can use the Cholesky decomposition, however I keep being told that this only works ...
2
votes
2answers
1k views

Whitening data before regression, should I whiten the response variable too?

I have some data X where the samples are not independent (they're correlated with each other), and I'm trying to do a regression of some continuous variable y on X. This sample correlation could ...
5
votes
1answer
2k views

nearPD function in Matrix package

Does anyone know how the eigenvalues are adjusted to make a non-positive definite matrix into a positive definite matrix in Matrix package? I mean in nearPD function.
2
votes
1answer
665 views

Eigenvalue and eigen vector of covariance matrix in mean*mean'

I know the rank of $\Sigma^{-1}m m'$ is $1$. How can I find the eigenvalue and eigenvector of it? ($\Sigma^{-1}$ is the inverse of a covariance matrix and $m$ is a mean vector.)
1
vote
1answer
5k views

Relation between best fit line and eigenvector of maximum eigen value of an estimated covariance matrix [closed]

(This question is from my pattern recognition course.) There is this exercise: Imagine we have $N$ samples with $n$ dimensions. First it asks to find a point $m$ where the summation of Euclidean ...
3
votes
2answers
1k views

I've been trying to wrap my head around the use of eigenvalues in cluster analysis. What does it tell me about my clustering behavior?

In a typical hierarchical cluster output from using SAS, the first table given lists all of the eigenvalues. From what I understand, eigenvalues are derived from covariance between the variables. ...
5
votes
3answers
2k views

Perform PCA. Extract PCs. Can one then tell what the most important _original_ features were, from the PCs? [duplicate]

Suppose that you have 1000 features, and a data set made up of say, 50,000 points. Suppose then that we perform PCA, and we extract the top 5 PCs, since they explain 99.99 percent of the variance, and ...
43
votes
3answers
46k views

Why does correlation matrix need to be positive semi-definite and what does it mean to be or not to be positive semi-definite?

I have been researching the meaning of positive semi-definite property of correlation or covariance matrices. I am looking for any information on Definition of positive semi-definiteness; Its ...
6
votes
1answer
8k views

How to interpret variation explained by principal coordinates?

I have recently seen a couple of Principal Coordinates Analysis (PCoA) projection plots which show "percentage variation explained" by the respective principal coordinates. Given that the analysis is ...
0
votes
1answer
1k views

Truncated SVD matrix reconstruction: what is the meaning of the real values?

Im my algorithm, I am working with Singular Value Decomposition (SVD). I have an input matrix $A_{in} \in \{0,1\}^{(m * n)} $, made by $n$ rows and $m$ colums. All the entries are 0 or 1. I ...
2
votes
1answer
911 views

Standardization of compositional data in PCA versus using real data

I have a question about conducting a PCA between variables that are measured in different units. I understand the importance of using a correlation matrix versus a covariance matrix to minimize ...
3
votes
0answers
3k views

Difference beween supplementary and active variables in PCA - Interpretation on obsevations?

I would like to introduce two supplementary variables into a PCA I'm conducting on a set of data measuring concentration in different material phases. However I'm unclear as to how to interpret the ...
3
votes
0answers
98 views

Sparsity regularization for eigenvectors

One way to think about finding the eigenvectors of a matrix $A$ is that they are the critical points of the functional $\vec x^\top A \vec x$ subject to $\|\vec x\|_2=1$. To regularize this problem, ...
2
votes
1answer
4k views

Analysis of compounds using PCA - selecting the right PCA “type” for the data…?

I'm completing scientific analysis of chemical compounds in consumer products. As a non-statistician, I would really appreciate any thoughts from the experts here. My data is non-normal so I've used ...
2
votes
1answer
266 views

Physical significance: multiplying matrix by outer product of its eigenvector

I stumbled around this piece of code: v1 <- eigen(X.center %*% t(X.center))$vectors[,1] X.0 <- v1 %*% t(v1) %*% X.center while v1 is the eigenvector ...
3
votes
1answer
1k views

Why do R function `eigen()' and Armadillo's `eig_sym()` give different results

I am trying to compute eigenvalues in C++ using the Armadillo function eig_sym via RcppArmadillo. The results are not entirely ...
2
votes
1answer
71 views

Regression Estimation difficulties

My regression problem is properly formulated, but is encountering serious computational difficulties. Dependent: $Y$ = multinomial Independent: $X_1, \dots, X_{90}$ = linearly independent set of ...
0
votes
0answers
532 views

Eigenvectors of a shuffled correlation matrix

A question about a shuffled vs. unshuffled correlation matrix I took the correlation matrix and shuffled between its values symmetrically. (shuffled only the left lower triangle of the matrix and ...
8
votes
1answer
230 views

Distribution of eigenvalues given one is known

I'm familiar with using insights from Random Matrix Theory to determine the number of principal components from the PCA of a covariance/correlation matrix to use to form factors. If the eigenvalue ...
0
votes
1answer
1k views

Exploratory factor analysis and eigenvalues

So, I ran an EFA on 60 items. Analysis resulted in 19 components with an eigenvalue of a score greater than 1. The only factors that theoretically make sense and that include more then 3 items have ...
0
votes
0answers
1k views

How do I convert the variances explained by a principal component into an eigenvalue?

I'm using SigmaPlot to run PCA on various measurements that should all correspond to size of an animal. Running PCA using a covariance matrix (instead of a correlation matrix, since all of the ...
6
votes
1answer
1k views

Adversarial noise in PCA

PCA is known to be quite sensitive to outlier noise (and this is why several Robust PCA techniques exists.) However, I am looking for a concrete example of sensitivity of PCA to adversarial noise that ...
1
vote
1answer
2k views

Dimensionality Reduction using PCA, with SVD of correlation matrix

I have computed a correlation matrix from certain data set of dimension 6 The correlation matrix is: ...
0
votes
1answer
3k views

PCA eigenvectors with dimensionality reduction

I want to understand how I can compute the eigenvectors and the eigenvalues of a matrix using dimensional reduction.I have a Matrix $M$ of dimensions $n$ x $d$ using dimension reduction I can compute ...
-1
votes
1answer
2k views

Determining eigenvalues and eigenvectors in R

x<-model.matrix(d) [,-1] e<-eigen(t(x) %*% x) e values [1] 1174600.548 21261.741 16133.842 6206.181 1856.894 First my $\alpha_1=1174600.548$, but ...
9
votes
2answers
533 views

Why is the amount of variance explained by my 1st PC so close to the average pairwise correlation?

What is the relationship between the first principal component(s) and the average correlation in the correlation matrix? For example, in an empirical application I observe that the average ...
0
votes
1answer
68 views

Show that all the characteristic roots of a dispersion matrix of a random variable are non-negative

Show that all the characteristic roots of a dispersion matrix of a random variable are non-negative. $$\begin{vmatrix} \sigma_{11}-\lambda & \sigma_{12} & \cdots & \sigma_{1p}\\ \sigma_{...
4
votes
0answers
3k views

What is the correct way to calculate the explained variance of each EOF as calculated from a gappy data set?

I am trying to determine the correct amount of variance explained by each mode of an Empirical Orthogonal Function (EOF) analysis (similar to "PCA") as applied to a gappy data set. (i.e., containing ...
12
votes
1answer
3k views

Why are eigen and svd decompositions of a covariance matrix based on sparse data yielding different results?

I am trying to decompose a covariance matrix based on a sparse / gappy data set. I'm noticing that the sum of lambda (explained variance), as calculated with svd, ...
4
votes
1answer
742 views

Eigenvalues nearly all 1 in canonical correlation analysis

I ran a Canonical Correlation Analysis on about 845 cases with 1000 variables each. (It originally started with 1000 cases and 400 variables but by using a kernel I got a 1000x1000 matrix) As a ...
9
votes
1answer
502 views

Estimated distribution of eigenvalues for i.i.d. (uniform or normal) data

Assuming I have a data set with $d$ dimensions (e.g. $d=20$) so that each dimension is i.i.d. $X_i \sim U[0;1]$ (alternatively, each dimension $X_i \sim \mathcal N[0;1]$) and independent of each other....
1114
votes
28answers
684k views

Making sense of principal component analysis, eigenvectors & eigenvalues

In today's pattern recognition class my professor talked about PCA, eigenvectors and eigenvalues. I understood the mathematics of it. If I'm asked to find eigenvalues etc. I'll do it correctly like ...

1
3 4 5 6
7