Questions tagged [eigenvalues]

For questions involving calculation or interpretation of eigenvalues or eigenvectors.

146 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
0answers
2k views

Appropriate negative eigenvalue correction for PCoA of genetic distances

I am trying to find the best way to represent genetic distances in a plane so that they may use them as response variables in canonical redundancy analysis (using ...
5
votes
0answers
470 views

Why do PCA and PCoA give the same components but different explained variances?

I'm quite familiar with Principal Component Analysisis, as I use it to study genetic structure. Lately, I was revisiting some of the functions I was using in R (...
5
votes
0answers
319 views

Why are the discriminant axes in linear discriminant analysis (LDA) not orthogonal?

This may be a quite silly question and please correct me if I'm wrong. The discriminants (discriminant axes) are essentially eigenvectors of $\mathrm{Cov}_\mathrm{within}^{-1} \mathrm{Cov}_\mathrm{...
4
votes
0answers
33 views

Physical interpretation of $U$ and $V$ matrices in SVD

I have a question about the physical interpretation of $U$ and $V$ matrices in SVD. I collect measurements at multiple devices across time are collected into an $m$ × $T$ matrix $M$, where m is the ...
4
votes
0answers
81 views

PCA: inference on the proportion of explained variance, in a large p setting

I am interested in doing inference on the proportion of total variance explained by the first principal component, for a PCA based on the correlation matrix R. I want to know the (asymptotic) ...
4
votes
0answers
122 views

Determine largest magnitude given a covariance matrix

Let's say one has a univariate Gaussian distribution with mean $\mu = 0$ and standard deviation $\sigma$. It is easy to see that the distance from $\mu$ to +1$\sigma$ is...well...$\sigma$. Let's ...
4
votes
0answers
264 views

Efficient way to solve generalized eigenvalue problem when the number of dimensions is greater than the number of samples

I am trying to solve the generalized eigenvalue problem: $$C_e v = \lambda C_o v.$$ $C_e$ and $C_o$ are both covariance matrices generated from data with $10512$ dimensions and about $2000$ samples. ...
4
votes
0answers
966 views

Pagerank vs Eigenvector centrality

What are the practical differences between PageRank and Eigenvector centralities? I don't mean the differents in how to compute the centralities, but the information they provide of a set of nodes in ...
4
votes
0answers
320 views

How do random data eigenvalues change, as random variables are added?

I am using parallel analysis (Horn 1965) to determine how many principal components I can extract from my data. I can add more variables to my dataset, but I cannot add more cases (I know, that's ...
4
votes
0answers
5k views

What is the rationale behind the “eigenvalue > 1” criterion in factor analysis or PCA?

What is the meaning of "eigenvalue > 1" criterion? I understand what eigenvalues and eigenvectors are. This question is w.r.t. this link and this statement there: By default, VARCLUS stops ...
4
votes
0answers
3k views

What is the correct way to calculate the explained variance of each EOF as calculated from a gappy data set?

I am trying to determine the correct amount of variance explained by each mode of an Empirical Orthogonal Function (EOF) analysis (similar to "PCA") as applied to a gappy data set. (i.e., containing ...
3
votes
0answers
74 views

How to draw an ellipsoid corresponding to the eigenvectors and eigenvalues of covariance matrix?

I'm doing PCA with Python with dataset decathlon in which I'm interested in 3 variables 100m, ...
3
votes
0answers
232 views

Failure to replicate calculation of PCA residuals in linear regression with heteroscedasticity

In their preprint, Rocha et al. suggest a new type of residual for linear regression models with heteroscedasticity. They call their new residual PCA residuals. I have tried to replicate some of their ...
3
votes
0answers
71 views

Maximum likelihood: Why is the number of non-zero eigenvalues equal to $x^T \hat{\Sigma}^{-1} x$

I've been reading this code (based on this R package) and I found that the number of non-zero eigenvalues of the estimated covariance is roughly equal to $x_i^T \hat{\Sigma}^{-1} x_i$. I want to know ...
3
votes
0answers
123 views

Expected eigenvalues of a Wishart Matrix

I consider a $n\times n$ Wishart Matrix with expected value $p \cdot I_n$, i.e. a matrix of the form $$W = XX'$$ with $X$ a $n\times p$ matrix with independent standard normal entries. It is easy ...
3
votes
0answers
343 views

General PCA optimization problem

I was looking at the PCA optimization problem, which is finding a matrix $U \in \mathbb{R}^{d\times n}$, $n \le d$, that solves the problem $$\max{tr(U^TCU)},\ \ \ s.t. U^TU = I, $$ where $C$ is the ...
3
votes
0answers
389 views

Distribution of eigenvalues of a random matrix

Given a random symmetric matrix A whose entries are Poisson distributed, can anything be said about the distribution of A's eigenvalues? Would be great if someone could link a paper citing such a ...
3
votes
0answers
2k views

Principal Component Analysis: how to interpret the total contribution of variables on several dimensions

When we calculate the total contribution of a variable for a single dimension, the sum of all single contributions is equal to 100%, which makes perfect sense. The http://www.sthda.com suggests to ...
3
votes
0answers
2k views

How does NIPALS algorithm work?

I'm working on NIPALS algorithm and I found this procedure from here: I'm just confused with the 4th step which stated, "using the $k$th scores, re-estimate the eigenvalues". As I understand, ...
3
votes
0answers
126 views

Distributions of eigenvalues of random matrices: what can they be used for in data mining?

I've accidentally come across some papers discussing distributions of principal components of the sample covariance matrices. An example of such a paper is Johnstone, 2001, On the distribution of the ...
3
votes
0answers
4k views

How to use Singular Value Decomposition for time series?

I want to represent a time series using the SVD algorithm. Below are some representations from this presentation. The SVD representations is formed by summing k "eigenwaves" corresponding to the ...
3
votes
0answers
3k views

Difference beween supplementary and active variables in PCA - Interpretation on obsevations?

I would like to introduce two supplementary variables into a PCA I'm conducting on a set of data measuring concentration in different material phases. However I'm unclear as to how to interpret the ...
3
votes
0answers
98 views

Sparsity regularization for eigenvectors

One way to think about finding the eigenvectors of a matrix $A$ is that they are the critical points of the functional $\vec x^\top A \vec x$ subject to $\|\vec x\|_2=1$. To regularize this problem, ...
2
votes
0answers
39 views

Dimensionality reduction of a large covariance matrix

I have a large covariance matrix $\Sigma$ and I am reducing its dimensionality by using a truncated eigendecomposition. $\Sigma \approx VDV^T$. I remember somewhere that you could also decompose it as ...
2
votes
0answers
210 views

Eigenvalue decomposition of a covariance matrix using a fast Cholesky decomposition

Let $\mathbf{C}$ be a $n \times n$ covariance matrix and assume that the LDL' Cholesky decomposition can be obtained efficiently. Can we take advantage of this to obtain a fast eigenvalue ...
2
votes
0answers
61 views

Is it better to interpret PCA components using the eigenvectors or the rescaled loadings?

I have a dataset to which I am applying PCA, and looking to each PCA component. Initially I was using the eigenvectors as a way to understand what each component "means". When using the eigenvectors ...
2
votes
0answers
55 views

Limiting results for non-unique eigenvalues and eigenvectors for a sample covariance matrix

I am working on the limiting behavior for the eigenvalue and the corresponding eigenvectors, especially the minimum eigenvalues. For instance, let $S_X=\frac{1} {T} \sum_t X_t X_t ^\prime$ be a $p \...
2
votes
1answer
88 views

Can I use combination of eigenvectors as a single vector to explain most of variance?

I have a problem trying to find a combination (or weighted average) of variables (statistics) that best explains the sample statistics. A – n x p matrix (n: observations p: variables, here are ...
2
votes
0answers
397 views

Interpretation of Square of Covariance Matrix

I have a random variable, distributed as a sum of independent chi-squared random variables each with one degree of freedom. $$ X = \sum_{i=1}^n \lambda_i \chi^2_{i(1)}$$ where $ \lambda_i$ are ...
2
votes
0answers
203 views

Data compression using either Singular Values or Eigenvalues

In many applications, an SVD of a matrix is used to determine which features are important and which ones less important. For example, in image compression, the smallest singular values are often ...
2
votes
0answers
65 views

How do I apply AIC to test out how many eigenvalues are different from 1 in a diagonalised covariance matrix?

Given that we have $x_1$,...,$x_n$ with dimensions $p * 1$ each, and $X_i$ ~ N(0,Σ), we form the diagonalised covariance matrix Σ such that the first K eigenvalues are unknown and the ...
2
votes
0answers
39 views

Connection between canonical correlation and distribution of roots of characteristic equation

I'm trying to make sense of the following sentence from introduction "Multiple discoveries: Distribution of roots of determinantal equations" http://statweb.stanford.edu/~ckirby/ted/papers/...
2
votes
0answers
486 views

Intuition of KernelPCA

I'm dealing currently with kernels and kernel PCA. For this purpose I've been reading a few papers on these topics. In this context I've been reading the paper "Kernel Principal Component Analysis" by ...
2
votes
0answers
1k views

Which of the following is NOT true regarding eigenvalues?

I am having a hard time trying to figure out the correct answer to this question. Any insight? I don't understand this at all. Which of the following is NOT true regarding eigenvalues? Option 1: An ...
2
votes
0answers
135 views

Projection on weighted kernel PCA basis

I'm performing a sort of weighted kernel PCA, where the weights of samples can be negative. The weights of all samples are given by the diagonal weight matrix $D$. The data matrix is the $n \times d$ ...
2
votes
0answers
370 views

The effect of non-positive-definite covariance matrix (in $p>n$ case) on PCA

Gene data has large number of dimensions as compared to samples. This leads to a non-positive-definite covariance matrix. In R when I try to use princomp which does ...
2
votes
0answers
2k views

Different order and signs of eigenvectors when doing PCA via eig() or svd() functions in Matlab

Assume we have a matrix X = randn(5,3). I am doing two things: 1) [S D1 V1] = svd(X); 2) ...
2
votes
0answers
17 views

Test error spikes at dimension half of sample size in linear model without regularization

I am training a basic linear model, with input data $x(n)$ and target data $t(n)$. Now, I want to make a model of dimension $d$, such that $$ y(\textbf{x}) = w_0 + \sum_{i=1}^d w_ix_i = w_0 + \textbf{...
2
votes
0answers
40 views

eigen problem of random matrix

Suppose I need to find the eigen vectors of a matrix $M(X)$ for any sample $X$ to be my dimension reduction projector. Then since I have samples $X_1,...,X_n$, so we propose to use the eigen vectors ...
2
votes
1answer
236 views

How does eigenvalues work with binary data in redundancy analysis?

I am using the vegan package in R to do a redundancy analysis (RDA, a part of canonical correlation analysis). My response data is binary and my explanatory variables contains 0, 0.5 an 1. I get quite ...
1
vote
0answers
17 views

Kernel matrix decomposition

I had a look at the sklearn.kernel_approxiamtion.Nystroem implementation, which is also described in this post: Nystroem Method for Kernel Approximation Here, a ...
1
vote
0answers
29 views

Checking that $ 𝐸[𝑥𝑥′⊗𝑥𝑥′] \prec \Sigma\otimes I+I\otimes \Sigma$

I have a random variable with mean 0 and covariance $\Sigma$, and I need to check that the following condition is satisfied $$2\Sigma\otimes \Sigma+\text{vec} \Sigma(\text{vec}\Sigma)'\prec \Sigma\...
1
vote
0answers
16 views

How to find eigenvalues and eigenvectors of the cokurtosis matrix?

Kurtosis is the fourth statistical moment of a random variable's distribution. Unlike the variance-covariance matrix $\Sigma$, which had a shape of $p\times p$, the kurtosis-cokurtosis matrix is ...
1
vote
0answers
111 views

What's the importance of parallel eigenvectors?

I'm studying eigenvectors. I read that if a matrix is symmetric and if the eigenvalues are real numbers, the eigenvectors will be perpendicular. However, I have no idea what it means (if anything) ...
1
vote
0answers
15 views

Are haar bases eigenfunctions for any kernel?

Are haar wavelet bases eigenfunctions for any kernel? If so, what Kernel is it, and how would we find the eigenvalues?
1
vote
0answers
88 views

Angle between PCA vector spaces?

I have two datasets of the same shape, one for condition A, the other for condition B. I would like to test if the major axes of variance of condition A are different than those of B. Here is my idea. ...
1
vote
0answers
55 views

Understanding the output from the Johansen Cointegration test in R

I have a VECM model that Im using to determine the revenues for a firm, based on factors like Interest rates, S&P 500 and company specific variables, as follows: Stage 1: $$z_t= a+ bX_t+e_t$$ ...
1
vote
0answers
13 views

eigenstructure matching optimization

Is there any optimization loss functions that can approximately match the eigenstructure of the original samples and the transformed samples? For example, given a collection of samples $\mathbf{X}$ ...
1
vote
0answers
25 views

Why covariance matrix estimated pairwise is not guaranteed to be PSD?

As mentioned here Is a covariance matrix composed of matrixes derived from separate samples guaranteed to be positive definitive? and here Must a matrix of sample pairwise covariances be PSD? if you ...
1
vote
0answers
11 views

First component of non-centered data

Let $X$ be a random vector of dimension $p$ and $\{ X_1, \dots, X_n \}$ the $n$ observations of such vector. Let $\mathbb{X}$ be the matrix with rows $X_k$ and $\mathbb{X}_c$ is the matrix with rows $...