Questions tagged [eigenvalues]

For questions involving calculation or interpretation of eigenvalues or eigenvectors.

Filter by
Sorted by
Tagged with
0
votes
0answers
349 views

How to compute eigenvalues from loadings in a PCA?

I have a table with the factor loadings obtained from a PCA for each variable and each component. Is it possible only with these data to obtain the eigenvalues for each Principal Component?
0
votes
1answer
68 views

My PCA projected points don't look right

I checked my PCA code many times, and I can't find anything wrong. I do PCA with eigen-decomposition, by using the eigen function in R. My data is 2-dimensional, I want to reduce it to one dimension. ...
2
votes
1answer
962 views

Lower bound on smallest eigenvalue of covariance matrices

Assume that a class of $p\times p$ covariance matrices is characterized by a parameter $\theta$, i.e, $$\mathbb{F} = \left\{\Sigma(\theta), \theta\in R\right\}$$ Also suppose we know the following ...
2
votes
1answer
2k views

Geometric interpretation of covariance matrix

In the following graph, the diagonal elements in the covariance matrix are the same, which I suppose means the spread of data in either direction should be the same, but why the data points are still ...
1
vote
1answer
370 views

What is the fastest way to calculate the leading singular value of a very large matrix (10mln x 100k)?

I only know of the following power iteration. But it needs to create a huge matrix A'*A when both of rows and columns are pretty large. And A is a dense matrix as well. Is there any alternative to ...
0
votes
1answer
324 views

Generating a positive semi-definite covariance matrix (using KL transform)

I have a set of input data X consisting of S&P 500 returns, that provides me with a covariance matrix C that is non positive semi-definite. The reason for the non-semi definite nature of the ...
5
votes
1answer
1k views

How can eigenfaces (PCA eigenvectors on face image data) be displayed as images?

I am trying to clarify some concepts for face recognition. According to my understanding, given a training set of images with each image measuring 225 x 255 pixels, we will have a matrix of training ...
0
votes
1answer
803 views

How does the matrix being centered change the eigenvalues/eigenvectors?

I have this question and I think I can just use maths for (ii) X'Xa=lambda a XX'Xa=XX'(Xa)=lambda(Xa) And so if lambda and a are eigenvalues/eigenvectors of X'X then lambda and Xa will be eigenvalues/...
1
vote
1answer
142 views

Jackknife and matrix diagonalization

Suppose I have $M$ matrices $n\times n$ of the form: \begin{equation} C^{k} = \begin{bmatrix} x^{k}_{11} & x^{k}_{12} & x^{k}_{13} & \dots & x^{k}_{1n} \\ x^{k}_{21} ...
0
votes
0answers
96 views

Different eigenvalues in R and SPSS

I'm trying to understand some canonical correlation outputs, and I found differences between eigenvalues results for R and SPSS. Some code: ...
4
votes
1answer
2k views

spectral clustering/theory, is there any meaning for the magnitude of values in eigenvectors?

Basically, spectral clustering is an application of spectral graph theory, which utilizes the eigenvalues and eigenvectors of a Laplacian matrix or adjacency matrix to disclose the connected ...
4
votes
1answer
4k views

Covariance matrix decomposition and coregionalization

The original question (that can be seen at the bottom of this post) was replaced by this first edit (below) EDIT I I give more details about my problem. First of all let suppose to have K vectors $\...
4
votes
0answers
130 views

Determine largest magnitude given a covariance matrix

Let's say one has a univariate Gaussian distribution with mean $\mu = 0$ and standard deviation $\sigma$. It is easy to see that the distance from $\mu$ to +1$\sigma$ is...well...$\sigma$. Let's ...
2
votes
0answers
68 views

How do I apply AIC to test out how many eigenvalues are different from 1 in a diagonalised covariance matrix?

Given that we have $x_1$,...,$x_n$ with dimensions $p * 1$ each, and $X_i$ ~ N(0,Σ), we form the diagonalised covariance matrix Σ such that the first K eigenvalues are unknown and the ...
3
votes
0answers
2k views

Principal Component Analysis: how to interpret the total contribution of variables on several dimensions

When we calculate the total contribution of a variable for a single dimension, the sum of all single contributions is equal to 100%, which makes perfect sense. The http://www.sthda.com suggests to ...
11
votes
2answers
4k views

Why are PCA eigenvectors orthogonal and what is the relation to the PCA scores being uncorrelated?

I'm reading up on PCA, and I'm understanding most of what's going on in terms of the derivation apart from the assumption that eigenvectors need to be orthogonal and how it relates to the projections (...
2
votes
0answers
41 views

Connection between canonical correlation and distribution of roots of characteristic equation

I'm trying to make sense of the following sentence from introduction "Multiple discoveries: Distribution of roots of determinantal equations" http://statweb.stanford.edu/~ckirby/ted/papers/...
1
vote
0answers
25 views

How to get loadings and scores of a matrix basing of PCA [duplicate]

What are the mathematical steps to get loadings and scores matrices of a 3x3 matrix basing of PCA and what is the relationship relating eigenvalues eigenvectors with loadings and score?
3
votes
1answer
458 views

Givens rotation with Eigendecomposition

Setup Let $\Sigma_t$ be the time $t$ positive-definite covariance matrix of some $N$-dimensional random vector $X_t$. Denote $\Sigma_t = P_t \Lambda_t P_t$ be the spectral/eigen-decomposition of this ...
1
vote
1answer
163 views

What is the interpretation of an S-shaped curve in the plot of eigenvalues of covariance matrix

From PCA's point of view, eigenvalues of covariance matrix should correspond to the principle components. Therefore, when we plot these eigenvalues on a data sets with some clustering pattern, we ...
9
votes
2answers
708 views

Why cannot I obtain a valid SVD of X via eigenvalue decomposition of XX' and X'X?

I am trying to do SVD by hand: ...
1
vote
0answers
74 views

Largest eigenvalue of the determinantal equation

I'm reading a paper and it defines $\lambda(\omega)$ as the largest eigenvalue of the determinantal equation: $$|A'f^{re}(\omega)A-\lambda A'VA|=0$$ Where $f^{re}(\omega)$ is the real part of $f(\...
3
votes
1answer
2k views

Implementing fancy PCA augmentaiton

I am really struggling to implement this fancy PCA augmentation method described in this paper, here is what I believe I must do (correct me if I am wrong): 1) Create a Matrix where the first column ...
2
votes
0answers
510 views

Intuition of KernelPCA

I'm dealing currently with kernels and kernel PCA. For this purpose I've been reading a few papers on these topics. In this context I've been reading the paper "Kernel Principal Component Analysis" by ...
1
vote
1answer
2k views

Reconstruction of original dataset through loadings in PCA [duplicate]

I am very new to PCA and I was trying, just as excercize, to reconstruct original dataset from loadings. Let's suppose I have a matrix A corresponding to the original dataset and C that is the z-...
1
vote
0answers
165 views

Explicit Eigenfunction Approximation from Eigendecomposition of Radial Basis Kernel Matrix

I am currently studying Kernel Ridge Regression. Specifically, I am considering the radial basis function kernel. Throughout literature I am seeing plots of the eigenfunctions that are the ...
1
vote
2answers
2k views

Orthogonality in PCA vectors [duplicate]

Why should the second PCA vector i.e. vector with largest variance in reduced subspace be orthogonal to the first PCA vector?
0
votes
0answers
16 views

Understanding PCA from the perspective of eigenspaces [duplicate]

This question is from the perspective of a student who has only a fundamental idea of eigenvectors and eigenspaces (and linear algebra in general). If my understanding is correct, an eigenvector E of ...
0
votes
1answer
445 views

Can someone explain the simple intution between Principal component 1, 2, … etc in PCA? [duplicate]

I see that in PCA the first principal component maximizes the variances amongst all the points within the data set. What exactly does this mean, what does it show and what does every other principal ...
8
votes
2answers
1k views

Why does the first eigenvector in PCA resemble the derivative of an underlying trend?

I am using PCA to analyze several spatially related time series, and it appears that the first eigenvector corresponds to the derivative of the mean trend of the series (example illustrated below). I ...
2
votes
0answers
1k views

Which of the following is NOT true regarding eigenvalues?

I am having a hard time trying to figure out the correct answer to this question. Any insight? I don't understand this at all. Which of the following is NOT true regarding eigenvalues? Option 1: An ...
1
vote
2answers
2k views

Using PCA in Matlab: Is it based on the covariance or correlation matrix?

I want to produce a scree plot to assess if there is an 'elbow' in the eigenvalues to aid in my identification of the number of PCs to retain. However, upon reading further into the topic, I realised ...
6
votes
1answer
991 views

Why does the direction with highest eigenvalue have the largest semi-axis?

So, in PCA, we decompose the covariance matrix by its eigenvalues and eigenvectors. I understand that an ellipsoid is fully characterized by the eigenvalues and eigenvectors of a positive definite ...
2
votes
1answer
342 views

Kernel methods - How do the infinite dimensions arise?

I'm going through the Kernel chapter from Kevin Murphy's Machine Learning book and he talks about the importance of positive definite or Mercer kernels: My question comes from the last sentence: What ...
3
votes
0answers
229 views

Does it make sense to add the principal components together to produce a single index? [duplicate]

I want to build an index from, say, 13 variables. I run a PCA for these 13 variables to produce 13 principal components, 5 of which have an Eigenvalue of more than 1. While some researchers use only ...
4
votes
1answer
113 views

How to intuitively understand the reconstruction of the original data after dimension reduction

So I have been trying to understand PCA for the past day, and the part that I don't understand is when the original data is reconstructed after dimension reduction. Below is the code that I was follow ...
4
votes
1answer
2k views

Why do prcomp() and eigen(cov()) in R return different signs of PCA eigenvectors?

I understand the sign of the eigen vectors / PCA rotations can be positive or negative (see here or here). But I am curious why the following two approaches yield different results, from numerical ...
1
vote
0answers
90 views

is covariance matrix and eigen vector question

is every covariance matrix 's eigenvector is orthogonal? why symmetric matrix's eigenvector is orthogonal? can you show some example? and the reason? $$\Sigma {\bf U_i} = \lambda_i U_i$$ additional ...
3
votes
1answer
5k views

Eigenvalue vs Variance

I recently read that eigenvalue indicates the variance for an attribute/dimension. But is there a relation/equation between eigenvalue and variance? Is is right to say eigenvalue is equal to variance (...
2
votes
1answer
514 views

Covariance Matrix Eigenvalue Distribution Relation to Size [closed]

I'm trying to run PCA on sample covariance matrices of various sizes (ranging between 20 x 20 to 4000 x 4000). Assume the data follows a joint multivariate normal distribution. While derivations are ...
4
votes
1answer
645 views

Eigenvalue equation for kernel PCA

In Nonlinear component analysis as a kernel eigenvalue problem, Schölkopf et al start by describing PCA. Given a set of data instances $x_1, \dots, x_M$, with $x_k \in \mathbb{R}^N, k=1,\dots,M$, and ...
2
votes
0answers
144 views

Projection on weighted kernel PCA basis

I'm performing a sort of weighted kernel PCA, where the weights of samples can be negative. The weights of all samples are given by the diagonal weight matrix $D$. The data matrix is the $n \times d$ ...
3
votes
1answer
66 views

Can I reduce the AICc penalty for multiple variables, when some variables should be grouped?

I have a data set of mean trait values for each of 18 populations, and want to test whether several ecological variables are related to variation in traits. I'm using the corrected Akaike information ...
2
votes
0answers
391 views

The effect of non-positive-definite covariance matrix (in $p>n$ case) on PCA

Gene data has large number of dimensions as compared to samples. This leads to a non-positive-definite covariance matrix. In R when I try to use princomp which does ...
2
votes
1answer
2k views

How to calculate the information loss of PCA?

How would we calculate the information loss of reducing dimensions using PCA ? Would it be the amount of variance loss if we skip certain eigenvectors after the PCA ?
0
votes
0answers
57 views

Principal Component Analysis PCA Terms and relationships: eigenvalues, eigenvectors, loadings, score matrix, and SVD [duplicate]

I've read many websites, blogs, pdfs on this top but struggle to put the picture together in simple math terms, that explains how some of the terms relate to each other / are computed. Let's assume ...
10
votes
1answer
4k views

What is principal subspace in probabilistic PCA?

if $X$ is observed data matrix and $Y$ is latent variable then $$X=WY+\mu+\epsilon$$ Where $\mu$ is the mean of observed data, and $\epsilon$ is the Gaussian error/noise in data, and $W$ is called ...
2
votes
0answers
294 views

Calculating eigenvectors for PCA and obtaining a different result from Matlab/EViews

Based on the variance-covariance matrix $$\begin{pmatrix}0.62 & 0.62\\ 0.62 & 0.72\end{pmatrix},$$ I have calculated the following eigenvalues $$λ_1 = 0.0500, \;λ_2 = 1.2840.$$ I then ...
3
votes
1answer
11k views

Does the sign of eigenvectors matter? [duplicate]

I had an matrix ((2,1,1),(-11,4,5),(-1,1-0)) I got the eigen values to be -1,1,2 for the eigenvalue -1 I got an eigenvector (0,1,-1) on the answers it says the answer is (0,-1,1). Is there an actual ...
4
votes
1answer
12k views

What's the relationship between initial eigenvalues and sums of squared loadings in factor analysis?

On the one hand I read in a comment here that: You can't speak of "eigenvalues" after rotation, even orthogonal rotation. Perhaps you mean sum of squared loadings for a principal component, ...

1
4
5
6 7 8