Questions tagged [eigenvalues]
For questions involving calculation or interpretation of eigenvalues or eigenvectors.
412
questions
3
votes
1
answer
33
views
+50
Why is the Scaling Matrix in LDA unnormalized?
I was carrying out LDA (linear Discriminant Analysis) and noticed that the Scaling matrix produced by R is not normalized. Here is an example:
...
- 436
0
votes
0
answers
25
views
Information contained in eigenvalues of a matrix [migrated]
Crossposted on Mathematics SE
Consider a full-rank square matrix $A$ with dimension $N \times N$. We have no further information about it. Let $C = A'A$, where $A'$ means the transposed of $A$. Now ...
- 46
3
votes
2
answers
170
views
Does the average eigenvalue equals 1 in PCA applied to standardised data?
From what I understood when we are doing PCA, we can work both with raw or standardised data, depending on the situation we're in. Is it true that the average of the eigenvalues is equal to 1 when we ...
- 103
1
vote
1
answer
24
views
Can I apply Kaiser Rule without knowing the eigenvalues?
Kaiser's rule suggests the number of principal components to be included in an analysis by looking at eigenvalues. If I'm given standard deviations only, instead of eigenvalues, can I still somehow ...
- 103
0
votes
0
answers
10
views
How to understand eigenvalue equation from inner product point of view? [migrated]
In equation (14.4) page 429 of scholkopf's Learning with Kernels book, it states that the eigenvalue equation $\lambda v = C v$ is equivalent to $\lambda \langle x_i, v \rangle = \langle x_i, C v \...
- 1
1
vote
0
answers
41
views
Are low-rank kernel approximations implementing implicit regularization?
Consider a kernel estimation problem as follows. We have functions $f^* \sim GP(0, C^*)$ drawn from a Gaussian process. We want to construct a kernel $K$ that does well in regressing functions drawn ...
- 173
2
votes
0
answers
17
views
Eigenvalues of a block matrix
Let $\bar{\lambda}$ be the smallest eigenvalue of $$M=\Omega^{-1/2}Y'Z(Z'Z)^{-1}Z'Y\Omega^{-1/2}$$, where $\Omega$, $Y$, and $Z$ are $(l\times l)$, $(N\times l)$, and $(N \times k)$ matrices, ...
- 45
1
vote
0
answers
48
views
is it fair to use a subset of eigenvalues to evaluate the multidimensional variance
I want to find a single metric to assess how spread (or how much variance) a multidimensional dataset (a large number of features) is. I learned that the determinant (or pseudo-determinant) of the ...
- 11
1
vote
0
answers
42
views
How to reconstruct Cartesian coordinates from a Gram matrix?
I read this article and am wondering why we can reconstruct Cartesian coordinates from a Gram matrix generated by taking dot product of the distance from the origin.
They had a Euclidean distance ...
- 11
0
votes
0
answers
22
views
Finding Eigen Vector [duplicate]
I followed this tutorial from this question. The dataset and example used in the mentioned tutorial seems famous one and it gives following eigen vectors:-
...
- 101
0
votes
0
answers
23
views
Why do we multiply the centered data with eigen vector instead of taking inverse of the eigen vector while performing PCA
The question may sound stupid but I really don't understand the logic behind this. Whenever we do a PCA, we take a covariance matrix on the centered data and do eigen decomposition. In order to ...
0
votes
0
answers
22
views
PCA for 3D Lidar data
I'm trying to work with Lidar data to classify road. I'm wondering how to use pca (or something else?) to identify if region has connectivity > threshold then label the region to be road? Would ...
- 101
0
votes
0
answers
24
views
Expectation of a function of the sample covariance matrix
Let $X \sim N(\mu, \Sigma)$ and let $\hat\Sigma$ be the sample covariance matrix calculated from an iid sample $X_1, \dots, X_n$. Also, let $\phi_i$ and $\phi_j$ be the orthonormal eigenvectors of $\...
0
votes
1
answer
36
views
Why do eigenvalues of $\mathbf\Phi^T\mathbf\Phi$ increase with the size of data set?
The question comes from a paragraph in page 171 of "Pattern Recognition and Machine Learning" by Christopher M. Bishop:
Here $\mathbf\Phi$ is the design matrix for a data set of $N$ samples ...
- 183
3
votes
2
answers
281
views
Why is $\mathbf\Phi^{\top}\mathbf\Phi$ a positive definite matrix?
I had this question when reading section 3.5.3 on page 170 of "Pattern Recognition and Machine Learning" written by Christopher M. Bishop:
Here $\mathbf\Phi$ represents the design matrix ...
- 183
0
votes
0
answers
31
views
Does invariance of PCA under orthogonal transformation hold for data that is not centered?
I read the proof in the top answer to this question, but that page assumes that $\overline{A} = 0$. If the data instead has some nonzero mean $\mu$, I'm not sure if the same logic applies: ...
0
votes
0
answers
11
views
What is the variable v in this case? Is it standard deviation or variance?
The code below is from an example on the scikit-learn website for plotting the Gaussians of a GMM as ellipses. However, I don't know whether the final definition of the variable ...
- 111
3
votes
0
answers
82
views
Relation between generalization bounds of Kernel Ridge Regression and largest eigenvalue of the kernel Gram matrix
Consider a positive-definite, symmetric function $k(x_1, x_2)$ which is used, given the dataset $\{(x_i, y_i)\}_{i=1}^m$, to construct the Gram matrix $K = [k(x_i, x_j)]_{i,j \in 1, ..., m}$.
What is ...
- 41
1
vote
1
answer
68
views
How to derive the three matrices of SVD from eigenvalue decomposition in Kernel PCA?
Kernel PCA is usually done via eigenvalue decomposition of the Kernel Matrix $\mathbf{K}$ and standard PCA via SVD of the input $\mathbf{X}$.
In standard PCA as far as I know we can derive $\mathbf{S}...
- 23
2
votes
1
answer
72
views
Confusion about principal component and major axis of the ellipse corresponding to the covariance matrix
Based on my understanding, in PCA, we try to find a linear combination of axes such that the variance in that direction is maximized. If variables have the covariance matrix $\Sigma$, then, the first ...
- 583
1
vote
0
answers
50
views
The convergence of random variables to standard normal distribution
Let $V_s$ be $n\times s$ real matrix and consisting i.i.d $\mathcal{N}(0,1)$ random variables [*]. Suppose that $O_s^1$ is the orthogonal matrix, its first column being the normalization of the first ...
0
votes
0
answers
47
views
Finding the vectors/modes "maximally responsible" for the fluctutation of a quantity?
I have a question which I fear might be simple but i am completely unable to figure out.
Lets say we have a time-series of a vector of coordinates which define a molecule ( column vector of x,y,z) ...
- 1
0
votes
0
answers
36
views
The relationship between eigenvalues of a covariance matrix and the variances of the same data matrix after using eigenvectors as bases
Suppose we have a data matrix $\mathbf{X}\in \mathbb{R}^{M\times N}$ with $M$ features, $N$ samples and zero means ($M \lt N$). The covariance matrix of $\mathbf{X}$ is $\mathbf{C_x}=\frac{1}{N}\...
- 1
0
votes
0
answers
25
views
Are "weights", "loading scores", and "singular values" all synonyms?
I'm currently learning to use "eigenfaces" for facial image classification. Unfortunately, I've encountered some confusion with the following lines of code:
...
- 641
1
vote
0
answers
85
views
Generate a random covariance matrix with specified eigenspectra and diagonal elements and first off-diagonal?
I want to generate a random covariance matrix ($c \in \mathcal{R}^{n \times n} $) whose eigenspectra, i.e., $n$ eigenvalues $e_0 \in \mathcal{R}^{n\times 1}$ and diagonal elements $c_{ii} \,\, i=1 \,\,...
- 31
1
vote
0
answers
34
views
Eigenvalues of Johansen Trace Test
I'm currently taking a course in time series and have been struggling with understanding the Johansen trace test. Specifically, the calculation of the eigenvalues for the Likelihood ratio statistic. ...
0
votes
0
answers
76
views
EFA: Eigenvalues or Loadings after extraction (SPSS)
I've been doing a EFA with ML extraction and Promax rotation, whereby three factors were extracted. For reporting the results, I was wondering whether to use the 'Initial Eigenvalues' or the '...
0
votes
0
answers
38
views
Is it possible to derive the expected sampled eigenvalues given a population correlation matrix?
My question is : Is it possible to derive the expected sampled eigenvalues given a population correlation matrix and a sample size $n$.
I am pondering about how behaves eigenvalues of correlation ...
- 493
0
votes
0
answers
57
views
Popular symmetric positive definite matrices with explicitly known eigenvalues (at least largest eigenvalue)
I am looking for popular symmetric positive definite matrices with explicitly known eigenvalues (at least largest eigenvalue) arising as autocovariance matrices in time series (for example). In fact, ...
- 1,515
4
votes
2
answers
260
views
Why is sum of squares equal to eigenvalue in PCA?
We fit a line or a hyperplane to a set of points. We project the points onto the hyperplane. The sum of squared distances of the projected points to origin is equal to the eigenvalue. Why is that?
- 255
1
vote
0
answers
269
views
Positive semi definite matrix with negative eigenvalues?
From what I know, for any square real matrix A, a matrix generated with the following should be a positive semidefinite (PSD) matrix:
...
- 43
1
vote
1
answer
71
views
Eigenvector centrality comparison
Let's say we calculate the eigenvector centrality of the same set of nodes in different years (we have a network each year). Note that the eigenvector centrality is normalized in such a way that the ...
- 591
1
vote
0
answers
27
views
Finding the eigenvalues of covariance matrix [duplicate]
Came across this as an interview question that I saw online: given a covariance matrix with diagonals being all 1 and the off-diagonals being c, what are its eigenvalues?
Going by the definition $Av = ...
- 311
0
votes
0
answers
114
views
Show that the autocovariance matrix is positive definite
I've been working through the textbook Time Series Analysis and its Applications (R. H. Shumway & D. S. Stoffer 2ed). The topic I'm looking at is forecasting using ARMA models. The below assumes ...
- 303
0
votes
0
answers
173
views
The discrepancy of results of PCA via Eigendecomposition vs via SVD in Python with scipy.linalg [duplicate]
I recently learned about different methods of PCA. I decided to manually implement PCA in Python with Eigendecomposition of cov(X) and the Singular Value ...
- 175
2
votes
0
answers
44
views
Is a bivariate copula relevant in this physics setting manifesting uniform univariate marginals--and, if so, how can it be constructed?
To quickly place our probabilistic (copula) question in its subject matter setting, we note that a fundamental concept in quantum theory is that of entanglement QuantumEntanglement.
The states of ...
- 161
0
votes
0
answers
438
views
All eigenvalues less than 1 in factor analysis
Is it possible to have all eigenvalues less than 1 during factor analysis? then what can be concluded here? since a factor is considered a factor when the eigenvalue of the factor is greater than 1. ...
- 1
1
vote
0
answers
97
views
How do I get the stationary distribution of a Markov chain matrix from SVD?
I have a matrix that represents a Markov chain.
...
- 1,337
0
votes
0
answers
66
views
Eigenvalue decomposition
Why is the eigenvalue decomposition of the covariance matrix formula, $UΛU^T$ (assuming T means transpose) and not $UΛU^{-1}$? Where $U$ is eigenvectors and $Λ$ is a diagonal matrix consisting of ...
2
votes
1
answer
227
views
Find first Principal Component (and loading) using a fast iterative algorithm without covariance matrix
I have a matrix $X$ and I would like to find its first principal component and the corresponding loadings. I would like to do this without computing the covariance matrix of $X$. How can I do so?
This ...
- 529
3
votes
2
answers
218
views
Inequality on norm in terms of eigenvalue
I came across this inequality but not sure if it's true:
$|\lambda_{\text{min}}|^2\|\hat{\beta}-\beta\|^2_2\leq (\hat{\beta}-\beta)'\hat{\Sigma}(\hat{\beta}-\beta)$
where $\lambda_{\text{min}}$ is the ...
- 503
1
vote
0
answers
148
views
Eigenvalues of time lagged covariance matrix - should be always real and positive?
Consider two random variables $X_t$ and $X_{t+\tau}$; where both come from random variable $X$ but are lagged by $\tau$. Assume that both are mean-free. If we want calculate covariance matrix of them, ...
- 547
2
votes
0
answers
540
views
Negative Eigenvalues in EFA
Assuming a set of data meet all assumptions for EFA and we are doing a factor analysis (with the SMC used to define the shared variance), how do negative eigenvalues appear and what does that say ...
- 395
1
vote
1
answer
96
views
Why is the first canonical direction equal to the left singular vector i.e. why is $w_1 = a = u_1$ in CCA (Canonical Correlation Analysis)?
I want to understand why the canonical direction $a$ is equal to the left singular value of $M = \Sigma^{-1/2}_X \Sigma_{X, Y} \Sigma^{-1/2}_Y$ and not $a = \sigma_1 u_1$.
My calculation tell me that ...
- 6,336
1
vote
0
answers
45
views
Relation between PCA eigen values and data visualization
We have matrix data $X$ which is $n\times d$. We use the covariance matrix/ design matrix/ gram matrix $X^T X$ to perform least-squares/ PCA. I compute the eigen basis representation of said matrix
$$...
- 181
2
votes
1
answer
67
views
number and size of eigenvectors in PCA
As I understand, the size of eigenvector produced in PCA should be min{n,N}, where N=number of samples and n=dimension of each sample (Right?). However, I have seen in couple of cases that this size ...
- 43
1
vote
1
answer
427
views
Making sure that the design matrix is positive (semi-) definite
In Bayesian linear regression, how do I make sure that the design matrix produced by a neural network $ \Phi$ is positive definite?
Computing the covariance matrix on the weight requires inverting --- ...
- 11
8
votes
3
answers
924
views
Get accurate eigenvectors, when eigenvalues are minuscule
I have a symmetric matrix A. I'm not able to compute all the eigenvectors accurately, and I believe it is due to the last few eigenvalues for ...
- 117
1
vote
0
answers
292
views
What are the metrics to assess the quality of a multiple correspondence analysis (MCA) model?
We are trying to implement a multiple correspondence analysis (MCA) model. I was looking for metrics to assess the quality of an MCA to evaluate our model. Sadly, I didn’t find much literature about ...
- 23
1
vote
0
answers
78
views
How to interpret a PCA Biplot? [duplicate]
I constructed a PCA plot from a very high-dimensional dataset that contains features relating to fraud. After creating the PCA plot, I created a biplot with the features to see how they interact. The ...
- 153