Questions tagged [eigenvalues]
For questions involving calculation or interpretation of eigenvalues or eigenvectors.
315
questions
1
vote
1answer
27 views
Is the covariance matrix almost always postive definite?
I understand a covariance matrix is always positive semi-definite, but it seems that the covariance matrix would almost always be positive definite (although theoretically is only guaranteed to be ...
0
votes
0answers
20 views
Interpreting SVD on non-centered matrix
I have a very large, very sparse matrix $A \in \mathbb{N}^{n \times m}$ I'd like to perform SVD on. It is non-centered. When I center it to $A'$, I can't even fit it in memory (because $A'$ is in $\...
0
votes
0answers
13 views
Variability of K SVD components [duplicate]
Let's say I have a SVD of a matrix $A = U \Sigma V^T$, $A \in \mathbb{R}^{n \times m}$, and I'm using top-k components corresponding to $\sigma_1, ..\sigma_k$, the k largest values on the diagonal of $...
1
vote
0answers
31 views
Angle between PCA vector spaces?
I have two datasets of the same shape, one for condition A, the other for condition B. I would like to test if the major axes of variance of condition A are different than those of B. Here is my idea. ...
1
vote
0answers
36 views
Understanding the output from the Johansen Cointegration test in R
I have a VECM model that Im using to determine the revenues for a firm, based on factors like Interest rates, S&P 500 and company specific variables, as follows:
Stage 1:
$$z_t= a+ bX_t+e_t$$
...
2
votes
1answer
35 views
What is this projection matrix doing?
Let’s say we have a $m\times d$ zero mean multivariate Gaussian matrix $X$. Its covariance matrix is $X^{T}X$. Let $V$ be the $d\times d$ matrix of eigenvectors of $X^{T}X$, with the columns sorted in ...
0
votes
0answers
12 views
Two different approaches to the linear discriminant analysis(LDA)
I have seen two different approaches to the explanation of the Linear Discriminant Analysis. The following is the description of the rough understanding of the approaches:
1) The first one refers to ...
1
vote
0answers
13 views
eigenstructure matching optimization
Is there any optimization loss functions that can approximately match the eigenstructure of the original samples and the transformed samples?
For example, given a collection of samples $\mathbf{X}$ ...
0
votes
0answers
7 views
Covariance matrix distance variability
I have been preparing an analysis of a data set from which I have extracted a number of covariance matrices and used the Log-Euclidean algorithms to calculate distances, geodesics etc. This data set ...
1
vote
0answers
19 views
Why covariance matrix estimated pairwise is not guaranteed to be PSD?
As mentioned here
Is a covariance matrix composed of matrixes derived from separate samples guaranteed to be positive definitive?
and here Must a matrix of sample pairwise covariances be PSD?
if you ...
0
votes
1answer
30 views
What can you say about spread of data by looking at singular values and clusters?
I have dataset 250X5 and its singular values are [200 50 25.2 2.3 0.35].
Singular values are directly related to variance. Can you say something about the clustering of data and how much is the spread ...
0
votes
0answers
8 views
Investigate if 1st principal component can discriminate between two classes
Consider the following cases
$X|Y$ = 1 ~ $N_d(0,\sigma^2I_d)$
$X|Y$ = -1 ~ $N_d(0,\tau^2I_d)$
Derive an expression for the eigenvector $v$ corresponding to the largest eigenvalue of $\Sigma = E[XX^...
1
vote
0answers
10 views
First component of non-centered data
Let $X$ be a random vector of dimension $p$ and $\{ X_1, \dots, X_n \}$ the $n$ observations of such vector. Let $\mathbb{X}$ be the matrix with rows $X_k$ and $\mathbb{X}_c$ is the matrix with rows $...
0
votes
0answers
26 views
Sketch ellipse with variance-covariance matrix that got after PCA
For X = (X1, X2, X3) distributed as N3(µ, Σ),
mean of the original data is mu and variance-covarinace matrix of the original data is Sigma.
I found in this section that we can derive the variance-...
7
votes
1answer
137 views
Why eigenvectors reveals the groups in Spectral Clustering
According to Handbook of Cluster Analysis Spectral Clustering is done with following algorithm:
Input Similarity Matrix $S$, number of clusters $K$
Form the transition matrix $P$ with $P_{ij} = S_{...
0
votes
0answers
8 views
Interpretation of Johanson co-integration test results
if the Max-eigenvalue test indicates no cointegration at the 0.05 level, while the Trace test indicates 2 cointegrating eqn(s) at the 0.05 level, how do i interpret the results?
1
vote
0answers
28 views
Almost sure convergence of the minimum eigenvalue of a sample covariance matrix
I was wondering if someone could provide a reference to the following result. Consider the $p\times p $ sample matrix
$$\frac{1}{n} \sum_{i=1}^n x_i x_i',$$
where $x_i$ are i.i.d. $p\times 1$ random ...
1
vote
1answer
31 views
Get the eigenvalues when you know the explained variances of a PCA plot
I'm performing a PCA using the sklearn.decomposition.pca function.
It appears to work as it should. Acording to this question, I can get the eigenvalues like this:...
0
votes
0answers
41 views
Scaling by Eigenvalues in PCA and Probabilistic PCA
I have a question about probabilistic PCA (PPCA) and regular PCA, particularly regarding transforming to and from the latent space. The main question (detailed in the following) is: when are the ...
3
votes
1answer
24 views
Dimensions Of The Covariance Matrix
I know that PCA can be obtained by eigendecomposition of the covariance matrix, and the covariance matrix $S$ is obtained by the equation: $S = X^TX $, where $X$ is the centered data matrix.
But I ...
0
votes
0answers
14 views
Second order moment of multivariate Gaussian-completeness of the set of eigenvectors (bishop p. 83)
I don't know how 2.60 euqation is derived. Even if I have read the contents in previous pages, it is still hard for me to get an idea of how the 2.60 euqation is calculated. Also, I don't know the ...
2
votes
1answer
107 views
Projecting new samples onto PCA space is failing
After performing PCA I would like to project any new samples to the principal component space (I would like to see how samples cluster together). I did the PCA analysis in R:
...
0
votes
0answers
11 views
Intuitions behind Singular Value Decomposing generalizing eigen decomposition [duplicate]
Let M be mxn matrix then SVD of M will be UXW^* (sorry for X, assume summation). Then how does it generalizes eigen decomposition ? Since eigen decomposition is possible for nxn matrix and that are ...
2
votes
1answer
51 views
Decomposing Gradient Decent Error in Eigenvector Space
I'm going through Why Momentum Really Works and am unable to understand the following line in the article.
"By writing the contributions of each eigenspace’s error to the loss
$$f(w^{k})-f(w^{\star})=...
2
votes
1answer
48 views
Why do we compute eigenpairs of $X^TX$ instead of $X$ in Principal Component Analysis?
What is the intuition behind finding the eigen-vectors and eigen-values of $X^TX$ or $XX^T$ instead of the matrix $X$? One reason that I see immediately is when the matrix $X$ is non-square. I am ...
0
votes
0answers
14 views
Jointly Gaussian RVs: Relation between the slope of the major axis and Pearson correlation coefficient
Given a covariance matrix $\Sigma$ and their eigenvectors $V$ and eigenvalues $e$. $\rho$ is Pearson correlation coefficient.
$$
\begin{array}{l}
\Sigma = \left[ {\begin{array}{*{20}{c}}
{.25}&{....
0
votes
1answer
29 views
PCA, highly explained component with no value above 0.3
I was analyzing PCA output, and when I reached "pca $xlist, comp($ncomp) blanks(.3)" to name variables, I found out that component 1. which explains more than 50% didn't have any variable above 3, yet ...
0
votes
0answers
13 views
How to quantify the similarity of EOF loading in multiple matrices?
I have five 3-D matrices (time, latitude, and longitude) representing the same variable but from different sources, denoted as A, B, C, D, and E. I calculated the first five EOF loadings for each of ...
0
votes
0answers
114 views
Derive Confidence Intervals for Eigenvalues of a Covariance Matrix
I am working with a dataset of n = 273 observations, p = 9 variables for which I have generated principal components. The task I am faced with is:
Assume the eigenvalues of a covariance matrix ${cov(...
0
votes
1answer
114 views
Upper bound trace of inverse of covariance matrix
Let C be the covariance matrix from any normal distribution. If the trace of C is upper-bounded by a constant k (i.e., tr(C)<=k), can I find an upper bound for the trace of the inverse of C (i.e., ...
0
votes
0answers
10 views
What is the correct explanation for the definition of Eigen vectors of covariance matrix, Principal components and Eigenfaces?
We have an input matrix X consisting of n images. We need to do PCA on this matrix.
We compute covariance matrix of X, and find the Eigen values. The Eigen vector corresponding to highest Eigen value ...
0
votes
0answers
29 views
Relation between eigenvalues of original and transformed matrices
Let the matrix $X$ be some data arranged in rows. Consider the following eigenvalue decomposition $X^\top X = Q \Theta Q^\top=\sum_{i=1}^n \theta_iq_iq_i^\top$ where $q_i$ are the eigenvectors and $\...
1
vote
0answers
14 views
Matrix GLM estimation Example Explanation
**Below is my question:
It is an example from the book " A Primer on linear modes" by Monahan**
I have 3 questions
Question 1:
is how did he calculated the matrix λ
Question 2:
Why $λ $ ...
1
vote
0answers
42 views
Getting sense from loadings plot of scaled eigenvectors
I wanted to confirm my intution about the meaning of "loadings" I have made out of eigenvalue/eigenvector decomposition, but I still fail to do that.
Note that I made 3 pairs of highly correlated ...
0
votes
1answer
63 views
Scatter Plot - Basics [closed]
I am stuck in understanding a basic scatter plot. I am working in two dimensions i.e. there are two variables X & Y.
So, the question is that in the scatter plot, what do the two axes mean?
...
1
vote
1answer
38 views
How can I adjust for negative eigenvalues?
I wish to run a path analysis from a pooled correlation matrix that I have imputed using the maximum-likelihood procedure. There was considerable missing data. The resulting correlation matrix is:
<...
0
votes
0answers
136 views
Using NbClust on datasets that produce some negative eigenvalues. When to exclude data, when to force to positive, when to exclude test index?
Background on why I am using clustering:
I am analyzing data from a multistep biological experiment, where each step is done in batches of varying sizes. I want to account for any biases that might ...
0
votes
0answers
67 views
Why are the eigenvalues of $X'X$ equal to that of $XX'$ when $X$ is a design matrix? [duplicate]
The title says it all. If $X$ is a design matrix (columns containing variables, rows containing observations), I have observed that eigs($X'X$)=eigs($XX'$). I actually found this by accident when I ...
1
vote
2answers
93 views
How to apply Principal components (Eigenvectors) in PCA?
I struggle to understand how to get further in my PCA analysis after computing the eigenvectors of the covariance matrix.
I choose the number of principal components which contain 95% of the ...
1
vote
1answer
391 views
Is the first principal component is the one with the largest eigenvalue and how to convert it to explained variance?
In PCA, after we calculate the eigenvalues of each variable, we need to get the explained variance, I read an article which suggests:
...
0
votes
0answers
42 views
Procedure to quickly find the near zero eigenvalues (and corresponding eigenvectors) of a positive semidefinite square matrix?
I have an ill-conditioned positive semidefinite $n \times n$ square matrix (Hessian), with unknown rank $r$, that I need to compute the inverse for. I'd prefer not to compute the SVD for performance ...
0
votes
0answers
30 views
eigen value decomposition of co-variance a series generated by factor model
Let's assume $N\times T$ series $Y_t$ is generated by the following equation.
$$ Y_t = \begin{bmatrix}A_x & A_m\end{bmatrix}\begin{bmatrix}x_t \\ m_t \end{bmatrix}$$
Where $A_x$ and $A_m$ are $N\...
0
votes
0answers
9 views
How to measure changes in condition indices over time
I am trying to understand how adding data, one observation at a time, affects the condition indices of a model. A similar question is how adding individual observations affects the principal ...
0
votes
1answer
25 views
Does the first principle component maximize variance on the two variables with greatest covariance or on all variables simultaneously?
Suppose I am performing PCA on 3 standardized variables: height, weight, and income. I understand that each principle component maximizes variance along a new line, but there are two ways I can see ...
0
votes
0answers
75 views
How to show that VIF's are the main diagonal elements of $\mathbf{T\Lambda^{-1}T'}$?
I'm stucked in the Exercise 9.29 of Introduction to Linear Regression Analysis (5th edition), by Montgomery:
9.29) Show that if $\mathbf{X'X}$ is in correlation form, $\mathbf{\Lambda}$ is the ...
0
votes
0answers
15 views
Parallel analysis for principle components analysis and Multi-dimensional scaling
Does the same method for conducting a parallel analysis for principal component analyses apply to find the cut-off point for multidimensional scaling?
Under the pretense that PCA and linear MDS are ...
0
votes
1answer
95 views
Interpretation of Eigenvalue vs. Singular Value plot
I'm doing some preliminary analysis on the feature matrix for a certain dataset (rows are observations, columns are feature dimensions).
I have computed the SVD and PCA decompositions for this matrix ...
0
votes
0answers
29 views
Eigenvectors - principle lines of force
In the article about PCA and coavariance matrix I've read the following:
Finding the eigenvectors and eigenvalues of the covariance matrix is the equivalent of fitting those straight, principal-...
2
votes
1answer
19 views
Determine missing eigenvalues given only correlations between variables and components
Given that I just have a correlation matrix ($X$ Variables vs. $Y$ Principal Components), and that I am trying to find 2 missing eigenvalues (e.g., missing $\lambda_1$ and $\lambda_5$) from the total $...
0
votes
1answer
57 views
Gradient descent derivation in Eigenspace [duplicate]
I am trying to decode article on https://distill.pub/2017/momentum/
I was able to follow everything until the part with a change of basis x$^k=Q^T(w^k−w^⋆)$ to eigenspace... I conceptually understand ...
