Questions tagged [eigenvalues]

For questions involving calculation or interpretation of eigenvalues or eigenvectors.

Filter by
Sorted by
Tagged with
1
vote
1answer
27 views

Is the covariance matrix almost always postive definite?

I understand a covariance matrix is always positive semi-definite, but it seems that the covariance matrix would almost always be positive definite (although theoretically is only guaranteed to be ...
0
votes
0answers
20 views

Interpreting SVD on non-centered matrix

I have a very large, very sparse matrix $A \in \mathbb{N}^{n \times m}$ I'd like to perform SVD on. It is non-centered. When I center it to $A'$, I can't even fit it in memory (because $A'$ is in $\...
0
votes
0answers
13 views

Variability of K SVD components [duplicate]

Let's say I have a SVD of a matrix $A = U \Sigma V^T$, $A \in \mathbb{R}^{n \times m}$, and I'm using top-k components corresponding to $\sigma_1, ..\sigma_k$, the k largest values on the diagonal of $...
1
vote
0answers
31 views

Angle between PCA vector spaces?

I have two datasets of the same shape, one for condition A, the other for condition B. I would like to test if the major axes of variance of condition A are different than those of B. Here is my idea. ...
1
vote
0answers
36 views

Understanding the output from the Johansen Cointegration test in R

I have a VECM model that Im using to determine the revenues for a firm, based on factors like Interest rates, S&P 500 and company specific variables, as follows: Stage 1: $$z_t= a+ bX_t+e_t$$ ...
2
votes
1answer
35 views

What is this projection matrix doing?

Let’s say we have a $m\times d$ zero mean multivariate Gaussian matrix $X$. Its covariance matrix is $X^{T}X$. Let $V$ be the $d\times d$ matrix of eigenvectors of $X^{T}X$, with the columns sorted in ...
0
votes
0answers
12 views

Two different approaches to the linear discriminant analysis(LDA)

I have seen two different approaches to the explanation of the Linear Discriminant Analysis. The following is the description of the rough understanding of the approaches: 1) The first one refers to ...
1
vote
0answers
13 views

eigenstructure matching optimization

Is there any optimization loss functions that can approximately match the eigenstructure of the original samples and the transformed samples? For example, given a collection of samples $\mathbf{X}$ ...
0
votes
0answers
7 views

Covariance matrix distance variability

I have been preparing an analysis of a data set from which I have extracted a number of covariance matrices and used the Log-Euclidean algorithms to calculate distances, geodesics etc. This data set ...
1
vote
0answers
19 views

Why covariance matrix estimated pairwise is not guaranteed to be PSD?

As mentioned here Is a covariance matrix composed of matrixes derived from separate samples guaranteed to be positive definitive? and here Must a matrix of sample pairwise covariances be PSD? if you ...
0
votes
1answer
30 views

What can you say about spread of data by looking at singular values and clusters?

I have dataset 250X5 and its singular values are [200 50 25.2 2.3 0.35]. Singular values are directly related to variance. Can you say something about the clustering of data and how much is the spread ...
0
votes
0answers
8 views

Investigate if 1st principal component can discriminate between two classes

Consider the following cases $X|Y$ = 1 ~ $N_d(0,\sigma^2I_d)$ $X|Y$ = -1 ~ $N_d(0,\tau^2I_d)$ Derive an expression for the eigenvector $v$ corresponding to the largest eigenvalue of $\Sigma = E[XX^...
1
vote
0answers
10 views

First component of non-centered data

Let $X$ be a random vector of dimension $p$ and $\{ X_1, \dots, X_n \}$ the $n$ observations of such vector. Let $\mathbb{X}$ be the matrix with rows $X_k$ and $\mathbb{X}_c$ is the matrix with rows $...
0
votes
0answers
26 views

Sketch ellipse with variance-covariance matrix that got after PCA

For X = (X1, X2, X3) distributed as N3(µ, Σ), mean of the original data is mu and variance-covarinace matrix of the original data is Sigma. I found in this section that we can derive the variance-...
7
votes
1answer
137 views

Why eigenvectors reveals the groups in Spectral Clustering

According to Handbook of Cluster Analysis Spectral Clustering is done with following algorithm: Input Similarity Matrix $S$, number of clusters $K$ Form the transition matrix $P$ with $P_{ij} = S_{...
0
votes
0answers
8 views

Interpretation of Johanson co-integration test results

if the Max-eigenvalue test indicates no cointegration at the 0.05 level, while the Trace test indicates 2 cointegrating eqn(s) at the 0.05 level, how do i interpret the results?
1
vote
0answers
28 views

Almost sure convergence of the minimum eigenvalue of a sample covariance matrix

I was wondering if someone could provide a reference to the following result. Consider the $p\times p $ sample matrix $$\frac{1}{n} \sum_{i=1}^n x_i x_i',$$ where $x_i$ are i.i.d. $p\times 1$ random ...
1
vote
1answer
31 views

Get the eigenvalues when you know the explained variances of a PCA plot

I'm performing a PCA using the sklearn.decomposition.pca function. It appears to work as it should. Acording to this question, I can get the eigenvalues like this:...
0
votes
0answers
41 views

Scaling by Eigenvalues in PCA and Probabilistic PCA

I have a question about probabilistic PCA (PPCA) and regular PCA, particularly regarding transforming to and from the latent space. The main question (detailed in the following) is: when are the ...
3
votes
1answer
24 views

Dimensions Of The Covariance Matrix

I know that PCA can be obtained by eigendecomposition of the covariance matrix, and the covariance matrix $S$ is obtained by the equation: $S = X^TX $, where $X$ is the centered data matrix. But I ...
0
votes
0answers
14 views

Second order moment of multivariate Gaussian-completeness of the set of eigenvectors (bishop p. 83)

I don't know how 2.60 euqation is derived. Even if I have read the contents in previous pages, it is still hard for me to get an idea of how the 2.60 euqation is calculated. Also, I don't know the ...
2
votes
1answer
107 views

Projecting new samples onto PCA space is failing

After performing PCA I would like to project any new samples to the principal component space (I would like to see how samples cluster together). I did the PCA analysis in R: ...
0
votes
0answers
11 views

Intuitions behind Singular Value Decomposing generalizing eigen decomposition [duplicate]

Let M be mxn matrix then SVD of M will be UXW^* (sorry for X, assume summation). Then how does it generalizes eigen decomposition ? Since eigen decomposition is possible for nxn matrix and that are ...
2
votes
1answer
51 views

Decomposing Gradient Decent Error in Eigenvector Space

I'm going through Why Momentum Really Works and am unable to understand the following line in the article. "By writing the contributions of each eigenspace’s error to the loss $$f(w^{k})-f(w^{\star})=...
2
votes
1answer
48 views

Why do we compute eigenpairs of $X^TX$ instead of $X$ in Principal Component Analysis?

What is the intuition behind finding the eigen-vectors and eigen-values of $X^TX$ or $XX^T$ instead of the matrix $X$? One reason that I see immediately is when the matrix $X$ is non-square. I am ...
0
votes
0answers
14 views

Jointly Gaussian RVs: Relation between the slope of the major axis and Pearson correlation coefficient

Given a covariance matrix $\Sigma$ and their eigenvectors $V$ and eigenvalues $e$. $\rho$ is Pearson correlation coefficient. $$ \begin{array}{l} \Sigma = \left[ {\begin{array}{*{20}{c}} {.25}&{....
0
votes
1answer
29 views

PCA, highly explained component with no value above 0.3

I was analyzing PCA output, and when I reached "pca $xlist, comp($ncomp) blanks(.3)" to name variables, I found out that component 1. which explains more than 50% didn't have any variable above 3, yet ...
0
votes
0answers
13 views

How to quantify the similarity of EOF loading in multiple matrices?

I have five 3-D matrices (time, latitude, and longitude) representing the same variable but from different sources, denoted as A, B, C, D, and E. I calculated the first five EOF loadings for each of ...
0
votes
0answers
114 views

Derive Confidence Intervals for Eigenvalues of a Covariance Matrix

I am working with a dataset of n = 273 observations, p = 9 variables for which I have generated principal components. The task I am faced with is: Assume the eigenvalues of a covariance matrix ${cov(...
0
votes
1answer
114 views

Upper bound trace of inverse of covariance matrix

Let C be the covariance matrix from any normal distribution. If the trace of C is upper-bounded by a constant k (i.e., tr(C)<=k), can I find an upper bound for the trace of the inverse of C (i.e., ...
0
votes
0answers
10 views

What is the correct explanation for the definition of Eigen vectors of covariance matrix, Principal components and Eigenfaces?

We have an input matrix X consisting of n images. We need to do PCA on this matrix. We compute covariance matrix of X, and find the Eigen values. The Eigen vector corresponding to highest Eigen value ...
0
votes
0answers
29 views

Relation between eigenvalues of original and transformed matrices

Let the matrix $X$ be some data arranged in rows. Consider the following eigenvalue decomposition $X^\top X = Q \Theta Q^\top=\sum_{i=1}^n \theta_iq_iq_i^\top$ where $q_i$ are the eigenvectors and $\...
1
vote
0answers
14 views

Matrix GLM estimation Example Explanation

**Below is my question: It is an example from the book " A Primer on linear modes" by Monahan** I have 3 questions Question 1: is how did he calculated the matrix λ Question 2: Why $λ $ ...
1
vote
0answers
42 views

Getting sense from loadings plot of scaled eigenvectors

I wanted to confirm my intution about the meaning of "loadings" I have made out of eigenvalue/eigenvector decomposition, but I still fail to do that. Note that I made 3 pairs of highly correlated ...
0
votes
1answer
63 views

Scatter Plot - Basics [closed]

I am stuck in understanding a basic scatter plot. I am working in two dimensions i.e. there are two variables X & Y. So, the question is that in the scatter plot, what do the two axes mean? ...
1
vote
1answer
38 views

How can I adjust for negative eigenvalues?

I wish to run a path analysis from a pooled correlation matrix that I have imputed using the maximum-likelihood procedure. There was considerable missing data. The resulting correlation matrix is: <...
0
votes
0answers
136 views

Using NbClust on datasets that produce some negative eigenvalues. When to exclude data, when to force to positive, when to exclude test index?

Background on why I am using clustering: I am analyzing data from a multistep biological experiment, where each step is done in batches of varying sizes. I want to account for any biases that might ...
0
votes
0answers
67 views

Why are the eigenvalues of $X'X$ equal to that of $XX'$ when $X$ is a design matrix? [duplicate]

The title says it all. If $X$ is a design matrix (columns containing variables, rows containing observations), I have observed that eigs($X'X$)=eigs($XX'$). I actually found this by accident when I ...
1
vote
2answers
93 views

How to apply Principal components (Eigenvectors) in PCA?

I struggle to understand how to get further in my PCA analysis after computing the eigenvectors of the covariance matrix. I choose the number of principal components which contain 95% of the ...
1
vote
1answer
391 views

Is the first principal component is the one with the largest eigenvalue and how to convert it to explained variance?

In PCA, after we calculate the eigenvalues of each variable, we need to get the explained variance, I read an article which suggests: ...
0
votes
0answers
42 views

Procedure to quickly find the near zero eigenvalues (and corresponding eigenvectors) of a positive semidefinite square matrix?

I have an ill-conditioned positive semidefinite $n \times n$ square matrix (Hessian), with unknown rank $r$, that I need to compute the inverse for. I'd prefer not to compute the SVD for performance ...
0
votes
0answers
30 views

eigen value decomposition of co-variance a series generated by factor model

Let's assume $N\times T$ series $Y_t$ is generated by the following equation. $$ Y_t = \begin{bmatrix}A_x & A_m\end{bmatrix}\begin{bmatrix}x_t \\ m_t \end{bmatrix}$$ Where $A_x$ and $A_m$ are $N\...
0
votes
0answers
9 views

How to measure changes in condition indices over time

I am trying to understand how adding data, one observation at a time, affects the condition indices of a model. A similar question is how adding individual observations affects the principal ...
0
votes
1answer
25 views

Does the first principle component maximize variance on the two variables with greatest covariance or on all variables simultaneously?

Suppose I am performing PCA on 3 standardized variables: height, weight, and income. I understand that each principle component maximizes variance along a new line, but there are two ways I can see ...
0
votes
0answers
75 views

How to show that VIF's are the main diagonal elements of $\mathbf{T\Lambda^{-1}T'}$?

I'm stucked in the Exercise 9.29 of Introduction to Linear Regression Analysis (5th edition), by Montgomery: 9.29) Show that if $\mathbf{X'X}$ is in correlation form, $\mathbf{\Lambda}$ is the ...
0
votes
0answers
15 views

Parallel analysis for principle components analysis and Multi-dimensional scaling

Does the same method for conducting a parallel analysis for principal component analyses apply to find the cut-off point for multidimensional scaling? Under the pretense that PCA and linear MDS are ...
0
votes
1answer
95 views

Interpretation of Eigenvalue vs. Singular Value plot

I'm doing some preliminary analysis on the feature matrix for a certain dataset (rows are observations, columns are feature dimensions). I have computed the SVD and PCA decompositions for this matrix ...
0
votes
0answers
29 views

Eigenvectors - principle lines of force

In the article about PCA and coavariance matrix I've read the following: Finding the eigenvectors and eigenvalues of the covariance matrix is the equivalent of fitting those straight, principal-...
2
votes
1answer
19 views

Determine missing eigenvalues given only correlations between variables and components

Given that I just have a correlation matrix ($X$ Variables vs. $Y$ Principal Components), and that I am trying to find 2 missing eigenvalues (e.g., missing $\lambda_1$ and $\lambda_5$) from the total $...
0
votes
1answer
57 views

Gradient descent derivation in Eigenspace [duplicate]

I am trying to decode article on https://distill.pub/2017/momentum/ I was able to follow everything until the part with a change of basis x$^k=Q^T(w^k−w^⋆)$ to eigenspace... I conceptually understand ...

1
2 3 4 5
7