# Questions tagged [eigenvalues]

For questions involving calculation or interpretation of eigenvalues or eigenvectors.

412 questions
Filter by
Sorted by
Tagged with
33 views
+50

### Why is the Scaling Matrix in LDA unnormalized?

I was carrying out LDA (linear Discriminant Analysis) and noticed that the Scaling matrix produced by R is not normalized. Here is an example: ...
• 436
25 views

### Information contained in eigenvalues of a matrix [migrated]

Crossposted on Mathematics SE Consider a full-rank square matrix $A$ with dimension $N \times N$. We have no further information about it. Let $C = A'A$, where $A'$ means the transposed of $A$. Now ...
• 46
170 views

### Does the average eigenvalue equals 1 in PCA applied to standardised data?

From what I understood when we are doing PCA, we can work both with raw or standardised data, depending on the situation we're in. Is it true that the average of the eigenvalues is equal to 1 when we ...
• 103
1 vote
24 views

### Can I apply Kaiser Rule without knowing the eigenvalues?

Kaiser's rule suggests the number of principal components to be included in an analysis by looking at eigenvalues. If I'm given standard deviations only, instead of eigenvalues, can I still somehow ...
• 103
10 views

36 views

### Why do eigenvalues of $\mathbf\Phi^T\mathbf\Phi$ increase with the size of data set?

The question comes from a paragraph in page 171 of "Pattern Recognition and Machine Learning" by Christopher M. Bishop: Here $\mathbf\Phi$ is the design matrix for a data set of $N$ samples ...
• 183
281 views

### Why is $\mathbf\Phi^{\top}\mathbf\Phi$ a positive definite matrix?

I had this question when reading section 3.5.3 on page 170 of "Pattern Recognition and Machine Learning" written by Christopher M. Bishop: Here $\mathbf\Phi$ represents the design matrix ...
• 183
31 views

### Does invariance of PCA under orthogonal transformation hold for data that is not centered?

I read the proof in the top answer to this question, but that page assumes that $\overline{A} = 0$. If the data instead has some nonzero mean $\mu$, I'm not sure if the same logic applies: ...
11 views

### What is the variable v in this case? Is it standard deviation or variance?

The code below is from an example on the scikit-learn website for plotting the Gaussians of a GMM as ellipses. However, I don't know whether the final definition of the variable ...
• 111
82 views

### Relation between generalization bounds of Kernel Ridge Regression and largest eigenvalue of the kernel Gram matrix

Consider a positive-definite, symmetric function $k(x_1, x_2)$ which is used, given the dataset $\{(x_i, y_i)\}_{i=1}^m$, to construct the Gram matrix $K = [k(x_i, x_j)]_{i,j \in 1, ..., m}$. What is ...
• 41
1 vote
68 views

Kernel PCA is usually done via eigenvalue decomposition of the Kernel Matrix $\mathbf{K}$ and standard PCA via SVD of the input $\mathbf{X}$. In standard PCA as far as I know we can derive $\mathbf{S}... • 23 2 votes 1 answer 72 views ### Confusion about principal component and major axis of the ellipse corresponding to the covariance matrix Based on my understanding, in PCA, we try to find a linear combination of axes such that the variance in that direction is maximized. If variables have the covariance matrix$\Sigma$, then, the first ... • 583 1 vote 0 answers 50 views ### The convergence of random variables to standard normal distribution Let$V_s$be$n\times s$real matrix and consisting i.i.d$\mathcal{N}(0,1)$random variables [*]. Suppose that$O_s^1$is the orthogonal matrix, its first column being the normalization of the first ... 0 votes 0 answers 47 views ### Finding the vectors/modes "maximally responsible" for the fluctutation of a quantity? I have a question which I fear might be simple but i am completely unable to figure out. Lets say we have a time-series of a vector of coordinates which define a molecule ( column vector of x,y,z) ... • 1 0 votes 0 answers 36 views ### The relationship between eigenvalues of a covariance matrix and the variances of the same data matrix after using eigenvectors as bases Suppose we have a data matrix$\mathbf{X}\in \mathbb{R}^{M\times N}$with$M$features,$N$samples and zero means ($M \lt N$). The covariance matrix of$\mathbf{X}$is$\mathbf{C_x}=\frac{1}{N}\...
25 views

I'm currently learning to use "eigenfaces" for facial image classification. Unfortunately, I've encountered some confusion with the following lines of code: ...
• 641
1 vote
85 views

• 311
114 views

### Show that the autocovariance matrix is positive definite

I've been working through the textbook Time Series Analysis and its Applications (R. H. Shumway & D. S. Stoffer 2ed). The topic I'm looking at is forecasting using ARMA models. The below assumes ...
• 303
173 views

### The discrepancy of results of PCA via Eigendecomposition vs via SVD in Python with scipy.linalg [duplicate]

I recently learned about different methods of PCA. I decided to manually implement PCA in Python with Eigendecomposition of cov(X) and the Singular Value ...
• 175
44 views

### Is a bivariate copula relevant in this physics setting manifesting uniform univariate marginals--and, if so, how can it be constructed?

To quickly place our probabilistic (copula) question in its subject matter setting, we note that a fundamental concept in quantum theory is that of entanglement QuantumEntanglement. The states of ...
438 views

### All eigenvalues less than 1 in factor analysis

Is it possible to have all eigenvalues less than 1 during factor analysis? then what can be concluded here? since a factor is considered a factor when the eigenvalue of the factor is greater than 1. ...
1 vote
97 views

### How do I get the stationary distribution of a Markov chain matrix from SVD?

I have a matrix that represents a Markov chain. ...
• 1,337
66 views

### Eigenvalue decomposition

Why is the eigenvalue decomposition of the covariance matrix formula, $UΛU^T$ (assuming T means transpose) and not $UΛU^{-1}$? Where $U$ is eigenvectors and $Λ$ is a diagonal matrix consisting of ...
227 views

### Find first Principal Component (and loading) using a fast iterative algorithm without covariance matrix

I have a matrix $X$ and I would like to find its first principal component and the corresponding loadings. I would like to do this without computing the covariance matrix of $X$. How can I do so? This ...
218 views

### Inequality on norm in terms of eigenvalue

I came across this inequality but not sure if it's true: $|\lambda_{\text{min}}|^2\|\hat{\beta}-\beta\|^2_2\leq (\hat{\beta}-\beta)'\hat{\Sigma}(\hat{\beta}-\beta)$ where $\lambda_{\text{min}}$ is the ...
• 503
1 vote
148 views

### Eigenvalues of time lagged covariance matrix - should be always real and positive?

Consider two random variables $X_t$ and $X_{t+\tau}$; where both come from random variable $X$ but are lagged by $\tau$. Assume that both are mean-free. If we want calculate covariance matrix of them, ...
540 views

### Negative Eigenvalues in EFA

Assuming a set of data meet all assumptions for EFA and we are doing a factor analysis (with the SMC used to define the shared variance), how do negative eigenvalues appear and what does that say ...
• 395
1 vote
96 views

### Why is the first canonical direction equal to the left singular vector i.e. why is $w_1 = a = u_1$ in CCA (Canonical Correlation Analysis)?

I want to understand why the canonical direction $a$ is equal to the left singular value of $M = \Sigma^{-1/2}_X \Sigma_{X, Y} \Sigma^{-1/2}_Y$ and not $a = \sigma_1 u_1$. My calculation tell me that ...
• 6,336
1 vote
45 views

### Relation between PCA eigen values and data visualization

We have matrix data $X$ which is $n\times d$. We use the covariance matrix/ design matrix/ gram matrix $X^T X$ to perform least-squares/ PCA. I compute the eigen basis representation of said matrix ...
• 181
67 views

### number and size of eigenvectors in PCA

As I understand, the size of eigenvector produced in PCA should be min{n,N}, where N=number of samples and n=dimension of each sample (Right?). However, I have seen in couple of cases that this size ...
• 43
1 vote
427 views

### Making sure that the design matrix is positive (semi-) definite

In Bayesian linear regression, how do I make sure that the design matrix produced by a neural network $\Phi$ is positive definite? Computing the covariance matrix on the weight requires inverting --- ...
924 views

### Get accurate eigenvectors, when eigenvalues are minuscule

I have a symmetric matrix A. I'm not able to compute all the eigenvectors accurately, and I believe it is due to the last few eigenvalues for ...
• 117
1 vote
292 views

### What are the metrics to assess the quality of a multiple correspondence analysis (MCA) model?

We are trying to implement a multiple correspondence analysis (MCA) model. I was looking for metrics to assess the quality of an MCA to evaluate our model. Sadly, I didn’t find much literature about ...
• 23
1 vote