Questions tagged [eigenvalues]
For questions involving calculation or interpretation of eigenvalues or eigenvectors.
356
questions
0
votes
0answers
10 views
Why eigen vector of a covariance matrix is the largest principle components? [duplicate]
I am self studying principle component analysis using this tutorial, I got most of the reasoning behind PCA but I don't get the intuitive reason why eigen vectors of a covariance matrix is also its ...
1
vote
0answers
23 views
Order of eigenvalues when using different methods
I'm doing PCA in a covariance matrix where each column and row represents tenors of the yield curve. I have coded the Jacobi rotation method and I also have a QR algorithm based on numpy.linalg.qr in ...
0
votes
1answer
16 views
Eigenvectors of covariance matrix and inertia tensor
The moment of inertia tensor from physics looks very similar to the covariance matrix, used for PCA. How are their eigenvectors and eigenvalues related?
1
vote
0answers
35 views
Are there problems associated with choosing the smallest eigenvalue/eigenvector when performing PCA?
In relation to PCA which is usually used in a setting where one wishes to maximize the variance as well as a reduction in dimensionality thereby choosing the eigenvalues with corresponding ...
0
votes
1answer
27 views
Eigenvalues of idempotent matrix of rank $r$
In the proof for the following theorem in Linear Models in Statistics, Render & Schaalje
$\textbf{Theorem 5.5}$ Let y be distributed as $N_p\left({\mathbf{\mu}, \mathbf{\Sigma}}\right)$, let $\...
5
votes
0answers
60 views
What is the precise relation between the eigenvalues of a covariance *function* and the eigenvalues of a covariance *matrix*?
Assume we have a temporal Gaussian Process $\mathcal{GP}(t;\ m,k)$ (GP) with mean $m$ and covariance function (aka. kernel) $k$ on some compact time interval $[0,T]$. Then, the eigenvalues $\lambda$ ...
0
votes
0answers
22 views
Multivariate normal distribution - diagonalizable matrix
Let $X_{1},..,X_{n}$ be independent random vectors drawn from the $N_{p}(\mu ,\Sigma )$ distribution and let $\bar{X}$ be their mean.
Find a $p\times np$ matrix $A$ and vector $v\in \mathbb{R}^{np}$ ...
1
vote
0answers
49 views
OLS estimator is consistent if the smallest eigenvalue of $X^TX$ goes to infinity as $n\to\infty$
I want to show that if $\lambda_{min}(X^T X)$ (i.e., the smallest eigenvalue of $X^TX$) goes to infinity as $n\to\infty$, then $\hat{\beta}$ is a consistent estimator of $\beta$.
My approach is the ...
0
votes
0answers
24 views
Variables' weights using PCA
I have recently read a paper where the authors applied PCA to determine the weights of the variables used to calculate a composite index. In the methodology, they mentioned that for a set of $N$ ...
1
vote
0answers
18 views
Why are the directions of eigenvectors in SVD and Eigen-Decomposition for PCA opposite? [duplicate]
As you may know, scikit-learn library utilizes singular value decomposition (SVD) of data matrix X to produce eigenvectors for PCA. I decided to code PCA by using ...
5
votes
1answer
110 views
How to prove positive (semi-)definitness in matrix notation without numbers
I'd like to show that $$V[\hat\beta_{OLS}]-V[\hat\beta_{GLS}]=\sigma^2(X'X)^{-1}(X'\Omega X)(X'X)^{-1}-\sigma^2(X'\Omega X)^{-1}\geq 0$$ is positive (semi-) definite. $\Omega$ and $X$ are square-...
0
votes
0answers
32 views
Fisher's formalism : How to find a complementary matrix to respect the Maximum Likelihood Estimator (MLE)?
I make following a previous post : Bad attempt to do cross-correlations between 2 matrices
Indeed, I say "Bad attempt" since the beginning of this study, I did a major error.
By wanting to ...
2
votes
1answer
45 views
How to get the eigenvalue expansion of the covariance matrix?
Working through Bishops’s Pattern Recognition and Machine Learning and have the following question regarding the Eigenvalue expansion of a covariance matrix:
“
Assume we have a symmetric real-valued ...
0
votes
0answers
27 views
Eigenvectors and eigenvalues of covariance matrix or its inverse in drawing ellipsoid
I'm trying to draw an ellipsoid of the $3 \times 3$ covariance matrix. Usually, I see the sentence
an ellipsoid corresponding to the eigenvectors and eigenvalues of covariance matrix.
But from the ...
3
votes
0answers
145 views
How to draw an ellipsoid corresponding to the eigenvectors and eigenvalues of covariance matrix?
I'm doing PCA with Python with dataset decathlon in which I'm interested in 3 variables 100m, ...
0
votes
1answer
77 views
How to Compute the Reconstruction error in Principal Component Analysis at lower dimensions
I have m examples and d features where m<<d. So I managed to compute the eigen value and corresponding its eigen vector ... I want to compute the reconstruction error for various value of ...
4
votes
0answers
52 views
Physical interpretation of $U$ and $V$ matrices in SVD
I have a question about the physical interpretation of $U$ and $V$ matrices in SVD.
I collect measurements at multiple devices across time are collected into an $m$ × $T$ matrix $M$, where m is the ...
4
votes
0answers
91 views
PCA: inference on the proportion of explained variance, in a large p setting
I am interested in doing inference on the proportion of total variance explained by the first principal component, for a PCA based on the correlation matrix R. I want to know the (asymptotic) ...
1
vote
0answers
18 views
Kernel matrix decomposition
I had a look at the sklearn.kernel_approxiamtion.Nystroem implementation, which is also described in this post:
Nystroem Method for Kernel Approximation
Here, a ...
3
votes
1answer
65 views
Eigenvalues in Ridge regression [duplicate]
The ridge regression estimate is given by
$$\beta^{*}=(X'X+kI)^{-1}X'y, k≥0,$$
where $X$ is the feature matrix. The original paper, Hoerl and Kennard's Ridge Regression: Biased Estimation for ...
2
votes
0answers
39 views
Dimensionality reduction of a large covariance matrix
I have a large covariance matrix $\Sigma$ and I am reducing its dimensionality by using a truncated eigendecomposition. $\Sigma \approx VDV^T$. I remember somewhere that you could also decompose it as ...
1
vote
1answer
44 views
Question about the Proof of PCA in “Learning from Data” by Shwartz and Ben-David, P. 280-281
Does anyone know how to justify the red and blue line in the attached proof of PCA?
Red line: $B \in \mathbb{R}^{ d \times n}$, arrange $B = [B_{j,1} | B_{j,2} | \cdots | B_{j,n}]$, then $B^\top B = \...
0
votes
1answer
85 views
how to get total Fisher matrix that makes cross synthesis of 2 Fisher matrix
I have initially posted on physics.stackexchange but I think my issue is more adapted on Cross-Validated (so I am going to delete the initial post on physics.stackexchange).
I have 2 Fisher matrixes ...
1
vote
1answer
26 views
What do eigenvectors of a data matrix consisting of house features/prices tell us? [duplicate]
I know this is one of the most repetitive question but bear with me please. I am trying to gain an intuitive understanding of eigenvectors. I had this example in my mind where there is a matrix A, the ...
1
vote
0answers
30 views
Checking that $ 𝐸[𝑥𝑥′⊗𝑥𝑥′] \prec \Sigma\otimes I+I\otimes \Sigma$
I have a random variable with mean 0 and covariance $\Sigma$, and I need to check that the following condition is satisfied
$$2\Sigma\otimes \Sigma+\text{vec} \Sigma(\text{vec}\Sigma)'\prec \Sigma\...
1
vote
0answers
17 views
How to find eigenvalues and eigenvectors of the cokurtosis matrix?
Kurtosis is the fourth statistical moment of a random variable's distribution. Unlike the variance-covariance matrix $\Sigma$, which had a shape of $p\times p$, the kurtosis-cokurtosis matrix is ...
7
votes
1answer
166 views
Multiplying vectors by the covariance matrix?
I thought I knew covariance but I'm starting to think that there's more to it. For example, what happens when you multiply observations by their corresponding covariance matrix? ...
1
vote
0answers
126 views
What's the importance of parallel eigenvectors?
I'm studying eigenvectors. I read that if a matrix is symmetric and if the eigenvalues are real numbers, the eigenvectors will be perpendicular. However, I have no idea what it means (if anything) ...
0
votes
0answers
13 views
Interpreting SAS output - Roots of AR Characteristic Polynomial
I urgently need help on interpreting the numbers from a SAS output on Characteristic Roots:
The VARMAX Procedure
The VARMAX Procedure Roots of AR Characteristic Polynomial
Index Real Imaginary Modulus ...
0
votes
0answers
12 views
Can trials have a differing number of samples when running PCA ? Why not?
Can somebody confirm that the number of "samples for each trial" doesn't matter(i guess that's right the language) for PCA.
The case at hand is this: i have 5 sets of 3-dimensional ...
4
votes
1answer
58 views
0
votes
0answers
14 views
Interpretation of eigenvalues from `decorana()` summary table
I just want to make sure I understand the example in the vegan package. Per the summary table on page 2 of this document, can I say that the first two axes of the ...
0
votes
0answers
44 views
Can Principal Component Analysis Be Used Here?
My professor has given us test prep in the form of a scenario essay question (For studying purposes/not graded) I want to see if my method of Principal Component Analysis would be applicable here. I ...
0
votes
1answer
47 views
0
votes
0answers
43 views
Principal Component Analysis: eigenvectors that maximise variance
I am currently starting to learn about Principal Component Analysis. In a presentation, the following is said:
Let $\mathbf{X} \sim (\mathbf{\mu}, \Sigma)$, $(\lambda_k, \mathbf{\eta}_k)$ the $k$th ...
0
votes
0answers
12 views
What is the meaning of the regressor characteristic root?
As described by Greene's Econometric Analysis (7th Edition), the regressor matrix's condition number measures how singular the matrix is. Therefore, the condition number is a measure of ...
1
vote
0answers
15 views
Are haar bases eigenfunctions for any kernel?
Are haar wavelet bases eigenfunctions for any kernel? If so, what Kernel is it, and how would we find the eigenvalues?
4
votes
1answer
55 views
What are the conditions for a graph's adjacency matrix to not have a negative eigenvalue with magnitude>=1?
Say I have a (directed) graph $G$ with an adjacency matrix $A$. For the sake of the question, let's assume it's normalized column-wise (edge weights are normalized so the sum of out-edge weights per ...
1
vote
1answer
22 views
Calculating eigen values from principal components and deciding on the number of principal components?
I calculated PCs for my samples and I am showing here data frame that has samples as my rows and PCs as my columns. My question is in order to decide on the number of PCs to keep for my regression ...
0
votes
0answers
62 views
Eigenvalues in exploratory factor analysis in R using psych::fa
I've run an EFA in R using the fa() function, extracting 6 factors from a pool of 22 items. From what I understand, the line in the fa output labeled 'SS loadings' presents the eigenvalues of each ...
0
votes
0answers
18 views
Variance of a principal component
Suppose that we have a SVD for our data matrix centered $X = UDV^T$.
Then it is stated that the i-th principal component, $Xv_i$, has variance $\frac{d_i^2}{N}$. Consider these steps.
$$
var(Xv_i) = ...
0
votes
0answers
10 views
How does partial least squares algorithm return more than one factor?
My understanding of PLS regression is that we find an eigen vector such that it maximises the covariance between X(matrix of independent variables) and Y(vector/matrix of dependent variable) i.e. find ...
0
votes
0answers
35 views
Should the sample size of principal component analysis and principal component regression be the same?
We have 1,000 observations for principal component analysis (PCA). In the following principal component regression(PCR) modeling, I found that only 500 of the 1,000 observations having the outcome we ...
0
votes
0answers
37 views
In Probabilistic PCA, Where does the arbitrary orthogonal matrix(rotation matrix) come from?
I'm working on studying Probabilistic PCA based on the paper (Tipping & Bishop, 1999), I can follow the idea that the maximum likelihood function would reach the stationary point when the the ...
0
votes
0answers
36 views
Eigenvalue bias in covariance estimation with limited number of samples
In the paper regularized discriminant analysis by Friedman, after introducing the sample covariance estimation as
where the coefficient $W_k$ is related to the class priors in multi-class ...
1
vote
1answer
46 views
GLMER Overdispersion and Error messages
I have a data set which involves 30 binomial absence/presences totalled for a ratio out of 1, which is the total score of a test out of 30 marks.
The data requires fitting one of my predictor ...
2
votes
1answer
61 views
Is $tr(B(B^TWB + D)^{-1}B^TW) = tr((I + D(B^TWB)^{-1})^{-1})$?
I am reading Eilers and Marx (1996) and at the beginning of page 94 they write, for $Q = B^TWB$, $D$ a symmetric positive definite matrix and $W$ a diagonal matrix,
\begin{align}
tr\left(B(Q + D)^{-1}...
0
votes
0answers
22 views
Connection between samples and dimensions of a matrix with the covariance matrix in PCA
In PCA, for a given matrix $M_{S\times D}$ where s = samples and d = dimensions, computing covariance matrix of dimension vector and then an eigen decomposition on it leads to eigenvectors which can ...
1
vote
1answer
46 views
Is the covariance matrix almost always postive definite?
I understand a covariance matrix is always positive semi-definite, but it seems that the covariance matrix would almost always be positive definite (although theoretically is only guaranteed to be ...
0
votes
0answers
24 views
Interpreting SVD on non-centered matrix
I have a very large, very sparse matrix $A \in \mathbb{N}^{n \times m}$ I'd like to perform SVD on. It is non-centered. When I center it to $A'$, I can't even fit it in memory (because $A'$ is in $\...
