Questions tagged [eigenvalues]

For questions involving calculation or interpretation of eigenvalues or eigenvectors.

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

Why eigen vector of a covariance matrix is the largest principle components? [duplicate]

I am self studying principle component analysis using this tutorial, I got most of the reasoning behind PCA but I don't get the intuitive reason why eigen vectors of a covariance matrix is also its ...
1
vote
0answers
23 views

Order of eigenvalues when using different methods

I'm doing PCA in a covariance matrix where each column and row represents tenors of the yield curve. I have coded the Jacobi rotation method and I also have a QR algorithm based on numpy.linalg.qr in ...
0
votes
1answer
16 views

Eigenvectors of covariance matrix and inertia tensor

The moment of inertia tensor from physics looks very similar to the covariance matrix, used for PCA. How are their eigenvectors and eigenvalues related?
1
vote
0answers
35 views

Are there problems associated with choosing the smallest eigenvalue/eigenvector when performing PCA?

In relation to PCA which is usually used in a setting where one wishes to maximize the variance as well as a reduction in dimensionality thereby choosing the eigenvalues with corresponding ...
0
votes
1answer
27 views

Eigenvalues of idempotent matrix of rank $r$

In the proof for the following theorem in Linear Models in Statistics, Render & Schaalje $\textbf{Theorem 5.5}$ Let y be distributed as $N_p\left({\mathbf{\mu}, \mathbf{\Sigma}}\right)$, let $\...
5
votes
0answers
60 views

What is the precise relation between the eigenvalues of a covariance *function* and the eigenvalues of a covariance *matrix*?

Assume we have a temporal Gaussian Process $\mathcal{GP}(t;\ m,k)$ (GP) with mean $m$ and covariance function (aka. kernel) $k$ on some compact time interval $[0,T]$. Then, the eigenvalues $\lambda$ ...
0
votes
0answers
22 views

Multivariate normal distribution - diagonalizable matrix

Let $X_{1},..,X_{n}$ be independent random vectors drawn from the $N_{p}(\mu ,\Sigma )$ distribution and let $\bar{X}$ be their mean. Find a $p\times np$ matrix $A$ and vector $v\in \mathbb{R}^{np}$ ...
1
vote
0answers
49 views

OLS estimator is consistent if the smallest eigenvalue of $X^TX$ goes to infinity as $n\to\infty$

I want to show that if $\lambda_{min}(X^T X)$ (i.e., the smallest eigenvalue of $X^TX$) goes to infinity as $n\to\infty$, then $\hat{\beta}$ is a consistent estimator of $\beta$. My approach is the ...
0
votes
0answers
24 views

Variables' weights using PCA

I have recently read a paper where the authors applied PCA to determine the weights of the variables used to calculate a composite index. In the methodology, they mentioned that for a set of $N$ ...
1
vote
0answers
18 views

Why are the directions of eigenvectors in SVD and Eigen-Decomposition for PCA opposite? [duplicate]

As you may know, scikit-learn library utilizes singular value decomposition (SVD) of data matrix X to produce eigenvectors for PCA. I decided to code PCA by using ...
5
votes
1answer
110 views

How to prove positive (semi-)definitness in matrix notation without numbers

I'd like to show that $$V[\hat\beta_{OLS}]-V[\hat\beta_{GLS}]=\sigma^2(X'X)^{-1}(X'\Omega X)(X'X)^{-1}-\sigma^2(X'\Omega X)^{-1}\geq 0$$ is positive (semi-) definite. $\Omega$ and $X$ are square-...
0
votes
0answers
32 views

Fisher's formalism : How to find a complementary matrix to respect the Maximum Likelihood Estimator (MLE)?

I make following a previous post : Bad attempt to do cross-correlations between 2 matrices Indeed, I say "Bad attempt" since the beginning of this study, I did a major error. By wanting to ...
2
votes
1answer
45 views

How to get the eigenvalue expansion of the covariance matrix?

Working through Bishops’s Pattern Recognition and Machine Learning and have the following question regarding the Eigenvalue expansion of a covariance matrix: “ Assume we have a symmetric real-valued ...
0
votes
0answers
27 views

Eigenvectors and eigenvalues of covariance matrix or its inverse in drawing ellipsoid

I'm trying to draw an ellipsoid of the $3 \times 3$ covariance matrix. Usually, I see the sentence an ellipsoid corresponding to the eigenvectors and eigenvalues of covariance matrix. But from the ...
3
votes
0answers
145 views

How to draw an ellipsoid corresponding to the eigenvectors and eigenvalues of covariance matrix?

I'm doing PCA with Python with dataset decathlon in which I'm interested in 3 variables 100m, ...
0
votes
1answer
77 views

How to Compute the Reconstruction error in Principal Component Analysis at lower dimensions

I have m examples and d features where m<<d. So I managed to compute the eigen value and corresponding its eigen vector ... I want to compute the reconstruction error for various value of ...
4
votes
0answers
52 views

Physical interpretation of $U$ and $V$ matrices in SVD

I have a question about the physical interpretation of $U$ and $V$ matrices in SVD. I collect measurements at multiple devices across time are collected into an $m$ × $T$ matrix $M$, where m is the ...
4
votes
0answers
91 views

PCA: inference on the proportion of explained variance, in a large p setting

I am interested in doing inference on the proportion of total variance explained by the first principal component, for a PCA based on the correlation matrix R. I want to know the (asymptotic) ...
1
vote
0answers
18 views

Kernel matrix decomposition

I had a look at the sklearn.kernel_approxiamtion.Nystroem implementation, which is also described in this post: Nystroem Method for Kernel Approximation Here, a ...
3
votes
1answer
65 views

Eigenvalues in Ridge regression [duplicate]

The ridge regression estimate is given by $$\beta^{*}=(X'X+kI)^{-1}X'y, k≥0,$$ where $X$ is the feature matrix. The original paper, Hoerl and Kennard's Ridge Regression: Biased Estimation for ...
2
votes
0answers
39 views

Dimensionality reduction of a large covariance matrix

I have a large covariance matrix $\Sigma$ and I am reducing its dimensionality by using a truncated eigendecomposition. $\Sigma \approx VDV^T$. I remember somewhere that you could also decompose it as ...
1
vote
1answer
44 views

Question about the Proof of PCA in “Learning from Data” by Shwartz and Ben-David, P. 280-281

Does anyone know how to justify the red and blue line in the attached proof of PCA? Red line: $B \in \mathbb{R}^{ d \times n}$, arrange $B = [B_{j,1} | B_{j,2} | \cdots | B_{j,n}]$, then $B^\top B = \...
0
votes
1answer
85 views

how to get total Fisher matrix that makes cross synthesis of 2 Fisher matrix

I have initially posted on physics.stackexchange but I think my issue is more adapted on Cross-Validated (so I am going to delete the initial post on physics.stackexchange). I have 2 Fisher matrixes ...
1
vote
1answer
26 views

What do eigenvectors of a data matrix consisting of house features/prices tell us? [duplicate]

I know this is one of the most repetitive question but bear with me please. I am trying to gain an intuitive understanding of eigenvectors. I had this example in my mind where there is a matrix A, the ...
1
vote
0answers
30 views

Checking that $ 𝐸[𝑥𝑥′⊗𝑥𝑥′] \prec \Sigma\otimes I+I\otimes \Sigma$

I have a random variable with mean 0 and covariance $\Sigma$, and I need to check that the following condition is satisfied $$2\Sigma\otimes \Sigma+\text{vec} \Sigma(\text{vec}\Sigma)'\prec \Sigma\...
1
vote
0answers
17 views

How to find eigenvalues and eigenvectors of the cokurtosis matrix?

Kurtosis is the fourth statistical moment of a random variable's distribution. Unlike the variance-covariance matrix $\Sigma$, which had a shape of $p\times p$, the kurtosis-cokurtosis matrix is ...
7
votes
1answer
166 views

Multiplying vectors by the covariance matrix?

I thought I knew covariance but I'm starting to think that there's more to it. For example, what happens when you multiply observations by their corresponding covariance matrix? ...
1
vote
0answers
126 views

What's the importance of parallel eigenvectors?

I'm studying eigenvectors. I read that if a matrix is symmetric and if the eigenvalues are real numbers, the eigenvectors will be perpendicular. However, I have no idea what it means (if anything) ...
0
votes
0answers
13 views

Interpreting SAS output - Roots of AR Characteristic Polynomial

I urgently need help on interpreting the numbers from a SAS output on Characteristic Roots: The VARMAX Procedure The VARMAX Procedure Roots of AR Characteristic Polynomial Index Real Imaginary Modulus ...
0
votes
0answers
12 views

Can trials have a differing number of samples when running PCA ? Why not?

Can somebody confirm that the number of "samples for each trial" doesn't matter(i guess that's right the language) for PCA. The case at hand is this: i have 5 sets of 3-dimensional ...
4
votes
1answer
58 views

PCA: Using R to generate and plot eigenvalues

I have the following data: ...
0
votes
0answers
14 views

Interpretation of eigenvalues from `decorana()` summary table

I just want to make sure I understand the example in the vegan package. Per the summary table on page 2 of this document, can I say that the first two axes of the ...
0
votes
0answers
44 views

Can Principal Component Analysis Be Used Here?

My professor has given us test prep in the form of a scenario essay question (For studying purposes/not graded) I want to see if my method of Principal Component Analysis would be applicable here. I ...
0
votes
1answer
47 views

Eigenvalues from `prcomp`

I used prcomp to calculate the follow PCA values: ...
0
votes
0answers
43 views

Principal Component Analysis: eigenvectors that maximise variance

I am currently starting to learn about Principal Component Analysis. In a presentation, the following is said: Let $\mathbf{X} \sim (\mathbf{\mu}, \Sigma)$, $(\lambda_k, \mathbf{\eta}_k)$ the $k$th ...
0
votes
0answers
12 views

What is the meaning of the regressor characteristic root?

As described by Greene's Econometric Analysis (7th Edition), the regressor matrix's condition number measures how singular the matrix is. Therefore, the condition number is a measure of ...
1
vote
0answers
15 views

Are haar bases eigenfunctions for any kernel?

Are haar wavelet bases eigenfunctions for any kernel? If so, what Kernel is it, and how would we find the eigenvalues?
4
votes
1answer
55 views

What are the conditions for a graph's adjacency matrix to not have a negative eigenvalue with magnitude>=1?

Say I have a (directed) graph $G$ with an adjacency matrix $A$. For the sake of the question, let's assume it's normalized column-wise (edge weights are normalized so the sum of out-edge weights per ...
1
vote
1answer
22 views

Calculating eigen values from principal components and deciding on the number of principal components?

I calculated PCs for my samples and I am showing here data frame that has samples as my rows and PCs as my columns. My question is in order to decide on the number of PCs to keep for my regression ...
0
votes
0answers
62 views

Eigenvalues in exploratory factor analysis in R using psych::fa

I've run an EFA in R using the fa() function, extracting 6 factors from a pool of 22 items. From what I understand, the line in the fa output labeled 'SS loadings' presents the eigenvalues of each ...
0
votes
0answers
18 views

Variance of a principal component

Suppose that we have a SVD for our data matrix centered $X = UDV^T$. Then it is stated that the i-th principal component, $Xv_i$, has variance $\frac{d_i^2}{N}$. Consider these steps. $$ var(Xv_i) = ...
0
votes
0answers
10 views

How does partial least squares algorithm return more than one factor?

My understanding of PLS regression is that we find an eigen vector such that it maximises the covariance between X(matrix of independent variables) and Y(vector/matrix of dependent variable) i.e. find ...
0
votes
0answers
35 views

Should the sample size of principal component analysis and principal component regression be the same?

We have 1,000 observations for principal component analysis (PCA). In the following principal component regression(PCR) modeling, I found that only 500 of the 1,000 observations having the outcome we ...
0
votes
0answers
37 views

In Probabilistic PCA, Where does the arbitrary orthogonal matrix(rotation matrix) come from?

I'm working on studying Probabilistic PCA based on the paper (Tipping & Bishop, 1999), I can follow the idea that the maximum likelihood function would reach the stationary point when the the ...
0
votes
0answers
36 views

Eigenvalue bias in covariance estimation with limited number of samples

In the paper regularized discriminant analysis by Friedman, after introducing the sample covariance estimation as where the coefficient $W_k$ is related to the class priors in multi-class ...
1
vote
1answer
46 views

GLMER Overdispersion and Error messages

I have a data set which involves 30 binomial absence/presences totalled for a ratio out of 1, which is the total score of a test out of 30 marks. The data requires fitting one of my predictor ...
2
votes
1answer
61 views

Is $tr(B(B^TWB + D)^{-1}B^TW) = tr((I + D(B^TWB)^{-1})^{-1})$?

I am reading Eilers and Marx (1996) and at the beginning of page 94 they write, for $Q = B^TWB$, $D$ a symmetric positive definite matrix and $W$ a diagonal matrix, \begin{align} tr\left(B(Q + D)^{-1}...
0
votes
0answers
22 views

Connection between samples and dimensions of a matrix with the covariance matrix in PCA

In PCA, for a given matrix $M_{S\times D}$ where s = samples and d = dimensions, computing covariance matrix of dimension vector and then an eigen decomposition on it leads to eigenvectors which can ...
1
vote
1answer
46 views

Is the covariance matrix almost always postive definite?

I understand a covariance matrix is always positive semi-definite, but it seems that the covariance matrix would almost always be positive definite (although theoretically is only guaranteed to be ...
0
votes
0answers
24 views

Interpreting SVD on non-centered matrix

I have a very large, very sparse matrix $A \in \mathbb{N}^{n \times m}$ I'd like to perform SVD on. It is non-centered. When I center it to $A'$, I can't even fit it in memory (because $A'$ is in $\...

1
2 3 4 5
8