Questions tagged [eigenvalues]
For questions involving calculation or interpretation of eigenvalues or eigenvectors.
343
questions
0
votes
0answers
6 views
How to Compute the Reconstruction error in Principal Component Analysis at lower dimensions
I have m examples and d features where m<<d. So I managed to compute the eigen value and corresponding its eigen vector ... I want to compute the reconstruction error for various value of ...
4
votes
0answers
31 views
Physical interpretation of $U$ and $V$ matrices in SVD
I have a question about the physical interpretation of $U$ and $V$ matrices in SVD.
I collect measurements at multiple devices across time are collected into an $m$ × $T$ matrix $M$, where m is the ...
4
votes
0answers
76 views
PCA: inference on the proportion of explained variance, in a large p setting
I am interested in doing inference on the proportion of total variance explained by the first principal component, for a PCA based on the correlation matrix R. I want to know the (asymptotic) ...
1
vote
0answers
14 views
Kernel matrix decomposition
I had a look at the sklearn.kernel_approxiamtion.Nystroem implementation, which is also described in this post:
Nystroem Method for Kernel Approximation
Here, a ...
3
votes
1answer
41 views
Eigenvalues in Ridge regression [duplicate]
The ridge regression estimate is given by
$$\beta^{*}=(X'X+kI)^{-1}X'y, k≥0,$$
where $X$ is the feature matrix. The original paper, Hoerl and Kennard's Ridge Regression: Biased Estimation for ...
2
votes
0answers
39 views
Dimensionality reduction of a large covariance matrix
I have a large covariance matrix $\Sigma$ and I am reducing its dimensionality by using a truncated eigendecomposition. $\Sigma \approx VDV^T$. I remember somewhere that you could also decompose it as ...
1
vote
1answer
40 views
Question about the Proof of PCA in “Learning from Data” by Shwartz and Ben-David, P. 280-281
Does anyone know how to justify the red and blue line in the attached proof of PCA?
Red line: $B \in \mathbb{R}^{ d \times n}$, arrange $B = [B_{j,1} | B_{j,2} | \cdots | B_{j,n}]$, then $B^\top B = \...
1
vote
1answer
76 views
how to get total Fisher matrix that makes cross synthesis of 2 Fisher matrix
I have initially posted on physics.stackexchange but I think my issue is more adapted on Cross-Validated (so I am going to delete the initial post on physics.stackexchange).
I have 2 Fisher matrixes ...
1
vote
1answer
24 views
What do eigenvectors of a data matrix consisting of house features/prices tell us? [duplicate]
I know this is one of the most repetitive question but bear with me please. I am trying to gain an intuitive understanding of eigenvectors. I had this example in my mind where there is a matrix A, the ...
1
vote
0answers
29 views
Checking that $ 𝐸[𝑥𝑥′⊗𝑥𝑥′] \prec \Sigma\otimes I+I\otimes \Sigma$
I have a random variable with mean 0 and covariance $\Sigma$, and I need to check that the following condition is satisfied
$$2\Sigma\otimes \Sigma+\text{vec} \Sigma(\text{vec}\Sigma)'\prec \Sigma\...
1
vote
0answers
16 views
How to find eigenvalues and eigenvectors of the cokurtosis matrix?
Kurtosis is the fourth statistical moment of a random variable's distribution. Unlike the variance-covariance matrix $\Sigma$, which had a shape of $p\times p$, the kurtosis-cokurtosis matrix is ...
6
votes
1answer
148 views
Multiplying vectors by the covariance matrix?
I thought I knew covariance but I'm starting to think that there's more to it. For example, what happens when you multiply observations by their corresponding covariance matrix? ...
1
vote
0answers
79 views
What's the importance of parallel eigenvectors?
I'm studying eigenvectors. I read that if a matrix is symmetric and if the eigenvalues are real numbers, the eigenvectors will be perpendicular. However, I have no idea what it means (if anything) ...
0
votes
0answers
12 views
Interpreting SAS output - Roots of AR Characteristic Polynomial
I urgently need help on interpreting the numbers from a SAS output on Characteristic Roots:
The VARMAX Procedure
The VARMAX Procedure Roots of AR Characteristic Polynomial
Index Real Imaginary Modulus ...
0
votes
0answers
12 views
Can trials have a differing number of samples when running PCA ? Why not?
Can somebody confirm that the number of "samples for each trial" doesn't matter(i guess that's right the language) for PCA.
The case at hand is this: i have 5 sets of 3-dimensional ...
4
votes
1answer
45 views
0
votes
0answers
7 views
Interpretation of eigenvalues from `decorana()` summary table
I just want to make sure I understand the example in the vegan package. Per the summary table on page 2 of this document, can I say that the first two axes of the ...
0
votes
0answers
44 views
Can Principal Component Analysis Be Used Here?
My professor has given us test prep in the form of a scenario essay question (For studying purposes/not graded) I want to see if my method of Principal Component Analysis would be applicable here. I ...
0
votes
1answer
29 views
0
votes
0answers
43 views
Principal Component Analysis: eigenvectors that maximise variance
I am currently starting to learn about Principal Component Analysis. In a presentation, the following is said:
Let $\mathbf{X} \sim (\mathbf{\mu}, \Sigma)$, $(\lambda_k, \mathbf{\eta}_k)$ the $k$th ...
0
votes
0answers
12 views
What is the meaning of the regressor characteristic root?
As described by Greene's Econometric Analysis (7th Edition), the regressor matrix's condition number measures how singular the matrix is. Therefore, the condition number is a measure of ...
1
vote
0answers
15 views
Are haar bases eigenfunctions for any kernel?
Are haar wavelet bases eigenfunctions for any kernel? If so, what Kernel is it, and how would we find the eigenvalues?
4
votes
1answer
47 views
What are the conditions for a graph's adjacency matrix to not have a negative eigenvalue with magnitude>=1?
Say I have a (directed) graph $G$ with an adjacency matrix $A$. For the sake of the question, let's assume it's normalized column-wise (edge weights are normalized so the sum of out-edge weights per ...
1
vote
1answer
19 views
Calculating eigen values from principal components and deciding on the number of principal components?
I calculated PCs for my samples and I am showing here data frame that has samples as my rows and PCs as my columns. My question is in order to decide on the number of PCs to keep for my regression ...
0
votes
0answers
25 views
Eigenvalues in exploratory factor analysis in R using psych::fa
I've run an EFA in R using the fa() function, extracting 6 factors from a pool of 22 items. From what I understand, the line in the fa output labeled 'SS loadings' presents the eigenvalues of each ...
0
votes
0answers
18 views
Variance of a principal component
Suppose that we have a SVD for our data matrix centered $X = UDV^T$.
Then it is stated that the i-th principal component, $Xv_i$, has variance $\frac{d_i^2}{N}$. Consider these steps.
$$
var(Xv_i) = ...
0
votes
0answers
9 views
How does partial least squares algorithm return more than one factor?
My understanding of PLS regression is that we find an eigen vector such that it maximises the covariance between X(matrix of independent variables) and Y(vector/matrix of dependent variable) i.e. find ...
0
votes
0answers
29 views
Should the sample size of principal component analysis and principal component regression be the same?
We have 1,000 observations for principal component analysis (PCA). In the following principal component regression(PCR) modeling, I found that only 500 of the 1,000 observations having the outcome we ...
0
votes
0answers
22 views
In Probabilistic PCA, Where does the arbitrary orthogonal matrix(rotation matrix) come from?
I'm working on studying Probabilistic PCA based on the paper (Tipping & Bishop, 1999), I can follow the idea that the maximum likelihood function would reach the stationary point when the the ...
0
votes
0answers
33 views
Eigenvalue bias in covariance estimation with limited number of samples
In the paper regularized discriminant analysis by Friedman, after introducing the sample covariance estimation as
where the coefficient $W_k$ is related to the class priors in multi-class ...
1
vote
1answer
39 views
GLMER Overdispersion and Error messages
I have a data set which involves 30 binomial absence/presences totalled for a ratio out of 1, which is the total score of a test out of 30 marks.
The data requires fitting one of my predictor ...
2
votes
1answer
58 views
Is $tr(B(B^TWB + D)^{-1}B^TW) = tr((I + D(B^TWB)^{-1})^{-1})$?
I am reading Eilers and Marx (1996) and at the beginning of page 94 they write, for $Q = B^TWB$, $D$ a symmetric positive definite matrix and $W$ a diagonal matrix,
\begin{align}
tr\left(B(Q + D)^{-1}...
0
votes
0answers
18 views
Connection between samples and dimensions of a matrix with the covariance matrix in PCA
In PCA, for a given matrix $M_{S\times D}$ where s = samples and d = dimensions, computing covariance matrix of dimension vector and then an eigen decomposition on it leads to eigenvectors which can ...
1
vote
1answer
39 views
Is the covariance matrix almost always postive definite?
I understand a covariance matrix is always positive semi-definite, but it seems that the covariance matrix would almost always be positive definite (although theoretically is only guaranteed to be ...
0
votes
0answers
23 views
Interpreting SVD on non-centered matrix
I have a very large, very sparse matrix $A \in \mathbb{N}^{n \times m}$ I'd like to perform SVD on. It is non-centered. When I center it to $A'$, I can't even fit it in memory (because $A'$ is in $\...
0
votes
0answers
13 views
Variability of K SVD components [duplicate]
Let's say I have a SVD of a matrix $A = U \Sigma V^T$, $A \in \mathbb{R}^{n \times m}$, and I'm using top-k components corresponding to $\sigma_1, ..\sigma_k$, the k largest values on the diagonal of $...
1
vote
0answers
75 views
Angle between PCA vector spaces?
I have two datasets of the same shape, one for condition A, the other for condition B. I would like to test if the major axes of variance of condition A are different than those of B. Here is my idea. ...
1
vote
0answers
49 views
Understanding the output from the Johansen Cointegration test in R
I have a VECM model that Im using to determine the revenues for a firm, based on factors like Interest rates, S&P 500 and company specific variables, as follows:
Stage 1:
$$z_t= a+ bX_t+e_t$$
...
2
votes
1answer
43 views
What is this projection matrix doing?
Let’s say we have a $m\times d$ zero mean multivariate Gaussian matrix $X$. Its covariance matrix is $X^{T}X$. Let $V$ be the $d\times d$ matrix of eigenvectors of $X^{T}X$, with the columns sorted in ...
0
votes
0answers
12 views
Two different approaches to the linear discriminant analysis(LDA)
I have seen two different approaches to the explanation of the Linear Discriminant Analysis. The following is the description of the rough understanding of the approaches:
1) The first one refers to ...
1
vote
0answers
13 views
eigenstructure matching optimization
Is there any optimization loss functions that can approximately match the eigenstructure of the original samples and the transformed samples?
For example, given a collection of samples $\mathbf{X}$ ...
0
votes
0answers
9 views
Covariance matrix distance variability
I have been preparing an analysis of a data set from which I have extracted a number of covariance matrices and used the Log-Euclidean algorithms to calculate distances, geodesics etc. This data set ...
1
vote
0answers
25 views
Why covariance matrix estimated pairwise is not guaranteed to be PSD?
As mentioned here
Is a covariance matrix composed of matrixes derived from separate samples guaranteed to be positive definitive?
and here Must a matrix of sample pairwise covariances be PSD?
if you ...
0
votes
1answer
35 views
What can you say about spread of data by looking at singular values and clusters?
I have dataset 250X5 and its singular values are [200 50 25.2 2.3 0.35].
Singular values are directly related to variance. Can you say something about the clustering of data and how much is the spread ...
0
votes
0answers
8 views
Investigate if 1st principal component can discriminate between two classes
Consider the following cases
$X|Y$ = 1 ~ $N_d(0,\sigma^2I_d)$
$X|Y$ = -1 ~ $N_d(0,\tau^2I_d)$
Derive an expression for the eigenvector $v$ corresponding to the largest eigenvalue of $\Sigma = E[XX^...
1
vote
0answers
11 views
First component of non-centered data
Let $X$ be a random vector of dimension $p$ and $\{ X_1, \dots, X_n \}$ the $n$ observations of such vector. Let $\mathbb{X}$ be the matrix with rows $X_k$ and $\mathbb{X}_c$ is the matrix with rows $...
0
votes
0answers
32 views
Sketch ellipse with variance-covariance matrix that got after PCA
For X = (X1, X2, X3) distributed as N3(µ, Σ),
mean of the original data is mu and variance-covarinace matrix of the original data is Sigma.
I found in this section that we can derive the variance-...
10
votes
1answer
203 views
Why eigenvectors reveals the groups in Spectral Clustering
According to Handbook of Cluster Analysis Spectral Clustering is done with following algorithm:
Input Similarity Matrix $S$, number of clusters $K$
Form the transition matrix $P$ with $P_{ij} = S_{...
0
votes
0answers
10 views
Interpretation of Johanson co-integration test results
if the Max-eigenvalue test indicates no cointegration at the 0.05 level, while the Trace test indicates 2 cointegrating eqn(s) at the 0.05 level, how do i interpret the results?
1
vote
0answers
38 views
Almost sure convergence of the minimum eigenvalue of a sample covariance matrix
I was wondering if someone could provide a reference to the following result. Consider the $p\times p $ sample matrix
$$\frac{1}{n} \sum_{i=1}^n x_i x_i',$$
where $x_i$ are i.i.d. $p\times 1$ random ...
