Linked Questions

2
votes
0answers
2k views

Varying lengths of eigenvectors on a PCA biplot [duplicate]

I'm conducting a PCA in Matlab on standardized variables. My goal is to interpret angles = loadings, correlations bw. variables and PC-axis directions = vectors point to the direction of the ...
2
votes
1answer
734 views

Selecting subset of variables most associated with the principal components of the data [duplicate]

I have a large data matrix that I'm trying to reduce to a reasonably sized basis set. The original matrix is 916x225, and I need to reduce the number of variables (its columns) to around 50, but I ...
999
votes
28answers
595k views

Making sense of principal component analysis, eigenvectors & eigenvalues

In today's pattern recognition class my professor talked about PCA, eigenvectors and eigenvalues. I understood the mathematics of it. If I'm asked to find eigenvalues etc. I'll do it correctly like ...
215
votes
14answers
230k views

What are the differences between Factor Analysis and Principal Component Analysis?

It seems that a number of the statistical packages that I use wrap these two concepts together. However, I'm wondering if there are different assumptions or data 'formalities' that must be true to use ...
157
votes
7answers
120k views

PCA on correlation or covariance?

What are the main differences between performing principal component analysis (PCA) on the correlation matrix and on the covariance matrix? Do they give the same results?
110
votes
6answers
13k views

Is there an intuitive interpretation of $A^TA$ for a data matrix $A$?

For a given data matrix $A$ (with variables in columns and data points in rows), it seems like $A^TA$ plays an important role in statistics. For example, it is an important part of the analytical ...
72
votes
4answers
21k views

How to visualize what canonical correlation analysis does (in comparison to what principal component analysis does)?

Canonical correlation analysis (CCA) is a technique related to principal component analysis (PCA). While it is easy to teach PCA or linear regression using a scatter plot (see a few thousand examples ...
71
votes
5answers
112k views

Loadings vs eigenvectors in PCA: when to use one or another?

In principal component analysis (PCA), we get eigenvectors (unit vectors) and eigenvalues. Now, let us define loadings as $$\text{Loadings} = \text{Eigenvectors} \cdot \sqrt{\text{Eigenvalues}}.$$ I ...
37
votes
2answers
14k views

How does Factor Analysis explain the covariance while PCA explains the variance?

Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis": According to the highlighted part, factor analysis captures the covariance between ...
32
votes
3answers
49k views

PCA on correlation or covariance: does PCA on correlation ever make sense? [closed]

In principal component analysis (PCA), one can choose either the covariance matrix or the correlation matrix to find the components (from their respective eigenvectors). These give different results (...
38
votes
1answer
13k views

PCA and Correspondence analysis in their relation to Biplot

Biplot is often used to display results of principal component analysis (and of related techniques). It is a dual or overlay scatterplot showing component loadings and component scores simultaneously. ...
18
votes
1answer
9k views

Positioning the arrows on a PCA biplot

I am looking to implement a biplot for principal component analysis (PCA) in JavaScript. My question is, how do I determine the coordinates of the arrows from the $U,V,D$ output of the singular vector ...
27
votes
1answer
13k views

Converting similarity matrix to (euclidean) distance matrix

In Random forest algorithm, Breiman (author) constructs similarity matrix as follows: Send all learning examples down each tree in the forest If two examples land in the same leaf increment ...
11
votes
1answer
21k views

What is the difference between “loadings” and “correlation loadings” in PCA and PLS?

One common thing to do when doing Principal Component Analysis (PCA) is to plot two loadings against each other to investigate the relationships between the variables. In the paper accompanying the ...
9
votes
2answers
12k views

In Factor Analysis (or in PCA), what does it mean a factor loading greater than 1?

I've just run a FA using a oblique rotation (promax) and an item yielded a factor loading of 1.041 on one factor, (and factor loadings of -.131, -.119 and .065 on the other factors using pattern ...

15 30 50 per page