Questions tagged [pca]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

893 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
18
votes
1answer
426 views

Blind source separation of convex mixture?

Suppose I have $n$ independent sources, $X_1, X_2, ..., X_n$ and I observe $m$ convex mixtures: \begin{align} Y_1 &= a_{11}X_1 + a_{12}X_2 + \cdots + a_{1n}X_n\\ ...&\\ Y_m &= a_{m1}X_1 + ...
12
votes
0answers
406 views

Can I use optimally scaled variables for a factor analysis to account for rotation? If I can then how?

I have discussed this issue several times in this site, but I am asking it again for a final justification from the experts of our community. I wanted to extract four factors (I should call dimensions ...
6
votes
0answers
388 views

What, if any, dissimilarity is preserved in partial least squares (PLS)?

When we perform a principal components analysis (PCA) on a multivariate data set we are interested in finding orthogonal components that explain maximal variance in the data set. We can form a biplot ...
5
votes
0answers
208 views

Why do PCA and PCoA give the same components but different explained variances?

I'm quite familiar with Principal Component Analysisis, as I use it to study genetic structure. Lately, I was revisiting some of the functions I was using in R (...
5
votes
0answers
223 views

What exactly should be called “projection matrix” in the context of PCA?

At the end of the PCA algorithm one gets a $D\times d$ matrix $U$ such that $z=U^Tx$ (here $x$ is $D$-dimensional and $z$ is $d$ dimensional with $d\leq D$). In multiple sources on the Web I found ...
5
votes
1answer
3k views

How to understand “factor loadings” in PCA?

I have seen that some people are talking about "factor loadings" in PCA. It is a topic that I do not manage to understand, despite some googling. I managed to obtain some code that generates the ...
5
votes
0answers
158 views

What is the fastest way to compute PC1 scores, without performing the whole PCA?

I want to compute only the first principal component's scores $t_1$ of a large number $n$ of data points x with a high dimensionality $p$. Assume the data has been centered about zero. Data points ...
5
votes
0answers
2k views

Interpretation of biplot in PCA

Blue points all appear in the lower right-hand quadrant in the plane formed by the first two principal components. Is it a good interpretation of the biplot (right panel) to say that blue points are ...
5
votes
0answers
606 views

can I use PCA and PAF on Kendall's and Spearman's correlation matrix?

I have a dataset of 77 items, ranked by 17 people, with many ties (actually: Q-sorted under a forced quasi-normal distribution ...
5
votes
1answer
4k views

Polychoric PCA and component loadings in Stata

I’m using Stata 12.0, and I’ve downloaded the polychoricpca command written by Stas Kolenikov, which I wanted to use with data that includes a mix of categorical ...
4
votes
0answers
166 views

PCA, SMOTE and cross validation - how to combine them together?

I was reading a lot recently about PCA and cross validation and it seems that the majority call it malpractice to do PCA before cross validation. I would also like to perform SMOTE, but there is a ...
4
votes
0answers
23 views

Mutual dependence/associations

I have three related questions about measuring the association within sets of data: A. Imagine I were to try and test the hypothesis that the IQ of individuals in same sex couples are correlated (e.g....
4
votes
0answers
22 views

How would one carry out principal component analysis in the Beltrami-Klein model of hyperbolic space?

This is a follow up to a comment by @whuber in response to this question: Is there an extension of PCA for data embedded in hyperbolic spaces? Sorry, I would have posted it there as a comment but it ...
4
votes
0answers
184 views

Mathematically Describing PCA chained with Logistic Regression

Python's scikit-learn package has a convenient pipe function that can combine machine learning techniques into one model with ...
4
votes
0answers
65 views

Analogue of spectral gap but for *smallest* eigenvalues/singular values

The difference between the largest eigenvalue and the next-largest of a graph Laplacian (equivalently, of the random walk Markov chain on the graph) is the spectral gap, related to the Cheeger ...
4
votes
0answers
437 views

Most important original features in PCA: can one multiply eigenvectors by the explained variance?

I would like to know the importance of the original features in principle component analysis. See this Stackoverflow link for an example of what I mean (with a code example). The question is: can you ...
4
votes
0answers
414 views

How to check t-SNE performance

We use t-SNE to visualize multidimensional vectors into 2D or 3D space, and if my understanding of this algorithm is correct, we can compare it to PCA as it also provides dimensionality reduction. My ...
4
votes
0answers
98 views

Categorical PCA: Merge categories based on Transformation Plots?

A tutorial on categorical pca (CATPCA) (Linting et al. 2012) explains that a decision to merge categories of an ordinal variable can be made based on the category quantification ("none of the ...
4
votes
0answers
221 views

Efficient way to solve generalized eigenvalue problem when the number of dimensions is greater than the number of samples

I am trying to solve the generalized eigenvalue problem: $$C_e v = \lambda C_o v.$$ $C_e$ and $C_o$ are both covariance matrices generated from data with $10512$ dimensions and about $2000$ samples. ...
4
votes
0answers
2k views

Why is Pearson Principal Component Analysis so much more common than Spearman or Kendall PCA?

As I understand it, Pearson PCA finds eigenvectors of the Pearson correlation matrix. The result is a coordinate system with dimensions that are linearly uncorrelated. But, a Spearman PCA does this ...
4
votes
0answers
220 views

Kernel PCA vs principal curve analysis

Both principal curve analysis and kernel PCA provide the ability to find nonlinear PCA. Kernel PCA does this by finding principal components in a higher dimensional space. Principal curve analysis is ...
4
votes
1answer
82 views

Does it make sense to do PCA after robust PCA?

I was wondering whether it makes sense to do PCA after robust PCA. Suppose I have a matrix $X$ and if I do robust PCA I would get: $$X=A+E$$ And if I do PCA over $A$ would this make sense as a ...
4
votes
0answers
272 views

How do random data eigenvalues change, as random variables are added?

I am using parallel analysis (Horn 1965) to determine how many principal components I can extract from my data. I can add more variables to my dataset, but I cannot add more cases (I know, that's ...
4
votes
0answers
4k views

What is the rationale behind the “eigenvalue > 1” criterion in factor analysis or PCA?

What is the meaning of "eigenvalue > 1" criterion? I understand what eigenvalues and eigenvectors are. This question is w.r.t. this link and this statement there: By default, VARCLUS stops ...
4
votes
0answers
561 views

Factor analysis across different levels of data aggregation

I have survey data for thousands of individuals from hundreds of towns. I want to identify factors underlying certain characteristics at the town level and the individual level. The individual level ...
4
votes
0answers
3k views

What is the correct way to calculate the explained variance of each EOF as calculated from a gappy data set?

I am trying to determine the correct amount of variance explained by each mode of an Empirical Orthogonal Function (EOF) analysis (similar to "PCA") as applied to a gappy data set. (i.e., containing ...
4
votes
0answers
361 views

Comparing principal components if number of variables changes

I have a database that contains daily stock returns of more than a thousand stocks for many decades. I would like to achieve the following goals: Construct a time-varying measure of co-movement (or ...
3
votes
0answers
50 views

Having only binary variables, why is the use of PCA still appropriate?

I often see the use of PCA on large datasets with a lot of binary variables. As i recall, the computation of the principal components is done via eigenvalue decomposition (or SVD) of the correlation ...
3
votes
0answers
174 views

stationarity and fractional differencing

This is a methodology question. I would like to make the data stationary but not transform it "too much" (information loss), before it is fit for statistical/ML purposes such as regression or PCA. ...
3
votes
0answers
89 views

Why is it necessary to eliminate components in PCR in order to 'solve' multicollinearity?

Running some form of regression on an input dataset that exhibits strong multicollinearity can cause unstable regression coefficients, because the regression algorithm can somewhat arbitrarily ...
3
votes
0answers
94 views

Understanding kernel PCA when the target space is infinite-dimensional

The PCA optimization problem is known as $$ \max_{U \in \mathbb{R}^{d\times r}, U^TU = I} tr(U^T\Sigma U), $$ where $\Sigma$ is a covariance matrix of a dataset $\{x_1,\dots,x_n\} \subset \mathbb{R}^d$...
3
votes
1answer
79 views

How to analyze this biplot of PCA?

I have dones a PCA analysis about measurement of a fish morphometric between female and male. After the PCA result came out with biplot graph, I was a little bit confused to interpret this data. It ...
3
votes
0answers
401 views

PCA - Background removal

I saw this example in a python notebook on Fast.ai. In the notebook they are removing the background and keeping objects in the foreground in a video sequence by using different methods(SVD, random ...
3
votes
0answers
241 views

Detrending time series data prior to PCA

In a recent conversation with a colleague who has much more experience in statistics, she commented that I should detrend my time series data (using a linear model) prior to undertaking PCA. I'm ...
3
votes
0answers
215 views

General PCA optimization problem

I was looking at the PCA optimization problem, which is finding a matrix $U \in \mathbb{R}^{d\times n}$, $n \le d$, that solves the problem $$\max{tr(U^TCU)},\ \ \ s.t. U^TU = I, $$ where $C$ is the ...
3
votes
0answers
212 views

Testing a low rank estimator of a covariance matrix

I am exploring ways to reduce the noise of a covariance matrix estimator when the number of variables is greater than the number of observations, i.e. $n > t$. First, I tried using a low rank ...
3
votes
0answers
39 views

Is LinearDiscriminantAnalysis legit for classifying images?

this was moved from SO, hope this is a better place to ask :) on this context: LDA = LinearDiscriminantAnalysis I tried classifying images' descriptors with SVM SVC linear kernel which gave bad ...
3
votes
0answers
87 views

PCA with one “known” component

(The title might not be very clear, any recommendation welcome) I have a $1 + n$ dimensional dataset. The first dimension measures a specific concept I am interested in and the $n$ remaining ...
3
votes
0answers
284 views

Quiz: Determine first principal component from data-plots

We see four data plots. The goal: How does the first principal component look for each plot a-d. For plot d, it is true that both clusters have same number of datapoints. First principal component ...
3
votes
1answer
745 views

Understanding direction of greatest variance in PCA

In the picture below, why is u the direction of greatest variance? Aren't the data points further away from v than from ...
3
votes
0answers
597 views

If I want to do PCA before k - means, is it mandatory to do it for all variables?

I have 10 variables and some of them are highly correlated. So before I do k - means, I want to get lower number of variables that are not correlated, but retain as much information as possible. Thus, ...
3
votes
0answers
770 views

Are principal components reflective, formative, both, or neither?

In reading various summaries on the similarities and differences between principal components and common factor models, I have noticed that there seems to be conflicting information about whether PCA ...
3
votes
0answers
75 views

What is the relationship between Mohr's Circle and Principal Component Analysis?

While I was studying PCA, I was told that it is related to Mohr's Circle. I don't know what that means. I don't know if they are related or not. I was just told, so I want to make sure here. If they ...
3
votes
0answers
1k views

Principal Component Analysis: how to interpret the total contribution of variables on several dimensions

When we calculate the total contribution of a variable for a single dimension, the sum of all single contributions is equal to 100%, which makes perfect sense. The http://www.sthda.com suggests to ...
3
votes
0answers
496 views

Making sense of PCA and t-SNE representations

I am wondering what (qualitative) sense one can make from PCA and t-SNE plots of weights in a neural network. I'm training a neural network that regresses five different control values, including a ...
3
votes
0answers
192 views

How does PCA maximise Total Variance without maximising Co-variance?

https://stats.stackexchange.com/a/3374/92071 - In PCA, the components are actual orthogonal linear combinations that maximize the total variance. In FA, the factors are linear combinations that ...
3
votes
0answers
160 views

Statistical significance for analysis/inference with PCA

I've got the following comment from a peer reviewer: the interpretations of PCA were described in not objective manner. The PC values can be statistically accessed and everything should be ...
3
votes
0answers
1k views

How to create a scree plot for factor analysis given that eigenvalues depend on the number of extracted factors?

I understand how Kaiser rule works for PCA, as no matter how many components I extract I always get the same eigenvalues. For example, with 3 components I get ...
3
votes
0answers
229 views

Observations aligned on PCA diagnostic plot: geometric interpretation?

The image below depicts the distance-distance plot for a (robust) PCA fit of a real data set. The distance-distance plot is described in greater detail on page 30-31 of (1) or page 2--3 of (0). It ...
3
votes
0answers
999 views

PCA loadings of correlation matrix

I know similar questions have been asked before already but I'm still a bit confused about the understanding of 'loadings'. Eigenvectors are unit-scaled loadings; and they are the coefficients (...