# Questions tagged [pca]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

893 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
426 views

### Blind source separation of convex mixture?

Suppose I have $n$ independent sources, $X_1, X_2, ..., X_n$ and I observe $m$ convex mixtures: \begin{align} Y_1 &= a_{11}X_1 + a_{12}X_2 + \cdots + a_{1n}X_n\\ ...&\\ Y_m &= a_{m1}X_1 + ...
406 views

### Can I use optimally scaled variables for a factor analysis to account for rotation? If I can then how?

I have discussed this issue several times in this site, but I am asking it again for a final justification from the experts of our community. I wanted to extract four factors (I should call dimensions ...
388 views

### What, if any, dissimilarity is preserved in partial least squares (PLS)?

When we perform a principal components analysis (PCA) on a multivariate data set we are interested in finding orthogonal components that explain maximal variance in the data set. We can form a biplot ...
208 views

### Why do PCA and PCoA give the same components but different explained variances?

I'm quite familiar with Principal Component Analysisis, as I use it to study genetic structure. Lately, I was revisiting some of the functions I was using in R (...
223 views

### What exactly should be called “projection matrix” in the context of PCA?

At the end of the PCA algorithm one gets a $D\times d$ matrix $U$ such that $z=U^Tx$ (here $x$ is $D$-dimensional and $z$ is $d$ dimensional with $d\leq D$). In multiple sources on the Web I found ...
3k views

I have seen that some people are talking about "factor loadings" in PCA. It is a topic that I do not manage to understand, despite some googling. I managed to obtain some code that generates the ...
158 views

### What is the fastest way to compute PC1 scores, without performing the whole PCA?

I want to compute only the first principal component's scores $t_1$ of a large number $n$ of data points x with a high dimensionality $p$. Assume the data has been centered about zero. Data points ...
2k views

### Interpretation of biplot in PCA

Blue points all appear in the lower right-hand quadrant in the plane formed by the first two principal components. Is it a good interpretation of the biplot (right panel) to say that blue points are ...
606 views

### can I use PCA and PAF on Kendall's and Spearman's correlation matrix?

I have a dataset of 77 items, ranked by 17 people, with many ties (actually: Q-sorted under a forced quasi-normal distribution ...
4k views

I’m using Stata 12.0, and I’ve downloaded the polychoricpca command written by Stas Kolenikov, which I wanted to use with data that includes a mix of categorical ...
166 views

### PCA, SMOTE and cross validation - how to combine them together?

I was reading a lot recently about PCA and cross validation and it seems that the majority call it malpractice to do PCA before cross validation. I would also like to perform SMOTE, but there is a ...
23 views

### Mutual dependence/associations

I have three related questions about measuring the association within sets of data: A. Imagine I were to try and test the hypothesis that the IQ of individuals in same sex couples are correlated (e.g....
22 views

### How would one carry out principal component analysis in the Beltrami-Klein model of hyperbolic space?

This is a follow up to a comment by @whuber in response to this question: Is there an extension of PCA for data embedded in hyperbolic spaces? Sorry, I would have posted it there as a comment but it ...
184 views

### Mathematically Describing PCA chained with Logistic Regression

Python's scikit-learn package has a convenient pipe function that can combine machine learning techniques into one model with ...
65 views

### Analogue of spectral gap but for *smallest* eigenvalues/singular values

The difference between the largest eigenvalue and the next-largest of a graph Laplacian (equivalently, of the random walk Markov chain on the graph) is the spectral gap, related to the Cheeger ...
437 views

### Most important original features in PCA: can one multiply eigenvectors by the explained variance?

I would like to know the importance of the original features in principle component analysis. See this Stackoverflow link for an example of what I mean (with a code example). The question is: can you ...
414 views

### How to check t-SNE performance

We use t-SNE to visualize multidimensional vectors into 2D or 3D space, and if my understanding of this algorithm is correct, we can compare it to PCA as it also provides dimensionality reduction. My ...
98 views

### Categorical PCA: Merge categories based on Transformation Plots?

A tutorial on categorical pca (CATPCA) (Linting et al. 2012) explains that a decision to merge categories of an ordinal variable can be made based on the category quantification ("none of the ...
221 views

### Efficient way to solve generalized eigenvalue problem when the number of dimensions is greater than the number of samples

I am trying to solve the generalized eigenvalue problem: $$C_e v = \lambda C_o v.$$ $C_e$ and $C_o$ are both covariance matrices generated from data with $10512$ dimensions and about $2000$ samples. ...
2k views

### Why is Pearson Principal Component Analysis so much more common than Spearman or Kendall PCA?

As I understand it, Pearson PCA finds eigenvectors of the Pearson correlation matrix. The result is a coordinate system with dimensions that are linearly uncorrelated. But, a Spearman PCA does this ...
220 views

### Kernel PCA vs principal curve analysis

Both principal curve analysis and kernel PCA provide the ability to find nonlinear PCA. Kernel PCA does this by finding principal components in a higher dimensional space. Principal curve analysis is ...
82 views

### Does it make sense to do PCA after robust PCA?

I was wondering whether it makes sense to do PCA after robust PCA. Suppose I have a matrix $X$ and if I do robust PCA I would get: $$X=A+E$$ And if I do PCA over $A$ would this make sense as a ...
272 views

### How do random data eigenvalues change, as random variables are added?

I am using parallel analysis (Horn 1965) to determine how many principal components I can extract from my data. I can add more variables to my dataset, but I cannot add more cases (I know, that's ...
4k views

### What is the rationale behind the “eigenvalue > 1” criterion in factor analysis or PCA?

What is the meaning of "eigenvalue > 1" criterion? I understand what eigenvalues and eigenvectors are. This question is w.r.t. this link and this statement there: By default, VARCLUS stops ...
561 views

### Factor analysis across different levels of data aggregation

I have survey data for thousands of individuals from hundreds of towns. I want to identify factors underlying certain characteristics at the town level and the individual level. The individual level ...
3k views

### What is the correct way to calculate the explained variance of each EOF as calculated from a gappy data set?

I am trying to determine the correct amount of variance explained by each mode of an Empirical Orthogonal Function (EOF) analysis (similar to "PCA") as applied to a gappy data set. (i.e., containing ...
361 views

### Comparing principal components if number of variables changes

I have a database that contains daily stock returns of more than a thousand stocks for many decades. I would like to achieve the following goals: Construct a time-varying measure of co-movement (or ...
50 views

### Having only binary variables, why is the use of PCA still appropriate?

I often see the use of PCA on large datasets with a lot of binary variables. As i recall, the computation of the principal components is done via eigenvalue decomposition (or SVD) of the correlation ...
174 views

### stationarity and fractional differencing

This is a methodology question. I would like to make the data stationary but not transform it "too much" (information loss), before it is fit for statistical/ML purposes such as regression or PCA. ...
89 views

### Why is it necessary to eliminate components in PCR in order to 'solve' multicollinearity?

Running some form of regression on an input dataset that exhibits strong multicollinearity can cause unstable regression coefficients, because the regression algorithm can somewhat arbitrarily ...
94 views

### Understanding kernel PCA when the target space is infinite-dimensional

The PCA optimization problem is known as $$\max_{U \in \mathbb{R}^{d\times r}, U^TU = I} tr(U^T\Sigma U),$$ where $\Sigma$ is a covariance matrix of a dataset $\{x_1,\dots,x_n\} \subset \mathbb{R}^d$...
79 views

### How to analyze this biplot of PCA?

I have dones a PCA analysis about measurement of a fish morphometric between female and male. After the PCA result came out with biplot graph, I was a little bit confused to interpret this data. It ...
401 views

### PCA - Background removal

I saw this example in a python notebook on Fast.ai. In the notebook they are removing the background and keeping objects in the foreground in a video sequence by using different methods(SVD, random ...
241 views

### Detrending time series data prior to PCA

In a recent conversation with a colleague who has much more experience in statistics, she commented that I should detrend my time series data (using a linear model) prior to undertaking PCA. I'm ...
215 views

### General PCA optimization problem

I was looking at the PCA optimization problem, which is finding a matrix $U \in \mathbb{R}^{d\times n}$, $n \le d$, that solves the problem $$\max{tr(U^TCU)},\ \ \ s.t. U^TU = I,$$ where $C$ is the ...
212 views

### Testing a low rank estimator of a covariance matrix

I am exploring ways to reduce the noise of a covariance matrix estimator when the number of variables is greater than the number of observations, i.e. $n > t$. First, I tried using a low rank ...
39 views

### Is LinearDiscriminantAnalysis legit for classifying images?

this was moved from SO, hope this is a better place to ask :) on this context: LDA = LinearDiscriminantAnalysis I tried classifying images' descriptors with SVM SVC linear kernel which gave bad ...
87 views

### PCA with one “known” component

(The title might not be very clear, any recommendation welcome) I have a $1 + n$ dimensional dataset. The first dimension measures a specific concept I am interested in and the $n$ remaining ...
284 views

### Quiz: Determine first principal component from data-plots

We see four data plots. The goal: How does the first principal component look for each plot a-d. For plot d, it is true that both clusters have same number of datapoints. First principal component ...
745 views

### Understanding direction of greatest variance in PCA

In the picture below, why is u the direction of greatest variance? Aren't the data points further away from v than from ...
597 views

### If I want to do PCA before k - means, is it mandatory to do it for all variables?

I have 10 variables and some of them are highly correlated. So before I do k - means, I want to get lower number of variables that are not correlated, but retain as much information as possible. Thus, ...
770 views

### Are principal components reflective, formative, both, or neither?

In reading various summaries on the similarities and differences between principal components and common factor models, I have noticed that there seems to be conflicting information about whether PCA ...
75 views

### What is the relationship between Mohr's Circle and Principal Component Analysis?

While I was studying PCA, I was told that it is related to Mohr's Circle. I don't know what that means. I don't know if they are related or not. I was just told, so I want to make sure here. If they ...
1k views

### Principal Component Analysis: how to interpret the total contribution of variables on several dimensions

When we calculate the total contribution of a variable for a single dimension, the sum of all single contributions is equal to 100%, which makes perfect sense. The http://www.sthda.com suggests to ...
496 views

### Making sense of PCA and t-SNE representations

I am wondering what (qualitative) sense one can make from PCA and t-SNE plots of weights in a neural network. I'm training a neural network that regresses five different control values, including a ...
192 views

### How does PCA maximise Total Variance without maximising Co-variance?

https://stats.stackexchange.com/a/3374/92071 - In PCA, the components are actual orthogonal linear combinations that maximize the total variance. In FA, the factors are linear combinations that ...
160 views

### Statistical significance for analysis/inference with PCA

I've got the following comment from a peer reviewer: the interpretations of PCA were described in not objective manner. The PC values can be statistically accessed and everything should be ...
1k views

### How to create a scree plot for factor analysis given that eigenvalues depend on the number of extracted factors?

I understand how Kaiser rule works for PCA, as no matter how many components I extract I always get the same eigenvalues. For example, with 3 components I get ...