# Questions tagged [pca]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

552 questions
Filter by
Sorted by
Tagged with
8k views

### Assessing reliability of a questionnaire: dimensionality, problematic items, and whether to use alpha, lambda6 or some other index?

I am analyzing scores given by participants attending an experiment. I want to estimate the reliability of my questionnaire which is composed of 6 items aimed at estimating the attitude of the ...
• 519
10k views

### How does "Fundamental Theorem of Factor Analysis" apply to PCA, or how are PCA loadings defined?

I'm currently going through a slide set I have for "factor analysis" (PCA as far as I can tell). In it, the "fundamental theorem of factor analysis" is derived which claims that the correlation ...
• 593
23k views

### How to perform cross-validation for PCA to determine the number of principal components?

I'm trying to write my own function for principal component analysis, PCA (of course there's a lot already written but I'm just interested in implementing stuff by myself). The main problem I ...
• 652
9k views

### How to determine significant principal components using bootstrapping or Monte Carlo approach?

I am interested in determining the number of significant patterns coming out of a Principal Component Analysis (PCA) or Empirical Orthogonal Function (EOF) Analysis. I am particularly interested in ...
• 3,702
10k views

### PCA, LDA, CCA, and PLS

How are PCA, LDA, CCA, and PLS related? They all seem "spectral" and linear algebraic and very well understood (say 50+ years of theory built around them). They are used for very different things (PCA ...
• 3,245
35k views

### Reversing PCA back to the original variables [duplicate]

I have a set of data that has $n$ samples described by $m$ variables. I do a PCA to reduce it to just 2 dimensions so I can make a nice 2D plot of the data. I understand that the $x,y$ coordinates (i....
• 373
22k views

### PCA of non-Gaussian data

I have a couple of quick questions about PCA: Does the PCA assume that the dataset is Gaussian? What happens when I apply a PCA to inherently non-linear data? Given a dataset, the process is to ...
• 1,200
17k views

### What is the "horseshoe effect" and/or the "arch effect" in PCA / correspondence analysis?

There are many techniques in ecological statistics for exploratory data analysis of multidimensional data. These are called 'ordination' techniques. Many are the same or closely related to common ...
5k views

### What is the connection between partial least squares, reduced rank regression, and principal component regression?

Are reduced rank regression and principal component regression just special cases of partial least squares? This tutorial (Page 6, "Comparison of Objectives") states that when we do partial least ...
• 465
28k views

### Is there any required amount of variance captured by PCA in order to do later analyses?

I have a dataset with 11 variables and PCA (orthogonal) was done to reduce the data. Deciding on the number of components to keep it was evident for me from my knowledge about the subject and the ...
• 1,135
39k views

### Why are principal components in PCA (eigenvectors of the covariance matrix) mutually orthogonal? [duplicate]

Why are principal components in PCA mutually orthogonal? I know that PCA can be calculated by eig(cov(X)), where X is centered. ...
• 331
13k views

### Can I do a PCA on repeated measures for data reduction?

I have 3 trials each on 87 animals in each of 2 contexts (some missing data; no missing data = 64 animals). Within a context, I have many specific measures (time to enter, number of times returning ...
• 221
42k views

### Normalizing vs Scaling before PCA

I know there's a lot of content about PCA pre-processing, but I am still somewhat confused. I have a dataset that contains some clear patterns: 1 variable is whether a person has financial resources (...
• 431
9k views

### Is it acceptable to reverse a sign of a principal component score? [duplicate]

I have two datasets from similar psycholinguistic experiments. In both of them, information about the participant's reading and spelling ability was collected, then converted into standardized scores ...
• 506
10k views

### Should PCA be performed before I do classification?

I have got a problem about doing a classification. I have got around 50 datasets. Each of them has 15 features. I am trying to use these features to classify the 50 datasets to either 'Good' or 'Bad'....
• 1,749
12k views

### How to project a new vector onto the PC space using kernel PCA?

Let $X_{N \times d}$ be the data matrix, where $N$ is the number of samples and $d$ the size of the features space. Using kernel PCA (kPCA), one first computes a kernel matrix $K_{N \times N}$, and ...
• 555
4k views

### Volume of the 95% confidence ellipsoid

I'm dealing with 3D data that are the trajectory of a point over time. I would like to have an indication of how much it is "spread" in space and I thought about using the volume of the 95% confidence ...
• 311
3k views

### Can averaging all the variables be seen as a crude form of PCA?

This question just occurred to me out of the blue. PCA is a way to reduce dimensions. Another way that is often (perhaps too often) used is to take mean values of two or more variables. This is done a ...
• 1,109
18k views

### What's the relationship between initial eigenvalues and sums of squared loadings in factor analysis?

On the one hand I read in a comment here that: You can't speak of "eigenvalues" after rotation, even orthogonal rotation. Perhaps you mean sum of squared loadings for a principal component, ...
3k views

### How to compute PCA scores from eigendecomposition of the covariance matrix?

Given a data matrix $\mathbf X$ of $12 \times 7$ size with samples in rows and variables in columns, I have calculated centered data $\mathbf X_c$ by subtracting column means, and then computed ...
5k views

### Categorical Principal Component Analysis - using Count, Continuous, Ordinal variables together

I have some variables and I want to reduce their number for further analysis. I initially thought of combining them using factor analysis. But since the variables are of all kinds (rating, count, ...
• 53
6k views

### Is it valid to perform PCA if Kaiser-Meyer-Olkin (KMO) index is very low?

I have a dataset that contains data from $307$ subjects and nine variables for each subject. I would like to run a PCA. My problem is that I get a Kaiser-Meyer-Olkin (KMO) value of $0.06$. Can it be ...
• 33
13k views

### Is PCA always recommended?

I was wondering if PCA can be always applied for dimensionality reduction before a classification or regression problem. My intuition tells me that the answer is no. If we perform PCA then we ...
• 744
43k views

### PCA when the dimensionality is greater than the number of samples

I've come across a scenario where I have 10 signals/person for 10 people (so 100 samples) containing 14000 data points (dimensions) that I need to pass to a classifier. I would like to reduce the ...
• 345
18k views

### Is there any advantage of SVD over PCA?

I know how to calculate PCA and SVD mathematically, and I know that both can be applied to Linear Least Squares regression. The main advantage of SVD mathematically seems to be that it can be applied ...
• 1,763
30k views

### Not normalizing data before PCA gives better explained variance ratio

I normalized my dataset then ran 3 component PCA to get small explained variance ratios ([0.50, 0.1, 0.05]). When I didn't normalize but whitened my dataset then ran 3 component PCA, I got high ...
32k views

### Why log-transforming the data before performing principal component analysis?

Im following a tutorial here: http://www.r-bloggers.com/computing-and-visualizing-pca-in-r/ to gain a better understanding of PCA. The tutorial uses the Iris dataset and applies a log transform prior ...
3k views

### The limit of "unit-variance" ridge regression estimator when $\lambda\to\infty$

Consider ridge regression with an additional constraint requiring that $\hat{\mathbf y}$ has unit sum of squares (equivalently, unit variance); if needed, one can assume that $\mathbf y$ has unit sum ...
• 106k
3k views

### Geometric understanding of PCA in the subject (dual) space

I am trying to get an intuitive understanding of how principal component analysis (PCA) works in subject (dual) space. Consider 2D dataset with two variables, $x_1$ and $x_2$, and $n$ data points (...
• 106k
12k views

### Supervised dimensionality reduction

I have a data set consisting of 15K labeled samples (of 10 groups). I want to apply dimensionality reduction into 2 dimensions, that would take into consideration the knowledge of the labels. When I ...
• 839
8k views

### When do we combine dimensionality reduction with clustering?

I am trying to perform document-level clustering. I constructed the term-document frequency matrix and I am trying to cluster these high dimensional vectors using k-means. Instead of directly ...
• 4,542
8k views

### Doing CCA vs. building a dependent variable with PCA and then doing regression

Given two multidimensional datasets, $X$ and $Y$, some people perform multivariable analysis by building a surrogate dependent variable using principal component analysis (PCA). That is, run PCA on $Y$...
• 2,240
5k views

### The first principal component does not separate classes, but other PCs do; how is that possible?

I ran PCA on 17 quantitative variables in order to obtain a smaller set of variables, that is principal components, to be used in supervised machine learning for classifying instances into two classes....
• 155
21k views

### Arrows of underlying variables in PCA biplot in R

At the risk of making the question software-specific, and with the excuse of its ubiquity and idiosyncrasies, I want to ask about the function biplot() in R, and, ...
• 26.6k
7k views

### Which independent variables are most important in predicting the response variable? [duplicate]

I'm a biologist, and I have a large dataset that I'm trying to analyze. Here are the variables I'm working with: levels of 211 different metabolites in 16 different blood samples (predictor variables)...
• 131
15k views

### On the use of oblique rotation after PCA

Several statistical packages, such as SAS, SPSS, and R, allow you to perform some kind of factor rotation following a PCA. Why is a rotation necessary after a PCA? Why would you apply an oblique ...
• 1,251
4k views

### Does the first principal component differ from simply computing the mean of all variables?

I was just wondering if the first principal component, while I am trying to find it for a dataset of 18 variables, is different from simply adding all variables and finding the mean? I.e. to compute ...
• 127
12k views

### Proportion of explained variance in PCA and LDA

I have some basic questions regarding PCA (principal component analysis) and LDA (linear discriminant analysis): In PCA there is a way to calculate the proportion of variance explained. Is it also ...
• 111
9k views

### Is it possible to use kernel PCA for feature selection?

Is it possible to use kernel principal component analysis (kPCA) for Latent Semantic Indexing (LSI) in the same way as PCA is used? I perform LSI in R using the ...
• 101
5k views

### PCA: Eigenvectors of opposite sign and not being able to compute eigenvectors with solve in R

I'm learning PCA in R language. I met two problems right now that I don't understand. I am performing a PCA analysis in R on a 318×17 dataset using some custom code. I take eigen function in R to ...
• 83
44k views

### What is the difference between PCA and PAF method in factor analysis?

What is the difference between principal component analyses (PCA) and principal axis factoring (PAF)? Also, I understand the difference between varimax and oblimin rotations, but is that the same as ...
2k views

### Is dimensionality reduction almost always useful for classification?

Is singular value decomposition almost always useful in practice for enhancing the predicative power of a trained classification model? E.x. A dataset for classification has 20,000 features. Run SVD ...
• 858
2k views

### Choosing number of PCA components when multiple samples for each data point are available

Update: I posted an answer below describing my current attempts to approach this problem. I am facing a perennial problem of identifying significant principal components. This question has been ...
• 106k
992 views

### Scree plot: $m$ vs $m-1$ components/factors

@ttnphns comments here that there exist two expositions of the Cattell scree-plot rule: If the "elbow" is the m-th eigenvalue, (1) choose to extract m components; or (2) choose to extract m-...
3k views

### Strange results of varimax rotation of principal component analysis in Stata: rotated components are all zeros and ones

This is my initial output of Principal Component Analysis (PCA) using Stata and correlation matrix (because different scales and measurement units of inputs): <...
• 4,271
12k views

### What are the units in this PCA biplot? [duplicate]

This is a plot of my data These are the values: ...
• 151
18k views

### Use of PCA analysis to select variables for a regression analysis [duplicate]

I have too many environmental variables to use in a multiple regression analysis. If I use all the variables the models are just too complex. The use of the PCA axes in the regression analysis was ...
• 21
59k views

### Are there cases where PCA is more suitable than t-SNE?

I want to see how 7 measures of text correction behaviour (time spent correcting the text, number of keystrokes, etc.) relate to each other. The measures are correlated. I ran a PCA to see how the ...
• 897
30k views

### Why is t-SNE not used as a dimensionality reduction technique for clustering or classification?

In a recent assignment, we were told to use PCA on the MNIST digits to reduce the dimensions from 64 (8 x 8 images) to 2. We then had to cluster the digits using a Gaussian Mixture Model. PCA using ...
• 643