# Questions tagged [pca]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

532 questions
Filter by
Sorted by
Tagged with
5k views

### Why do all the PLS components together explain only a part of the variance of the original data?

I have a dataset consisting of 10 variables. I ran partial least squares (PLS) to predict a single response variable by these 10 variables, extracted 10 PLS components, and then computed the variance ...
6k views

### What are good metrics to assess the quality of a PCA fit, in order to select the number of components?

What is a good metric for assessing the quality of principal component analysis (PCA)? I performed this algorithm on a dataset. My objective was to reduce the number of features (the information was ...
5k views

### Combining PCA, feature scaling, and cross-validation without training-test data leakage

The sci-kit learn documentation for cross-validation says the following about using feature-scaling and cross-validation: Just as it is important to test a predictor on data held-out from training,...
9k views

### What exactly is the procedure to compute principal components in kernel PCA?

In kernel PCA (principal component analysis) you first choose a desired kernel, use it to find your $K$ matrix, center the feature space via the $K$ matrix, find its eigenvalues and eigenvectors, then ...
13k views

### Using principal components analysis vs correspondence analysis

I am analyzing a data set concerning intertidal communities. The data are percent cover (of seaweed, barnacles, mussels, etc) in quadrats. I am used to thinking about correspondence analysis (CA) in ...
1k views

### A paper mentions a "Monte Carlo simulation to determine the number of principal components"; how does it work?

I'm doing a Matlab analysis on MRI data where I have performed PCA on a matrix sized 10304x236 where 10304 is the number of voxels (think of them as pixels) and 236 is the number of timepoints. The ...
4k views

### Mahalanobis distance via PCA when $n<p$

I have an $n\times p$ matrix, where $p$ is the number of genes and $n$ is the number of patients. Anyone whose worked with such data knows that $p$ is always larger than $n$. Using feature selection I ...
16k views

### How to use principal components as predictors in GLM?

How would I use the output of a principal components analysis (PCA) in a generalized linear model (GLM), assuming the PCA is used for variable selection for the GLM? Clarification: I want to use PCA ...
5k views

### Advice/literature on combining items with different response scales into composite scales?

Let's say I have some self-report items measured on a 5-point Likert scale (Strongly Disagree to Strongly Agree) and other items measured on a 4-point Likert scale (Never, Rarely, Sometimes, Often). ...
12k views

### Understanding cluster plot and component variability

I have run k-means clustering. I have also plotted the results using the following code in R: ...
1k views

### Will I miss anomalies/outliers due to PCA?

My final goal is to detect anomalies (outliers) in a high dimensional space. I was planning to use PCA to reduce the dimensionality as to be able to notice such anomalies better. But then I thought ...
10k views

### Very different results of principal component analysis in SPSS and Stata after rotation

For my PhD thesis I have to do a Principal Component Analysis (PCA). I didn't find it too difficult in Stata and was happy interpreting the results (I know there is a difference between factor and ...
3k views

### Visualizing multiple "histograms" (bar-charts)

I am having difficulties to select the right way to visualize data. Let's say we have bookstores that sells books, and every book has at least one category. For a bookstore, if we count all the ...
16k views

### How to find which variables are most correlated with the first principal component?

I came across an article where the authors did a Principal Component Analysis on gene expression data, and found out the genes that are most correlated to the 1st principal component, and they used ...
6k views

### Why is the eigenvector in PCA taken to be unit norm?

In deriving the eigenvectors for PCA, the vector is subject to the condition that it should be of unit length. Why is this so?
7k views

### Fitting a plane to a set of points in 3D using PCA

I am trying to estimate a midplane of a 3D model using the midpoints of paired landmarks, in order to reconstruct missing data (midplane refers here to the middle/saggital plane of the cranium which ...
9k views

### Why with two classes, LDA gives only one dimension?

I am working with dimensionality reduction algorithms. Linear Discriminant Analysis (LDA) is a supervised algorithm that takes into account the class label (which is not the case of PCA for example). ...
643 views

### Probabilistic models for partial least squares, reduced rank regression, and canonical correlation analysis?

This question results from the discussion following a previous question: What is the connection between partial least squares, reduced rank regression, and principal component regression? For ...
357 views

### Null distribution of subspaces similarity, or what is the distribution of $\mathrm{tr}(AA'BB')$?

What is the distribution of $\mathrm{tr}(AA'BB')$ where $A$ and $B$ are two random matrices of $d \times k$ size with orthonormal columns? Maybe the expected value is easier to compute? A fallback ...
2k views

### Construct artificial slightly overlapping data for PCA plot

I am trying to construct artificial data which show two distinct groups in a PCA plot. However, the two groups should still slightly overlap. The following approaches came the closest but I am still ...
3k views

### Is principal component regression (PCR) using principal component scores for regression?

Principal component regression (PCR) in fact is regression on PC scores but not PCs. Why then in so many books and tutorials do they say something like, in statistics, principal component ...
1k views

### How to do primary component analysis on multi-mode data with non-orthogonal primary components?

Consider the following picture representing the experimental data sequence obtained by two 1D-sensors (each point of the sequence is plotted on XY plane according to the respective sensor reading): ...
632 views

### Regression in $p\gg N$ setting (predicting drug efficiency from gene expression with 30k predictors and ~30 samples)

I have a dataset of 29 cell lines and the IC50 values of a test drug. I want to find a relation between the gene expression profiles of each cell line (nearly 31000 genes) and the IC50 values. My ...
5k views

### How is PCA applied to new data?

I understand the basic intuition behind PCA: reducing the dimensionality of data by finding the eigenvectors along which there is most variance in the data, and projecting the data along these ...
3k views

### Interpretation of biplot in PCA

Blue points all appear in the lower right-hand quadrant in the plane formed by the first two principal components. Is it a good interpretation of the biplot (right panel) to say that blue points are ...
3k views

### Whether to use EFA or PCA to assess dimensionality of a set of Likert items

This follows on from my previous question on assessing reliability. I designed a questionnaire (six 5-points Likert items) to evaluate the attitude of a group of users toward a product. I would like ...
2k views

### One component in PCA is always the mean vector in two-dimensions but not three [duplicate]

I've been testing PCA via SVD to decompose a simple time series data matrix, $X$. I have two signals $x_1(t)$ and $x_2(t)$ in a data matrix where $M$ rows represents each timepoint sample and each ...
1k views

### How can 8 dimensions be reduced to 3?

I can't seem to understand how PCA works. My (lack of) math knowledge don't help me either. I have read that the new set of variables has to be a linear combination of the old set. What does that ...
2k views

### Fourier bases for a stationary signal & relation to PCA for natural images

Why does PCA of a translation-invariant signal give a Fourier basis? I've found proofs for this, but I'd love some intuitions. Any help is greatly appreciated! EDIT: Sorry that this question ...
2k views

### How to create a scree plot for factor analysis given that eigenvalues depend on the number of extracted factors?

I understand how Kaiser rule works for PCA, as no matter how many components I extract I always get the same eigenvalues. For example, with 3 components I get ...
482 views

### Interpreting PCA figures in layman terms

Image you have to present your PCA findings to a managerial board. An example: After having read answers from here, I've understood that The left and bottom axis are correlations The right and ...
14k views

### PCA and FA example - calculation of communalities

I'm trying to understand how Principal Component Analysis and Factor Analysis work by implementing examples. Although I'm mainly using Python and Numpy here, this isn't Python-specific, as I'd like to ...
4k views

### Fundamental difference between PCA and FA?

According to this, the fundamental difference between PCA and FA can be illustrated via the following image: So, the direction of arrows changes. According to this answer and a few others: ...
8k views

### Are dimensionality reduction techniques useful in deep learning?

I have been working on Machine learning and noticed that most of the time, dimensionality reduction techniques like PCA and t-SNE are used in machine learning. But, i rarely noticed anyone doing it ...
3k views

### What is the advantage of transforming variables from nominal to ordinal/numerical when it reduces variance explained in CatPCA?

Context I have a dataset of 8 categorical variables. And I want to apply Categorical Principal Component Analysis (CatPCA). Before doing that, I have been advised to look at the transformation plots ...
354 views

### Does it make sense to use criteria from PCA to select the numbers of factors in a factor analysis?

Looking at both the practice of colleagues and also the practices instantiated in popular programs (e.g. SPSS, and commonly used syntax for SPSS), it seems common to use criteria based on a PCA to ...
1k views

### PCA, dimensionality, and k-means results: reaction to duplicating of variables

There are many excellent conversations on CV about the curse of dimensionality when applied to methods like k-means. The answer in the same post and other research (e.g., the paper titled "When Is ‘...
809 views

### Using metric MDS with non-metric distances and assessing the fit quality

I'm going to perform MDS by means of cmdscale function of standard R library. I spent several hours googling it and finally have ...
679 views

### PCA on count-based fractions, taking uncertainties into account

I'm looking to do a PCA analysis on count based data itself rather than averages. I'm hoping this will help for variable observation depths; for example, 3/4 reads is not really equivalent to 15/20. ...
2k views

### Do I need to run PCA over all predictors in a regression model? Can I run it only over the continuous ones?

I'm looking at the Lending Club data from Kaggle and I'm just building a pretty simple model to predict defaults. The data has a large amount of both continuous and categorical variables (I have ...
5k views

### What is the difference between Exploratory Factor Analysis and Principal Components Analysis (PCA)? [duplicate]

I know what you're thinking, this is a duplicate of "What are the differences between Factor Analysis and Principal Component Analysis", but it isn't really. That other question deals with ...
7k views

### How to avoid multicolinearity in SVM input data?

Do you know of any techniques that allows one to avoid and get rid of multicolinearity in SVM input data? We all know that if multicolinearity exists, explanatory variables have a high degree of ...
2k views

### Is it acceptable to rotate factors with PCA for binary data?

What issues, if any, might there be in rotating factors in order to obtain factor/component loadings of binary data? Is it acceptable to rotate the factors when doing a traditional PCA? (Assuming I’m ...
12k views

### How to use SVD for dimensionality reduction [duplicate]

After reading several "tutorials" on SVD I am left still wondering how to use it for dimensionality reduction. Here is my confusion in an applied setting. If I limit svd to only considering the first ...
4k views

### What do empty (missing) values in the output of PCA or factor analysis mean in R?

I am doing factor analysis with factanal in R and observe empty spaces in the loadings table: ...
737 views

### Removing nuisance PCA components from the fMRI data

So in attempting to replicate analysis pipeline from Tambini & Davachi, PNAS 2013, Persistence of hippocampal multivoxel patterns into postencoding rest is related to memory I'm hoping to use PCA ...
1k views

### How to understand optimal Scaling in R: The Package homals for novices

Does anyone know of a step-by-step guide for the practical implementation of Gifi Methods for Optimal Scaling in R: The Package homals? Although I have an OK theoretical understanding (thanks chl for ...
8k views

### Can non normal data be used for factor analysis and multiple regression? If so what is the procedure to justify it?

While I was writing up the analysis in my thesis, I just came across when rechecking my test for normality, that the p-value for most continuous variables was .000, which is less than .05, and it ...