Questions tagged [pca]
Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.
532
questions
12
votes
1
answer
5k
views
Why do all the PLS components together explain only a part of the variance of the original data?
I have a dataset consisting of 10 variables. I ran partial least squares (PLS) to predict a single response variable by these 10 variables, extracted 10 PLS components, and then computed the variance ...
- 407
12
votes
2
answers
6k
views
What are good metrics to assess the quality of a PCA fit, in order to select the number of components?
What is a good metric for assessing the quality of principal component analysis (PCA)?
I performed this algorithm on a dataset. My objective was to reduce the number of features (the information was ...
- 899
11
votes
4
answers
5k
views
Combining PCA, feature scaling, and cross-validation without training-test data leakage
The sci-kit learn documentation for cross-validation says the following about using feature-scaling and cross-validation:
Just as it is important to test a predictor on data held-out from training,...
- 293
11
votes
1
answer
9k
views
What exactly is the procedure to compute principal components in kernel PCA?
In kernel PCA (principal component analysis) you first choose a desired kernel, use it to find your $K$ matrix, center the feature space via the $K$ matrix, find its eigenvalues and eigenvectors, then ...
- 283
11
votes
2
answers
13k
views
Using principal components analysis vs correspondence analysis
I am analyzing a data set concerning intertidal communities. The data are percent cover (of seaweed, barnacles, mussels, etc) in quadrats. I am used to thinking about correspondence analysis (CA) in ...
- 1,256
11
votes
1
answer
1k
views
A paper mentions a "Monte Carlo simulation to determine the number of principal components"; how does it work?
I'm doing a Matlab analysis on MRI data where I have performed PCA on a matrix sized 10304x236 where 10304 is the number of voxels (think of them as pixels) and 236 is the number of timepoints. The ...
- 359
10
votes
3
answers
4k
views
Mahalanobis distance via PCA when $n<p$
I have an $n\times p$ matrix, where $p$ is the number of genes and $n$ is the number of patients. Anyone whose worked with such data knows that $p$ is always larger than $n$. Using feature selection I ...
- 1,611
10
votes
3
answers
16k
views
How to use principal components as predictors in GLM?
How would I use the output of a principal components analysis (PCA) in a generalized linear model (GLM), assuming the PCA is used for variable selection for the GLM?
Clarification: I want to use PCA ...
- 123
10
votes
1
answer
5k
views
Advice/literature on combining items with different response scales into composite scales?
Let's say I have some self-report items measured on a 5-point Likert scale (Strongly Disagree to Strongly Agree) and other items measured on a 4-point Likert scale (Never, Rarely, Sometimes, Often). ...
- 721
9
votes
1
answer
12k
views
Understanding cluster plot and component variability
I have run k-means clustering. I have also plotted the results using the following code in R:
...
- 389
9
votes
1
answer
1k
views
Will I miss anomalies/outliers due to PCA?
My final goal is to detect anomalies (outliers) in a high dimensional space. I was planning to use PCA to reduce the dimensionality as to be able to notice such anomalies better. But then I thought ...
- 2,597
9
votes
4
answers
10k
views
Very different results of principal component analysis in SPSS and Stata after rotation
For my PhD thesis I have to do a Principal Component Analysis (PCA). I didn't find it too difficult in Stata and was happy interpreting the results (I know there is a difference between factor and ...
- 93
9
votes
2
answers
3k
views
Visualizing multiple "histograms" (bar-charts)
I am having difficulties to select the right way to visualize data. Let's say we have bookstores that sells books, and every book has at least one category.
For a bookstore, if we count all the ...
- 413
9
votes
2
answers
16k
views
How to find which variables are most correlated with the first principal component?
I came across an article where the authors did a Principal Component Analysis on gene expression data, and found out the genes that are most correlated to the 1st principal component, and they used ...
- 309
9
votes
2
answers
6k
views
Why is the eigenvector in PCA taken to be unit norm?
In deriving the eigenvectors for PCA, the vector is subject to the condition that it should be of unit length. Why is this so?
- 373
9
votes
1
answer
7k
views
Fitting a plane to a set of points in 3D using PCA
I am trying to estimate a midplane of a 3D model using the midpoints of paired landmarks, in order to reconstruct missing data (midplane refers here to the middle/saggital plane of the cranium which ...
- 93
9
votes
3
answers
9k
views
Why with two classes, LDA gives only one dimension?
I am working with dimensionality reduction algorithms. Linear Discriminant Analysis (LDA) is a supervised algorithm that takes into account the class label (which is not the case of PCA for example). ...
- 103
8
votes
1
answer
643
views
Probabilistic models for partial least squares, reduced rank regression, and canonical correlation analysis?
This question results from the discussion following a previous question: What is the connection between partial least squares, reduced rank regression, and principal component regression?
For ...
- 415
8
votes
1
answer
357
views
Null distribution of subspaces similarity, or what is the distribution of $\mathrm{tr}(AA'BB')$?
What is the distribution of $\mathrm{tr}(AA'BB')$ where $A$ and $B$ are two random matrices of $d \times k$ size with orthonormal columns?
Maybe the expected value is easier to compute? A fallback ...
- 477
8
votes
1
answer
2k
views
Construct artificial slightly overlapping data for PCA plot
I am trying to construct artificial data which show two distinct groups in a PCA plot. However, the two groups should still slightly overlap. The following approaches came the closest but I am still ...
- 651
7
votes
2
answers
3k
views
Is principal component regression (PCR) using principal component scores for regression?
Principal component regression (PCR) in fact is regression on PC scores but not PCs. Why then in so many books and tutorials do they say something like,
in statistics, principal component ...
- 71
7
votes
2
answers
1k
views
How to do primary component analysis on multi-mode data with non-orthogonal primary components?
Consider the following picture representing the experimental data sequence obtained by two 1D-sensors (each point of the sequence is plotted on XY plane according to the respective sensor reading):
...
- 807
7
votes
3
answers
632
views
Regression in $p\gg N$ setting (predicting drug efficiency from gene expression with 30k predictors and ~30 samples)
I have a dataset of 29 cell lines and the IC50 values of a test drug. I want to find a relation between the gene expression profiles of each cell line (nearly 31000 genes) and the IC50 values.
My ...
- 141
7
votes
1
answer
5k
views
How is PCA applied to new data?
I understand the basic intuition behind PCA: reducing the dimensionality of data by finding the eigenvectors along which there is most variance in the data, and projecting the data along these ...
- 1,234
7
votes
0
answers
3k
views
Interpretation of biplot in PCA
Blue points all appear in the lower right-hand quadrant in the plane formed by the first two principal components.
Is it a good interpretation of the biplot (right panel) to say that blue points are ...
- 2,089
7
votes
3
answers
3k
views
Whether to use EFA or PCA to assess dimensionality of a set of Likert items
This follows on from my previous question on assessing reliability.
I designed a questionnaire (six 5-points Likert items) to evaluate the attitude of a group of users toward a product. I would like ...
- 509
6
votes
2
answers
2k
views
One component in PCA is always the mean vector in two-dimensions but not three [duplicate]
I've been testing PCA via SVD to decompose a simple time series data matrix, $X$. I have two signals $x_1(t)$ and $x_2(t)$ in a data matrix where $M$ rows represents each timepoint sample and each ...
- 751
6
votes
3
answers
1k
views
How can 8 dimensions be reduced to 3?
I can't seem to understand how PCA works. My (lack of) math knowledge don't help me either. I have read that the new set of variables has to be a linear combination of the old set.
What does that ...
- 161
6
votes
1
answer
2k
views
Fourier bases for a stationary signal & relation to PCA for natural images
Why does PCA of a translation-invariant signal give a Fourier basis?
I've found proofs for this, but I'd love some intuitions.
Any help is greatly appreciated!
EDIT:
Sorry that this question ...
- 103
5
votes
0
answers
2k
views
How to create a scree plot for factor analysis given that eigenvalues depend on the number of extracted factors?
I understand how Kaiser rule works for PCA, as no matter how many components I extract I always get the same eigenvalues.
For example, with 3 components I get
...
5
votes
1
answer
482
views
Interpreting PCA figures in layman terms
Image you have to present your PCA findings to a managerial board. An example:
After having read answers from here, I've understood that
The left and bottom axis are correlations
The right and ...
- 195
5
votes
2
answers
14k
views
PCA and FA example - calculation of communalities
I'm trying to understand how Principal Component Analysis and Factor Analysis work by
implementing examples. Although I'm mainly using Python and Numpy here, this isn't Python-specific, as I'd like to ...
- 287
5
votes
1
answer
4k
views
Fundamental difference between PCA and FA?
According to this, the fundamental difference between PCA and FA can be illustrated via the following image:
So, the direction of arrows changes.
According to this answer and a few others:
...
- 1,047
5
votes
3
answers
8k
views
Are dimensionality reduction techniques useful in deep learning?
I have been working on Machine learning and noticed that most of the time, dimensionality reduction techniques like PCA and t-SNE are used in machine learning. But, i rarely noticed anyone doing it ...
- 151
5
votes
1
answer
3k
views
What is the advantage of transforming variables from nominal to ordinal/numerical when it reduces variance explained in CatPCA?
Context
I have a dataset of 8 categorical variables. And I want to apply Categorical Principal Component Analysis (CatPCA).
Before doing that, I have been advised to look at the transformation plots ...
- 51
5
votes
1
answer
354
views
Does it make sense to use criteria from PCA to select the numbers of factors in a factor analysis?
Looking at both the practice of colleagues and also the practices instantiated in popular programs (e.g. SPSS, and commonly used syntax for SPSS), it seems common to use criteria based on a PCA to ...
4
votes
4
answers
1k
views
PCA, dimensionality, and k-means results: reaction to duplicating of variables
There are many excellent conversations on CV about the curse of dimensionality when applied to methods like k-means. The answer in the same post and other research (e.g., the paper titled "When Is ‘...
- 496
4
votes
1
answer
809
views
Using metric MDS with non-metric distances and assessing the fit quality
I'm going to perform MDS by means of cmdscale function of standard R library. I spent several hours googling it and finally have ...
- 439
4
votes
1
answer
679
views
PCA on count-based fractions, taking uncertainties into account
I'm looking to do a PCA analysis on count based data itself rather than averages. I'm hoping this will help for variable observation depths; for example, 3/4 reads is not really equivalent to 15/20. ...
- 608
4
votes
1
answer
2k
views
Do I need to run PCA over all predictors in a regression model? Can I run it only over the continuous ones?
I'm looking at the Lending Club data from Kaggle and I'm just building a pretty simple model to predict defaults.
The data has a large amount of both continuous and categorical variables (I have ...
- 278
4
votes
1
answer
5k
views
What is the difference between Exploratory Factor Analysis and Principal Components Analysis (PCA)? [duplicate]
I know what you're thinking, this is a duplicate of "What are the differences between Factor Analysis and Principal Component Analysis", but it isn't really.
That other question deals with ...
- 9,692
4
votes
1
answer
7k
views
How to avoid multicolinearity in SVM input data?
Do you know of any techniques that allows one to avoid and get rid of multicolinearity in SVM input data? We all know that if multicolinearity exists, explanatory variables have a high degree of ...
- 105
4
votes
1
answer
2k
views
Is it acceptable to rotate factors with PCA for binary data?
What issues, if any, might there be in rotating factors in order to obtain factor/component loadings of binary data? Is it acceptable to rotate the factors when doing a traditional PCA? (Assuming I’m ...
- 41
4
votes
1
answer
12k
views
How to use SVD for dimensionality reduction [duplicate]
After reading several "tutorials" on SVD I am left still wondering how to use it for dimensionality reduction.
Here is my confusion in an applied setting. If I limit svd to only considering the first ...
- 7,990
3
votes
2
answers
4k
views
What do empty (missing) values in the output of PCA or factor analysis mean in R?
I am doing factor analysis with factanal in R and observe empty spaces in the loadings table:
...
- 964
3
votes
1
answer
737
views
Removing nuisance PCA components from the fMRI data
So in attempting to replicate analysis pipeline from Tambini & Davachi, PNAS 2013, Persistence of hippocampal multivoxel patterns into
postencoding rest is related to memory I'm hoping to use PCA ...
- 359
3
votes
2
answers
1k
views
How to understand optimal Scaling in R: The Package homals for novices
Does anyone know of a step-by-step guide for the practical implementation of Gifi Methods for Optimal Scaling in R: The Package homals? Although I have an OK theoretical understanding (thanks chl for ...
- 71
3
votes
1
answer
8k
views
Can non normal data be used for factor analysis and multiple regression? If so what is the procedure to justify it?
While I was writing up the analysis in my thesis, I just came across when rechecking my test for normality, that the p-value for most continuous variables was .000, which is less than .05, and it ...
- 31
3
votes
1
answer
7k
views
PCA using princomp in MATLAB
I'm trying to do dimensionality reduction using matlab princomp, but i'm not sure i'm do it right.
here is the my code just for test, but I'm not sure that I'm doing projection right:
...
- 2,107
3
votes
1
answer
219
views
covariate selection in inference problems in logistic regression
For my specific problem, but a common situation in the medical field, I have several hundred patients, and about 10-20 exaplnatory variables. the goal is to examine a specific predictor("treatment") ...
- 307