As of May 31, 2023, we have updated our Code of Conduct.

# Questions tagged [pca]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

3,357 questions
Filter by
Sorted by
Tagged with
1 vote
568 views

### Differentiable PCA? [closed]

Is there a differentiable method for dimensionality reduction that is either based on PCA or has the properties of: Mathematically or algorithmically defined, e.g. not trained like an ML model or t-...
1 vote
159 views

### How to find complete log likelihood for mixture of PPCA

In Appendix C of a paper by Michael E. Tipping and Christopher M. Bishop about mixture models for probabilistic PCA, the probability of a single data vector $\mathbf{t}$ is expressed as a mixture of ...
84k views

While reading about PCA, I came across the following explanation: Suppose we have a data set where each data point represents a single student's scores on a math test, a physics test, a reading ...
946 views

### FAMD explained variance of components very low

I am dealing with a dataset composed of 50 features. There are both categorical (some with many levels, others dichotomous) and numerical features, so I decided to use FAMD in order to reduce the ...
529 views

### Which rotation type for principal component regression?

I would like to perform a principal component regression (PCR), but feel a little confused about the rotation type to be used in the principal component analysis (PCA) step. First I perform a PCA to ...
341 views

### PCA Questions on the principal() function of psych package

I recently learned PCA and have the following questions on the use of principal() function of psych package: From 20 variables I decided to keep 4 components / factors. I used principal() function ...
454 views

### Why does second component have to be orthogonal to the first component in PCA?

PCA is done through series of orthogonal rotation. My impression of PCA precedure is: First component is on the direction of largest variance and second component is on the orthogonal direction to the ...
262 views

### Adding/removing variables to PCA

If I have a PCA that I ran on some set of variables, how (if at all) will it relate to the PCA results if I add or remove one variable? Will the PCA components change in some well-defined way, or is ...
1 vote
442 views

### Clustering leading to visually overlapping clusters on scatterplot

I am dealing with a dataset with 13 features. After going through some standard scaling and missing data imputation, I use kmeans from sklearn to create clusters. Now the point is that, although the ...
273 views

### PCA for dimension reduction on repeated measures [closed]

I have a dataset with 1000 individuals who each came in for a minimum of 1 to a maximum of 10 visits during which we measured their arm strength in various directions (forward, back, sideways etc.). I ...
330 views

### Principal Component's Direction for a Matrix

Can anyone give a brief mathematical derivation on how to calculate principal components in PCA for a given covariance matrix let's say - \begin{pmatrix} 5 & 2\\ 2 & 5 \end{pmatrix} ?
488 views

### Multiple T Tests

On one of my questionnaires, which measure learning behaviours, after PCA, I have 4 subgroups, Efficacy/Perseverance/Effort/Achievement - I have run a T-Test and I have 4 sets of data comparing my ...
27 views

### Recommendations on best papers / blogs / existing literature on constructing risk and vulnerability indices?

I'm interested in constructing a risk index by indexing a large number of identified risk factors into a composite measure (that ideally then has sub-dimensions that can be explored further if one so ...
1 vote
452 views

### PCA explained variance and clustering

Social scientist here with little background in stats. I have a question regarding a PCA I've carried out on my data. I have 17 variables catching different properties of neighbourhoods (geometrical ...
1 vote
50 views

### Deterministic time-aggregation of principal component factors. Is it wrong?

I have estimated the first factor/score using PCA on a set of 190 monthly timeseries. For my analysis I also need the quarterly factor. Two choices come to mind: Take the 3-month average of the ...
1k views

### Best book for learn principal component analysis

Which book could you recommend for me to study principal component analysis at an intermediate level? I have studied multivariate statistics, but I want to delve into this topic.
20 views

### How does centering the data reduce the risk of numerical problems when doing PCA?

In Mathematics for Machine Learning (page 336), the authors state that centering the data (subtracting from the data its the empirical mean) reduces the risk of numerical problems. Which numerical ...
409 views

### Principal component analysis in two dimensions

During my studies, I stumbled upon the following exercise: We have the following joint probability distribution: $$p(x,y) = p(x) p(y|x)$$ p(x) = \mathcal{N}(0,1), p(y \mid x) = \frac{1}{2} \delta(y ...
337 views

### Why is sum of squares equal to eigenvalue in PCA?

We fit a line or a hyperplane to a set of points. We project the points onto the hyperplane. The sum of squared distances of the projected points to origin is equal to the eigenvalue. Why is that?
13 views

36 views

### Interpretation and application of PCA - positive factor patterns for first component for all variables

I am attempting to reduce the number of variables in a dataset for regression purposes and I suspect that many of the variables are correlated. Hence, I attempt a PCA, which I must admit I'm very new ...
1 vote
25 views

### Distribute feature importance to the components of the features in a PCA regression?

I read some interesting speculation over on the Data Science Stack. The setup is that there are multiple correlated features in a regression problem, and the goal is to determine feature importance. ...
323 views

### PCA: should standardization be applied on features or samples?

I am struggling a little bit with PCA. I understand that standardization is an important part of the algorithm but I do not understand which elements should be standardized. Let's say I have a 10x100 ...
42 views

### (PCA) Is it possible for PCs loading scores to completely change sign after adding data? [duplicate]

I recently ran a PCA on a dataset of self-report data from 226 subjects to zoom in on which specific individual differences might account for participants’ predicted choices in a separate task we have ...
14 views

### Time invariance when the latent structure is unknown

Imagine that participants completed a series of measures indexing different abilities (memory capacity, learning, etc.) at two timepoints. The only thing I would like to test at this stage is whether ...
37 views

### Maximize Variance of Linear Combination of Matrix Columns

Let $A$ be a $k \times 1$ random vector, and $\mathbf{A}$ be a $n \times k$ matrix of observations. Letting $t \in \mathbb{R}^{k}$ be a vector of weights s.t. $||t||_2 = 1$, suppose we are interested ...
267 views

### What is the proof that non-linearly separable data can't become linearly separable with the results of PCA?

Give a non-linearly separable dataset $X,$ I want to proof that after performing PCA on it, the resulting dataset is guaranteed to be still non-linearly separable. I think we could argue that we still ...
479 views

### "PCA" based on distance metric other than $L_2$

PCA is based on $L_2$ distance and is maximizing variance along the PC axes. What if we try a different distance measure (something else than $L_2$)? Do any methods corresponding to PCA but with ...
1k views

### Criteria for choosing between PCA and sparse PCA

A bit of a neophyte question: I want to conduct data reduction on an NLP dataset 2000+ variables and 100000 plus cases. I am looking at different data reduction techniques discussed in "Robust ...
1 vote
I have a dataset $X$ which I want to perform PCA on- however, I don't care so much about explaining the variance of certain features. So instead of using the "normal" definition of ...