Questions tagged [pca]
Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.
3,357
questions
1
vote
0
answers
568
views
Differentiable PCA? [closed]
Is there a differentiable method for dimensionality reduction that is either based on PCA or has the properties of:
Mathematically or algorithmically defined, e.g. not trained like an ML model or t-...
- 708
1
vote
1
answer
159
views
How to find complete log likelihood for mixture of PPCA
In Appendix C of a paper by Michael E. Tipping and Christopher M. Bishop about mixture models for probabilistic PCA, the probability of a single data vector $\mathbf{t}$ is expressed as a mixture of ...
- 5,522
19
votes
2
answers
84k
views
How to interpret PCA loadings?
While reading about PCA, I came across the following explanation:
Suppose we have a data set where each data point represents a single student's scores on a math test, a physics test, a reading ...
0
votes
1
answer
946
views
FAMD explained variance of components very low
I am dealing with a dataset composed of 50 features. There are both categorical (some with many levels, others dichotomous) and numerical features, so I decided to use FAMD in order to reduce the ...
CommunityBot
- 1
0
votes
1
answer
529
views
Which rotation type for principal component regression?
I would like to perform a principal component regression (PCR), but feel a little confused about the rotation type to be used in the principal component analysis (PCA) step.
First I perform a PCA to ...
CommunityBot
- 1
3
votes
1
answer
341
views
PCA Questions on the principal() function of psych package
I recently learned PCA and have the following questions on the use of principal() function of psych package:
From 20 variables I decided to keep 4 components / factors.
I used principal() function ...
CommunityBot
- 1
2
votes
4
answers
454
views
Why does second component have to be orthogonal to the first component in PCA?
PCA is done through series of orthogonal rotation. My impression of PCA precedure is: First component is on the direction of largest variance and second component is on the orthogonal direction to the ...
- 832
2
votes
1
answer
262
views
Adding/removing variables to PCA
If I have a PCA that I ran on some set of variables, how (if at all) will it relate to the PCA results if I add or remove one variable?
Will the PCA components change in some well-defined way, or is ...
CommunityBot
- 1
1
vote
1
answer
442
views
Clustering leading to visually overlapping clusters on scatterplot
I am dealing with a dataset with 13 features.
After going through some standard scaling and missing data imputation, I use kmeans from sklearn to create clusters.
Now the point is that, although the ...
CommunityBot
- 1
3
votes
1
answer
273
views
PCA for dimension reduction on repeated measures [closed]
I have a dataset with 1000 individuals who each came in for a minimum of 1 to a maximum of 10 visits during which we measured their arm strength in various directions (forward, back, sideways etc.). I ...
CommunityBot
- 1
0
votes
1
answer
330
views
Principal Component's Direction for a Matrix
Can anyone give a brief mathematical derivation on how to calculate principal components in PCA for a given covariance matrix let's say -
\begin{pmatrix}
5 & 2\\
2 & 5
\end{pmatrix}
?
CommunityBot
- 1
0
votes
1
answer
488
views
Multiple T Tests
On one of my questionnaires, which measure learning behaviours, after PCA, I have 4 subgroups, Efficacy/Perseverance/Effort/Achievement - I have run a T-Test and I have 4 sets of data comparing my ...
CommunityBot
- 1
0
votes
0
answers
27
views
Recommendations on best papers / blogs / existing literature on constructing risk and vulnerability indices?
I'm interested in constructing a risk index by indexing a large number of identified risk factors into a composite measure (that ideally then has sub-dimensions that can be explored further if one so ...
- 71.6k
1
vote
2
answers
452
views
PCA explained variance and clustering
Social scientist here with little background in stats. I have a question regarding a PCA I've carried out on my data. I have 17 variables catching different properties of neighbourhoods (geometrical ...
CommunityBot
- 1
1
vote
1
answer
50
views
Deterministic time-aggregation of principal component factors. Is it wrong?
I have estimated the first factor/score using PCA on a set of 190 monthly timeseries. For my analysis I also need the quarterly factor. Two choices come to mind:
Take the 3-month average of the ...
CommunityBot
- 1
0
votes
2
answers
1k
views
Best book for learn principal component analysis
Which book could you recommend for me to study principal component analysis at an intermediate level?
I have studied multivariate statistics, but I want to delve into this topic.
CommunityBot
- 1
0
votes
0
answers
20
views
How does centering the data reduce the risk of numerical problems when doing PCA?
In Mathematics for Machine Learning (page 336), the authors state that centering the data (subtracting from the data its the empirical mean) reduces the risk of numerical problems.
Which numerical ...
- 1,655
2
votes
2
answers
409
views
Principal component analysis in two dimensions
During my studies, I stumbled upon the following exercise:
We have the following joint probability distribution:
$$p(x,y) = p(x) p(y|x)$$
$$p(x) = \mathcal{N}(0,1), p(y \mid x) = \frac{1}{2} \delta(y ...
- 64.7k
4
votes
2
answers
337
views
Why is sum of squares equal to eigenvalue in PCA?
We fit a line or a hyperplane to a set of points. We project the points onto the hyperplane. The sum of squared distances of the projected points to origin is equal to the eigenvalue. Why is that?
CommunityBot
- 1
0
votes
0
answers
13
views
In a principal component space, is a change in any coordinate going to result in the same distance between PC-projected observations?
Suppose that we have observations in a 2-dimensional principal coordinate space. Let's denote one observation $\mathbf{y}=(y_1,y_2)$ in our PC space. Further, suppose that we have another observation $...
- 1,799
1
vote
1
answer
38
views
Latent Semantic Indexing vs. PCA
I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition.
Let's denote $X$ a $N \times D$ document-term matrix. The $D$ rows of $X$ ...
- 41k
1
vote
1
answer
802
views
Euclidean distance between points in PCA space along different principal component dimensions
I've picked up this project half way through, and I'm working through the last guy's code, so please bear with me.
So the original data consists of 500+ points in 150 dimensions, and I want to ...
CommunityBot
- 1
1
vote
1
answer
353
views
Interpretation of a PCA plot
I have a PC1-PC2 plot generated by applying PCA to the combination of 2 sets of samples where red means one set and blue means the other set.
I can see that the two sets of samples are very different ...
CommunityBot
- 1
0
votes
0
answers
62
views
Optimal predictive factors
Assume I am interested in predicting a time series variable $y_t$ using a vector of possible predictors $X_t$ of dimension $N_x$. I am interested in finding the optimal $N_z < N_x$ predictive ...
- 320
0
votes
0
answers
20
views
Do low-angle loadings necessarily indicate correlation between variables in PCA?
I keep reading that the angle between loadings in a PCA plot indicates some degree of correlation between the loadings (presumably lower angles lead to higher correlation, and vice versa).
I don't ...
- 249
1
vote
0
answers
39
views
How can I compare one full PCA model to two smaller ones?
I have nearly 30 variables going in to a large PCA, but the variables really fall into two conceptual categories. I want to test whether leaving all the variables to correlate freely with one another ...
- 11
1
vote
0
answers
31
views
How useful is PCA on its own? [closed]
In my machine learning course we have covered the key ideas behind principal component analysis. To round this part of the course off, we have learned to interpret the results of PCA, specifically ...
- 11
0
votes
2
answers
197
views
Model causality: graphical models and PCA
If we build a graphical model (DAG) we (may) interpret the arrows as causal dependences.
If we build a graphical model based on the variables returned by principal component analysis (PCA) we should ...
- 1,896
1
vote
0
answers
39
views
Interpreting results of PCA [duplicate]
I've recently started an introductory machine learning course and the first topic we have covered is prinicpal component analysis (PCA) and overall I am finding the whole topic quite tricky to wrap my ...
- 103
0
votes
1
answer
429
views
How to minimize influence of outliers in PCA for anomaly detection?
Let $\mathbf{X}$ be a dataset of size $n \times d$, where $n$ is the number of samples (days) and $d$ is the number of variables (daily observations). All observations are taken at the same times each ...
CommunityBot
- 1
1
vote
0
answers
45
views
PCA - Feature Scaling [closed]
I have been reading that the features should be standardized before performing PCA but I couldn't relate to my understanding of the same.
PCA try to project the dataset in the direction of maximum ...
- 21
0
votes
0
answers
22
views
Can we find dependence among Principal Components?
Principal Components (PCs) in PCA are linearly uncorrelated by definition.
However, uncorrelation does not imply independence (let aside the fact that each PC constructed at step t is necessarily part ...
- 401
2
votes
2
answers
194
views
What is the maximum number of dimensions in MDS?
If I have an arbitrary Euclidean distance matrix $D=(d_{ij}:i=1,\ldots, I; j=1,\ldots, I)$ and I want to reconstruct its elements (pairwise Euclidean distances) via classical Euclidean MDS. That is ...
- 501
0
votes
0
answers
22
views
Definition of Generalized of Low-Rank model
Recently, I looked into this paper "Generalized Low-Rank models" and found it very interesting as it gives general perspective on Principal Component Analysis (PCA) related methods.
In the ...
- 71.6k
7
votes
3
answers
2k
views
Can PC1 explain more than 90% of variance?
I've been running some analysis using morphological data. However, when I run a PCA, PC1 explains more than 90% of variance. I'm not sure if this is possible. If not, what should I need to do?
- 38.7k
2
votes
1
answer
245
views
Can I use combination of eigenvectors as a single vector to explain most of variance?
I have a problem trying to find a combination (or weighted average) of variables (statistics) that best explains the sample statistics.
A – n x p matrix (n: observations p: variables, here are ...
CommunityBot
- 1
1
vote
1
answer
748
views
Creating a composite indicator using PCA, CFA or addition
I need some advice to decide which aggregation method is better in my case. I want to create an aggregated variable which is a composite of 16 variables. All 16 variables are part of one big concept, ...
CommunityBot
- 1
0
votes
1
answer
658
views
Frequency parameter of robust PCA for anomaly detection
I am using the R implementation of robust PCA here for anomaly detection.
I have a vector of time series data, and a vector of dates. The algorithm works fine when the length of the vector is a ...
CommunityBot
- 1
0
votes
2
answers
381
views
Non-linear transformation to increase separability between clusters
I want to do a classification on PC scores. I have a $400$ dimensional matrix, e.g. $2000\times 400$ ($2000$ number of samples and $400$ dimensions). I first apply PCA on it and take it to 3D, i.e. $...
CommunityBot
- 1
0
votes
1
answer
36
views
Interpretation and application of PCA - positive factor patterns for first component for all variables
I am attempting to reduce the number of variables in a dataset for regression purposes and I suspect that many of the variables are correlated. Hence, I attempt a PCA, which I must admit I'm very new ...
- 369
1
vote
0
answers
25
views
Distribute feature importance to the components of the features in a PCA regression?
I read some interesting speculation over on the Data Science Stack. The setup is that there are multiple correlated features in a regression problem, and the goal is to determine feature importance.
...
- 51.1k
0
votes
1
answer
323
views
PCA: should standardization be applied on features or samples?
I am struggling a little bit with PCA.
I understand that standardization is an important part of the algorithm but I do not understand which elements should be standardized. Let's say I have a 10x100 ...
CommunityBot
- 1
2
votes
0
answers
42
views
(PCA) Is it possible for PCs loading scores to completely change sign after adding data? [duplicate]
I recently ran a PCA on a dataset of self-report data from 226 subjects to zoom in on which specific individual differences might account for participants’ predicted choices in a separate task we have ...
- 21
0
votes
0
answers
14
views
Time invariance when the latent structure is unknown
Imagine that participants completed a series of measures indexing different abilities (memory capacity, learning, etc.) at two timepoints.
The only thing I would like to test at this stage is whether ...
- 75
0
votes
0
answers
37
views
Maximize Variance of Linear Combination of Matrix Columns
Let $A$ be a $k \times 1$ random vector, and $\mathbf{A}$ be a $n \times k$ matrix of observations.
Letting $t \in \mathbb{R}^{k}$ be a vector of weights s.t. $||t||_2 = 1$, suppose we are interested ...
- 388
0
votes
2
answers
267
views
What is the proof that non-linearly separable data can't become linearly separable with the results of PCA?
Give a non-linearly separable dataset $X,$ I want to proof that after performing PCA on it, the resulting dataset is guaranteed to be still non-linearly separable.
I think we could argue that we still ...
- 64.7k
4
votes
1
answer
479
views
"PCA" based on distance metric other than $L_2$
PCA is based on $L_2$ distance and is maximizing variance along the PC axes.
What if we try a different distance measure (something else than $L_2$)?
Do any methods corresponding to PCA but with ...
- 41k
4
votes
2
answers
1k
views
Criteria for choosing between PCA and sparse PCA
A bit of a neophyte question:
I want to conduct data reduction on an NLP dataset 2000+ variables and 100000 plus cases.
I am looking at different data reduction techniques discussed in
"Robust ...
- 5,143
1
vote
0
answers
47
views
How to visualize time series using PCA?
I have two multivariate data sets comprised of 100s of time series, one is the actual recorded data set of time series and the other is a synthetically generated data set based on the recorded one.
...
- 31
0
votes
0
answers
40
views
PCA with non-Euclidean norm?
I have a dataset $X$ which I want to perform PCA on- however, I don't care so much about explaining the variance of certain features. So instead of using the "normal" definition of ...
