Questions tagged [pca]
Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.
3,338
questions
2
votes
2
answers
28
views
Does the average eigenvalue equals 1 in PCA applied to standardised data?
From what I understood when we are doing PCA, we can work both with raw or standardised data, depending on the situation we're in. Is it true that the average of the eigenvalues is equal to 1 when we ...
- 8,246
1
vote
1
answer
21
views
Can I apply Kaiser Rule without knowing the eigenvalues?
Kaiser's rule suggests the number of principal components to be included in an analysis by looking at eigenvalues. If I'm given standard deviations only, instead of eigenvalues, can I still somehow ...
- 8,246
3
votes
2
answers
1k
views
Is it OK to tune the k parameter in PCA?
Principal Component Analysis (PCA) is used to reduce n-dimensional data to k-dimensional data to speed things up in machine learning. After PCA is applied, one can check how much of the variance of ...
Tim♦
- 128k
0
votes
0
answers
6
views
Clustering sparse dataset with mix of continuous and categorical variables
I am trying to cluster sparse heterogeneous datasets containing demographics and diagnosis variables ( mix of categorical and numerical variables).
How should I start my clustering endeavors ? start ...
0
votes
0
answers
20
views
Creating a standardized composite "score" made up of multiple continuous, dependent variables for analysis in SPSS
I am trying to evaluate how a surgical intervention (insertion of a spinal fusion cage), resulting in distance changes between two vertebral bodies, led to new symptoms (yes/no) in some patients but ...
0
votes
0
answers
39
views
Plotting all the points of PCA to only one PCA axis, first PC1 and then on the PC2
I am required to project all the points on PCA to PC1 axis and then on PC2 axis as to see whether there is a good separation between the points. Through this I have to determine which dimension can ...
- 1
1
vote
0
answers
27
views
duplicated variables for different components
I'm presently evaluating the position of individuals of an 3 populations of an animal (according to their sexe) in function of the environmental factors (12) present in their habitat. To detect which ...
- 11
12
votes
8
answers
2k
views
Practical usefulness of PCA
I asked a similar question in the past, but I've thought about the message I am trying to convey a bit more and feel I can articulate it better. For context, I am on an introductory course in machine ...
- 1,109
0
votes
1
answer
121
views
How to use slopes in PCA?
I would like to use slope values in PCA. The problem I face is that the slopes I calculate per group could be within different ranges of values. We know that it is important to normalize your data ...
- 932
0
votes
1
answer
38
views
Can I perform multiple Kruskal-Wallis tests with different explanatory variables against the same response variable?
My data is observational data, and that's made it all kinds of ugly, and I can't decide what statistical test is needed. I have one response variable, which is categorical (Species 1, Species 2, or ...
- 437
0
votes
1
answer
2k
views
Principal Component Analysis with time series and index construction
I am doing a pca analysis to construct a financial stress index from different variables which I expect they will move together in a period of "financial stress". As I have read in different papers I ...
CommunityBot
- 1
25
votes
2
answers
723
views
Blind source separation of convex mixture?
Suppose I have $n$ independent sources, $X_1, X_2, ..., X_n$ and I observe $m$ convex mixtures:
\begin{align}
Y_1 &= a_{11}X_1 + a_{12}X_2 + \cdots + a_{1n}X_n\\
...&\\
Y_m &= a_{m1}X_1 + ...
- 12.2k
0
votes
0
answers
26
views
What does the quality of representation of a variable mean in PCA?
I understand that the quality of representation of an individual by a certain axis is measured by the cosine of the angle between the axis and the individual; the more the vector representing the ...
- 208
1
vote
1
answer
2k
views
Dealing with high dimension in principal component analysis
Does PCA work well or does it work at all in extremely high-dimensional problems, i.e. when the number of dimensions $p$ is larger than the sample size $N$? By 'work' I mean if it works mathematically....
- 8,246
3
votes
1
answer
1k
views
PCA with variables in different Likert scales
Can I run PCA (principal components analysis) if my Likert variables are measured differently? E.g. one Likert variable is in the range 0-6, another one is in the range 0-10 and yet another one is in ...
0
votes
1
answer
1k
views
Checking Multicollinearity and building a classification model when dependent is a factor and other independent variables are numerical in r
Problem statement
Y - Dependent variable is a factor (with levels A, B, and C)
Independent variables are all numerical variables.
Important: I have only 70 data points.
End Goal: Building a ...
CommunityBot
- 1
17
votes
5
answers
47k
views
Meaning of "reconstruction error" in PCA and LDA
I am implementing PCA and LDA for compression and classification respectively (implementing both an LDA for compression and classification).
I have the code written and everything works. What I need ...
- 316
8
votes
1
answer
897
views
Is it okay to perform PerMANOVA on PCA values?
Is it acceptable to first perform principal components analysis on a dataset, and then use permutational MANOVA on those principal components values, rather than on the original values in the dataset? ...
- 1
3
votes
2
answers
2k
views
PCA: How can the first principal component both maximize variance AND define the line that most closely fits the data?
I'm reviewing Chapter 6 from An Introduction to Statistical Learning. I'm having trouble understanding PCA and the provided example.
Can someone explain how the first principal component direction ...
1
vote
0
answers
527
views
Differentiable PCA? [closed]
Is there a differentiable method for dimensionality reduction that is either based on PCA or has the properties of:
Mathematically or algorithmically defined, e.g. not trained like an ML model or t-...
- 698
1
vote
3
answers
533
views
Why does more variance imply more information?
I just started to read about PCA in machine learning , and got to know that the main goal to determine principal components is to maximize variance so that more information is retained.But, why does ...
- 5,137
562
votes
3
answers
439k
views
Relationship between SVD and PCA. How to use SVD to perform PCA?
Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix ...
CommunityBot
- 1
1
vote
1
answer
155
views
How to find complete log likelihood for mixture of PPCA
In Appendix C of a paper by Michael E. Tipping and Christopher M. Bishop about mixture models for probabilistic PCA, the probability of a single data vector $\mathbf{t}$ is expressed as a mixture of ...
- 5,137
19
votes
2
answers
82k
views
How to interpret PCA loadings?
While reading about PCA, I came across the following explanation:
Suppose we have a data set where each data point represents a single student's scores on a math test, a physics test, a reading ...
0
votes
1
answer
773
views
FAMD explained variance of components very low
I am dealing with a dataset composed of 50 features. There are both categorical (some with many levels, others dichotomous) and numerical features, so I decided to use FAMD in order to reduce the ...
CommunityBot
- 1
0
votes
0
answers
10
views
How to understand eigenvalue equation from inner product point of view? [migrated]
In equation (14.4) page 429 of scholkopf's Learning with Kernels book, it states that the eigenvalue equation $\lambda v = C v$ is equivalent to $\lambda \langle x_i, v \rangle = \langle x_i, C v \...
- 1
0
votes
1
answer
511
views
Which rotation type for principal component regression?
I would like to perform a principal component regression (PCR), but feel a little confused about the rotation type to be used in the principal component analysis (PCA) step.
First I perform a PCA to ...
CommunityBot
- 1
3
votes
1
answer
297
views
PCA Questions on the principal() function of psych package
I recently learned PCA and have the following questions on the use of principal() function of psych package:
From 20 variables I decided to keep 4 components / factors.
I used principal() function ...
CommunityBot
- 1
2
votes
4
answers
414
views
Why does second component have to be orthogonal to the first component in PCA?
PCA is done through series of orthogonal rotation. My impression of PCA precedure is: First component is on the direction of largest variance and second component is on the orthogonal direction to the ...
- 822
2
votes
1
answer
255
views
Adding/removing variables to PCA
If I have a PCA that I ran on some set of variables, how (if at all) will it relate to the PCA results if I add or remove one variable?
Will the PCA components change in some well-defined way, or is ...
CommunityBot
- 1
1
vote
1
answer
290
views
Clustering leading to visually overlapping clusters on scatterplot
I am dealing with a dataset with 13 features.
After going through some standard scaling and missing data imputation, I use kmeans from sklearn to create clusters.
Now the point is that, although the ...
CommunityBot
- 1
3
votes
1
answer
241
views
PCA for dimension reduction on repeated measures [closed]
I have a dataset with 1000 individuals who each came in for a minimum of 1 to a maximum of 10 visits during which we measured their arm strength in various directions (forward, back, sideways etc.). I ...
CommunityBot
- 1
0
votes
1
answer
256
views
Principal Component's Direction for a Matrix
Can anyone give a brief mathematical derivation on how to calculate principal components in PCA for a given covariance matrix let's say -
\begin{pmatrix}
5 & 2\\
2 & 5
\end{pmatrix}
?
CommunityBot
- 1
0
votes
1
answer
485
views
Multiple T Tests
On one of my questionnaires, which measure learning behaviours, after PCA, I have 4 subgroups, Efficacy/Perseverance/Effort/Achievement - I have run a T-Test and I have 4 sets of data comparing my ...
CommunityBot
- 1
0
votes
0
answers
27
views
Recommendations on best papers / blogs / existing literature on constructing risk and vulnerability indices?
I'm interested in constructing a risk index by indexing a large number of identified risk factors into a composite measure (that ideally then has sub-dimensions that can be explored further if one so ...
- 71k
1
vote
2
answers
440
views
PCA explained variance and clustering
Social scientist here with little background in stats. I have a question regarding a PCA I've carried out on my data. I have 17 variables catching different properties of neighbourhoods (geometrical ...
CommunityBot
- 1
1
vote
1
answer
45
views
Deterministic time-aggregation of principal component factors. Is it wrong?
I have estimated the first factor/score using PCA on a set of 190 monthly timeseries. For my analysis I also need the quarterly factor. Two choices come to mind:
Take the 3-month average of the ...
CommunityBot
- 1
0
votes
2
answers
1k
views
Best book for learn principal component analysis
Which book could you recommend for me to study principal component analysis at an intermediate level?
I have studied multivariate statistics, but I want to delve into this topic.
CommunityBot
- 1
0
votes
0
answers
19
views
How does centering the data reduce the risk of numerical problems when doing PCA?
In Mathematics for Machine Learning (page 336), the authors state that centering the data (subtracting from the data its the empirical mean) reduces the risk of numerical problems.
Which numerical ...
- 1,655
2
votes
2
answers
390
views
Principal component analysis in two dimensions
During my studies, I stumbled upon the following exercise:
We have the following joint probability distribution:
$$p(x,y) = p(x) p(y|x)$$
$$p(x) = \mathcal{N}(0,1), p(y \mid x) = \frac{1}{2} \delta(y ...
- 61.5k
4
votes
2
answers
257
views
Why is sum of squares equal to eigenvalue in PCA?
We fit a line or a hyperplane to a set of points. We project the points onto the hyperplane. The sum of squared distances of the projected points to origin is equal to the eigenvalue. Why is that?
CommunityBot
- 1
0
votes
0
answers
11
views
In a principal component space, is a change in any coordinate going to result in the same distance between PC-projected observations?
Suppose that we have observations in a 2-dimensional principal coordinate space. Let's denote one observation $\mathbf{y}=(y_1,y_2)$ in our PC space. Further, suppose that we have another observation $...
- 1,777
1
vote
1
answer
32
views
Latent Semantic Indexing vs. PCA
I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition.
Let's denote $X$ a $N \times D$ document-term matrix. The $D$ rows of $X$ ...
- 39.3k
1
vote
1
answer
609
views
Euclidean distance between points in PCA space along different principal component dimensions
I've picked up this project half way through, and I'm working through the last guy's code, so please bear with me.
So the original data consists of 500+ points in 150 dimensions, and I want to ...
CommunityBot
- 1
1
vote
1
answer
338
views
Interpretation of a PCA plot
I have a PC1-PC2 plot generated by applying PCA to the combination of 2 sets of samples where red means one set and blue means the other set.
I can see that the two sets of samples are very different ...
CommunityBot
- 1
0
votes
0
answers
49
views
Optimal predictive factors
Assume I am interested in predicting a time series variable $y_t$ using a vector of possible predictors $X_t$ of dimension $N_x$. I am interested in finding the optimal $N_z < N_x$ predictive ...
- 320
0
votes
0
answers
15
views
Do low-angle loadings necessarily indicate correlation between variables in PCA?
I keep reading that the angle between loadings in a PCA plot indicates some degree of correlation between the loadings (presumably lower angles lead to higher correlation, and vice versa).
I don't ...
- 249
1
vote
0
answers
17
views
How can I compare one full PCA model to two smaller ones?
I have nearly 30 variables going in to a large PCA, but the variables really fall into two conceptual categories. I want to test whether leaving all the variables to correlate freely with one another ...
- 11
1
vote
0
answers
30
views
How useful is PCA on its own? [closed]
In my machine learning course we have covered the key ideas behind principal component analysis. To round this part of the course off, we have learned to interpret the results of PCA, specifically ...
- 11
0
votes
2
answers
171
views
Model causality: graphical models and PCA
If we build a graphical model (DAG) we (may) interpret the arrows as causal dependences.
If we build a graphical model based on the variables returned by principal component analysis (PCA) we should ...
- 1,856