Questions tagged [pca]
Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.
3,343
questions
0
votes
1
answer
34
views
Is this approach of PCA correct?
My project is based in a clinical trial in which we measured in gene expression in three groups (OO, NUTS, LFD). The individuals (n = 151) are almost equally distributed and the variables to measure ...
1
vote
0
answers
13
views
Given a psd matrix $Q$ and a kernel function $f(y_i, y_j)$, how do I find $Y \in \mathbb{R}^{n \times d}$ that best approximates $Q$? [duplicate]
The question is basically the title. I have a matrix $Q$ that I know is positive semi-definite. I now want to find the $Y$ that approximates this matrix under some kernel function $f(y_i, y_j)$. I ...
- 111
0
votes
1
answer
24
views
Smallest set of words contained in a number of documents
I have a collection of text documents. I would like to find the smallest set of words such that searching by those words allows discovering each document.
It is quite natural to describe this data as ...
- 939
0
votes
0
answers
7
views
Difference between LDA and LR? [duplicate]
I want a brief difference about Linear Discriminant Analysis and Linear Regression. Isnt it the same process? I heard the difference is that when it comes to multidimensional target/categories LDA is ...
- 1
0
votes
0
answers
13
views
Do we always want to do projection when we using PCA with SVD? [duplicate]
Assume that we have a matrix $X$ and we want to do Singular Value Decompotision(SVD) with $X$.
First we need to find the average of $X$ row wise, called $\mu$
...
- 193
0
votes
0
answers
11
views
Correlation or Covariance when applying PCA to standardized data?
I am trying to apply PCA to a data set. For context I have standardized my data using the "scale ()" function in R and now I am wondering what matrix I should use in PCA?
- 1
2
votes
2
answers
40
views
Can one variable load onto different components in (varimax-rotated) PCA?
I performed PCA on 33 items with 133 observations. Considering the criteria to take components with eigenvalues >1, 4 factors can be extracted. I then did varimax rotation of those. However, I ...
- 21
0
votes
0
answers
10
views
What is the difference between Multiple Correspondence Analysis and Principal Component Analysis result?
Several articles I've read stated that MCA and PCA both work as "reducing dimensionality" tools but MCA is used for categorical variables and PCA is used for numerical variables. But is ...
- 29
3
votes
2
answers
169
views
Does the average eigenvalue equals 1 in PCA applied to standardised data?
From what I understood when we are doing PCA, we can work both with raw or standardised data, depending on the situation we're in. Is it true that the average of the eigenvalues is equal to 1 when we ...
- 103
1
vote
1
answer
24
views
Can I apply Kaiser Rule without knowing the eigenvalues?
Kaiser's rule suggests the number of principal components to be included in an analysis by looking at eigenvalues. If I'm given standard deviations only, instead of eigenvalues, can I still somehow ...
- 103
0
votes
0
answers
11
views
Clustering sparse dataset with mix of continuous and categorical variables
I am trying to cluster sparse heterogeneous datasets containing demographics and diagnosis variables ( mix of categorical and numerical variables).
How should I start my clustering endeavors ? start ...
0
votes
0
answers
22
views
Creating a standardized composite "score" made up of multiple continuous, dependent variables for analysis in SPSS
I am trying to evaluate how a surgical intervention (insertion of a spinal fusion cage), resulting in distance changes between two vertebral bodies, led to new symptoms (yes/no) in some patients but ...
0
votes
0
answers
50
views
Plotting all the points of PCA to only one PCA axis, first PC1 and then on the PC2
I am required to project all the points on PCA to PC1 axis and then on PC2 axis as to see whether there is a good separation between the points. Through this I have to determine which dimension can ...
- 1
1
vote
0
answers
27
views
duplicated variables for different components
I'm presently evaluating the position of individuals of an 3 populations of an animal (according to their sexe) in function of the environmental factors (12) present in their habitat. To detect which ...
- 11
13
votes
8
answers
2k
views
Practical usefulness of PCA
I asked a similar question in the past, but I've thought about the message I am trying to convey a bit more and feel I can articulate it better. For context, I am on an introductory course in machine ...
0
votes
1
answer
40
views
Can I perform multiple Kruskal-Wallis tests with different explanatory variables against the same response variable?
My data is observational data, and that's made it all kinds of ugly, and I can't decide what statistical test is needed. I have one response variable, which is categorical (Species 1, Species 2, or ...
0
votes
1
answer
125
views
How to use slopes in PCA?
I would like to use slope values in PCA. The problem I face is that the slopes I calculate per group could be within different ranges of values. We know that it is important to normalize your data ...
- 301
0
votes
0
answers
10
views
How to understand eigenvalue equation from inner product point of view? [migrated]
In equation (14.4) page 429 of scholkopf's Learning with Kernels book, it states that the eigenvalue equation $\lambda v = C v$ is equivalent to $\lambda \langle x_i, v \rangle = \langle x_i, C v \...
- 1
0
votes
0
answers
27
views
Recommendations on best papers / blogs / existing literature on constructing risk and vulnerability indices?
I'm interested in constructing a risk index by indexing a large number of identified risk factors into a composite measure (that ideally then has sub-dimensions that can be explored further if one so ...
0
votes
0
answers
28
views
What does the quality of representation of a variable mean in PCA?
I understand that the quality of representation of an individual by a certain axis is measured by the cosine of the angle between the axis and the individual; the more the vector representing the ...
- 208
0
votes
0
answers
19
views
How does centering the data reduce the risk of numerical problems when doing PCA?
In Mathematics for Machine Learning (page 336), the authors state that centering the data (subtracting from the data its the empirical mean) reduces the risk of numerical problems.
Which numerical ...
- 1,655
0
votes
0
answers
11
views
In a principal component space, is a change in any coordinate going to result in the same distance between PC-projected observations?
Suppose that we have observations in a 2-dimensional principal coordinate space. Let's denote one observation $\mathbf{y}=(y_1,y_2)$ in our PC space. Further, suppose that we have another observation $...
- 1,777
1
vote
1
answer
33
views
Latent Semantic Indexing vs. PCA
I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition.
Let's denote $X$ a $N \times D$ document-term matrix. The $D$ rows of $X$ ...
- 249
0
votes
0
answers
51
views
Optimal predictive factors
Assume I am interested in predicting a time series variable $y_t$ using a vector of possible predictors $X_t$ of dimension $N_x$. I am interested in finding the optimal $N_z < N_x$ predictive ...
- 320
0
votes
0
answers
15
views
Do low-angle loadings necessarily indicate correlation between variables in PCA?
I keep reading that the angle between loadings in a PCA plot indicates some degree of correlation between the loadings (presumably lower angles lead to higher correlation, and vice versa).
I don't ...
- 249
1
vote
0
answers
19
views
How can I compare one full PCA model to two smaller ones?
I have nearly 30 variables going in to a large PCA, but the variables really fall into two conceptual categories. I want to test whether leaving all the variables to correlate freely with one another ...
- 11
1
vote
0
answers
30
views
How useful is PCA on its own? [closed]
In my machine learning course we have covered the key ideas behind principal component analysis. To round this part of the course off, we have learned to interpret the results of PCA, specifically ...
- 11
1
vote
0
answers
26
views
Interpreting results of PCA [duplicate]
I've recently started an introductory machine learning course and the first topic we have covered is prinicpal component analysis (PCA) and overall I am finding the whole topic quite tricky to wrap my ...
- 101
1
vote
0
answers
44
views
PCA - Feature Scaling [closed]
I have been reading that the features should be standardized before performing PCA but I couldn't relate to my understanding of the same.
PCA try to project the dataset in the direction of maximum ...
- 21
0
votes
0
answers
18
views
Can we find dependence among Principal Components?
Principal Components (PCs) in PCA are linearly uncorrelated by definition.
However, uncorrelation does not imply independence (let aside the fact that each PC constructed at step t is necessarily part ...
- 391
0
votes
2
answers
171
views
Model causality: graphical models and PCA
If we build a graphical model (DAG) we (may) interpret the arrows as causal dependences.
If we build a graphical model based on the variables returned by principal component analysis (PCA) we should ...
- 758
7
votes
3
answers
1k
views
Can PC1 explain more than 90% of variance?
I've been running some analysis using morphological data. However, when I run a PCA, PC1 explains more than 90% of variance. I'm not sure if this is possible. If not, what should I need to do?
- 71
0
votes
1
answer
34
views
Interpretation and application of PCA - positive factor patterns for first component for all variables
I am attempting to reduce the number of variables in a dataset for regression purposes and I suspect that many of the variables are correlated. Hence, I attempt a PCA, which I must admit I'm very new ...
- 3
1
vote
0
answers
22
views
Distribute feature importance to the components of the features in a PCA regression?
I read some interesting speculation over on the Data Science Stack. The setup is that there are multiple correlated features in a regression problem, and the goal is to determine feature importance.
...
- 47k
2
votes
0
answers
41
views
(PCA) Is it possible for PCs loading scores to completely change sign after adding data? [duplicate]
I recently ran a PCA on a dataset of self-report data from 226 subjects to zoom in on which specific individual differences might account for participants’ predicted choices in a separate task we have ...
- 21
0
votes
0
answers
11
views
Time invariance when the latent structure is unknown
Imagine that participants completed a series of measures indexing different abilities (memory capacity, learning, etc.) at two timepoints.
The only thing I would like to test at this stage is whether ...
- 75
0
votes
0
answers
29
views
Maximize Variance of Linear Combination of Matrix Columns
Let $A$ be a $k \times 1$ random vector, and $\mathbf{A}$ be a $n \times k$ matrix of observations.
Letting $t \in \mathbb{R}^{k}$ be a vector of weights s.t. $||t||_2 = 1$, suppose we are interested ...
- 313
1
vote
0
answers
37
views
How to visualize time series using PCA?
I have two multivariate data sets comprised of 100s of time series, one is the actual recorded data set of time series and the other is a synthetically generated data set based on the recorded one.
...
- 31
0
votes
0
answers
28
views
PCA with non-Euclidean norm?
I have a dataset $X$ which I want to perform PCA on- however, I don't care so much about explaining the variance of certain features. So instead of using the "normal" definition of ...
0
votes
0
answers
19
views
Definition of Generalized of Low-Rank model
Recently, I looked into this paper "Generalized Low-Rank models" and found it very interesting as it gives general perspective on Principal Component Analysis (PCA) related methods.
In the ...
- 391
2
votes
0
answers
31
views
How to run a PCA or Factor Analysis when you know one column of the factor loadings
I have this application where I have a direction that I want to keep fixed when I'm running a PCA or factor analysis. Is this possible? I just want to keep a column of the loadings matrix fixed.
How ...
- 257
2
votes
0
answers
28
views
Why not scale before PCA as a default step? [duplicate]
In ISLR 2nd edition, it says that you may not want to scale before PCA if the features are all in the same units (below). However, I don't see the nuance. Why not just have the "default" ...
- 513
0
votes
2
answers
170
views
What is the proof that non-linearly separable data can't become linearly separable with the results of PCA?
Give a non-linearly separable dataset $X,$ I want to proof that after performing PCA on it, the resulting dataset is guaranteed to be still non-linearly separable.
I think we could argue that we still ...
- 29
0
votes
0
answers
28
views
Does RDA axis explanatory power affect arrow length interpretation in RDA plot (scaling 2)?
In an RDA plot with scaling 2, arrow lengths reflect how strongly an independent variable is related to the response matrix. But is their effect also influenced by which axis they are pointing along? ...
0
votes
0
answers
39
views
Choosing Which latent variables to display and analyze
Attached is a plot of the R-squared for each additional latent variable (LV) included in a PLSR.
When choosing which LVs to display on a biplot (and ultimately determining which factors best explain ...
- 1
0
votes
0
answers
33
views
Lag variables on OLS after PCA (Principal Components Analysis)
I have about 60 macroeconomic and financial indicators; all of them stationary (logs and differencing), with monthly data for the last 25 years. I am trying to predict changes in a financial variable (...
1
vote
1
answer
28
views
How to form a single index?
My goal is to order companies based on their profitability in a specified period (from the most profitable, to the least profitable). For that I need a single index.
I have got a dataset similar to ...
- 11
0
votes
0
answers
31
views
high feature correlation but good OLS prediction
I have regression results where unconstrained OLS is near optimal - out of sample scores are almost the best when compared to some other constrained regression models. Although the ratio of number of ...
- 1
6
votes
1
answer
166
views
Can we consider the loadings as a proxy for correlation, in a Principal Component Analysis (PCA)?
In a PCA, the loadings can be understood as the weights for each original variable when calculating the principal component, or "how much each variable influence a principal component".
Thus,...
- 85
0
votes
0
answers
9
views
PLS and LDA analogies
I need some information to clarify some concept about PLS, LDA.
LDA is able to decompose the independent variable to maximize the classes separation. the approach used is to develop a PCA on the ...
