Questions tagged [pca]
Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.
3,343
questions
1
vote
0
answers
28
views
Relations between clustering, graph-theory and principal components
I am trying to give some theoretical foundations to the intuitive idea that three branches of mathematics are indeed tightly connected, specifically: clustering, graph-theory and principal components.
...
- 63
0
votes
0
answers
14
views
Relative importance of scores in PCA / SVD
Let $X$ be an $n\times p$ matrix with $n$ observations and $p$ variables. After performing
$[U,S,V] = svd(X)$
I can obtain the scores using $scores = XV$. If I want to compare the relative importance ...
- 337
0
votes
0
answers
86
views
PCA evaluation with Shannon Entropy
I'm working on a fairly large dataset (5e5 samples in a 22 dimensional space); I'm reducing the dimensionality of this dataset with PCA.
Since a lot of variables are cross-correlated, I group the ...
- 11
1
vote
1
answer
48
views
What exactly does PCA show that I can't figure out otherwise?
First off, apologies if this is basic but I'm still learning about PCA. I am somewhat confused as to what exactly PCA can provide that I can't find out through other means. For example:
...
- 11
0
votes
0
answers
22
views
PCA for 3D Lidar data
I'm trying to work with Lidar data to classify road. I'm wondering how to use pca (or something else?) to identify if region has connectivity > threshold then label the region to be road? Would ...
- 101
1
vote
0
answers
26
views
When might the mean of PCA be useful? [closed]
I wish for a better understanding of the mean from principal component analysis. I am told this merely projects the origin to a different coordinate space. However, is there use to using the mean ...
- 23
0
votes
0
answers
93
views
Testing differences between groups using PCA / mahalanobis distance
I would appreciate any thoughts on the below problem, as well as recommended reading for PCA, mahalanobis distance, and hypothesis testing as it pertains to the below vignette!
I'm dealing with some ...
- 1
0
votes
0
answers
33
views
Can I perform cluster analysis before PCA?
I would like to know what are pros and cons of performing cluster analysis before/after PCA?
My data are not very well explained by the first and the second PC (49%), so I am thinking it would be more ...
- 11
1
vote
1
answer
93
views
Accounting for 0 observations in presence/absence community data, PCoA/PCA: is this idea credible?
I have a solution in mind for this problem, but I'm not sure if it is defensible (which is why I'm asking you all!). I have a data frame where each row represents an individual site, each column ...
- 55
1
vote
0
answers
58
views
Outliers in a PCA score plot
I have this dataset of 104 tissue samples from two different types of tumors (B and C) along and 182 observations (gene expression profile). I do not need to understand the underlying biological ...
0
votes
0
answers
37
views
PCA: Does the sum of all the features (not the components) is always linearly correlated to F1?
I'm trying to figure out if we always have:
$$pearson\_corr(F_1, \sum_{k=1}^{n}X_i) = 1$$
with:
$F_1$ the principal component,
$X_i$ the numerical features we used for the PCA,
$n$ the number of ...
0
votes
0
answers
45
views
Procrustes rotation when computing confidence intervals for PCA weights
I would like to compute confidence intervals for my feature weights derived from a PCA using a resampling procedure. I found a lot of different threads that deal with two major issues here when doing ...
- 161
0
votes
0
answers
18
views
Gene expression clusters
I am working on a gene expression project in which my database is based in a cohort of individuals distributed in 3 groups (according to treatment), and 2 measurements (baseline and post). I am ...
0
votes
0
answers
41
views
How to interpret relationships between variables in a PCA Biplot?
I have a PCA biplot about the detection of chemicals found in lakes and groundwater and I am feeling confused about how to interpret the relationships between the chemical variables in the biplot. I ...
- 283
1
vote
1
answer
31
views
Principal component vs. Exploratory Factor Analysis [duplicate]
Problem: Which method should I use to observe whether there is one common factor/component explaining six different measures (i.e., face and objects)?
Study: In this study, we have four measures of ...
- 13
2
votes
0
answers
34
views
Is it sound to use PLS components in a model trained with the same response?
I have some high-dimensional spectral data I want to use in modeling plant productivity using a supervised model like random forest. I want to use the model for inference as well as for prediction in ...
- 21
0
votes
0
answers
35
views
Is short vector length in PCA problematic? Why are my points so far from my vectors?
I am using PCA to examine the relationships between environmental factors of a couple of different types (physical properties, concentrations, relative abundance of genes). I have log-transformed the ...
1
vote
0
answers
33
views
Principal component analysis on binary variables (i.e. 0 or 1)
Can I do principal component analysis when I have only dummy variables? Is it preferable to construct quantitative continuous variables (when possible) using the dummy variables and then run the pca ...
- 11
2
votes
1
answer
51
views
PCA : how to cluster data to differenciate my data the most while considering their groups
I have to do a PCA in R for a project, but I have 300 data in 15 differents groups, and I want to find the reduced space which gives me the most variability between the groups and cluster my data in ...
- 21
0
votes
0
answers
34
views
PCA : how to cluster data to differenciate my data the most while considering their groups [duplicate]
I have to do a PCA in R for a project, but I have 300 data in 15 different groups, and I want to find the variables which gives me the most variability between the groups and cluster my data in their ...
- 21
1
vote
1
answer
68
views
First principal component does not correlate with its variables
I am building an indicator for developable land as the inverse of the principal component of four variables that are all negatively correlated with land availability (e.g. terrain slope, fraction of ...
- 2,602
0
votes
0
answers
32
views
Cochran Q test & Correspondance Analysis on CATA data
I have done Cochran Q test and Principal Correspondance Analysis on Sensory CATA data. I have 72 attributes and they are treated as variables in this case. The first two axes of PCA only account for ...
- 1
0
votes
0
answers
56
views
PCA with zero-mean noise and covariance $\boldsymbol{\Sigma}_i$ for each observation $\mathbf{x}_i$
Given $n$ noisy observations $\{\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_n\}$ (each $\mathbf{x}_i\in\mathbb{R}^d$), perturbed each with heterocedastic Gaussian noise $\mathcal{N}(\mathbf{0}, \...
- 1
0
votes
0
answers
48
views
How do I deal with low loadings in the first principal component of PCA?
I have a matrix data consist of 40 variables (year data) with 1500 obs. I used this code:
jjas <- df
pca.jjas <- prcomp (jjas, center = T, scale =T)
Here are ...
1
vote
1
answer
61
views
Spiked Covariance Model and PCA
Consider the spiked covariance model $Y_i\sim^{iid}N(\mu,\Sigma)$, where $Y_1,\ldots,Y_n\in \mathbb{R}^p$, $\Sigma=U\Lambda U^\top+\sigma^2 I_p$ be the eigendecomposition: $U\in\mathbb{R}^{p\times r}$ ...
0
votes
0
answers
48
views
How to interpret PCA loading values for categorical features when doing FAMD for classification tasks
I'm trying to tackle a binary classification problem. My dataset has 20+ features which are numeric continuous, numeric discrete, and categorical ordinal. I need to provide a breakdown of feature ...
- 1
5
votes
1
answer
65
views
Estimating largest eigenvalue of $N_{d\gg 1}(0,\Sigma)$ from small data
I am trying to estimate the largest eigenvalue of some $d$-dimensional normal distribution $N_d(0,\Sigma)$ from the sample data
$$X_1, \ldots, X_N \sim_{iid} N(0,\Sigma)$$
where $N$ is much smaller ...
- 203
0
votes
0
answers
24
views
How do I have to interpret positive/negative of eigenvector of PCA results?
Following is the eigenvector of my PCA result for 4 variables in R.
...
- 23
0
votes
0
answers
28
views
How to perform multilevel regression on Y?
We know multiple linear regression has the equation
$$
Y_1 = \beta_0 + \beta_1x_{11} + ... + \beta_k x_{1k} + \epsilon_1 \\
Y_2 = \beta_0 + \beta_1x_{21} + ... + \beta_k x_{2k} + \epsilon_2 \\
\dots \...
- 171
0
votes
0
answers
20
views
Why is data whitening (decorrelation) not tried with a post multiplication of a whitening matrix?
The general procedure of whitening(decorrelating) a data $X$ with dimensions $(D, n)$ (n being the number of samples) rests upon finding a matrix $W$ with dimensions $(D, D)$ such that the transformed ...
- 2,139
0
votes
0
answers
23
views
KMeans data treatment
I have a dataset with a set of features regarding:
Age Country TypeA1 TypeB1 TypeC1
Where Country is a feature that can have 195 different potential values, ...
- 123
0
votes
0
answers
14
views
How to create a matrix with known number of signals for use in PCA testing
I'm looking for a way to create a matrix with a known number of signals and background error for use in PCA. The following example attempts this by combining signals of known amplitude, followed by ...
- 3,682
0
votes
0
answers
9
views
PCA on a matrix to produce a sequence
My dataset is organized as matrix with 6 variables (Each matrix shows a specific type of fault). I want to convert it to a sequence by applying PCA on matrix and choose the first principle component (...
1
vote
0
answers
56
views
Using Gradient Decent For PCA Optimization [closed]
I'm trying to solve the PCA problem:
For $k\in N$ some number and $X\in R(n\times d)$ where I'm trying to find $w\in R(k\times d)$ such that:
$w = argmax( E(WXX^T))$
(I might be wrong with the ...
- 11
0
votes
0
answers
11
views
How to white a complex multivariate random process
I'm trying to apply a whitening process to a complex multivariate random process ($\boldsymbol X$) in a way to satisfy the following conditions simultaneously:
$E[\boldsymbol X_w\boldsymbol X_w^H]=\...
0
votes
1
answer
30
views
Principle Component Analysis for Feature extraction from Voltage and Current Signals
I am doing research work on fault classification in power transmission lines. I generated fault datasets in MATLAB/Simulink and collected it in matrix format with 6 various i.e. 6 variables in 6 ...
0
votes
0
answers
12
views
How best to quantify how much one part (i.e., a group of items in a scale) contributes to the whole pool?
I am trying to determine how much specific types of stressors contribute to the overall pool of stress experienced, e.g., X type of stress contributes X% to the overall pool of stress, or be able to ...
0
votes
0
answers
29
views
Why do we scale features in PCA? Wouldn't that mean the variance in all dimensions is just $1$? [duplicate]
According to https://scikit-learn.org/stable/auto_examples/preprocessing/plot_scaling_importance.html,
Feature scaling through standardization (or Z-score normalization) can be an important ...
- 205
0
votes
0
answers
6
views
why all loadings of a component is negative [duplicate]
I want to deal with principal component analysis(PCA) with ordinal variables. More specifically, all variables are ordinal. What I use is "princals", which is a command in package "Gifi&...
0
votes
0
answers
31
views
Does invariance of PCA under orthogonal transformation hold for data that is not centered?
I read the proof in the top answer to this question, but that page assumes that $\overline{A} = 0$. If the data instead has some nonzero mean $\mu$, I'm not sure if the same logic applies: ...
0
votes
1
answer
110
views
Understanding estimates of qualitative supplementary variables by dimdesc() of FactoMineR package
FactoMineR package is helpful when doing PCA and much more. Coming across the output using dimdesc() function and looking at the estimates of supplementary ...
- 1,085
0
votes
0
answers
49
views
First principal component separate the clusters but other PCs not
I have two sets of vcf files with different samples but from the same population. I merged the two and want to compare the two sets via PCA analysis.
Since the two set is from the same population I ...
2
votes
0
answers
23
views
PCA on X to capture the most variance of Y
PCA maximises the variation in $X$. Now, suppose $X$ is causal to $Y$. Is there then any analogous way to decompose $X$ into the principal components that cause the most variation in $Y$.
For an ...
- 21
0
votes
0
answers
31
views
How to choose tuning parameters for SparsePCA
Good morning to everyone.
I'm studying the paper "Sparse Principal Component Analysis Hui ZOU, Trevor HASTIE, and Robert TIBSHIRANI" (Link: https://hastie.su.domains/Papers/spc_jcgs.pdf)
At ...
1
vote
0
answers
41
views
Is there a multilinear kernel principal components analysis?
PCA can be extended to kPCA using the kernel trick.
MPCA is a multilinear extension of PCA that involves multiple matrices for the different modes of the data tensor.
Can MCPA be similarly extended ...
- 6,135
1
vote
0
answers
42
views
Is there a difference between kernel PCA with a non-linear kernel vs PCA with a non-linear change of variables?
I see that kernel PCA with a linear kernel is the same as PCA.
On Wikipedia's introduction of the kernel to PCA they suggest that there exists a non-trivial arbitrary choice of map $\Phi$ that is ...
- 6,135
1
vote
0
answers
20
views
Is there a multilinear principal component regression?
PCR is a linear regression problem on top of PCA.
Analogically, is there a 'multilinear PCR' as a linear or multilinear regression problem (e.g. CP tensor regression and Tucker tensor regression for ...
- 6,135
1
vote
1
answer
302
views
Clustering leading to visually overlapping clusters on scatterplot
I am dealing with a dataset with 13 features.
After going through some standard scaling and missing data imputation, I use kmeans from sklearn to create clusters.
Now the point is that, although the ...
1
vote
1
answer
129
views
Elastic Net Collinearity
When performing linear regression it is often assumed that the predictors are independent with Gaussian noise:
\begin{equation}
Y = X\beta + \epsilon \quad
\epsilon \sim \mathcal{N}(0, \sigma)
\end{...
- 121
0
votes
0
answers
13
views
Obtain principal components from the loading vectors
Good morning, everyone.
I am trying to use the "SparsePCA()" function of Matlab whose documentation link is below available.
https://www.ml.uni-saarland.de/code/sparsePCA/sparsePCA.html
This ...