Questions tagged [pca]
Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.
3,394
questions
0
votes
0
answers
10
views
Comparing scree plots or explained variance of two groups with different number of features after PCA
I want to define the dimensionality of a group as the number of PC features that can explain 80% of the variance in the group dataset. This intuition seems to work for a single group, however, if I ...
- 1
1
vote
1
answer
69
views
Geometric Intuition Behind Whitening for ICA
I know there are a couple posts asking about why we whiten the data for ICA. I understand why we whiten to fix scaling invariants between the sources and to increase the computationally efficiency.
...
- 11
0
votes
0
answers
21
views
Is FAMD (Factor Analysis of Mixed Data) truly a factor analysis technique? or it is a dimension reduction technique?
PCA is distinct from factor analysis; it's a dimension reduction technique. PCA does not account for individual variable noise. On the other hand, FAMD (Factor Analysis of Mixed Data) combines PCA and ...
0
votes
0
answers
26
views
Interpretation of pca plot [duplicate]
I am looking at an example of using PCA:
https://wiki.q-researchsoftware.com/wiki/Principal_Components_Analysis_Biplot
Specifically, Example 2 has data
What are the points I'm looking at the biplot? ...
- 101
0
votes
0
answers
28
views
Precise definition of a principal component in PCA
To the following question regarding how PCA is performed, two well-detailed answers are given: one by ameoba and one by Andre P.
In the first, ameoba takes an $n\times p$ centred matrix $X$ and ...
0
votes
0
answers
21
views
Principal component analysis for "uneven" data
Suppose I have data consisting of three layers. Layer $i$ consists of $n_i$ 2D points, $i=1,2,3$. Here $n_1$, $n_2$ and $n_3$ may be different. I would like to somehow apply the Principal Component ...
- 101
0
votes
1
answer
60
views
Variances explained by each feature on PC in PCA
I came across this article with an associated Python codebase.
In brief, there is a section "Understanding How Features Contribute to PCs
", where...
One method for understanding which ...
- 577
0
votes
0
answers
39
views
Can participants be columns in a data frame for PCA?
I am very new to statistical analysis, so I apologize in advance if this question is too naïve.
I would like to know whether participants be columns in data frames for PCA, but I first want to offer ...
0
votes
0
answers
16
views
Is there a difference between Principal Component and weighted mean using PC loadings? How to get Principal Component on scale of original variables?
I was interested in doing a Principal Component analysis but returning a Principal Component on the scale of the original variables. Principal component analysis in R defaults to scaling and centering,...
- 525
0
votes
0
answers
18
views
Why am I getting negative components with my custom NIPALS algorithm
I've recently been learning about the Nonlinear Iterative Partial Least Squares (NIPALS) algorithm for computing the principal components of a dataset. I am trying to code a NIPALS class from scratch ...
- 41
1
vote
0
answers
36
views
Orthonormal Basis assumption in PCA derivation [migrated]
I'm doing the Mathematics for Machine Learning course on Coursera (Course 3, Week 4). I am trying to understand the derivation of PCA.
Specifically from:
$J =\frac{1}{N} \sum_{n=1}^{N}\Vert \sum_{j=M+...
- 349
0
votes
0
answers
35
views
How can to visualize/plot correlation matrix as a distance matrix of points in space?
It seems to me that the various options for visualizing the correlation matrix in R are quite unintuitive for laymen. They focus on the graphical representation of the correlation matrix as different ...
- 141
0
votes
1
answer
31
views
PCA and scree plot and slope
This may be really a simple question.
I know that if there are correlated variables, they may not be good to use for modeling. We have to use the PCA such that each PC is orthogonal to one meaning ...
- 11
0
votes
0
answers
13
views
What is Bayesian PCA and its cousin?
When I think of the phrase "Bayesian PCA" I think of two things, but these two things are what I have contrived rather than conventional notions. I would appreciate guidance on what these ...
- 6,998
1
vote
0
answers
22
views
Comparing the PCA modes from two different covariances
Suppose I have a set of $n$ vectors $x_i$ arranged as columns of a matrix $X$ and I want to perform PCA to reduce the number of dimensions needed to explain some set of observations. I have developed ...
- 281
0
votes
0
answers
8
views
Using cross correlation and an uncorrelated eigenvector basis to minimize exposure
I have the following problem:
We are looking at stock market data and have a historic price dataset $X$ with two stock types:
\begin{equation}
X = \begin{bmatrix}
x_1 & x_2
\end{bmatrix}
=
\begin{...
- 43
0
votes
0
answers
22
views
Running all possible fixed effects combinations, LMER, PCA [duplicate]
my data looks like that:
...
0
votes
0
answers
23
views
Running all possible fixed effects combinations for linear mixed effects models
my data looks like this:
...
0
votes
0
answers
10
views
Kernel + Mutliple SVM's + Platt Scaling = 1 layer neural network?
I have built my own Support Vector Machine by using quadratic programming and I'm using Kernel PCA with SVM. The output is tanh e.g Platt scaling.
When I combinde ...
- 349
0
votes
0
answers
5
views
Why does Kernel PCA works with validation data?
Assume that you have a matrix $X$ and you want to do Principal Component Analysis on that data. But the data contains nonlinearities, so you decided to use Kernel Principal Component Analysis instead.
...
- 349
1
vote
0
answers
48
views
Nonlinear Sparse PCA
Given data $x_1, \dots, x_n \in\mathbb{R}^d$, I am looking for a nonlinear dimensionality reduction technique $f: \mathbb{R}^d \rightarrow \mathbb{R}^q$ that only uses a limited number of dimensions ...
- 298
1
vote
0
answers
28
views
PCA with soil cover of plant species
8 plots (quadrants 3 x 3 meters), 4 invaded with an alien plant, 4 without the alien plant.
I have measured the soil cover (as %) of the species found in these 8 plots.
If I make a PCA with log (n+1)...
- 31
0
votes
0
answers
38
views
How do I measure the "dispersions" of a group of time series
I have a group of time series $X_1, X_2, ... X_n$. I want to measure how much they have "dispersed" over time. i.e. are they moving "more together" in 2023, comparing to 2022. $n$ ...
- 111
0
votes
1
answer
36
views
Analysis of the bias resulting from PCA [closed]
Suppose that we generate some dataset from $y = X \beta + \epsilon,$ where $\epsilon$ is some independent error, and the rows of $X$ come from some distribution (unspecified for now). Suppose you run ...
- 211
0
votes
0
answers
18
views
PCA with gram matrix produces different results from PCA done using covariance matrix?
I was trying PCA on a dataset (#samples=24, #dims=42) via eigendecomposition using numpy. I read that for matrices where the number of features exceeds the number of samples, we should use the gram ...
- 121
0
votes
0
answers
3
views
Comparing two algorithms, one is parameter free while the other is not
I wish to compare two algorithm for subspace approximation (similar to PCA). One algorithm is parameter free, while the other is not. I use cross validation to set value to this parameter, and then ...
- 799
0
votes
0
answers
64
views
Identifying stress in financial stress index constructed using PCA using Hodrick-Prescott filter
I am reading:
https://www.econstor.eu/bitstream/10419/128519/1/ewp-356.pdf
The footnote on page 19 says:
The trend was derived using the Hodrick-Prescott method where the smoothing parameter λ is set ...
- 231
0
votes
1
answer
71
views
Difference between conducting PCA on $XX^\top$ vs $X^\top X$?
PCA: For a given set of centered data $\mathscr D =\{x_i\}_{i=1}^N \subset \mathbb R^d$, i.e. the data has $N$ examples with dimension $d$. Then the principal directions of PCA can be obtained from ...
- 1
0
votes
1
answer
24
views
How to project kernel PCA?
I have an $m\times n$ matrix $X$. To apply a Kernel PCA to my $X$ matrix I need to warp it into a function $K = \Phi(X)$.
The problem here is that $K$ get the size $m \times m$. If I'm doing ...
- 349
2
votes
0
answers
27
views
Functional Principal Component Analysis - Explaining Functional Principal Component Scores
I was wondering if someone can help with explaining Functional Principal Component Scores?
I am working with a dataset which reflects participants in a weight loss management trial (longitudinal data)....
2
votes
3
answers
418
views
Can you only apply PCA feature reduction only when there are linear relationship between the variables?
My coworker and I were discussing this. He insists that PCA only makes sense just when you know at least one variable is linear combination of the rest but I think it can be applied whenever there are ...
0
votes
1
answer
69
views
Can I conduct a PCA on binary presence-absence data?
I have presence-absence data for 53 wildlife species at 60 different sites. I am a bit confused as to whether a PCA would be appropriate for this sort of data. If so, is there a specific R package I ...
BiggestNoob
0
votes
0
answers
23
views
How do I associate or assign a large amount of continuous variables with zero-heavy distributions to different groups?
I have a dataset with about 70 continuous numeric variables. I have about 80 samples which divide more or less equally into two groups. I want to figure out which of these variables is most strongly ...
- 11
0
votes
1
answer
19
views
Multiple linear models and multiple comparison correction
I have run a PCA from which I have extracted 4 components. Each component will be used in a multilevel model as the outcome variable. The four model will have the exact same predictor variables, 4 ...
- 11
1
vote
0
answers
33
views
How can I perform statistical analysis, (PCA sPLS-DA) etc on a dataset which has many missing values? [closed]
I have a dataset of mass spec analysis which was sent to me in 200 excel files. Importing into R, I get a dataframe that looks like this:
| Unique Sample ID | Chemical Name | Concentration |
| --------...
- 11
0
votes
0
answers
11
views
How can a linear autoencoder with $h=1$ hidden unit reconstruct any rank 1 matrix?
I've had this as a homework problem as a true or false type of question and I'm trying to wrap my head around why this is true.
Is the reason simply represent each datapoint as a scaled version of a ...
- 290
1
vote
0
answers
35
views
How can I test significance of 2 PCA results/PCA polygons
The story so far is that I am comparing the climate niche of wild native UK tree species, with the climate niche of commercial growers that grow that species in the UK. Using the species code 'agl' as ...
- 11
0
votes
0
answers
23
views
Understanding PCA plot built on data normalized by two normalization methods
I tried two different normalization methods, generated the PCA plot above on the combined data, and colored the samples by the normalization type. Both normalization methods should give similar ...
- 153
0
votes
0
answers
12
views
Kernel PCA on each data set, not the whole matrix - Possible?
I have a matrix $X$ that has $M * n$ in size. I'm going to apply that with PCA. The problem is that $X$ contains nonlinear structures. So one good thing is to use The Kernel Trick.
My data is ...
- 349
1
vote
1
answer
54
views
Why does Non-negative Matrix Factorization not give me 100% R^2 at full rank?
IMPORTANT: Please read the question carefully before labeling it a "code question" & sending it to stack overflow. The question is in fact a theoretical stats question (barring the ...
- 265
0
votes
0
answers
21
views
Can a linear autoencoder perform projection as PCA with less layers than components? [duplicate]
Assume that you have a matrix $X$ and you want to project it onto a subspace by using PCA. It will work. Then you are trying to use a linear autoencoder to projecting $X$ onto the same subspace. It ...
- 349
0
votes
1
answer
61
views
Using Ridge Regression to estimate importance of multicollinear variables in python
New to statistical analysis so bear with me.
I have a dataset with 1 (say y) dependent variable and 5 independent variables (say x1,x2,x3,x4,x5) which are highly correlated. I know that y depends (...
- 101
9
votes
2
answers
572
views
Why am I getting different PCA loadings for the same data set using two different r packages?
I have applied PCA to a dataset using two different R packages viz. "stats" and "FactoMineR". For both models, I am getting different loadings. What may be the probable reason ...
4
votes
2
answers
81
views
PCA to reconstruct Binary Data
I'm working with binary 3D matrices. I calculate their PCA (or REOF or SVD) and as a test I would like to reconstruct these matrices from the PCA results. However I realized that because I only keep ...
- 143
0
votes
0
answers
13
views
Is there an eigenvector based decomposition scheme that will find the longest axis of variation collinear with a named variable?
So far as I understand, algorithms like principal component analysis, spectral decomposition, etc. are similar algorithms that identify orthogonal vectors in a dataset using a different set of ...
- 284
0
votes
1
answer
30
views
I decomposed only the first four components from PCA/EOF. Can I get the percent variance of each component against the entire dataset?
I am using NIPALS (non-linear iterative partial least squares) to get the loadings and scores from a PCA analysis. The reason I am using NIPALS rather than singular value decomposition is because ...
1
vote
1
answer
56
views
Which features corresponds to which eigenvalues when use SVD in PCA?
Today, after learning about performing $PCA$ using $SVD$, I know $PCA$ will choose $K$ components that have the highest eigenvalues. I have a question which feature will correspond to which ...
- 113
0
votes
1
answer
22
views
How do I determine whether there is a correlation between geographic groups and PC2?
I have used PCA on a group of individuals based on a specific feature of their genetic data. I now want to see if PC1 or PC2 correlates with their grouping based on geographic location (each ...
- 1
2
votes
2
answers
93
views
PCA Derivation with maximizing projection length
I was reading a post about deriving PCA, So it considered an arbitrary row datum, $x_i$ and tried to maximize projection length for each row of data matrix, $X \in \mathbb{R}^{N\times D}$:
$$\sum_{i=1}...
- 21
0
votes
0
answers
24
views
Project Matrix Data onto PCA Space
Given a matrix A with dimension m by n which is a matrix of m samples and n features. Also SVD of Matrix A = U * Sigma * V^T, How to project matrix A onto its k principal components with only U and ...
- 31