Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Make your voice heard. Take the 2019 Developer Survey now

Questions tagged [pca]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

5
votes
1answer
148 views

Using PCA to reduce dimensionality of training and testing data

I've read so many contradicting opinions that I feel like I need to ask the question myself. Say I use PCA on a dataset with 60 variables and find that I can explain 98% of variance with 6 principal ...
0
votes
0answers
11 views

How to apply PCA using statsmodels library [on hold]

I am using statsmodel for developing linear regression model. I want to use PCA using the same statsmodels library. How to do it ? Is it advisable to do PCA using SKlearn and then doing regression in ...
1
vote
0answers
33 views

Indications about the population when PCA essentially “fails” to reduce dimentionality

This is more of a conceptual question rather than a methodological one I guess. Let's assume that we have a dataset coming from a questionnaire and after some feature scaling we run a PCA to reduce ...
0
votes
0answers
7 views

How do I calculate ensemble-averaged PCA scores?

For context, please see https://arxiv.org/pdf/1808.00084.pdf (page 6) I am trying to replicate the results of the above link, which uses dimensionality reduction to observe a phase transition in an ...
1
vote
0answers
45 views

How to interpret feature importance in a decision tree after applying Factor Analysis

I'm using SKlearn to apply Factor Analysis (FA) to my data before training a Decision Tree. I then want to do an importance analysis. If I had not applied FA to my data, I could just call clf....
0
votes
0answers
28 views

Can we use Min-Max Normalization instead of Scaling in PCA

I have 11 variables & performing a PCA before applying clustering on the component data. I adopted two approaches for data pre-processing as the variables are of different scales. Approach 1: I ...
0
votes
0answers
15 views

Use PCA in order to show impact of variables on clusters

I did a clustering of selected variables. Is it possible to use the PCA (contribution, correlation) in order to show the impact of each variable in the clustering? (cfr. image 1) and add complementary ...
0
votes
1answer
24 views

One-hot-encoding gives untractable amount of classes

I'm performing regression on the price of bycicles based on their brand, model and submodel. These features are hierarchical: one model belongs only to one brand but one brand can have many models. ...
1
vote
0answers
15 views

Using PCA in domain adaptation

In literature, I see people using (Kernelized) Principle Component Analysis, not for feature extraction, but for domain adaptation. In other words, I have data from a source domain and I would like to ...
0
votes
0answers
20 views

Using Principal Components from one data set to visualize data in another [closed]

I have two correlation matrices, matrix1 and matrix. I want to be able to do something like this where I plot matrix2 according to the first and second principal components of matrix1. I know that <...
1
vote
1answer
47 views

How do I use principal components as predictors in linear regression?

I followed the instructions from this open Stanford lecture on PCR. I have a couple of questions, but first I'll post the code with my comments. ...
0
votes
0answers
27 views

Is it normal for principle components to have the same SD?

I'm trying to determine outliers of a PCA by calculating the sd of the eigenvectors - however it returns the sd as equal for each PC. 1) Is this normal and 2) Is this the right approach for ...
3
votes
1answer
36 views

How to quantify the agreement between the same parameter from two different data sets

I am looking at Arctic ice thickness from two different Earth-orbiting satellites A and B. I'm interested in quantifying how well these two datasets agree, but I'm struggling over what parameters to ...
0
votes
0answers
21 views

Dealing with correlated variables and choosing between models

One of the things I've always been confused about is the framework around model selection in the cases where $n$ predictor variables $x_i$'s are correlated. My thoughts on the different approaches: ...
0
votes
0answers
13 views

How to construct PCA projections uncorrelated with another variable?

Let's say I have 11 variables. I want to run a PCA on first 10 variables such that the projections are uncorrelated with variable 11. By construction and orthogonality of the eigenvectors, the ...
0
votes
0answers
37 views

Determining the Direction of Eigenvectors in PCA [duplicate]

I'm using R to get the principal components for several datasets. An example result, using prcomp yields: ...
1
vote
0answers
18 views

How to handle missing data in PPCA

I am implementing a paper "Probabilistic Principal Component Analysis" (PPCA) which deals with a dataset where each vector suffers from at least one missing value missing values. Generally, PPCA ...
0
votes
0answers
27 views

Idea behind change of basis and how it relates to projecting your points onto principal components

I would like to clarify if my understanding is correct. In the traditional X-Y coordinate system, our choice of basis vectors are $\vec{i} = (1, 0)$ and $\vec{j} = (0, 1)$ and when you I have a point $...
2
votes
1answer
43 views

Can I use PCA after lasso variable selection?

I have a data regarding life satisfaction, of more than 2000 observations and 265 variables (most are categorical variables). I want to build a model, estimating the effects of society problems on the ...
0
votes
1answer
28 views

How to use log probabilities in PCA mixture EM algorithm

I'm trying to implement PCA mixtures (Tipping & Bishop 2006 Appendix C) on the Tobomovirus. I'll summarize the mathematical background and algorithm here: For a single PCA model, we assume a ...
0
votes
0answers
9 views

Promax PCA interpretation in order to validate singular items - Structure or Pattern Matrix?

I am doing a promax PCA analysis. I have a big dataset (over 1000 subjects) and about 50 items. I am trying to validate the singular items. My final aim is to exclude non-relevant items (those which ...
0
votes
4answers
63 views

LDA - solving singularity problem of within classes matrix

I would like to solve the problem in LDA where the within classes matrix is singular if the number of samples is lower than the number of dimensions (which is true in my case, used on images of faces)....
0
votes
1answer
42 views

Principal Component Analysis Singluar value

While studying PCA, I saw this question but not able to solve. Given a data matrix X that is taller than it is wide, prove that every right singular vector of X with singular value s is an eigenvector ...
0
votes
0answers
55 views

PCA: violation of the normal distribution assumption

I try to understand why normal distribution is an assumption for PCA and what might happens when it is violated. I found one answer on this plattform and a lot of different answers in literature. It ...
2
votes
1answer
36 views

Normalizing vs Scaling before PCA

I know there's a lot of content about PCA pre-processing, but I am still somewhat confused. I have a dataset that contains some clear patterns: 1 variable is whether a person has financial resources (...
0
votes
0answers
12 views

How do I calculate my RCs for SSA for prediction (like I would for principle components)?

So the essential structure of what I am hoping to do is to create a neural network that uses the Reconstructed Components (RCs) from Singular Spectrum Analysis (SSA) to predict my outcome variable in ...
0
votes
0answers
13 views

Smoothing of experimental data for PCA

I am applying PCA to a set of spectrophotometric measurements with the aim of differentiating two groups of substances. The many small wiggles on the right-hand side of the curves (the region above ~...
0
votes
0answers
22 views

Principal component analysis for variable reduction

In the textbook “Principal Component Analysis” Jolliffe (§9.2) suggests the following method for variable reduction: “When the variables fall into well-defined clusters, there will be one high-...
1
vote
0answers
10 views

How to find complete log likelihood for mixture of PPCA

In Appendix C of a paper by Michael E. Tipping and Christopher M. Bishop about mixture models for probabilistic PCA, the probability of a single data vector $\mathbf{t}$ is expressed as a mixture of ...
0
votes
1answer
18 views

How to interpret a given 2D co-variance matrix?

I am trying to solve a problem regarding revision for my Big Data module. I have two main questions. 1) Given a predefined co-variance matrix: A cluster of points is distributed in a two-...
0
votes
2answers
50 views

PCA to recover factors used during data generation. Why doesn't it work?

I often found that the results of a PCA or any kind of factor analysis are interpreted in a "causal" fashion. I.e. if a principal component with high variance explanation is found, this is interpreted,...
0
votes
0answers
8 views

How does the three-component PCA model in Bishop and Tipping 1999 work?

In section 4.2 of Bishop and Tipping's "Probabilistic Principal Component Analysis" (Microsoft Research), we are shown three different plots that are said to have something to do with a three-...
0
votes
0answers
24 views

If I center my kernel does it no longer remain positive semidefinite?? If so why is it being used in algorithms like kernel pca?

If I center my kernel then can it still be used in operations where a positive semi-definite kernel is required such as SVM and ridge regression? I am centering my kernel as follows: $$K_c(\mathbf{t}...
1
vote
0answers
24 views

Equity Risk Model using an autoencoder

I am trying to create a statistical equity risk model using an autoencoder in a similar fashion to how one would use PCA to derive the systematic and specific risk components of a stock's returns. I ...
0
votes
0answers
14 views

Feature reduction of Biological time series signals

I have a data set of biological signals (PSG signals); the dimension of the signals is high (850 features for each sample). I am looking for the best way to reduce the dimensionality of the signals. ...
0
votes
0answers
32 views

PCA: should standardization be applied on features or samples?

I am struggling a little bit with PCA. I understand that standardization is an important part of the algorithm but I do not understand which elements should be standardized. Let's say I have a 10x100 ...
0
votes
0answers
22 views

Detecting insufficient communalities in R

Given is this sorted factor analysis: ...
0
votes
2answers
30 views

Use PCA to discover the most impactful variables on the original data set?

I am trying to find a way to statistically show that some variables in my data set are more important than others to determine its classification. I have an example data set with three variables from ...
0
votes
1answer
19 views

Interpreting PCA with varimax rotation

I have problems understanding the Factor Component Analysis of the paper: "Measuring thirty facets of the Five Factor Model with a 120-item public domain inventory: Development of the IPIP-NEO-120". ...
1
vote
0answers
24 views

PCA used for regression

Can anybody point me to a thorough example where PCA was used for predictive purposes and it compared favorably to regular linear regression (i.e., perhaps a lower RMSE using the PCA components ...
0
votes
0answers
7 views

is it necessary to have same scale of IDV in PCA [duplicate]

Can PCA be done if my data have both likert data and discrete (ordinal and nominal)as independent variables?
1
vote
0answers
28 views

What is connection between Pearson correlation coefficient and proportion of variance explained in PCA?

PCA procedure includes SVD of Covariance matrix. Based on eigenvalues we can find a proportion of variance explained by related Principal Components (eigenvectors). ...
2
votes
0answers
83 views

Why do PCA loadings given by sqrt(eigenvalue)*eigenvector yield correlations between PCs and original variables?

I did a lot of reading in this blog and elsewhere about PCA, SVD, loadings etc. But I still don't understand why loadings, which represent correlations between principal components and the original ...
1
vote
0answers
23 views

Screening data prior to PCA v. PLS

I have a very large time series matrix $X$, where the number of observations (rows) $n$ is much smaller than the number of input variables (columns) $p$. My aim is to use the information in $X$ to ...
1
vote
1answer
66 views

Units in Principal Component Analysis

If I have a data set with 6 variables (A, B, C, D) all measured in different units (eg. m, kg, s, K) and I perform principal component analysis to transform this into two Principal Axes, how do I ...
0
votes
0answers
25 views

PCA interpreting coefficients

down vote favorite had a question on something I thought was pretty basic, but just not getting this. I have three variables, and three principal components rateprev1 rateprev2 rateprev3 0.03831 0....
-1
votes
1answer
37 views

PCA influence of duplicates

I am using sklearn IPCA decomposition and surprised that if I delete duplicates from my dataset, the result differs from the "unclean" one. What is the reason? As I think, the variance is the same. ...
0
votes
0answers
15 views

Cross Correlation between two RVs and PCA

What is the difference between the maximum value of cross-correlation value of RVs X, Y and maximum eigenvalue of Covariance matrix of these same RVs X and Y? Are both same and just represents the ...
0
votes
0answers
33 views

Principal Component Analysis with weights

In a Principal Component setting, I want to solve the minimization problem $$\min_{\{f_t\}^T_{t=1}, \Gamma} SSR$$ where $$ SSR = \sum^T_{t=1}(y_t - \Gamma f_t)' R'_t R_t (y_t - \Gamma f_t)$$ $...
0
votes
1answer
25 views

A question about pca and gene function analysis

I'm new to bioinformatics, and I have a pretty basic question. Let's say I have a bunch of genes {Xi} and I want to know which one has the most significant on some measurable phenotypic trait Y. Now I ...