Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the ...

learn more… | top users | synonyms (1)

0
votes
0answers
15 views

2 features and 2 principal components

Whats the difference between 2 features and 2 principal components? I know what a PCA is, I just have the following problem: If my data has 2 features, the PCA will produce 2 components. So why does ...
0
votes
1answer
24 views

Feature selection: PCA vs intuition? [on hold]

Which one should I choose? How can I combine them (i.e. in series or parallel)? What if there are dummy features in my data? What if my intuition messes things up?
1
vote
0answers
11 views

Interpreting PCA results on gene expression data

I'm working on a gene expression dataset from patients who either have systemic sclerosis or not. I normalized the data with housekeeping gene values and scaled the expression values for each DNA ...
0
votes
0answers
24 views

How would Y-aware PCA for binaries look?

I recently stumbled upon Y-aware PCA in the blog of win-vector. They describe how PCA can be adjusted not to explain variation in $X$ but covariation of $X$ and $Y$. This is explained for the case ...
-1
votes
1answer
30 views

Why should I choose features or plot them manually when there are built-in functions to do that?

Why should I select variables due to my intuition if there are builtin functions in sklearn python like SelectKBest() and PCA() If I plot graphs of features of the data to see if they can detect the ...
0
votes
1answer
23 views

Interpreting PCA results when there a lot of PCs to reach 95% of variance

I'm working on a simple classification project to better understand PCA, but I don't understand the results. My dataset has 10 features, and I'm trying to predict the label (SeriousDlqin2yrs). ...
0
votes
0answers
17 views

Weighting components in questionnaire?

I'm working on a questionnaire that is built on 3 large divisions that each exist out of multiple questions (mostly of the likert-scale or categorical type). My research question is what the ...
0
votes
1answer
23 views

Accessing PCA components from caret object in R

I know how to build a model using PCA components in caret package, however I don't know which variables explain which PCA components. I need some help on it. When I perfom the preProcessing ...
2
votes
1answer
50 views

Why do the loadings returned by psych::principal() in R change with the number of components?

I have been using the principal() function of the psych package in R and setting the number of components after a scree plot analysis (...
1
vote
1answer
27 views

Confused about PCA transformation vectors

I'm trying to get the intuition of how PCA works. So far I understood that: I start from the input matrix $X = [X_{1},...,X_{p}]$ where each $X_{i}$ is composed by $n$ elements that are the $n$ ...
0
votes
1answer
28 views

Principal Components for dependent variable in a regression

My question is related to PCA. I want to estimate the effect of agriculture variability on the welfare measures for financialy included people against the excluded. For this purpose I am trying to ...
0
votes
0answers
7 views

Euclidean distances of PCA first axis

I want to calculate the Euclidean distances along the first axis of a principal component analysis (PCA) of 19 variables. I need a pairwise matrix of those distances for downstream analysis. I am ...
0
votes
1answer
53 views

Principal component analysis with random effects?

How can I do a Principal Component Analyisis considering also the Random Effects? (*) Is there any R package able to do so? Something like PCA+lme4 or PCA+nlme. (*) I mean I want to transform my ...
0
votes
1answer
41 views

Creating a new PC variable based on PCA loadings

I am trying to find variables which would be good predictors for the "stool" variable in my data. I was thinking I would use PCA to create a new variable which accounted for most of the variance in ...
0
votes
0answers
16 views

Principal Coordinates Analysis (PCoA) with Longitudinal Data

I am interested in running a PCoA on a distance matrix derived from longitudinal data. I'm concerned about biasing the PCoA towards overrepresented subjects (those with more time-points and samples). ...
0
votes
1answer
22 views

If I use PCA before clustering, do I need to use PCA scores on new axes(principal components) to run clustering?

I want to use PCA before clustering, and then I want to run a clustering algorithm such as K-Means. My understanding is that I run PCA and find loadings for each original variable, then calculate ...
-1
votes
0answers
22 views

Can somebody set an exmaple of the steps for doing K-means with PCA below? [on hold]

I have found a paper on the Internet and have read it. But there are some steps which are not really clear to me. If you already understand, help me understand what those steps say. Input: X={d1, d2, ...
0
votes
2answers
16 views

Non-linear transformation to increase separability between clusters

I want to do a classification on PC scores. I have a 400 dimensional matrix, e.g. 2000*400 (2000 number of samples and 400 dimensions). I fist apply PCA on it and take it to 3D, i.e. 2000*3. There are ...
1
vote
1answer
31 views

PCA on the time series data yields first PC that has an opposite trend from all original time series

I have time series data with five variables that have common variation and trends and they are very noisy. I want to extract their common variation (most likely the first principal component) and use ...
0
votes
0answers
12 views

matrix signal deconvolution

I measured the dietary habits of thousands of animals throughout the world, giving me the following matrix: X = (M x N), where Xij = measurement of food j in animal i M = number of animals N = ...
0
votes
0answers
16 views

How to generate new random variables after using PCA for dimension reduction?

I want to be able to generate random variables, that (more or less) match the distribution of some observed data set. The data set is high dimension and I have reduced the dimension using PCA. Only ...
3
votes
1answer
107 views

Why do deep learning practitioners forego PCA for ZCA?

I have an understanding of PCA and ZCA, read a similar question on the subject which, unfortunately, does not have the specific answer to my question. I understand the benefits of data whitening: ...
0
votes
0answers
19 views

Why is the reconstruction error for my training set larger than my test error using PCA on the MNIST data set? [duplicate]

I have a very strange behavior where I am trying to run PCA on the MNIST data set and then I check the test and train error. However, it seems that I get that the test error is lower than my train ...
3
votes
1answer
52 views

When to use distance biplot vs. correlation biplot in PCA

I wonder what could be good examples of using scaling 1 and 2 for a principal component analysis biplot. By examples, I mean ecological examples or applied examples of the PCA scaling so that one can ...
0
votes
0answers
32 views

Principal Component Analysis intuition

I am trying to understand the basics of PCA. I am trying to figure out how this helps us reduce dimensionality. From what I understand, we have a set of data in N-Dimensional Space We find N ...
1
vote
1answer
31 views

Is the matrix coeff from MATLAB's pca the same as the left singular vectors of the centered data?

Consider the SVD of a centered data matrix: $$ X_{centered} = U \Sigma V^T$$ where a column of $X_{centered}$ is: $$ X_{centered} = x^{(i)} - \frac{1}{N} \sum^N_{n=1} x^{(n)} $$ is the matrix $ U ...
0
votes
0answers
19 views

Distributed PCA

I have a large data set. Large means many instances (~50000000) and many features (~25000). I will call my data matrix X where the rows are instances and columns are features. Lets say the number of ...
0
votes
0answers
5 views

What's the relationship between detection and estimation?

I'm a bit confused by the relationship between detection (hypothesis testing) and estimation. For example, in sparse PCA setting, we may want to estimate the leading eigenvectors of the covariance ...
0
votes
0answers
21 views

How does the fraction of retained PCA variance affect the accuracy of a model?

I was checking various tools for classification and optimisation; I trained a sample dataset using KNN. I got 100% accuracy with 95% PCA explained variance and 99.2% accuracy with 5% PCA explained ...
0
votes
1answer
125 views

PCA vs FA vs ICA for dimensionality reduction in questionaire data

I am trying to identify personality traits underlying the multidimensional data from a questionnaire. In more abstract terms this means reducing the dimensionality of my data from N-dimensional (where ...
1
vote
0answers
17 views

Are the terminologies of score and principal component in PCA equivalent? [duplicate]

I am confused with the terminologies of score and principal component in PCA, it seems they are equivalent but there is also some difference. Could anyone explain to me?
3
votes
1answer
64 views

Do I need to run PCA over all predictors in a regression model? Can I run it only over the continuous ones?

I'm looking at the Lending Club data from Kaggle and I'm just building a pretty simple model to predict defaults. The data has a large amount of both continuous and categorical variables (I have ...
3
votes
0answers
30 views

How to draw a Shepard diagram from PCA scores?

In Legendre & Legendre 2012, they are saying that it is possible to diagnostic a PCA with the Shepard diagram. What would be the procedure to draw the diagram? I don't know what to put in relation ...
2
votes
1answer
77 views

What can be the reason to do feature selection based on variance before doing PCA?

I have noticed that when applying PCA to large datasets, people often will first subset the data considerably. Sometimes people just randomly take a subset of the features/variables, but often they ...
0
votes
1answer
34 views

Qualitative implications of Linear Discriminant Analysis (LDA)

I'm a beginner to LDA, and my question is about its qualitative implications. Say I have two classes of medical data, already classified as: $C_1=\{x_{11},x_{12}, ...x_{1n_1}\}, C_2=\{x_{21}, x_{22} ...
1
vote
0answers
31 views

How to determine time complexity of EM algorithm of probabilistic PCA?

I was studying probabilistic PCA from Bishop's book. There an EM algorithm is provided to calculate principal subspace: Here $\mathbf M$ is $M\times M$ matrix, $\mathbf W$ is $D\times M$ matrix ...
0
votes
0answers
24 views

Principal Component Analysis PCA Terms and relationships: eigenvalues, eigenvectors, loadings, score matrix, and SVD [duplicate]

I've read many websites, blogs, pdfs on this top but struggle to put the picture together in simple math terms, that explains how some of the terms relate to each other / are computed. Let's assume ...
1
vote
0answers
28 views

Multilevel Principal Component Analysis

I needed to run a PCA on a dataset with a multilevel structure. My question is similar to the one asked here: Principal components analysis on nested data In my case, however, the two levels are ...
1
vote
0answers
69 views

When can I use $XX^\top/n$ as covariance matrix for PCA?

Given a data matrix $\mathbf X$, can I always obtain its covariance matrix (to use in PCA) by centering (subtracting the column means) and then computing $\mathbf X \mathbf X^\top/n$? Is this always ...
1
vote
0answers
30 views

Is/are there any threshold value(s) to determine to see if PCA is useful at all, specially for high dimensional data?

Apologies if this is a naïve question, but it's not so naïve to me! Let's first assume we have 2D data which are perfectly linear but not along the x- or y-axis. PCA will rotate it so that it becomes ...
2
votes
0answers
55 views

If almost all variance is explained by the few first principal components, what can we say about the dataset?

What can we say about a dataset if we apply PCA and observe that there is a high percentage of variance in the first principal component(s)? Can we say that this dataset has linear structure? Can we ...
1
vote
1answer
33 views

Building regression model with multicollinear continuous and categorical variables: can I use PCA?

I am trying to build a regression model that has continuous and categorical predictors. Furthermore, the continuous variables suffer from collinearity. My understanding is that PCA can handle ...
1
vote
1answer
42 views

Rotating a new matrix into the same basis as another using SVD

I have collected some data and stored them in an $N\times P$ matrix $A$. Using SVD, we can rotate $A=UDV^T$ into a new basis, also discarding some dimensions: $A\approx U\tilde{D}V^T$, where ...
5
votes
1answer
90 views

Should the correlation PCA projection be computed on original or normalized samples?

Suppose we compute the correlation PCA of a dataset $X$ (with $m$ variables and $n$ observations) by first normalizing the input variables. That is: mean -> 0 and standard deviation -> 1. Let us ...
0
votes
1answer
56 views

Why convert categorical data into numerical using one hot encoding

I don't have very strong statistical background, and I'm new in data science... Now, I am practicing PCA (Principle Component Analysis) for dimension reduction. This tutorial looks very complete, but ...
2
votes
1answer
45 views

Variable standardization / scaling for PCA when all dimensions already have same scale [duplicate]

Often when PCA is performed on exam results where all variables (dimensions) have the same $0$ to $100$ scale, scaling is none the less applied. For different scales I can see the purpose of it, but ...
6
votes
1answer
150 views

Is PCA a non-linear transform?

In the article Relative Information Loss in the PCA, the authors make, at some point (in the introductory section), the following statement: In case the orthogonal matrix is not known a priori, ...
2
votes
0answers
54 views

What to do with principal components?

I'm running some basic R and there are a lot of great tutorials about PCA but they always stop once they've obtained a biplot. My question is what do you do with the principal components? I have two ...
3
votes
1answer
71 views

What is principal subspace in probabilistic PCA?

if $X$ is observed data matrix and $Y$ is latent variable then $$X=WY+\mu+\epsilon$$ Where $\mu$ is the mean of observed data, and $\epsilon$ is the Gaussian error/noise in data, and $W$ is called ...
0
votes
1answer
13 views

Using original centroid as cluster identifier after applying PCA

Take a look at my original data. (masked with purely random alphabetic here) : a b c d e f g h i j A = k l m n o p q r s t u v w x y I'm running ...