Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the ...

learn more… | top users | synonyms (1)

0
votes
0answers
11 views

Relation between percentage of variance in first components of PCA and linear separability of a data set?

What we can say about a data set, if we apply PCA on the data set and observe there is high percentage of variance in the first principal component? Can we say this data set is linearly separable? Can ...
0
votes
1answer
13 views

Building regression model with multicollinear continuous and categorical data types - Use PCA?

I am trying to build a regression model that has continuous and categorical datasets. Furthermore, the continuous variables suffer from collinearity. I am of the understanding that PCA can handle ...
1
vote
1answer
31 views

Rotating a new matrix into the same basis as another using SVD

I have collected some data and stored them in an $N\times P$ matrix $A$. Using SVD, we can rotate $A=UDV^T$ into a new basis, also discarding some dimensions: $A\approx U\tilde{D}V^T$, where ...
3
votes
0answers
54 views

Should the correlation PCA projection be computed on original or normalized samples?

Suppose we compute the correlation PCA of a dataset $X$ (with $m$ variables and $n$ observations) by first normalizing the input variables. That is: mean -> 0 and standard deviation -> 1. Let us ...
0
votes
1answer
35 views

Why convert categorical data into numerical using one hot encoding

I don't have very strong statistical background, and I'm new in data science... Now, I am practicing PCA (Principle Component Analysis) for dimension reduction. This tutorial looks very complete, but ...
0
votes
0answers
50 views

Will PCA also produce multicollinearity? [closed]

It's well known that PCA will generate orthogonal basis. But in practice, I found that even after PCA, I can still face the problem of multicollinearity. Is it due to the numerical limit or I did ...
2
votes
1answer
42 views

Variable standardization / scaling for PCA when all dimensions already have same scale [duplicate]

Often when PCA is performed on exam results where all variables (dimensions) have the same $0$ to $100$ scale, scaling is none the less applied. For different scales I can see the purpose of it, but ...
6
votes
1answer
134 views

Is PCA a non-linear transform?

In the article Relative Information Loss in the PCA, the authors make, at some point (in the introductory section), the following statement: In case the orthogonal matrix is not known a priori, ...
0
votes
0answers
19 views

kernel PCA with RBF Kernel projected values [closed]

Let's suppose we have two points and we apply kPCA with an RBF Kernel : The projection values onto the first eigenvector are the same, whereas on the second eigenvector they are different. I don't ...
0
votes
0answers
46 views

What to do with principal components?

I'm running some basic R and there are a lot of great tutorials about PCA but they always stop once they've obtained a biplot. My question is what do you do with the principal components? I have two ...
3
votes
1answer
67 views

What is principal subspace in probabilistic PCA?

if $X$ is observed data matrix and $Y$ is latent variable then $$X=WY+\mu+\epsilon$$ Where $\mu$ is the mean of observed data, and $\epsilon$ is the Gaussian error/noise in data, and $W$ is called ...
0
votes
1answer
12 views

Using original centroid as cluster identifier after applying PCA

Take a look at my original data. (masked with purely random alphabetic here) : a b c d e f g h i j A = k l m n o p q r s t u v w x y I'm running ...
0
votes
0answers
18 views

kPCA with RBF Kernel and Circle data [closed]

When applying Kpca with RBF kernel to a circle data, the projection onto the first eigenvector gives the same value (all points have the same projection value). I don't understand why since kPCA is a ...
0
votes
0answers
24 views

Question: next step after regression analysis? how to tell if multiple variables all co-correlate?

I am new to regression analysis and I have found that my 'independent variable' or predictor correlates with several dependent variables (through multiple linear regression tests) in a way that ...
0
votes
1answer
18 views

Picking more than desired number of features in PCA

I have encountered the presentation and one of the ideas mentioned there is as follows. Suppose, that there is a sample of objects with 100 features, only 5 of which are informative. On the 5th slide ...
3
votes
1answer
39 views

Using metric MDS with non-metric distances and assessing the fit quality

I'm going to perform MDS by means of cmdscale function of standard R library. I spent several hours googling it and finally have ...
0
votes
1answer
18 views

How to use principal component to fit linear regression for pairwise relation in R

I have been struggling with this problem for several months. I would really appreciate if someone could help me solve this. I am working on a pairwise relationship as shown in the data below (...
0
votes
0answers
12 views

Would it make sense to train a naive bayes model with PCA compressed data?

I would like to improve my result from my training, from which i was thinking maybe PCA could be able to help determine which features are useful for my classification task. But would that help?
0
votes
0answers
18 views

Using PCA to determine which features are useful in classification [duplicate]

Is it possible to use PCA to determine what features can be used in classification (to determine a class)? I have a dataset consisting of 40000 observation from which 324 features are extracted. I ...
0
votes
1answer
29 views

Correct way to report the results of a Correspondence Analysis and a Principal Components Analysis

I am currently writing up my dissertation and have done a few CA and PCA tests. I know that the output gives principal inertias that have both decimal value and percentage of variance output. I am ...
3
votes
0answers
41 views

Running PCA on all predictor variables in regression vs. running separate PCAs on groups of related variables

I am new to PCA and was hoping for some insight. I currently have a dataset with ~150 observations and ~80 variables, and the goal is to determine which of these variables are associated with our ...
2
votes
1answer
41 views

Is there such thing as a case-weighted PCA?

Say I have 300 samples from a population containing two groups, A and B, and data for several variables. I have 150 from Group A and 150 from Group B. However, I know that Group A makes up roughly 20% ...
0
votes
0answers
20 views

Principal Component Analysis data visualization and interpretation

How would you go about interpreting correlations between two components? Are there any general inferences that one can make? i.e. a positive correlation can imply...? I pulled this plot from a SAS ...
1
vote
1answer
17 views

Standard deviation in principal component space

I am running a large number of simulations in which I have a 3D parameter space, and for each set of parameters (point in the 3D space) I run 100 simulations. I then use 14 measures to quantify the ...
0
votes
0answers
11 views

Something like PCA for nonlinear discrete data?

I have a multi-dimensional discrete dataset. I know that the relationship between the components is nonlinear, but I don't know anything else. Preliminary inspection of some components of the data ...
1
vote
0answers
15 views

Calculating change in PC scores using PCA for paired temporal data

I'm trying to calculate change in scores on a depression questionnaire - a very simple problem. However, what I care about is not the change in raw score, but rather the change in principal component ...
5
votes
1answer
92 views

Probabilistic models for partial least squares, reduced rank regression, and canonical correlation analysis?

This question results from the discussion following a previous question: What is the connection between partial least squares, reduced rank regression, and principal component regression? For ...
0
votes
0answers
8 views

Creating subsets of data with a particular target distribution

Is it possible to select a subset of observations from a dataset, while enforcing particular distributions across the data dimensions? In essence, create subsets of data with particular target ...
0
votes
0answers
8 views

What is a better approach to create operational performance score, LSM or PCA?

I want to create one operational performance score for each day so that we know if the performance was good or bad in our call centers, to be able to compare days, and to be proactive in maximizing ...
0
votes
0answers
23 views

diagnostics for PLSR?

I am trying to apply sPLS2 type pf regression my matrix y has a set of clinical variables and matrix X has some gene data .I am using mixOmics package in R. My question is how to decide that my ...
5
votes
1answer
131 views

What is the connection between partial least squares, reduced rank regression, and principal component regression?

Are reduced rank regression and principal component regression just special cases of partial least squares? This tutorial (Page 6, "Comparison of Objectives") states that when we do partial least ...
0
votes
0answers
20 views

PCA + Parallel Analysis

When I realize the Factor Analysis (I have 16 items), the PCA says I have 5 factors. But in the scree plot there is no elbow at all, just a decreasing line, that makes me think maybe I shouldn't be ...
0
votes
0answers
22 views

Calculating Eigenvalues in Factor Analysis and Calculating Factor Loads

So, as I wade back into the world of stats, I am finding myself confused about a few things when it comes to Factor Analysis. So I understand the bits about calculating Eigenvalues from a Correlation ...
1
vote
0answers
12 views

PCA for model coefficients

I have a regression problem with strong multi-colinearity. The question is about the effect of the variables more-so than the prediction from the model. Is it valid to divide the coefficient of a ...
0
votes
0answers
79 views

Principal components analysis: relationship between first and second principal component

I'm really struggling with understanding the idea of Principal Component Analysis and would appreciate any help. We have a m multivariate input time series $ \begin{align} X_{t} &= ...
0
votes
0answers
9 views

What's this model called? Regression with a rotated bivariate gaussian variable as outcome. Rotation is based on eigenvalue decomposition

In my latest modeling effort I came up with the following model. Observed variables $y$ - outcome variable, $N \times S \times 2$ array where $S$ is number of sessions/repetitions and $N$ is ...
0
votes
1answer
39 views

PCA for data containing IP address

How can I calculate PCA for variables which contain IP address (which is not a continuous variable)?
1
vote
0answers
22 views

Calculating eigenvectors for PCA and obtaining a different result from Matlab/EViews

Based on the variance-covariance matrix $$\begin{pmatrix}0.62 & 0.62\\ 0.62 & 0.72\end{pmatrix},$$ I have calculated the following eigenvalues $$λ_1 = 0.0500, \;λ_2 = 1.2840.$$ I then ...
0
votes
1answer
36 views

PCA on a Likert scale data

I am trying to conduct a small experiment based on Likert style data. I have a total of 20 questions, 10 are referring to a latent construct of happiness, and the other 10 to a latent construct of ...
0
votes
0answers
11 views

PCA for mixed variables

I have a dataset which contains many variables : continuous and discrete variables. I have a discrete variable adress which can include many value (if for example the customer changes his location, ...
0
votes
0answers
15 views

Factor analysis Geometric interpretation of common variance [duplicate]

I am trying to understand the fundamental differences between PCA and Factor Analysis. PCA is straight forward in that you take the eigenvectors of the data's variance (and hence considering all the ...
4
votes
1answer
322 views

Combining multiple variables into one “score”

My question is very similar to this one, which was not solved unfortunately. I am working on a project for which I want to rank countries by means of their HIV/AIDS burden. So I collected a lot of ...
0
votes
1answer
68 views

Standardization before PCA with data in same units and similar interval? [duplicate]

We have 16 variables which are indices produced by calculations based on ratio (unitless in fact). Some examples of the ranges of our variables are (0.450-0.750), (0.000 - 0.800) and (0.000 - 1.000). ...
3
votes
1answer
90 views

What's the relationship between initial eigenvalues and sums of squared loadings in factor analysis?

On the one hand I read in a comment here that: You can't speak of "eigenvalues" after rotation, even orthogonal rotation. Perhaps you mean sum of squared loadings for a principal component, ...
0
votes
0answers
16 views

PCA covariates to fit the linear model

I have pairwise relationship (relation) for items in ColA and ColB in table mydf which is also affected by various covariates ...
1
vote
0answers
11 views

PCA, reverse coding, TIPI questionnaire used as predictor, ANOVA vs Multiple Regression

I am in the process of analyzing a questionnaire measuring attitudes: recording as predictors 1) gender, 2) religion, AND 3) the 10-item personality inventory (TIPI, 2003); dependent variables ...
0
votes
0answers
28 views

Tukey's test of additivity or Tukey's one degree of freedom test

Does anyone know anything about this test? I have a scale of 9 questions and I don't know how to explain Tukey's test of additivity or Tukey's one degree of freedom test. How to report this test and ...
-5
votes
0answers
16 views

HELP with Principal Components Analysys (PCA) in WEKA [duplicate]

I have questions about the PCA filter . Question 1: How can I improve the speed of filtering PCA? My base has 7200 attributes, but takes many hours to complete the process. 2nd Question: After I do ...
6
votes
1answer
63 views

What does a wedge-like shape of the PCA plot indicate?

In their paper on autoencoders for text classification Hinton and Salakhutdinov demonstrated the plot produced by 2-dimensional LSA (which is closely related to PCA): . Applying PCA to absolutely ...
1
vote
0answers
46 views

How to compute explained variances in PCA with varimax rotation?

I've done a PCA and varimax-rotated the EOF loadings (i.e. eigenvectors of the covariance matrix scaled by the square roots of the respective eigenvalues) and calculated the rotated PCs by multiplying ...