Principal component analysis is a technique to decompose an array of numerical data into a set of orthogonal vectors (uncorrelated linear combinations of the variables) called principal components. The first few principal components often suffice to grasp nearly all the multivariate variability of ...

learn more… | top users | synonyms (1)

0
votes
0answers
19 views

How to find original features corresponding to the first two principal components? [duplicate]

I have a set of data described by $n$ features. I do a principal component analysis (PCA) to reduce it to just 2 dimensions so I can make a 2D plot of the data, with the first two Principal ...
0
votes
0answers
36 views

In PCA, can the values in the principle component vectors which are close to zero be removed to see the important features? [duplicate]

In PCA, when I extract the principle component vectors, I am choosing the first vector with the largest corresponding eigenvalue. I notice that some of the values in this vector are close to zero. Can ...
0
votes
0answers
17 views

How is a dot calculated in PCA plot?

I only have a little background in statistics. I use SIMCA-p for metabolomic analysis. The PCA plot shows a dot for each sample. I usually analyze 100 samples with 80 variables. How is a dot assigned? ...
14
votes
3answers
731 views

Weird correlations in the SVD results of random data; do they have a mathematical explanation or is it a LAPACK bug?

I observe a very weird behaviour in the SVD outcome of random data, which I can reproduce in both Matlab and R. It looks like some numerical issue in the LAPACK library; is it? I draw $n=1000$ ...
0
votes
0answers
22 views

Correlation of principal component [closed]

I am asked to calculate the correlation coefficients each of the first 3 principal components with each of the 3 factors respectively. These 3 factors are the market return, HML/SMB and risk free ...
0
votes
0answers
9 views

How to use the components from PCA in discriminant analysis? [migrated]

Any clue on how to do this in SAS Enterprise Guide?
0
votes
0answers
13 views

Principal Components Analysis Multiple Loadings [duplicate]

I currently ran an PCA using SPSS and found that many of the items loaded on multiple components. I tried a variety of rotations (Oblique etc. due to correlated items) but these did not help. There ...
2
votes
1answer
47 views

Why would one remove items which load on more than one component or factor in PCA or FA?

Why would a researcher remove items which load onto more than one component after rotation in PCA using Varimax? A couple of studies I'm using as the basis for a study I'm conducting have done this ...
9
votes
1answer
436 views

Are PCA components of multivariate Gaussian data statistically independent?

Are PCA components (in principal component analysis) statistically independent if our data is multivariate normally distributed? If so, how can this be demonstrated/proven? I ask because I saw this ...
0
votes
0answers
29 views

If my classifier is trained on PCA compressed images, do I have to use PCA compressed images during live?

Recently I discovered how to use PCA to compress my images (for dimensionality reduction). These images are then used to train an ANN image classifier. The purpose of the classifier is to classify ...
0
votes
2answers
32 views

The meaning of units on the axes of a PCA plot

I've used the R packages DESeq2 and ggplot2 and the following code ...
2
votes
2answers
63 views

Is it appropriate to do an ANOVA on a feature selected via inspecting PCA results?

I've been given a dataset consisting of 8 dimensional feature vectors for 4 classes of objects. I was asked to find the features that best distinguish the classes, and write up a short report. My ...
1
vote
0answers
55 views

PCA and diagonalization of the covariance matrix

I am preparing for an upcoming exam and having looked at older exams I found one PCA related question I am having trouble understanding: You have a dataset of $N$ two-dimensional points $\ y^t $. ...
1
vote
1answer
30 views

Understanding proportion of variance in PCA

One of my exam question was as follows: Assume the covariance matrix of your dataset $X $ is $\Sigma$ and while doing Principal Component Analysis, you found $\Sigma = CDC^T $. Here $D $ is the ...
1
vote
1answer
46 views

How to interpret this PCA biplot?

I am approaching PCA analysis for the first time, and have difficulties on interpreting the results. This is my biplot (produced by Matlab's functions pca and ...
0
votes
0answers
15 views

Obatining PCA residusals in Python's scikit-learn [migrated]

I'm using scikit-learn to conduct PCA on a large dataset with the goal of removing large, common sources of variance from a matrix X. Thus, I'd like to produce a ...
3
votes
1answer
67 views

What is the difference between feature selection and dimensionality reduction?

I know that both feature selection and dimensionality reduction aim towards reducing the number of features in the original set of features. What is the exact difference between the two if we are ...
0
votes
0answers
20 views

Problems using pcr (from pls library) in R with large number of qualitative variables

I'm trying to classify a variable into either 0 or 1, using 50 factors, with a sample size of 2000. 25% of the dependent variables are 0 and the rest are 1. Of these factors, 30 are categorical. I've ...
0
votes
1answer
29 views

How to distinguish one data set to the other using PCA

I have 5 set data, each for different observations, i.e.: o1, o2, o3, o4, o5. All those 5 set have 10 similar variables. I want to distinguish data set o1 to the other four data set but only want to ...
2
votes
1answer
42 views

Should ordinal variables be normalized for PCA?

I need to analyze my (ecological) data with PCA, but the data don't seem to meet the assumption of normality very well. The problem is, that out of my 9 variables only two are continuous and the ...
5
votes
1answer
97 views

First principal component of 2D data forming a rectangle?

What is the first principal component of points that form a "filled" rectangle in the 2D space? Is it one of the diagonals? Or are the first two principal components basically the sides of the ...
0
votes
1answer
36 views

Multiple T Tests

On one of my questionnaires, which measure learning behaviours, after PCA, I have 4 subgroups, Efficacy/Perseverance/Effort/Achievement - I have run a T-Test and I have 4 sets of data comparing my ...
0
votes
1answer
57 views

LDA, PCA and k-means: how are they related?

I am trying to understand how linear discriminant analysis (LDA) is related to principal component analysis (PCA) and k-means clustering method. As an example, here is a comparison between PCA and ...
0
votes
0answers
17 views

PCA reduction and low-reliability components

I'm working on a survey with 288 observation in total (108 complete answers used) and around 200 variables. I'm working on reducing those number using Principal Components Analysis, using R. Suppose ...
0
votes
1answer
58 views

PCA in r using prcomp: should data variable in prcomp function be correlation matrix?

I am hoping someone can check this code to ensure that I have interpreted the various pieces of PCA correctly. I am trying to figure out a way to identify the leading contributors to the performance ...
1
vote
0answers
62 views

How to reduce dimension of the sampling procedure?

I am stuck with this problem for a long time, hopefully I can get help here! Basically, I want to sample from a posterior distribution that looks like, \begin{align*} X &\sim ...
2
votes
1answer
89 views

Variance estimation in a one-factor linear model

I was given a dataset (a mat file) of $100\: 000$ observations, each with $50$ dimensions (coordinates). Denote matrix $X$ a $50\times 100\:000$ matrix in which each column was generated according to: ...
0
votes
0answers
14 views

Is there a framework for dealing with time-varying sequences of matrices (specifically for applications to finance, involving PCA)?

I am using principal component analysis to model yield curves for certain financial instruments. My dataset is a matrix $X$ where $[X]_{ti}$ is the observation of the $i$th instrument on day $t$. I am ...
2
votes
0answers
34 views

Issue with the proof of PCA

I found a very nice PCA proof over here PCA_proof and I'm trying to understand it (I don't know what Langrange multipliers are so I'm trying my best). From the second page of the previous link, ...
3
votes
1answer
37 views

Principal Component Regression with an additional factor

I am looking to tease out the significance and contribution of a particular variable to 2 different continuous responses. I have 7 continuous variables I know to be influential on the two responses ...
0
votes
1answer
39 views

Possible ways to convert predicted scores from PCA analysis?

I have my predicted scores from PCA analysis, and my predicted scores have both negative and positive numbers. For instance, minimum value is - 4 and maximum value is 4. I plan to use the predicted ...
0
votes
0answers
14 views

Dealing with seasonality when doing dimensionality reduction

I want to perform dimensionality reduction (in particular, PCA) on a data set that is highly seasonal. One approach that I came across when researching this is "seasonal PCA", where you split your ...
0
votes
0answers
35 views

What are the potential disadvantages of doing kernel PCA?

I was trying to learn more of the motivation around kernel PCA. Its clear to me that one might need to change the representation of the data if it lies in a non-linear space, hence, the projection ...
2
votes
0answers
14 views

Defining the probability distribution of a Random vector given the probability over a “sub-vector”

Suppose I want the probability distribution over a random vector $X={X_1 ,X_2 ... X_n }$. What I already have with me is the distribution over a subvector $X_i , X_{i+1}...X_m$, $m<n$ which I ...
17
votes
1answer
553 views

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix ...
5
votes
2answers
237 views

Why do we divide by the standard deviation and not some other standardizing factor before doing PCA?

I was reading the following justification (from cs229 course notes) on why we divide the raw data by its standard deviate: even though I understand what the explanation is saying, it is not clear ...
0
votes
0answers
31 views

PCA and cross-validation [duplicate]

I am fairly new to the machine learning, and I have been going over all the great posts about cross-validation today and I have a question regarding PCA and cross-validation, I don't have enough ...
2
votes
0answers
42 views

Creating a single index from several principal components retained from PCA

I am using Principal Component Analysis (PCA) to create an index required for my research. My question is how I should create a single index by using the retained principal components calculated ...
8
votes
1answer
66 views

Is there any required amount of variance captured by PCA in order to do later analyses?

I have a dataset with 11 variables and PCA (orthogonal) was done to reduce the data. Deciding on the number of components to keep it was evident for me from my knowledge about the subject and the ...
0
votes
0answers
10 views

Intrinsic topology and metrics… (looking for name of a method)

Suppose I have an n-dimensional dataset and its points are roughly in the shape of an n-dimensional horseshoe or something along those lines. Using euclidian distance might be a bad idea, since points ...
4
votes
1answer
105 views

Why is variance (instead of standard deviation) the default measure of information content in principal components?

The information content of principal components is almost always expressed as a variance (e.g., in scree plots or in statements like "the first three PCs contain 95% of the total data variance"). The ...
3
votes
1answer
148 views

Combining several variables into one outcome score: How is it done in the machine learning community?

I have got 8 cognitive (continuous) behaviour variables and would like to combine them into a composite score. I would then like to find the best predictors of this outcome (from about 50 predictors). ...
2
votes
1answer
86 views

Does PCA mean selecting most important features and ignoring the others?

Principal component analysis (PCA) is used to reduce the dimensions in our data set. While explaining PCA, they say that they are projecting the data to where there is huge variance; is that the same ...
4
votes
2answers
64 views

How many components to use in PCA in order to preserve a certain amount of variance?

I want to reduce the dimensionality of my data with PCA, until it preserves $\alpha = 0.99$ of the variance. How do I decide how many eigenvectors I should use? So I'm looking for a function ...
2
votes
0answers
45 views

How do I get from the eigenvectors of the covariance matrix to the regression parameters? [duplicate]

I have a linear regression problem $$ y = a x + b$$ with errors on $x$ and $y$ that are uncorrelated and unitary and I have to find $a$ and $b$. To do this, I want to use principal component ...
0
votes
0answers
31 views

Reduction of species variables in vegetative analysis

Edited following helpful feedback. I have vegetation species data for a number of grassland habitat sites, and am preparing to begin Exploratory Data Analysis. Data was collected in 100 quadrats over ...
0
votes
1answer
27 views

What should can I do with some items loadings on unexpected construct?

I conducted a Principal Component Analysis to reduce the items and dimensions. But some items loaded on unexpected construct and the items have a low face validity with the construct. Is that a ...
4
votes
1answer
102 views

How to choose a kernel for kernel PCA?

What are the ways to choose what kernel would result in good data separation in the final data output by kernel PCA (principal component analysis), and what are the ways to optimize parameters of the ...
3
votes
1answer
60 views

What exactly is the procedure to compute principal components in kernel PCA?

In kernel PCA (principal component analysis) you first choose a desired kernel, use it to find your $K$ matrix, center the feature space via the $K$ matrix, find its eigenvalues and eigenvectors, then ...
1
vote
1answer
41 views

Integrating length for input-space feature PC projections in kernel PCA

I read a paper detailing the algebraic process of kernel PCA. I have question though: the paper details the projection of new points onto the new eigenvectors in the feature space, but what if I want ...