Principal component analysis is a technique to decompose an array of numerical data into a set of orthogonal vectors (uncorrelated linear combinations of the variables) called principal components. The first few principal components often suffice to grasp nearly all the multivariate variability of ...

learn more… | top users | synonyms (1)

2
votes
1answer
59 views

Dimensionality reduction when number of samples is much larger than number of features

I was wondering what happens when the number of samples is much larger (e.g. $\times 200\:000$ times more) than the number of features? Is there any recommended way of reducing the samples' ...
-1
votes
0answers
12 views

PCA correlation req normalizing data? [duplicate]

Am I required to normalize the data (eg. using zscore) before performing PCA? I performed PCA on correlation matrix. I think correlation matrix is a standardized (both centered and rescaled) data. I ...
3
votes
1answer
116 views

Linear independence vs statistical independence (PCA and ICA)

I'm reading this interesting paper on application of ICA to gene expression data. The authors write: [T]here is no requirement for PCA components to be statistically independent. That is true, ...
1
vote
2answers
49 views

How to perform PCA in Matlab when number of dimensions is larger than number of observations?

I have a data matrix of say, $3000 \times 200$, i.e. I have $3000$-dimensional observations from $200$ subjects. How can I reduce the dimensionality to $1000$ in MATLAB? With bigger numbers, ...
0
votes
0answers
31 views

How to obtain an exact value of the determinant of a correlatiom matrix when doing PCA in SPSS?

When I perform a Principal Components Analysis (PCA) in SPSS 22 I get a footnote under the correlation matrix which states that the value of the determinant $=.000$. I am not able to display more ...
1
vote
0answers
21 views

How to fit a single quadratic term to a regression

I have a high dimensional multivariate model and am fitting linear weights to each of the $N$ free variables using a classic stable SVD matrix solver. This works. I want to improve the fit by using a ...
4
votes
2answers
79 views

How to interpret and translate this statement from a Wikipedia article on PCA?

I have to write a Wikipedia article about Principal Component Analysis in my own language and partially I translate it from the English version, but I don't understand a statement from it, more ...
2
votes
2answers
78 views

Do I need to take out any predictors from multiple regression if I put in some principal components as additional predictors?

I have an assignment which involves one area-level dataset made of $366$ scale variables. I have to perform PCA, compare it with rates of an additional response variable $X$, and comment on its face ...
-1
votes
0answers
31 views

How can representation in kernel PCA change with permuting the rows of data matrix?

I observe a phenomenon I cannot explain, maybe you can: I perform Kernel PCA with RBF kernel for my data, but no matter which order I put them in on the data matrix (i.e. if I permute the rows, ...
2
votes
1answer
41 views

Eigenfaces in R: How to reconstruct original features from principal components? [duplicate]

I've performed PCA on face images dataset and I'm not sure how can I use the most informative principal components to show the "reduced" image. The original image is 96*96 pixels (96*96 = 9216) and I ...
2
votes
1answer
81 views

Where is the indeterminacy of factor values on this plot explaining factor analysis?

It is a well-known fact that in principal component analysis (PCA) we can obtain true values of components but in factor analysis (FA) we cannot obtain true values of common factors. We can compute ...
2
votes
1answer
147 views

What is component (factor) score coefficient matrix in PCA or factor analysis and how is it calculated?

As per my understanding, in PCA based on correlations we get factor (= principal component in this instance) loadings which are nothing but the correlations between variables and factors. Now when I ...
8
votes
1answer
103 views

What criteria to use for separating variables into explanatory variables and responses for ordination methods in ecology?

I have different variables that interact within a population. Basically I have been doing an inventory of millipedes and measuring some other values of the terrain, like: The species and the amount ...
2
votes
0answers
48 views

Logistic principal component regression where PCs are correlated with an additional binary predictor

My scenario is this: I collected a bunch of vegetation data (% cover counts in a quadrant at different heights) in patches where birds were seen foraging and also in control patches where no foraging ...
4
votes
1answer
73 views

Under which conditions do PCA and FA yield similar results?

Under which conditions can principal components analysis (PCA) and factor analysis (FA) be expected to yield similar results?
1
vote
1answer
49 views

Sparse coding vs. sparse PCA, are they the same thing?

Are they the same thing? If not, could someone possibly explain the difference or point to the seminal papers describing the approaches? I am looking not for a detailed technical exposition, but a ...
-1
votes
0answers
19 views

Principal Component Analysis (hyperspectral image analysis) [duplicate]

I have two dataset (Shortwave Infrared and Thermal Infrared hyperspectral images acquired by two different airborne remote sensing sensors), the Shortwave infrared image has 82 bands, while the ...
-1
votes
1answer
71 views

PCA Feature Selection [duplicate]

Actually, i have 540x46 matrix, (540 observations and 46 features) and after using PCA by considering 95% variance, it is reduced to 540x12 matrix. So, is it possible to know which 12 features from ...
2
votes
1answer
60 views

Why are faces so low-dimensional? (i.e. why does the eigenface approach work so well?)

I (think I) understand PCA, but it still blows my mind that you can get such good reconstruction of the faces with only like 20 eigenface components. How is that possible?? Why should it be the case ...
1
vote
1answer
48 views

Is it possible to project a new vector onto the PC space using kernel PCA?

Let $X_{N \times d}$ be the data matrix, where $N$ is the number of samples and $d$ the size of the features space. Using kernel PCA (kPCA), one first computes a kernel matrix $K_{N \times N}$, and ...
2
votes
0answers
60 views

What does it mean to compute eigenvectors of a covariance matrix if the data were not centered first? [duplicate]

Say $\mathbf{X} \in \mathbb{R}^{n \times p}$ and $\boldsymbol{\Sigma} = \frac{1}{n}\mathbf{X}'\mathbf{X}$. The eigenvector decomposition of $\boldsymbol{\Sigma}$ gives $\boldsymbol{\Sigma} = ...
8
votes
1answer
287 views

How does “Fundamental Theorem of Factor Analysis” apply to PCA, or how are PCA loadings defined?

I'm currently going through a slide set I have for "factor analysis" (PCA as far as I can tell). In it, the "fundamental theorem of factor analysis" is derived which claims that the correlation ...
-1
votes
0answers
13 views

How to reconstruct the data? [duplicate]

I have compressed the data( seven sensors are measuring 125 days temperature) using Principal Component Analysis . Then How to reconstruct the data ? What formula should i use?
4
votes
3answers
153 views

How to interpret PCA on time-series data?

I am trying to understand the use of PCA in a recent journal article titled "Mapping brain activity at scale with cluster computing" Freeman et al., 2014 (free pdf available on the lab website). They ...
1
vote
2answers
87 views

PCA on train and test datasets: should I run one PCA on train+test or two separate on train and on test? [duplicate]

I'm doing an image classification task and the number of features of each example image is pretty huge (3,072: # pixels in each image). I'm thinking of using PCA to reduce the # features of each image ...
2
votes
1answer
65 views

Why doesn't it make sense to preprocess data with PCA before classification?

For some classification algorithms, assuming independence of data helps reduce the number of parameters to estimate. Why then not just to apply a method like PCA (or ICA) to the original features to ...
1
vote
2answers
36 views

Choosing the number of principal components to retain before training a neural network for classification

I am working on neural networks and I am currently creating a perceptron that will work as a classifier for a data set of images with faces. I am required to perform pca (principal component analysis) ...
10
votes
2answers
572 views

How to understand “nonlinear” as in “nonlinear dimensionality reduction”?

I am trying to understand the differences between the linear dimensionality reduction methods (e.g., PCA) and the nonlinear ones (e.g., Isomap). I cannot quite understand what the (non)linearity ...
5
votes
1answer
99 views

Meaningful inference about data structure based on components with low variance in PCA

A lot of microbiome (microbial ecology) papers that I have come across use either principal component analysis (PCA) or principal coordinate analysis (PCoA) to make conclusions about the data. A lot ...
7
votes
1answer
500 views

Confused about the visual explanation of eigenvectors: how can visually different datasets have the same eigenvectors?

A lot of statistics textbooks provide an intuitive illustration of what the eigenvectors of a covariance matrix are: The vectors u and z form the eigenvectors (well, eigenaxes). This makes sense. ...
1
vote
0answers
37 views

Non-decaying eigenvalues in Kernel PCA with small kernel width

I noticed that when I use a small width kernel (RBF) with PCA, I get my desired result (clustering in this case), but I do not get a decay in the eigenvalues (they stay about the same value). Is that ...
2
votes
1answer
46 views

How to select a subsample of fixed size to maximize its total PCA variance?

I would like to use PCA to help design my genomics experiment. I can only afford to perform my experiment on a limited number of genotypes so would like to maximize the variation of the ones I ...
2
votes
1answer
25 views

Can nonlinear clustering produce 'fake' results?

I know that overfitting in classification is possible when using, for instance, an RBF kernel, due to its infinite dimension. But, is it possible to get (in a similar manner) fake clustering results ...
1
vote
1answer
50 views

How can I estimate a principal component from incomplete data? [duplicate]

I would like to know the best way to estimate a principal component's latest value, if I only have partial information about the latest variable data points: Assuming I have 5 variables: ...
5
votes
1answer
102 views

What is PCA doing with autocorrelated data?

Just because some correspondent posed an interesting question concerning methods of computation of autocorrelation, I began to play with it, nearly without any knowledge about time series and ...
3
votes
1answer
63 views

Why are there only $N-1$ principal axes for $N$ data points if the number of dimensions is larger than $N$?

In PCA, when the number of dimensions $d$ is much much greater than the number of samples $N$, why is it that you will have at most $N-1$ non-zero eigenvectors? In other words, the rank of the ...
1
vote
1answer
56 views

PCA clustering results 'ruined' by standartization

I have some data that I want to classify. As an initial measure, I did PCA for the data and I saw two distinct clusters of my data. However, when standardizing the data, the two clusters disappear. ...
32
votes
6answers
1k views

Is there any good reason to use PCA instead of EFA?

In some disciplines, PCA (principal component analysis) is systematically used without any justification, and PCA and EFA (exploratory factor analysis) are considered as synonyms. I therefore ...
1
vote
0answers
66 views

What's wrong with t-SNE vs PCA for dimensional reduction using R?

I have a matrix of 336x256 floating point numbers (336 bacterial genomes (columns) x 256 normalized tetranucleotide frequencies (rows), e.g. every column adds up to 1). I get nice results when I run ...
4
votes
1answer
73 views

Why are the singular values of a standardized data matrix not equal to the eigenvalues of its correlation matrix?

Conceptually, aren't the eigenvalues of a correlation matrix and the singular values of the associated scaled data matrix supposed to be the same? The below illustration is saying that it isn't so. ...
0
votes
1answer
36 views

PCA or MCA for binarized data

I am working with bioinformatics and I have data that looks like the following: ...
1
vote
0answers
33 views

Principal components analysis on nested data

I'm working on a piece of analysis that requires identifying a small set of variables that summarize the variation found in a larger set of principal observations on teacher practice. Given the nature ...
0
votes
0answers
41 views

Does it mean anything when all items load negatively on one factor when several factors are output?

So I am fairly familiar with factor analysis, and am aware of answers here and here that tangentially address my question. I believe I am right in my answer to the question I'm asking, but I wanted to ...
1
vote
2answers
61 views

Motive behind preserving variance

Dimensionality reduction techniques preserve some properties of the data. I was wondering how preserving variance (as PCA does) can be helpful? Precisely speaking, PCA takes the covariance matrix and ...
1
vote
2answers
86 views

How to transform test set to the PCA space of the training set, if the features in train and test are different?

I'm working on a text classification project, and I want to reduce the tf-idf matrix dimension with Principal Component Analysis (PCA) and then train my model with this, which is pretty ...
8
votes
1answer
350 views

Is there any advantage of SVD over PCA?

I know how to calculate PCA and SVD mathematically and I know that both can be applied to Linear Least Squares regression. The main advantage of SVD mathematically seems to be that it can be applied ...
2
votes
0answers
45 views

Imputation of missing values for doing PCA in R [duplicate]

I have a dataset with approximately 4000 rows and 150 columns. I want to predict the values of a single column (= target). The data is on cities (demography, social, economic, ... indicators). A lot ...
3
votes
2answers
149 views

Why does my loading matrix following PCA with a varimax rotation contain only ones and zeros?

I'm running a PCA using the R function prcomp. This is the function: d2.pca <- prcomp(sel.d2, center=TRUE, scale.=TRUE) So ...
0
votes
1answer
53 views

Bi-normal separation feature selection (BNS) in R

I'm doing binary classification on highly dimensional text data, with a biased class distribution. After reading this paper, i found out about BNS feature selection. Is there any package that ...
1
vote
0answers
25 views

How to program automated shrinkage for a subset of terms in R?

I've got data from a randomized experiment that includes a lot of covariates. I'm interested $\delta$ from a model of the form $y = g(\delta T + X'\beta+ \epsilon)$, where $T$ is randomly assigned and ...