Principal component analysis is a technique to decompose an array of numerical data into a set of orthogonal vectors (uncorrelated linear combinations of the variables) called principal components. The first few principal components often suffice to grasp nearly all the multivariate variability of ...

learn more… | top users | synonyms (1)

-1
votes
0answers
10 views

How to explain the correlation between the mean of each element and it's corresponding PC1 coefficient in a data.frame?

I am doing some PCA analysis by R now,I have a data,which is a data.frame:a,ncol(a) =1000,nrow(a)=100,each row represent a sample,each col represent a feature,so when I use R do PCA analysis like ...
0
votes
0answers
20 views

Log transformation on iris dataset - why [on hold]

What is the reason for applying log transformation on iris dataset when it is used in PCA? As shown in this link: https://tgmstat.wordpress.com/2013/11/28/computing-and-visualizing-pca-in-r/ Thanks ...
2
votes
1answer
81 views

Variance estimation in a one-factor linear model

I was given a dataset (a mat file) of $100\: 000$ observations, each with $50$ dimensions (coordinates). Denote matrix $X$ a $50\times 100\:000$ matrix in which each column was generated according to: ...
0
votes
0answers
12 views

Is there a framework for dealing with time-varying sequences of matrices (specifically for applications to finance, involving PCA)?

I am using principal component analysis to model yield curves for certain financial instruments. My dataset is a matrix $X$ where $[X]_{ti}$ is the observation of the $i$th instrument on day $t$. I am ...
2
votes
0answers
32 views

Issue with the proof of PCA

I found a very nice PCA proof over here PCA_proof and I'm trying to understand it (I don't know what Langrange multipliers are so I'm trying my best). From the second page of the previous link, ...
0
votes
0answers
17 views

How to add supplementary variables after PCA using principal function (psych package) in R? [closed]

Is there a way to add supplementary variables to a PCA plot when the PCA was conducted using the principal function from the psych package in R? I know that the PCA from the FactoMiner package has ...
3
votes
1answer
37 views

Principal Component Regression with an additional factor

I am looking to tease out the significance and contribution of a particular variable to 2 different continuous responses. I have 7 continuous variables I know to be influential on the two responses ...
0
votes
1answer
30 views

Possible ways to convert predicted scores from PCA analysis?

I have my predicted scores from PCA analysis, and my predicted scores have both negative and positive numbers. For instance, minimum value is - 4 and maximum value is 4. I plan to use the predicted ...
0
votes
0answers
14 views

Dealing with seasonality when doing dimensionality reduction

I want to perform dimensionality reduction (in particular, PCA) on a data set that is highly seasonal. One approach that I came across when researching this is "seasonal PCA", where you split your ...
0
votes
0answers
30 views

What are the potential disadvantages of doing kernel PCA?

I was trying to learn more of the motivation around kernel PCA. Its clear to me that one might need to change the representation of the data if it lies in a non-linear space, hence, the projection ...
2
votes
0answers
13 views

Defining the probability distribution of a Random vector given the probability over a “sub-vector”

Suppose I want the probability distribution over a random vector $X={X_1 ,X_2 ... X_n }$. What I already have with me is the distribution over a subvector $X_i , X_{i+1}...X_m$, $m<n$ which I ...
16
votes
1answer
255 views

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix ...
4
votes
1answer
178 views

Why do we divide by the standard deviation and not some other standardizing factor before doing PCA?

I was reading the following justification (from cs229 course notes) on why we divide the raw data by its standard deviate: even though I understand what the explanation is saying, it is not clear ...
0
votes
0answers
30 views

PCA and cross-validation [duplicate]

I am fairly new to the machine learning, and I have been going over all the great posts about cross-validation today and I have a question regarding PCA and cross-validation, I don't have enough ...
2
votes
0answers
32 views

Creating a single index from several principal components retained from PCA

I am using Principal Component Analysis (PCA) to create an index required for my research. My question is how I should create a single index by using the retained principal components calculated ...
7
votes
1answer
52 views

Is there any required amount of variance captured by PCA in order to do later analyses?

I have a dataset with 11 variables and PCA (orthogonal) was done to reduce the data. Deciding on the number of components to keep it was evident for me from my knowledge about the subject and the ...
0
votes
0answers
10 views

Intrinsic topology and metrics… (looking for name of a method)

Suppose I have an n-dimensional dataset and its points are roughly in the shape of an n-dimensional horseshoe or something along those lines. Using euclidian distance might be a bad idea, since points ...
4
votes
1answer
84 views

Why is variance (instead of standard deviation) the default measure of information content in principal components?

The information content of principal components is almost always expressed as a variance (e.g., in scree plots or in statements like "the first three PCs contain 95% of the total data variance"). The ...
3
votes
1answer
122 views

Combining several variables into one outcome score: How is it done in the machine learning community?

I have got 8 cognitive (continuous) behaviour variables and would like to combine them into a composite score. I would then like to find the best predictors of this outcome (from about 50 predictors). ...
2
votes
1answer
71 views

Does PCA mean selecting most important features and ignoring the others?

Principal component analysis (PCA) is used to reduce the dimensions in our data set. While explaining PCA, they say that they are projecting the data to where there is huge variance; is that the same ...
4
votes
2answers
54 views

How many components to use in PCA in order to preserve a certain amount of variance?

I want to reduce the dimensionality of my data with PCA, until it preserves $\alpha = 0.99$ of the variance. How do I decide how many eigenvectors I should use? So I'm looking for a function ...
1
vote
0answers
23 views

How do I get from the eigenvectors of the covariance matrix to the regression parameters?

I have a linear regression problem $ y = a x + b$ with errors on $x$ and $y$ that are uncorrelated and unitary and I have to find $a$ and $b$. To do this, I want to use PCA. 1) Calculate the $n ...
0
votes
0answers
24 views

Reduction of species variables in vegetative analysis

Edited following helpful feedback. I have vegetation species data for a number of grassland habitat sites, and am preparing to begin Exploratory Data Analysis. Data was collected in 100 quadrats over ...
0
votes
1answer
26 views

What should can I do with some items loadings on unexpected construct?

I conducted a Principal Component Analysis to reduce the items and dimensions. But some items loaded on unexpected construct and the items have a low face validity with the construct. Is that a ...
-1
votes
0answers
10 views

which could be better for rationale weight assignment to different properties, PCA or Shannon Entropy?

Rationale weight assignment to the factors normally done by either using Shannon entropy or by using PCA, but I am bit confused which could be better between these two analysis, please help?
4
votes
1answer
77 views

How to choose a kernel for kernel PCA?

What are the ways to choose what kernel would result in good data separation in the final data output by kernel PCA (principal component analysis), and what are the ways to optimize parameters of the ...
3
votes
1answer
56 views

What exactly is the procedure to compute principal components in kernel PCA?

In kernel PCA (principal component analysis) you first choose a desired kernel, use it to find your $K$ matrix, center the feature space via the $K$ matrix, find its eigenvalues and eigenvectors, then ...
1
vote
0answers
22 views

Integrating length for input-space feature PC projections in kernel PCA

I read a paper detailing the algebraic process of kernel PCA. I have question though: the paper details the projection of new points onto the new eigenvectors in the feature space, but what if I want ...
17
votes
4answers
333 views

What makes the Gaussian kernel so magical for PCA, and also in general?

I was reading about kernel PCA (1, 2, 3) with Gaussian and polynomial kernels. How does the Gaussian kernel separate seemingly any sort of nonlinear data exceptionally well? Please give an intuitive ...
2
votes
2answers
188 views

Why are principal components in PCA (eigenvectors of the covariance matrix) mutually orthogonal? [duplicate]

Why are principal components in PCA mutually orthogonal? I know that PCA can be calculated by eig(cov(X)), where X is centered. ...
4
votes
1answer
85 views

What norm of the reconstruction error is minimized by the low-rank approximation matrix obtained with PCA?

I am a bit confused between $\parallel \cdot \parallel_2$ and $\parallel \cdot \parallel_F$. Given a PCA approximation of matrix $X$ with a matrix $\hat X$, we know that $\hat X$ is the best ...
7
votes
0answers
152 views

Blind source separation of convex mixture?

Suppose I have $n$ independent sources, $X_1, X_2, ..., X_n$ and I observe $m$ convex mixtures: \begin{align} Y_1 &= a_{11}X_1 + a_{12}X_2 + \cdots + a_{1n}X_n\\ ...&\\ Y_m &= a_{m1}X_1 + ...
2
votes
0answers
46 views

Factor analysis with varimax rotation gives very different answer from PCA?

I asked expert raters to evaluate several subject on six dimensions of creativity. Now, I am using factor analysis (factanal()) and PCA(princomp()) to see of these dimensions are measuring distinct ...
1
vote
0answers
36 views

Principal component regression on polynomial terms

One of the data sets I am working upon had 3 variables which were having almost 100% correlation among themselves. Since I am learning regression modelling I thought I'll do principal component ...
9
votes
3answers
371 views

Is there any value in dimensionality reduction of a data set where all variables are approximately orthogonal?

Suppose I have an $N$-dimensional data set where the $N$ dimensions are roughly orthogonal (have correlation zero). Is there any utility in terms of: Visualization Representation (for classifier ...
1
vote
2answers
120 views

What can be inferred from a short multivariate time series?

I have annual observations of 24 variables over 10 years, and I would like to identify evidence of structural change (regime shift). The data pertain to university enrollment & spending, so I ...
12
votes
3answers
227 views

Interpretation of ridge regularization in regression

I have several questions regarding the ridge penalty in the least squares context: $$\beta_{ridge} = (\lambda I_D + X'X)^{-1}X'y$$ 1) The expression suggests that the covariance matrix of X is ...
2
votes
1answer
76 views

Dimensionality reduction when number of samples is much larger than number of features

I was wondering what happens when the number of samples is much larger (e.g. $\times 200\:000$ times more) than the number of features? Is there any recommended way of reducing the samples' ...
3
votes
1answer
134 views

Linear independence vs statistical independence (PCA and ICA)

I'm reading this interesting paper on application of ICA to gene expression data. The authors write: [T]here is no requirement for PCA components to be statistically independent. That is true, ...
1
vote
2answers
73 views

How to perform PCA in Matlab when number of dimensions is larger than number of observations?

I have a data matrix of say, $3000 \times 200$, i.e. I have $3000$-dimensional observations from $200$ subjects. How can I reduce the dimensionality to $1000$ in MATLAB? With bigger numbers, ...
0
votes
0answers
62 views

How to obtain an exact value of the determinant of a correlatiom matrix when doing PCA in SPSS?

When I perform a Principal Components Analysis (PCA) in SPSS 22 I get a footnote under the correlation matrix which states that the value of the determinant $=.000$. I am not able to display more ...
1
vote
0answers
24 views

How to fit a single quadratic term to a regression

I have a high dimensional multivariate model and am fitting linear weights to each of the $N$ free variables using a classic stable SVD matrix solver. This works. I want to improve the fit by using a ...
4
votes
2answers
81 views

How to interpret and translate this statement from a Wikipedia article on PCA?

I have to write a Wikipedia article about Principal Component Analysis in my own language and partially I translate it from the English version, but I don't understand a statement from it, more ...
2
votes
2answers
79 views

Do I need to take out any predictors from multiple regression if I put in some principal components as additional predictors?

I have an assignment which involves one area-level dataset made of $366$ scale variables. I have to perform PCA, compare it with rates of an additional response variable $X$, and comment on its face ...
2
votes
1answer
57 views

Eigenfaces in R: How to reconstruct original features from principal components? [duplicate]

I've performed PCA on face images dataset and I'm not sure how can I use the most informative principal components to show the "reduced" image. The original image is 96*96 pixels (96*96 = 9216) and I ...
2
votes
1answer
120 views

Where is the indeterminacy of factor values on this plot explaining factor analysis?

It is a well-known fact that in principal component analysis (PCA) we can obtain true values of components but in factor analysis (FA) we cannot obtain true values of common factors. We can compute ...
3
votes
1answer
317 views

What is component (factor) score coefficient matrix in PCA or factor analysis and how is it calculated?

As per my understanding, in PCA based on correlations we get factor (= principal component in this instance) loadings which are nothing but the correlations between variables and factors. Now when I ...
8
votes
1answer
108 views

What criteria to use for separating variables into explanatory variables and responses for ordination methods in ecology?

I have different variables that interact within a population. Basically I have been doing an inventory of millipedes and measuring some other values of the terrain, like: The species and the amount ...
2
votes
0answers
50 views

Logistic principal component regression where PCs are correlated with an additional binary predictor

My scenario is this: I collected a bunch of vegetation data (% cover counts in a quadrant at different heights) in patches where birds were seen foraging and also in control patches where no foraging ...
4
votes
1answer
104 views

Under which conditions do PCA and FA yield similar results?

Under which conditions can principal components analysis (PCA) and factor analysis (FA) be expected to yield similar results?