Questions tagged [pca]
Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.
1,226
questions with no upvoted or accepted answers
27
votes
4
answers
893
views
Blind source separation of convex mixture?
Suppose I have $n$ independent sources, $X_1, X_2, ..., X_n$ and I observe $m$ convex mixtures:
\begin{align}
Y_1 &= a_{11}X_1 + a_{12}X_2 + \cdots + a_{1n}X_n\\
...&\\
Y_m &= a_{m1}X_1 + ...
- 835
17
votes
1
answer
523
views
Can I use optimally scaled variables for a factor analysis to account for rotation? If I can then how?
I have discussed this issue several times in this site, but I am asking it again for a final justification from the experts of our community. I wanted to extract four factors (I should call dimensions ...
- 3,595
9
votes
1
answer
6k
views
Polychoric PCA and component loadings in Stata
I’m using Stata 12.0, and I’ve downloaded the polychoricpca command written by Stas Kolenikov, which I wanted to use with data that includes a mix of categorical ...
8
votes
0
answers
426
views
What is the precise relation between the eigenvalues of a covariance *function* and the eigenvalues of a covariance *matrix*?
Assume we have a temporal Gaussian Process $\mathcal{GP}(t;\ m,k)$ (GP) with mean $m$ and covariance function (aka. kernel) $k$ on some compact time interval $[0,T]$. Then, the eigenvalues $\lambda$ ...
- 81
8
votes
0
answers
3k
views
Principal components: Can I interpret PCA as essentially a change of basis
I was hoping that someone could simply validate or correct my interpretation of Principal Components Analysis. There are a lot of questions on this site about Principal Components analysis--some ...
- 1,512
7
votes
0
answers
3k
views
Interpretation of biplot in PCA
Blue points all appear in the lower right-hand quadrant in the plane formed by the first two principal components.
Is it a good interpretation of the biplot (right panel) to say that blue points are ...
- 2,173
7
votes
0
answers
410
views
What, if any, dissimilarity is preserved in partial least squares (PLS)?
When we perform a principal components analysis (PCA) on a multivariate data set we are interested in finding orthogonal components that explain maximal variance in the data set. We can form a biplot ...
- 45.8k
6
votes
0
answers
77
views
Asymptotic properties of functional models
When working in Functional Data Analysis, a classical "preprocessing" step is to represent the "observations" using a B-spline expansion:
$$
X_i(t) \approx \sum_{j=1}^J \lambda_{ij}...
- 540
6
votes
0
answers
223
views
How would one carry out principal component analysis in the Beltrami-Klein model of hyperbolic space?
This is a follow up to a comment by @whuber in response to this question: Is there an extension of PCA for data embedded in hyperbolic spaces?
Sorry, I would have posted it there as a comment but it ...
- 161
6
votes
0
answers
735
views
stationarity and fractional differencing
This is a methodology question. I would like to make the data stationary but not transform it "too much" (information loss), before it is fit for statistical/ML purposes such as regression or PCA.
...
- 61
6
votes
0
answers
265
views
Mathematically Describing PCA chained with Logistic Regression
Python's scikit-learn package has a convenient pipe function that can combine machine learning techniques into one model with ...
- 573
6
votes
0
answers
12k
views
What is the Rotation Matrix in PCA?
I'm trying to implement th Local Coordinate System (LCS) of this paper.
It's all clear to me about how it works, but the only thing that I' dont understand is the "rotation" mechanism. Quoting the ...
- 399
6
votes
0
answers
957
views
can I use PCA and PAF on Kendall's and Spearman's correlation matrix?
I have a dataset of 77 items, ranked by 17 people, with many ties (actually: Q-sorted under a forced quasi-normal distribution ...
- 742
5
votes
2
answers
2k
views
Reconstruction Error: Principal component analysis vs Probabilistic prinicpal component analysis
I am working through the book "Machine Learning: A Probabilistic Perspective". After introducing PCA and Probabilistic PCA, the following graphic is shown (the upper two graphics ...
user290388
5
votes
0
answers
4k
views
Is MCA equivalent to PCA when all variables are binary?
I am looking to apply principal component analysis on binary (true/false) data, and I have come across the "equivalence between PCA and MCA" (Multiple Correspondense Analysis) for binary data, but ...
- 51
5
votes
0
answers
182
views
PCA with one "known" component
(The title might not be very clear, any recommendation welcome)
I have a $1 + n$ dimensional dataset. The first dimension measures a specific concept I am interested in and the $n$ remaining ...
- 3,661
5
votes
1
answer
194
views
What is the relationship between Mohr's Circle and Principal Component Analysis?
While I was studying PCA, I was told that it is related to Mohr's Circle. I don't know what that means. I don't know if they are related or not. I was just told, so I want to make sure here.
If they ...
- 1,673
5
votes
0
answers
254
views
What is the fastest way to compute PC1 scores, without performing the whole PCA?
I want to compute only the first principal component's scores $t_1$ of a large number $n$ of data points x with a high dimensionality $p$. Assume the data has been centered about zero. Data points ...
- 231
5
votes
0
answers
2k
views
How to create a scree plot for factor analysis given that eigenvalues depend on the number of extracted factors?
I understand how Kaiser rule works for PCA, as no matter how many components I extract I always get the same eigenvalues.
For example, with 3 components I get
...
5
votes
0
answers
443
views
How do random data eigenvalues change, as random variables are added?
I am using parallel analysis (Horn 1965) to determine how many principal components I can extract from my data.
I can add more variables to my dataset, but I cannot add more cases (I know, that's ...
- 742
5
votes
0
answers
8k
views
Principal Component Analysis on Time series data and panel data
I am trying to build an index on infrastructure and compare the index over the years and between nations. Since the variables are highly correlated with each other, review has suggested me to proceed ...
- 501
5
votes
0
answers
794
views
Factor analysis across different levels of data aggregation
I have survey data for thousands of individuals from hundreds of towns. I want to identify factors underlying certain characteristics at the town level and the individual level. The individual level ...
- 1,272
5
votes
0
answers
2k
views
Factor analysis for categorical target variable
I'm doing some research into factor analysis and I've hit that barrier where I don't know what search terms to use.
I'm trying to see if something is possible. Basically I have a data set with ~100 ...
- 533
4
votes
0
answers
198
views
PCA: inference on the proportion of explained variance, in a large p setting
I am interested in doing inference on the proportion of total variance explained by the first principal component, for a PCA based on the correlation matrix R. I want to know the (asymptotic) ...
- 3,063
4
votes
0
answers
358
views
What is the appropriate metric for determining distance / dissimilarity of sparse, high dimensional data in PCA space?
I'm working with scRNA-seq data (~96% sparse, high dimensional), and am trying to determine distances between the cells in PCA space - NOT for the specific purpose of clustering. The principal ...
- 51
4
votes
2
answers
466
views
Is there an eigenfaces equivalent for PCA analysis of time series, eigen-time series?
I am trying to better understand PCA as applied to time series by drawing parallels with this explanation of PCA as applied to images of faces. In particular, I would like to visualize the resulting "...
- 763
4
votes
0
answers
73
views
Where is the “energy” analogy coming from in PCA?
In some publications or materials on POD/PCA, I've encountered sentences like: the first PCs carry most energy, or the first PCs identify the most energetic directions. This analogy to energy seems ...
- 802
4
votes
0
answers
159
views
Is high dimensional PCA regression consistent?
Consider a set $(y_i, x_i), i=1\ldots,n$. The OLS estimator, which is a $\sqrt{n}$-consistent estimator of $\beta$ is obtained as $$\hat\beta=(X^tX)^{-1}X^ty$$.
Now perform a PCA on the matrix $X\in\...
- 1,284
4
votes
0
answers
40
views
Productionize (applied) PCA to detect outlier etc. long term with new data?
I was wondering how one could use PCA in e.g. a dashboard for non Subject Matter Expert.
For example, you are quite certain that 2 PCs are sufficient based on the current data. It also makes sense for ...
4
votes
0
answers
1k
views
How is explained variance in sparse PCA calculated?
Sparse PCA is a technique proposed by Zou et all in this paper. In usual PCA the obtained loadings are orthonormal, and the resulting scores are uncorrelated. However, in sparse PCA you give up these ...
- 1,284
4
votes
0
answers
374
views
PCA, SMOTE and cross validation - how to combine them together?
I was reading a lot recently about PCA and cross validation and it seems that the majority call it malpractice to do PCA before cross validation. I would also like to perform SMOTE, but there is a ...
- 211
4
votes
0
answers
35
views
Mutual dependence/associations
I have three related questions about measuring the association within sets of data:
A. Imagine I were to try and test the hypothesis that the IQ of individuals in same sex couples are correlated (e.g....
- 41
4
votes
0
answers
1k
views
PCA - Background removal
I saw this example in a python notebook on Fast.ai. In the notebook they are removing the background and keeping objects in the foreground in a video sequence by using different methods(SVD, random ...
- 141
4
votes
1
answer
2k
views
Feature Selection Using Principal Feature Analysis and Variables Factor Map
I am trying to select the most important features that explain the variability of my data using an unsupervised approach in python (would consider R though).
This is after I performed a PCA and ...
- 163
4
votes
0
answers
161
views
Analogue of spectral gap but for *smallest* eigenvalues/singular values
The difference between the largest eigenvalue and the next-largest of a graph Laplacian (equivalently, of the random walk Markov chain on the graph) is the spectral gap, related to the Cheeger ...
- 141
4
votes
0
answers
467
views
Quiz: Determine first principal component from data-plots
We see four data plots. The goal: How does the first principal component look for each plot a-d. For plot d, it is true that both clusters have same number of datapoints.
First principal component ...
- 331
4
votes
0
answers
3k
views
Principal Component Analysis: how to interpret the total contribution of variables on several dimensions
When we calculate the total contribution of a variable for a single dimension, the sum of all single contributions is equal to 100%, which makes perfect sense.
The http://www.sthda.com suggests to ...
- 251
4
votes
0
answers
1k
views
Most important original features in PCA: can one multiply eigenvectors by the explained variance?
I would like to know the importance of the original features in principle component analysis. See this Stackoverflow link for an example of what I mean (with a code example).
The question is: can you ...
- 141
4
votes
0
answers
2k
views
Statistically correct to apply Multi-dimensional scaling or PCA to cosine similarity matrix?
Supposing I have a document-term matrix as scripted below:
...
- 237
4
votes
0
answers
489
views
Principal component regression (PCR) with some of the original predictors left out of PCA
I just recently started learning about principal component regression (PCR) and I'm wondering if it's possible to use both principal components and original variables as predictors of a given outcome (...
- 41
4
votes
0
answers
138
views
Categorical PCA: Merge categories based on Transformation Plots?
A tutorial on categorical pca (CATPCA) (Linting et al. 2012) explains that a decision to merge categories of an ordinal variable can be made based on the category quantification ("none of the ...
- 443
4
votes
0
answers
373
views
Efficient way to solve generalized eigenvalue problem when the number of dimensions is greater than the number of samples
I am trying to solve the generalized eigenvalue problem:
$$C_e v = \lambda C_o v.$$
$C_e$ and $C_o$ are both covariance matrices generated from data with $10512$ dimensions and about $2000$ samples. ...
- 253
4
votes
0
answers
2k
views
How does NIPALS algorithm work?
I'm working on NIPALS algorithm and I found this procedure from here:
I'm just confused with the 4th step which stated, "using the $k$th scores, re-estimate the eigenvalues". As I understand, ...
- 41
4
votes
0
answers
166
views
Distributions of eigenvalues of random matrices: what can they be used for in data mining?
I've accidentally come across some papers discussing distributions of principal components of the sample covariance matrices. An example of such a paper is Johnstone, 2001, On the distribution of the ...
- 591
4
votes
0
answers
293
views
Kernel PCA vs principal curve analysis
Both principal curve analysis and kernel PCA provide the ability to find nonlinear PCA. Kernel PCA does this by finding principal components in a higher dimensional space. Principal curve analysis is ...
- 562
4
votes
0
answers
8k
views
What is the rationale behind the "eigenvalue > 1" criterion in factor analysis or PCA?
What is the meaning of "eigenvalue > 1" criterion? I understand what eigenvalues and eigenvectors are.
This question is w.r.t. this link and this statement there:
By default, VARCLUS stops ...
- 116
4
votes
0
answers
2k
views
How to compute component or factor scores when the analysis is based on polychoric/tetrachoric correlations?
[This question is modified based on suggestion from @ttnphns]
I am doing linear principal component analysis (PCA) based on polychoric correlations between the variables (rather than on native Pearson ...
- 534
4
votes
0
answers
593
views
Reducing size of dataset to a fixed size - retaining maximum information in all dimensions
I was wondering about about the following problem:
I have a set of $N=10^5$ observations with dimensionality $D=2$, and I would like to reduce it to a set of size with $M=10^3$, or some other $(M \ll ...
- 1,661
4
votes
0
answers
196
views
Should I run measurement invariance if confirmatory factor analysis is not affirmative?
I have a repeated measures design for a Likert scale with 10 items. This scale is assumed to be unidimensional and this is one of the research questions along with the scales measurement invariance (...
- 555
4
votes
0
answers
4k
views
Difference beween supplementary and active variables in PCA - Interpretation on obsevations?
I would like to introduce two supplementary variables into a PCA I'm conducting on a set of data measuring concentration in different material phases.
However I'm unclear as to how to interpret the ...
- 103