# Questions tagged [pca]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

3,445 questions
Filter by
Sorted by
Tagged with
10 views

• 211
1 vote
26 views

### Variable weighted PCA

I have seen a lot of "weighted PCA" but they are really all on "observations". For example Weighted principal components analysis if you have K variables, N observations, the ...
21 views

### How to make an index with PCA?

I'm new to PCA and have a question about interpreting the results. In R, I can obtain the component loadings and scores. To create an index, should I just add these scores together? Are the scores ...
• 543
22 views

### Comparing the angle between PCA loading plots

I am analyzing temperature, precipitation, and snow depth in three datasets. I used PCA for the analysis, and I obtained angles of 13.1°, 19.6°, and 102.0° between precipitation and snow depth for ...
1 vote
24 views

### Performing a PCA on data of different hierarchical levels

I (novice) plan to use a PCA on several different, related, i.e. non-orthogonal questionnaire measures. These measures have composite scores (item sums etc.) and some of them have sub-facets. Also, ...
• 11
31 views

### Confusion regarding PCA, FA, and PCR?

I learned here: Is PCA followed by a rotation (such as varimax) still PCA? About the relationship between PCA and FA and how they each provide a perspective for looking at the same thing. However, at ...
• 1
19 views

### Creating an Index using PCA?

Apologies if this question seems basic—I'm new to PCA. In R, I've figured out how to obtain the component loadings and scores. To create an index, should I simply add these scores together? In other ...
• 543
1 vote
28 views

### What is the interpretation of outlier-robust principal component analysis?

There's a set of methods called "robust" principal component analysis (here, "robust" means resistant to influence from outliers). One example is Hubert et al., "ROBPCA: A new ...
• 1,373
297 views

### When is multidimensional scaling exact for a graph?

For an undirected graph with one connected component and distance matrix given by the shortest path between nodes, I would like to embed the nodes in a high dimensional Euclidean space where all ...
• 403
17 views

### Reproducing PCA results of pca.fit_transform() using pca.fit()? [duplicate]

I have a data frame called data_principal_components with dimensions (306x21154), so 306 observations and 21154 features. Using PCA, I want to project the data into 10 dimensions. As far as I ...
• 301
21 views

### What dimensions to expect of Principal Components Analysis? [duplicate]

In both Python and R, the matrix of eigenvectors of Principal Component Analysis (PCA) is a matrix of principal components with dimensions (Number of Observations x Number of Principal Components). ...
• 301
1 vote
18 views

### What is the difference between direct effect and correlation in factor analysis?

I am reading the short course material of factor analysis from: https://www.hsph.harvard.edu/wp-content/uploads/sites/59/2016/10/harvard-lecture-series-session-4_Factor-analysis.pdf#page=38.00 Here, ...
• 250
87 views

### Do features with high variance contribute more to top principal components that explain much of the variance in dataset and vice versa?

The first principal components capture the most variance in the dataset. Does that mean, when we look at the weights (or the loading scores), the top principal components will have high absolute ...
11 views

### I want to plot the decision boundaries of an SVM model with more than 2 variables

I understand that that is impossible to visualize, so I went in and PCA-transformed the variables. The problem is that I still need more than 2 principal components to get "good" ...
1 vote
36 views

### PCA applied to non-linear data

Assume I apply standard principal component analysis to data, where the observed variables are non-linear functions of factors. That is I have a panel variable $Y_{i} \in \mathbb{R}^{N_{Y}}$, which ...
• 340
34 views

### Applying PCA Before Training Multiple SVM Binary Classifiers To Reduce Data

I am working on a project which has a goal to determine if a new sample is part of Class A or Class A'. I need multiple of those classifiers. I will have an SVM to classify between: ClassA - ClassA' ...
• 121
1 vote
25 views

### Difference between weight matrix and loading matrix in PCA

Currently I am working with PCA techniques (specifically sparse PCA techniques) but my question revolves around obtaining the weight matrix in PCA. This reference provides the following representation ...
• 11
18 views

### Curse of dimensionality in Time series with K-means

I have been looking at the following notebook: time series clustering where the writer says that the dataset is affected by the "Curse of Dimensionality", so applying TimeSeriesKMeans ...
15 views

### Meaning of within-class Covariance in Linear Discriminant Analysis Dimensionality Reduction

In section 4.3.3 of Elements of Statistical Learning by Hastie, Tibshirani, and Friedman the authors listed a procedure to reduce the dimensions of an input matrix $\mathbf{X}$, first using Linear ...
• 103
12 views

### Estimating impact of correlated dependent variables on the independent variable using Principal Component Regression

I have a large no. observations of 50 columns of correlated data and I want to estimate their individual impact on the dependent/target variable. The approach I believe might work is first estimate ...
1 vote
19 views

### why we need use covarience matrix to calculate eigenvectors and eigenvalues in pca [duplicate]

I am stuck at understanding the idea behind using covariance matrix for calculating eigenvectors and eigenvalues and select the eigenvector which has highest eigenvalue
• 11
22 views

### KMO measure in principal component analysis [duplicate]

I just started to be interested in PCA. I have a data set of 21 variables with 27 samples, and I am trying to make an application in Jamovi with this data set. I performed KMO and Bartlett's test ...
• 33
31 views

### PCA factor model - insignificant factor loadings

I have a time-series of N assets for which I am trying to estimate a factor model. Let $Z_{t}$ be one of these assets' prices at time $t$. We can write it as: $$Z_{t} = \beta F_{t} + \theta_{t}$$ ...
• 243
26 views

### Statistical test for comparing PCA variation between two groups

I have conducted a Procrustes transformed PCA. I have data from multiple ancient human populations transformed onto the same background of modern reference data. I have a few populations that I am ...
• 51
1 vote
32 views

### Would test set MSE and r^2 be the same with OLS and PC Regression with all PCs? [duplicate]

For this question, I define: PC Regression = Standardize Variables, Fit PCA, then apply OLS to all PCs. OLS = Standardize Variables, then apply OLS with all variables. This question makes me think the ...
• 340
18 views

### Can samples be grouped for principal component analysis? Are there any tests needed?

I have two groups of people, those who are involved in public affairs and those who are not. At the same time, I have five types of capital for these people. I want to explore the mix of capital ...
21 views

### Is the behavior of log-likelihood and number of parameters correct in probabilistic PCA?

I am studying the behavior of Probabilistic PCA as described by Tipping and Bishop (1999). I am using the R package "Rdimtools" to help. I am puzzled about the number of parameters in the ...
70 views

### The Math Behind the Conditional Probability of a Probabilistic PCA

I am trying to understand how to calculate the conditional distribution of probabilistic principal component analysis. This is explained in the book "Pattern Recognition and Machine Learning"...
• 75
31 views

### Linearity preservation after PCA covariate transformation

thank you for this interesting discussion about linearity of PCA. enter link description here I have a question, which I do not know if it's trivial or not, but I do need to clarify it. I want to fit ...
1 vote
10 views

### PCA time series Index in R: Score Plot Doesn't Match Expected Index

I'm working on creating a PCA Index in R to understand how it works. I've used 'make up' data for this purpose. However, when I plot the scores of the first component, the results are not as expected. ...
14 views

### PCA site scores much smaller than 'species' scores?

This question is similar to another one on DCA (Spread of species and site scores in R (vegan) DCA ordination), though I believe I'm using the correct analysis and data pre-processing. I'm using PCA (...
57 views

### Principal Component Analysis using Panel Data

I have a panel data with identifiers(a,b,c,....z) and different times(t=1,2,3,....100) I have 6 different variables (A,B,C,D,E,F) for every identifier-time observation. I attempt to use those 6 ...
• 1
8 views

### After removing the first principal component I reverse the PCA, but some of the data flip compared to original some not

In order to remove noise from a dataset (n variable, m measurements), I apply PCA on it, remove the first PC, and reconstruc the original data. This perfectly removes the noise from my data. However, ...
131 views

### Are there any situations where orthogonality is not optimal?

Data reduction is often used to avoid overfitting and to enhance explainability. Popular data reduction techniques, such as SVD or PCA map/project high-dimensional data to a lower-dimensional ...
• 39
1 vote
24 views

### In PCA, can certain variables be weighted?

I am working with a large dataset of disease symptoms which are all rated on a numeric scale (0-4 points per symptom). Some symptoms concern the left body half, some the right half, and some are axial ...
• 111
16 views

### Can the first latent variable of a PLSR model explain less variation in either X or Y compared to the second latent variable?

I currently have a n x m dataset as the X block and a n x 1 dataset as the Y block. I am using the ROPLS package in R, and I've noticed that there are times when R2Y is greater in the second latent ...
• 125
18 views

### Demonstrating $SU=U(\sigma^2 I+D^2)$ as a Sufficient Condition in Maximum Likelihood Estimation

I am working on an exercise related to maximum likelihood estimation (in the context of principal component analysis) for the distribution $$p(x) = Gauss(b, WW^T+\sigma^2I)$$ In particular, I want to ...
• 153
31 views

### Why apply PCA instead of just removing highly correlated variables? Specially in prediction tasks [duplicate]

First of all let's assume we have variables that are correlated or highly correlated. When we apply PCA we want to reduce dimensionality, PCA works better when we have a linear correlation between the ...
51 views

### PCA with correlated variables

I'm analyzing data from around 10 survey questions focused on regulatory issues. I've noticed these questions are highly correlated (of course since they are all about regulation), and I'm concerned ...
• 543
10 views

### What is the maximum variance direction of a binary mask of the letter T?

I am performing a PCA analysis on a binary mask with the aim of determining the major axis of the shape. I am trying to understand if the first eigenvector generated in this process makes sense. ...
• 11
1 vote
41 views

### Variability in the dataset explained by a subset of variables [closed]

Let's assume a centred data matrix $X_{n \times p}$ of $p$ variables, where each variable has unit variance. PCA can be thought about as a rotation of the coordinate system that diagonalises the ...
• 390
15 views

### Opposite facing loadings of seemingly the same features

I am performing PCA analysis of NIR spectra for the analysis of the progression of a film coating process I have noticed that one of my components goes up with the film coating process (with upwards ...
22 views

### MD, tICA and (random, stationary) processes

I am asking myself wether PCA and tICA mandatorily need: 1) 2) Random data as input, i.e. the values sampled per each feature need to have "no memory" of the other ones; Indeed, I was ...
12 views

### Using PCA to check if parameters simulated from a hierarchical Bayesian model are close to real parameters

I have a hierarchical Bayesian model that learns a 5-parameter function for each of the N participants. The priors on each of the 5 parameters are parameterized by a scale parameter, so, it also ...
• 111
37 views

### post processing in PCA and making sense of an example

The example is as follows: A bunch of doctors were asked to score a list of desirable characteristics of sales representatives. The questions were like: "in-depth knowledge about his/her product&...
• 2,593
21 views

### Can principal components changed by a normalization method be used to construct original data shape with SVD

I'm planning to use an algorithm called Harmony, designed for data normalization, particularly in the context of single cell data analysis. Harmony operates by taking principal components (PCs) as ...
• 133