Skip to main content

Questions tagged [pca]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Posterior mean in Probabilistic PCA shifts towards 0 when $\sigma^2 > 0$

We know that in probabilistic PCA, where $$x = Wz + \mu + \epsilon, \epsilon \sim N(0, I)$$ The posterior can be derived to be $$z|x \sim N\left(M^{-1}W^T(x-\mu), \sigma^{-1}M\right)$$ where $M = W^TW ...
statslearner's user avatar
0 votes
0 answers
21 views

Factor Analysis with Multiple Imputation

I have a dataset with 49 Items of a questionnaire (ordinal; 0,1,2,3,4) with 5 diagnostic group of samples (n1: 50, n2: 25; n3:30, n4:23, n5:60). However, the dataset have missings like for 10-12 ...
dododabird's user avatar
0 votes
0 answers
18 views

Is this a reasonable way to check the quality of simulated data in MCMC inference?

I have a hierarchical Bayesian model that looks like this: $\alpha_i \sim \mathcal{N}\left(\mu_\alpha, \sigma_\alpha\right) \tag{1}$ $\beta_i \sim \mathcal{N}\left(\mu_\beta, \sigma_\beta\right) \tag{...
chesslad's user avatar
  • 211
1 vote
0 answers
26 views

Variable weighted PCA

I have seen a lot of "weighted PCA" but they are really all on "observations". For example Weighted principal components analysis if you have K variables, N observations, the ...
Brandon Bolden's user avatar
0 votes
0 answers
21 views

How to make an index with PCA?

I'm new to PCA and have a question about interpreting the results. In R, I can obtain the component loadings and scores. To create an index, should I just add these scores together? Are the scores ...
chunguc1004's user avatar
0 votes
0 answers
22 views

Comparing the angle between PCA loading plots

I am analyzing temperature, precipitation, and snow depth in three datasets. I used PCA for the analysis, and I obtained angles of 13.1°, 19.6°, and 102.0° between precipitation and snow depth for ...
Arathi's user avatar
  • 1
1 vote
1 answer
24 views

Performing a PCA on data of different hierarchical levels

I (novice) plan to use a PCA on several different, related, i.e. non-orthogonal questionnaire measures. These measures have composite scores (item sums etc.) and some of them have sub-facets. Also, ...
Livster's user avatar
  • 11
0 votes
0 answers
31 views

Confusion regarding PCA, FA, and PCR?

I learned here: Is PCA followed by a rotation (such as varimax) still PCA? About the relationship between PCA and FA and how they each provide a perspective for looking at the same thing. However, at ...
Ozz's user avatar
  • 1
0 votes
0 answers
19 views

Creating an Index using PCA?

Apologies if this question seems basic—I'm new to PCA. In R, I've figured out how to obtain the component loadings and scores. To create an index, should I simply add these scores together? In other ...
chunguc1004's user avatar
1 vote
1 answer
28 views

What is the interpretation of outlier-robust principal component analysis?

There's a set of methods called "robust" principal component analysis (here, "robust" means resistant to influence from outliers). One example is Hubert et al., "ROBPCA: A new ...
cgmil's user avatar
  • 1,373
4 votes
2 answers
297 views

When is multidimensional scaling exact for a graph?

For an undirected graph with one connected component and distance matrix given by the shortest path between nodes, I would like to embed the nodes in a high dimensional Euclidean space where all ...
user3433489's user avatar
0 votes
0 answers
17 views

Reproducing PCA results of pca.fit_transform() using pca.fit()? [duplicate]

I have a data frame called data_principal_components with dimensions (306x21154), so 306 observations and 21154 features. Using PCA, I want to project the data into 10 dimensions. As far as I ...
george1994's user avatar
0 votes
0 answers
21 views

What dimensions to expect of Principal Components Analysis? [duplicate]

In both Python and R, the matrix of eigenvectors of Principal Component Analysis (PCA) is a matrix of principal components with dimensions (Number of Observations x Number of Principal Components). ...
george1994's user avatar
1 vote
0 answers
18 views

What is the difference between direct effect and correlation in factor analysis?

I am reading the short course material of factor analysis from: https://www.hsph.harvard.edu/wp-content/uploads/sites/59/2016/10/harvard-lecture-series-session-4_Factor-analysis.pdf#page=38.00 Here, ...
doraemon's user avatar
  • 250
5 votes
1 answer
87 views

Do features with high variance contribute more to top principal components that explain much of the variance in dataset and vice versa?

The first principal components capture the most variance in the dataset. Does that mean, when we look at the weights (or the loading scores), the top principal components will have high absolute ...
Aditya D Modak's user avatar
0 votes
0 answers
11 views

I want to plot the decision boundaries of an SVM model with more than 2 variables

I understand that that is impossible to visualize, so I went in and PCA-transformed the variables. The problem is that I still need more than 2 principal components to get "good" ...
maglorismyspiritanimal's user avatar
1 vote
0 answers
36 views

PCA applied to non-linear data

Assume I apply standard principal component analysis to data, where the observed variables are non-linear functions of factors. That is I have a panel variable $Y_{i} \in \mathbb{R}^{N_{Y}} $, which ...
fes's user avatar
  • 340
0 votes
0 answers
34 views

Applying PCA Before Training Multiple SVM Binary Classifiers To Reduce Data

I am working on a project which has a goal to determine if a new sample is part of Class A or Class A'. I need multiple of those classifiers. I will have an SVM to classify between: ClassA - ClassA' ...
guitardenver's user avatar
1 vote
0 answers
25 views

Difference between weight matrix and loading matrix in PCA

Currently I am working with PCA techniques (specifically sparse PCA techniques) but my question revolves around obtaining the weight matrix in PCA. This reference provides the following representation ...
Aryan's user avatar
  • 11
0 votes
0 answers
18 views

Curse of dimensionality in Time series with K-means

I have been looking at the following notebook: time series clustering where the writer says that the dataset is affected by the "Curse of Dimensionality", so applying TimeSeriesKMeans ...
Zackbord's user avatar
0 votes
0 answers
15 views

Meaning of within-class Covariance in Linear Discriminant Analysis Dimensionality Reduction

In section 4.3.3 of Elements of Statistical Learning by Hastie, Tibshirani, and Friedman the authors listed a procedure to reduce the dimensions of an input matrix $\mathbf{X}$, first using Linear ...
Jack Guan's user avatar
  • 103
0 votes
0 answers
12 views

Estimating impact of correlated dependent variables on the independent variable using Principal Component Regression

I have a large no. observations of 50 columns of correlated data and I want to estimate their individual impact on the dependent/target variable. The approach I believe might work is first estimate ...
Shouray Duggal 's user avatar
1 vote
0 answers
19 views

why we need use covarience matrix to calculate eigenvectors and eigenvalues in pca [duplicate]

I am stuck at understanding the idea behind using covariance matrix for calculating eigenvectors and eigenvalues and select the eigenvector which has highest eigenvalue
daadkfjd's user avatar
0 votes
0 answers
22 views

KMO measure in principal component analysis [duplicate]

I just started to be interested in PCA. I have a data set of 21 variables with 27 samples, and I am trying to make an application in Jamovi with this data set. I performed KMO and Bartlett's test ...
ali's user avatar
  • 33
0 votes
0 answers
31 views

PCA factor model - insignificant factor loadings

I have a time-series of N assets for which I am trying to estimate a factor model. Let $Z_{t}$ be one of these assets' prices at time $t$. We can write it as: $$ Z_{t} = \beta F_{t} + \theta_{t} $$ ...
deblue's user avatar
  • 243
2 votes
1 answer
26 views

Statistical test for comparing PCA variation between two groups

I have conducted a Procrustes transformed PCA. I have data from multiple ancient human populations transformed onto the same background of modern reference data. I have a few populations that I am ...
dnv89's user avatar
  • 51
1 vote
0 answers
32 views

Would test set MSE and r^2 be the same with OLS and PC Regression with all PCs? [duplicate]

For this question, I define: PC Regression = Standardize Variables, Fit PCA, then apply OLS to all PCs. OLS = Standardize Variables, then apply OLS with all variables. This question makes me think the ...
user2330624's user avatar
0 votes
0 answers
18 views

Can samples be grouped for principal component analysis? Are there any tests needed?

I have two groups of people, those who are involved in public affairs and those who are not. At the same time, I have five types of capital for these people. I want to explore the mix of capital ...
Zhi WANG's user avatar
0 votes
0 answers
21 views

Is the behavior of log-likelihood and number of parameters correct in probabilistic PCA?

I am studying the behavior of Probabilistic PCA as described by Tipping and Bishop (1999). I am using the R package "Rdimtools" to help. I am puzzled about the number of parameters in the ...
Daniel Caetano's user avatar
3 votes
1 answer
70 views

The Math Behind the Conditional Probability of a Probabilistic PCA

I am trying to understand how to calculate the conditional distribution of probabilistic principal component analysis. This is explained in the book "Pattern Recognition and Machine Learning"...
CAM_etal's user avatar
0 votes
1 answer
31 views

Linearity preservation after PCA covariate transformation

thank you for this interesting discussion about linearity of PCA. enter link description here I have a question, which I do not know if it's trivial or not, but I do need to clarify it. I want to fit ...
sinectica's user avatar
1 vote
0 answers
10 views

PCA time series Index in R: Score Plot Doesn't Match Expected Index

I'm working on creating a PCA Index in R to understand how it works. I've used 'make up' data for this purpose. However, when I plot the scores of the first component, the results are not as expected. ...
searchandprint's user avatar
0 votes
0 answers
14 views

PCA site scores much smaller than 'species' scores?

This question is similar to another one on DCA (Spread of species and site scores in R (vegan) DCA ordination), though I believe I'm using the correct analysis and data pre-processing. I'm using PCA (...
JChase's user avatar
  • 1
0 votes
1 answer
57 views

Principal Component Analysis using Panel Data

I have a panel data with identifiers(a,b,c,....z) and different times(t=1,2,3,....100) I have 6 different variables (A,B,C,D,E,F) for every identifier-time observation. I attempt to use those 6 ...
Ken's user avatar
  • 1
0 votes
0 answers
8 views

After removing the first principal component I reverse the PCA, but some of the data flip compared to original some not

In order to remove noise from a dataset (n variable, m measurements), I apply PCA on it, remove the first PC, and reconstruc the original data. This perfectly removes the noise from my data. However, ...
Elham's user avatar
  • 1
3 votes
1 answer
131 views

Are there any situations where orthogonality is not optimal?

Data reduction is often used to avoid overfitting and to enhance explainability. Popular data reduction techniques, such as SVD or PCA map/project high-dimensional data to a lower-dimensional ...
Chris M's user avatar
  • 39
1 vote
0 answers
24 views

In PCA, can certain variables be weighted?

I am working with a large dataset of disease symptoms which are all rated on a numeric scale (0-4 points per symptom). Some symptoms concern the left body half, some the right half, and some are axial ...
Nereus's user avatar
  • 111
0 votes
0 answers
16 views

Can the first latent variable of a PLSR model explain less variation in either X or Y compared to the second latent variable?

I currently have a n x m dataset as the X block and a n x 1 dataset as the Y block. I am using the ROPLS package in R, and I've noticed that there are times when R2Y is greater in the second latent ...
Syuma's user avatar
  • 125
0 votes
0 answers
18 views

Demonstrating $SU=U(\sigma^2 I+D^2)$ as a Sufficient Condition in Maximum Likelihood Estimation

I am working on an exercise related to maximum likelihood estimation (in the context of principal component analysis) for the distribution $$p(x) = Gauss(b, WW^T+\sigma^2I)$$ In particular, I want to ...
Andrea's user avatar
  • 153
0 votes
0 answers
31 views

Why apply PCA instead of just removing highly correlated variables? Specially in prediction tasks [duplicate]

First of all let's assume we have variables that are correlated or highly correlated. When we apply PCA we want to reduce dimensionality, PCA works better when we have a linear correlation between the ...
Gabriel_86400's user avatar
2 votes
1 answer
51 views

PCA with correlated variables

I'm analyzing data from around 10 survey questions focused on regulatory issues. I've noticed these questions are highly correlated (of course since they are all about regulation), and I'm concerned ...
chunguc1004's user avatar
0 votes
0 answers
10 views

What is the maximum variance direction of a binary mask of the letter T?

I am performing a PCA analysis on a binary mask with the aim of determining the major axis of the shape. I am trying to understand if the first eigenvector generated in this process makes sense. ...
Loperh's user avatar
  • 11
1 vote
0 answers
41 views

Variability in the dataset explained by a subset of variables [closed]

Let's assume a centred data matrix $X_{n \times p}$ of $p$ variables, where each variable has unit variance. PCA can be thought about as a rotation of the coordinate system that diagonalises the ...
thesecond's user avatar
  • 390
0 votes
0 answers
15 views

Opposite facing loadings of seemingly the same features

I am performing PCA analysis of NIR spectra for the analysis of the progression of a film coating process I have noticed that one of my components goes up with the film coating process (with upwards ...
phil27's user avatar
  • 1
0 votes
0 answers
22 views

MD, tICA and (random, stationary) processes

I am asking myself wether PCA and tICA mandatorily need: 1) 2) Random data as input, i.e. the values sampled per each feature need to have "no memory" of the other ones; Indeed, I was ...
Jacopo's user avatar
  • 1
0 votes
0 answers
12 views

Using PCA to check if parameters simulated from a hierarchical Bayesian model are close to real parameters

I have a hierarchical Bayesian model that learns a 5-parameter function for each of the N participants. The priors on each of the 5 parameters are parameterized by a scale parameter, so, it also ...
vishu's user avatar
  • 111
0 votes
1 answer
37 views

post processing in PCA and making sense of an example

The example is as follows: A bunch of doctors were asked to score a list of desirable characteristics of sales representatives. The questions were like: "in-depth knowledge about his/her product&...
figs_and_nuts's user avatar
0 votes
0 answers
21 views

Can principal components changed by a normalization method be used to construct original data shape with SVD

I'm planning to use an algorithm called Harmony, designed for data normalization, particularly in the context of single cell data analysis. Harmony operates by taking principal components (PCs) as ...
MadmanLee's user avatar
  • 133
0 votes
0 answers
63 views

Principal components analysis missing data SPSS

I have a large data set (27k) There are 63 variables of interest. I attempted PCA as a dimensionality reduction strategy. The items are scored on different metrics ranging from dichotomous variables ...
Danny's user avatar
  • 1
2 votes
0 answers
54 views

Finding parameters which reveal clustering in t-SNE

These data are from SAMHSA, Mental Health Client-Level Data. I am trying to find the right parameters to obtain clustering as in this paper. Code here. For now, I'm dropping columns which aren't ...
Jackson Walters's user avatar

1
2 3 4 5
69