Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the ...

0
votes
0answers
12 views

Can you use factor analysis on a table has count data? [on hold]

I am working on a project where my predecesor has been analyzing a table of rows by columns of count data. Brands represent the columns, and statements about those brands represent the rows. The cells ...
0
votes
0answers
6 views

PCA on continuous and binary data - further explained [duplicate]

I would like to perform a PCA on continuous data and binary data. Actually, I want to submit 8 continuous variables and 1 binary variable to a PCA. Is it a problem to include one binary variable ? ...
-1
votes
0answers
23 views

Why are my first two eigenvectors inverted with respect to the first two PCs? [duplicate]

Here is the working code: # the eigen vectors eigen(cov(mtcars))$vectors # the principal components prcomp(mtcars, scale. = T) The first five eigenvectors that ...
1
vote
0answers
28 views

Dimensionality reduction using arbitrarily matrix factorizations

As far as I can see from basic dimensionality techniques like PCA, EFA, ICA etc. the problem is reduced to ($X$ is dataset with observations as rows and $D$ columns; $A$ has $d<D$ columns) : $$X \...
1
vote
0answers
18 views

Machine Learning point of view regarding the purpose of EFA [on hold]

From the point of view of Machine Learning, what is the purpose of Factor Analysis (FA)? I used to think that it is a dimensionality reduction, because it is so connected to PCA, but when I read in ...
0
votes
0answers
17 views

Should I use the covariance matrix for PCA on ecological data when I have species percent or number of species per gram

I have done a lot of reading about PCA using the covariance and correlation matrix. I understand that if the variables are measured on different scales i.e. number of leaves, height in m, width in mm, ...
1
vote
0answers
37 views

Rotational invariance of PPCA

From here (slide 23) and here (page 5, 4th slide) I understand that it is said that PPCA (probabilistic PCA) is rotational invariant. It can be written as follows: $$\text{PPCA}(X) = [\mu, W, \sigma^...
0
votes
0answers
4 views

Signs in SPSS's PCA with rotations with the FACTOR algorithm

I am trying to reproduce the results of the PCA with rotations from SPSS in python. But there is some information I didn't find in their documentation. I am trying to do the PCA like in the FACTOR ...
0
votes
0answers
11 views

How can I write the resulted new representation of data, using LDA, from WEKA to an arff file? [on hold]

Linear Discriminant Analysis (LDA) is a supervised dimensionality reduction algorithm, so from d dimensional data (input data) we want to obtain p new dimensions, where d>>p. It is the same principle ...
0
votes
0answers
7 views

Compare principal components/differences between two groups

I have four groups of individuals which have a certain illness and are part of a clinical trial: Taking drug, illness gets better Taking drug, illness is not getting better Taking placebo, illness ...
0
votes
0answers
11 views

Pearson correlation after principal component analysis and varimax rotation

Is it possible (or does it make sense) to check for correlation after varimax rotation, since varimax assumes that there aren't any correlation between factors (or components)?
1
vote
1answer
39 views

Factor-loadings vs Variable-loadings

In PCA and Factor Analysis, there is the term loadings, which refers to factor loadings (onto the original variable). Does the term (original) variable loading (onto the latent factor) exist?
4
votes
1answer
53 views

Fundamental difference between PCA and FA?

According to this, the fundamental difference between PCA and FA can be illustrated via the following image: So, the direction of arrows changes. According to this answer and a few others: ...
1
vote
0answers
13 views

How can I calculate the standardized root mean square residual (SRMR) from the psych package in R?

The psych package in R provides the root mean square of the residuals (RMSR) when using the principal (principal components analysis) or fa (factor analysis) functions. How could I calculate the ...
0
votes
0answers
13 views

feature selection and classification - train and test on the sample?

I have a dataset of 93 records and 45 radiomics variables from various CT scans. I wanted to check if age and sex could be classified by the variables so I made a new variable with both sex and age. I ...
0
votes
0answers
12 views

How to interpret linear regression coefficients on variables that have been transformed by PCA

Let's say I use PCA to reduce the dimensionality of my dataset before building a linear regression model. See R example: ...
0
votes
0answers
28 views

Principal component regression using python [closed]

I have strain temperature data and I have read the article https://www.idtools.com.au/principal-component-regression-python-2/ I'm trying to build a model and predict strain from temperature. I have ...
1
vote
0answers
20 views
+50

Kernel function for use in Kernel-PCA given a known piecewise linear true data generating process

If I know that a multivariate dataset has a piecewise-linear data generating process with known knots (or breakpoints), then what is the appropriate kernel function to use in Kernel-PCA? For example, ...
3
votes
2answers
63 views

Eckart-Young-Mirsky theorem: rank $≤k$ or rank $=k$

The Eckart-Young-Mirsky theorem is sometimes stated with rank $\le k$ and sometimes with rank $= k$. Why? More specifically, given a matrix $X \in \mathbb{R}^{n \times d}$, and a natural number $k \...
0
votes
0answers
9 views

Principal Component Analysis - Centering/Scaling Question

I was wondering if there is precedent for centering to the median and scaling to the median absolute deviation (MAD) (as opposed to arithmetic mean and standard deviation) prior to conducting PCA. I ...
1
vote
1answer
15 views

Apply statsmodels PCA to new data

I am working on a personal project, and I want to use Statsmodels' PCA on a dataset. The ultimate goal is to then perform a linear regression and evaluate its prediction. I know scikit-learn may be ...
0
votes
1answer
38 views

Can PCA be used for detecting multicollinearity?

The definition of multicollinearity is: Given a set of $N \times 1$ predictors $X = (x_1, x_2, \cdots, x_m)$, if $$x_j = \sum_{i \neq j}a_ix_i$$ then we say there is multicollinearity among the ...
0
votes
1answer
14 views

Which analysis to use to discriminate morphometrics measurements from different species from 2 different environment?

So I have a dataset of measurements (lengths, surface areas, volumes...) from 3 species from 2 different environments, with 3 individuals per species. Can be summarised like that: ...
0
votes
1answer
19 views

Is a chi-square test for independence appropriate on a contingency table where one category is the unsupervised learning cluster?

I have a data set that has been partitioned into four clusters by executing a clustering algorithm that used principal components from a principal component analysis (PCA). I then make a contingency ...
1
vote
0answers
11 views

Question about pca proof in deeplearning by ian goodfellow [duplicate]

Can anyone help me with this part? I don’t understand why d vector should be eigen vector of covariance matirx of X nor the generalization part
0
votes
0answers
15 views

When applying PCA to a dataset consisting of regression coefficients, should one use PCA on correlation or on covariance?

This is a follow-up question from the post: PCA on correlation or covariance? The accepted answer quotes: You tend to use the covariance matrix when the variable scales are similar and the ...
0
votes
0answers
11 views

Appropriate Statistical Analysis for Variable Reduction

I'm planning to conduct a study for clustering a set of observations. For the beginning, I'm planning to include more than 50 variables to my study. Therefore I must apply a relevant statistical ...
-1
votes
0answers
16 views

performing PCA on data sets of high variance [duplicate]

I am reading a documentation from Matlab (https://www.mathworks.com/help/stats/quality-of-life-in-u-s-cities.html#d119e60095) on performing PCA which makes this claim: When all variables are in ...
0
votes
0answers
75 views

Using a single principle component (PC) space to describe how a dataset changes across conditions

Given a design matrix that consists of N (>100) variables and J (>100) observations (the data, itself, is actual time-series): ...
0
votes
0answers
17 views

Robust Anomaly Detection Algorithm from Netflix?

I have read a lot about the robust anomaly detection of Netflix which they open sourced as part of their Surus Project (https://github.com/Netflix/Surus). The project anomaly detector is based on the ...
1
vote
0answers
33 views

how to optimize reduced rank regression with constant diagnoal constraint?

I am trying to optimize a panel regression $G=\beta G+e$. $G$ is $N \times T$ matrix. $\beta$ is $N\times N$ unknown coefficient matrix. Each column of G represents the growth of all stocks at period $...
1
vote
0answers
26 views

How to interpret low loadings all over PC 1?

My PCA with prcomp in R results in very low "loadings" (i.e. eigenvectors, see figure below). I've tried a rotation with ...
2
votes
0answers
97 views

Is MCA equivalent to PCA when all variables are binary?

I am looking to apply principal component analysis on binary (true/false) data, and I have come across the "equivalence between PCA and MCA" (Multiple Correspondense Analysis) for binary data, but ...
0
votes
0answers
14 views

PCA and erratic scoring

I have been thinking about PCA when finding latent variables from a questionnaire and I was wondering if PCA can deal with the following example: I give two 1-5 Likert scale questions to a sample, ...
0
votes
1answer
73 views

In case of semi-supervised data, and PCA pre-processing, should I pre-process all the data or only the labeled data?

suppose I have labeled data and unlabeled data, should I do a PCA process on all the data, and then feed the labeled data through a classifier? Or take only the labeled data through the PCA process ...
0
votes
0answers
23 views

LDA vs PCA 2d plot

PCA (Principal Component Analysis) is often used to represent 2d or 3d plot of the data, where y=PC2 and x=PC1 (eventually z=PC3). Given that there is an 'order' between components, it makes sense to ...
0
votes
0answers
15 views

PCA on layers of deep neural network in time series prediction (or linear regression in general) for explainability

Are there any references on applying PCA or some other dimensionality reduction to the intermediate layers of deep neural networks in the context of time series or regression? I was thinking that ...
0
votes
0answers
57 views

Is it possible to calculate the volume of a point cloud using principal eigenvalues from PCA?

Assume I have a point cloud in $\mathbf{R}^N$ and we take the $N$ principal component eigenvalues: $\lambda_1 , ... , \lambda_N$. Does $\prod_i \lambda_i$ represent the effective volume surrounding ...
0
votes
0answers
7 views

Performing Scoring Indicator : With principal component

iam trying to generate an engagement indicator from 5 variables which i feel that explained engaged behaviours in my application, for each specific user i will need to score it in an indicator so as ...
1
vote
0answers
30 views

PCA generating less principal components than the number of original variables [duplicate]

I've been applying PCA to some data sets and in all previous cases, the number of principal components generated by prcomp() was always equal to the number of original variables (m). However, I've ...
1
vote
2answers
60 views

How to plot High Dimensional supervised K-means on a 2D plot chart

I'm Having a ML problem where my data set contains 80 features labelled into 3 groups (0, 1, -1). I want to plot the data on a 2D surface to see how "close" (similar) data with ...
0
votes
0answers
34 views

Variable selection using PCA in R [duplicate]

I need to preselect my environmental variables by a PCA before running RDA or CCA because the models are overparameterized. Please have a look at this publication if you ask whether or comment whether ...
0
votes
0answers
10 views

PCA combined with 2nd stage multinomial logit regression

I am running a PCA with the following kind of data. Var1 - Var 25 take only discrete values between -4 and 4 in an imposed normal distribution (i.e. 4 and -4 can be the chosen value once, -3 and 3 can ...
1
vote
1answer
15 views

PCA returns the same pair of principal axes for completely different 2D datasets [duplicate]

I noticed a (seemingly) weird behavior while using sklearn's PCA on 2D standardized datasets: I kept getting the same principal axes: $\pm\left(\begin{gathered}\sqrt{0.5}\\ \sqrt{0.5} \end{gathered} \...
3
votes
0answers
28 views

What exactly should be called “projection matrix” in the context of PCA?

At the end of the PCA algorithm one gets a $D\times d$ matrix $U$ such that $z=U^Tx$ (here $x$ is $D$-dimensional and $z$ is $d$ dimensional with $d\leq D$). In multiple sources on the Web I found ...
1
vote
0answers
11 views

Intuition: What is the difference between linear factor models and regular linear regression?

So, I have a very vexing theoretical question that I hope some experienced econometricians can help me with. Being in finance, I have recently been exposed to linear factor models, which are models ...
0
votes
1answer
60 views

PCA in psych package with more columns than rows

Why is it impossible to do a PCA in R using principal from psych package without warnings with a matrix, which has more columns ...
1
vote
0answers
22 views

PCA for three-dimensional linear fit on time-resolved trajectory

I study the behavior of organisms that are able of self-locomotion and that show directed motion toward one another. This directed motion occurs through the detection of chemical trails released by ...
0
votes
0answers
18 views

How do I interpret Principal Components in k-means analysis?

This might not be an appropriate question to make here, but Cross Validated has helped me before so I'll at least try to get help here again. Anyway, here are the relevant parts of the code ...
0
votes
1answer
24 views

Dimensionality Reduction on VGG Image Vector

I have a random forest model which I am using to make retail demand predictions. I am looking at trying to leverage product image data to improve the predictions and have put the images through VGG-...