# Questions tagged [pca]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

3,343 questions
Filter by
Sorted by
Tagged with
34 views

### Is this approach of PCA correct?

My project is based in a clinical trial in which we measured in gene expression in three groups (OO, NUTS, LFD). The individuals (n = 151) are almost equally distributed and the variables to measure ...
1 vote
13 views

### Given a psd matrix $Q$ and a kernel function $f(y_i, y_j)$, how do I find $Y \in \mathbb{R}^{n \times d}$ that best approximates $Q$? [duplicate]

The question is basically the title. I have a matrix $Q$ that I know is positive semi-definite. I now want to find the $Y$ that approximates this matrix under some kernel function $f(y_i, y_j)$. I ...
24 views

### Smallest set of words contained in a number of documents

I have a collection of text documents. I would like to find the smallest set of words such that searching by those words allows discovering each document. It is quite natural to describe this data as ...
• 939
7 views

### Difference between LDA and LR? [duplicate]

I want a brief difference about Linear Discriminant Analysis and Linear Regression. Isnt it the same process? I heard the difference is that when it comes to multidimensional target/categories LDA is ...
• 1
13 views

### Do we always want to do projection when we using PCA with SVD? [duplicate]

Assume that we have a matrix $X$ and we want to do Singular Value Decompotision(SVD) with $X$. First we need to find the average of $X$ row wise, called $\mu$ ...
• 193
11 views

### Correlation or Covariance when applying PCA to standardized data?

I am trying to apply PCA to a data set. For context I have standardized my data using the "scale ()" function in R and now I am wondering what matrix I should use in PCA?
• 1
40 views

### Can one variable load onto different components in (varimax-rotated) PCA?

I performed PCA on 33 items with 133 observations. Considering the criteria to take components with eigenvalues >1, 4 factors can be extracted. I then did varimax rotation of those. However, I ...
10 views

### What is the difference between Multiple Correspondence Analysis and Principal Component Analysis result?

Several articles I've read stated that MCA and PCA both work as "reducing dimensionality" tools but MCA is used for categorical variables and PCA is used for numerical variables. But is ...
• 29
169 views

### Does the average eigenvalue equals 1 in PCA applied to standardised data?

From what I understood when we are doing PCA, we can work both with raw or standardised data, depending on the situation we're in. Is it true that the average of the eigenvalues is equal to 1 when we ...
• 103
1 vote
24 views

### Can I apply Kaiser Rule without knowing the eigenvalues?

Kaiser's rule suggests the number of principal components to be included in an analysis by looking at eigenvalues. If I'm given standard deviations only, instead of eigenvalues, can I still somehow ...
• 103
11 views

### Clustering sparse dataset with mix of continuous and categorical variables

I am trying to cluster sparse heterogeneous datasets containing demographics and diagnosis variables ( mix of categorical and numerical variables). How should I start my clustering endeavors ? start ...
22 views

### Creating a standardized composite "score" made up of multiple continuous, dependent variables for analysis in SPSS

I am trying to evaluate how a surgical intervention (insertion of a spinal fusion cage), resulting in distance changes between two vertebral bodies, led to new symptoms (yes/no) in some patients but ...
50 views

### Plotting all the points of PCA to only one PCA axis, first PC1 and then on the PC2

I am required to project all the points on PCA to PC1 axis and then on PC2 axis as to see whether there is a good separation between the points. Through this I have to determine which dimension can ...
1 vote
27 views

### duplicated variables for different components

I'm presently evaluating the position of individuals of an 3 populations of an animal (according to their sexe) in function of the environmental factors (12) present in their habitat. To detect which ...
2k views

### Practical usefulness of PCA

I asked a similar question in the past, but I've thought about the message I am trying to convey a bit more and feel I can articulate it better. For context, I am on an introductory course in machine ...
40 views

### Can I perform multiple Kruskal-Wallis tests with different explanatory variables against the same response variable?

My data is observational data, and that's made it all kinds of ugly, and I can't decide what statistical test is needed. I have one response variable, which is categorical (Species 1, Species 2, or ...
125 views

### How to use slopes in PCA?

I would like to use slope values in PCA. The problem I face is that the slopes I calculate per group could be within different ranges of values. We know that it is important to normalize your data ...
• 301
10 views

• 1,777
1 vote
33 views

### Latent Semantic Indexing vs. PCA

I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition. Let's denote $X$ a $N \times D$ document-term matrix. The $D$ rows of $X$ ...
• 249
51 views

### Optimal predictive factors

Assume I am interested in predicting a time series variable $y_t$ using a vector of possible predictors $X_t$ of dimension $N_x$. I am interested in finding the optimal $N_z < N_x$ predictive ...
• 320
15 views

• 249
1 vote
19 views

### How can I compare one full PCA model to two smaller ones?

I have nearly 30 variables going in to a large PCA, but the variables really fall into two conceptual categories. I want to test whether leaving all the variables to correlate freely with one another ...
1 vote
30 views

### How useful is PCA on its own? [closed]

In my machine learning course we have covered the key ideas behind principal component analysis. To round this part of the course off, we have learned to interpret the results of PCA, specifically ...
1 vote
26 views

### Interpreting results of PCA [duplicate]

I've recently started an introductory machine learning course and the first topic we have covered is prinicpal component analysis (PCA) and overall I am finding the whole topic quite tricky to wrap my ...
• 101
1 vote
44 views

### PCA - Feature Scaling [closed]

I have been reading that the features should be standardized before performing PCA but I couldn't relate to my understanding of the same. PCA try to project the dataset in the direction of maximum ...
18 views

### Can we find dependence among Principal Components?

Principal Components (PCs) in PCA are linearly uncorrelated by definition. However, uncorrelation does not imply independence (let aside the fact that each PC constructed at step t is necessarily part ...
• 391
171 views

### Model causality: graphical models and PCA

If we build a graphical model (DAG) we (may) interpret the arrows as causal dependences. If we build a graphical model based on the variables returned by principal component analysis (PCA) we should ...
• 758
1k views

### Can PC1 explain more than 90% of variance?

I've been running some analysis using morphological data. However, when I run a PCA, PC1 explains more than 90% of variance. I'm not sure if this is possible. If not, what should I need to do?
34 views

### Interpretation and application of PCA - positive factor patterns for first component for all variables

I am attempting to reduce the number of variables in a dataset for regression purposes and I suspect that many of the variables are correlated. Hence, I attempt a PCA, which I must admit I'm very new ...
1 vote
22 views

### Distribute feature importance to the components of the features in a PCA regression?

I read some interesting speculation over on the Data Science Stack. The setup is that there are multiple correlated features in a regression problem, and the goal is to determine feature importance. ...
• 47k
41 views

### (PCA) Is it possible for PCs loading scores to completely change sign after adding data? [duplicate]

I recently ran a PCA on a dataset of self-report data from 226 subjects to zoom in on which specific individual differences might account for participants’ predicted choices in a separate task we have ...
• 21
11 views

### Time invariance when the latent structure is unknown

Imagine that participants completed a series of measures indexing different abilities (memory capacity, learning, etc.) at two timepoints. The only thing I would like to test at this stage is whether ...
29 views

### Maximize Variance of Linear Combination of Matrix Columns

Let $A$ be a $k \times 1$ random vector, and $\mathbf{A}$ be a $n \times k$ matrix of observations. Letting $t \in \mathbb{R}^{k}$ be a vector of weights s.t. $||t||_2 = 1$, suppose we are interested ...
• 313
1 vote
37 views

### How to visualize time series using PCA?

I have two multivariate data sets comprised of 100s of time series, one is the actual recorded data set of time series and the other is a synthetically generated data set based on the recorded one. ...
• 31
28 views

### PCA with non-Euclidean norm?

I have a dataset $X$ which I want to perform PCA on- however, I don't care so much about explaining the variance of certain features. So instead of using the "normal" definition of ...
19 views

### Definition of Generalized of Low-Rank model

Recently, I looked into this paper "Generalized Low-Rank models" and found it very interesting as it gives general perspective on Principal Component Analysis (PCA) related methods. In the ...
• 391
31 views

### How to run a PCA or Factor Analysis when you know one column of the factor loadings

I have this application where I have a direction that I want to keep fixed when I'm running a PCA or factor analysis. Is this possible? I just want to keep a column of the loadings matrix fixed. How ...
28 views

### Why not scale before PCA as a default step? [duplicate]

In ISLR 2nd edition, it says that you may not want to scale before PCA if the features are all in the same units (below). However, I don't see the nuance. Why not just have the "default" ...
• 513
170 views

### What is the proof that non-linearly separable data can't become linearly separable with the results of PCA?

Give a non-linearly separable dataset $X,$ I want to proof that after performing PCA on it, the resulting dataset is guaranteed to be still non-linearly separable. I think we could argue that we still ...
28 views

### Does RDA axis explanatory power affect arrow length interpretation in RDA plot (scaling 2)?

In an RDA plot with scaling 2, arrow lengths reflect how strongly an independent variable is related to the response matrix. But is their effect also influenced by which axis they are pointing along? ...
39 views

### Choosing Which latent variables to display and analyze

Attached is a plot of the R-squared for each additional latent variable (LV) included in a PLSR. When choosing which LVs to display on a biplot (and ultimately determining which factors best explain ...
33 views

### Lag variables on OLS after PCA (Principal Components Analysis)

I have about 60 macroeconomic and financial indicators; all of them stationary (logs and differencing), with monthly data for the last 25 years. I am trying to predict changes in a financial variable (...
1 vote
28 views

### How to form a single index?

My goal is to order companies based on their profitability in a specified period (from the most profitable, to the least profitable). For that I need a single index. I have got a dataset similar to ...
• 11
31 views

### high feature correlation but good OLS prediction

I have regression results where unconstrained OLS is near optimal - out of sample scores are almost the best when compared to some other constrained regression models. Although the ratio of number of ...