Questions tagged [pca]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

3,159 questions
Filter by
Sorted by
Tagged with
12 views

Principal components in linear regression for prediction [closed]

I have question regarding principal component analysis (PCA). I understood that you can use, let say first principal component and its scores in linear regression. For a simple example in R with ...
13 views

Can I consider interaction effect between two principal components?

I ran a PCA on five variable and got PC1, PC2 as the main axes. Then I want to run a GLMM with binomial distribution to see how these two PCs influence the response variable. My question is if I can ...
30 views

The discrepancy of results of PCA via Eigendecomposition vs via SVD in Python with scipy.linalg [duplicate]

I recently learned about different methods of PCA. I decided to manually implement PCA in Python with Eigendecomposition of cov(X) and the Singular Value ...
20 views

Looking for a PCA equivalent for regression? [closed]

I'm curious if there is a name for a regression which returns a set of orthogonal regression coefficient vectors in order of predictive power? I guess maybe a PCA that weights the dimensions of only ...
22 views

best way of dimensionality reduction with respect to model(classifier) performance

in my theoretical studies I faced the term Curse of Dimensionality so many times that when I started implementing those ideas in real world(in particular music genre classification) after getting ...
26 views

Why do variables with large values influence PCA stronger?

In principal component analysis, the loadings are weights which are used to transform (linearly) the original values. If the values are values of gene expression, then ...
11 views

Is there any methods to identify when PCA transformation is not optimal

I am working on unsupervized clustering problematic. I am currently reducing the dimensions of my data by applying first a PCA transformation through which I keep 80% of the variance (reducing ...
50 views

When to use "fixed number of components" in a PCA

The question is relatively simple (and stupid maybe). When do I use a fixed number of components (especially in analyzing a questionnaire). Is it useful when I have literature to a validated scale/...
35 views

In PCA, what is the difference between computing the eigenvectors for $X^T X$ and for the covariance matrix?

Why do we calculate eigenvectors based on $X^TX$ and other times based on the covariance matrix? (What's the difference?) I have found resources online some of which computes the (eigenvector,...
22 views

Do need to calculate PCA every time I add new item to dataset?

I want to visualise dataset of high dimensional data. And there comes PCA with dimensionality reduction. When I upload a new image and want to see where it is placed compared to images that are ...
2 views

Recombining Principal Components after PCA To Satisfy Exclusion Restrictions

I am reading a paper which uses a technique that extend the PCA method to obtain the reduced dimensions in a desired style. In particular, say we have a $N \times P$ matrix $\mathbf{M}$, where $N$ is ...
38 views

PCA to select few stocks to mimic and index

I am looking for a way to select a subset of stocks which returns can approximately mimick the return of an 'index' they corresponds to. These are all self created factor portfolios, and not traded ...
36 views

Why is the Mean of the 1st Principal Components 0?

I am reading a text on principal components which has the following excerpt: Since $\frac1n \cdot \sum_{i=1}^n x_{ij} =0$, the sample mean of the first principal component scores, $\bar{z}_1$, equals ...
16 views

How to use principal components in R to apply a multinomial logistic regression?

Context: I have an original dataset of +- 20k rows (samples) and 253 features (deletions, insertions and substitutions). These columns/dimensions/features are called SNVs (single nucleotide variants ...
31 views

All eigenvalues less than 1 in factor analysis

Is it possible to have all eigenvalues less than 1 during factor analysis? then what can be concluded here? since a factor is considered a factor when the eigenvalue of the factor is greater than 1. ...
31 views

In a pca plot from unscaled data, does the magnitude of the loadings still represent the contribution to the PC?

I want to infer the variables that contribute most strongly to a specific PC. However, the data was not scaled. Had it been scaled, then the contribution of each variable to the PC would be precisely ...
26 views

PCA with unbalanced panel

As far as I know, to use Principal Component Analysis (PCA) on a panel of data, data must be balanced. As an example, consider the returns of the constituents of S&P500 from 1967 to 2020. Because ...
17 views

Should I center and standarize in a PCA of relative growth?

Im quite familiar with PCA, but as a non-pure-statician I have come here to ask for help and comments. I have a dataset with multiple meassures (End and initial for a lot of meassures), and the time ...
23 views

I have a data set in which two variables are collinear (r^2 ≈ 0.7). I decided to extract the principal components, and then include these in a regression analysis to see which of the two variables ...
36 views

Explain the correlation between 7 critical success factors and 8 factors representing firms characteristics with ANOVA

For my dissertation I need to check if how 7 factors vary is somehow explained by how 24 other factors vary. (how the critical success factors (which are 7) vary is explained by the characteristics of ...
19 views

kernel PCA similarity matrix analogy

The standard explanation to linear PCA begins with the covariance matrix. That is, for a dataset $D$ of dimension $N \times d$, the covariance matrix is given as $\sum = \frac{D^{T}D}{N}$ where the ...
8 views

Why doing an ANOVA exits an error saying that the factors after the first one cannot be estimated and therefore they are removed?

For my dissertation I need to check if how 7 factors vary is somehow explained by how 24 other factors vary. I reduced the number of 24 factors to 8 via a PCA. So now I should check if the variance of ...
13 views

Using UMAP on the space of principal components, a valid proposition?

I have a genomics dataset with roughly 16,000 features. Currently, I'm in the process of clustering for cell subtype identification, which I'll then build a classifier on. However, I've run into two ...
14 views

Proof of SVD generates equal eigenvectors as PCA [duplicate]

Figure link: https://people.cs.pitt.edu/~milos/courses/cs3750-Fall2014/lectures/class9.pdf In process of PCA, we either decompose covariance matrix, or do SVD on X.  C = \frac{1}{n-1} X^T X = \frac{...
19 views

PCA for Time Series with Different Frequency in R

I am trying to perform the Principal Component Analysis in R on my dataset. My dataset consists of time series data with different frequencies. For example, I could have x1 in quarter ending (Mar, Jun,...
33 views

Should I scale my data before time series PCA?

I am doing PCA on 22 time series. All the series are log transformed and in their stationary form. The plot of all the input series is given below. My main aim is to forecast GDP. I noticed that ...
6 views

By multiplying data by eigenvectors of PCA, do we actually recover part of the left singular vector?

In software like sklearn or matlab, often we project data through first $r$ principal components of PCA. For example if we have column data matrix $X$ (mean centered) and principal component $v_r$, to ...
2 views

data projection on k - dime hyperplane (for eigenvector with largest eigenvalue) provides corrected representation

This is in reference to the text, outlier analysis by Charu C. Aggarwal: The author mentions, the projection of data points on the k - dimensional hyperplane corresponding to the largest eigenvalues (...
42 views

Principal Component Analysis to Time Series Data (PCA)

What is the procedure to do PCA on time series data? I followed the following method and I want to know whether it is correct Scaled the stationary time series Did PCA on the series obtained by ...
13 views

Do I need to normalize data for PCA with prcomp in R [duplicate]

I am trying to get the PCA for my data with the 'prcomp' function in R. My dataset are in different unit, as such do I need to normalize my dataset with z-score for example before proceeding to find ...
37 views

Why use covariance, instead of hankel when using PCA? [closed]

I found something intresting today! I have always been doing PCA by using SVD with covariance when I want to reduce the dimension. Assume that we have a vector $X = {x_0, x_1, x_2, \dots, x_n}$. $X$ ...
27 views

My setup: ...
55 views

How to understand the formula：patient score = ∑PC1A−∑PC1B? [closed]

Today I read a paper (doi: 10.1016/j.omtn.2020.08.030), in this paper, they used many methods and finally they got a matrix, and each row represents a gene, each column represents a patient sample. ...
36 views

Does linear discriminant analysis expands or only rotate?

Assume that we have two classes, $X$ and $Y$ and we find the mean $X_\mu$ and $Y_\mu$ and variance $X_\sigma$ and $Y_\sigma$. With that, we could use linear discriminant analysis to expend the ...
49 views

Reconstruction error in PCA?

I'm using PCA for a while, but recently I read about reconstruction error which I cannot understand... For example let's consider dataset consisting 5 variables: $X_1, X_2, X_3, X_4, X_5$. ...
48 views

Find first Principal Component (and loading) using a fast iterative algorithm without covariance matrix

I have a matrix $X$ and I would like to find its first principal component and the corresponding loadings. I would like to do this without computing the covariance matrix of $X$. How can I do so? This ...
30 views

Alternative data standardization procedures before PCA analysis

I'm working with morphometric data including 50 different variables (with very different scales) measured in 180 individuals (these 180 individuals belong to 4 different groups which received ...
14 views

Optimum number of PCs to retain when finding BIC

I am using the Bayesian Information Criterion (BIC) to find my optimal value of clusters (k) in a dataset of 1774 individuals with each 5903 SNPs. I used the following R code from the adegenet package:...
15 views

Kernel pca and kernel svm

PCA is a way to reduces dimension and complexities, but is it ok to use kernel PCA with radial basis function and then use kernel SVM using the same.
145 views

I don't completely understand the concept of PCA analysis [closed]

First of all, PCA analysis is not something I came across in my economics studies. But, recently, I wanted to make a PCA analysis of American GDP. I started to read about the fundamentals of PCA and ...
55 views

When are latent analyses useful?

As far as I understand, latent profile analysis, clustering or similar latent analyses are about finding something hidden in the data. Are there any guidelines or thoughts on when these techniques are ...
24 views

Performing PCA on multistate variables (vector rather than scalar values)

I have a data set of individuals genotyped over thousands of polymorphic (variable) loci. At each locus, each individual has two nucleotides from the set {A,C,G,T}, either a pair of identical ...
37 views

Why do we try to "Reproduce" Hilbert Spaces in Statistics?

I am trying to better understand why people are interested in "reproducing" Hilbert Spaces in Statistics and Machine Learning. I (think) understand the general idea behind Hilbert Spaces. ...
44 views

PC-Vector Autoregression (PC-VAR)

When using PC-VAR model for forecasting purposes, can we define it in the following manner? where a k-dimensional vector of intercepts is denoted by φ0 , Φ represents a k × k matrix of coefficients ...
27 views

PCA: is linearity important?

I have a 6 dimensions dataset where I want to apply PCA to remove one dimension. I did a small analysis to check for relationships in my data and concluded that there is very low linear correlation ...
20 views

PCA: does outlier detection make sense with low linear correlation? [duplicate]

I am experimenting PCA to detect outliers based on the reconstruction error. What I do: I start with a 6 dimensions dataset and reduce it to 5 dimensions. Then, I reconstruct the initial dataset and ...
13 views

Cross-Validation PCA in selecting number of PCA components [duplicate]

If I have 3 datasets and make comparison between them and after calculation of PCA components for each dataset. For instance, If D1 gave us 7 pc components , D2 gave 5pcs and D3 gave 4 pcs . How can I ...
35 views

Principal component analysis of RT-qPCR data

Greetings to all biostatisticians, I am analyzing a gene expression data set consisting of around 100 genes that were measured by RT-qPCR and expession values are given as 2^-delta Ct. Expression of ...