Questions tagged [pca]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

How are the signs of the loadings in ICA interpreted?

In my novice understanding of ICA, we generate two matrices: a source matrix, which describes the contribution of variables to the independent components (analogous to loadings in PCA..?) and the ...
0
votes
0answers
14 views

Model Examples always with 1 or 2 features

Why are all the model examples that I see on sklearn (e.g., https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.LocalOutlierFactor.html or https://scikit-learn.org/stable/auto_examples/...
1
vote
1answer
27 views

Combining continuous and binary data in unsupervised learning

I am working on cluster detection in a data set consisting of housing data. Each data point has some continuous features, such as house size, and some discrete ones, such as the number of garages (0 ...
0
votes
0answers
15 views

Combining datasets when working with PCA

I'm largely a statistics novice, and I've gotten into a discussion with my lab mate about the application of PCA. Previously I've applied PCA only to a uniform dataset (results from cells all ...
2
votes
0answers
18 views

How is explained variance in sparse PCA calculated?

Sparse PCA is a technique proposed by Zou et all in this paper. In usual PCA the obtained loadings are orthonormal, and the resulting scores are uncorrelated. However, in sparse PCA you give up these ...
0
votes
0answers
10 views

Linear Discriminant Analysis with Random Forest

Does it make sense to combine the 2? I'm testing out several model combinations. I used LDA (Linear Discriminant Analysis) as a dimensional reduction method and layered a SVM model for classification -...
3
votes
0answers
29 views

Reconstruction Error in PCA

I've read that if we have a data matrix $X$ and the eigenvectors $V$ of its covariance matrix $Cov(X) = X^{T}X$, then whereas $$XV$$ gives us the projection of our data on low dimension, $$XVV^{T}$$ ...
6
votes
1answer
131 views

What does Sparse PCA implementation in Python do?

I am interested on using sparse PCA in python and I found the sklearn implementation. However, I think this python implementation solves a different problem than the original sparse pca algorithm ...
0
votes
0answers
21 views

Multiple species & environment variables: how to reduce variable space (PCA, CCA, RDA?)

I am completely new to multivariate analyses and I need an advice how to get it applied to my data and which analyses to choose for which purpose. My dataset is presence/absence (or relative ...
2
votes
0answers
13 views

Is there a way to get predicted returns from a predicted covariance matrix?

To obtain an "eigenportfolio" of returns based on PCA analysis of the covariance matrix of a basket of $M$ stocks, we have the formula [1][2] $$ P = RW_k\Lambda^{1/2}_k $$ where $R$ is a matrix of ...
1
vote
0answers
20 views

Can I combine independent components from different models using PCA?

I have a set of independent components for each subject in my dataset (i.e. an ica model was generated for each subject). The samples used to generate each set of ICs are aligned across subjects, and ...
7
votes
1answer
192 views

Is the PCA estimator used in regression root-n-consistent?

Consider a sample of $n$ observations $(y_i, x_i)$, $i=1,\ldots,n$ and assume without loss of generality that the samples are centered. The true model is $y_i=x_i^t\beta+\epsilon$, and the OLS ...
1
vote
0answers
6 views

Labeling KPCA in R

The function $\texttt{biplot}$ in R is very useful for creating visualizations when performing PCA. However, when performing kernel PCA in R, I cannot find a way to label the loadings on the graph. ...
1
vote
1answer
20 views

What are the problems of using every feature to predict an outcome?

I've got a problem I'm trying to solve at work where we have over 500 features to predict a binary outcomes (buys/ not buy). I'm being asked to throw everything into a PCA and then run a model. ...
0
votes
0answers
12 views

How to quantify the similarity of EOF loading in multiple matrices?

I have five 3-D matrices (time, latitude, and longitude) representing the same variable but from different sources, denoted as A, B, C, D, and E. I calculated the first five EOF loadings for each of ...
2
votes
1answer
34 views

PCA in R: different results for caret and prcomp

Can some tell my why the preProcess function from the caret packages gives a different result than the ...
1
vote
0answers
10 views

Improving the Naive Bayes classifier performance through decorrelation?

I was wondering if it is possible to improve the performance of the Naïve Bayes classifier by decorrelating the data. The Naïve Bayes assumes conditional independence of the features given some class $...
0
votes
0answers
12 views

Why is the entropy of principal component scores always the same?

Suppose I have a matrix, and I find its principal components. Then I project onto that matrix to get the scores. I then compute the entropy of every row, and it's always the same constant! This is not ...
1
vote
0answers
10 views

Are there any package for stratified bootstrap re-sampling for principal component analysis in R? [closed]

It is possible to bootstrap principal component analysis (see package bootSVD) It is possible to conduct stratified bootstrapping (see package boot, option strata). However, is it possible to conduct ...
0
votes
0answers
10 views

Addressing spatially nested data in nMDS?

I have an experiment comparing bird community composition in different forest stands, having subsampled each stand a few times. Is there any way to account for this subsampling (i.e., non-independent ...
0
votes
0answers
23 views

PCA on TF-IDF matrix

I want to perform PCA on TF-IDF matrix, but I am not sure, should I center this matrix first or not? And should I do scaling or just centering?
0
votes
0answers
28 views

Using PCA to estimate a single index

I am using a set of different variables of a dataset that have been used to test the cognitive ability of each individual. However, I would like to synthesize these variables into a single one ...
0
votes
1answer
29 views

PCA provides principal directions, what does tSNE provide?

One of my main frustrations with the current state of single cell transcriptome analysis is representations of cells within $tSNE$ plots. These $tSNE$ plots provide amazing separation of the data and ...
0
votes
0answers
43 views

Derive Confidence Intervals for Eigenvalues of a Covariance Matrix

I am working with a dataset of n = 273 observations, p = 9 variables for which I have generated principal components. The task I am faced with is: Assume the eigenvalues of a covariance matrix ${cov(...
0
votes
0answers
18 views

Differences Between Pearson'S Correlation And PCA Biplots

I have a scaled PCA biplot of the first two PCs of a data set with 43 observations and 5 variables. The biplot indicates certain relationships between variables, based on the angles between the ...
0
votes
1answer
35 views

Why do we use PCs instead of transformed data as new variables in linear regression?

PCA and SVD Using the SVD on matrix X (column as features) We have X = U\sigmaV* where V contains the PCs and U\sigma would be the transformed data PCA with linear regression The ...
0
votes
0answers
37 views

pca interpretation

I'm analyzing the expression of a dataset of genes in 12 tissues(the rownames). I constructed a matrix with genes as rows and tissues as columns. I transposed the matrix to observe the PCA of tissues. ...
0
votes
0answers
10 views

Can one normalize a PCA for specific features?

When dealing with data sets that have hundreds of dimensions, some phenotypic and some metadata, I would like to "normalize" the effect of specific (multiple) features on PCAs. I can get the ...
0
votes
0answers
15 views

Removing multi-collinearity with PCA for regression analysis

I'm interested in studying the impact or importance of each feature on the response variable. I'm thinking running multiple linear regression with multiple features, and running regression analysis ...
1
vote
1answer
25 views

Comparing the first principal component with an observed variable (mean)?

I want to see how using the mean of my variables instead of the first principal component helps represent (the first dimension) of my data. The idea is that if those are similar enough, I might as ...
-2
votes
1answer
32 views

Dimension reduction (matrix) using eigenvalues and eigenvectors. PCA?

I am stuck with one task where I need to reduce dimensions of the matrix 20000 by 10, compute eigenvectors and eigenvalues and determine how much the dimension can be reduced and transform the data. ...
0
votes
0answers
26 views

why is the first column of my PCA always the largest?

I have a set of samples, compute the covariance matrix, and calculate the eigenvalues. The largest eigenvalue corresponds to the first column. I then re-order my original samples and repeat the ...
0
votes
1answer
21 views

Function of Feature Transformation using PCA

I completely understood the math behind PCA. I have a doubt here while calculating the function that will do the transformation. According to the book : Deep Learning by Ian Goodfellow, Yoshua Bengio ...
0
votes
0answers
10 views

What is the correct explanation for the definition of Eigen vectors of covariance matrix, Principal components and Eigenfaces?

We have an input matrix X consisting of n images. We need to do PCA on this matrix. We compute covariance matrix of X, and find the Eigen values. The Eigen vector corresponding to highest Eigen value ...
0
votes
0answers
13 views

Classification after using PCA analysis for Target variable

New to data mining, I'm very lost on how to use the components generated by PCA for classification. I have a dataset that contains peoples' phone usage(duration of every session) and their self-...
0
votes
1answer
50 views

Predictive model based on Principal Components when new data has different variables

I built a logistic regression model to classify a corpus of documents. The dependent variable is the type of document (eg A or B) while the dependent variables, because of dimensionality, are the ...
0
votes
0answers
15 views

Running PCA on cointegrated variables

I know that there are a lot of questions about the topic but I would like some extra explanation. What is the effect of running PCA on cointegrated variables? I'm doing PCA on financial data trough ...
2
votes
0answers
26 views

How to do principal component regression for repeated measures?

I have between-subject repeated measures data. I want to select the parameters for my regression model. PCR /PCA is one option to reduce dimensionality. How can I do this in the case of a repeated ...
0
votes
0answers
7 views

How many observations need to be in place for multiple correspondence analysis with a particular number of questions/categories

I'm wondering about how many observations need to be in place for a particular set of questions. If I have data as follows: ...
1
vote
0answers
22 views

which interactions to explore between sex, age, height to predict weight

As a learning exercise I'm running linear regression to predict person's weight from: sex, age, height. here's a few sample lines ...
0
votes
0answers
44 views

Different reconstruction errors using different PCA algorithms

In Matlab, I perform PCA on a centered and scaled (std-scaled) data set X_cs in four ways: builtin pca using the builtin ...
0
votes
0answers
36 views

How to compare two PCAs

I am working on a deep learning research and came across the following problem: I have a network (let's call it A) that performs a certain task with X% ...
0
votes
1answer
40 views

How can I calculate principal components scores in a compositional data PCA?

I hope someone can help. Please let me know if my question is not clear. I have compositional data, where there are three variables which sum to 100% for each row, something like this: ...
0
votes
0answers
27 views

Time series plots, polynomial coefficients and PCA

I have several time series plots that I have their polynomial coefficients (curve fitting using Matlab polyfit). Is it possible and valid to use Principal Component Analysis (PCA) to try to classify ...
0
votes
1answer
37 views

Are my PCA groups significantly different?

I am studying the feeding behaviour of deep sea fishes, and have produced a dataset containing percentages for 20 different fatty acids (totalling 100%) for 32 individual fish. I have performed PCA (...
0
votes
1answer
17 views

Sampling and Standardization of data before applying dimentionality reduction?

I'm trying to solve a classification problem with 4 parameters, next_action - binary variable(0/1) total_visits- numerical value days_Since_last_visit - numerical lead_source- categorical variable (5 ...
0
votes
0answers
10 views

Using dimension reduction techniques for poverty/wealth indicator

I would like to create an indicator/index of a person's wealth (or socio-economic status, SES). I have about 20 variables that are a combination of education, household assets, access to money, and ...
0
votes
0answers
11 views

Principal Components Analysis, Reading PC Plot

For Question 2v, can someone please explain to me why each subject is located where they are in terms of the two principal components.
0
votes
0answers
9 views

Multiple factor analysis: Getting more number of factors than the number of dimensions/ features

I am trying to apply multiple factor analysis on a survey data, which has all sorts of features - numerical, categorical and ordinal. In total, there are 109 features. Now, when I did multiple factor ...
0
votes
0answers
17 views

In max-variance PCA, why is the variance of the projected data equals to $\sum_{j=1}^M\mathbf{u}^T_j\mathbf{S}\mathbf{u}_j$?

In my machine learning course we have been taught that given a new axis $\mathbf{u}_j$ and a datapoint $\mathbf{x}_n$, the projection is $z_j = \mathbf{u}^T\mathbf{x}_n$. The variance of $z_j$ can be ...