Linked Questions
16
votes
4
answers
25k
views
How does PCA improve the accuracy of a predictive model? [duplicate]
I've seen in a kaggle challenge about digit recognition someone who used PCA before decision tree or other techniques.
I thought it was just for compressing data but he aimed to improve his score.
...
- 193
15
votes
2
answers
27k
views
PCA before Random Forest Regression provide better predictive scores for my dataset than just Random Forest Regression, how to explain it? [duplicate]
I have a regression dataset where the features are on the order of ~ 400 variables and the dataset itself is around 300 samples. I tried to use Random Forest Regression (RFR) on the dataset and used ...
- 165
6
votes
1
answer
473
views
How can preprocessing with PCA but keeping the same dimensionality improve random forest results? [duplicate]
I found these sentences:
PCA before random forest can be useful not for dimensionality
reduction but to give you data a shape where random forest can perform
better.
I am quiet sure that ...
1
vote
3
answers
280
views
Why and when would one begin with PCA on $X$ when predicting $y$? [duplicate]
From a very general point of view, when you have a dataset $X$ and want to predict a label $y$, what is the purpose of beginning with a PCA (principal component analysis) first, and then doing the ...
- 11
1
vote
0
answers
404
views
Is it realistic to achieve better results when doing PCA before neural network classification? [duplicate]
I'm doing a data analysis on data with more than 100 dimensions.
After that different ML-Algorithms like NN are applied to it.
When I do a PCA in the first place to reduce dimensionality to somewhat ...
- 71
1
vote
0
answers
309
views
When using principal components as predictors in linear regression, PC1 comes out not significant [duplicate]
I have a dataset with 37 independent variables and a dependent variable. In order to take care of multicollinearity among the independent variables, I conducted a PCA on them. My first principal ...
1
vote
0
answers
92
views
How to efficiently do PCA/SVD on dataset with thousands of features (both continuous and OHE) [duplicate]
I am currently dealing with a dataset with about 300,000 records, there are a wide variety of categories in several columns and naturally when one-hot-encoding these the number of features increases ...
- 11
1
vote
0
answers
73
views
Adding more features improves the variance explained by PCA but the prediction model performs worse [duplicate]
With my 37 base features, I obtain a PCA 7 components whose explained variance is 62584.661. Below is my code (I use scikit-learn):
...
- 877
0
votes
0
answers
16
views
PCA on predictor variables before supervised learning is appplied [duplicate]
Say I have 20 predictor vars (X's) and 1 response var (Y) and I'm attempting to build a supervised model y=f(x). Is it advisable or is it "OK" to firstly run PCA on all of the Predictor variables - ...
- 835
33
votes
5
answers
23k
views
What can cause PCA to worsen results of a classifier?
I have a classifier that I'm doing cross-validation on, along with a hundred or so features that I'm doing forward selection on to find optimal combinations of features. I also compare this against ...
- 779
43
votes
6
answers
14k
views
Examples of PCA where PCs with low variance are "useful"
Normally in principal component analysis (PCA) the first few PCs are used and the low variance PCs are dropped, as they do not explain much of the variation in the data.
However, are there examples ...
- 433
64
votes
3
answers
13k
views
Why does shrinkage work?
In order to solve problems of model selection, a number of methods (LASSO, ridge regression, etc.) will shrink the coefficients of predictor variables towards zero. I am looking for an intuitive ...
33
votes
4
answers
14k
views
Is PCA always recommended?
I was wondering if PCA can be always applied for dimensionality reduction before a classification or regression problem. My intuition tells me that the answer is no.
If we perform PCA then we ...
- 744
28
votes
6
answers
17k
views
Dealing with correlated regressors
In a multiple linear regression with highly correlated regressors, what is the best strategy to use? Is it a legitimate approach to add the product of all the correlated regressors?
- 1,579
33
votes
2
answers
28k
views
Does it make sense to combine PCA and LDA?
Assume I have a dataset for a supervised statistical classification task, e.g., via a Bayes' classifier. This dataset consists of 20 features and I want to boil it down to 2 features via ...
user39663