Linked Questions

2
votes
0answers
115 views

Random Forest after PCA results don't make sense [duplicate]

I'm playing around with random forest classification and principal component analysis using scikit-learn and have found a point of confusion. I want to fit two models that predict 1 of 5 target ...
89
votes
4answers
104k views

PCA and proportion of variance explained

In general, what is meant by saying that the fraction $x$ of the variance in an analysis like PCA is explained by the first principal component? Can someone explain this intuitively but also give a ...
25
votes
5answers
4k views

How can top principal components retain the predictive power on a dependent variable (or even lead to better predictions)?

Suppose I am running a regression $Y \sim X$. Why by selecting top $k$ principle components of $X$, does the model retain its predictive power on $Y$? I understand that from dimensionality-reduction/...
25
votes
2answers
15k views

Does it make sense to combine PCA and LDA?

Assume I have a dataset for a supervised statistical classification task, e.g., via a Bayes' classifier. This dataset consists of 20 features and I want to boil it down to 2 features via ...
10
votes
3answers
926 views

The first principal component does not separate classes, but other PCs do; how is that possible?

I ran PCA on 17 quantitative variables in order to obtain a smaller set of variables, that is principal components, to be used in supervised machine learning for classifying instances into two classes....
4
votes
2answers
4k views

Why does PCA feature reduction make accuracy dramatically worse?

I'm trying to estimate how much feature reduction using PCA can help with increasing accuracy in case of classification using different ml methods. I'm using digits dataset available in scikit-learn. ...
7
votes
1answer
7k views

Should PCA be performed before I do classification?

I have got a problem about doing a classification. I have got around 50 datasets. Each of them has 15 features. I am trying to use these features to classify the 50 datasets to either 'Good' or 'Bad'....
6
votes
4answers
3k views

Does PCA followed by LDA make sense, when there is more data available for PCA than for LDA?

This is a question about classification. I am a neuroscience student with little experience of classification methods and I'd be grateful for any advice about the best way to implement a linear ...
2
votes
2answers
524 views

What classifier to use after performing PCA?

Which is the most used classifier applied once PCA is performed? I computed PCA on my data, and now I have the points projected on a line, I was wondering what is the preferred classifier. Is a ...
4
votes
2answers
315 views

How is Hyndman's explanation of proper Time Series Cross Validation different from Leave-One-Out?

Hyndman's great explanation of proper time series CV is at the bottom of the page in the following link: http://robjhyndman.com/hyndsight/crossvalidation/ Leave-One-Out illustration in the following ...
2
votes
3answers
2k views

Do we need to use one-hot if a feature has values {1,2}?

I'm wondering whether it's necessary to use one-hot if a feature has only two values but not {0,1}. I'm also wondering whether there is a good way to reduce the number of features after one-hot. Is ...
0
votes
1answer
945 views

PCA dramatically reduce the accuracy of classification

I am doing classification of this UCI Dataset in Matlab. I represented dataset as matrix (instances x dimensions) and 2nd matrix as (instances x label [instances x 1]). With Naive Bayess I get ...