1
$\begingroup$

Could someone explain the intuition behind the difference of feature importance using Factor Analysis vs. Random Forest Feature importance. Does there lie an advantage in RF due to the fact that it does not need an explicit underlying model?

$\endgroup$
1
$\begingroup$

It is not easy to compare two things concretely that are so different. But on an abstract level, there are many differences.

1) Factor analysis is purely unsupervised. The idea is to explain the observations $X$. Random forests are supervised, as their aim is to explain $Y|X$.

2) Factor analysis finds a latent representation of the data that is good at explaining it, i.e. $WZ \approx X$. Random forest feature importance tries to find a subset of the features with $f(VX) \approx Y$, where $f$ is the random forest in question and $V$ is binary.

Thus, both methods reflect different purposes. One tries to explain the data, the other tries to find those features of $X$ which are helping prediction.

Adding to that, factor analysis has a statistic interpretation--I am not aware of any such thing for RF feature selection.

$\endgroup$
  • $\begingroup$ Factor analysis is purely unsupervised or, to put it differently, it is multivariate, not a sequential or hierarchiacal univariate approach. $\endgroup$ – ttnphns Sep 16 '14 at 10:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.