2
$\begingroup$

Random Forest algorithm has been recently proposed for estimating parameter values within the context of Approximate Bayesian Computation (Raynal et al 2017). The idea consists of training regression trees on a reference table composed of the parameters values (response variables) used for generating pseudo data from which summary statistics (predictors) were computed. Once the trees are trained, it is possible to predict the expected parameter value from the summary statistics computed on the observations.

As I'm new to Random Forest algorithm, I would like to know if there is a way to access some how the posterior distribution, and not only the expectation of the parameter of interest. I'm particularly interested in doing posterior predictive checks.

$\endgroup$
0
$\begingroup$

You'd have to be able to have access to the internals of any RF algorithms you used. For Bayesian, this could be a hornets' nest, since there are commonly thousands of trained trees used. Recall, Bayesian is known to potentially overfit, so it's not a cure-all to the overfitting problem.

$\endgroup$
  • 1
    $\begingroup$ Your third sentence is quite unclear to me. Could you explain what you mean by 'Bayesian is known to potentially overfit'? $\endgroup$ – mkt - Reinstate Monica Apr 19 '18 at 6:00
  • $\begingroup$ Sure, the main citation on abandoning the empirical Bayes approach due to over-fitting is Cawley, Nicola L. C. Talbot. On over-fitting in model selection and subsequent selection bias in performance valuation. Journal of Machine Learning Research, 11(Jul): 2079–2107, 2010. $\endgroup$ – wrktsj Apr 19 '18 at 16:03
  • 1
    $\begingroup$ Thanks for the citation, I will take a look at it. My main question was basically, "Bayesian what"? And your comment mostly clarifies this for me ("empirical Bayes approach") $\endgroup$ – mkt - Reinstate Monica Apr 19 '18 at 20:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.