3
$\begingroup$

Random Forest algorithm has been recently proposed for estimating parameter values within the context of Approximate Bayesian Computation (Raynal et al 2017). The idea consists of training regression trees on a reference table composed of the parameters values (response variables) used for generating pseudo data from which summary statistics (predictors) were computed. Once the trees are trained, it is possible to predict the expected parameter value from the summary statistics computed on the observations.

As I'm new to Random Forest algorithm, I would like to know if there is a way to access some how the posterior distribution, and not only the expectation of the parameter of interest. I'm particularly interested in doing posterior predictive checks.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
-1
$\begingroup$

You'd have to be able to have access to the internals of any RF algorithms you used. For Bayesian, this could be a hornets' nest, since there are commonly thousands of trained trees used. Recall, Bayesian is known to potentially overfit, so it's not a cure-all to the overfitting problem.

Improve this answer
$\endgroup$
4
  • 2
    $\begingroup$ Your third sentence is quite unclear to me. Could you explain what you mean by 'Bayesian is known to potentially overfit'? $\endgroup$
    – mkt
    Apr 19, 2018 at 6:00
  • $\begingroup$ Sure, the main citation on abandoning the empirical Bayes approach due to over-fitting is Cawley, Nicola L. C. Talbot. On over-fitting in model selection and subsequent selection bias in performance valuation. Journal of Machine Learning Research, 11(Jul): 2079–2107, 2010. $\endgroup$
    – user32398
    Apr 19, 2018 at 16:03
  • 3
    $\begingroup$ Thanks for the citation, I will take a look at it. My main question was basically, "Bayesian what"? And your comment mostly clarifies this for me ("empirical Bayes approach") $\endgroup$
    – mkt
    Apr 19, 2018 at 20:25
  • $\begingroup$ Given the comment, this needs edit, since you don’t mean “Bayesian” but “empirical Bayesian” approach, those are not the same. Bayesian approach is not known for overfitting. $\endgroup$
    – Tim ♦
    Aug 30, 2020 at 8:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.