3
$\begingroup$

I'm a user of ctree function from partykit package in R. I always wondered for which purpose we want to use Monte Carlo to compute the distribution of test statistics? The literature suggest that it goes for large samples. I checked vignettes for partykit and coin but I didn't get clear answer. Are there any different conditions we might use it? I noticed that when the MC is used, ctree returns more variables for a given tree. Why?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

The default method for obtaining the p-values in conditional inference trees is by using the asymptotic permutation distribution (which is normally or chi-squared distributed, depending on the type of test). Alternatively, you can approximate the exact finite-sample permutation distribution by drawing a sufficiently large number of Monte Carlo samples. The two approaches will typically be similar - increasingly so for larger sample sizes. In small(er) samples there might be a bit of a difference with the approximated finite-sample distribution sometimes leading to slightly more powerful tests. Hence, in your case I suspect that the sample size is small to moderate and you get a few more significant splits in the tree.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you, that's highly informative. It's very nice that someone like you gives the answer. In my study I have 2500 observations for 120 variables. I understand that if I use permutation tests I shouldn't be worried about the distribution of the response. Is there any difference beetween Monte Carlo and Bonferroni when using non-normal data? With non-normal data is it safe to assume higher expectations towards nonmonotonic relations between the variables and switch to maximally selected statistics instead of linear? $\endgroup$
    – Tom
    May 10, 2020 at 11:02
  • 1
    $\begingroup$ Yes, asymptotics should kick in earlier for approximately normal data - but they also work otherwise. Maximally selected statistics often work better for nonmonotonic relationships. $\endgroup$
    – Achim Zeileis
    May 10, 2020 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.