2
$\begingroup$

I am exploring a tree structured as follows:

A/ parent node = parent population, size N, with a proportion of success for an experiment (eg. p = nb success / N)

B/ child nodes = sub populations for a give parent node. Ex.: level one child sizes are n1, n2, ... nn, where n1 + n2 + ... + nn = N. Success proportions are also computed (p1 = nb1 success / n1, ..., pn = nbn success / nn)

This could be summarized by the enclosed picture. The problem What I am trying to do is cut the nodes where the proportions of success are not statistically significant. In other words, how can I trust the proportion of success for a given node, according to its population size?

The context of this question is pretty specific: p, p1, ... are always very low (between 0 and 0.0002, and node size can vary a lot, from 10 to several thousands of observations.

So how can I compute the minimum node size, so that I can consider a node proportion of success as statistically significant? Moreover, do I have to do a computation for each of the tree node, do I have to take the dependencies between the nodes into account, or is a global approximation for all the nodes acceptable?

I'm working with R. My first insight was use the prop.test function for all node and check the p-value, but I realize this completely wrong.

As anyone an idea to deal with this tricky problem?

Improve this question
$\endgroup$
1
  • $\begingroup$ Maybe a starting point can be found here : grokbase.com/t/r/r-help/10b80sxv1s/… Unfortunately, this thread is about comparing 2 proportions. I am a bit confused: how could this be applied to my case? $\endgroup$
    – PLOTZ
    Commented Apr 16, 2014 at 19:37

1 Answer 1

Reset to default
1
$\begingroup$

What you are describing is recursive partitioning, implemented, for example, in the R rpart package. It is with difficulty that this is accomplished, because the data are not capable of revealing the correct connections/configurations except when the signals are very strong or the sample size is extremely large. Whenever you wish to use a single dendrogram to represent the patterns (as opposed to random forests, bagging, and boosting trees) it is incumbent upon you to demonstrate that the patterns are replicable. This can be done by bootstrapping the entire process and seeing if the same or similar trees emerge each time. See http://biostat.mc.vanderbilt.edu/ComplexDataJournalClub for an example where this was done.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you Frank for this answer and for the interesting link. However, my problem is slighlty different. I do not really want to use a regression or classification tree algorithm, such as CART for instance. In my case, the structure of the tree (split and nodes hierarchy) is completely defined by expert knwoledge. So I am just counting trials and sucess in each node, and would like to know whether a node counting can be considered as significiant or not. Not sure if I'm clear... $\endgroup$
    – PLOTZ
    Commented Apr 16, 2014 at 12:00
  • $\begingroup$ Sorry I misunderstood. But "cutting the nodes where the proportions of success are not statistically significant" needs clarification then. Exactly what is the hypothesis being tested, if you are interested in "significance"? $\endgroup$
    – Frank Harrell
    Commented Apr 16, 2014 at 12:36
  • $\begingroup$ Let's consider p1 and p2 and imagine they are approximatively equal, but with n1 >> n2. How can I compare both subpopulations by using this information and, precisely, is there a minimum size of n2 to compute the proportion of success with confidence ? I would like to cut the node, for which the population size is so small that we can't trust the proportion of sucess. – $\endgroup$
    – PLOTZ
    Commented Apr 17, 2014 at 8:56
  • $\begingroup$ nobody can help ??? :-( $\endgroup$
    – PLOTZ
    Commented Apr 18, 2014 at 12:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.