# Tree structured proportion of success : statistical significance?

I am exploring a tree structured as follows:

A/ parent node = parent population, size N, with a proportion of success for an experiment (eg. p = nb success / N)

B/ child nodes = sub populations for a give parent node. Ex.: level one child sizes are n1, n2, ... nn, where n1 + n2 + ... + nn = N. Success proportions are also computed (p1 = nb1 success / n1, ..., pn = nbn success / nn)

This could be summarized by the enclosed picture. What I am trying to do is cut the nodes where the proportions of success are not statistically significant. In other words, how can I trust the proportion of success for a given node, according to its population size?

The context of this question is pretty specific: p, p1, ... are always very low (between 0 and 0.0002, and node size can vary a lot, from 10 to several thousands of observations.

So how can I compute the minimum node size, so that I can consider a node proportion of success as statistically significant? Moreover, do I have to do a computation for each of the tree node, do I have to take the dependencies between the nodes into account, or is a global approximation for all the nodes acceptable?

I'm working with R. My first insight was use the prop.test function for all node and check the p-value, but I realize this completely wrong.

As anyone an idea to deal with this tricky problem?

• Maybe a starting point can be found here : grokbase.com/t/r/r-help/10b80sxv1s/… Unfortunately, this thread is about comparing 2 proportions. I am a bit confused: how could this be applied to my case? Apr 16 '14 at 19:37

## 1 Answer

What you are describing is recursive partitioning, implemented, for example, in the R rpart package. It is with difficulty that this is accomplished, because the data are not capable of revealing the correct connections/configurations except when the signals are very strong or the sample size is extremely large. Whenever you wish to use a single dendrogram to represent the patterns (as opposed to random forests, bagging, and boosting trees) it is incumbent upon you to demonstrate that the patterns are replicable. This can be done by bootstrapping the entire process and seeing if the same or similar trees emerge each time. See http://biostat.mc.vanderbilt.edu/ComplexDataJournalClub for an example where this was done.

• Thank you Frank for this answer and for the interesting link. However, my problem is slighlty different. I do not really want to use a regression or classification tree algorithm, such as CART for instance. In my case, the structure of the tree (split and nodes hierarchy) is completely defined by expert knwoledge. So I am just counting trials and sucess in each node, and would like to know whether a node counting can be considered as significiant or not. Not sure if I'm clear... Apr 16 '14 at 12:00
• Sorry I misunderstood. But "cutting the nodes where the proportions of success are not statistically significant" needs clarification then. Exactly what is the hypothesis being tested, if you are interested in "significance"? Apr 16 '14 at 12:36
• Let's consider p1 and p2 and imagine they are approximatively equal, but with n1 >> n2. How can I compare both subpopulations by using this information and, precisely, is there a minimum size of n2 to compute the proportion of success with confidence ? I would like to cut the node, for which the population size is so small that we can't trust the proportion of sucess. – Apr 17 '14 at 8:56
• nobody can help ??? :-( Apr 18 '14 at 12:14