In the literature about decision tress and especially the family of tree approaches that avoid selection bias (conditional inference trees e.g. here: ctree: Conditional Inference Trees by Hothorn, Hornik and Zeileis) the term "case weight" is used - what does it exactly mean? As we speak about weights - can we clarify how a "node weight" in a tree compares to it? Could you provide a pedagogic example?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
5
The term "case weights" is used to distinguish weights in a regression/classification model from "proportionality weights" that are commonly used in least-squares regression. In the former case the number of observations is sum(weights) whereas in the latter case it is length(weights) (or sum(weights > 0)). For point estimates this typically does not make a difference but for the corresponding tests/p-values it does.
For some more comments in the context of regression (not trees), see e.g.: Weighted censored regression.
As for "node weight" I'm not sure what exactly you mean by this. I would expect that this is simply the sum of (case) weights in a certain node.
answered Oct 28, 2015 at 23:57
-
$\begingroup$ Sorry for my poor understanding. is it true that case weights are more or less a vector of 0s and 1s? A bit like indicators? Can I say that "case weights" more or less tell me the number of occurences of a specific set-up of input variables? Do you have a toy example at hand? Thank you very much for your efforts - this time and in the past! $\endgroup$– Richi WCommented Oct 29, 2015 at 10:16
-
$\begingroup$ No, case weights do not need to be binary, that's the point. If all weights are actually 0 or 1, then the case weights and proportionality weights approach coincide. And a small example is included in the weighted censored regression thread I linked. The point is that doubling all weights does not change the standard errors when using the proportionality approach. However, when doing the same for case weights, will decrease the standard errors by 1/sqrt(2) because this means that you doubled the sample size. $\endgroup$ Commented Oct 29, 2015 at 11:51
-
1$\begingroup$ In the question you provide an example where you apply the weights and in the mail conversation this is done too. Do you have any example in a blog post or a paper somewhere? Sorry for being so ignorant .. most probably the notion is rather easy to understand, I just don't find a clear reference .. thanks one more time. $\endgroup$– Richi WCommented Oct 29, 2015 at 15:19
-
$\begingroup$ In the mail they write " Thus, the scaling in the variances is done by the number of (non-zero) weights, not by the sum of weights." I get a feeling for this but still don't get it why this happens ... $\endgroup$– Richi WCommented Oct 29, 2015 at 15:20
-
$\begingroup$ When having data with the same observation several times, then using the corresponding frequencies as case weights should be very natural, I hope. For example, this occurs in in categorical data where frequencies are typically listed in contingency tables (see e.g.
as.data.frame(UCBAdmissions)in R). Proportionality weights are used used if you have aggregated (typically averaged) data from several observations but not the original unaggregated data. Then you know that the variance is decreased by a factor of 1/sqrt(n[i]). See any textbook on weighted least squares for more details. $\endgroup$ Commented Oct 29, 2015 at 23:36
$\begingroup$
$\endgroup$
"Case weight" is defined in Hothorn et. al's paper: Unbiased recursive partitioning: A conditional inference framework. I agree that it is a little confusing as written.
First, let's define exactly what a case weight is mathematically using the paper: it is a vector of "nonnegative integer value[s]" that indicate if a record is in the node. Additionally, this sentence from the paper is useful as well, "The algorithm induces a partition {B1,...,Br} of the covariate space X, where each cell B ∈{B1,...,Br} is associated with a vector of case weights."
So, the case weight is simply a vector indicating if an observation is in a node or in a particular final cell (final cell can be read as 'terminal node') of the partition space. The sum of case weights in a terminal node is the same as how many observations are in that final cell. Note that the length of the case weight vector is equal to the number of records.
I believe the term "node weight" is just an integer value that equals the row sum or the column sum of a case weight vector.
Case weights exist for every node in a tree and simply indicate which observations are being included.
A natural question might be, "Are case weights are always vectors of 0s and 1s?" In some cases, yes. However, sometimes it is desirable to weight records in terms of modeling (for example, suppose a study suffers from missing data and has decided to use inverse probability weighting. A rare observation (meaning one that had a high probability of being missing but we were able to record it) is weighted more heavily in the model (perhaps it gets a weight of 5 to represent 5 observations that were missing as is the concept behind inverse probability weighting).
Therefore, the case weights allow records to be counted multiple times in nodes. This could be done for a variety of reasons such as the one I gave above.
Let me further add an example of these terms used in the R package cforest's ctree_control and ctree documentations (which fit these kinds of trees described by Hothorn et al.).
"minsplit" is an argument for the ctree_control funciton defined as "the minimum sum of [case] weights in a node in order to be considered for splitting." This determined how many records (that may be weighted) must be in the node for it to be considered for splitting. Said another way, the node weights of the two daughter nodes must sum to greater than the minsplit argument.
"minbucket" is another argument defined as "the minimum sum of [case] weights in a terminal node." It means that no terminal node will contain fewer observations than its value. Said another way, the node weight of a terminal node will not be lower than its value.
These two criteria determine how small nodes can be.
