8
$\begingroup$

The documentation for Hmisc::describe (page 77 of the PDF) says:

For numeric variables, describe adds an item called Info which is a relative information measure using the relative efficiency of a proportional odds/Wilcoxon test on the variable relative to the same test on a variable that has no ties. Info is related to how continuous the variable is, and ties are less harmful the more untied values there are. The formula for Info is one minus the sum of the cubes of relative frequencies of values divided by one minus the square of the reciprocal of the sample size. The lowest information comes from a variable having only one distinct value following by a highly skewed binary variable. Info is reported to two decimal places.

My questions are:

  1. How those "relative frequency" is defined for continuous variables that would have infinite values?
  2. Why is it defined in this way instead of some other way? What is the advantage of adopting this definition?
  3. Where can I find a bibliographic reference for this variable?
  4. Can I have an example of a variable with low info and a variable with high info?
  5. How do I interpret "practically" this measure? Why I need it?

I guess I don't fully understand this measure and I need a more "practical approach" with it.

Related questions:

Help interpreting output of describe function from Hmisc package in R

Describe function by hmisc in R

$\endgroup$

1 Answer 1

13
$\begingroup$

Great questions. Information measure Info in Hmisc::describe is generated from a variance / information measure coming from a two-sample Wilcoxon/proportional odds comparison with equal sized groups even though we are interested in describing the information content in a single numeric variable. By conceiving of a hypothetical two-group comparison of the variable values in question we have an efficiency anchor. Info is approximately the ratio of the variance of a log odds ratio (a simple transformation of the Wilcoxon statistic) when there are no ties in the data to the variance computed on the observed data. Info is a 0-1 relative information measure when 0=no information/worst possible statistical efficiency (e.g. binary outcome variable with no events) and 1=information for a continuous variable with no ties in the data.

Whitehead 1993 provided the approximate variance formula for log odds ratios in two-group comparisons.

The Wilcoxon two-sample test is equivalent to the proportional odds model and gives us a unified way to look at statistical efficiency for all levels of categorization of a response variable Y. I say unified because you can compute odds ratios for binary, discrete ordinal, continuous ordinal, and continuous interval-scaled Y that all have the same interpretation.

You can also think of Info, after multiplying by the sample size, as giving you the effective sample size (in comparison to continuous Y) in comparison with a Y with no ties. For example a very imbalanced binary Y may have Info of 0.2 indicating roughly speaking that you could achieve the same statistical efficiency and power from a continuous Y with only $\frac{1}{5}$ the sample size. An ordinal Y with 5 well-populated levels may have Info of 0.98 so has almost the same statistical information (especially for the purpose of studying associations with other variables) as a truly continuous Y with no ties.

Info for binary Y is $3p(1-p)$ where $p$ is the proportion of Y=1, so the effective sample size is $3np(1-p)$. So the maximum efficiency of a binary Y is $\frac{3}{4}$.

The definition of Info is as follows. Relative frequencies refer to frequencies of distinct values of Y in the data sample, so for a truly continuous Y each observation has relative frequency $\frac{1}{n}$. Let $n_{1}, n_{2}, \ldots, n_{k}$ denote the frequencies of all $k$ distinct value of Y. Info is defined as the ratio of $1 - \frac{1}{n^{3}} \sum_{i=1}^{k} n_{i}^{3}$ to the value of this expression when all $n_{i}=1$, which is $1 - \frac{1}{k^{2}}$ since with no ties $k=n$.

One of the purposes of having Info in the output is to alert analysts about information loss from binning numeric variables, and to constantly remind them that binary Y has minimum statistical information. Info also shows that if you have an almost continuous Y with one value having a good many ties, the relative information content of the variable is still high. When Y is the dependent variable in a statistical model or with machine learning, Info helps you derive the sample size needed to have a reliable model or to achieve a certain statistical power for a certain comparison.

Info doesn't know anything about measurement error. If there is significant measurement error in Y, efficiency and effective sample size will be lower than what Info indicates.

See also this and this.

It would nice to also develop a relative information measure for unordered categorical variables. For these the effective sample size might be something like the median sample size of the variable's categories. A binary unordered variable would have the same sample size as a binary ordered variable, so median doesn't quite work in that case.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.