31
$\begingroup$

I'm working through a statistics textbook while learning R and I've run into a stumbling block on the following example:

enter image description here

After looking at ?quantile I attempted to recreate this in R with the following:

> nuclear <- c(7, 20, 16, 6, 58, 9, 20, 50, 23, 33, 8, 10, 15, 16, 104)
> quantile(nuclear)
   0%   25%   50%   75%  100% 
  6.0   9.5  16.0  28.0 104.0 

Given that the text and R have different results, I'm gathering that R is utilizing the median in the calculation of the first and third quartiles.

Question:

Should I include the median in calculating the first and third quartiles?

More specifically, does the textbook or R have this correct? If the textbook has this correct, is there a way to properly achieve this in R?

Thanks in advance.

$\endgroup$
  • 6
    $\begingroup$ A few threads here discuss the many ways quantiles can be computed or estimated. Here is one with an authoritative answer, but others are available by searching our site. In brief, your textbook appears to present a non-standard method of computing quartiles, but quantile types 1, 2, and 6 will reproduce them for a dataset of this particular size. None of the R methods corresponds to your textbook. (One wonders about the quality of this text...) $\endgroup$ – whuber Jan 20 '15 at 16:58
  • $\begingroup$ @whuber Thanks for this comment, it helped a lot since I fear I don't yet have the technical background to distinguish exactly what the different types in quantile are doing. $\endgroup$ – user60305 Jan 20 '15 at 17:27
  • $\begingroup$ @whuber: it is clearly non-standard (which is probably mentioned somewhere in the book), but not unintuitive. Do you think it is wrong mathematically? $\endgroup$ – Michael M Jan 20 '15 at 17:30
  • 6
    $\begingroup$ @Michael You can define a "quartile" to be anything you like, so there's nothing wrong mathematically. It is clear that asymptotically these definitions work for large $n$. But introducing a novel definition into a textbook does a disservice to thoughtful students like Chuck D. who notice that they cannot get their calculations to agree with software, publications, or anything else but their text. $\endgroup$ – whuber Jan 20 '15 at 17:43
  • 1
    $\begingroup$ R uses nine different definitions of quantiles (by default it uses definition 7). See ?quantile $\endgroup$ – Glen_b Mar 15 '16 at 7:01
40
$\begingroup$

Your textbook is confused. Very few people or software define quartiles this way. (It tends to make the first quartile too small and the third quartile too large.)

The quantile function in R implements nine different ways to compute quantiles! To see which of them, if any, correspond to this method, let's start by implementing it. From the description we can write an algorithm, first mathematically and then in R:

  1. Order the data $x_1 \le x_2 \le \cdots \le x_n$.

  2. For any set of data the median is its middle value when there are an odd number of values; otherwise it is the average of the two middle values when there are an even number of values. R's median function calculates this.

    The index of the middle value is $m = (n+1)/2$. When it is not an integer, $(x_l + x_u)/2$ is the median, where $l$ and $u$ are $m$ rounded down and up. Otherwise when $m$ is an integer, $x_m$ is the median. In that case take $l=m-1$ and $u=m+1$. In either case $l$ is the index of the data value immediately to the left of the median and $u$ is the index of the data value immediately to the right of the median.

  3. The "first quartile" is the median of all $x_i$ for which $i \le l$. The "third quartile" is the median of $(x_i)$ for which $i \ge u$.

Here is an implementation. It can help you do your exercises in this textbook.

quart <- function(x) {
  x <- sort(x)
  n <- length(x)
  m <- (n+1)/2
  if (floor(m) != m) {
    l <- m-1/2; u <- m+1/2
  } else {
    l <- m-1; u <- m+1
  }
  c(Q1=median(x[1:l]), Q3=median(x[u:n]))
}

For instance, the output of quart(c(6,7,8,9,10,15,16,16,20,20,23,33,50,58,104)) agrees with the text:

Q1 Q3 
 9 33 

Let's compute quartiles for some small datasets using all ten methods: the nine in R and the textbook's:

y <- matrix(NA, 2, 10)
rownames(y) <- c("Q1", "Q3")
colnames(y) <- c(1:9, "Quart")
for (n in 3:5) {
  j <- 1
  for (i in 1:9) {
    y[, i] <- quantile(1:n, probs=c(1/4, 3/4), type=i)
  }
  y[, 10] <- quart(1:n)
  cat("\n", n, ":\n")
  print(y, digits=2)
}

When you run this and check, you will find that the textbook values do not agree with any of the R output for all three sample sizes. (The pattern of disagreements continues in cycles of period three, showing that the problem persists no matter how large the sample may be.)

The textbook might have misconstrued John Tukey's method of computing "hinges" (aka "fourths"). The difference is that when splitting the dataset around the median, he includes the median in both halves. That would produce $9.5$ and $28$ for the example dataset.

$\endgroup$
  • 3
    $\begingroup$ A big thanks for such a detailed answer along with providing me the tools to do work on my own and evaluate the different methods. I'm going to fire them up now and go through things in more detail. $\endgroup$ – user60305 Jan 20 '15 at 17:52
2
$\begingroup$

Within the field of statistics (which I teach, but in which I am not a researcher), quartile calculations are particularly ambiguous (in a way that is not necessarily true of quantiles, more generally). This has a lot of history behind it, in part because of the use (and perhaps abuse) of inter-quartile range (IQR), which is insensitive to outliers, as a check or alternative to standard deviation. It remains an open contest, with three distinctive methods for computing Q1 and Q3 being co-canonical.

As is often the case, the Wikipedia article has a reasonable summary: https://en.m.wikipedia.org/wiki/Quartile The Larson and Farber text, like most elementary statistics texts, uses what's described in the Wikipedia article as "Method 1." If I follow the descriptions above, r uses "Method 3". You'll have to decide for yourself which is canonically appropriate in your own field.

$\endgroup$
  • $\begingroup$ You make good points (+1). But given that the references for "Method 1" are the TI-83 calculator and Excel (whose lack of credibility is well known), that this method is demonstrably biased, and that it is no more difficult to compute than Tukey's hinges, it would seem difficult to justify or recommend its use. $\endgroup$ – whuber Jul 26 '17 at 16:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy