In this question the answerer goes into what the R quantile() function generally does, but I don't really feel satisfied that I understand the different possible modes of the function.

I've been using quantile() to find the nth percentile of a set of data, as it turns out it was suggested in the top answer here, but now that I'm looking into it further, I don't know how reliable that solution is.

I'm using the default type - type 7 - now, but the documentation for the function states that type 8 is the "recommended" type. Recommended for what? What is it finding, and how is that superior? Is it better for finding the percentiles?

I generally do not understand the difference between what exactly the different quantile() modes are finding. To make things even more confusing, my coworker ran a test of the two types and the percentiles produced by type 8 were not bijective, which would be impossible -- but I haven't been able to replicate his results, so I don't know how type 8 could or could not produce non-bijective percentiles.

Any explanation would be appreciated.

  • 3
    $\begingroup$ The documentation provides a reference. Read it! $\endgroup$
    – Roland
    May 20, 2016 at 7:04

1 Answer 1


Hyndman and Fan (1996) (also here) described 9 algorithms to calculate a quantile.

Sample quantiles of type i are defined by Q[i](p) = (1 - gamma) x[j] + gamma x[j+1]

where i in [1,9], p in [(j-m)/n,(j-m+1)/ n] , x[j] is the j-th order statistic, n is the sample size, and m is a constant determined by the sample quantile type. Here gamma depends on the fractional part of g = np+m-j.

For the continuous sample quantile types (4 through 9), the sample quantiles can be obtained by linear interpolation between the k-th order statistic and p(k):

p(k) = (k - alpha) / (n - alpha - beta + 1)

where alpha and beta are constants determined by the type. Further, m = alpha + p(1 - alpha - beta), and gamma = g.

About the two types you are using (7 and 8, the default):

  • Type 7 (used by S)

p(k) = (k - 1) / (n - 1) In this case, p(k) = mode[F(x[k])].

  • Type 8

p(k) = (k - 1/3) / (n + 1/3)

Then p(k) =~ median[F(x[k])]. The resulting quantile estimates are approximately median-unbiased regardless of the distribution of 'x'.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.