How to obtain a confidence interval for a percentile? I have a bunch of raw data values that are dollar amounts and I want to find a confidence interval for a percentile of that data.  Is there a formula for such a confidence interval?
 A: Derivation
The $\tau$-quantile $q_\tau$ (this is the more general concept than percentile) of a random variable $X$ is given by $F_X^{-1}(\tau)$.  The sample counterpart can be written as $\hat{q}_\tau = \hat{F}^{-1}(\tau)$ -- this is just the sample quantile.  We are interested in the distribution of:
$\sqrt{n}(\hat{q}_\tau - q_\tau)$
First, we need the asymptotic distribution of the empirical cdf.
Since $\hat{F}(x) = \frac{1}{n} \sum 1\{X_i < x\}$, you can use the central limit theorem.  $1\{X_i < x\}$ is a bernoulli random variable, so the mean is $P(X_i < x) = F(x)$ and the variance is $F(x)(1-F(x))$.
$\sqrt{n}(\hat{F}(x) - F(x)) \rightarrow N(0, F(x)(1-F(x))) \qquad (1)$
Now, because inverse is a continuous function, we can use the delta method.
[**The delta method says that if $\sqrt{n}(\overline{y} - \mu_y) \rightarrow N(0,\sigma^2)$, and $g(\cdot)$ is a continuous function, then $\sqrt{n}(g(\overline{y}) - g(\mu_y)) \rightarrow N(0, \sigma^2 (g'(\mu_y))^2)$  **]
In the left hand side of (1), take $x=q_\tau$, and $g(\cdot) = F^{-1}(\cdot)$
$\sqrt{n}(F^{-1}(\hat{F}(q_\tau)) - F^{-1}(F(q_\tau))) = \sqrt{n}(\hat{q}_\tau - q_\tau)$
[** note that there is a bit of a slight of hand in the last step because $F^{-1}(\hat{F}(q_\tau)) \neq \hat{F}^{-1}(\hat{F}(q_\tau)) = \hat{q}_\tau$, but they are the asymptotically equal if tedious to show **]
Now, apply the delta method mentioned above.
Since $\frac{\textrm{d}}{\textrm{d}x} F^{-1}(x) = \frac{1}{f(F^{-1}(x))}$ (inverse function theorem)
$\sqrt{n}(\hat{q}_\tau - q_\tau) \rightarrow N\left(0, \frac{F(q_\tau)(1-F(q_\tau))}{f(F^{-1}(F(q_\tau)))^2}\right) = N\left(0, \frac{F(q_\tau)(1-F(q_\tau))}{f(q_\tau)^2}\right)$
Then, to construct the confidence interval, we need to calculate the standard error by plugging in sample counterparts of each of the terms in the variance above:
Result
So $se(\hat{q}_\tau) = \sqrt{\frac{\hat{F}(\hat{q}_\tau)(1-\hat{F}(\hat{q}_\tau))}{n \hat{f}(\hat{q}_\tau)^2}} =$ $\sqrt{\frac{\tau (1 - \tau)}{n \hat{f}(\hat{q}_\tau)^2}}$
And $CI_{0.95}(\hat{q}_\tau) = \hat{q}_\tau \pm 1.96 se(\hat{q}_\tau)$
This will require you to estimate the density of $X$, but this should be pretty straightforward.  Alternatively, you could bootstrap the CI pretty easily too.
A: A brute-force computing intensive solution is to use the bootstrap resampling method. The following function returns the bootstrap confidence intervals of a quantile.
quantile.CI.via.bootstrap <- function(x, p, alpha = 0.1) {
    ## Purpose:
    ##   Calculate a two-sided confidence interval with confidence level of (1 - alpha) for
    ##   a quantile, based on the (computing intensive) bootstrap resampling method. 
    ##
    ## Arguments:
    ##   - x: a vector of values, representing a data sample. 
    ##   - p: probability cutpoint for the quantile (between 0 and 1).
    ##   - alpha: type I error level (default to 0.1 so a 90% CI is calculated)
    ##
    ## Return:
    ##   - CI: the lower and upper limits of the two-sided CI. 

    q <- quantile(x, probs = p)         
    message("Quantile Point Estimate = ", q, " (Probability Cutpoint = ", p, ")\n")

    ## Bootstrap resampling with 2000 replications
    library(boot)
    set.seed(1)
    b <- boot(x, function(x, i) quantile(x[i], probs = p), R = 2000)

    boot.ci(b, conf = 1 - alpha, type = c("norm", "basic", "perc", "bca"))

}
if (F) {                                # Unit Test
    x <- 1:100
    p <- 0.9
    alpha <- 0.05
    quantile.CI.via.bootstrap(x, p, alpha)
    ## Intervals : 
    ## Level      Normal              Basic         
    ## 95%   (84.50, 96.34 )   (85.10, 97.10 )  

    ## Level     Percentile            BCa          
    ## 95%   (83.1, 95.1 )   (83.3, 95.1 )  
}

You may note the "Basic" bootstrap method returns an interval [85, 97] that aligns with the analytical method (binomial distribution) in the previous post.
