calculate p-values based on an arbitrary distribution

Question

I have bootstrapped a distribution of values that is non-normal. Now I would like to calculate the p-value of a parameter mu against that distribution (i.e. how likely, given the bootstrapped distribution, is it to receive mu). Using example data from

require(SuppDists)

## make a weird dist with Kurtosis and Skew
a <- rnorm( 5000, 0, 2 )
b <- rnorm( 1000, -2, 4 )
c <- rnorm( 3000,  4, 4 )
babyGotKurtosis <- c( a, b, c )

This threat gives the intution to find the probability density function numerically. For a normal, they propose:

mu = 1.64

dF <- function(x)dnorm(x)
pF <- function(q)integrate(dF,-Inf,q)$value 

> pF(mu)
[1] 0.9494974
> pnorm(mu)
[1] 0.9494974

Just how to apply this approach to a simulated non-normal distribution?

Answer 1

You seem to be looking for the empirical cumulative distribution function, which simply returns which proportion of a sample is less than a given cutoff. In R, use ecdf():

> xx <- rnorm(100)
> ecdf(xx)(2)
[1] 0.99

To get a p-value, you will need to do some judicious subtraction from one, taking absolute values and/or division by or multiplication by two, depending on whether you are looking for a one-sided or a two-sided test.