Efficient estimation of conditional means from pdf, CDF, & quantile function supplied numerically

Question

Suppose I have a a probability distribution that I know to have a continuous differentiable unimodal pdf, with pdf(x) strictly greater than zero for all x in the positive half-plane. In addition, I have a black box that provides me with the CDF, pdf and quantile value for any given x.

However, I do not want those things. Instead I want the functions which are to the conditional means (or conditional totals) as the CDF and the survival function are to the conditional probability, i.e. the two functions that that provide the the mean (or total) values conditional on x being less than or greater than, respectively, some constant c. In other word, I am looking two functions of c, given the distribution.

Is there an efficient method, ideally analytic but probably numeric, to estimate these conditional mean values from what I can extract from my black box? You may assume the distribution has a finite mean, though it may be heavy-tailed.

If we are looking at a numeric approach, please assume that I can program, in R or a little in C++, but that I don't have any real training in numerical methods.

Also, I would like to know if these two conditional mean functions described above have commonly-used names in statistics, economics, or elsewhere. I need to do a bunch of things with them, and I am looking for a term or terms I can use to locate a literature.

Answer 1

I’m going to take a stab at answering this one myself. Suppose bb(X; Θ) is your black box function for the pdf with parameter vector Θ, and [lb, ub] are your lower and upper bounds. Then the integral of X*bb(X) from lb to ub is the proportion of the integral over the entire range that lies between lb and ub. If I want the mean given that x falls within that range, I need to treat it as a truncated distribution and increase that value by a factor of the inverse of the probability that X will fall within that range, which is the integral of the pdf over that range (but not, as before, multiplied by X).

Here is R code to do this. Note, first, that R’s integrate( ) function accepts -Inf and Inf as range values (though there can be convergence problems); and second, that it returns a list of which only the first element is the value of the integral.

This function returns the unconditional mean with the default parameters. If the pdf is only defined for positive values, the -INF should be replaced by zero:

cr_moment <- function(lb = -Inf, ub = Inf, dfun, params, v=1, ...){
  x_pdf         <- function(X){
    X^v * do.call(what=dfun, args=c(list(x=X), params))
    }
  prob_interval <- function(X){
    do.call(what=dfun, args=c(list(x=X), params))
    }
  integral_val  <- integrate(f = x_pdf, lower = lb, upper = ub)
  integral_prob <- integrate(f = prob_interval, lower = lb, upper = ub)
  crm <-  interval_val[[1]] / interval_prob[[1]]
  out <- list(value = integral_val[[1]], probability = integral_prob[[1]],  
              cond_moment = crm)
  out
}

This function specializes the function above to the black box distribution (or the pdf function of whatever distribution you like, in place of the bb). (“cr” for conditional on the range):

cr_bb <- function(lb = -Inf, ub = Inf, v = 1, params = Θ){
  cr_moment(lb, ub, dfun = bb, params = get("params"))
}

Then the actual functions I originally requested would be:

upper_tail_mean <- function(c){cr_bb(lb = c)}

and

lower_tail_mean <- function(c){cr_bb(ub = c)} 

Kudos to Mikko Marttila on stackoverflow for assistance in constructing this function.