Suppose I have a a probability distribution that I know to have a continuous differentiable unimodal pdf, with pdf(x) strictly greater than zero for all x in the positive half-plane. In addition, I have a black box that provides me with the CDF, pdf and quantile value for any given x.

However, I do not want those things. Instead I want the functions which are to the conditional means (or conditional totals) as the CDF and the survival function are to the conditional probability, i.e. the two functions that that provide the the mean (or total) values conditional on x being less than or greater than, respectively, some constant c. In other word, I am looking two functions of c, given the distribution.

Is there an efficient method, ideally analytic but probably numeric, to estimate these conditional mean values from what I can extract from my black box? You may assume the distribution has a finite mean, though it may be heavy-tailed.

If we are looking at a numeric approach, please assume that I can program, in R or a little in C++, but that I don't have any real training in numerical methods.

Also, I would like to know if these two conditional mean functions described above have commonly-used names in statistics, economics, or elsewhere. I need to do a bunch of things with them, and I am looking for a term or terms I can use to locate a literature.

  • 1
    $\begingroup$ There is much about this post that makes it too vague to answer. Your introductory paragraph is puzzling: since the PDF is defined "in the positive half-plane," evidently your distribution is bivariate. What, then, could you mean by "quantile value"? Are you attempting to describe the conditional distributions given the first coordinate? If so, your "black box" sounds like it is no different than the conditional distribution function, which means you have full information about the joint distribution. What exactly does it mean, then, to "extract from my black box"? Could you edit to clarify? $\endgroup$
    – whuber
    Mar 13 '18 at 22:26

I’m going to take a stab at answering this one myself. Suppose bb(X; Θ) is your black box function for the pdf with parameter vector Θ, and [lb, ub] are your lower and upper bounds. Then the integral of X*bb(X) from lb to ub is the proportion of the integral over the entire range that lies between lb and ub. If I want the mean given that x falls within that range, I need to treat it as a truncated distribution and increase that value by a factor of the inverse of the probability that X will fall within that range, which is the integral of the pdf over that range (but not, as before, multiplied by X).

Here is R code to do this. Note, first, that R’s integrate( ) function accepts -Inf and Inf as range values (though there can be convergence problems); and second, that it returns a list of which only the first element is the value of the integral.

This function returns the unconditional mean with the default parameters. If the pdf is only defined for positive values, the -INF should be replaced by zero:

cr_moment <- function(lb = -Inf, ub = Inf, dfun, params, v=1, ...){
  x_pdf         <- function(X){
    X^v * do.call(what=dfun, args=c(list(x=X), params))
  prob_interval <- function(X){
    do.call(what=dfun, args=c(list(x=X), params))
  integral_val  <- integrate(f = x_pdf, lower = lb, upper = ub)
  integral_prob <- integrate(f = prob_interval, lower = lb, upper = ub)
  crm <-  interval_val[[1]] / interval_prob[[1]]
  out <- list(value = integral_val[[1]], probability = integral_prob[[1]],  
              cond_moment = crm)

This function specializes the function above to the black box distribution (or the pdf function of whatever distribution you like, in place of the bb). (“cr” for conditional on the range):

cr_bb <- function(lb = -Inf, ub = Inf, v = 1, params = Θ){
  cr_moment(lb, ub, dfun = bb, params = get("params"))

Then the actual functions I originally requested would be:

upper_tail_mean <- function(c){cr_bb(lb = c)}


lower_tail_mean <- function(c){cr_bb(ub = c)} 

Kudos to Mikko Marttila on stackoverflow for assistance in constructing this function.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.