5
$\begingroup$

I have been presented a problem of this kind: suppose I know the values of k quantiles for a continuous random variable $X$

$$X_{1\%} = x_1, X_{5\%} = x_2, \dots , X_{99\%} = x_{k}$$

so that

$$ F_X(x_1)=1\%, F_X(x_2)=5\%, \dots, F_X(x_k)=99\% $$

From these informations I want to draw the chart of the PDF.

I thought that I could proceed this way:

  • interpolate $F_X(x)$ to get a smooth CDF (for instance spline interpolation)
  • find the derivative (numerical) of the smoothed CDF at some points to obtain the PDF.

Are there other more direct methods to address this problem? Do you think my solution is solid?

Thank you.

Improve this question
$\endgroup$
11
  • 2
    $\begingroup$ If you can assume a parametric distribution, you can also fit the CDF function to the quantiles using black-box optimizer to find relevant parameters. $\endgroup$
    – Tim ♦
    May 24, 2018 at 10:02
  • $\begingroup$ Thanks for the comment ... In my particular case I don't want to assume any parametric distribution. But just in case how doese the optimization work for example if I have more conditions than parameters to be estimated? For instance if I have 3 quantiles and I want to use optimizer to find $\mu$ and $\sigma$ of a normal distribution. $\endgroup$
    – gioxc88
    May 24, 2018 at 10:15
  • 1
    $\begingroup$ Say you have $q_p = x$, then you seek such parameters $\theta$ to minimize the difference between $F_\theta(x)$ and $p$ as measured by some loss, using a black-box optimizer. $\endgroup$
    – Tim ♦
    May 24, 2018 at 10:30
  • $\begingroup$ How many quantiles do you have? Are the true values known, or is there some kind of sampling or noise involved? $\endgroup$
    – user20160
    May 28, 2018 at 8:57
  • $\begingroup$ I have 7 quantiles, but regardless, I would like to have an opinion about the procedure I described. Does it make sense? because I tried cubic spline interpolation I it does not work well $\endgroup$
    – gioxc88
    May 28, 2018 at 9:23

1 Answer 1

Reset to default
6
+50
$\begingroup$

Your approach is valid, if you use the integral of a cubic B-spline to interpolate the quantiles with the condition that all B-spline coefficients are nonnegative. https://en.m.wikipedia.org/wiki/B-spline This preserves the monotonicty of the quantiles in the interpolating function, while normal cubic splines do not.

There is no need then to numerically differentiate the spline, you get the PDF from the normal evaluation of the B-spline. The integral of the B-spline curve is also available without numerical computation. You get it from any serious library that implements splines, for example, in Python's scipy module.

The remaining question is how to find the B-spline coefficients in this case. I don't have perfect answer right now, although an efficient algorithm should be possible. A simple brute-force way would be to minimize the squared residuals between the integrated spline and the quantiles with a numerical minimizer, while constraining the coefficients to be nonnegative.

I can add a numerical example in a few days if desired.

Improve this answer
$\endgroup$
2
  • $\begingroup$ thank you very much for your answer. I use Matlab and my problem was just the one you mentioned of the negative coefficient. I will expand my original question in order to show you my results. Thanks for the time you spent looking into it!! $\endgroup$
    – gioxc88
    May 31, 2018 at 7:40
  • 1
    $\begingroup$ Cool, I am glad I could help! Sorry for not providing a numerical example right away, but I am traveling and only have my phone at hand. $\endgroup$
    – olq_plo
    May 31, 2018 at 23:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.