5
$\begingroup$

Imagine we have a partial data and we know that this partial data represent only the left 5% of the log-normal distribution, which the overall data follow. How can we calculate the mean-log and sd-log of the overall log-normal distribution having this partial data? The answer in R is preferable.

Improve this question
$\endgroup$
4
  • 5
    $\begingroup$ There are some standard ways to do this, including maximum likelihood and regression on order statistics. But if by "left" you mean the lower five percent, then you should be concerned about the potential for enormous standard errors. Do you have a very large dataset? How confident are you that the upper 95% truly is lognormally distributed? $\endgroup$
    – whuber ♦
    Jun 17, 2015 at 14:14
  • 1
    $\begingroup$ My specific example is about tasks execution durations (which do follow log-normal distribution with p close to 0.8) on the micro-task crowdsourcing platform (CrowdFlower). As all tasks start at the same time it is obvious that I receive the results for the fast tasks first. My goal is to predict the 95% percentile duration time, having only data describing the lower k % (where k is 5 or 10 or 20) of the distribution. I can not call my dataset very large - the overall dataset is about 500 items only. $\endgroup$
    – Pavel Kucherbaev
    Jun 17, 2015 at 21:20
  • $\begingroup$ @whuber, is it possible for you to introduce an example of code to use maximum likelihood and regression on order statistics in this specific usecase? $\endgroup$
    – Pavel Kucherbaev
    Jun 18, 2015 at 7:46
  • 1
    $\begingroup$ See my answer at stats.stackexchange.com/questions/130156. It includes R code illustrating both ML and ROS. The R package NADA includes ROS capabilities. The free USEPA software ProUCL incorporates ROS. (It's so cumbersome that it's of little use for extensive data, but it could be valuable for double-checking of calculations.) $\endgroup$
    – whuber ♦
    Jun 18, 2015 at 11:16

0

Reset to default