2
$\begingroup$

My sample size is around 100k - should be enough for CLT to work.

One intuitive approach is to estimate population mean by sample mean y.

However, after plotting the sample data, I notice the distribution is definitively a delta-lognormal distribution, with P(Y>0)=99% and X={lnY|Y>0} being normally distributed with parameters u and sigma.

Please note that Y=0 really means zero value. ( not out of detection)

Given Y conforms to a delta-lognormal distribution, E(Y)=P(Y>0)exp(u+0.5sigma^2).

Here is the problem: two approaches give different results - completely different. Is there anything missing in the calculation?

$\endgroup$

1 Answer 1

1
$\begingroup$

If you have $n=100000$ sample values then the two methods should give you estimated values that are very close to one another. Since you haven't supplied your calculations I cannot say what went wrong, but here is the correct way to do it.

#Generate lognormal data
set.seed(1)
n    <- 10^6
DATA <- rlnorm(n)

#Compute and print the sample mean
mean(DATA)
[1] 1.64861

#Compute the MLE of the distribution mean 
MLE <- fitdistrplus::mledist(DATA, distr = 'lnorm', lower = 0)
MLE.mu    <- MLE$estimate[1]
MLE.sigma <- MLE$estimate[2]
MLE.mean  <- unname(exp(MLE.mu + 0.5*MLE.sigma^2))
MLE.mean
[1] 1.649103

As you can see, these two estimates of the mean of the distribution are quite close to each other. (The true mean is $\exp(0.5) = 1.648721$ in this case.) As a general rule, if you have the choice of a non-parametric estimator like the sample mean, versus a parametric estimator like the MLE, with lots of data, you should prefer the non-parametric estimator, since it is robust to violations of your assumptions. In the present case this means it is generally preferable to use the sample mean, since this estimator is robust to misspecification of the distribution.

$\endgroup$
3
  • 1
    $\begingroup$ I ran a simulation similar to yours, and the results matched from two methods. I guess the issue goes back to misspecification of distribution- even though from graph the logarithm of Y looks like a perfect normal distribution. Probably I should go with sample mean. $\endgroup$
    – Lovnlust
    Commented Dec 17, 2021 at 10:12
  • $\begingroup$ Now my sample size is large enough to apply CLT. What if the sample size is small? Is there any prevalence of parametric estimator over non-parametric one? $\endgroup$
    – Lovnlust
    Commented Dec 17, 2021 at 10:14
  • $\begingroup$ As a general rule, in small samples a (good) parametric estimator will usually be more accurate than a non-parametric estimator when the model assumptions for the former are correct. This is because the parametric estimator incorporates assumed distributional information. $\endgroup$
    – Ben
    Commented Dec 17, 2021 at 21:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.