# How to estimate log-normal distribution parameters from a set of data?

I don't have any background in statistics, so maybe I may say things that are incorrect.

I have to model the production of waste deposition at each container of a set of containers and I have access to the values of the fill-up rates for certain intervals of time. I need to generate future values based on the historical past values.

First I thought that a Normal Distribution could be adopted, but since there's no negative waste deposition rates I thought on using the Log-Normal Distribution as it starts at the zero value.

Imagine that I have the following ten values: 5; 0; 9; 2; 6; 4; 1; 0; 5; 3

My question is: How can I obtain the parameters μ and σ that are needed to model the probability distribution?

I see that an approach is to use:

But then, since I have values of xk that are 0, I have ln (0) which is a problem that I don't know how to work around.

From the wikipedia page I see also that when the individual values x1,x2,...,xn are not available I can calculate the parameters using the following formulas:

So, from my sample, $$\bar{x}$$ = 3.5 and $$\hat{\sigma}$$ = 2.8771. So, can I say that μ and σ are equal to 0.9946 and to 0.7185, respectively?

If yes, then what bugs me the most is that I have the individual values but some of them happen to be 0. How could it be that I could use those first formulas only if the values are different than 0?

Any help would be great! Thank you in advance!

• The support of the lognormal distribution is $(0, +\infty)$. The fact that you have values of $0$ thus immediately rules out the lognormal distribution as a suitable model for these data. What exactly do these fill-up rates mean (e.g. what does a value of $5$ mean)? Nov 29, 2019 at 13:54
• A value of 5 means that for a specific day a container was filled up by 5%. A value of 0 represents that no waste was deposited during that day. If there are 0 values but no negative ones, what suitable models could be used to characterize this? Nov 29, 2019 at 14:17
• So it's always a percentage between 0% and 100%? Am I understanding this correctly? Nov 29, 2019 at 14:24
• Yes. It could get overfilled, but let's neglect that Nov 29, 2019 at 14:31
• @COOLSerdash given we know that there are exact 0's, the beta itself would be unsuitable for the same reason the lognormal is. A 0- & 1- inflated beta might be a better choice. Nov 30, 2019 at 3:27

Because your data appear to be rounded to the nearest integer, "0" really means some value in the interval $$[0, 1/2]$$ and "$$k$$" for $$k\gt 0$$ means some value in the interval $$[k-1/2, k+1/2].$$ Let's denote that interval as $$[k^-,k^+],$$ even for $$k=0.$$

Writing $$\Phi$$ for the Normal distribution, the likelihood that the parameters of $$\log X$$ are $$(\mu, \sigma)$$ for data $$x_1, x_2, \ldots, x_n$$ therefore is

$$\mathcal L = \prod_{i=1}^n \left(\Phi(\log x_i^+;\mu,\sigma) - \Phi(\log x_i^-;\mu,\sigma)\right).$$

Here, when $$x_i=0$$ and (therefore) $$x_i^-=0,$$ take $$\log 0 = -\infty$$ for the sake of computing. Since $$\Phi(-\infty;\mu, \sigma) = \lim_{x\to-\infty}\Phi(x;\mu,\sigma)= 0$$ there is no problem.

This is readily maximized numerically to give a Maximum Likelihood estimate for the parameters. For the data in the question, the estimates are $$\hat\mu = 0.7652$$ and $$\hat\sigma = 1.2155.$$ These are close to crude estimates $$\hat\mu = 0.7613,$$ $$\hat\sigma = 0.1.2856$$ obtained by using the midpoints of the intervals: that is, replacing $$[0,1/2]$$ by $$1/4$$ and replacing $$[k-1/2, k+1/2]$$ by $$k$$ throughout and estimating $$\mu$$ as the mean logarithm and $$\sigma$$ as the standard deviation of the logarithms. But there is a difference in the estimates of $$\sigma,$$ suggesting something might be gained by the more careful calculation.

(In response to a long-standing answer in this thread, replacing the zeros by $$0.00001$$ gives $$\hat\sigma = 5.4,$$ which is grossly wrong. The reason is predictable: the logarithms of $$0.00001$$ are so far from the data that they exaggerate the value of $$\sigma.$$)

The following R code does the work. It minimizes $$-\log \mathcal L$$ as a function of $$\mu$$ and $$\log \sigma$$ (thereby avoiding the usual numerical problems and requiring no constraints).

With the estimated parameters in hand, you can generate additional random values easily: use them to generate Normally distributed values and then exponentiate and round them.

#
# Negative log likelihood of theta = (mu, log sigma).
#
lambda <- function(theta) {
mu <- theta[1]
sigma <- exp(theta[2])
-sum(log(pnorm(log(x + 1/2), mu, sigma) - pnorm(log(pmax(0, x - 1/2)), mu, sigma)))
}
#
# The data (nonnegative integers).
#
x <- c(5,0,9,2,6,4,1,0,5,3)
#
# Use the midpoints for an initial estimate.
#
y <- log(pmax(1/4, x))
theta <- c(mean(y), log(sd(y)))
#
# Maximize the likelihood.
#
b <- nlm(lambda, theta)
#
# Extract the estimates.
#
m <- b$estimate[1] s <- exp(b$estimate[2])


Pad it with an arbitrary small number, e.g. 0.00001; basically your minimum precision. It will yield a highly negative value of the logarithm, but that's fine.

Assuming the production was continuous in time, you can never actually measure a point where it's exactly zero in reality anyway, it's asymptotic too, so it's not entirely unprincipled.

• The "yield a highly negative value" looks like it's going to bias any estimation procedure strongly, so please explain why you think this is "fine."
– whuber
Nov 29, 2019 at 17:32
• +1 or +0.1 or +0.01 turns out to be very arbitrary and very hard to generalize in predictive modeling in my experience. Jun 22, 2023 at 15:17