# Method of moments and MLE

Suppose a random variable follows the logistic distribution, $$X ∼ Logistic (\mu, \sigma)$$ and we restrict our attention to random samples drawn from this random variable $$X$$.

What would be the MoM and MLE? Is it possible to obtain a closed-form solution?

• It seems unlikely (the regular logistic regression does not lead to closed-form solutions). Dec 15, 2020 at 13:21

If we denote your random samples as $$x_i$$, then the MLE for $$\mu, \sigma$$ will be the solutions to the following system of equations:

$$\sum_{i=1}^n \frac{1}{1+e^{z_i}} = \frac{n}{2}$$

$$\sum_{i=1}^n z_i \cdot \frac{1-e^{z_i}}{1+e^{z_i}} = n$$

where

$$z_i = \frac{x_i - \hat{\mu}_{mle}}{\hat{\sigma}_{mle}}$$

There does not appear to be a closed-form solution. To see where the equations above come from, notice that the likelihood for a sample $$x_1, \ldots, x_n$$ can be written as $$\mathcal{L}(x_1, \ldots, x_n) = \prod_{i=1}^n \frac{e^{-(x_i-\mu)/\sigma}}{\sigma(1+e^{-(x_i-\mu)/\sigma})^2},$$ which means that the log-likelihood can be written as $$l(x_1, \ldots, x_n) = \sum_{i=1}^n \left(-\frac{x_i-\mu}{\sigma} - \log \sigma - 2\log(1+e^{-(x_i-\mu)/\sigma})\right).$$ We take the derivative with respect to $$\mu$$ and set it equal to 0, which results in:

$$\frac{\partial l}{\partial \mu} = \sum_{i=1}^n \left(\frac{1}{\sigma} - 2\frac{\frac{1}{\sigma}e^{-(x_i-\mu)/\sigma}}{1+e^{-(x_i-\mu)/\sigma}}\right) = 0$$

We can multiply both sides by $$\sigma$$ and re-arrange to obtain $$\sum_{i=1}^n 1 = 2\sum_{i=1}^n \frac{e^{-(x_i-\mu)/\sigma}}{1+e^{-(x_i-\mu)/\sigma}}.$$

Now notice that $$\sum_{i=1}^n 1 = n$$, and then we can divide both sides by $$2$$. Finally, we can rewrite $$\frac{e^{-(x_i-\mu)/\sigma}}{1+e^{-(x_i-\mu)/\sigma}} = \frac{1}{\frac{1}{e^{-(x_i-\mu)/\sigma}} + \frac{e^{-(x_i-\mu)/\sigma}}{e^{-(x_i-\mu)/\sigma}}} = \frac{1}{e^{(x_i-\mu)/\sigma} + 1},$$ and if we define $$z_i$$ as above, then we obtain the first equation from that system of equations.

To obtain the second equation, we again start from the log-likelihood, but now differentiate with respect to $$\sigma$$, and set the result equal to 0. This results in $$\frac{\partial l}{\partial \sigma} = \sum_{i=1}^n \left(\frac{x_i-\mu}{\sigma^2} - \frac{1}{\sigma} - 2\frac{\frac{x_i-\mu}{\sigma^2}e^{-(x_i-\mu)/\sigma}}{1+e^{-(x_i-\mu)/\sigma}}\right) = 0.$$ Now we multiply both sides by $$\sigma$$ and re-arrange, which results in $$\sum_{i=1}^n \left(\frac{x_i-\mu}{\sigma} - 2\frac{\frac{x_i-\mu}{\sigma}e^{-(x_i-\mu)/\sigma}}{1+e^{-(x_i-\mu)/\sigma}}\right) = \sum_{i=1}^n 1.$$ Making the same simplifications as before, this becomes $$\sum_{i=1}^n \left(z_i - 2 \frac{z_i}{1+e^{z_i}}\right) = n,$$ and finally, we simplify the left side by noticing that $$z_i - 2\frac{z_i}{1+e^{z_i}} = z_i\left(1 - \frac{2}{1+e^{z_i}}\right) = z_i \cdot \frac{1+e^{z_i}-2}{1+e^{z_i}} = z_i\frac{e^{z_i}-1}{e^{z_i}+1},$$ which we can plug in to match the form of the second equation in the system.

The MOM estimators, however, are simpler. Since the mean of a logistic distribution is $$\mu$$, we can equate $$\hat{\mu}_{mom} = \frac{1}{n}\sum_{i=1}^n x_i$$ and since the standard deviation is $$\frac{\sigma \pi}{\sqrt{3}},$$ we can write $$\hat{\sigma}_{mom} = \frac{\sqrt{3}}{\pi} s,$$ where $$s$$ denotes the sample standard deviation.

See here, which also lists references for these results.

• I already did this following way but I wasn't sure of my results. Thank you and have a good day :) Dec 15, 2020 at 16:29
• could you please help me identify why the MLE looks like that? Usually, it's based on its log-likelihood but this one is kind of confusing me. Thanks in advance Dec 15, 2020 at 17:59
• This result is based on the log-likelihood of the samples x_1 through x_n, but it just doesn't have nice solutions. You can get the two equations by first writing down the derivative with respect to mu, and setting it equal to 0, and then writing down the derivative with respect to sigma, and then setting that equal to 0. The forms presented here are just from re-arranging those results, and then just defining z_i to make the equations a little nicer. Let me know if that makes sense -- also happy to write it out in more detail if that would be helpful!
– Izzy
Dec 15, 2020 at 18:12
• If you could write it, I'd be so thankful. I've looked everywhere but it's only making me even more confused. Should there be some conditions for deriving it or something? Dec 15, 2020 at 18:22
• I edited my original comment to show all the work! Let me know if any steps are unclear! It's the usual process for getting MLE estimates, but there is just a bit of algebraic manipulation involved. In a simpler case, you could then solve this system of equations to get the solutions, but here, you wouldn't be able to solve and get closed-form solutions, so we typically would just leave it in this form.
– Izzy
Dec 15, 2020 at 20:18