Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Suppose a random variable follows the logistic distribution, $X ∼ Logistic (\mu, \sigma)$ and we restrict our attention to random samples drawn from this random variable $X$.
What would be the MoM and MLE? Is it possible to obtain a closed-form solution?
If we denote your random samples as $x_i$, then the MLE for $\mu, \sigma$ will be the solutions to the following system of equations:
$$\sum_{i=1}^n \frac{1}{1+e^{z_i}} = \frac{n}{2}$$
$$\sum_{i=1}^n z_i \cdot \frac{1-e^{z_i}}{1+e^{z_i}} = n$$
where
$$z_i = \frac{x_i - \hat{\mu}_{mle}}{\hat{\sigma}_{mle}}$$
There does not appear to be a closed-form solution. To see where the equations above come from, notice that the likelihood for a sample $x_1, \ldots, x_n$ can be written as $$\mathcal{L}(x_1, \ldots, x_n) = \prod_{i=1}^n \frac{e^{-(x_i-\mu)/\sigma}}{\sigma(1+e^{-(x_i-\mu)/\sigma})^2},$$ which means that the log-likelihood can be written as $$l(x_1, \ldots, x_n) = \sum_{i=1}^n \left(-\frac{x_i-\mu}{\sigma} - \log \sigma - 2\log(1+e^{-(x_i-\mu)/\sigma})\right).$$ We take the derivative with respect to $\mu$ and set it equal to 0, which results in:
$$\frac{\partial l}{\partial \mu} = \sum_{i=1}^n \left(\frac{1}{\sigma} - 2\frac{\frac{1}{\sigma}e^{-(x_i-\mu)/\sigma}}{1+e^{-(x_i-\mu)/\sigma}}\right) = 0$$
We can multiply both sides by $\sigma$ and re-arrange to obtain $$\sum_{i=1}^n 1 = 2\sum_{i=1}^n \frac{e^{-(x_i-\mu)/\sigma}}{1+e^{-(x_i-\mu)/\sigma}}.$$
Now notice that $\sum_{i=1}^n 1 = n$, and then we can divide both sides by $2$. Finally, we can rewrite $$\frac{e^{-(x_i-\mu)/\sigma}}{1+e^{-(x_i-\mu)/\sigma}} = \frac{1}{\frac{1}{e^{-(x_i-\mu)/\sigma}} + \frac{e^{-(x_i-\mu)/\sigma}}{e^{-(x_i-\mu)/\sigma}}} = \frac{1}{e^{(x_i-\mu)/\sigma} + 1},$$ and if we define $z_i$ as above, then we obtain the first equation from that system of equations.
To obtain the second equation, we again start from the log-likelihood, but now differentiate with respect to $\sigma$, and set the result equal to 0. This results in $$\frac{\partial l}{\partial \sigma} = \sum_{i=1}^n \left(\frac{x_i-\mu}{\sigma^2} - \frac{1}{\sigma} - 2\frac{\frac{x_i-\mu}{\sigma^2}e^{-(x_i-\mu)/\sigma}}{1+e^{-(x_i-\mu)/\sigma}}\right) = 0.$$ Now we multiply both sides by $\sigma$ and re-arrange, which results in $$\sum_{i=1}^n \left(\frac{x_i-\mu}{\sigma} - 2\frac{\frac{x_i-\mu}{\sigma}e^{-(x_i-\mu)/\sigma}}{1+e^{-(x_i-\mu)/\sigma}}\right) = \sum_{i=1}^n 1.$$ Making the same simplifications as before, this becomes $$\sum_{i=1}^n \left(z_i - 2 \frac{z_i}{1+e^{z_i}}\right) = n,$$ and finally, we simplify the left side by noticing that $$z_i - 2\frac{z_i}{1+e^{z_i}} = z_i\left(1 - \frac{2}{1+e^{z_i}}\right) = z_i \cdot \frac{1+e^{z_i}-2}{1+e^{z_i}} = z_i\frac{e^{z_i}-1}{e^{z_i}+1},$$ which we can plug in to match the form of the second equation in the system.
The MOM estimators, however, are simpler. Since the mean of a logistic distribution is $\mu$, we can equate $$\hat{\mu}_{mom} = \frac{1}{n}\sum_{i=1}^n x_i$$ and since the standard deviation is $$\frac{\sigma \pi}{\sqrt{3}},$$ we can write $$\hat{\sigma}_{mom} = \frac{\sqrt{3}}{\pi} s,$$ where $s$ denotes the sample standard deviation.
See here, which also lists references for these results.
