6
$\begingroup$

such as find MLE of $P(X>12)$ when $X_i$ ~ $N(\mu,\sigma^2) $

$\endgroup$
6
  • $\begingroup$ Maybe write out $P$ as an integral and try to derive the likelihood function. I’m curious where this goes. +1 $\endgroup$
    – Dave
    Oct 16 '20 at 12:58
  • 1
    $\begingroup$ [not sure if I'm correctly interpreting the question] The higher the value of $\mu$, the higher the value of $P(X>12)$, so it seems there would be no finite answer. For a fixed $\mu>12$, the variance should be as small as possible (to approach 1). For a fixed $\mu<12$, the variance should be as large as possible (to approach 0.5) $\endgroup$
    – jpneto
    Oct 16 '20 at 13:32
  • $\begingroup$ @jpneto You’ve misinterpreted something about maximum likelihood estimation. Even the maximum likelihood estimator of $\mu$, the usual sample mean $\bar{X}$, has no bounds. $\endgroup$
    – Dave
    Oct 16 '20 at 13:40
  • 1
    $\begingroup$ @Dave There appear to be several forms of misinterpretation in this comment thread, so I posted a solution in the hope that might clear things up a little. $\endgroup$
    – whuber
    Oct 16 '20 at 14:30
  • 1
    $\begingroup$ @jpneto P(X>12) should not be as high as possible. The estimate of P(X>12) should be as close as possible to the true value of P(X>12). And you can do that estimation by estimating maximizing the likelihood, but the likelihood is not equal to this P(X>12). The value P(X>12) is the parameter to be optimized. $\endgroup$ Oct 16 '20 at 20:10
17
$\begingroup$

As I'm sure you know (or can easily derive), the maximum likelihood estimator of $(\mu,\sigma^2)$ is

$$(\hat\mu,\hat\sigma^2) = \left(\bar X, \operatorname{Var}(X)\right)$$

where, as usual, $n\bar X = X_1+X_2+\cdots + X_n$ and, a little unusually,

$$\operatorname{Var}(X) = \frac{1}{n}\left((X_1-\bar X)^2 + (X_2-\bar X)^2 + \cdots + (X_n-\bar X)^2\right)$$

(notice the $n$ rather than the $n-1$ in the denominator).

A fundamental property of MLEs is that a maximum likelihood estimator of any function $f$ of the parameters equals $f$ applied to the MLEs of the parameters. In this case, letting $\Phi$ be the standard Normal CDF

$$f(\mu,\sigma^2) = \Pr(X \gt 12) = 1 - \Phi\left(\frac{12-\mu}{\sqrt{\sigma^2}}\right).$$

Therefore the MLE of $\Pr(X \gt 12)$ is $$\hat f = 1 - \Phi\left(\frac{12-\hat\mu}{\sqrt{\hat\sigma^2}}\right).$$

Because maximum likelihood estimation is a procedure intended for relatively large sample sizes, here is a simulation based on 100,000 samples of size $n=240.$ The parameter was set to $(\mu,\sigma)=(5,3).$

Figure

You can see how the estimates are spread roughly evenly around the true value (marked as a red vertical segment), indicating this solution is a reasonable estimator. Further simulations with ever larger sample sizes (I went up to $n=24000$ but limited the simulation to just a thousand iterations to keep the computation time down to a second or two) indicate this solution converges around the true value, as it ought.

#
# Specify the problem.
#
mu <- 5
sigma <- 3
threshold <- 12
n <- 240
#
# Draw samples.
#
set.seed(17)
n.sim <- 1e5
x <- matrix(rnorm(n.sim*n, mu, sigma), n.sim)
#
# Compute the MLEs.
#
mu.hat <- rowMeans(x)
sigma2.hat <- rowMeans((x - mu.hat)^2)
p.hat <- pnorm((threshold - mu.hat) / sqrt(sigma2.hat), lower.tail = FALSE)
#
# Plot the MLEs.
#
hist(p.hat, freq=FALSE, col="#f0f0f0", breaks=50,
     xlab=expression(hat(f)), cex.lab=1.25,
     main=expression(paste("Histogram of ", Pr(X>12))))
abline(v = pnorm(threshold, mu, sigma, lower.tail = FALSE), lwd=2, col="#d01010")
$\endgroup$
7
  • $\begingroup$ Small doubt: are we finding here the MLE of the random variable $P \equiv Pr(X>12)$ here? If so, how does the distribution of this random variable (as given in your answer) depends on the observed sample? $\endgroup$
    – Dayne
    Oct 16 '20 at 14:54
  • $\begingroup$ @Dayne The probability is not a random variable: it is a number, as always. We are finding the MLE of that number. $\endgroup$
    – whuber
    Oct 16 '20 at 15:00
  • $\begingroup$ Yes, my bad. What I meant was that we maximize likelihood function which is interpreted as a function of parameter, given observations. So this function depends on observations. Here the likelihood function didn't seem to be dependent on observations. Anyway, perhaps we are estimating the conditional probability $Pr(X>12 | x_1, x_2,...)$ $\endgroup$
    – Dayne
    Oct 16 '20 at 15:08
  • 1
    $\begingroup$ Got it. Thanks for comments. Without the conditional part it seemed as the estimate of parameter $Pr(X>12)$ would just be independent of observations, as also alluded by @jpneto. And I agree this method (using the MLE property) is much more elegant. $\endgroup$
    – Dayne
    Oct 16 '20 at 15:25
  • 2
    $\begingroup$ @elemolotiv When $X$ is a random variable, $\Pr(X\gt 12)$ is a property of its distribution, usually termed a parameter. You describe the problem of estimating a parameter and the theory is called statistical (point) estimation. One of the classic texts is Lehmann & Casella's Theory of Point Estimation. $\endgroup$
    – whuber
    Feb 28 at 17:40

Not the answer you're looking for? Browse other questions tagged or ask your own question.