# Maximum likelihood of function of the mean on a restricted parameter space

I've been trying to teach myself some of the fundamentals of statistics by trying to work through old qualifying exams. Here's a problem:

Suppose $X_1, \ldots, X_n$ are a random sample from a normal distribution with mean $\theta$ and variance $\sigma^2$, where $\sigma^2$ is fixed and $\theta>0$ is a parameter. Find the maximum likelihood estimator of $\sqrt{\theta}$.

My work so far:

I have the likelihood function $$L(\theta|\mathbf{x})=(2\pi \sigma^2)^{-n/2} \text{exp}\Big(-\frac{1}{2\sigma^2}\sum_{i=1}^{n}(x_i-\theta) \Big).$$

Then, differentiating $L(\theta|\mathbf{x})$ with respect to theta, and equating the result to zero yields $$\sum_{i=1}^{n}(x_i-\theta)=0 \implies \theta=\frac{1}{n}\sum_{i=1}^{n}x_i \hspace{3mm} (=\bar{x}).$$ The second derivative is negative here, hence the MLE of $\theta$ is $\hat{\theta}=\bar{X}$. So, by the invariance property of MLEs, the MLE of $\sqrt{\theta}$ is $\sqrt{\hat{\theta}}=\sqrt{\bar{X}}.$

Question:

My question is about the $\theta>0$ assumption (I'm pretty sure $\sqrt{\bar{X}}$ would be fine if there were no restrictions on $\theta$). So, for $\theta>0$, would the MLE for $\theta$ be something like $\max\{\bar{X},0\}$?

I would greatly appreciate any feedback, corrections, etc.

-
If $\theta < 0$, then $\sqrt{\bar{X}}$ is complex-valued with (arbitrarily) high probability as $n$ increases, so in that sense, there is a problem. However, your intuition regarding the MLE on the restricted parameter space is correct. How would you prove it? (Hint: Use the definition of "maximum likelihood" directly.) – cardinal Jan 10 at 23:14
The definition of MLE is just the parameter that maximizes likelihood, right? This seems pretty intuitive to me - we're just looking at the joint probability of our sample and trying to figure out what parameter is most probable. Showing this directly appears to be easy enough. Thanks for the comment! – pedrosuavo Jan 11 at 2:49

So, I'll modify your problem slightly to avoid dealing with boundary issues. Instead of your constraint $\theta > 0$, I'll replace it with $\theta \geq 0$.

You want to maximize the likelihood subject to $\theta \geq 0$.

After taking the logarithm of your likelihood and ignoring constant terms, we get the problem:

$$\min_{\theta} f(\theta) \text{ s.t. } \theta \geq 0$$

where

$$f(\theta) := \sum_{i=1}^n (X_i - \theta)^2.$$

You are correct that if we didn't have the constraint, we could simply differentiate the objective function and get $\theta^{unconstrained} := \bar{X}$.

However, due to the constraint, we can't just differentiate. So, let us consider the two cases separately:

1. If $\theta^{unconstrained}$ is positive, then it is also the solution for your constrained MLE problem (the additional constraint can only increase the value of the minimization problem above).

2. If $\theta^{unconstrainted} < 0$, then it doesn't satisfy your constraint and is not feasible. However, you can check for yourself (a bit of algebra) that $f(\theta) \geq f(0)$ for all $\theta \geq 0$ when $\theta^{constrainted} < 0$. Therefore, $\theta^0=0$ minimizes $f(\theta)$ over $\theta \geq 0$.

So for this problem, the MLE for $\theta$ is $\theta^{ML} = \max{(\bar{X}, 0)}$. And using the equivariance property, the MLE of $\sqrt{\theta}$ is $\sqrt{\theta^{ML}}$.

-
 This might be a silly question, but why can't we simply just differentiate, then consider the parameter restriction(s)? – pedrosuavo Jan 11 at 2:53 So basically ignore the parameter constraints, maximize (which in this case can be done by differentiation), then somehow project onto the constraint set? I don't think that this will in general give the best value. – Dapz Jan 11 at 6:16