Maximum likelihood of function of the mean on a restricted parameter space

Question

I've been trying to teach myself some of the fundamentals of statistics by trying to work through old qualifying exams. Here's a problem:

Suppose $X_1, \ldots, X_n$ are a random sample from a normal distribution with mean $\theta$ and variance $\sigma^2$, where $\sigma^2$ is fixed and $\theta>0$ is a parameter. Find the maximum likelihood estimator of $\sqrt{\theta}$.

My work so far:

I have the likelihood function $$L(\theta|\mathbf{x})=(2\pi \sigma^2)^{-n/2} \text{exp}\Big(-\frac{1}{2\sigma^2}\sum_{i=1}^{n}(x_i-\theta) \Big).$$

Then, differentiating $L(\theta|\mathbf{x})$ with respect to theta, and equating the result to zero yields $$\sum_{i=1}^{n}(x_i-\theta)=0 \implies \theta=\frac{1}{n}\sum_{i=1}^{n}x_i \hspace{3mm} (=\bar{x}).$$ The second derivative is negative here, hence the MLE of $\theta$ is $\hat{\theta}=\bar{X}$. So, by the invariance property of MLEs, the MLE of $\sqrt{\theta}$ is $\sqrt{\hat{\theta}}=\sqrt{\bar{X}}.$

Question:

My question is about the $\theta>0$ assumption (I'm pretty sure $\sqrt{\bar{X}}$ would be fine if there were no restrictions on $\theta$). So, for $\theta>0$, would the MLE for $\theta$ be something like $\max\{\bar{X},0\}$?

I would greatly appreciate any feedback, corrections, etc.

Answer 1

So, I'll modify your problem slightly to avoid dealing with boundary issues. Instead of your constraint $\theta > 0$, I'll replace it with $\theta \geq 0$.

You want to maximize the likelihood subject to $\theta \geq 0$.

After taking the logarithm of your likelihood and ignoring constant terms, we get the problem:

$$ \min_{\theta} f(\theta) \text{ s.t. } \theta \geq 0$$

where

$$f(\theta) := \sum_{i=1}^n (X_i - \theta)^2.$$

You are correct that if we didn't have the constraint, we could simply differentiate the objective function and get $\theta^{unconstrained} := \bar{X}$.

However, due to the constraint, we can't just differentiate. So, let us consider the two cases separately:

1. If $\theta^{unconstrained}$ is positive, then it is also the solution for your constrained MLE problem (the additional constraint can only increase the value of the minimization problem above).

2. If $\theta^{unconstrainted} < 0$, then it doesn't satisfy your constraint and is not feasible. However, you can check for yourself (a bit of algebra) that $f(\theta) \geq f(0)$ for all $\theta \geq 0$ when $\theta^{constrainted} < 0$. Therefore, $\theta^0=0$ minimizes $f(\theta)$ over $\theta \geq 0$.

So for this problem, the MLE for $\theta$ is $\theta^{ML} = \max{(\bar{X}, 0)}$. And using the equivariance property, the MLE of $\sqrt{\theta}$ is $\sqrt{\theta^{ML}}$.