Estimating $\theta$ based on censored data when $X_i\sim \text{Uniform}(0,\theta)$ with $\theta\ge 1$

Question

Suppose $(X_i)_{1\le i\le n}$ are i.i.d $\text{Uniform}(0,\theta)$ random variables where $\theta \ge 1$. We observe $Y_i=\min(X_i,1)$ instead of $X_i$. I wish to estimate $\theta$ based on the data $(Y_i)_{1\le i\le n}$.

I think the likelihood takes the form

\begin{align} L(\theta\mid \boldsymbol y)&=\prod_{i:x_i<1} \left(\frac1{\theta}\mathbf1_{0<y_i<\theta}\right)\prod_{i:x_i\ge 1}\left(1-\frac1{\theta}\right) \\&=\theta^{-r}(1-\theta^{-1})^{n-r}\mathbf1_{0<y_{(n)}<\theta}\,, \end{align}

where $r=\sum_{i=1}^n \mathbf1_{x_i<1}=\sum_{i=1}^n \mathbf1_{y_i<1}$ is the number of observations less than $1$.

If this is the correct likelihood, then a sufficient statistic for $\theta$ seems to be $T=(R,Y_{(n)})$. Or is the sufficient statistic simply $R$? Is the sufficient statistic complete? This would help to answer if there is UMVUE of $\theta$.

On the other hand, what is MLE of $\theta$? If I ignore the indicator $\mathbf1_{0<y_{(n)}<\theta}$ in the likelihood and assume $\theta\ne 1$, then differentiation leads to the stationary point $\hat\theta =\frac{n}{r}$. I am not sure if this is a valid answer.

Answer 1

Some more details than the other answers. The probability distribution of $Y$ will be a mixture distribution with two components, one continuous and one discrete. To find the distribution of $Y$, write $$ \DeclareMathOperator{\P}{\mathbb{P}} \P(Y\in A) = \P(Y\in A \mid Y<1)\P(Y<1)+\P(Y\in A\mid Y=1)\P(Y=1)\\ =\frac1\theta\cdot\int_{A\cap [0,1]} \; dy + \frac{\theta-1}{\theta}\cdot \mathbb{1}(Y\in A) $$ leading directly to the likelihood $$ L_Y(\theta) = \left(\frac{\theta-1}{\theta}\right)^{n-r}\cdot \left( \frac1\theta\right)^r $$ showing that $R$ alone is a sufficient statistic. Now the usual procedure leads to the maximum likelihood estimator $$ \hat{\theta}_{ML}= 1+ \frac{n-r}r $$ which also seems intuitively reasonable.

Some more details Likelihood defined in this situation with a mixture distribution might be new to many, so some details might help. But first, this is discussed onsite earlier at Maximum likelihood function for mixed type distribution and Weighted normal errors regression with censoring. We will need some concepts from measure theory ... let $\mu^*$ be the measure given by $$ \mu^*(A)= \mu(A) + \mathbb{1}\{ 1\in A\} $$ where $\mu$ is Leb (Lebesgue) measure and the second term is an atom at $1$. Now, the distribution of $Y$ can be written as a density (in the sense of the Radon-Nikodym theorem) with respect to $\mu^*$. This looks like $$ \P(Y\in A) =\int_A f(y) \; \mu^*(dy) =\int_A f(y) \; \mu(dy) + \int_A f(y)\; d\delta_1(y) $$ where $\delta_1$ is the atom at $1$. The Radon-Nikodym density $f$ can be written $$ f(y)= \frac1\theta \mathbb{1}\{0\le y < 1\} + \frac{\theta-1}\theta\cdot \mathbb{1}\{y=1\} $$ Note that the first term in the density only contributes to the integral with respect to $d\mu$, the second term only to the integral with respect to $d\delta_1$.

So, defining the likelihood using the RN-derivative $f$, we get $$ L_Y(\theta)=\prod_i^n f(y_i)=\prod_i^n \left\{ \frac1\theta \mathbb{1}\{0\le y_i < 1\} + \frac{\theta-1}\theta\cdot \mathbb{1}\{y_i=1\} \right\} $$ and simplifying gives the likelihood above.

Answer 2

It's redundant to have the $\textbf{1}_{0 < y_i < \theta}$ in the likelihood, because the product is already over samples where that is true. So you don't need that factor there. Then the likelihood is nicely differentiable and easy to optimize for $\theta$. And yes, your likelihood is correct without that factor.

When you do, you should get a result that seems intuitive. For example, if $\theta=2$, we should expect half the samples to be less than one. Think, in general, what proportion of samples (what $\frac{r}{n}$ in your notation) you should expect to see $<1$ for a given value of $\theta$, and see if that's consistent with your ML estimate $\hat\theta$.

Answer 3

For censoring problems like this you are dealing with an observable random variable that is a mixture of a continuous and discrete part, with the discrete part occurring at the censoring value. For this type of data, it is sometimes easier to derive the likelihood function by starting with the CDF of the censored values and then using this to get their PDF. I'm going to do the derivation without your assumption that $\theta \geqslant 1$ initially, and then I'll add that assumption at the end, so that you can see how it is done in general.

---

Deriving the PDF for the censored values: The simplest way to do this is to first obtain the CDF. Since $X \sim \text{U}(0, \theta)$ you have:

$$\mathbb{P}(X \leqslant x) = \frac{\min(x, \theta)}{\theta} \quad \quad \quad \text{for all } x \geqslant 0.$$

Thus, for all $y \geqslant 0$ we have the CDF:

$$\begin{align} F_Y(y) &\equiv \mathbb{P}(Y \leqslant y) \\[8pt] &= \mathbb{P}(\min(X,1) \leqslant y) \\[8pt] &= \mathbb{P}(X \leqslant y) \cdot \mathbb{I}(y \leqslant 1) + \mathbb{I}(y > 1) \\[6pt] &= \frac{\min(y, \theta)}{\theta} \cdot \mathbb{I}(y \leqslant 1) + \mathbb{I}(y > 1). \\[6pt] \end{align}$$

To get the PDF for this mixture variable we use the Dirac-delta function $\delta$ for the discrete part. Differentiating the CDF and using the Dirac delta function gives the PDF:$^\dagger$

$$\begin{align} f_Y(y) &= \frac{dF_Y}{dy}(y) \\[6pt] &= \bigg[ \frac{d}{dy} \frac{\min(y, \theta)}{\theta} \bigg] \mathbb{I}(y \leqslant 1) + \bigg[ 1 - \frac{\min(\theta, 1)}{\theta} \bigg] \delta(1) \\[6pt] &= \bigg[ \frac{1}{\theta} \cdot \mathbb{I}(y \leqslant \theta) \bigg] \mathbb{I}(y \leqslant 1) + \frac{\theta - \min(\theta, 1)}{\theta} \cdot \delta(1) \\[6pt] &= \frac{1}{\theta} \cdot \mathbb{I}(y \leqslant \min(\theta, 1)) + \frac{\max(0, \theta-1)}{\theta} \cdot \delta(1). \\[6pt] \end{align}$$

Now, if you add in your assumption that $\theta \geqslant 1$ then you get the simplified PDF:

$$f_Y(y) = \frac{1}{\theta} \cdot \mathbb{I}(y \leqslant 1) + \frac{\theta-1}{\theta} \cdot \delta(1).$$

---

The likelihood function and MLE: Now that we have the PDF we can write the likelihood function. Using your notation, suppose we let $R \equiv R(\mathbf{y}) \equiv \sum_{i=1}^n \mathbb{I}(y_i < 1)$ be the number of non-censored data points. We can then write the likelihood as:

$$\begin{align} L_\mathbf{y}(\theta) &= \prod_{i=1}^n f_Y(y_i) \\[6pt] &= \bigg( \frac{1}{\theta} \bigg)^r \times \bigg( \frac{\theta-1}{\theta} \bigg)^{n-r} \\[6pt] &= \frac{(\theta-1)^{n-r}}{\theta^n}, \\[6pt] \end{align}$$

which gives the log-likelihood function:

$$\ell_\mathbf{y}(\theta) = (n-r) \log(\theta-1) - n \log (\theta) \quad \quad \quad \quad \quad \text{for } \theta \geqslant 1.$$

The statistic $R$ is sufficient for this problem, and the MLE is:

$$\hat{\theta}_\text{MLE} = \frac{n}{r}.$$

I will leave the rest of the analysis (completeness, etc.) to you. Further analysis should be reasonably simple due to the simple form of the log-likelihood function

---

$^\dagger$ We use the convention that $0 \cdot \delta(x) = 0$ so that the last term disappears if $\theta < 1$.