Let $\epsilon_i \sim \text{Exp}(\lambda)$, $\lambda > 0$, and iid for all $i = 1,2, \dots$ Suppose for we have the linear model $$ Y_i = \beta X_i + \epsilon_i,$$ where $X_i > 0$ for all $i = 1, 2, \dots$

I would like to find the distribution of $\hat{\beta} - \beta$ for this model. The cdf and pdf of $Y_i$ is

$$F(y_i) = 1 - e^{\lambda(\beta x_i - y_i)}\mathbb{1}\{y_i \geq \beta x_i\}~~~~~~~\text{and}~~~~~~~f(y_i) = \frac{d}{dy_i}F(y_i) = \lambda e^{\lambda(\beta x_i - y_i)}\mathbb{1}\{y_i \geq \beta x_i\}.$$

Now, the likelihood is

$$L(Y_i; \beta, \lambda) = \prod_{i=1}^n \lambda e^{\lambda(\beta x_i - y_i)}\mathbb{1}\{y_i \geq \beta x_i\}.$$

We want $\beta x_i$ as large as possible without making the indicator 0. Then

$$\beta x_i \leq \min\{y_i\} ~~\Rightarrow~~ \hat{\beta} = \frac{\min\{y_i\}}{x_i}.$$

But I'm not sure how I would go about finding the distribution of

$$ \hat{\beta} - \beta = \frac{\min\{y_i\}}{x_i} - \beta.$$

  • $\begingroup$ Find its CDF... $\endgroup$ – StubbornAtom Mar 10 '20 at 6:59
  • $\begingroup$ @StubbornAtom I'm unsure how. The min gives me the idea: $P((X_i)^{-1}\min\{Y_1, \dots, Y_n\} - \beta > y) = P(\min\{Y_1, \dots, Y_n\} > X_i(y - \beta)) = \dots = exp\{-\lambda(y - \beta)\sum X_i\}?$ $\endgroup$ – xuzhang23 Mar 10 '20 at 7:16

The problem here is that you have not correctly identified the MLE. Your likelihood function can be written as:

$$\begin{equation} \begin{aligned} L_{\mathbf{y},\mathbf{x}}(\beta, \lambda) &= \prod_{i=1}^n \lambda \exp( - \lambda(y_i - \beta x_i)) \cdot \mathbb{I}(y_i \geqslant \beta x_i) \\[6pt] &= \prod_{i=1}^n \lambda \exp( - \lambda(y_i - \beta x_i)) \cdot \mathbb{I} \Big( \beta \leqslant \frac{y_i}{x_i} \Big) \\[6pt] &= \lambda^n \exp \Big( - \lambda \sum_{i=1}^n (y_i - \beta x_i) \Big) \cdot \mathbb{I} \Big( \beta \leqslant \min \Big\{ \frac{y_i}{x_i} \Big\} \Big) \\[6pt] &= \lambda^n \exp \Big( - \lambda n (\bar{y}_n - \beta \bar{x}_n) \Big) \cdot \mathbb{I} \Big( \beta \leqslant \min \Big\{ \frac{y_i}{x_i} \Big\} \Big). \\[6pt] \end{aligned} \end{equation}$$

The corresponding log-likelihood function is:

$$\begin{equation} \begin{aligned} \ell_{\mathbf{y},\mathbf{x}}(\beta, \lambda) &= \begin{cases} n \ln (\lambda) - \lambda n (\bar{y}_n - \beta \bar{x}_n) & & \text{for } \beta \leqslant \min \{ y_i/x_i \}, \\[6pt] -\infty & & \text{for } \beta > \min \{ y_i/x_i \}. \\[6pt] \end{cases} \end{aligned} \end{equation}$$

Since this function is monotonically increasing in $\beta$ over the range of the first part, it is maximised at the boundary point $\hat{\beta} = \min \{ y_i/x_i \}$ (which is different to the solution you have given). Now, to obtain the distribution of this statistic, we can use the fact that:

$$\frac{Y_i}{x_i} - \beta = \frac{\epsilon_i}{x_i} \sim \text{Exp}(\lambda x_i).$$

We therefore have the CDF:

$$\begin{equation} \begin{aligned} F_{\hat{\beta}-\beta}(t) \equiv \mathbb{P}(\hat{\beta}-\beta \leqslant t) &= \mathbb{P} \Big( \min \Big\{ \frac{Y_i}{x_i} - \beta \Big\} \leqslant t \Big) \\[6pt] &= 1 - \mathbb{P} \Big( \min \Big\{ \frac{Y_i}{x_i} - \beta \Big\} > t \Big) \\[6pt] &= 1 - \prod_{i=1}^n \mathbb{P} \Big( \frac{Y_i}{x_i} - \beta > t \Big) \\[6pt] &= 1 - \prod_{i=1}^n \exp ( - \lambda t x_i ) \\[6pt] &= 1 - \exp \Big( - t \cdot \lambda n \bar{x}_n \Big). \\[6pt] \end{aligned} \end{equation}$$

This means that we have the distribution:

$$\hat{\beta}-\beta \sim \text{Exp}( \lambda n \bar{x}_n ).$$

(Note that one consequence of this is that we have the bias $\mathbb{E}(\hat{\beta}-\beta) = 1/\lambda n \bar{x}_n$, so the "bias corrected" version of the estimator is $\hat{\beta}_* = \min \{ y_i/x_i \} + 1/\lambda n \bar{x}_n$.)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.