9
$\begingroup$

Suppose that we have a linear model $y_i = \beta_0 + \beta_1 x_i + \epsilon_i$ that meets all the standard regression (Gauss-Markov) assumptions. We are interested in $\theta = 1/\beta_1$.

Question 1: What assumptions are necessary for the distribution of $\hat{\theta}$ to be well defined? $\beta_1 \neq 0$ would be important---any others?

Question 2: Add the assumption that the errors follow a normal distribution. We know that, if $\hat{\beta}_1$ is the MLE and $g(\cdot)$ is a monotonic function, then $g\left(\hat{\beta}_1\right)$ is the MLE for $g(\beta_1)$. Is monotonicity only necessary in the neighborhood of $\beta_1$? In other words, is $\hat{\theta} = 1/\hat{\beta}$ the MLE? The continuous mapping theorem at least tells us that this parameter is consistent.

Question 3: Are both the Delta Method and the bootstrap both appropriate means for finding the distribution of $\hat{\theta}$?

Question 4: How do these answer changes for the parameter $\gamma = \beta_0 / \beta_1$?

Aside: We might consider rearranging the problem to give $$\begin{align*} x_i &= \frac{\beta_0}{\beta_1} + \frac{1}{\beta_1} y_i + \frac{1}{\beta_1} \epsilon_i \\ &= \gamma + \theta y_i + \frac{1}{\beta_1} \epsilon_i \end{align*}$$ to estimate the parameters directly. This doesn't seem to work to me as the Gauss-Markov assumptions no longer make sense here; we can't talk about $\text{E}[\epsilon \mid y]$, for example. Is this interpretation correct?

Improve this question
$\endgroup$
6
  • $\begingroup$ Do the "standard" assumptions include the Normality of the $\epsilon_i$ or not? $\endgroup$ – whuber♦ Dec 13 '11 at 6:09
  • $\begingroup$ Good point; I added that assumption to the part about the MLE. It shouldn't be necessary for the others, though. $\endgroup$ – Charlie Dec 13 '11 at 7:06
  • 1
    $\begingroup$ The sampling distribution of $\beta_1$ is normal, whence that of $\theta$ is the reciprocal of a normal. This is bimodal with a divergent (infinite) mean, no matter what the mean of $\beta_1$ may be, and is infinitely flat at 0. The Delta method will therefore be horrible, the usual asymptotic MLE approximations will be poor, and even the bootstrap may be suspect. $\endgroup$ – whuber♦ Dec 13 '11 at 17:11
  • $\begingroup$ @whuber, Could you expand upon that? My intuition doesn't see how the reciprocal of a normal should be bimodal; my guess would be that all the mass would be at the reciprocal of the mean of the normal (here, $1/\hat{\beta}_1$). I was worried about the infinite mean possibility because of mass near 0. The bootstrap and asymptotic results require existence of the moments being estimated, so that is ultimately what this question hinges upon. $\endgroup$ – Charlie Dec 13 '11 at 17:41
  • 1
    $\begingroup$ The PDF of a reciprocal normal is $\exp(-(1/x-\mu)^2/(2\sigma^2))/(\sqrt{2\pi}x^2\sigma)dx$. At 0 all derivatives equal 0; finding critical points of its logarithm identifies a positive and negative mode (easily computed in terms of $\sigma$ and $\mu/\sigma$); the integral of $|x|$ diverges like the integral of $|x|/x^2=1/|x|$. The problem with infinite first moments attaches to the reciprocal of any random variable having positive probability density at 0, which includes all normals. $\endgroup$ – whuber♦ Dec 13 '11 at 18:23
3
$\begingroup$

Q1. If $\hat\beta_1$ is the MLE of $\beta_1$, then $\hat\theta$ is the MLE of $\theta$ and $\beta_1 \neq 0$ is a sufficient condition for this estimator to be well-defined.

Q2. $\hat\theta = 1/\hat\beta$ is the MLE of $\theta$ by invariance property of the MLE. In addition, you do not need monotonicity of $g$ if you do not need to obtain its inverse. There is only need for $g$ to be well-defined at each point. You can check this in Theorem 7.2.1 pp. 350 of "Probability and Statistical Inference" by Nitis Mukhopadhyay.

Q3. Yes, you can use both methods, I would also check the profile likelihood of $\theta$.

Q4. Here, you can reparameterise the model in terms of the parameters of interest $(\theta,\gamma)$. For instance, the MLE of $\gamma$ is $\hat\gamma=\hat\beta_0/\hat\beta_1$ and you can calculate the profile likelihood of this parameter or its bootstrap distribution as usual.

The approach you mention at the end is incorrect, you are actually considering a "calibration model" which you can check in the literature. The only thing you need is to reparameterise in terms of the parameters of interest.

I hope this helps.

Kind regards.

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ Thanks for the response. I don't have the book that you cite, but often these properties require existence of the moments being estimated. I'm not sure that the reciprocal of a normal has the requisite moments. I should have made this point clearer in my question. $\endgroup$ – Charlie Dec 13 '11 at 17:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.