Beta confidence intervals in transformed linear regression

Question

Let's say I have a model: $$Y_i = \beta_0 \beta_1^{X_i} \epsilon_i$$

(note: This is slightly different than the more common example case of $Y_i = \alpha e^{\beta x_i}\epsilon_i$.)

I can take the log of $Y$: $$\log Y_i = \log\beta_0 + X_i\log\beta_1 + \log\epsilon_i$$

Assuming all the usual assumptions around the errors, etc, hold, I can estimate this with simple linear regression. However, I only get an estimate, and therefore a confidence interval, for $\log\beta_1$, not $\beta_1$ itself.

Can I backtransform with $\hat{\beta_1} = \exp(\widehat{\log\beta_1})$? If so, for a confidence interval for $\hat{\beta_1}$, would I just transform the endpoints of the interval?

Answer 1

Actually, the interval carries over just fine. The transformation is monotonic; the probability statement that applies on the log-scale transforms directly to the original scale, so as long as the assumptions under which the original interval was computed do apply, then it works as an interval for the original population parameter after transformation.

It's the estimate that may be problematic (but may be okay, depending on what you want). Note that $E[\exp(X)]\neq \exp[E(X)]$ if $\sigma_X^2>0$. If the log-scale estimate is unbiased, the transformed estimate is biased.

If you're happy to have an estimate that's median-unbiased, then the back-transformed estimate is also okay, for the same reason that the interval works.

If you seek mean-unbiasedness there are some choices. For example, if you're prepared to assume a normal distribution on $\hat\beta$ you can unbias it by using the properties of the lognormal. Alternatively, you can use a Taylor expansion to get an approximate adjustment (details are also in a number of posts on this site). If the standard error of the estimate is small, it won't matter much. There are other things that are done.

Answer 2

Here's a simple example with simulated data where $\beta_0=3$ and $\beta_1=2$:

. #delimit;
delimiter now ;
. clear;

. set seed 10011979;

. set obs 10000;
obs was 0, now 10000

. gen x = rnormal();

. gen y = 3 * (2^x) * rnormal(1,.25);

. gen lny = ln(y);

. reg lny x;

      Source |       SS       df       MS              Number of obs =   10000
-------------+------------------------------           F(  1,  9998) =60377.64
       Model |  4697.83357     1  4697.83357           Prob > F      =  0.0000
    Residual |  777.919401  9998  .077807502           R-squared     =  0.8579
-------------+------------------------------           Adj R-squared =  0.8579
       Total |  5475.75297  9999   .54763006           Root MSE      =  .27894

------------------------------------------------------------------------------
         lny |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           x |   .6883164   .0028012   245.72   0.000     .6828254    .6938074
       _cons |   1.065971   .0027894   382.15   0.000     1.060503    1.071438
------------------------------------------------------------------------------

Using the delta method and the fact that the derivative of $\exp$ is itself, you know that

$$SE(exp (\beta_1)) = \exp (0.6883164) \cdot 0.0028012 = 0.0055754 $$

This is the standard error that Stata calculates with nlcom:

. nlcom (exp_b1:exp(_b[x]));

      exp_b1:  exp(_b[x])

------------------------------------------------------------------------------
         lny |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      exp_b1 |   1.990362   .0055755   356.98   0.000     1.979434    2.001289
------------------------------------------------------------------------------

To get the 95% CI, it's the usual formula:

$$1.990362 \pm 1.96 \cdot 0.0055755 = [1.979434,2.00129] $$

We can also use non-linear least squares to fit this model without transforming it:

. nl (y = {b0=1}*{b1=1}^x), nolog;
(obs = 10000)

      Source |       SS       df       MS
-------------+------------------------------         Number of obs =     10000
       Model |   235659.16     2   117829.58         R-squared     =    0.9386
    Residual |  15404.5955  9998   1.5407677         Adj R-squared =    0.9386
-------------+------------------------------         Root MSE      =  1.241277
       Total |  251063.755 10000  25.1063755         Res. dev.     =  32699.58

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         /b0 |   2.996304   .0128667   232.87   0.000     2.971083    3.021525
         /b1 |   1.994413   .0048325   412.71   0.000     1.984941    2.003886
------------------------------------------------------------------------------

This gives similar results.

Answer 3

Your $\hat{log(\beta)}$ estimator is asymptotically normal. So, use delta method to derive asymptotic distribution (and confidence intervals) of a function of the estimator:

Let $\gamma=log(\beta)$ and $h(\gamma)$ is some differentiable function s.t. $h'(\gamma)\not=0$. Under some basic conditions

$\sqrt{n}(\hat\gamma-\gamma)\rightarrow N(0, V)$

Then

$\sqrt{n}(h({\hat\gamma})-h(\gamma))\rightarrow N(0, (h'(\gamma))^TV(h'(\gamma)))$

---

In your case:

$h(\gamma)=\begin{bmatrix} e^{\gamma_0} \\ e^{\gamma_1} \end{bmatrix}=\begin{bmatrix} \beta_0 \\ \beta_1 \end{bmatrix}$

$h'(\gamma)=\begin{bmatrix} e^{\gamma_0} & 0 \\ 0 & e^{\gamma_1} \end{bmatrix}$

$V=\begin{bmatrix} \sigma_{\gamma_0}^2 & \sigma_{\gamma_0\gamma_1} \\ \sigma_{\gamma_0\gamma_1} & \sigma_{\gamma_1}^2 \end{bmatrix}$

$\sqrt{n}({\hat\beta}-\beta)\rightarrow N(0, \begin{bmatrix} e^{2\gamma_0}\sigma_{\gamma_0}^2 & e^{\gamma_0+\gamma_1}\sigma_{\gamma_0\gamma_1} \\ e^{\gamma_0+\gamma_1}\sigma_{\gamma_0\gamma_1} & e^{2\gamma_1}\sigma_{\gamma_1}^2 \end{bmatrix})$

Finally, you can estimate the s.e. of $\hat\beta_1$ as $e^{\hat\gamma_1}\hat\sigma_{\gamma_1}/\sqrt{n}$. So, the relevant confidence interval is

$CI_{95\%}(\hat\beta_1)=[e^{\hat\gamma_1}-z_{97.5\%}e^{\hat\gamma_1}\hat\sigma_{\gamma_1}/\sqrt{n}, e^{\hat\gamma_1}+z_{97.5\%}e^{\hat\gamma_1}\hat\sigma_{\gamma_1}/\sqrt{n}]$

where $z_{0.975}$ is $97.5\%$ quantile of the standard normal distribution.

Answer 4

Calculate a simple linear regression and get values a couple of values. Try 1 + 2x. Then take the exponent of the values. That's what the values should be. Now try calculating the 1 + exp(2)x.

In R it would be:

exp( 1 + 2 * 1:3)
# versus
1 + exp(2) * 1:3

It should be pretty obvious these won't be equal. Given that finding it should be clear that you don't even back transform the beta coefficient let alone make a CI around it. The back transformed coefficient is useless.

What you can do is use the equation to generate points in the linear function and a CI around that function within the transform space and then back transform those final values.