I have been reading about the delta method in regards to auto regressive distributed lag models. This is very new to me, so excuse any beginner mistakes.

The problem is as follows:

We have a model for gasoline consumption. $g$ is the per capita consumption, $y$ disposable income, $p$ is price, $g_{t-1}$ is lagged consumption. All the values are in logs.

$$g_t = \alpha_0 + \beta_1 p_t + \beta_2 y_t + \omega g_{t-1} + u_t$$

$\beta_i$ denote the short-run effects and $\beta_i/(1-\omega)$ denote the long-run effect. The problem is that these long-run estimates do not have standard errors calculated in most studies. I found only two papers that do: Bentzen & Engsted (2001) and Pesaran & Shin (1997). They propose to calculate the standard error using the delta method.

The problem that I see is that $y_t$ (or $p_t$) and $g_{t-1}$ are highly correlated, thus violating the delta method assumption (as far as I understand it). The correlation is quite clear since both $y_t$ and $p_t$ are significant in the regression above, so taking

$$g_{t-1} = \alpha_0 + \beta_1 p_{t-1} + \beta_2 y_{t-1} + \omega g_{t-2} + u_{t-1},$$

we know that there is correlation between $g_{t-1}$ and $p_{t-1}$ (or $y_{t-1})$, given price (or income) persistency, correlation between $g_{t-1}$ and $p_t$ or $y_t$ is surely there. I even dug a whole lot of data from Eurostat to confirm sample correlation and it was there, higher than 0.5 in absolute value.

You can also see the estimated standard error using the delta method is much larger than the ones estimated using other methods. That indicates the omitted correlation might cause the overestimation of the standard error.

So the question is: Can I use the delta method to estimate the standard error of the non-linear transformation while knowing these variables are correlated? Or does the non-linear nature of the transformation change things?


1 Answer 1


Yes, you can still use the delta method with correlated variables.

Let us label your function $f(\theta)$, where $\theta = (\beta, \omega)^T$ and $f(\theta) = \beta / (1-\omega)$. The delta method is based upon the Taylor expansion:

$f(\hat{\theta}) \approx f(\theta) + (\hat{\theta} - \theta)^Tf'(\theta)$

Rearranging terms and squaring both sides results in:

$(f(\hat{\theta}) - f(\theta))^2 \approx (\hat{\theta} - \theta)^Tf'(\theta)f'(\theta)^T(\hat{\theta} - \theta)$

Taking expectations:

$\text{Var} f(\hat{\theta}) \approx \mathbb{E}(\hat{\theta} - \theta)^Tf'(\theta)f'(\theta)^T(\hat{\theta} - \theta)$

Taking derivatives of $f$ and evaluating $f'$ at $\hat{\theta}$ gives:

$f'(\hat{\theta})f'(\hat{\theta})^T = \frac{1}{(1-\hat{\omega})^2} \begin{bmatrix} 1 & \hat{\beta} / (1 - \hat{\omega}) \\ \hat{\beta} / (1 - \hat{\omega}) & \hat{\beta}^2 / (1 - \hat{\omega})^2 \end{bmatrix} $

Writing out the full expression for $\text{Var}f(\hat{\theta})$ and substituting estimates:

$\widehat{\text{Var}} f(\hat{\theta}) = \frac{1}{(1-\hat{\omega})^2}(\hat{\sigma}^2_{\beta} + 2\hat{\sigma}_{\beta \omega} \hat{\beta} / (1-\hat{\omega}) + \hat{\sigma}^2_{\omega}\hat{\beta}^2 / (1 - \hat{\omega})^2)$

You can see that positive correlation between $\beta$ and $\omega$ is going to increase the variance of the estimate of the long-run effect; it means there's a negative correlation between the estimates of $\beta$ and $1 - \omega$, the numerator and denominator of the long-run effect, so the estimated numerator and denominator tend to move in opposite directions, which naturally increases variability relative to the uncorrelated case.

Note that the delta method can fail miserably, so you might want to check its performance via simulation, e.g., by specifying all the parameters and creating many data sets with different errors, estimating the long run effect for each data set, calculating the standard deviation of the long run effect estimates, and comparing that to the delta method estimates of the standard error for the various data sets.

  • 1
    $\begingroup$ Wow! So 1) If the correlation increases the variance, then if I do not know the correlation, I in fact underestimate (in case of positive corr) the standard error? (I only have results from regressions, I do not have the data, correlation matrices, nothing) 2) You are saying it may fail miserably -- could you recommend an article/paper on these issues with the delta method? Thanks! $\endgroup$
    – Ondrej
    Commented Jan 29, 2012 at 10:06
  • 1
    $\begingroup$ Generally, a regression package will have an option to print out the estimated covariance matrix of the parameter estimates, which is all you need. As for failure of the delta method; if the first order Taylor expansion is a poor approximation to the function, and if the higher-order moments of the parameter estimates aren't small enough to effectively cancel out the higher-order terms in the Taylor expansion which make the first-order approximation poor, then the delta method will likely give poor results too. $\endgroup$
    – jbowman
    Commented Jan 29, 2012 at 16:56
  • $\begingroup$ Ok, I get the failure. As to the correlation: it is not that I don't know how to get it... rather I cannot get it. I work with regression estimation results (from published studies) that only present the usual set of values, not correlation matrices. So in this case the delta method is quite useless, because if I used it, omitting the non-zero correlation, I would get biased estimates. Bummer, but still useful for me in other scenarios. That is all, thank you very much, you have been very helpful. $\endgroup$
    – Ondrej
    Commented Jan 29, 2012 at 18:34
  • 2
    $\begingroup$ One thing you can do - take a published study, assume all the parameter estimates are exactly correct, then simulate a "duplicate" study - same sample size etc. - as best you can. Then estimate the parameters using the simulated data, and look at the estimated covariance matrix. This may get you in the ballpark - esp. with a few repeats - and it may be that close is good enough, in the sense that varying the covariance a little has very little effect on the target standard deviation estimate. $\endgroup$
    – jbowman
    Commented Jan 29, 2012 at 19:53
  • $\begingroup$ I have found this paper to be very useful: Powell, L. A. 2007. Approximating variance of demographic parameters using the delta method: a reference for avian biologists. The Condor 109:949-954. $\endgroup$
    – RioRaider
    Commented Aug 2, 2012 at 1:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.