A distribution looks like this:


modeled by an equation $y=1.0333x^2 - .5382x + 1.6905.$

Find the rate of change (i.e. the slope at that point of the regression equation) at point 6 (the x axis value), and give the standard error of that slope.

The data set is this:

x   y
1   1.685583341
2   0.283701371
3   12.46623075
4   18.72989511
5   25.80864106
6   37.87660867
7   50.31247602
8   59.85196297
9   77.95083301
10  99.94633518
1   1.16826204
2   5.472501855
3   7.018634811
4   18.20518892
5   21.1307888
6   33.77248541
7   46.63896027
8   63.82182222
9   84.0457413
10  99.50407873
1   2.580495548
2   6.153600293
3   10.37034361
4   16.88383006
5   28.39918421
6   32.9756888
7   46.8185963
8   63.48607854
9   80.27429267
10  101.7215886
  • 1
    $\begingroup$ Re the "nonlinear" in the original title: this is a linear regression. The answer therefore uses the standard mechanisms for obtaining the standard error of any linear combination of parameters. See stats.stackexchange.com/questions/148638. $\endgroup$ – whuber Jun 3 '16 at 19:53
  • 1
    $\begingroup$ I think my main question has an underlying assumption that I'm not sure about. Do slopes of nonlinear relationships even have standard errors to begin with? I know the overall regression does, but what about the slopes at each x? $\endgroup$ – Tom Jun 3 '16 at 21:46

The model is

$$\mathbb{E}(y) = \beta_0 + \beta_1 x + \beta_2 x^2.$$

Adding a fixed (usually small) quantity $\delta x$ to $x$ and comparing gives the difference

$$\eqalign{ \frac{\delta\,\mathbb{E}(y)}{\delta\,x} &= \frac{\beta_0 + \beta_1(x+\delta x) + \beta_2(x+\delta x)^2 - (\beta_0 + \beta_1 x + \beta_2 x^2)}{\delta x} \\ &= \beta_1 + \beta_2 (2x + \delta x). }$$

This is the first difference in $y$. For the slope itself, take the limit as $\delta x \to 0$, giving

$$\frac{d\,\mathbb{E}(y)}{d\,x} = \beta_1 + 2\beta_2 x.$$

Like the model for $y$ itself, this is a linear combination of the parameters $(\beta_0, \beta_1, \beta_2)$ (with coefficients $c_0=0,c_1=1,c_2=2x$). That is key.

Obtain estimates of the coefficients, $(\hat\beta_1, \hat\beta_2)$, in any way you like, along with their covariance matrix


Thus, $\Sigma_{ii}$ gives the estimation variance of $\beta_i$ and $\Sigma_{12}=\Sigma_{21}$ gives their covariance. With these in hand, estimate the slope at any given $x$ as

$$\widehat{\frac{d\,\mathbb{E}(y)}{d\,x}} = \hat\beta_1 + 2\hat\beta_2 x.$$

Using the standard rules to compute variances of linear combinations, its estimation variance is

$$\operatorname{Var}\left(\widehat{\frac{d\,\mathbb{E}(y)}{d\,x}}\right) = \text{Var}(\hat\beta_1 + 2\hat\beta_2 x)= \Sigma_{11} + 4x\Sigma_{12} + 4x^2\Sigma_{22}.\tag{1}$$

Its square root is the standard error of the slope at $x$.

This easy calculation of the standard error was possible due to the key observation previously made: the estimated slope is a linear combination of the parameter estimates.

More generally, to obtain the variance of a linear combination, compute

$$\operatorname{Var}\left(c_1\hat\beta_1 + c_2\hat\beta_2\right) = c_1^2\Sigma_{11} + 2c_1c_2\Sigma_{12} + c_2^2\Sigma_{22}.\tag{1}$$

Its square root is the standard error of this linear combination of coefficients.

Estimate higher derivatives, partial derivatives (or indeed any linear combination of the coefficients) and all their variances in a multiple regression model using the same techniques: differentiate, plug in the estimated parameters, and compute the variance.

For these data, $\Sigma$ is calculated (in R) to be

            (Intercept)      x I(x^2)
(Intercept)       2.427 -0.921  0.073
x                -0.921  0.423 -0.037
I(x^2)            0.073 -0.037  0.003

Using this, I drew one thousand randomly generated tangent lines for $x=6$ (assuming a trivariate normal distribution for $(\hat\beta_0,\hat\beta_1,\hat\beta_2)$) to depict the variance of the slope. Each line was drawn with high transparency. The black bar on the figure is the cumulative effect of all thousand of those tangents. (On top of it is drawn, in red, the estimated tangent itself.) Evidently, the slope is known with some certainty: its variance (by formula $(1)$) is only $0.024591$. Since the intercept of the curve itself is much less certain (its variance is $2.427$), most of these tangents differ only in elevation, not in angle, forming the tightly collimated black bar you see.


To show what else can occur, I added independent Normal errors of standard deviation $10$ to each data point and performed the same construction for the base point $x=2$. Now the slope, being much less certain, is manifest as a spreading fan of tangents.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.