1
$\begingroup$

The following is excerpted from An Introduction to Statistical Learning by Tibshiriani et. al. In page 66, they introduce the standard error for linear regression coefficients: $$Y = \beta_0+\beta_1X+\epsilon,\quad \epsilon \sim \mathcal N (0, \sigma)$$ $$SE(\hat \beta_0)^2 = \sigma^2 \Big[\frac{1}{n} +\frac{\bar x^2 }{\sum_{i=1}^n(x_i-\bar x)^2} \Big] $$ and $$SE(\hat \beta_0)^2 =\frac{\sigma^2}{\sum_{i=1}^n(x_i-\bar x)^2}.$$ The passage then proceeds as below:

"For these formulas to be strictly valid, we need to assume that the errors $\epsilon_i$ for each observation are uncorrelated with common variance $σ^2$. This is clearly not true in Figure 3.1, but the formula still turns out to be a good approximation." Here is the figure: enter image description here

I understand that the assumption of independent errors can be quite unrealistic in many scenarios. But if some correlation is suspected, I would think of, for example, the Durbin-Watson test to detect it.

Now, first, what do the authors mean by "this" in the sentence "this is clearly not true"? Are they referring to the common variance assumption or to the uncorrelated errors assumption?

And in general, is it possible to detect correlation between observation errors from a scatter plot (particularly this one)?

$\endgroup$
3
  • 1
    $\begingroup$ The data appears to curve downwards at the left side of the plot. If the data were fit well by a straight line, this would not be the case. I can see from the shape of the scatterplot that a model with some curvature in that part of the plot would be a better fit to the data. $\endgroup$ Commented Sep 21, 2019 at 23:43
  • $\begingroup$ You should not apply the Durbin-Watson test to non-time series data. The inherent ordering of the data is what makes serial correlation possible and the DW test meaningful. Otherwise, you can randomly shuffle the ordering of data and find orderings that "suggest serial correlation" when in fact this is nonsensical because they're not inherently ordered (i.e. it doesn't matter if the 10th student is listed as the first in a sample of students, but if you try to list the 10th closing stock price as the 1st in a series of daily prices, you're messing up inherent ordering in the latter). $\endgroup$
    – LSC
    Commented Sep 22, 2019 at 12:05
  • $\begingroup$ Ran out of characters... the authors, as Peter Flom mentioned, are referring to the assumption of constant variance over all settings of the predictor(s). This is revealed by the funnel-shaped patter with increasing spread as you move rightward on the x-axis. $\endgroup$
    – LSC
    Commented Sep 22, 2019 at 12:08

1 Answer 1

1
$\begingroup$

Assuming that this is a plot of x vs. y, the common variance assumption is clearly off. There is much greater variance on the right side of the plot than the left, indicating that y is less well predicted by x as x increases (at least in absolute terms).

You could do various tests for heteroscedasticity, but I suggest using both OLS regression and a model that does not assume homoscedasticity (such as quantile regression) and seeing if median regression gives very similar results to mean regression - if the DV is symmetric. If the DV is asymmetric, you could choose a quantile that approximates the mean.

$\endgroup$
1
  • 1
    $\begingroup$ You could also still use OLS but employ a heteroskedasticity-robust standard error or you could boostrap your way into the error variance and standard error estimates. $\endgroup$
    – LSC
    Commented Sep 22, 2019 at 12:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.