Homoscedasticity and independence of errors

In linear regression I often see homoscedasticity and independence of errors listed as assumptions (for example on wikipedia). But I would think that independence of errors would imply homoscedasticity. Look at this error plot example:

Could the errors be independent? I would think that they are dependent, in the sense that a large error would mean an increased chance of a large error on the following observation. What am I missing?

• You confuse serial correlation with variance.
– whuber
Jan 9, 2021 at 17:52
• So independence means serial correlation? In the sense that the value of one error should predict (with some uncertainty) the value of the next one?
– bgst
Jan 9, 2021 at 18:12
• Independence implies no serial correlation. It doesn't imply anything about heteroscedasticity.
– whuber
Jan 9, 2021 at 18:13
• it is not true that independence imply nothing about heteroschedasticity. It imply scorrealation but scorrealation among squared too, usual kind of heteroschedasticity. Also others go away. However a bit of formalization about the model and assumptions you refers on is needed. Terminology can produce ambiguities. Jan 9, 2021 at 19:50
• @bgst; from the graph I suppose that your data are in cross section. It's so? Jan 9, 2021 at 20:42

As I said at the beginning in the comments some detail more would be useful for avoid ambiguities and misconceptions. Anyway I propose here a quite general basic answer.

Starting to define the linear regression model as

$$y = X’\beta + \epsilon$$

In a widely shared definition (for example from Bruce Hansen) heteroskedasticity is:

$$V[\epsilon|X=x] = E[\epsilon^2|X=x] = \sigma^2(x)$$ (a function of $$X$$)

while homoskedasticity mean that

$$V[\epsilon|X=x] = E[\epsilon^2|X=x] = \sigma^2$$ (a positive constant)

Therefore if regression error ($$\epsilon$$) and regressors ($$X$$) are independent we cannot have heteroskedasticity, then homoskedasticity must hold. In your (@bgst) plot residuals are showed as function of $$X$$, so heteroskedasticity (defined as above) appear evident. Indeed graph like that is frequently used for show how heteroskedasticity appear.

However heteroskedasticity can be defined differently. For example:

$$V[\epsilon|Z=z] = E[\epsilon^2|Z=z] = \sigma^2(z)$$

where $$Z$$ is a set of variable not necessarily shared with the regressors set. So even if regression error ($$\epsilon$$) and regressors ($$X$$) are independent, heteroskedasticity can appear.

Moreover heteroschedaticity sometimes is presented as a property of error matrix, then as temporal or spatial dependence among errors. In time series framework a GARCH process is a common example of heteroskedasticity and it imply some dependency among errors; more precisely imply correlation among some squared error terms.

The last case can be included in the definition $$V[\epsilon|Z=z] = \sigma^2(z)$$

where $$Z$$ represent past value of squared errors.

The message in that: the concept of heteroskedasticity have much to do with that of stochastic dependance, however context/definitions/details matters a lot.

• But if heteroscedasticity disappear, then there homoscedasticity right? So you are saying that independence imply homoscedasticity?
– bgst
Jan 9, 2021 at 21:43
• Exactly. Things are so. Jan 9, 2021 at 21:52
• So why list homoscedasticity as a separate assumption if it is implied by independence?
– bgst
Jan 9, 2021 at 22:38
• The list of assumption needed can be debatable, is not precisely shared in any ref and context matters. However both, full independent error and/or homoschedasticity are not mandatory. Some detail about your model can help. However in your case heteroschedasticity appear and residuals are not independent from predictors. You are new here, i remember you that If you consider my reply helpful can upvote it. Jan 9, 2021 at 23:25
• I'm very thankful for your replies but right now they are contradicting the other replies as far as I can tell. So I need to figure out who is right.
– bgst
Jan 9, 2021 at 23:45

What you are missing is information about a study's design. Independence is somehing that comes from the study design - it is NOT implied by homoscedasticity.

To give you a simple example, imagine that you are measuring temperature (degrees Celsius) at your local airport using a sensor, with the goal being to see how temperature changes over time. If you measure temperature every day (once per day, say at noon), then you can expect the resulting daily values of temperature to be correlated with each other if they come from days which are close to each other. If you aggregated the daily temperature values collected within each year to get a yearly temperature value, then it is possible that the yearly temperature values may no longer be correlated over time (because the temporal distance of one year between consecutive values is large enough).

Staying with this example of measuring a response variable (e.g., temperature) over time (e.g., every year) and trying to regress that variable against time, you can expect to encounter all possible combinations of situations, depending on what variable you are measuring and how frequent your measurements are:

(1) independent errors and homoscedasticity;
(2) independent errors and heteroscedasticity;
(3) temporally dependent errors and homoscedasticity;
(4) temporally dependent errors and heteroscedasticity.


This fact alone hints that you are wrong to assume that independence implies homoscedasticity; if that were the case, we would never encounter situation (2) in practice. However, the statistical literature is full of examples to the contrary!

@whuber already hinted that you are confusing two distinct concepts: dependence of errors and variance of errors.

In the context of the temperature versus time example, the variance of errors simply quantifies the amount of spread you can expect to encounter in your temperature values about the underlying temporal trend in these values. Under the assumption of homoscedasticity, the amount of spread is unaffected by the passage of time (i.e., it remains constant over time). However, under the assumption of heteroscedasticity, the amount of spread is affected by the passage of time - for example, the amount of spread can increase over time. Spread is about how far you can expect an observed temperature value to be at time $$t$$ relative to what is 'typical' for that time $$t$$.

The dependence of errors looks at something different altogether: if you know something about the value of temperature at the current time $$t$$, does that tell you anything about the value of temperature at time $$t + 1$$? If the temperature values are independent from each other, knowing that temperature is high today should have no bearing on what the temperature value will be tomorrow. If the temperature values are NOT independent from each other (e.g., they are positively correlated), knowing that temperature is high today will tell you that the temperature value will also be high tomorrow. How high it will be will depend on the strength of the correlation.

• Lets say that the temperatures from your example behaves as in the error plot in my original post. I think I'm supposed to say that it looks likes independent errors and heteroscedasticity right? But if the temperature at t is extreme (very high or very low), it is more likely that the temperature at t+1 is extreme? Isn't this called dependence? Shouldn't the temperature at t+1 be completely unrelated to the temperature at t if there is independence?
– bgst
Jan 9, 2021 at 23:43
• @Isabella Ghement, you are right that study design and context matters. However situation 2 and 3 of your list seems me problematic. Some equations can help us to understand each other. Jan 10, 2021 at 0:03
• You cannot assess independence or dependence from the plot you shared in your post. If you truly were in a setting where your response variable was temperature and your predictor variable was time (e.g., year), you would assess whether temporal dependence is present among the model errors by examining ACF and PACF plots of the residuals from a simple linear regression of temperature on time. Note that these plots make sense when time is measured at equally spaced intervals (e.g., every year). Jan 10, 2021 at 0:57
• I provided my 2 cents - now it is up to you to read up more on this subject and make sure you understand the flaws in your thinking. Cross Validated can point you in the right direction but you need to go deeper and do more self-study. You had multiple people confirm that you are not framing this properly - this should be enough to make you question your current understanding of independence and variability. Jan 10, 2021 at 1:04
• By the way, if temperature and time in my example were linearly related (e.g., temperature tends to increase over time), there is a way to look for signs of positive temporal dependence among temperature values in the scatterplot of temperature versus time: you would just look for a series of several consecutive temperature values located above the linear trend line, followed by a series of several consecutive values of temperature values located below the trend line, and so on. This is totally different from looking for increasing/decreasing spread in temperature values over time! Jan 10, 2021 at 1:11