Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
In linear regression I often see homoscedasticity and independence of errors listed as assumptions (for example on wikipedia). But I would think that independence of errors would imply homoscedasticity. Look at this error plot example:
Could the errors be independent? I would think that they are dependent, in the sense that a large error would mean an increased chance of a large error on the following observation. What am I missing?
As I said at the beginning in the comments some detail more would be useful for avoid ambiguities and misconceptions. Anyway I propose here a quite general basic answer.
Starting to define the linear regression model as
$y = X’\beta + \epsilon$
In a widely shared definition (for example from Bruce Hansen) heteroskedasticity is:
$V[\epsilon|X=x] = E[\epsilon^2|X=x] = \sigma^2(x)$ (a function of $X$)
while homoskedasticity mean that
$V[\epsilon|X=x] = E[\epsilon^2|X=x] = \sigma^2$ (a positive constant)
Therefore if regression error ($\epsilon$) and regressors ($X$) are independent we cannot have heteroskedasticity, then homoskedasticity must hold. In your (@bgst) plot residuals are showed as function of $X$, so heteroskedasticity (defined as above) appear evident. Indeed graph like that is frequently used for show how heteroskedasticity appear.
However heteroskedasticity can be defined differently. For example:
$V[\epsilon|Z=z] = E[\epsilon^2|Z=z] = \sigma^2(z)$
where $Z$ is a set of variable not necessarily shared with the regressors set. So even if regression error ($\epsilon$) and regressors ($X$) are independent, heteroskedasticity can appear.
Moreover heteroschedaticity sometimes is presented as a property of error matrix, then as temporal or spatial dependence among errors. In time series framework a GARCH process is a common example of heteroskedasticity and it imply some dependency among errors; more precisely imply correlation among some squared error terms.
The last case can be included in the definition $V[\epsilon|Z=z] = \sigma^2(z)$
where $Z$ represent past value of squared errors.
The message in that: the concept of heteroskedasticity have much to do with that of stochastic dependance, however context/definitions/details matters a lot.
What you are missing is information about a study's design. Independence is somehing that comes from the study design - it is NOT implied by homoscedasticity.
To give you a simple example, imagine that you are measuring temperature (degrees Celsius) at your local airport using a sensor, with the goal being to see how temperature changes over time. If you measure temperature every day (once per day, say at noon), then you can expect the resulting daily values of temperature to be correlated with each other if they come from days which are close to each other. If you aggregated the daily temperature values collected within each year to get a yearly temperature value, then it is possible that the yearly temperature values may no longer be correlated over time (because the temporal distance of one year between consecutive values is large enough).
Staying with this example of measuring a response variable (e.g., temperature) over time (e.g., every year) and trying to regress that variable against time, you can expect to encounter all possible combinations of situations, depending on what variable you are measuring and how frequent your measurements are:
(1) independent errors and homoscedasticity;
(2) independent errors and heteroscedasticity;
(3) temporally dependent errors and homoscedasticity;
(4) temporally dependent errors and heteroscedasticity.
This fact alone hints that you are wrong to assume that independence implies homoscedasticity; if that were the case, we would never encounter situation (2) in practice. However, the statistical literature is full of examples to the contrary!
@whuber already hinted that you are confusing two distinct concepts: dependence of errors and variance of errors.
In the context of the temperature versus time example, the variance of errors simply quantifies the amount of spread you can expect to encounter in your temperature values about the underlying temporal trend in these values. Under the assumption of homoscedasticity, the amount of spread is unaffected by the passage of time (i.e., it remains constant over time). However, under the assumption of heteroscedasticity, the amount of spread is affected by the passage of time - for example, the amount of spread can increase over time. Spread is about how far you can expect an observed temperature value to be at time $t$ relative to what is 'typical' for that time $t$.
The dependence of errors looks at something different altogether: if you know something about the value of temperature at the current time $t$, does that tell you anything about the value of temperature at time $t + 1$? If the temperature values are independent from each other, knowing that temperature is high today should have no bearing on what the temperature value will be tomorrow. If the temperature values are NOT independent from each other (e.g., they are positively correlated), knowing that temperature is high today will tell you that the temperature value will also be high tomorrow. How high it will be will depend on the strength of the correlation.
