5
$\begingroup$

What are good examples to show that, even if a regression of $Y$ on $X$ is heteroscedastic, a regression of $Y$ on a different independent variable $Z$ could be homoscedastic?

More formally, what are populations of triples $(Y_i, X_i, Z_i)$ for which all of the following hold:

a) $E[Y_i | X_i]$ is an increasing (or decreasing) function of $X_i$;

b) $Var[Y_i | X_i]$ is an increasing (or decreasing) function of $X_i$;

c) $E[Y_i | Z_i]$ is an increasing (or decreasing) function of $Z_i$;

d) $Var[Y_i | Z_i]$ is constant?

I am interested in examples where it is known or plausible that these properties apply to a population (ie they are not just quirks of particular samples that may not be representative). Although this scenario seems quite possible in theory, it is (to me at least) quite hard to find convincing examples.

Eventually I thought of the following. $Y_i$ is the number of years individual i is in full-time education. $X_i$ is i's average family income during the period of i's education. $Z_i$ is i's month of birth (1,...,12) in terms of the school year. Suppose the individuals i are in a country in which:

1) children are required to start school on reaching a certain birthday (so that they start at different times of the academic year, depending when their birthday falls);

2) compulsory education ends at the end of the academic year in which children reach a certain age, and most education beyond the compulsory period also ends at the end of the academic year;

3) children from families with higher incomes are more likely to stay in education beyond the compulsory period.

Then children from higher-income families are likely to spend longer in education on average, but the time they spend in education is likely to vary according to aptitudes and preferences, while children from lower-income families are more likely to spend in education only the compulsory period. Children born towards the beginning of the academic year are likely to spend longer in education on average, but there is unlikely to be much variation with month of birth in how much longer they spend (those born in different months being equally likely to be from higher-income families and to stay in education beyond the compulsory period).

Are there better (simpler, more obvious) examples?

Addendum: whuber has given an excellent theoretical answer showing how many examples can be generated. Answers involving specific examples with familiar variables (eg $Y$ = Income) would also be of interest.

$\endgroup$
4
$\begingroup$

All you need is to suppose the underlying relations among the variables differ from those implicitly assumed in your model.

Suppose, for instance, that $Y = \beta X + \varepsilon$ and $Z = \gamma Y + \alpha X\delta$ where $\varepsilon$ and $\delta$ are iid random variables and $\alpha, \beta,$ and $\gamma$ are nonzero fixed numerical parameters. Then--obviously--$Y$ has a homoscedastic relationship to $X$ but--perhaps not quite so obviously, but plausibly--$Y$ has a heteroscedastic relationship to $Z$. Indeed, by eliminating $X$ we can rewrite the relationship between $Y$ and $Z$ as

$$Y = \frac{\beta Z + \alpha \delta \varepsilon}{\alpha\delta + \beta\gamma} \approx \left(\frac{1}{\gamma}\right)Z + \frac{\alpha(Z-\gamma\varepsilon)}{\beta\gamma^2}\left(-\delta\right) + O(\delta^2)$$

whence the fluctuations due to $\delta$ introduce fluctuations in $Y - Z/\gamma$ whose standard deviation is approximately $|\frac{\alpha(Z-\gamma\varepsilon)}{\beta\gamma^2}|$ times the standard deviation of $\delta$--and this multiple, because it depends on $Z$, will evidently be heteroscedastic when $|Z|$ is large compared to $\gamma\ \text{SD}(\varepsilon).$

To really see this, run a simulation. (We may take the simulated values as an example of an actual population as requested.)

Scatterplot matrix

x <- 1:800 / 100
y <- x + rnorm(length(x))
z <- y + rnorm(length(x), sd=x/3)
pairs(cbind(x,z,y))

The plot in the lower left relating $Y$ to $X$ is homoscedastic: the vertical spreads in $Y$ are the same (up to chance variation) regardless of horizontal position. The plot in the middle bottom relating $Y$ to $Z$ is heteroscedastic: the spread clearly increases as $Z$ increases.

We can make up myriad scenarios corresponding to this picture: all it needs is for us to choose to regress $Y$ against $Z$ and $X$ when the actual influences are of the form $X\to Y$ and $(X,Y)\to Z$, with $X$ influencing the variation of $Z$.

$\endgroup$
  • 1
    $\begingroup$ Thank you, a much deeper answer than I was anticipating! To get to the approximation for $Y$ in terms of $Z$ and $\delta$, both numerator and denominator of the fraction on the left are multiplied by $\beta\gamma - \alpha\delta$ and then the terms in $\delta^2$ are separated off? $\endgroup$ – Adam Bailey Nov 14 '13 at 9:35
  • $\begingroup$ I don't know where the $\beta\gamma-\alpha\delta$ combination came from, but yes, the approximation (which was not needed for the result but helped motivate it) is $O(\delta^2).$ $\endgroup$ – whuber Nov 14 '13 at 15:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.