1
$\begingroup$

Suppose that I want to study the relationship between two variables $Y$ and $X$ using the linear model $Y \sim X$. Unfortunately, both $Y$ and $X$ are not normally distributed, say they are both skewed and have long right tails. So I'd do the trick of log-transforming both of them to make them look more like bell-shaped and do the regression.

Now I consider doing something quite different. Say I have $n$ independent observations of $(X, Y)$. Suppose that $n$ is a very large number. Now I'd do $k << n$ iterations of the following process to get $k$ independent observations whose both dependent and independent variables are guaranteed to be normally distributed: For each iteration $i$, I randomly sample $m$ pairs of $(y_i, x_i)$, I then do the log-transformation on those $m$ pairs and take the average, call them $(\bar{y}_i, \bar{x}_i)$. I finally do the linear regression on those means of samples $(\bar{y}_i, \bar{x}_i)$ -- there are $k$ of them.

My question is: Will the two methods described above produce the same results? That is, are they equivalent? Or are they completely different from each other? Which one would you choose if you want to look at the relationship between $Y$ and $X$ using a linear model?

Improve this question
$\endgroup$
5
  • $\begingroup$ It is clear the two approaches are different, because your proposal fails to obtain (correct) information about the uncertainty in the estimates. So in what sense do you mean "equivalent"? $\endgroup$
    – whuber ♦
    Commented Apr 1, 2013 at 18:29
  • $\begingroup$ "equivalent" means the linear regression results of the two methods (coefficient estimates, etc.) will be more or less the same. In other words, they both arrive at the same relationship between X and Y. $\endgroup$
    – Joe
    Commented Apr 1, 2013 at 18:33
  • $\begingroup$ Please explain how we are to construe "more or less": if that doesn't mean exactly the same, then how are we to determine how close is close enough? (We would have a built-in way to do that if your procedure provided accurate error estimates, but it does not.) $\endgroup$
    – whuber ♦
    Commented Apr 1, 2013 at 18:37
  • $\begingroup$ It seems like you could simulate date under one "model" and then estimate the relationship with the other. $\endgroup$
    – D L Dahly
    Commented Apr 1, 2013 at 18:43
  • $\begingroup$ For intuition, suppose $Y = X$ up to a relatively small (independent, homoscedastic) random error and that the values of $X$ are skewed. Then the values of $Y$, looking like those of $X$, will also be skewed: but you would have no reason to, and gain no benefit from, taking logarithms. For intuition about your method, consider that most of the averages $\bar{x}_i$ will be close to the mean of all the $x$'s and most of the $\bar{y}_i$ will be close to the mean of all the $y$'s. Thus any regression based on them will grossly extrapolate to the full range of $x$'s and probably be wildly wrong. $\endgroup$
    – whuber ♦
    Commented Apr 1, 2013 at 22:43

1 Answer 1

Reset to default
3
$\begingroup$

Suppose that I want to study the relationship between two variables Y and X using the linear model Y∼X. Unfortunately, both Y and X are not normally distributed,

There is no distributional assumption about the $X$'s in linear regression; they are simply conditioned on.

There is actually no distributional assumption about the $Y$'s either; they're only required to be conditionally normal (that is, the error term is assumed normal; the distribution of the $Y$'s will depend on the particular arrangement of the $X$'s). In any case it's only the distributional inferences that depend on the conditional distribution of the $Y$'s (and in large samples, perhaps not so much even then; prediction intervals would more strongly depend on normality of course).

So I'd do the trick of log-transforming both of them to make them look more like bell-shaped and do the regression.

Don't transform the $X$'s to make them bell shaped. Don't transform $Y$'s to make them bell-shaped. There may be a point to transforming the $Y$'s if it makes the residuals more normal but then you have to be very careful about what you're actually doing and why. (In any case, conditional heteroskedasticity may be more of an issue than nonnormality.) There can be a point to transforming the $X$'s but it isn't to make them normal - you might do it to make the regression more linear for example.

My question is: Will the two methods described above produce the same results? That is, are they equivalent?

No.

Or are they completely different from each other?

Completely? No. For example, they'd share a common mean point, I think.

Which one would you choose if you want to look at the relationship between Y and X using a linear model?

I'd think much more carefully about the point of what I was doing and the actual assumptions before attempting to solve a problem I don't necessarily have.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.