9
$\begingroup$

Let's suppose I have a $N$-vector $Y$ of dependent variables, and an $N$-vector $X$ of independent variable. When $Y$ is plotted against $\frac{1}{X}$, I see that there is a linear relationship (upward trend) between the two. Now, this also means that there is a linear downward trend between $Y$ and $X$.

Now, if I run the regression: $Y = \beta * X + \epsilon$ and get the fitted value $\hat{Y} = \hat{\beta}X$

Then I run the regression: $Y = \alpha * \frac{1}{X} + \epsilon$ and get the fitted value $\tilde{Y} = \hat{\alpha} \frac{1}{X}$

Will the two predicted values, $\hat{Y}$and $\tilde{Y}$ be approximately equal?

$\endgroup$
9
$\begingroup$

 When Y is plotted against $\frac{1}{X}$, I see that there is a linear relationship (upward trend) between the two. Now, this also means that there is a linear downward trend between Y and X

The last sentence is wrong: there is a downward trend, but it is by no means linear: Y ~ 1 / X Y ~ X

I used a $f(x) = \frac{1}{x}$ as function plus a bit of noise on $Y$. As you can see, while plotting $Y$ against $\frac{1}{X}$ yields a linear behaviour, $Y$ against $X$ is far from linear.

(@whuber points out that the $Y$ against $\frac{1}{X}$ plot doesn't look homoscedastic. I think it appears to have higher variance for low $Y$ because the much higher case density leads to larger range which is essentially what we perceive. Actually, the data is homoscedastic: I used Y = 1 / X + rnorm (length (X), sd = 0.1) to generate the data, so no dependency on the size of $X$.)

So in general the relationship is very much non-linear. That is, unless your range of $X$ is so narrow that you can approximate $\frac{d \frac{1}{x}}{dx} = - \frac{1}{x^2} \approx const.$ Here's an example:

Y ~ 1 / X Y ~ X

Bottomline:

  • In general, it is very hard to approximate a $\frac{1}{X}$-type function by a linear or polynomial function. And without offset term you'll never get a reasonable approximation.
  • If the $X$ interval is narrow enough to allow a linear approximation, you'll anyways not be able from the data to guess the relation should be $\frac{1}{X}$ and not linear ($X$).
$\endgroup$
  • $\begingroup$ You begin with an invalid assumption: the OP never asserted $Y$ and $X$ are linearly related. The only assertion was that $Y$ and $1/X$ appear to be linearly related (with a negative slope). That, of course, indicates that $Y$ and $X$ are nonlinearly related. I think this is such a severe departure from what the question posits that the remainder of your post might only further mislead readers. $\endgroup$ – whuber Dec 4 '14 at 17:30
  • 2
    $\begingroup$ @whuber: I'm totally sorry but seem to be pretty dense right now. Question says: "When Y is plotted against 1/X, I see that there is a linear relationship (upward trend)". That's what I tried to depict in the 1st and 3rd image: Y over 1/X linearly increasing. I then plotted the corresponding Y over X (nonlinear, decreasing). Where do I misunderstand the OP? $\endgroup$ – cbeleites Dec 4 '14 at 17:40
  • 1
    $\begingroup$ Don't be sorry--I simply misread your post (by transposing the labels of the X axes in the first image)! The fault is all mine. I am therefore upvoting your answer, which is correct and informative. If you get a chance, though, you might wish to comment on the effect of this transformation on the homoscedasticity (or lack thereof) of the residuals (which can be detected in your $Y$ vs $1/X$ plot). $\endgroup$ – whuber Dec 4 '14 at 18:34
  • $\begingroup$ Thank you for the observations on homoscedasticity. By transforming the independent variable you do not change the homoscedasticity of the response--but its appearance certainly can change, as you point out, which is useful to know. (We have seen this phenomenon in several other posts, where people mis-attribute heteroscedasticity to mere differences in group populations, for instance.) $\endgroup$ – whuber Dec 4 '14 at 19:10
  • $\begingroup$ Very thorough answer and comments! Thanks @cbeleites and @whuber! $\endgroup$ – Mayou Dec 5 '14 at 13:30
2
$\begingroup$

I see no reason for them to be "approximately equal" in general -- but what exactly do you mean by approximately equal?

Here's a toy example:

library(ggplot2)
n <- 10^3
df <- data.frame(x=runif(n, min=1, max=2))
df$y <- 5 / df$x + rnorm(n)
p <- (ggplot(df, aes(x=x, y=y)) +
      geom_point() +
      geom_smooth(method="lm", formula=y ~ 0 + x) +  # Blue, OP's y hat
      geom_smooth(method="lm", formula=y ~ 0 + I(x^-1), color="red"))  # Red, OP's y tilde
p

The picture:

I'd say these are far from "approximately equal"

The "blue" model would do much better if it were allowed to have an intercept (i.e. constant) term...

$\endgroup$
  • $\begingroup$ It's hard to tell what you're doing with the blue model, but it certainly is nothing like what the OP describes! The red one is much closer to the situation presented in the question. $\endgroup$ – whuber Dec 4 '14 at 17:31
  • $\begingroup$ @cbeleites The lack of offset explains what otherwise was rather a mystery. Adrian, you are correct--but I doubt your example has much bearing on the OP's data. The post describes a situation in which the $Y$ vs $1/X$ plot (without any intercept) appears to be the good one, whereas you illustrate a situation where the good relationship is between $Y$ and $X$--and so of course the $Y$ vs $1/X$ fit is awful. $\endgroup$ – whuber Dec 4 '14 at 18:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.