3
$\begingroup$

What transformation can be used to establish relationship between two variables whose scatter plots looks like the plot below?

I have applied log Y, log X, exp Y, exp . But a relationship could not be established with high R-squared.

enter image description here

$\endgroup$

2 Answers 2

2
$\begingroup$

Note that the spread of $y$ increases linearly as $x$ increases.

You also seem to have (at least approximately) a linear increase $y$ as $x$ increases.

That is $E(Y|x) = \beta_0+\beta_1x$ but also $\sigma(Y|x) = \gamma_0+\gamma_1x$.

The conditional distribution doesn't seem to be particularly far from uniform (this might be important if you're interested in more efficient estimates, or if you're interested in prediction intervals).

There are approaches for the general case, such as iteratively reweighted least squares approaches; alternatively one might write down a suitable model for the conditional mean, variance and error distribution and solve by maximum likelihood, but some special cases (which seem to occur fairly often) are worth discussing.


If it were the case (impossible to tell from your plot) that the spread was proportional to $x$, i.e. that $\gamma_0$ was $0$, then we could proceed as follows:

That is $Y_i = \beta_0+\beta_1x_i+x_i\epsilon_i$, for $i=1,2,...,n$, where $\text{Var}(\epsilon)=\sigma^2I$.

Now divide through by $x_i$:

$Y_i/x_i = \beta_0 (1/x_i)+\beta_1+\epsilon_i$

and let $Y^*_i=Y_i/x_i$ and $x^*_i = 1/x_i$ we have:

$Y^*_i = \beta_1+\beta_0 x^*_i+\epsilon_i$

a constant variance-regression with the roles of the two coefficients swapped. When it applies, this is highly convenient.

Another possibility for this $\gamma_0=0$ case if all $y$ values are positive is to use a GLM with variance proportional to mean-squared (the gamma), or a similar constant coefficient of variance model (such as a Weibull regression). Again, the conditional distributions are more nearly uniform, so these regressions won't be as efficient as a choice closer to that. A third possibility when the variables are both positive and the regression intercept is also $0$ is to take logs of both sides; the relationship should remain linear and the conditional variance should become constant, but the error distribution might be quite skew. If the intercept on the regression line were not zero but was known, a shift could be applied before taking logs.


It won't matter much what method you use to deal with this; you will not get a considerably better fit than just sticking a like through the middle of the data (roughly, it looks like the intercept is just below 3 and the slope just above 1):

scatterplot of data taken from the original post, showing fairly uniform conditional distributions; the low end of the y's is pretty flat near y=2; the high end goes from y=4 at x=0 to y=14 at x=5; approximate line of best fit drawn in.

Any line that approximates $E(Y|x)$ for these data simply won't have a high $R^2$ (not that I'd pay much attention to $R^2$ in the best of circumstances, but even less so with heterosckedastic data).

$\endgroup$
5
  • $\begingroup$ On applying the Yi/Xi and 1/Xi transformation I get the spread which looks like a rectangle. So, the prediction power is nearly equal to 0.1. Since there's no relationship that could be established between Yi/Xi and 1/Xi. $\endgroup$
    – vp_050
    Aug 15 at 14:11
  • $\begingroup$ I have revised the plot with values of X and Y and both are positive $\endgroup$
    – vp_050
    Aug 15 at 14:12
  • 1
    $\begingroup$ 1. Note that the intercept is the coefficient of interest in the transformed regression; it tells you about the slope in the original regression. As I said, the roles of the two variables swap. $\:$ 2. Clearly - now that we can see that when x=0 that the spread is not 0 - γ0 is not 0, so that approach may not work well at all. Consider estimating all of the parameters by maximum likelihood -- or at least to get an initial unweighted estimate of beta 0 and beta 1 and then estimate gamma 0 and gamma 1 and then use that to do a weighted regression (you can iterate; it should converge fast) $\endgroup$
    – Glen_b
    Aug 15 at 23:01
  • 1
    $\begingroup$ Note that you can directly estimate the intercept and slope by eye from the plot. Near x=0 the y-values are more-or-less uniform between 2 and 4 (perhaps slightly more points in the lower half but let's ignore that for the purpose of a rough assessment). The midrange at x=0 is about 3. Near x=5 the y-values go from about 2 to 14, and again are roughly uniform. The midrange is around 8. So the midrange y-values increase by 5 as x changes by 5; clearly the slope should be very close to 1. Similarly the intercept should be about 3. This gives a good reasonableness check on any estimates we get. $\endgroup$
    – Glen_b
    Aug 16 at 3:43
  • 1
    $\begingroup$ The slope may end up a tad higher than 1 and the intercept will likely tend to be just under 3, but 3 and 1 are very much in the right ballpark. $\endgroup$
    – Glen_b
    Aug 16 at 3:45
2
$\begingroup$

Based on that $x$ feature alone, the relationship seems pretty clear: increasing conditional mean of $y$ as $x$ increases, with the conditional variance of $y$ also increasing as $x$ increases.

I do not see any transformations or expansions of the $x$ feature (e.g., polynomials or splines) that will do much for you.

You say that you want a high $R^2$, but I do not believe you to have the features to achieve that goal with any kind of generalizability (e.g., out of sample performance, adjusted $R^2$). You are missing some component that drives $y$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.