I'm not sure which method to use to model the relationship between two variables ($x$ and $y$) in the experiment described as follows:

  • There are 3 variables: $x_{aim}$, $x$ and $y$.
  • The value of $x_{aim}$ is set when operating the experiment. However, $x$ and $x_{aim}$ aren't always equal.
  • The Pearson's correlation coefficient between $x_{aim}$ and $x$ is about 0.9.
  • The Pearson's correlation coefficient between $x$ and $y$ is much less: about 0.5.
  • $y$ has a maximum possible value ($y_{max}$) which can't be exceeded.
  • Each data point is obtained after setting $x_{aim}$ and reading $x$ and $y$.

Although the Pearson's correlation coefficient between $x$ and $y$ isn't great, it looks like $y$ tends to increase with $x$.

After doing simple linear regressions of $y=f(x)$ and $x=g(y)$ (and converting the latter back as $g^{-1}$, so as to be displayed on the same graph as $f$ for example), both slopes are positive, but the slope of $g^{-1}$ is greater than that of $f$.

Does it make sense to say $x_{max} = f^{-1}(y_{max})$ or $x_{max} = g(y_{max})$? ($x_{max}$ would be reached earlier in the second case.)

Considering that $y$ is bound by $y_{max}$, what can be said about the possible maximum value of $x$ that could be reached?

As far as I understand, it makes sense to do a linear regression of the form $y=f(x)$ when $x$ is the independent variable and $y$ is the dependent variable. However, in this context, I'm not sure whether it makes sense to consider that $x$ is independent and $y$ is dependent.

Would a total least square regression be more appropriate? Are there other methods to determine which values of $x_{max}$ can be reached (and with which likelihood)?

(If this matters, $x$ and $y$ don't seem to follow a normal distribution, as more attempts have been made to try to reach higher values of $x$.)

  • $\begingroup$ What will you do with this relationship, if you'll find it? Will you test the hypotheses, or are just interested how it looks? If there are a lot of data points, you should consider non-linear models. $\endgroup$ – mpiktas Oct 7 '11 at 12:26
  • $\begingroup$ @mpiktas, ultimately, I'd like to know which x_max is a reasonable target that I could try to reach on a regular basis (not just once), considering that reaching or going above y_max makes the experiment void (effectively implying x=x_min for that attempt). $\endgroup$ – Bruno Oct 7 '11 at 19:08
  • $\begingroup$ Total least squares (or errors-in-variables) regression is indicated when the variance of $x$ becomes sizable compared to that of $y$. The 90% correlation with $x_\text{aim}$ suggests that variance of $x$ may be sufficiently small that you can safely treat it as an independent variable. This is something you can check post-regression by comparing the RMSE of residuals of $x_\text{aim}$ vs. $x$ to the RMSEs of residuals of $y$ vs. $x_\text{aim}$. Whether $y_\text{max}$ is a problem depends; if you see an upper cutoff in the scatterplot with $x_\text{aim}$, it's an important consideration. $\endgroup$ – whuber Dec 19 '11 at 20:56

I want to second @King's points. It is very intuitive to suspect that regressing $y$ onto $x$ ('direct regression') and regressing $x$ onto $y$ ('reverse regression') ought to be the same. However, this is neither true mathematically nor with respect to how the regression is related to the situation you're analyzing. If you plot $y$ on the vertical axis of a graph and $x$ on the horizontal axis, you can see what's happening. Direct regression finds the line that minimizes the vertical distances between the data points and the line, whereas reverse regression minimizes the horizontal distances. The line that minimizes the one will only minimize the other if $r_{xy}=1.0$. You need to decide what you want to explain, and what you want to use to explain it. The answer to that question gives you which variable is $y$ and $x$ and specifies your model. Also, (again following @King), I disagree with trying to say $x_{max}=f^{-1}(y_{max})$, for the same reasons.

Regarding the issue of a bounded variable, typically it is conceivable that the 'real' amount could go higher, but that you just can't measure it. For example, an outside thermometer out my window goes up to 120, but it could be 140 outside in some places, and you would only have 120 as your measurement. Thus, the variable would have an upper bound, but the thing you really wanted to think about doesn't. If this is the case, tobit models exist for just such situations.

Another approach would be to use something more robust like loess, which may be perfectly adequate for your needs.

  • $\begingroup$ Apologies for the delay, I hadn't noticed your answer. I'll need to read up about the Tobit model. $\endgroup$ – Bruno May 24 '12 at 14:26
  • $\begingroup$ No problem. For more on the nature of regression (vs. reverse regression) see here. For some help w/ applying tobit regression using various software try here. $\endgroup$ – gung - Reinstate Monica May 24 '12 at 14:40

Firstly, I don't think it makes sense to say $x_{max}=f^{-1}(y_{max})$ here, that's like implying that it's a one-to-one function although $x_{max}$ is explained by other unobserved variables.

Secondly, it really depends on the context for which one to treat as an independent or dependent variable. From my experience, unless theory strongly suggests one way; either way is ok. From your comments on Oct 7, it seems like $x$ is the dependent while $y$ is the independent.

If possible, look at the residuals and see if you can squeeze anything out of it. There could be another variable that you forgot; or it may help to transform your variables.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.