Which parameter on x axis for linear regression?

Question

Here's a problem. I'm a geologist, so if you think you can help me... well, go slowly please.

I want to predict a continuous variable like porosity from remote sensing data. Let's say I have a measure of the reflectivity of the surface of the earth, densely sampled over an area. And I have rock porosity at some locations. So I'd like to calibrate the former to the latter, and transform reflectivity to porosity.

Since porosity p is the unknown, I instinctively plot it on the y-axis, against reflectivity r on the x. Then do a linear regression (let's pretend life is this simple for a mo), and get my equation $p = a r + b$, and go off on my merry way.

But this feels wrong. Reflectivity depends on porosity (in my hypothesis, and let's say that we know it does in nature). So in fact I should plot reflectivity on the y-axis, vs porosity on the x. Then get my linear equation, then rearrange to solve for $p = (r - b)/a$.

These approaches give different answers. I found this a bit surprising. Most people I've asked about this say, 'does it matter?'. It does, and the difference can be quite big (several percent).

Which one is the correct approach? Why? (For what it's worth, my answer is the latter approach, because it 'respects' nature, to anthropomorphize a bit... Not sure I can really explain myself any better).

Thanks for any insight anyone can offer,
Matt

Answer 1

As far as I can tell, you have a lot of reflectivity measurements and a few porosity measurements, and you want to estimate porosity in areas where you only have reflectivity information.

In that case, you want to regress porosity ($y$) on reflectivity ($x$) to minimise the sum of squares error in the porosity estimates. Any physical causal relationship should not affect this.

Answer 2

It's not a good idea to rearrange an OLS fit equation. If you do, you're changing the assumptions of what variables were assumed to be "independent" versus "dependent".

If you have to rearrange things, one way around this is to use Principal Component Analysis rather than Ordinary Least Squares. Here's an article that shows why this makes a difference:

http://www.cerebralmastication.com/2010/09/principal-component-analysis-pca-vs-ordinary-least-squares-ols-a-visual-explination/

Edit 1 ======================================

Oops, I forgot that I no longer filter these posts with the [r] tag and the code in that link is in R. So, I've got some explaining to do.

I don't know if you're using Excel or some other software to fit your data, but in R there's a lot more freedom. The dependent variable might be plotted on the horizontal (x) axis or the vertical (y) axis. It depends on how the code is written. In the above link, whenever you see:

 y ~ x

That is an R formula and it basically means that y is the "dependent" variable and x is the "independent" variable. When you see:

 x ~ y

That means that x is used as the "dependent" variable and y is the "independent" variable. When you see:

 lm(y ~ x)

That's where he did a least squares fit with y as the "dependent" variable and x as the "independent" variable. Obviously, the following:

 lm(x ~ y)

That's where he did a least squares fit with x as the "dependent" variable and y as the "independent" variable.

So, if you look closely at his code, you'll see that the RED line in the graph came from his least squares fit lm(y ~ x) where y is the "dependent" variable. And, the BLUE line came from lm(x ~ y) where x is the "dependent" variable.

Notice in the first graph that the two lines are NOT the same. That's because the Ordinary Least Squares (OLS) technique minimizes the error for the DEPENDENT variable. In his second graph, he shows the error (the distance from the dependent y variable to the RED line) in orange for the lm(y ~ x) case.

In the third graph, he shows the error (the distance from the dependent x variable to the BLUE line) in orange for the lm(x ~ y) case.

What does all of this mean? It means that if you OLS fit your data with p as the dependent variable, you'll get one set of coefficients. If you OLS fit your data with r as the dependent variable, you'll get a second set of coefficients. And, unless the data behaves in a very special way (where the red line is identical to the blue line), those sets of coefficients won't be related by a simple rearrangement of terms. If you want the ability to rearrange the equation, you have to do a fit that provides the proper coefficients. That's where the Principal Component Analysis method comes in.

So, if you fit your data using Ordinary Least Squares, and your output/calculated variable is porosity p, then you need to fit it with p as the dependent variable. If you want to "go both ways", then you'll need to use the above PCA method.