13
$\begingroup$

I am looking for methods which can be used to estimate the "OLS" measurement error model.

$$y_{i}=Y_{i}+e_{y,i}$$ $$x_{i}=X_{i}+e_{x,i}$$ $$Y_{i}=\alpha + \beta X_{i}$$

Where the errors are independent normal with unknown variances $\sigma_{y}^{2}$ and $\sigma_{x}^{2}$. "Standard" OLS won't work in this case.

Wikipedia has some unappealing solutions - the two given force you to assume that either the "variance ratio" $\delta=\frac{\sigma_{y}^{2}}{\sigma_{x}^{2}}$ or the "reliability ratio" $\lambda=\frac{\sigma_{X}^{2}}{\sigma_{x}^{2}+\sigma_{X}^{2}}$ is known, where $\sigma_{X}^2$ is the variance of the true regressor $X_i$. I am not satisfied by this, because how can someone who doesn't know the variances know their ratio?

Anyways, are there any other solutions besides these two which don't require me to "know" anything about the parameters?

Solutions for just the intercept and slope are fine.

$\endgroup$
  • $\begingroup$ the Wikipedia article itself provides you the answer to this question. If you assume normality of the "true" regressor, then you need further conditions on the distributions of the errors. If the true regressor is not Gaussian, then you have some hope. See Reiersol (1950). $\endgroup$ – cardinal Feb 26 '11 at 17:07
  • $\begingroup$ also, what do you mean by "Solutions for just the intercept and slope are fine". Those are your only two parameters! Or were you hoping to try to back out the "true" regressor as well? $\endgroup$ – cardinal Feb 26 '11 at 17:10
  • $\begingroup$ @cardinal - I meant that I didn't particularly care about the two scale parameters, and as you say, the "true" regressor $X_{i}$. $\endgroup$ – probabilityislogic Feb 26 '11 at 23:40
  • $\begingroup$ I see. That makes sense. $\endgroup$ – cardinal Feb 26 '11 at 23:45
7
$\begingroup$

There are a range of possibilities described by J.W. Gillard in An Historical Overview of Linear Regression with Errors in both Variables

If you are not interested in details or reasons for choosing one method over another, just go with the simplest, which is to draw the line through the centroid $(\bar{x},\bar{y})$ with slope $\hat{\beta}=s_y/s_x$, i.e. the ratio of the observed standard deviations (making the sign of the slope the same as the sign of the covariance of $x$ and $y$); as you can probably work out, this gives an intercept on the $y$-axis of $\hat{\alpha}=\bar{y}-\hat{\beta}\bar{x}.$

The merits of this particular approach are

  1. it gives the same line comparing $x$ against $y$ as $y$ against $x$,
  2. it is scale-invariant so you do not need to worry about units,
  3. it lies between the two ordinary linear regression lines
  4. it crosses them where they cross each other at the centroid of the observations, and
  5. it is very easy to calculate.

The slope is the geometric mean of the slopes of the two ordinary linear regression slopes. It is also what you would get if you standardised the $x$ and $y$ observations, drew a line at 45° (or 135° if there is negative correlation) and then de-standardised the line. It could also be seen as equivalent to making an implicit assumption that the variances of the two sets of errors are proportional to the variances of the two sets of observations; as far as I can tell, you claim not to know which way this is wrong.

Here is some R code to illustrate: the red line in the chart is OLS regression of $Y$ on $X$, the blue line is OLS regression of $X$ on $Y$, and the green line is this simple method. Note that the slope should be about 5.

X0 <- 1600:3600
Y0 <- 5*X0 + 700
X1 <- X0 + 400*rnorm(2001)
Y1 <- Y0 + 2000*rnorm(2001)
slopeOLSXY  <- lm(Y1 ~ X1)$coefficients[2]     #OLS slope of Y on X
slopeOLSYX  <- 1/lm(X1 ~ Y1)$coefficients[2]   #Inverse of OLS slope of X on Y
slopesimple <- sd(Y1)/sd(X1) *sign(cov(X1,Y1)) #Simple slope
c(slopeOLSXY, slopeOLSYX, slopesimple)         #Show the three slopes
plot(Y1~X1)
abline(mean(Y1) - slopeOLSXY  * mean(X1), slopeOLSXY,  col="red")
abline(mean(Y1) - slopeOLSYX  * mean(X1), slopeOLSYX,  col="blue")
abline(mean(Y1) - slopesimple * mean(X1), slopesimple, col="green")
$\endgroup$
  • $\begingroup$ @Henry, your definition of $\hat{\beta}$ doesn't make any sense to me. Are some "hats" missing? $\endgroup$ – cardinal Feb 26 '11 at 23:34
  • $\begingroup$ It is mean to be the observed standard deviation of $\{y_i\}$ divided by the observed standard deviation of $\{x_i\}$. I'll change $\sigma$ to $s$ $\endgroup$ – Henry Feb 26 '11 at 23:58
  • $\begingroup$ @Henry, can you clarify some of your comments? Something strikes me as being off based on your current description. Let $\hat{\beta}_{xy}$ be the slope assuming $y$ is the response and $x$ is the predictor. Let $\hat{\beta}_{yx}$ be the slope assuming $x$ is the response and $y$ the predictor. Then $\hat{\beta}_{xy} = \hat{\rho}s_y / s_x$ and $\hat{\beta}_{yx} = \hat{\rho} s_x / s_y$, where $\hat{\rho}$ is the sample correlation between $x$ and $y$. Hence the geometric mean of these two slope estimates is just $\hat{\rho}$. $\endgroup$ – cardinal Feb 27 '11 at 0:15
  • $\begingroup$ @cardinal: No - when I see $x = by+c$ I mean the slope is $1/b$ since it can be rewritten as $y=x/b-c/b$. When you try to draw the two OLS lines on the same graph together with the observed points (e.g. with $y$ on the vertical axis and $x$ on the horizontal axis) you have to invert one of the slopes. So I meant that you take the geometric mean of $\hat{\rho}s_y/s_x$ and $s_y/\hat{\rho}s_x$, which is simply $s_y/s_x$. Or, if you are unconventional enough to plot $y$ and $x$ the other way round for both lines and the observed points, then you get the inverse of that as the slope. $\endgroup$ – Henry Feb 27 '11 at 0:39
  • $\begingroup$ @Henry - that's quite an interesting answer. I don't necessarily doubt its validity, but one thing which does surprise me is that the correlation/covariance between $Y$ and $X$ is completely absent from the answer. Surely this should be relevant to the answer? $\endgroup$ – probabilityislogic Feb 27 '11 at 1:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.