How to account for measurement error when computing explained variance

Question

Let's say that I am fitting a linear regression model to predict random variable $Y$ based on random variable $X$, and I know as a fact that the only random variable that detemines the value of $Y$ is $X$. The value of $R^2$ describes how much of the variance in the samples from $Y$ is reduced by my model. The reason $R^2$ is not $1.0$ is two fold:

1. The linear model does not fully capture how $X$ affects $Y$
2. Measurement error

I want to remove the effect of measurement error from $R^2$, i.e. I like to know how much of the unexplained variance is caused because of the insufficiency of the model's complexity.

Let's say that for each $x$, I have multiple measurements of $y$ in the dataset. Can I use this to estimate the variance caused by measurement error and correct the value of $R^2$ accordingly? How does your answer generalize to deviance, for instance if I use GLM with Poisson distribution instead of the linear regression?

Answer 1

If the measurement error is only in $Y$, it won't bias the fitted model, but only add residual noise (lowering power). With replicated measurements at the same values of $X$, you can conduct a lack of fit test. You don't want a function of sufficient complexity to spear every conditional mean—since the measurement error is (believed to be) real, that guarantees overfitting. Instead, the degree to which the conditional mean of $Y|X=x_i$ should bounce randomly around your model's $\hat{y}_{x_i}$ due to measurement error alone. That's what the case with a lack of fit test assesses. This can be done with either a linear model (OLS) or a non-normal GLM (e.g., a Poisson regression). Fit a larger model by adding a dummy for each unique value of $X$ that has multiple replicates, then perform a nested model test. If it's significant, the dummies are adding information that was missed by your primary model. Here's a simple example, coded in R:

library(faraway)
data(corrosion)
d2 = sapply(with(corrosion, split(loss, Fe)), mean)
d2 = data.frame(Fe=f2n(names(d2)), loss=d2)
windows()
  plot(loss~Fe, corrosion)
  abline(lm(loss~Fe, corrosion), col="gray")
  text(1.75, 130, expression(R^2==0.9697))
  points(d2[,1], d2[,2], pch=3, col="lightcoral")

anova(lm(loss~Fe, corrosion), lm(loss~Fe+as.factor(Fe), corrosion))
# Analysis of Variance Table
# 
# Model 1: loss ~ Fe
# Model 2: loss ~ Fe + as.factor(Fe)
#   Res.Df     RSS Df Sum of Sq      F   Pr(>F)   
# 1     11 102.850                                
# 2      6  11.782  5    91.069 9.2756 0.008623 **
# ---
# Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1