# How to account for measurement error when computing explained variance

Let's say that I am fitting a linear regression model to predict random variable $$Y$$ based on random variable $$X$$, and I know as a fact that the only random variable that detemines the value of $$Y$$ is $$X$$. The value of $$R^2$$ describes how much of the variance in the samples from $$Y$$ is reduced by my model. The reason $$R^2$$ is not $$1.0$$ is two fold:

1. The linear model does not fully capture how $$X$$ affects $$Y$$
2. Measurement error

I want to remove the effect of measurement error from $$R^2$$, i.e. I like to know how much of the unexplained variance is caused because of the insufficiency of the model's complexity.

Let's say that for each $$x$$, I have multiple measurements of $$y$$ in the dataset. Can I use this to estimate the variance caused by measurement error and correct the value of $$R^2$$ accordingly? How does your answer generalize to deviance, for instance if I use GLM with Poisson distribution instead of the linear regression?

• You are assuming that there is no irreducible variation in the universe (or at least in this system)? Is there measurement error in $X$, $Y$, or both? Do you have an external estimate of the measurement error? – gung - Reinstate Monica Feb 24 at 15:44
• Yes, let's say that there is no variation in the system. The measurement error is only in $Y$, and there is no direct estimate of it. I am asking to estimate it based on different values of $y$ for a single $x$. – Feri Feb 24 at 18:04

If the measurement error is only in $$Y$$, it won't bias the fitted model, but only add residual noise (lowering power). With replicated measurements at the same values of $$X$$, you can conduct a lack of fit test. You don't want a function of sufficient complexity to spear every conditional mean—since the measurement error is (believed to be) real, that guarantees overfitting. Instead, the degree to which the conditional mean of $$Y|X=x_i$$ should bounce randomly around your model's $$\hat{y}_{x_i}$$ due to measurement error alone. That's what the case with a lack of fit test assesses. This can be done with either a linear model (OLS) or a non-normal GLM (e.g., a Poisson regression). Fit a larger model by adding a dummy for each unique value of $$X$$ that has multiple replicates, then perform a nested model test. If it's significant, the dummies are adding information that was missed by your primary model. Here's a simple example, coded in R:

library(faraway)
data(corrosion)
d2 = sapply(with(corrosion, split(loss, Fe)), mean)
d2 = data.frame(Fe=f2n(names(d2)), loss=d2)
windows()
plot(loss~Fe, corrosion)
abline(lm(loss~Fe, corrosion), col="gray")
text(1.75, 130, expression(R^2==0.9697))
points(d2[,1], d2[,2], pch=3, col="lightcoral")

anova(lm(loss~Fe, corrosion), lm(loss~Fe+as.factor(Fe), corrosion))
# Analysis of Variance Table
#
# Model 1: loss ~ Fe
# Model 2: loss ~ Fe + as.factor(Fe)
#   Res.Df     RSS Df Sum of Sq      F   Pr(>F)
# 1     11 102.850
# 2      6  11.782  5    91.069 9.2756 0.008623 **
# ---
# Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


• Thank you. I lack some background and need to read more to fully understand how your solution provides me with the value I want. But one quick question. Did you assume in your solution that the measurement error is from a normal distribution? Does it also work if it is a Poisson distribution? (e.g if the model is simple linear regression but we know for a fact that we have measurement error in form of a Poisson distribution with mean equal to the estimation of the linear model) – Feri Feb 24 at 19:12
• I'm not sure I follow that, @Feri. If you are fitting a Poisson GLM, that's fine, you can do the same thing as shown here. If you are fitting Poisson data w/ a linear regression, you are likely to have a poor model. – gung - Reinstate Monica Feb 24 at 19:35