# How to perform regression with different error variances

I have two series of measurements values, first series is X and second is Y. I need to model Y as a function of X, where I know the method that was used to measure X is two times better then the method used for measure Y in terms of error variance.

I red about regression with error term in the independent variable, but I'm not sure what to do when there is also error term in the dependent variable and they are different.

Which technique/tools/constraints should I use for this sort of case?

• If the data is $X_i$ and $Y_i$, then you’re looking for the $a$, $b$, and $W_i$ that minimize $$\sum 2(X_i - W_i)^2 + (Y_i -aW_i -b)^2.$$ You should be able to solve this numerically, starting from the solution to ordinary least squares, i.e. with accurate $X_i$, i.e. with $X_i=W_i$, and then improving it. – Matt F. Aug 10 '19 at 6:12

So you know that $$X$$ is measured with half the error variance as $$Y$$. Then you should used weighted regression, and it is known that the optimal weights (for least squares) is proportional to inverse variance.

With useful software the regression function have have an argument for the weights. I R with lm that is the argument weights=. With your two responses in vectors x and y this can be:

n <- length(x);   m <- length(y)
Z <- c(x,y)
mod.wlm <- lm(Z ~ <your predictors>, weights=c(rep(0.5,n), rep(1,m)), ... )


Any good book about linear regression would have a chapter on weighted regression.

• I believe this answer is incorrect. The question concerns a form of regression in which one needs to account for errors in the independent variable $X.$ You seem to have some different interpretation in mind. – whuber Aug 10 '19 at 22:47

Well, in Bayesian world, you could do something like this (bugs/JAGS notation):

intcept ~ dnorm(0, 0.01)
sigma ~ dunif(0, 10)
X_SE ~ dunif(0, 10)
Y_SE <- X_SE * sqrt(2) # measurement error variance of X is supposed to be two times lower than of X

b ~ dnorm(0, 0.01)

for (i in 1:N) {
X_real[i] ~ dnorm(X[i], 1/X_SE^2)
Y_real_exp[i] <- intcept + b * X_real[i]
Y_real[i] ~ dnorm(Y_real_exp[i], 1/sigma^2)
Y[i] ~ dnorm(Y_real[i], 1/Y_SE^2)
}


The X[i] and Y[i] are your measurements; X_real and Y_real are the real values, which you don't know. X_SE and Y_SE are your measurement errors of X and Y. This is actually beautiful in bayesian models, that you can model this very easily. And the regression itself is done on those latent (unknown) values of X_real and Y_real. It is advised to standardize X.

Not sure how to do this in non-bayesian setting. Gaussian processes should be also able to handle uncertain input data, but I have no experience with that.

EDIT: I realized that this model would have an issue to identify the parameters X_SE and Y_SE. It can only be applied if you have some estimate how big these errors are since the model has no information how to tell how big these errors actually are.

• Could you show us how your solution uses the assumption that the error variance in $Y$ is twice that of $X$? – whuber Aug 10 '19 at 22:48