Skip to main content
Added R example
Source Link
Corvus
  • 5.4k
  • 1
  • 31
  • 61

[Heavily edited/extended in light of comments. This is my understanding of this area of statistics - given the comments so far, I suspect this may be disputed.]

Prediction and Forecasting

Calibration is an intuitively sensible requirement of an instrument, but it can also cause confusion. Notice, that even a well calibrated instrument will not be showing you the expected value of $Y$! To get $E[Y|X]$ you still need to do the Y-on-X regression, even with a well calibrated instrument. This estimate will generally look like a shrunk version of the instrument value (remember the $\gamma$ term that crept in). In particular, to get a really good estimate of $E[Y|X]$ you should include your prior knowledge of the distribution of $Y$. This then leads to concepts such as regression-to-the-mean and empirical bayes.

Example in R One way to get a feel for what is going on here is to make some data and try the methods out. The code below compares X-on-Y with Y-on-X for prediction and calibration and you can quickly see that X-on-Y is no good for the prediction model, but is the correct procedure for calibration.

library(data.table)
library(ggplot2)

N = 100
beta = 0.7
c = 4.4

DT = data.table(Y = rt(N, 5), epsilon = rt(N,8))
DT[, X := 0.7*Y + c + epsilon]

YonX = DT[, lm(Y~X)]   # Y = alpha_1 X + alpha_0 + eta
XonY = DT[, lm(X~Y)]   # X = beta_1 Y + beta_0 + epsilon


YonX.c = YonX$coef[1]   # c = alpha_0
YonX.m = YonX$coef[2]   # m = alpha_1

# For X on Y will need to rearrage after the fit.
# Fitting model X = beta_1 Y + beta_0
# Y = X/beta_1 - beta_0/beta_1

XonY.c = -XonY$coef[1]/XonY$coef[2]      # c = -beta_0/beta_1
XonY.m = 1.0/XonY$coef[2]  # m = 1/ beta_1
   
ggplot(DT, aes(x = X, y =Y)) + geom_point() +  geom_abline(intercept = YonX.c, slope = YonX.m, color = "red")  +  geom_abline(intercept = XonY.c, slope = XonY.m, color = "blue")

# Generate a fresh sample

DT2 = data.table(Y = rt(N, 5), epsilon = rt(N,8))
DT2[, X := 0.7*Y + c + epsilon]

DT2[, YonX.predict := YonX.c + YonX.m * X]
DT2[, XonY.predict := XonY.c + XonY.m * X]

cat("YonX sum of squares error for prediction: ", DT2[, sum((YonX.predict - Y)^2)])
cat("XonY sum of squares error for prediction: ", DT2[, sum((XonY.predict - Y)^2)])

# Generate lots of samples at the same Y

DT3 = data.table(Y = 4.0, epsilon = rt(N,8))
DT3[, X := 0.7*Y + c + epsilon]

DT3[, YonX.predict := YonX.c + YonX.m * X]
DT3[, XonY.predict := XonY.c + XonY.m * X]

cat("Expected value of X at a given Y (calibrated using YonX) should be close to 4: ", DT3[, mean(YonX.predict)])
cat("Expected value of X at a gievn Y (calibrated using XonY) should be close to 4: ", DT3[, mean(XonY.predict)])

ggplot(DT3) + geom_density(aes(x = YonX.predict), fill = "red", alpha = 0.5) + geom_density(aes(x = XonY.predict), fill = "blue", alpha = 0.5) + geom_vline(x = 4.0, size = 2) + ggtitle("Calibration at 4.0")

The two regression lines are plotted over the data

enter image description here

And then the sum of squares error for Y is measured for both fits on a new sample.

> cat("YonX sum of squares error for prediction: ", DT2[, sum((YonX.predict - Y)^2)])
YonX sum of squares error for prediction:  77.33448
> cat("XonY sum of squares error for prediction: ", DT2[, sum((XonY.predict - Y)^2)])
XonY sum of squares error for prediction:  183.0144

Alternatively a sample can be generated at a fixed Y (in this case 4) and then average of those estimates taken. You can now see that the Y-on-X predictor is not well calibrated having an expected value much lower than Y. The X-on-Y predictor, is well calibrated having an expected value close to Y.

> cat("Expected value of X at a given Y (calibrated using YonX) should be close to 4: ", DT3[, mean(YonX.predict)])
Expected value of X at a given Y (calibrated using YonX) should be close to 4:  1.305579
> cat("Expected value of X at a gievn Y (calibrated using XonY) should be close to 4: ", DT3[, mean(XonY.predict)])
Expected value of X at a gievn Y (calibrated using XonY) should be close to 4:  3.465205

The distribution of the two prediction can been seen in a density plot.

enter image description here

[Heavily edited/extended in light of comments. This is my understanding of this area of statistics - given the comments so far, I suspect this may be disputed.]

Prediction and Forecasting

Calibration is an intuitively sensible requirement of an instrument, but it can also cause confusion. Notice, that even a well calibrated instrument will not be showing you the expected value of $Y$! To get $E[Y|X]$ you still need to do the Y-on-X regression, even with a well calibrated instrument. This estimate will generally look like a shrunk version of the instrument value (remember the $\gamma$ term that crept in). In particular, to get a really good estimate of $E[Y|X]$ you should include your prior knowledge of the distribution of $Y$. This then leads to concepts such as regression-to-the-mean and empirical bayes.

Prediction and Forecasting

Calibration is an intuitively sensible requirement of an instrument, but it can also cause confusion. Notice, that even a well calibrated instrument will not be showing you the expected value of $Y$! To get $E[Y|X]$ you still need to do the Y-on-X regression, even with a well calibrated instrument. This estimate will generally look like a shrunk version of the instrument value (remember the $\gamma$ term that crept in). In particular, to get a really good estimate of $E[Y|X]$ you should include your prior knowledge of the distribution of $Y$. This then leads to concepts such as regression-to-the-mean and empirical bayes.

Example in R One way to get a feel for what is going on here is to make some data and try the methods out. The code below compares X-on-Y with Y-on-X for prediction and calibration and you can quickly see that X-on-Y is no good for the prediction model, but is the correct procedure for calibration.

library(data.table)
library(ggplot2)

N = 100
beta = 0.7
c = 4.4

DT = data.table(Y = rt(N, 5), epsilon = rt(N,8))
DT[, X := 0.7*Y + c + epsilon]

YonX = DT[, lm(Y~X)]   # Y = alpha_1 X + alpha_0 + eta
XonY = DT[, lm(X~Y)]   # X = beta_1 Y + beta_0 + epsilon


YonX.c = YonX$coef[1]   # c = alpha_0
YonX.m = YonX$coef[2]   # m = alpha_1

# For X on Y will need to rearrage after the fit.
# Fitting model X = beta_1 Y + beta_0
# Y = X/beta_1 - beta_0/beta_1

XonY.c = -XonY$coef[1]/XonY$coef[2]      # c = -beta_0/beta_1
XonY.m = 1.0/XonY$coef[2]  # m = 1/ beta_1
   
ggplot(DT, aes(x = X, y =Y)) + geom_point() +  geom_abline(intercept = YonX.c, slope = YonX.m, color = "red")  +  geom_abline(intercept = XonY.c, slope = XonY.m, color = "blue")

# Generate a fresh sample

DT2 = data.table(Y = rt(N, 5), epsilon = rt(N,8))
DT2[, X := 0.7*Y + c + epsilon]

DT2[, YonX.predict := YonX.c + YonX.m * X]
DT2[, XonY.predict := XonY.c + XonY.m * X]

cat("YonX sum of squares error for prediction: ", DT2[, sum((YonX.predict - Y)^2)])
cat("XonY sum of squares error for prediction: ", DT2[, sum((XonY.predict - Y)^2)])

# Generate lots of samples at the same Y

DT3 = data.table(Y = 4.0, epsilon = rt(N,8))
DT3[, X := 0.7*Y + c + epsilon]

DT3[, YonX.predict := YonX.c + YonX.m * X]
DT3[, XonY.predict := XonY.c + XonY.m * X]

cat("Expected value of X at a given Y (calibrated using YonX) should be close to 4: ", DT3[, mean(YonX.predict)])
cat("Expected value of X at a gievn Y (calibrated using XonY) should be close to 4: ", DT3[, mean(XonY.predict)])

ggplot(DT3) + geom_density(aes(x = YonX.predict), fill = "red", alpha = 0.5) + geom_density(aes(x = XonY.predict), fill = "blue", alpha = 0.5) + geom_vline(x = 4.0, size = 2) + ggtitle("Calibration at 4.0")

The two regression lines are plotted over the data

enter image description here

And then the sum of squares error for Y is measured for both fits on a new sample.

> cat("YonX sum of squares error for prediction: ", DT2[, sum((YonX.predict - Y)^2)])
YonX sum of squares error for prediction:  77.33448
> cat("XonY sum of squares error for prediction: ", DT2[, sum((XonY.predict - Y)^2)])
XonY sum of squares error for prediction:  183.0144

Alternatively a sample can be generated at a fixed Y (in this case 4) and then average of those estimates taken. You can now see that the Y-on-X predictor is not well calibrated having an expected value much lower than Y. The X-on-Y predictor, is well calibrated having an expected value close to Y.

> cat("Expected value of X at a given Y (calibrated using YonX) should be close to 4: ", DT3[, mean(YonX.predict)])
Expected value of X at a given Y (calibrated using YonX) should be close to 4:  1.305579
> cat("Expected value of X at a gievn Y (calibrated using XonY) should be close to 4: ", DT3[, mean(XonY.predict)])
Expected value of X at a gievn Y (calibrated using XonY) should be close to 4:  3.465205

The distribution of the two prediction can been seen in a density plot.

enter image description here

Greatly extended to encompass the discussions from comments
Source Link
Corvus
  • 5.4k
  • 1
  • 31
  • 61

[Heavily edited/extended in light of comments. This is my understanding of this area of statistics - given the comments so far, I suspect this may be disputed.]

Prediction and Forecasting

Yes you are correct, when you view this as a problem of prediction, a Y-on-X regression will give you a model such that given a instrument measurement you can make an unbiased estimate of the accurate lab measurement, without doing the lab procedure.

However This may seem counter-intuitive because the error structure is not the "real" one. Assuming that the lab method is a gold standard error free method, if you know something aboutthen we "know" that the true data-generative model is

$X_i = \beta Y_i + \epsilon_i$

where $Y_i$ and $\epsilon_i$ are independent identically distribution, and $E[\epsilon]=0$

We are interested in getting the best estimate of Y$E[Y_i|X_i]$. Because of our independence assumption we can rearrange the above:

$Y_i = \frac{X_i - \epsilon}{\beta}$

Now, then you could do an X-ontaking expectations given $X_i$ is where things get hairy

$E[Y_i|X_i] = \frac{1}{\beta} X_i - \frac{1}{\beta} E[\epsilon_i|X_i]$

The problem is the $E[\epsilon_i|X_i]$ term -Y regression is it equal to zero? It doesn't actually matter, because you can never see it, and use that in your inferencewe are only modelling linear terms (or the argument extend up to whatever terms you are modelling). Any dependence between $\epsilon$ and $X$ can simply be absorbed into the constant we are estimating.

For exampleExplicitly, suppose thatwithout loss of generality we know that Y comes from some distributioncan let

$\epsilon_i = \gamma X_i + \eta_i$

Where $E[\eta_i|X] = 0$ by definition, butso that you are interested in flagging patients whosewe now have a Y > 100 say. It

$Y_I = \frac{1}{\beta} X_i - \frac{\gamma}{\beta} X_i - \frac{1}{\beta} \eta_i$

$Y_I = \frac{1-\gamma}{\beta} X_i - \frac{1}{\beta} \eta_i $

which satisfies all the requirements of OLS, since $\eta$ is now not necessarilyexogenous. It doesn't matter in the optimal approach toslightest that the error term also contains a $\beta$ since neither $\beta$ nor $\sigma$ are known anyway and must be estimated. We can therefore simply takereplace those constants with new ones and use the normal approach

$Y_I = {\alpha} X_i + \eta_i $

Notice that we have NOT estimated the quantity $\beta$ that I originally wrote down - we have built the best model we can for using X as a proxy for Y.

expectedInstrument Analysis value of Y given X

The person who set you this question, but instead to compare the likelihood ofclearly didn't want the two hypotheses (Y>100 or Y<100). You now need to doanswer above since they say X-on-Y regression, because you want to know whatis the distribution of X would be for any given Ycorrect method, so why might they have wanted that you can compare? Most likely they were considering the alternatives. This is particularly important when you classification costs are very imbalanced (as is oftentask of understanding the caseinstrument. As discussed in medicine) sinceVincent's answer, if you often much prefer falsely positiveswant to know about they want the false negatives. Itinstrument behaves, the X-on-Y is betterthe way to send someone's samplego.

Going back to the lab for extra analysis and it come back negative, than not send it off when it was actually positive.first equation above:

If$X_i = \beta Y_i + \epsilon_i$

The person setting the variabilityquestion could have been thinking of youcalibration. An instrument changes dramatically over the range of Y this canis said to be also very important.

Finally, as an aside (sincecalibrated when it wasn't inhas expectation equal to the original question), iftrue value - that is $E[X_i|Y_i] = Y_i$. Clearly in order to calibrate $X$ you wereneed to do multiple measurements on one patientfind $\beta$, and get one lab result Y, thenso to calibrate an instrument you mustneed to do X-on-Y regression. The fact that Y

Shrinkage

Calibration is shared across multiple X's meansan intuitively sensible requirement of an instrument, but it can also cause confusion. Notice, that you wouldn't have independent residuals ineven a well calibrated instrument will not be showing you the expected value of $Y$! To get $E[Y|X]$ you still need to do the Y-on-X regression, even with a well calibrated instrument. This estimate will generally look like a shrunk version of the instrument value (remember the $\gamma$ term that crept in). In particular, to get a really good estimate of $E[Y|X]$ you should include your prior knowledge of the distribution of $Y$. This then leads to concepts such as regression-to-the-mean and empirical bayes.

Yes you are correct, a Y-on-X regression will give you a model such that given a instrument measurement you can make an unbiased estimate of the accurate lab measurement, without doing the lab procedure.

However, if you know something about the distribution of Y, then you could do an X-on-Y regression and use that in your inference.

For example, suppose that we know that Y comes from some distribution, but that you are interested in flagging patients whose have a Y > 100 say. It is now not necessarily the optimal approach to simply take the expected value of Y given X, but instead to compare the likelihood of the two hypotheses (Y>100 or Y<100). You now need to do X-on-Y regression, because you want to know what the distribution of X would be for any given Y so that you can compare the alternatives. This is particularly important when you classification costs are very imbalanced (as is often the case in medicine) since you often much prefer falsely positives the false negatives. It is better to send someone's sample to the lab for extra analysis and it come back negative, than not send it off when it was actually positive.

If the variability of you instrument changes dramatically over the range of Y this can be also very important.

Finally, as an aside (since it wasn't in the original question), if you were to do multiple measurements on one patient, and get one lab result Y, then you must do X-on-Y. The fact that Y is shared across multiple X's means that you wouldn't have independent residuals in a Y-on-X regression.

[Heavily edited/extended in light of comments. This is my understanding of this area of statistics - given the comments so far, I suspect this may be disputed.]

Prediction and Forecasting

Yes you are correct, when you view this as a problem of prediction, a Y-on-X regression will give you a model such that given a instrument measurement you can make an unbiased estimate of the accurate lab measurement, without doing the lab procedure.

This may seem counter-intuitive because the error structure is not the "real" one. Assuming that the lab method is a gold standard error free method, then we "know" that the true data-generative model is

$X_i = \beta Y_i + \epsilon_i$

where $Y_i$ and $\epsilon_i$ are independent identically distribution, and $E[\epsilon]=0$

We are interested in getting the best estimate of $E[Y_i|X_i]$. Because of our independence assumption we can rearrange the above:

$Y_i = \frac{X_i - \epsilon}{\beta}$

Now, taking expectations given $X_i$ is where things get hairy

$E[Y_i|X_i] = \frac{1}{\beta} X_i - \frac{1}{\beta} E[\epsilon_i|X_i]$

The problem is the $E[\epsilon_i|X_i]$ term - is it equal to zero? It doesn't actually matter, because you can never see it, and we are only modelling linear terms (or the argument extend up to whatever terms you are modelling). Any dependence between $\epsilon$ and $X$ can simply be absorbed into the constant we are estimating.

Explicitly, without loss of generality we can let

$\epsilon_i = \gamma X_i + \eta_i$

Where $E[\eta_i|X] = 0$ by definition, so that we now have

$Y_I = \frac{1}{\beta} X_i - \frac{\gamma}{\beta} X_i - \frac{1}{\beta} \eta_i$

$Y_I = \frac{1-\gamma}{\beta} X_i - \frac{1}{\beta} \eta_i $

which satisfies all the requirements of OLS, since $\eta$ is now exogenous. It doesn't matter in the slightest that the error term also contains a $\beta$ since neither $\beta$ nor $\sigma$ are known anyway and must be estimated. We can therefore simply replace those constants with new ones and use the normal approach

$Y_I = {\alpha} X_i + \eta_i $

Notice that we have NOT estimated the quantity $\beta$ that I originally wrote down - we have built the best model we can for using X as a proxy for Y.

Instrument Analysis

The person who set you this question, clearly didn't want the answer above since they say X-on-Y is the correct method, so why might they have wanted that? Most likely they were considering the task of understanding the instrument. As discussed in Vincent's answer, if you want to know about they want the instrument behaves, the X-on-Y is the way to go.

Going back to the first equation above:

$X_i = \beta Y_i + \epsilon_i$

The person setting the question could have been thinking of calibration. An instrument is said to be calibrated when it has expectation equal to the true value - that is $E[X_i|Y_i] = Y_i$. Clearly in order to calibrate $X$ you need to find $\beta$, and so to calibrate an instrument you need to do X-on-Y regression.

Shrinkage

Calibration is an intuitively sensible requirement of an instrument, but it can also cause confusion. Notice, that even a well calibrated instrument will not be showing you the expected value of $Y$! To get $E[Y|X]$ you still need to do the Y-on-X regression, even with a well calibrated instrument. This estimate will generally look like a shrunk version of the instrument value (remember the $\gamma$ term that crept in). In particular, to get a really good estimate of $E[Y|X]$ you should include your prior knowledge of the distribution of $Y$. This then leads to concepts such as regression-to-the-mean and empirical bayes.

Source Link
Corvus
  • 5.4k
  • 1
  • 31
  • 61

Yes you are correct, a Y-on-X regression will give you a model such that given a instrument measurement you can make an unbiased estimate of the accurate lab measurement, without doing the lab procedure.

Put another way, if you are just interested in $E[Y|X]$ then you want Y-on-X regression.

However, if you know something about the distribution of Y, then you could do an X-on-Y regression and use that in your inference.

For example, suppose that we know that Y comes from some distribution, but that you are interested in flagging patients whose have a Y > 100 say. It is now not necessarily the optimal approach to simply take the expected value of Y given X, but instead to compare the likelihood of the two hypotheses (Y>100 or Y<100). You now need to do X-on-Y regression, because you want to know what the distribution of X would be for any given Y so that you can compare the alternatives. This is particularly important when you classification costs are very imbalanced (as is often the case in medicine) since you often much prefer falsely positives the false negatives. It is better to send someone's sample to the lab for extra analysis and it come back negative, than not send it off when it was actually positive.

If the variability of you instrument changes dramatically over the range of Y this can be also very important.

Finally, as an aside (since it wasn't in the original question), if you were to do multiple measurements on one patient, and get one lab result Y, then you must do X-on-Y. The fact that Y is shared across multiple X's means that you wouldn't have independent residuals in a Y-on-X regression.