2
$\begingroup$

I have one linear regression between a variable of interest (z) and measured values of another variable (y):

z = ay + b

and another regression between y and a proxy variable (x):

y = cx + d

I want to make the case that x is a reasonable proxy for y in estimating z, using these two consecutive regressions. Can I calculate the r2 for the regression between z and predicted y, accounting for the error produced by predicting y from z?

Thanks so much for any help.

$\endgroup$
  • $\begingroup$ The question is confusing. Do you mean that you want to show that x is also a good predictor of z? Also are you saying you only have the summary stats on x and not the individual x values. $\endgroup$ – Michael R. Chernick Jun 14 '17 at 19:52
  • $\begingroup$ Sorry for the unclear question. Basically, I want to calculate the error in z when I measure x, predict y from x, and then predict z from y. I want to compare that error to the error in z predicted directly from y. Thanks for your help. $\endgroup$ – quercus Jun 14 '17 at 20:08
1
$\begingroup$

Let's begin by simplifying the situation. By centering and scaling all data variables appropriately we may create new ones $X,Y,Z$ whose averages are zero and whose lengths, thought of as Euclidean vectors, are unity. When this is done, as is well-known, the regression models are equivalent to

$$Z = \alpha Y + \varepsilon_{ZY}$$

and

$$Y = \gamma X + \varepsilon_{YX}$$

and the errors $\varepsilon_{ZY}$ and $\varepsilon_{YX}$ still have zero means and are independent (within each model). The least-squares solutions are the cosines of the angles among these (unit vectors), $\hat\alpha=\cos(ZY)$ and $\hat\gamma = \cos(YX)$. These cosines are usually called the correlation coefficients. Their squares are the $R^2$ statistics for the regressions (which haven't changed as a result of the centering and scaling).

Using $x$ as a proxy for $y$ is equivalent to using $X$ as a proxy for $Y$. When that occurs, the least squares estimate in the model

$$Z = \beta X + \varepsilon_{ZX}$$

is $\hat\beta = \cos(ZX)$. This allows us to recast the question as one of (three-dimensional) Euclidean geometry:

How well can the (cosine of the) angle $XY$ be estimated based on estimates of the cosines of the other two angles, $ZY$ and $YX$?

The answer is fairly obvious from pictures: the angle $XY$ cannot be any greater than the sum of $ZY$ and $YX$ and it cannot be any less than the size of the difference of $ZY$ and $YX$. (This statement needs appropriate modification when the sum exceeds $90$ degrees: I'll leave that for the interested reader to work out.)

As an extreme example, suppose both angles are $45$ degrees: thus, their cosines are each $\sqrt{1/2} \approx 0.71$ (which is often considered a largish correlation) and both $R^2$ statistics are $1/2$. The angle $XZ$ may range from $45+45=90$ degrees down to $|45-45|=0$ degrees. In the former case, $Z$ and $X$ are orthgonal and $\hat\beta=0$: there is no "linear relation" between them, so $X$ is the worst possible proxy for $Y$. In the latter case $X$ and $Z$ are perfectly correlated and $X$ is the best possible proxy for $Y$.

Contemplation of a few such examples leads one to conclude that unless both $R^2$ statistics (for $Z$ against $Y$ and $Y$ against $X$) are sufficiently close to $1$, it is possible $X$ would be a poor proxy for $Y$. As an approximation, if by "good proxy" you mean the $R^2$ for $Z$ vs $X$ will be at least some (moderately large) threshold $C$, then you will want $R^2_{ZY} + R^2_{ZX} - 1$ to exceed $C$ to assure this.

Figure

This figure shows $X$ situated relative to $Y$. Given the depicted angle between $Z$ and $Y$, the possible locations of $Z$ lie on the red spherical circle shown. Parts of this circle (near the right) are relatively close to $X$, but other parts (near the back left) are nearly at right angles to $X$. Depending on where $Z$ actually is on that circle, $X$ might or might not be close to ("a good proxy") for $Y$.


An R implementation to simulate a dataset of any size and any valid set of correlation coefficients (as specified by angles or directly by modifying rho) follows. It will enable you to experiment with the possibilities.

library(MASS) # exports `mvrnorm`
#
# Specify dataset properties.
# The angle ZX must lie between the extremes of XY+YX and |XY-YX| in order
# for the specified dataset to be possible.
# 
n <- 30                 # Dataset size
alpha <- 45 / 180 * pi  # Angle XY (in radians)
gamma <- 45 / 180 * pi  # Angle YX (in radians)
beta <- 90 / 180 * pi   # Angle ZX (in radians)
#
# Generate a dataset with the specified correlations.
#
rho <- c(xy=cos(gamma), xz=cos(beta), yz=cos(alpha)) # Correlations
Sigma <- matrix(rep(rho, 5)[c(-1,-2,-15)], 4)[-4, ]  # Correlation matrix
diag(Sigma) <- 1
xyz <- try(as.data.frame(mvrnorm(n, c(x=0,y=0,z=0), Sigma, empirical=TRUE)),
           silent=TRUE)
if ("try-error" %in% class(xyz)) stop("These angles are inconsistent.")
#
# Display the three regression results.
#
summary(lm(z ~ y, xyz))$r.squared
summary(lm(y ~ x, xyz))$r.squared
summary(lm(z ~ x, xyz))$r.squared
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.