0
$\begingroup$

Consider the following data (I put the data as a table in the bottom of this question.)

All Data

These data are in two groups, blue and orange. In each group there is a positive relationship, while pooling across groups, there is a negative relationship. So there is Simpson's paradox. (In my application, the two groups blue and orange are school districts and the dots are schools, but that is not important for the question.)

If I run an OLS regression using all the data I get these estimates,

ppSpend = 9.481481 + -2.962963*pctPoor

Now if I average the data up to the group-level (points weighted equally) I get this scatterplot

Group means

If I run an OLS regression using the group averages (so this is the "between model") I get

ppSpend_GroupMean = 10.14286 + -4.285714*pctPoor_GroupMean

Finally, here is the data if I demean both the dependent and independent variables by group (plotting the points at different sizes so that they can be seen):

Demeaned

If I run an OLS regression on this "demeaned" model ("within model") I get

ppSpend_Demeaned = 0 + 10*pctPoor_Demeaned

Here's my question: is there an interpretable weight A such that

Total Slope = A*(Between Slope) + (1-A)*(Within Slope) ?

In my example,

-2.962963 = A*(-4.285714) + (1-A)*(10)

Of course the specific number in my example is 0.907407, but I would like to know if there's some general expression for that number from interpretable things calculated from the data.

Data used in the example (as csv):

group,pctPoor,ppSpend
1,0,8
1,.1,9
1,.2,10
1,.3,11
2,.7,5
2,.8,6
2,.9,7
2,1,8
$\endgroup$

1 Answer 1

1
$\begingroup$

Let g be the indicator of the first group. That is, it is a vector of length 8 whose first 4 elements are 1 and whose last 4 are 0.

Let P be the projection onto the space spanned by g and 1-g -- if there were k groups then we would consider the space spanned by k vectors but here we have only two -- and let Q=I-P be the orthogonal complement projection. Also let y be ppSpend and x be pctPoor.

Let b, w and t be the between, within and total slopes. That is they are the slopes of the regression (including intercept) of y on Px, y on Qx and y on x respectively. Then we interpret the question as asking what the relationship is among b, w and t and it is:

var(Px) * b + var(Qx) * w = var(x) * t

which follows from the fact that the slopes are given by the three expressions below and that the numerators of b and w sum to the numerator of t (and similarly for the denominators).

b = cov(Px, y) / var(Px)
w = cov(Qx, y) / var(Qx)
t = cov(x, y) / var(x)

Dividing through the equation involving b, w and t by var(x) and letting a = var(Px)/var(x) we can write it as this convex combination.

a * b + (1-a) * w = t

The formula var(Px) / var(x) can be regarded as the squared cosine of the angle between Px and x if we regard squared length to be var.

We can illustrate this using R.

g <- rep(1:0, each = 4)
x <- c(0, 0.1, 0.2, 0.3, 0.7, 0.8, 0.9, 1)
y <- c(8, 9, 10, 11, 5, 6, 7, 8)

n <- length(y)
G <- cbind(g, 1-g)
P <- G %*% solve(crossprod(G), t(G))
Q <- diag(n) - P

b = cov(P %*% x, y) / var(P %*% x); b  # or coef(lm(y ~ P %*% x))[[2]]
##           [,1]
## [1,] -4.285714

w = cov(Q %*% x, y) / var(Q %*% x); w  # or coef(lm(y ~ Q %*% x))[[2]]
##      [,1]
## [1,]   10

t = cov(x, y) / var(x); t  # or coef(lm(y ~ x))[[2]]
## [1] -2.962963

a <- var(P %*% x) / var(x); a
##           [,1]
## [1,] 0.9074074

# P %*% x also equals ave(x, g) in R so we can alternately write a as:
var(ave(x, g)) / var(x)
## [1] 0.9074074

# Using a, b and w from above, we see this equals the t shown above
a * b + (1-a) * w
##           [,1]
## [1,] -2.962963
$\endgroup$
3
  • $\begingroup$ Is this a special case of a general decomposition formula? (That maybe has a name?) I just ask because it seems g could have been any covariate matrix. Although, perhaps the idea of a "between effect" isn't that common unless g is a categorical variable. I'm not sure what other case we would want to know the relationship between the dependent variable and the predicted values of x using the other covariates. $\endgroup$ Mar 13 at 2:47
  • $\begingroup$ Here are some other notes, some just reminders for me. x = Px + Qx is a way to write that x is equal to its predicted value plus its residual. Use that in cov() to show that cov(x,y) = cov(Px,y) + cov(Qx,y) to get the first equation. Px and Qx are orthogonal by construction so var(x) = var(Px) + var(Qx). The rest is algebra. Also, since Px gives the fitted values, so var(predict(lm(x~factor(g))))/var(x) is also a. Which also means, summary(lm(x~factor(g)))$r.squared is also a. In words, a is the share of variance in x explained by the groups. $\endgroup$ Mar 13 at 2:49
  • 1
    $\begingroup$ Good idea. Another way of stating this is that a is the squared correlation between x and the projection of x on g where x and g are as in the answer.. In R we have a = cor(ave(x, g), x)^2 . $\endgroup$ Mar 13 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.