3
$\begingroup$

I have trouble understanding how R² in a regression analysis makes sense visually in Ballantine diagrams.

For instance:

Kennedy Figure 2

Obviously the red region is ignored when estimating the coefficients for x on y and w on y. But the red region must still be in the model somehow, otherwise we lose information that is useful in predicting y.

The typical practical example is high multicollinearity, a large R², but non-significant predictors.

I assume that the red region is implicitly part of the model although it is not used when estimating the coefficients. But how exactly?

$\endgroup$
8
  • $\begingroup$ This is called a "Ballantine" (not a Venn diagram as some respondents and even authors claim). The Cohens popularized it in the 1970's and 80's in the social sciences, but it did not become commonplace. I have found it to be less than useful. Some of its problems are described in Hunt (1986), who writes "there is an exact analogy between the percentage of one circle that intersects another and the squared correlations ... Unfortunately, the analogy ... does not hold for multiple, partial, and semipartial correlations." $\endgroup$
    – whuber
    Commented May 30, 2023 at 14:59
  • $\begingroup$ To state the criticism briefly, the overlaps among pairs of circles graphically indicated bivariate correlations, but nothing here directly represents the multivariate correlations in which one is usually interested. $\endgroup$
    – whuber
    Commented May 30, 2023 at 15:04
  • $\begingroup$ @whuber: Sure, Ballantine or Ballentine (as called by Kennedy) is more accurate, but I guess most people just call it Venn as this is better known. Linux is actually GNU/Linux but nobody calls it that way, there are thousands of such examples. Anyway, your answer is: the analogy does not hold, which is a good answer! $\endgroup$ Commented May 30, 2023 at 15:05
  • $\begingroup$ Because the relative areas (and even the actual shapes) in a Venn diagram are meaningless but are crucial in the Ballantine, it truly is important to distinguish the two kinds of diagram, even though superficially they might look similar. $\endgroup$
    – whuber
    Commented May 30, 2023 at 15:16
  • $\begingroup$ Alright, you convinced me, I changed Venn to Ballantine. $\endgroup$ Commented May 30, 2023 at 15:30

2 Answers 2

4
$\begingroup$

I cover collinearity here where you'll see reasons why collinearity is usually not to be feared when looking at overall model properties such as $R^2$. Collinearity comes into play when attempting to interpret an effect of a variable that is collinear with other variables. Descriptive analyses that help include variable clustering and redundancy analysis. Also, chunk tests are your friend.

$\endgroup$
1
  • 2
    $\begingroup$ Thanks, but that does not really answer my question. $\endgroup$ Commented May 30, 2023 at 14:28
3
$\begingroup$

From my experience, I have found the Venn diagram explanation for correlations (partial, semi-partial, etc.) to be somewhat confusing and not always an accurate model of what is actually happening (or might happen).

The key here is that there are two types of correlations that we could examine in the presence of another variable. The first is the partial correlation. This correlation removes all of the influence of the 2nd independent variable from both the predictor and the response. Thus, using the Venn diagram, this would be $\frac{B}{Y+B}$ (for yellow and blue)...and technically, this is the square of the partial correlation (the "partial" coefficient of determination, if you will).

However, there is also the semi-partial correlation. This correlation removes the influence only from the other predictor (but it does not remove the influence from the response variable). Thus, this correlation would be represented as $\frac{B}{Y+B+G+R}$ (again, this is technically the square of the semi-partial correlation).

Now the tricky part is that there are two types of $R^2$ we might talk about here...and these depend on the way we choose to break down the variation of the response variable. The first of these $R^2$ uses the semi-partial correlations, and as a consequence, that shared overlap is not actually ignored. The second of these $R^2$ uses the partial correlations...and in this case, if you were to add up the corresponding "variability" estimates for the response variable, you actually would be missing some of it. (Or weirder still, you can actually end up with more variability than you start with...and this is why I find these diagrams to not be the best model for explaining the partitioning process very well.)

Happy to clarify more if needed.

$\endgroup$
4
  • $\begingroup$ I am aware of partial and semipartial correlations. Obviously regression coefficients are similar to semipartial correlations (with the denominator being squared). This does not answer my question though. In your example B/(Y+B+G+R) the red part is still ignored because it is not treated as explained variance. Only blue and green are, right? That's what my question is about. How does the red region enter the model? $\endgroup$ Commented May 30, 2023 at 14:34
  • $\begingroup$ The red region would enter the model in the Type-I version of the decomposition of the sums of squares. The first variable would be the only predictor in the model, and the red region would show up there. The second variable would be the variable accounting for whatever is left over to be accounted for (no red region). This process would continue until all variables have been entered into the model. $\endgroup$
    – Gregg H
    Commented May 30, 2023 at 14:49
  • $\begingroup$ Are you sure? In your description high multicollinearity would lead to one predictor being significant, the other not. But this is not what usually happens. Imagine a correlation between predictors of close to 1 (e.g. 0.99). Almost no information is available to accurately estimate the coefficients so neither will be significant. Also: If both variables enter the regression, who decides what is the "first" one? $\endgroup$ Commented May 30, 2023 at 14:56
  • $\begingroup$ Your statement "one predictor being significant" suggests that you are focusing on Type-II sums of squares decomposition. This is not the only way you can assess the variance decomposition components. And your last query is moving into the realm of model building. With Type-II analyses, you get the same results if you enter the variables in any order. With Type-I analyses, you can get different results based on the order the variables are entered in the model. So, the model builder must decide. $\endgroup$
    – Gregg H
    Commented May 30, 2023 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.