1
$\begingroup$

How is multicollinearity calcuated for an ordered logistic regression if R^2 is not determined for this type of regression? Because the formula for vif says: 1/(1-R^2)
In R the looking up ?vif says that the polr function in MASS package can be sued to determine vif. However in the regression summary of polr R^2 is not determined. So I just got a bit confused. Anyone that can help me to clarify and understand this in an easy way? Any kind of source would also help.. I could not find a good one.

$\endgroup$
7
  • $\begingroup$ Because multicollinearity is a property only of the explanatory variables, you would calculate any indicators for it in the same way you would for ordinary least squares regression. $\endgroup$
    – whuber
    Dec 12, 2022 at 19:13
  • $\begingroup$ @whuber the R car package provides VIF measures based on the variance-covariance matrix, rather than the method based solely on values of explanatory variables. There's a special method for polr objects. See car:::vif.polr. $\endgroup$
    – EdM
    Dec 12, 2022 at 20:38
  • 1
    $\begingroup$ The linked duplicate page show the generic code for vif() in the car package. It uses the variance-covariance matrix of the model rather than the standard formula for ordinary least squares (OLS) models; the two methods agree for OLS but the method in the car package better illustrates potential instability in coefficient estimates in other model types. To see the specific method used for polr objects, type car:::vif.polr at the R command prompt. $\endgroup$
    – EdM
    Dec 12, 2022 at 20:49
  • $\begingroup$ That's interesting, @EdM -- thank you for sharing that. $\endgroup$
    – whuber
    Dec 12, 2022 at 21:02
  • 1
    $\begingroup$ @rr19 although the answer is included in the linked page, you are correct that how that answers your question is a bit opaque. I'll reopen and answer with more explication. $\endgroup$
    – EdM
    Dec 13, 2022 at 19:01

1 Answer 1

2
$\begingroup$

The variance whose inflation is at issue in a variance inflation factor (VIF) is the sampling variance of a regression coefficient. If there is correlation among the sampling variances of multiple coefficients, then the estimate of any one coefficient might change with changes in the estimates of its correlated coefficients. That inflates the sampling variance of any single coefficient.

Information needed to evaluate that problem is in the variance-covariance matrix of a regression model's coefficient estimates. That's typically available in R via the vcov() function applied to the model. There's nothing specific here about ordinary least squares (OLS) regression; this is the case for any model with a variance-covariance matrix of coefficient estimates.

Say that there are two regression coefficients $\beta_1$ and $\beta_2$, and that you have converted the variance-covariance matrix of their estimates to a correlation matrix, $\text{Cor}(\beta_1,\beta_2)$. The general form would be:

$$ \text{Cor}(\beta_1,\beta_2)= \begin{pmatrix} 1 & \rho\\\ \rho & 1\end{pmatrix}.$$

If there were no correlation between the estimates ($\rho=0$), the determinant of the matrix would be 1. The actual determinant, with the correlation, is $(1-\rho^2)$. That suggests using the ratio of the no-correlation (ideal) determinant value to the actual determinant as an estimate of the magnitude of the problem introduced by the correlation, for:

$$\frac{1}{1-\rho^2} .$$

In the particular case of OLS, the squared correlation $\rho^2$ between the coefficient estimates is identical to the squared correlation between the two sets of predictor values. You might think of the OLS case as a specific, simple instance of a more general estimate of coefficient variance inflation.

This can be generalized to a multiple-coefficient setting (for example, evaluating together all coefficients for a multi-level categorical predictor), with a block matrix of correlations among estimates. Say that $R$ is the full correlation matrix, $R_1$ is the correlation matrix restricted to the coefficients of main interest, $R_2$ is that restricted to the others, and $R_{12}$ represents the cross-correlations between the two sets of coefficients. Then you have for the determinant of the correlation matrix:

$$\det R = \det \begin{pmatrix} R_1 & R_{12}\\\ R_{12}^T & R_2\end{pmatrix}= \det R_1 \det (R_2 - R_{12}^T R_1^{-1} R_{12}) ,$$

which would be $\det R_1 \det R_2$ if there were no correlations among their estimates (with $R_{12}$ a matrix of zeros). Taking the ratio of determinants between the no-correlation and the actual situation as above, we have as an estimate of the inflation of the variance for estimates in $R_1$:

$$\frac{\det R_1 \det R_2}{\det R}. $$

That's what returned by the following line of code as the generalized variance inflation factor (GVIF) in the R car package for a polr object or other object with a variance-covariance matrix, scaled to a correlation matrix R and subsetted into two groups of coefficients:

det(as.matrix(R[subs, subs])) * det(as.matrix(R[-subs, -subs]))/detR

again without any need to restrict to the OLS situation. In OLS with only a single coefficient for predictor $X_j$ included in $R_1$, that reduces to the familiar form $1/(1-R_j^2)$, where $R_j^2$ is the squared multiple correlation coefficient of $X_j$ against the other predictors.

An example in this question shows that the same set of predictor values can have different GVIF values depending on whether they are included in an OLS model or a generalized linear model like a binomial regression.

$\endgroup$
2
  • $\begingroup$ Thanks for your answer! I should maked more clear when I create my question. The reason I asked this question was I thought that the R^2 in the vif formula was the R^2 we get in regression summaries. (i.e. R^2 if it's for instance close to 1 then the model is a "good" model) $\endgroup$
    – rr19
    Dec 13, 2022 at 19:14
  • 1
    $\begingroup$ @rr19 the confusion is understandable, given how many $R^2$ terms are involved. The $R_j^2$ for the standard OLS VIF is for a regression, but it's for the regression of predictor $X_j$ against all of the other predictors. $\endgroup$
    – EdM
    Dec 13, 2022 at 19:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.