6
$\begingroup$

I'm pretty new to regression analysis, and I'm using python's statsmodels to look at the relationship between GDP/health/social services spending and health outcomes (DALYs) across the OECD. Just to give an idea of the data I'm using, this is a scatter matrix with the diagonal being the kernel density estimate:

Scatter matrix results

When I run a simple regression of dalyrate on social_exp, the result shows a warning about a high condition number:

import pandas as pd
import numpy as np
import statsmodels.formula.api as smf
import statsmodels.api as sm

poly_1 = smf.ols(formula='dalyrate ~ 1 + social_exp', data=model_df, missing='drop').fit()
print poly_1.summary()

                            OLS Regression Results                            
==============================================================================
Dep. Variable:               dalyrate   R-squared:                       0.253
Model:                            OLS   Adj. R-squared:                  0.248
Method:                 Least Squares   F-statistic:                     46.85
Date:                Fri, 28 Oct 2016   Prob (F-statistic):           2.30e-10
Time:                        12:56:43   Log-Likelihood:                -1336.8
No. Observations:                 140   AIC:                             2678.
Df Residuals:                     138   BIC:                             2683.
Df Model:                           1                                         
Covariance Type:            nonrobust                                         
==============================================================================
                 coef    std err          t      P>|t|      [95.0% Conf. Int.]
------------------------------------------------------------------------------
Intercept   2.705e+04    635.828     42.541      0.000      2.58e+04  2.83e+04
social_exp    -0.9303      0.136     -6.845      0.000        -1.199    -0.662
==============================================================================
Omnibus:                       34.504   Durbin-Watson:                   0.907
Prob(Omnibus):                  0.000   Jarque-Bera (JB):               78.046
Skew:                           1.017   Prob(JB):                     1.13e-17
Kurtosis:                       6.039   Cond. No.                     1.03e+04
==============================================================================

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
[2] The condition number is large, 1.03e+04. This might indicate that there are
strong multicollinearity or other numerical problems.

From what I've read, multicollinearity shouldn't be an issue with a single variable regression, so what could be causing this problem? It seems like both the gdp_cap and dalyrate variables are positively skewed in the scatter matrix, so do I need to normalize or standardize the data in order to prevent this warning? It would be ideal if I didn't need to because the units on the coefficient are helpful for interpreting the results, but obviously I'd do so if necessary.

I also get a similar warning with a multivariate regression:

poly_2 = smf.ols(formula='dalyrate ~ 1 + social_exp + health_exp + gdp_cap', data=model_df, missing='drop').fit()
print poly_2.summary()


                            OLS Regression Results                            
==============================================================================
Dep. Variable:               dalyrate   R-squared:                       0.406
Model:                            OLS   Adj. R-squared:                  0.393
Method:                 Least Squares   F-statistic:                     30.98
Date:                Fri, 28 Oct 2016   Prob (F-statistic):           2.51e-15
Time:                        13:04:35   Log-Likelihood:                -1320.8
No. Observations:                 140   AIC:                             2650.
Df Residuals:                     136   BIC:                             2661.
Df Model:                           3                                         
Covariance Type:            nonrobust                                         
==============================================================================
                 coef    std err          t      P>|t|      [95.0% Conf. Int.]
------------------------------------------------------------------------------
Intercept    2.86e+04    631.270     45.308      0.000      2.74e+04  2.98e+04
social_exp     0.0379      0.208      0.182      0.855        -0.373     0.449
health_exp    -1.1555      0.889     -1.300      0.196        -2.914     0.603
gdp_cap       -0.1412      0.053     -2.662      0.009        -0.246    -0.036
==============================================================================
Omnibus:                       53.794   Durbin-Watson:                   0.663
Prob(Omnibus):                  0.000   Jarque-Bera (JB):              157.258
Skew:                           1.490   Prob(JB):                     7.11e-35
Kurtosis:                       7.251   Cond. No.                     7.24e+04
==============================================================================

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
[2] The condition number is large, 7.24e+04. This might indicate that there are
strong multicollinearity or other numerical problems.

Any help would be appreciated!

$\endgroup$
  • $\begingroup$ This may be irrelevant, but I don't think you need to include the 1 with the formula API. $\endgroup$ – Demetri Pananos Oct 28 '16 at 18:51
  • 1
    $\begingroup$ You can have multicollinearity between a single covariate and the intercept, but no one ever cares. I can't tell if that's what you have or not. I also note that the relationship appears to be curvilinear & probably heteroscedastic. $\endgroup$ – gung - Reinstate Monica Oct 28 '16 at 18:59
  • 1
    $\begingroup$ Do I understand correctly that you have a regression w/ only 1 x-variable, which is centered, & it tells you you have a large condition number? That is pretty strange. There is probably something wrong w/ Python here. $\endgroup$ – gung - Reinstate Monica Oct 28 '16 at 20:44
  • 1
    $\begingroup$ My guess is that this is not related to collinearity but to the scaling of the explanatory variables, i.e. an eigenvalue is large because the variable is large, and some variables, like constant, are small. (statsmodels reports the condition number of the design matrix and not of a standardized design matrix.) $\endgroup$ – Josef Oct 28 '16 at 20:54
  • 1
    $\begingroup$ @user333700 Thanks for the info. I scaled each of my explanatory variables down by a factor of 1000, and that removed the condition number warning. This issue might be related as well: github.com/statsmodels/statsmodels/issues/553 $\endgroup$ – pst0102 Oct 28 '16 at 23:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.