# Cause of a high condition number in a python statsmodels regression?

I'm pretty new to regression analysis, and I'm using python's statsmodels to look at the relationship between GDP/health/social services spending and health outcomes (DALYs) across the OECD. Just to give an idea of the data I'm using, this is a scatter matrix with the diagonal being the kernel density estimate:

When I run a simple regression of dalyrate on social_exp, the result shows a warning about a high condition number:

import pandas as pd
import numpy as np
import statsmodels.formula.api as smf
import statsmodels.api as sm

poly_1 = smf.ols(formula='dalyrate ~ 1 + social_exp', data=model_df, missing='drop').fit()
print poly_1.summary()

OLS Regression Results
==============================================================================
Dep. Variable:               dalyrate   R-squared:                       0.253
Method:                 Least Squares   F-statistic:                     46.85
Date:                Fri, 28 Oct 2016   Prob (F-statistic):           2.30e-10
Time:                        12:56:43   Log-Likelihood:                -1336.8
No. Observations:                 140   AIC:                             2678.
Df Residuals:                     138   BIC:                             2683.
Df Model:                           1
Covariance Type:            nonrobust
==============================================================================
coef    std err          t      P>|t|      [95.0% Conf. Int.]
------------------------------------------------------------------------------
Intercept   2.705e+04    635.828     42.541      0.000      2.58e+04  2.83e+04
social_exp    -0.9303      0.136     -6.845      0.000        -1.199    -0.662
==============================================================================
Omnibus:                       34.504   Durbin-Watson:                   0.907
Prob(Omnibus):                  0.000   Jarque-Bera (JB):               78.046
Skew:                           1.017   Prob(JB):                     1.13e-17
Kurtosis:                       6.039   Cond. No.                     1.03e+04
==============================================================================

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
[2] The condition number is large, 1.03e+04. This might indicate that there are
strong multicollinearity or other numerical problems.


From what I've read, multicollinearity shouldn't be an issue with a single variable regression, so what could be causing this problem? It seems like both the gdp_cap and dalyrate variables are positively skewed in the scatter matrix, so do I need to normalize or standardize the data in order to prevent this warning? It would be ideal if I didn't need to because the units on the coefficient are helpful for interpreting the results, but obviously I'd do so if necessary.

I also get a similar warning with a multivariate regression:

poly_2 = smf.ols(formula='dalyrate ~ 1 + social_exp + health_exp + gdp_cap', data=model_df, missing='drop').fit()
print poly_2.summary()

OLS Regression Results
==============================================================================
Dep. Variable:               dalyrate   R-squared:                       0.406
Method:                 Least Squares   F-statistic:                     30.98
Date:                Fri, 28 Oct 2016   Prob (F-statistic):           2.51e-15
Time:                        13:04:35   Log-Likelihood:                -1320.8
No. Observations:                 140   AIC:                             2650.
Df Residuals:                     136   BIC:                             2661.
Df Model:                           3
Covariance Type:            nonrobust
==============================================================================
coef    std err          t      P>|t|      [95.0% Conf. Int.]
------------------------------------------------------------------------------
Intercept    2.86e+04    631.270     45.308      0.000      2.74e+04  2.98e+04
social_exp     0.0379      0.208      0.182      0.855        -0.373     0.449
health_exp    -1.1555      0.889     -1.300      0.196        -2.914     0.603
gdp_cap       -0.1412      0.053     -2.662      0.009        -0.246    -0.036
==============================================================================
Omnibus:                       53.794   Durbin-Watson:                   0.663
Prob(Omnibus):                  0.000   Jarque-Bera (JB):              157.258
Skew:                           1.490   Prob(JB):                     7.11e-35
Kurtosis:                       7.251   Cond. No.                     7.24e+04
==============================================================================

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
[2] The condition number is large, 7.24e+04. This might indicate that there are
strong multicollinearity or other numerical problems.


Any help would be appreciated!

• This may be irrelevant, but I don't think you need to include the 1 with the formula API. Commented Oct 28, 2016 at 18:51
• You can have multicollinearity between a single covariate and the intercept, but no one ever cares. I can't tell if that's what you have or not. I also note that the relationship appears to be curvilinear & probably heteroscedastic. Commented Oct 28, 2016 at 18:59
• Do I understand correctly that you have a regression w/ only 1 x-variable, which is centered, & it tells you you have a large condition number? That is pretty strange. There is probably something wrong w/ Python here. Commented Oct 28, 2016 at 20:44
• My guess is that this is not related to collinearity but to the scaling of the explanatory variables, i.e. an eigenvalue is large because the variable is large, and some variables, like constant, are small. (statsmodels reports the condition number of the design matrix and not of a standardized design matrix.) Commented Oct 28, 2016 at 20:54
• @user333700 Thanks for the info. I scaled each of my explanatory variables down by a factor of 1000, and that removed the condition number warning. This issue might be related as well: github.com/statsmodels/statsmodels/issues/553 Commented Oct 28, 2016 at 23:56

I found this page in a search, because I had the same question, but I think I have figured out what's going on.

First, a demonstration of the problem:

import numpy as np
import statsmodels.api as sm
import matplotlib.pyplot as plt
x = np.arange(1000.,1030.,1.)
y = 0.5*x
plt.plot(x, y,'x')
plt.show()


mod_ols = sm.OLS(y, X)
res_ols = mod_ols.fit()
print(res_ols.summary())


Notice the very high condition number of 1.19e+05. This is because we're fitting a line to the points and then projecting the line all the way back to the origin (x=0) to find the y-intercept. That y-intercept will be very sensitive to small movements in the data points. The condition number takes into account high sensitivity in either fitted parameter to the input data, hence the high condition number when all of the data are far to one side of x=0.

To solve this, we simply center the x-values:

x -= np.average(x)