Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.
  1. Are Pandas, Statsmodels and scikits-learn different implementations of machine learning/statistical operations, or are these complementary to one another?

  2. Which of these has the most comprehensive functionality?

  3. Which one is actively developed and/or supported?

  4. I have to implement logistic regression. Any suggestions as to which of these I should use?

share|improve this question

2 Answers 2

up vote 13 down vote accepted
  1. scikits-learn (sklearn) is the best choice for machine learning, out of the three listed. While Pandas and statsmodels do contain some predictive learning algorithms, they are hidden/not production-ready yet. Often, as authors will work on different projects, the libraries are complimentary. For example, recently Pandas' Dataframes were integrated into Statsmodels. A relationship between sklearn and Pandas is not present (yet).

  2. Define functionality. They all run. If you mean what is the most useful, then it depends on your application. I would definitely give Pandas a +1 here, as it has added a great new data structure to Python (dataframes). Pandas also probably has the best api.

  3. They are all actively supported, though I would say Pandas has the best code base. Sklearn and Pandas are more active than Statsmodels.

  4. The clear choice is sklearn. It is easy and clear how to perform it.

    from sklearn.linear_models import LogisticRegression as LR
    logr = LR()
    logr.fit( X, Y )
    results = logr.predict( test_data)
    
share|improve this answer
    
Thanks so much for the explanations. It helped! –  Nik Jan 17 '13 at 3:46
3  
Be sure to read the longer explanation below! –  dartdog Jan 26 '13 at 16:20
    
This answer neglects to explain that Pandas is principally for data manipulation (e.g. indexing, selecting, aggregating), whereas the other two are for building models (e.g. for prediction or inference). –  Nathan Gould May 24 at 3:49

I would like to qualify and clarify a bit the accepted answer.

The three packages are complementary to each other since they cover different areas, have different main objectives, or emphasize different areas in machine learning/statistics.

  • pandas is mainly a package to handle and operate directly on data.
  • scikit-learn is doing machine learning with emphasis on predictive modeling with often large and sparse data
  • statsmodels is doing "traditional" statistics and econometrics, with much stronger emphasis on parameter estimation and (statistical) testing.

statsmodels has pandas as a dependency, pandas optionally uses statsmodels for some statistics. statsmodels is using patsy to provide a similar formula interface to the models as R.

There is some overlap in models between scikit-learn and statsmodels, but with different objectives. see for example The Two Cultures: statistics vs. machine learning?

some more about statsmodels

statsmodels has the lowest developement activity and longest release cycle of the three. statsmodels has many contributors but unfortunately still only two "maintainers" (I'm one of them.)

The core of statsmodels is "production ready": linear models, robust linear models, generalised linear models and discrete models have been around for several years and are verified against Stata and R. statsmodels also has a time series analysis part covering AR, ARMA and VAR (vector autoregressive) regression, which are not available in any other python package.

Some examples to show some specific differences between the machine learning approach in scikit-learn and the statistics and econometrics approach in statsmodels:

Simple linear Regression, OLS, has a large number of post-estimation analysis http://statsmodels.sourceforge.net/devel/generated/statsmodels.regression.linear_model.OLSResults.html including tests on parameters, outlier measures and specification tests http://statsmodels.sourceforge.net/devel/stats.html#residual-diagnostics-and-specification-tests

Logistic Regression can be done in statsmodels either as Logit model in discrete or as a family in generalized linear model (GLM). http://statsmodels.sourceforge.net/devel/glm.html#module-reference

GLM includes the usual families, discrete models contains besides Logit also Probit, multinomial and count regression.

Logit

Using Logit is as simple as this http://statsmodels.sourceforge.net/devel/examples/generated/example_discrete.html

>>> import statsmodels.api as sm
>>> x = sm.add_constant(data.exog, prepend=False)
>>> y = data.endog

>>> res1 = sm.Logit(y, x).fit()
Optimization terminated successfully.
         Current function value: 0.402801
         Iterations 7
>>> print res1.summary()
                           Logit Regression Results                           
==============================================================================
Dep. Variable:                      y   No. Observations:                   32
Model:                          Logit   Df Residuals:                       28
Method:                           MLE   Df Model:                            3
Date:                Sat, 26 Jan 2013   Pseudo R-squ.:                  0.3740
Time:                        07:34:59   Log-Likelihood:                -12.890
converged:                       True   LL-Null:                       -20.592
                                        LLR p-value:                  0.001502
==============================================================================
                 coef    std err          z      P>|z|      [95.0% Conf. Int.]
------------------------------------------------------------------------------
x1             2.8261      1.263      2.238      0.025         0.351     5.301
x2             0.0952      0.142      0.672      0.501        -0.182     0.373
x3             2.3787      1.065      2.234      0.025         0.292     4.465
const        -13.0213      4.931     -2.641      0.008       -22.687    -3.356
==============================================================================
>>> dir(res1)
...
>>> res1.predict(x.mean(0))
0.25282026208742708
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.