Are Pandas, Statsmodels and Scikit-learn different implementations of machine learning/statistical operations, or are these complementary to one another?
Which of these has the most comprehensive functionality?
Which one is actively developed and/or supported?
I have to implement logistic regression. Any suggestions as to which of these I should use?
Scikit-learn (sklearn) is the best choice for machine learning, out of the three listed. While Pandas and Statsmodels do contain some predictive learning algorithms, they are hidden/not production-ready yet. Often, as authors will work on different projects, the libraries are complimentary. For example, recently Pandas' Dataframes were integrated into Statsmodels. A relationship between sklearn and Pandas is not present (yet).
Define functionality. They all run. If you mean what is the most useful, then it depends on your application. I would definitely give Pandas a +1 here, as it has added a great new data structure to Python (dataframes). Pandas also probably has the best API.
They are all actively supported, though I would say Pandas has the best code base. Sklearn and Pandas are more active than Statsmodels.
The clear choice is Sklearn. It is easy and clear how to perform it.
from sklearn.linear_models import LogisticRegression as LR logr = LR() logr.fit( X, Y ) results = logr.predict( test_data)
I would like to qualify and clarify a bit the accepted answer.
The three packages are complementary to each other since they cover different areas, have different main objectives, or emphasize different areas in machine learning/statistics.
- pandas is mainly a package to handle and operate directly on data.
- scikit-learn is doing machine learning with emphasis on predictive modeling with often large and sparse data
- statsmodels is doing "traditional" statistics and econometrics, with much stronger emphasis on parameter estimation and (statistical) testing.
statsmodels has pandas as a dependency, pandas optionally uses statsmodels for some statistics. statsmodels is using
patsy to provide a similar formula interface to the models as R.
There is some overlap in models between scikit-learn and statsmodels, but with different objectives. see for example The Two Cultures: statistics vs. machine learning?
some more about statsmodels
statsmodels has the lowest developement activity and longest release cycle of the three. statsmodels has many contributors but unfortunately still only two "maintainers" (I'm one of them.)
The core of statsmodels is "production ready": linear models, robust linear models, generalised linear models and discrete models have been around for several years and are verified against Stata and R. statsmodels also has a time series analysis part covering AR, ARMA and VAR (vector autoregressive) regression, which are not available in any other python package.
Some examples to show some specific differences between the machine learning approach in scikit-learn and the statistics and econometrics approach in statsmodels:
Simple linear Regression,
OLS, has a large number of post-estimation analysis
http://statsmodels.sourceforge.net/devel/generated/statsmodels.regression.linear_model.OLSResults.html including tests on parameters, outlier measures and specification tests http://statsmodels.sourceforge.net/devel/stats.html#residual-diagnostics-and-specification-tests
Logistic Regression can be done in statsmodels either as
Logit model in discrete or as a family in generalized linear model (
GLM includes the usual families, discrete models contains besides
Probit, multinomial and count regression.
Logit is as simple as this
>>> import statsmodels.api as sm >>> x = sm.add_constant(data.exog, prepend=False) >>> y = data.endog >>> res1 = sm.Logit(y, x).fit() Optimization terminated successfully. Current function value: 0.402801 Iterations 7 >>> print res1.summary() Logit Regression Results ============================================================================== Dep. Variable: y No. Observations: 32 Model: Logit Df Residuals: 28 Method: MLE Df Model: 3 Date: Sat, 26 Jan 2013 Pseudo R-squ.: 0.3740 Time: 07:34:59 Log-Likelihood: -12.890 converged: True LL-Null: -20.592 LLR p-value: 0.001502 ============================================================================== coef std err z P>|z| [95.0% Conf. Int.] ------------------------------------------------------------------------------ x1 2.8261 1.263 2.238 0.025 0.351 5.301 x2 0.0952 0.142 0.672 0.501 -0.182 0.373 x3 2.3787 1.065 2.234 0.025 0.292 4.465 const -13.0213 4.931 -2.641 0.008 -22.687 -3.356 ============================================================================== >>> dir(res1) ... >>> res1.predict(x.mean(0)) 0.25282026208742708