# Multiple linear regression on a data set with Python?

I have a set of data. In this set I have one dependent variable and about 10 independent variables and the data set is growing regularly. It's rows of data in database with 10 columns of independent variables and one column of the dependent variable. You can see my previous question for an example of what I'm trying to do: Variables importance: who can do the most pushups?

The output of a linear regression is a formula right?

Now I want to write a python script (I could use R also but I'd greatly prefer python) to take this data as input and output the linear regression formula. Is there a python method to do this? Do I need to run a regression comparing each independent variable with the dependent variable one at a time? Or is there a python method to feed in the data with all 10 independent variables and come out with a formula?

-
Is your interest in the data or in writing the program? If it's the first, then forget about Python: use a statistics package and learn how to do linear regression, because it's far more than just a formula: you need capabilities of data manipulation, re-expression, and visualization; ways to obtain regression statistics and residuals; computations of regression diagnostics; and more ways to visualize and check the results. –  whuber Oct 6 '11 at 16:09
Ultimately it's in the data but my understanding was that for things like linear regression one could either use R or use python with NumPy or SciPy to do similar things. As a programmer I'd prefer python although if it's considerably harder in python then I'm open to using R (I've already used R some). –  User Oct 6 '11 at 16:19
R is immediate and interactive: for this kind of work it's really commanding, not programming. Sure you can do the job in Python (and even Excel for that matter), but the important thing here is for you to learn about regression: the program/command problem is trivial in comparison, but no amount of programming will overcome lack of understanding of how to use the software properly. –  whuber Oct 6 '11 at 16:23
Fair enough. I probably will learn to do this in R first and then figure out how to translate that into python. I did find this R tutorial which seems to cover what I want: gardenersown.co.uk/Education/Lectures/R/regression.htm –  User Oct 6 '11 at 17:45
With regard to visualization I'm assuming you mean plotting. In the case of multiple variables, is the idea that you plot each independent variable against the dependent variable one at a time? –  User Oct 7 '11 at 9:02

The simplest outcome from a regression is a set of coefficients, but that is not sufficient for a true regression analysis. You say that you have used R, in R there is a built in dataset called "anscombe" (after the person who created the data). Use that dataset and fit a regression of y1 vs. x1, then do a regression of y2 vs. x2, y3 vs. x3, and y4 vs. x4.

Compare the coefficients (formulas) for the 4 regressions and think about what your conclusions are. Now plot the pairs of data and compare the plots. How does the comparison of the plots compare to the comparison of the regression models?

You could also look up Anscombe's quartet on wikipedia or google, but it is much more informative to do it yourself.

A more complete regression analysis will include not only the coefficients but also things like residual, fitted values, standard errors, confidence intervals, diagnostic plots, etc. (the complete list of everything needed in an analysis depends on the specific data, science, and questions being asked). The above can be produced with about 4 lines of code in R, I don't know how much python code it would take (but there may be prewritten python code to do the same in much fewer lines than programming straight python would).

Also, unless your predictor variables (independent variables, but I don't like the independent/dependent names) are perfectly orthogonal to each other you will get different coefficients fitting one at a time than fitting them all together, and for any dataset of real interest the other important aspects (standard errors, etc) will differ whether you do things one at a time or all together.

-
(+1) I'd also recommend scikits.statsmodels (eventually, with pandas for data structure), rather than just scipy; but @whuber's remarks are worth thousand of words. –  chl Oct 7 '11 at 14:48