# Selecting most useful variables from Ordinary Least Squares using statsmodels in python [duplicate]

I have 6 independent x variables and have used OLS to get a polynomial model to describe their relationship with my dependent y variable.

Here is what I get from statsmodels:

==============================================================================
Dep. Variable:                      p   R-squared:                       0.209
Method:                 Least Squares   F-statistic:                     2.949
Date:                Sat, 09 Oct 2021   Prob (F-statistic):             0.0129
Time:                        15:06:33   Log-Likelihood:                -634.99
No. Observations:                  74   AIC:                             1284.
Df Residuals:                      67   BIC:                             1300.
Df Model:                           6
Covariance Type:            nonrobust
==========================================================================================
coef    std err          t      P>|t|      [0.025      0.975]
------------------------------------------------------------------------------------------
const                   1.177e+04   1107.772     10.629      0.000    9563.438     1.4e+04
Plasma Temperature (K)     0.0025      0.003      0.843      0.402      -0.003       0.008
Kp*10                     16.1861     27.491      0.589      0.558     -38.687      71.059
Dst                      -14.2056     14.523     -0.978      0.332     -43.195      14.783
Proton Density            83.2930     69.770      1.194      0.237     -55.969     222.555
f10.7                    -15.9405      8.069     -1.975      0.052     -32.047       0.166
Ey                      -197.5733    292.431     -0.676      0.502    -781.268     386.121
==============================================================================
Omnibus:                        6.537   Durbin-Watson:                   2.089
Prob(Omnibus):                  0.038   Jarque-Bera (JB):                6.412
Skew:                           0.485   Prob(JB):                       0.0405
Kurtosis:                       4.067   Cond. No.                     7.71e+05
==============================================================================


As someone with no experience with statistics, I looked up what all of these different parameters meant but am currently trying to determine how to get the most accurate polynomial using only some of the variables. How would I approach this? Is it simply running a case of running all of the combinations until I get the highest Adj. R-squared or lowest P>t?

Thanks!

• Hi scikitnoob and welcome! Your question is a very well known one, and goes by the name of "variable selection". It has been discussed on this site multiple times already, Check out this one stats.stackexchange.com/questions/122931/…. It has a recommendation to use lasso as an answer and some more links in the comment to the question. Oct 9, 2021 at 21:03
• @psarka Thank you so much for sharing the name of the problem!! I'll check it out, much appreciated! Oct 9, 2021 at 21:21
• Hi @psarka, correct me if I am wrong, but from my understanding Lasso simply removes any coefficients under a certain value (so, near-zero). Am I misunderstanding Lasso? I only ask because when using lasso, I receive the exact same polynomial I already had, but I know for sure that removing at least one of the x variables from the model will improve my model... Oct 9, 2021 at 21:35
• Lasso is more subtle than just normal regression + removing coefficients under a certain value. I find it surprising, that you get the same result. My advice would be to double check the code to make sure you don't have any bugs, and if not, post a new question. Oct 9, 2021 at 21:44
• You state you have fitted a polynomial but I see no sign of such terms. What do you mean by most accurate polynomial? Oct 10, 2021 at 13:50