0
$\begingroup$

I have the following code using statsmodels

x_ran = [random.random() for i in range(100)]
y_ran = [random.random()*800 + random.randint(1, 2000) for i in range(100)]

mod2= sm.OLS(y_ran, x_ran)
res = mod2.fit()
print(res.summary2())

The summary is as follows:

                        Results: Ordinary least squares
================================================================================
Model:                  OLS              Adj. R-squared (uncentered): 0.623     
Dependent Variable:     y                AIC:                         1629.9400 
Date:                   2020-01-23 18:04 BIC:                         1632.5452 
No. Observations:       100              Log-Likelihood:              -813.97   
Df Model:               1                F-statistic:                 166.4     
Df Residuals:           99               Prob (F-statistic):          6.37e-23  
R-squared (uncentered): 0.627            Scale:                       6.9493e+05
------------------------------------------------------------------------------------
             Coef.       Std.Err.        t        P>|t|        [0.025        0.975] 
------------------------------------------------------------------------------------
x1         1746.1500     135.3722     12.8989     0.0000     1477.5422     2014.7578
--------------------------------------------------------------------------------
Omnibus:                   3.045             Durbin-Watson:                1.861
Prob(Omnibus):             0.218             Jarque-Bera (JB):             2.607
Skew:                      0.289             Prob(JB):                     0.272
Kurtosis:                  2.460             Condition No.:                1    
================================================================================

Why are the p-value so low and the adjusted variance so high given that the data is randomly generated?

As per the request, here is the plot: enter image description here

Improve this question
$\endgroup$
5
  • 2
    $\begingroup$ Does that regression include an intercept? $\endgroup$ – Firebug Jan 23 '20 at 18:19
  • 2
    $\begingroup$ Not the question, but bizarre software design to throw out a Durbin-Watson statistic when there is no sign that you are dealing with time series data. Note that testing for conditional normality may be routine, but as I understand it your noise term is uniform and has positive mean, so the data generation process doesn't match the inferential machinery. $\endgroup$ – Nick Cox Jan 23 '20 at 18:28
  • 2
    $\begingroup$ @Nick Cox I agree that displaying "unwanted" statistics such as D-W here can be really misleading. $\endgroup$ – Rodolphe Jan 23 '20 at 19:14
  • $\begingroup$ Hello @Firebug, Coef is the intercept i think $\endgroup$ – user2707389 Jan 23 '20 at 19:37
  • 2
    $\begingroup$ That plot is not consistent with the summary, because the heights in the plot are around 1400 (as expected from the code) but the output estimates the mean height as around 1750. It looks like the plot and the output might have been produced with different random numbers, but this is rather confusing. $\endgroup$ – whuber♦ Jan 23 '20 at 19:39
7
$\begingroup$

There is no intercept term in this model, so the best-fit line must go through the origin. A best-fit line that goes through the origin will clearly have a positive slope for the data you're showing. It seems including 

x_ran = sm.add_constant(x_ran)

will add the constant term. You should then find that your intercept is significantly diffrent from zero, but the slope of the best-fit line is not.

Improve this answer
$\endgroup$
4
  • 2
    $\begingroup$ Good catch (+1). As I don't use statsmodels, I was surprised to see that the intercept isn't included by default in a linear regression, but that's the case. I'm sure there was some good reason for that choice... $\endgroup$ – EdM Jan 23 '20 at 21:34
  • $\begingroup$ Yeap, this fixed the problem !! $\endgroup$ – user2707389 Jan 23 '20 at 22:02
  • $\begingroup$ @EdM statsmodels model interface was designed under the assumption that the user provides a full array of explanatory variables and just wants to estimate it. The models don't try to second guess the user. The formula interface is more like R (or Stata) and adds an intercept by default. Related, statsmodels does not automatically drop variables if there are problems like perfect collinearity, or perfect separation in Logit (in contrast to some other packages that don't want to bother the user and just pick some arbitrary "fix", although statsmodels uses pinv in OLS which is also a "fix") $\endgroup$ – Josef Jan 24 '20 at 22:40
  • $\begingroup$ @Josef thanks for the explanation. It's often hard on this site to figure out which issues are software-specific and which are statistical. Here it was a little of both. I'll keep in mind this assumption underlying statsmodels going forward. $\endgroup$ – EdM Jan 24 '20 at 22:44
5
$\begingroup$

Whatever X1 is, the p-value you are referring to is so low because the null hypothesis implicitly made by your modeling software/package is that X1 coefficient is equal to zero.

You estimate a coef of 1746 $\pm$ SE 135... Which is definitely different from zero.

Improve this answer
$\endgroup$
4
  • $\begingroup$ @Rodolphe does not the null hypothesis come from the user? As in, using the data i am trying to test a hypothesis. How can you derive null hypothesis from data itself? $\endgroup$ – user2707389 Jan 23 '20 at 19:40
  • $\begingroup$ That being said, i am not sure about the signification of X1. It doesn't seem to be the average of all y_rand that, as @whuber said in the comments, should be around 1400. $\endgroup$ – Rodolphe Jan 23 '20 at 21:16
  • $\begingroup$ Maybe X1 is the slope of the implicit model, probably without intercept as was also pointed in the comments by @Firebug. $\endgroup$ – Rodolphe Jan 23 '20 at 21:16
  • $\begingroup$ @user2707389 this is to me the most obvious signification of the line about X1 in the table $\endgroup$ – Rodolphe Jan 23 '20 at 21:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.