0
$\begingroup$

I am having a problem with OLS regression. I am doing a relative time model (leads and lags model) to prove the robustness of my Difference in differences model. Problem is that some of my coefficients and standard errors are having 0 value. I think it means that there's something wrong with the model but I don't know what's causing the problem. Here are the results.

enter image description here

dist_5 is when the treatment effect first appeared. I want to prove that there is a positive significant effect starting from dist_5 (lags) and no significant effect on the leads.

Thank you

+) I'm running the model with python and the result is too long to attach so I had to cut the rest of the coefficients.

These are the coefficient of each parameters; enter image description here

and these are the standard error of the parameters; enter image description here

This is what the result summary looks like (not full result); enter image description here

$\endgroup$
7
  • $\begingroup$ Are you running the model using spreadsheet software or just using it to present the results ? Can you show us the actual results using actual statistical software ? $\endgroup$ Commented Jul 2, 2020 at 7:03
  • 1
    $\begingroup$ @RobertLong I just edited my question and added images of the results. $\endgroup$
    – kyrhee
    Commented Jul 2, 2020 at 7:20
  • $\begingroup$ Ok, so it's not that they are zero. It's just that they are very small, right ? $\endgroup$ Commented Jul 2, 2020 at 7:26
  • $\begingroup$ @RobertLong Yes. The coefficients and standard errors of the leads are too small (near zero) that I think I must have done something wrong or misunderstood the relative time model (leads and lags model). I saw some papers regarding the model but none of them had coefficients and std errors that were nearly zero. $\endgroup$
    – kyrhee
    Commented Jul 2, 2020 at 7:30
  • 1
    $\begingroup$ I think its unlikely that we can tell what, if anything, is wrong, without seeing the data. It looks like the results support what you are trying to "prove", but please note that you can't "prove" anything with statistics $\endgroup$ Commented Jul 2, 2020 at 7:36

1 Answer 1

3
$\begingroup$

The standard errors of your estimated coefficients

The standard errors of your estimated coefficients, $\hat\beta$, are estimated according to:

$$\widehat{s.e.}(\hat\beta_j ) = \sqrt{s^2(X^TX)_{jj}^{-1}}$$

where $s^2$ is an estimate of the deviation/variance of the noise and $X$ is the matrix of your regressors.

  1. In other words the standard error of the coefficient $\beta_j$ scales with the inverse of the root mean square of the column/regressor $X_j$. (you should be able to verify this easily which helps you with debugging)

  2. Note that these are marginal errors and the formula can be generalized to provide a covariance matrix of the variance of the estimated coefficients. If there is correlation then the marginal errors might be very large but will not reflect the distribution of error correctly


Example of direct (manual) computation of standard error of coefficients

Computational example (in R, but I hope that the annotations make it clear for you to turn it into python code):

###
### creating data 
###
set.seed(1)
 
### matrix with three columns
x <- cbind(c(1,1,1,1,1,1),
           c(1,2,3,4,5,6),
           c(0,0,0,1,1,1))

### some output based on the second column
### with added normal distributed noise with sigma=0.1
y <- x[,2] + rnorm(6,0,0.1)

### computing the model with the standard lm function
mod <- lm(y~0+x)
summary(mod)

### computing the std.err manually based on
### sd of the residuals
### the inverse (using the function solve) of the matrix product t(x)*x
### a correction sqrt(5/3) because the residuals should be based on n-p degrees of freedom and not n-1
###                                                                (and we have p=3, three parameters)
sd(residuals(mod))*sqrt(solve(t(x) %*% x)[c(1,5,9)])*sqrt(5/3)

output lm function:

> summary(mod)

Call:
lm(formula = y ~ 0 + x)

Residuals:
       1        2        3        4        5        6 
-0.08565  0.06098  0.02467  0.05709 -0.00386 -0.05323 

Coefficients:
   Estimate Std. Error t value Pr(>|t|)    
x1  0.08863    0.08888   0.997 0.392139    
x2  0.93438    0.03848  24.279 0.000153 ***
x3  0.27629    0.13145   2.102 0.126334    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.07697 on 3 degrees of freedom
Multiple R-squared:  0.9998,    Adjusted R-squared:  0.9996 
F-statistic:  5126 on 3 and 3 DF,  p-value: 4.625e-06

output manual:

> sd(residuals(mod))*sqrt(solve(t(x) %*% x)[c(1,5,9)])*sqrt(5/3)
[1] 0.08887551 0.03848422 0.13144865

This above code shows where those standard errors 0.089, 0.038, 0.131 come from.


Check the root mean square size of your regressors

So in your situation you probably have some very large (root mean squared) size for several of the columns/regressors $\vert X_j \vert$. And that is why the parameters (and the estimated error) have such low values.

(I would also suspect that there might be some computational errors, however this is difficult to inspect with the given information. Anyway, at least you can now verify with this answer whether it is the standard case, ie. the relation with $X^TX$)

In addition: Possibly you might want to do a mixed effects model? Those fixed effects per user do they make sense to be considered fixed effects, or can we more specifically regard them as following a normal distribution?

$\endgroup$
1
  • $\begingroup$ Thank you for giving a clear and detailed reply. I tried computing the standard errors manually and found out that there was nothing wrong with the regression and the problem was coming from another source. Anyways, your answer was very helpful. Thanks! $\endgroup$
    – kyrhee
    Commented Jul 3, 2020 at 9:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.