Suppose I have a toy example for linear regression




Whats the relationship between Std. Error and Confidence Interval (as shown in red boxes in the figure?)

enter image description here

I remember from some book, they are different things, one is point estimation another is a random interval estimation (population parameter is fixed and unknown). But is there any relationship between them?

  • $\begingroup$ I do not use R, but checking in Python it looks like confint is a 95% confidence interval based on the estimate and std. error, assuming the standardized PDF is a $t$ distribution with DF degrees of freedom? $\endgroup$ – GeoMatt22 Feb 15 '17 at 22:39
  • $\begingroup$ @GeoMatt22 I am little confused with "std of point estimation" vs "confidence interval" that is why I am asking this. $\endgroup$ – Haitao Du Feb 16 '17 at 0:17
  • $\begingroup$ OK. My earlier comment was just about the numerical relationship. I believe that both of your quoted terms would be in reference to the inferred sampling distribution of the statistic (as estimated from the data, i.e. that would be why a $t$ PDF vs. a standard-normal one). $\endgroup$ – GeoMatt22 Feb 16 '17 at 0:21
  • 1
    $\begingroup$ A better explanation is given by @gung here. $\endgroup$ – GeoMatt22 Feb 16 '17 at 0:39
  • $\begingroup$ Let us continue this discussion in chat. $\endgroup$ – GeoMatt22 Feb 16 '17 at 1:15

My attempt to answer (thanks GeoMat22 !!)

Confidence Interval minus point estimation divided by standard error of point estimation will satisfy T distribution with corresponding degree of freedom.

Here is the verification !

> point_est=fit$coefficients
> point_est_se=coef(summary(fit))[, "Std. Error"]
> c95_ci=as.vector(confint(fit))
> t_bnd=(c95_ci-point_est)/point_est_se
> pt(t_bnd,df=49)
[1] 0.025 0.975


> point_est+qt(0.025,df=49)*point_est_se
> point_est+qt(0.975,df=49)*point_est_se
> c95_ci
[1] 0.2859806 0.3944128
  • $\begingroup$ The implied underlying logic is something like: "The $z$ score of coefficients obtained from repeated experiments, when standardized using the point estimate + standard error of the current experiment, is expected to follow a $t$ PDF with $df$ degrees of freedom". $\endgroup$ – GeoMatt22 Feb 16 '17 at 2:05

In normal/Gaussian distribution, the confidence intervals can be derived as follows:

$$\text{95% CI lower limit} = \text{mean} - 1.96 \times \text{SE}$$

$$\text{95% CI upper limit} = \text{mean} + 1.96 \times \text{SE}$$

In your case, your mean estimate (or better, your estimated mean for the coefficient) is $0.34020$ and your standard error (SE)) is $0.02698$

Plug that into the equation above, and you get (roughly) the same values of your confidence interval.

I assume (but may be wrong) that the small difference in some of the digits after the decimal are due to the output rounding the model results to certain number of digits, while the confint(fit) probably uses the actual number (with as many digits as it has "inside").

  • 2
    $\begingroup$ This is close, but you may notice that the (CI - estimate)/std_err values give a systematically wider "z" interval, more like +/-2.01 (far larger than expected from rounding, given the # digits reported in the OP's output). This suggests some "finite sample uncertainty" correction, and indeed is consistent with a "t" interval using the reported $df = 49$ degrees of freedom. BTW if you delete this answer the negative rep. from the downvotes will disappear. (I did not downvote, but the above may be why some did.) $\endgroup$ – GeoMatt22 Feb 16 '17 at 1:53
  • 2
    $\begingroup$ I do not mind being down voted, but it is better to have some explanation on why it is wrong. $\endgroup$ – Haitao Du Feb 16 '17 at 2:12
  • 1
    $\begingroup$ @hxd1011 I agree, and commenting is definitely encouraged. $\endgroup$ – GeoMatt22 Feb 16 '17 at 2:23
  • $\begingroup$ I also don't see the point why this was down voted. It could have been easily demonstrated in the comments that a confidence interval based on the t-distribution is calculated as the standard error of the estimate multiplied with the quantile function for the t-distribution and the respective degrees of freedom. The reason why you got close is because in @hxd1011 example the degrees of freedom were large (in R compare qt(0.025, 49) vs. qnorm(0.025), see also here). $\endgroup$ – Stefan Feb 16 '17 at 4:15
  • 1
    $\begingroup$ I also would not mind understanding why the answer was downvoted, so any hints appreciated. $\endgroup$ – Tilen Feb 16 '17 at 9:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.