5
$\begingroup$

It's similar to the post Interpretation of R's lm() output.

lm(formula = iris$Sepal.Width ~ iris$Petal.Width)

however, just a point that I can't understand for the explanation of t-value.

enter image description here

It shows that, t-value is the ratio from the first two values

t-value = estimate_mean/std.error

Questions: Is this t-value exactly the t-score in student's t distribution?

Based on my understanding, from the definition, t-score is calculated as follows. enter image description here

If assuming a null hypothesis that response residual mean is 0, the correct t-score in this lm() case, in my understanding, should be as follows.

 t-score given H_null = estimated_mean / (std.error/sqrt(n)) 
                      = sqrt(n) * estimated_mean/std.error

Therefore, t-score I derived is sqrt(n) times larger than t-value given by lm() .... Any one know which part is wrong above? Thanks!

$\endgroup$
3
  • 3
    $\begingroup$ 1. What is the source of the orange image? 2. You seem to be confusing the standard deviation and the standard error in the ordinary one sample t-test (the orange image shows one, then you say the other; they're different). 3. Testing a regression coefficient is not (usually) the same as performing an ordinary one-sample t-test. $\endgroup$
    – Glen_b
    Commented Nov 8, 2015 at 15:10
  • $\begingroup$ @Glen_b, the orange image is the one I summarized from stattrek.com/probability-distributions/t-distribution.aspx. could you provide some more details on how to test a regression coefficient please? $\endgroup$ Commented Nov 8, 2015 at 15:34
  • $\begingroup$ @HappyCoding No, because you have a more fundamental problem as I already explained in my previous point (2). The fact that you just ignored it completely concerns me. Given (3), it might be better to look at the one sample t-statistic throughout, get that sorted out and then if you still have a question about regression, to ask about it separately (since dealing with the other issue will change what you ask). $\endgroup$
    – Glen_b
    Commented Nov 8, 2015 at 15:37

1 Answer 1

2
$\begingroup$

The equation in the orange image is for a single sample. If a sample is drawn from a normal distribution with mean $\mu$ and standard deviation $\sigma$, then $\bar{x}$ is normal with mean $\mu$ and variance $\sigma/\sqrt{n}$.

Hence $\frac{\bar{x}-\mu}{\sigma/\sqrt{n}}$ will have a standard normal distribution. However, if we estimate $\sigma/\sqrt{n}$ from the sample, by the estimated standard error of the mean, $s/\sqrt{n}$, (where $s$ is the standard deviation of the sample), then

$\frac{\bar{x}-\mu}{std.err(\bar{x})}=\frac{\bar{x}-\mu}{s/\sqrt{n}}$ has a $t$ distribution with $n-1$ degrees of freedom.

The orange image is consistent with what I just said.

But then you said:

t-score given H_null = estimated_mean / (std.error/sqrt(n))

This is not correct. Note that the standard error of the mean is $s/\sqrt{n}$, where $s$ is the standard deviation of the data.

The t-statistic in regression is slightly different (though analogous in form; its $t=\frac{b-\beta}{s.e.(b)}$ where $b$ is the estimated coefficient).

It's important to understand the problem here with the more basic case first.

$\endgroup$
4
  • $\begingroup$ thanks @Glen_b for pointing out t distribution is different from t-statistic in regression. follow yours suggestion, I did some search and read in stattrek.com/regression/slope-test.aspx?Tutorial=AP. it's much clearer now. $\endgroup$ Commented Nov 9, 2015 at 1:48
  • $\begingroup$ @HappyCoding Thanks for the heads up; I'll edit my answer a tiny bit; that should make things clearer for later readers (because there's fewer sources of confusion, the central issue about the standard error is clearer) $\endgroup$
    – Glen_b
    Commented Nov 9, 2015 at 1:52
  • $\begingroup$ thanks @Glen_b! That's great! It could be even better if you can also explain a bit on the differences and connections between t-statistic in sampling and regression, besides the formula difference as you pointed out above. $\endgroup$ Commented Nov 9, 2015 at 1:55
  • $\begingroup$ thanks. for comments are very helpful. I just highlighted here Note that the standard error of the mean is s/sqrt(n), where s is the standard deviation of the data. $\endgroup$ Commented Nov 9, 2015 at 2:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.