4
$\begingroup$

I have implemented a linear regression in R (lm) and would like to show the significance and direction of the coefficient by means of the t-value. But now I'm not sure how to compute the critical t-value which makes the coefficient significant.

$\endgroup$
  • 1
    $\begingroup$ Critical t-values are mostly a concept of the past. Students are still taught them because they are not allowed to use a computer during exams and have to used t-tables. In practice, you use software to calculate a p-value (or, preferably, a confidence interval). Since you are using software, you should calculate p-values and, in fact, the summary function calculates them for you. $\endgroup$ – Roland Feb 12 at 6:24
  • $\begingroup$ Yeah I totally agree, but in this case I need to make an exception and present the results solely based on the t-values, which is why I need the critical values. $\endgroup$ – Guiseppe Feb 12 at 10:17
  • 1
    $\begingroup$ So? Just calculate them. See help("qt"). $\endgroup$ – Roland Feb 12 at 10:19
  • $\begingroup$ So like this qt(.975, 200), if I want the 0.05 significance and have 200 degrees of freedom? I though the standard error must be part of it? $\endgroup$ – Guiseppe Feb 12 at 10:21
  • 1
    $\begingroup$ You need the standard error to calculate t-values. Critical t-values are calculated from the distribution. No standard error needed. $\endgroup$ – Roland Feb 12 at 10:23
1
$\begingroup$

I'm going to construe the question as meaning: What is the t-statistic and what is its probability distribution?

You have

\begin{align} Y_i = {} & \beta_0 +\beta_1 x_i + \varepsilon_i \quad \text{for } i=1,\ldots,n \\[10pt] & \varepsilon_1,\ldots,\varepsilon_n \sim \operatorname{iid} \operatorname N(0,\sigma^2). \\[10pt] \widehat\beta_1 = {} & \frac{\sum_{i=1}^n (Y_i-\overline Y)(x_i-\overline x)}{\sum_{i=1}^n (x_i - \overline x)^2} \\[10pt] & \text{where } \overline Y = (Y_1+\cdots+Y_n)/n, \\ & \text{ and } \overline x = (x_1 + \cdots + x_n)/n. \\[10pt] \text{and } \overline Y = {} & \widehat\beta_0 + \widehat\beta_1 \overline x. \quad (\text{This defines } \widehat\beta_0.) \\[10pt] \widehat{\varepsilon\,}_i = {} & Y_i -\left( \widehat\beta_0 + \widehat\beta_1 x_i \right). \end{align} Then

  • $\widehat\beta_1 \sim \operatorname N\left( \beta_1, \dfrac{\sigma^2}{\sum_{i=1}^n (x_i - \overline x)^2} \right)$

  • $\dfrac{\widehat{\varepsilon\,}_1^2 + \cdots + \widehat{\varepsilon\,}_n^2}{\sigma^2} \sim \chi^2_{n-2}.$

  • $\widehat\beta_1$ and $\widehat\sigma^2 = \dfrac{\widehat{\varepsilon\,}_1^2 + \cdots + \widehat{\varepsilon\,}_n^2}{n-2}$ are independent.

From these it follows that $$ \frac{\widehat\beta_1 - \beta_1}{\widehat\sigma/\sqrt{n-2}} \sim t_{n-2}. $$ Therefore $$ \widehat\beta_1 \pm A \frac{\widehat\sigma}{\sqrt n} $$ are the endpoints of a confidence interval for $\beta_1,$ where $A$ is a suitable percentage point of the $t_{n-2}$ distribution.

Here I have not included proofs of the points following the three typographical bullets above. Possibly proofs of those have been posted here before.

$\endgroup$
0
$\begingroup$

That answer by Michael Hardy gives you the formulae for manual calculation for simple linear regression. Since you have implemented your model in R the easiest thing would be just to generate the outputs using standard commands in that program:

#Fit a linear regression model
#Substitute the actual names of your data frame and variables
MODEL <- lm(y ~ x1 + ... + xm, data = DATA);

#Print structure of model
str(MODEL);

#Print summary output
summary(MODEL);

#Print ANOVA table
anova(MODEL);

#Generate studentised residuals
RESID <- resid(MODEL);

The summary table in the R output contains the coefficient estimates table, which includes the t-statistics and associated p-values for each of the terms in the model.

$\endgroup$
  • $\begingroup$ Many thanks, I agree that the summary table provides enough information for the significance. However, I need to present the results only with the t-values and thus I need the critical values. How can I do this particularly in R? $\endgroup$ – Guiseppe Feb 12 at 10:19

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.