Interpretation of R's lm() output

Question

the help pages in R assume I know what those numbers mean. I don't :) I'm trying to really intuitively understand every number here. I will just post the output and comment on what I found out. There might (will) be mistakes, as I'll just write what I assume. Please correct me, and I will edit the wrong parts.
Mainly I'd like to know what the t-value in the coefficients mean, and why they print the residual standard error. I hope someone can clarify that.

Call:
lm(formula = iris$Sepal.Width ~ iris$Petal.Width)

Residuals:
     Min       1Q   Median       3Q      Max 
-1.09907 -0.23626 -0.01064  0.23345  1.17532 

A 5-point-summary of the residuals (Their mean is always 0, right?). The numbers can be used (I'm guessing here) to quickly see if there are any big outliers. Also you can already see it here if the residuals are far from normal distributed (which they should be).

Coefficients:
                 Estimate Std. Error t value Pr(>|t|)    
(Intercept)       3.30843    0.06210  53.278  < 2e-16 ***
iris$Petal.Width -0.20936    0.04374  -4.786 4.07e-06 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Estimates $\hat{\beta_i}$ , computed by least squares regression. Also, the standard error $\sigma_{\beta_i}$ . I'd like to know how this is calculated.
Also, no idea where the t value and the corresponding p come from. I know $\hat{\beta}$ should be normal distributed, but how is the t value calculated?

Residual standard error: 0.407 on 148 degrees of freedom

$\sqrt{ \frac{1}{n-p} \epsilon^T\epsilon }$ , I guess. But why do we calculate that, and what does it say us?

Multiple R-squared: 0.134,  Adjusted R-squared: 0.1282 

$ R^2 = \frac{s_\hat{y}^2}{s_y^2} $ , which is $ \frac{\sum_{i=1}^n (\hat{y_i}-\bar{y})^2}{\sum_{i=1}^n (y_i-\bar{y})^2} $ . The ratio is close to 1 if the points lie on a straight line, and 0 if they are random.
What is the adjusted R-squared?

F-statistic: 22.91 on 1 and 148 DF,  p-value: 4.073e-06 

F and p for the whole model, not only for single $\beta_i$s as previous. The F value is $ \frac{s^2_{\hat{y}}}{\sum\epsilon_i} $ . The bigger it grows, the more unlikely it is that the $\beta$'s do not have any effect at all.

Answer 1

Five point summary: yes, the idea is to give a quick summary of the distribution. It should be roughly symmetrical about mean, the median should be close to 0, the 1Q and 3Q values should ideally be roughly of similar absolute magnitude etc.

Each coefficient in the model is a Gaussian (Normal) random variable. The $\hat{\beta_i}$ is the estimate of the mean of the distribution of that random variable, and the standard error is the square root of the variance of that distribution. It is a measure of the uncertainty in the estimate of the $\hat{\beta_i}$. You can look at how these are computed (well the mathematical formulae used) on here. Note that any self-respecting stats programme will not use the standard mathematical equations to compute the $\hat{\beta_i}$ because doing them on a computer can lead to a large loss of precision in the computations.

The $t$ statistics are the estimates ($\hat{\beta_i}$) divided by their standard errors ($\hat{\sigma_i}$), e.g. $t_i = \frac{\hat{\beta_i}}{\hat{\sigma_i}}$. Assuming you have the same model in object modas your Q:

> mod <- lm(Sepal.Width ~ Petal.Width, data = iris)

then the $t$ values R reports are computed as:

> (tstats <- coef(mod) / sqrt(diag(vcov(mod))))
(Intercept) Petal.Width 
  53.277950   -4.786461 

Where coef(mod) are the $\hat{\beta_i}$, and sqrt(diag(vcov(mod))) gives the square roots of the diagonal elements of the covariance matrix of the model parameters, which are the standard errors of the parameters ($\hat{\sigma_i}$).

The p-value is the probability of achieving a value of $t$ as larger or larger if the null hypothesis were true. Here the null hypothesis is the $\hat{\beta_i}$ are individually 0. They are computed as (using tstats from above):

> 2 * pt(abs(tstats), df = df.residual(mod), lower.tail = FALSE)
 (Intercept)  Petal.Width 
1.835999e-98 4.073229e-06

So we compute the upper tail probability of achieving the $t$ values we did from a $t$ distribution with degrees of freedom equal to the residual degrees of freedom of the model. This represents the probability of achieving a $t$ value greater than the absolute values of the observed $t$s. It is multiplied by 2, because of course $t$ can be large in the negative direction too.

The residual standard error is an estimate of the parameter $\sigma$. The assumption in ordinary least squares is that the residuals are individually described by a Guassian (normal) distribution with mean 0 and standard deviation $\sigma$. The $\sigma$ relates to the constant variance assumption; each residual has the same variance and that variance is equal to $\sigma^2$.

The adjusted $R^2$ is the same thing as $R^2$, but adjusted for the complexity of the model, i.e. the number of parameters in the model. Think about things for a minute. If we have a model with a single parameter. It will have a certain $R^2$. If we add another parameter to this model, the $R^2$ of the new model has to increase, even if the added parameter has no statistical power. The adjusted $R^2$ tries to account for this, by including information on the number of parameters in the model.

Adjusted $R^2$ is computed as:

$$1 - (1 - R^2) \frac{n - 1}{n - p - 1}$$

The $F$ is the ratio of two variances, the variance explained by the parameters in the model and the residual or unexplained variance. You can see this better if we get the ANOVA table for the model via anova():

> anova(mod)
Analysis of Variance Table

Response: Sepal.Width
             Df  Sum Sq Mean Sq F value    Pr(>F)    
Petal.Width   1  3.7945  3.7945   22.91 4.073e-06 ***
Residuals   148 24.5124  0.1656                      
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

First, notice the $F$s are the same in the ANOVA output and the summary(mod) output. The Mean Sq column contains the two variances and $3.7945 / 0.1656 = 22.91$. We can compute the probability of achieving an $F$ that large under the null hypothesis of no effect, from an $F$-distribution with 1 and 148 degrees of freedom. This is what is reported in the final column of the ANOVA table. In the simple case of a single, continuous predictor (as per your example), $F = t_{\mathrm{Petal.Width}}^2$, which is why the p-values are the same. This equivalence only holds in this simple case.