Algebraic definition of a residual from a regression

Question

Can anyone give some advice on how to start proving this algebraically?

Define the residual from a regression (one independent variable) algebraically and show that:

1. the mean of the residuals is zero
2. the correlation of the residuals and the independent variable is zero

Answer 1

Suppose you have the following regression model:

$$ y_i=\alpha+\beta x_i+\varepsilon_i $$

Least squares problem looks for $\alpha$ and $\beta$ which minimize the following function:

$$g(\alpha,\beta)=\sum_{i=1}^n(y_i-\alpha-\beta x_i)^2$$

Solution for this problem will satisfy

$$\frac{\partial g}{\partial \alpha}=0, \quad \frac{\partial g}{\partial \beta}=0.$$

Try differencing and look hard at the resulting expressions. You will see that this answers your problem.

Note that @cardinal is right, if you do not include $\alpha$, your first statement is false.

Update This is might be considered non-algebraic solution, so please state more clearly what do you mean by algebraic. If this is not helpful, I will retract my answer, which really is a long comment.