I am having troubles in deriving the RHS of this formula:

$$RSS= \hat{\beta}^{T}Z^{T}y-n\bar{y}^{2}$$ where $Z$ is the design matrix

I am given ${y}^{T}y$ (what exactly does ${y}^{T}y$ stand for and how does it help me with the above formula?)


You denote the expression as RSS, which would suggest that it is the residual sum of squares. However, the expression that you give is the explained sum of squares (ESS).

In order to derive this expression you should work with the elements in the following relationship, which holds when the regression model contains an intercept (TSS stands for the total sum of squares):

$$ \underbrace{y^{T}y}_{TSS} = \underbrace{\hat{u}^T\hat{u}}_{RSS} + \underbrace{\hat{y}^T\hat{y}}_{ESS} \,. $$

$y^{T}y$ is the sum of the squares of the dependent variable; $\hat{u}$ is the vector of residuals of the fitted model and $\hat{y}$ is the vector of fitted values.

It can be shown that the RSS can be written as (to see this replace the expression of the Ordinary Least Squares estimator $\hat{\beta}$ in the definition of the residuals $\hat{u} = y - Z\hat{\beta}$ and note that $My=Mu$):

$$ \hat{u}^T\hat{u} = u^TM u = y^T M y \,, \quad \hbox{where} \quad M = I - Z(Z^T Z)^{-1} Z^ T \quad \hbox{(idempotent matrix)} \,. $$

Then, we have:

$$ \hat{u}^T\hat{u} = y^{T}y - y^T Z\underbrace{(Z^TZ)^{-1}Z^Ty}_{\hat{\beta}} = y^Ty - \hat{\beta}^T Z^Ty \,. $$

Plugging this expression for the RSS in the relationship given above and subtracting in both sides the term $T\bar{y}^2$ yields the expression that you have.

  • $\begingroup$ thank you, but in our lectures we learnt RSS is the regression sum of squares and that is what I am looking for $\endgroup$ – Silver May 3 '15 at 19:48
  • $\begingroup$ @Silver you can get one or another from the relationship $TSS = RSS + ESS$. After removing $T\bar{y}^2$ in both sides of this relationship, you can rewrite for example $RSS = y^Ty - (\hat{\beta}^TZ^Ty - T\bar{y}^2)$. $\endgroup$ – javlacalle May 3 '15 at 20:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.