3
$\begingroup$

$\newcommand{\szdp}[1]{\!\left(#1\right)} \newcommand{\szdb}[1]{\!\left[#1\right]}$ Problem Statement: Show that the least squares prediction equation $$\hat{y}=\hat\beta_0+\hat\beta_1x_1+\cdots+\hat\beta_kx_k$$ passes through the point $(\overline{x}_1,\overline{x}_2,\dots,\overline{x}_k,\overline{y}).$

Note: This is Exercise 11.81 in Mathematical Statistics with Applications, 5th Ed., by Wackerly, Mendenhall, and Scheaffer.

My Work So Far: What we need to do is show that $$\overline{y}=\hat\beta_0+\hat\beta_1\overline{x}_1+\hat\beta_2\overline{x}_2 +\cdots+\hat\beta_k\overline{x}_k.$$ Now if we set up the matrices \begin{align*} \mathbf{x}&=\szdb{ \begin{matrix} 1&x_{11}&x_{21}&\cdots&x_{k1}\\ 1&x_{12}&x_{22}&\cdots&x_{k2}\\ \vdots &\vdots &\vdots &\ddots &\vdots\\ 1&x_{1n}&x_{2n}&\cdots&x_{kn} \end{matrix}}\\ \mathbf{a}&=\szdb{ \begin{matrix} 1\\ \overline{x}_1\\ \overline{x}_2\\ \vdots \\ \overline{x}_k \end{matrix} }\\ \mathbf{y}&=\szdb{ \begin{matrix} y_1\\y_2\\ \vdots \\ y_n \end{matrix}}, \end{align*} then we know ${\hat\beta}=(\mathbf{x}^T\mathbf{x})^{-1}\mathbf{x}^T\mathbf{y},$ and we are asked to prove $\overline{y}=\mathbf{a}^T\hat\beta.$ Here $n$ is the number of data points, and $k$ is the number of features.

My Questions: I have no idea where to go from here, or even whether this is the right approach to begin. What's a hint?

$\endgroup$

2 Answers 2

5
$\begingroup$

One place to start would be to replace your $a^T$ vector with $\frac1n$ times a row vector of $n$ 1's times $x$. This is just the matrix version of finding the means ($\bar{x}$) of the columns. When you substitute that (and $\hat{\beta}$) into your last equation you will have a piece that computes means times the hat matrix times the y vector. Look up the properties of the hat matrix and/or remember that $y_i = \hat{y_i} + \hat\epsilon_i$ and remember or look up what the mean of the observed residuals is to continue.

$\endgroup$
3
  • $\begingroup$ Thanks for your answer! Just to make sure I'm understanding, you're recommending $$\mathbf{a}^T=\frac1n[\underbrace{\begin{matrix}1&1&\dots&1\end{matrix}}_{n\times}]?$$ $\endgroup$ Oct 1, 2021 at 16:38
  • $\begingroup$ Or do you mean $$\mathbf{a}^T=\frac1n\left[ \underbrace{\begin{matrix} 1 &1 &1 &\dots &1 \end{matrix}}_{n\times} \right]\mathbf{x}?$$ $\endgroup$ Oct 1, 2021 at 17:48
  • $\begingroup$ I can see that you meant the latter expression, with $\mathbf{x}$ in it. Now I have a new question: I can see that your suggestion allows me to get to $$\mathbf{a}'\hat\beta =\frac1n\left[ \underbrace{\begin{matrix} 1 &1 &1 &\dots &1 \end{matrix}}_{n\times} \right]Py.$$ But why should $\overline{y}=\overline{\hat{y}}?$ Oh, your last comment. Gotta follow it out! $\endgroup$ Oct 1, 2021 at 18:06
4
$\begingroup$

An alternate approach:

  1. If the mean of all the $x$s was zero, the mean point would be $(0,\dots,0,\hat\beta_0)$. The matrix $X^TX$ would be block diagonal, with a block for the intercept and a block for everything else, which will imply $\hat\beta_0=\bar y$. So the fitted (hyper)plane passes through the mean point.

  2. Reparametrising $x_i\mapsto x_i-\bar x_i$ relabels the x axes but it doesn't change whether the fitted (hyper)plane goes through a point. So if the result holds when the $x$s have mean zero, it holds in general

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.