3
$\begingroup$

$\newcommand{\szdp}[1]{\!\left(#1\right)} \newcommand{\szdb}[1]{\!\left[#1\right]}$ Problem Statement: Show that the least squares prediction equation $$\hat{y}=\hat\beta_0+\hat\beta_1x_1+\cdots+\hat\beta_kx_k$$ passes through the point $(\overline{x}_1,\overline{x}_2,\dots,\overline{x}_k,\overline{y}).$

Note: This is Exercise 11.81 in Mathematical Statistics with Applications, 5th Ed., by Wackerly, Mendenhall, and Scheaffer.

My Work So Far: What we need to do is show that $$\overline{y}=\hat\beta_0+\hat\beta_1\overline{x}_1+\hat\beta_2\overline{x}_2 +\cdots+\hat\beta_k\overline{x}_k.$$ Now if we set up the matrices \begin{align*} \mathbf{x}&=\szdb{ \begin{matrix} 1&x_{11}&x_{21}&\cdots&x_{k1}\\ 1&x_{12}&x_{22}&\cdots&x_{k2}\\ \vdots &\vdots &\vdots &\ddots &\vdots\\ 1&x_{1n}&x_{2n}&\cdots&x_{kn} \end{matrix}}\\ \mathbf{a}&=\szdb{ \begin{matrix} 1\\ \overline{x}_1\\ \overline{x}_2\\ \vdots \\ \overline{x}_k \end{matrix} }\\ \mathbf{y}&=\szdb{ \begin{matrix} y_1\\y_2\\ \vdots \\ y_n \end{matrix}}, \end{align*} then we know ${\hat\beta}=(\mathbf{x}^T\mathbf{x})^{-1}\mathbf{x}^T\mathbf{y},$ and we are asked to prove $\overline{y}=\mathbf{a}^T\hat\beta.$ Here $n$ is the number of data points, and $k$ is the number of features.

My Questions: I have no idea where to go from here, or even whether this is the right approach to begin. What's a hint?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
5
$\begingroup$

One place to start would be to replace your $a^T$ vector with $\frac1n$ times a row vector of $n$ 1's times $x$. This is just the matrix version of finding the means ($\bar{x}$) of the columns. When you substitute that (and $\hat{\beta}$) into your last equation you will have a piece that computes means times the hat matrix times the y vector. Look up the properties of the hat matrix and/or remember that $y_i = \hat{y_i} + \hat\epsilon_i$ and remember or look up what the mean of the observed residuals is to continue.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for your answer! Just to make sure I'm understanding, you're recommending $$\mathbf{a}^T=\frac1n[\underbrace{\begin{matrix}1&1&\dots&1\end{matrix}}_{n\times}]?$$ $\endgroup$
    – Adrian Keister
    Oct 1, 2021 at 16:38
  • $\begingroup$ Or do you mean $$\mathbf{a}^T=\frac1n\left[ \underbrace{\begin{matrix} 1 &1 &1 &\dots &1 \end{matrix}}_{n\times} \right]\mathbf{x}?$$ $\endgroup$
    – Adrian Keister
    Oct 1, 2021 at 17:48
  • $\begingroup$ I can see that you meant the latter expression, with $\mathbf{x}$ in it. Now I have a new question: I can see that your suggestion allows me to get to $$\mathbf{a}'\hat\beta =\frac1n\left[ \underbrace{\begin{matrix} 1 &1 &1 &\dots &1 \end{matrix}}_{n\times} \right]Py.$$ But why should $\overline{y}=\overline{\hat{y}}?$ Oh, your last comment. Gotta follow it out! $\endgroup$
    – Adrian Keister
    Oct 1, 2021 at 18:06
4
$\begingroup$

An alternate approach:

  1. If the mean of all the $x$s was zero, the mean point would be $(0,\dots,0,\hat\beta_0)$. The matrix $X^TX$ would be block diagonal, with a block for the intercept and a block for everything else, which will imply $\hat\beta_0=\bar y$. So the fitted (hyper)plane passes through the mean point.

  2. Reparametrising $x_i\mapsto x_i-\bar x_i$ relabels the x axes but it doesn't change whether the fitted (hyper)plane goes through a point. So if the result holds when the $x$s have mean zero, it holds in general

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.