-1
$\begingroup$

For data $x=\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}$ and $y=\begin{bmatrix} 4 \\ 5 \\ 3 \end{bmatrix}$, fit a linear model $y=ax+b+\varepsilon$ using the formula $\hat{\beta}=\left ( X^TX\right)^{-1}X^TY$.

This exercise should be really simple, but I have some doubts about my reasoning.

Let $Y_i = \beta_0 + \beta_1X_i+\varepsilon$ describe the i-th sample. The matrix of regressors is $X=\begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 3\end{bmatrix}$.

Therefore $\left ( X^TX\right)^{-1} = \begin{bmatrix} 14 & 6\\ 6 & 3 \end{bmatrix}^{-1} = \begin{bmatrix} \frac{7}{3} & 1\\ 1 & \frac{1}{2}\end{bmatrix}$ and $X^TY=\begin{bmatrix}12\\ 23\end{bmatrix}$.

The product of the matrix multiplication is:

$\hat{\beta}=\left ( X^TX\right)^{-1}X^TY = \begin{bmatrix} \frac{7}{3} & 1\\ 1 & \frac{1}{2}\end{bmatrix} \cdot \begin{bmatrix}12\\ 23\end{bmatrix} = \begin{bmatrix}40\\ 18.5\end{bmatrix}$

Thus $\beta_0 = b = 40$ and $\beta_1 = a = 18.5$, meaning that the final linear model would be:

$y = 18.5x + 40 + \varepsilon$.

Hover, this seems unlikely, because graph of such line would be way above the given points. Am I making a mistake in my calculations? If so, where my understanding is incorrect?

$\endgroup$

closed as off-topic by gung Nov 30 '17 at 2:22

  • This question does not appear to be about statistics within the scope defined in the help center.
If this question can be reworded to fit the rules in the help center, please edit the question.

1
$\begingroup$

Your matrix of regressors is $$X=\begin{bmatrix} 1 & 1 \\ 1 & 2 \\ 1& 3\end{bmatrix}.$$

Then $$(X^TX)^{-1}=\begin{bmatrix} 3 & 6 \\ 6 & 14 \end{bmatrix}^{-1}=\begin{bmatrix} \frac{7}{3} & -1 \\ -1 & \frac{1}{2} \end{bmatrix}.$$

Also $$X^Ty=\begin{bmatrix} 12 \\ 23\end{bmatrix}.$$

Together, $$(X^TX)^{-1}(X^Ty)=\begin{bmatrix} 5 \\ -\frac{1}{2}\end{bmatrix}.$$

Your regression equation is $$y=5-\frac{1}{2}x+\varepsilon.$$

The predicted points you will find, with this regression line, are $$\hat{y}=5-\frac{1}{2}\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}=\begin{bmatrix} \frac{9}{2} \\ 4 \\ \frac{7}{2} \end{bmatrix}=\begin{bmatrix} 4.5 \\ 4 \\ 3.5 \end{bmatrix},$$

which all are close to the true values.

$\endgroup$
  • $\begingroup$ Please be cautious about answering homework questions. Our policies are detailed in the [self-study] tag's wiki. $\endgroup$ – gung Nov 30 '17 at 2:23
  • $\begingroup$ I thought the criteria was showing how much work they had already done and what approaches they had tried. I apologize. Thank you. $\endgroup$ – Anna SdTC Nov 30 '17 at 2:29
  • 1
    $\begingroup$ No, no, @AnnaSdTC, there is no need to apologize. My comment is just a heads up. $\endgroup$ – gung Nov 30 '17 at 2:34

Not the answer you're looking for? Browse other questions tagged or ask your own question.