Skip to main content
1 of 2
Siong Thye Goh
  • 7k
  • 3
  • 21
  • 31

Let's go from the right to the left.

\begin{align} n\sum_{i=1}^n (x_i - \bar{x})^2 &= n\sum_{i=1}^n (x_i^2-2\bar{x} x_i +\bar{x}^2)\\&= n\sum_{i=1}^n x_i^2-2n\bar{x} \sum_{i=1}^nx_i + n^2 \bar{x}^2 \\ &= n\sum_{i=1}^n x_i^2-2n^2\bar{x} \left( \frac{\sum_{i=1}^nx_i}{n} \right) + n^2 \bar{x}^2 \\ &=n\sum_{i=1}^n x_i^2- n^2 \bar{x}^2 \\ \end{align}

I will leave the last step for you to simplify.


Alternatively, here is an approach in terms of matrices, we want to minimize

$$L(\|A\tilde{z}-y\|^2$$ where $A$ include a column of $1$ to handle the intercept term and $\tilde{z}$ include the intercept term as well.

Differentiate it, we obtain $$\nabla_z(L)=A^T(A\tilde{x}-y)$$

and $$\nabla_z^2(L)=A^TA \succeq 0$$

Siong Thye Goh
  • 7k
  • 3
  • 21
  • 31