Skip to main content
added 74 characters in body
Source Link
Siong Thye Goh
  • 7k
  • 3
  • 21
  • 31

Let's go from the right to the left.

\begin{align} n\sum_{i=1}^n (x_i - \bar{x})^2 &= n\sum_{i=1}^n (x_i^2-2\bar{x} x_i +\bar{x}^2)\\&= n\sum_{i=1}^n x_i^2-2n\bar{x} \sum_{i=1}^nx_i + n^2 \bar{x}^2 \\ &= n\sum_{i=1}^n x_i^2-2n^2\bar{x} \left( \frac{\sum_{i=1}^nx_i}{n} \right) + n^2 \bar{x}^2 \\ &=n\sum_{i=1}^n x_i^2- n^2 \bar{x}^2 \\ \end{align}

I will leave the last step for you to simplify.

By the way, the conclusion should be nonnegative rather than positive.


Alternatively, here is an approach in terms of matrices, we want to minimize

$$L(\|A\tilde{z}-y\|^2$$ where $A$ include a column of $1$ to handle the intercept term and $\tilde{z}$ include the intercept term as well.

Differentiate it, we obtain $$\nabla_z(L)=A^T(A\tilde{x}-y)$$

and $$\nabla_z^2(L)=A^TA \succeq 0$$

Let's go from the right to the left.

\begin{align} n\sum_{i=1}^n (x_i - \bar{x})^2 &= n\sum_{i=1}^n (x_i^2-2\bar{x} x_i +\bar{x}^2)\\&= n\sum_{i=1}^n x_i^2-2n\bar{x} \sum_{i=1}^nx_i + n^2 \bar{x}^2 \\ &= n\sum_{i=1}^n x_i^2-2n^2\bar{x} \left( \frac{\sum_{i=1}^nx_i}{n} \right) + n^2 \bar{x}^2 \\ &=n\sum_{i=1}^n x_i^2- n^2 \bar{x}^2 \\ \end{align}

I will leave the last step for you to simplify.


Alternatively, here is an approach in terms of matrices, we want to minimize

$$L(\|A\tilde{z}-y\|^2$$ where $A$ include a column of $1$ to handle the intercept term and $\tilde{z}$ include the intercept term as well.

Differentiate it, we obtain $$\nabla_z(L)=A^T(A\tilde{x}-y)$$

and $$\nabla_z^2(L)=A^TA \succeq 0$$

Let's go from the right to the left.

\begin{align} n\sum_{i=1}^n (x_i - \bar{x})^2 &= n\sum_{i=1}^n (x_i^2-2\bar{x} x_i +\bar{x}^2)\\&= n\sum_{i=1}^n x_i^2-2n\bar{x} \sum_{i=1}^nx_i + n^2 \bar{x}^2 \\ &= n\sum_{i=1}^n x_i^2-2n^2\bar{x} \left( \frac{\sum_{i=1}^nx_i}{n} \right) + n^2 \bar{x}^2 \\ &=n\sum_{i=1}^n x_i^2- n^2 \bar{x}^2 \\ \end{align}

I will leave the last step for you to simplify.

By the way, the conclusion should be nonnegative rather than positive.


Alternatively, here is an approach in terms of matrices, we want to minimize

$$L(\|A\tilde{z}-y\|^2$$ where $A$ include a column of $1$ to handle the intercept term and $\tilde{z}$ include the intercept term as well.

Differentiate it, we obtain $$\nabla_z(L)=A^T(A\tilde{x}-y)$$

and $$\nabla_z^2(L)=A^TA \succeq 0$$

Source Link
Siong Thye Goh
  • 7k
  • 3
  • 21
  • 31

Let's go from the right to the left.

\begin{align} n\sum_{i=1}^n (x_i - \bar{x})^2 &= n\sum_{i=1}^n (x_i^2-2\bar{x} x_i +\bar{x}^2)\\&= n\sum_{i=1}^n x_i^2-2n\bar{x} \sum_{i=1}^nx_i + n^2 \bar{x}^2 \\ &= n\sum_{i=1}^n x_i^2-2n^2\bar{x} \left( \frac{\sum_{i=1}^nx_i}{n} \right) + n^2 \bar{x}^2 \\ &=n\sum_{i=1}^n x_i^2- n^2 \bar{x}^2 \\ \end{align}

I will leave the last step for you to simplify.


Alternatively, here is an approach in terms of matrices, we want to minimize

$$L(\|A\tilde{z}-y\|^2$$ where $A$ include a column of $1$ to handle the intercept term and $\tilde{z}$ include the intercept term as well.

Differentiate it, we obtain $$\nabla_z(L)=A^T(A\tilde{x}-y)$$

and $$\nabla_z^2(L)=A^TA \succeq 0$$