I have the equation $$ R_{\text{emp}}(w) = \frac{1}{2N} \lVert \Phi w -y \rVert ^2 = \frac{1}{2N} \sum_{i=1}^N \left( w^T \phi(x_i)- y_i \right)^2 $$ and am supposed to derive the least squares solution $\hat{w}=(\Phi^T\Phi)^{-2}\Phi^Ty$ (assume $\Phi^T\Phi$ is invertible).

I know that I have to take the gradient of the empirical risk and equate it to zero. The trouble I am having is in differentiating the formula with the summation in it. Appreciate any help in this matter.

  • 2
    $\begingroup$ Try re-writing $\|\Phi w-y\|^{2}$ as $(\Phi w-y)^T(\Phi w-y)$ and take derivatives with respect to $w$. $\endgroup$ Sep 13, 2021 at 14:27
  • $\begingroup$ Many more threads available via search: stats.stackexchange.com/… $\endgroup$
    – Sycorax
    Sep 13, 2021 at 14:36