# How does $\vec{\beta}=(H^TH)^{-1}H^T\vec{y}$ equivalent to least squares criteria for evaluating splines? [duplicate]

I'm learning about splines and the equation for a spline trying to predict the true function given data points is expressed as

$$f(x)=\sum^k_{m=1}\beta_mh_m(x)$$

Where $$\beta_m$$ is some linear coefficient, $$h_m(x)$$ is a basis function and $$k$$ is the number of regions/areas separated by knots.

I read that the least squares criterion is used to estimate the coefficients($$\beta$$), which makes perfect sense for evaluation. Then it goes on to say you can solve for all $$\beta_m$$ using matrices.

$$H = \begin{bmatrix}h_1(x_0) & ... & h_k(x_0)\\ ... & ... & ... \\ h_1(x_n) & ... & h_k(x_n)\end{bmatrix}$$

$$h_i, i\in[1,k] :$$ basis functions
$$x_l, l\in[0,n] :$$ Independent data points being approximated

$$\vec{\beta}=(H^TH)^{-1}H^T\vec{y}$$ $$\vec{y}$$ : Vector of response variables to each $$x_l, l\in[0,n]$$ so $$\vec{y}=[y_0,...,y_n]$$
$$\vec{\beta} = [\beta_1 , ..., \beta_k]$$

How does $$\vec{\beta}=(H^TH)^{-1}H^T\vec{y}$$ result in minimizing least square criteria?

• Did you mean $(H^TH)^{-1}$? That is very different from $(H^TH^{-1})$. Also, you switch between $(H^TH^{-1})H^T\vec{y}$ and $H(H^TH{-1})H^T\vec{y}$. The first one is correct, and can be derived using calculus on linear transformations. – Robby the Belgian Aug 28 '20 at 6:43
• @RobbytheBelgian the post has been corrected. Thanks for pointing out the errors – user8714896 Aug 28 '20 at 6:43
• If you're happy to admit that for linear regression that $\hat{\beta} = (H^T H )^{-1} H^T y$, and you can also see that splines are just a linear model (albeit, ones with funny basis functions) then hopefully you can see why the same formula minimises MSE in the spline case – jcken Aug 28 '20 at 8:06
• @jcken I'm not familiar with solving LR with this matrix form. I'm more used to $(1/n)\sum(y-\hat{f(}x_i))^2$. I don't quite understand how the matrix equation equates to MSE. – user8714896 Aug 28 '20 at 8:51
• @user8714896 wikipedia has a good explanation of the solution to minimising the MSE via matrices. Look under the estimation section en.m.wikipedia.org/wiki/Linear_regression – jcken Aug 28 '20 at 19:12