# Sum of Squares Constraint Regularisation

Suppose I have a function $g_0\in L_2(\mathbb{R})$ such that we observe $(X_i,Y_i)$, $i=1,...,n$ such that

\begin{align*} Y_i & = g_0(X_i) + V_i \end{align*}

We wish to estimate $g_0$, and since $g_0\in L_2(\mathbb{R})$, we can approximate $g_0$ by using an orthonormal series $\{h_k\}_{k\in\mathbb{N}}$. So for each sample size $n$, we approximate $g_0$ by considering the truncated series $g_{p_n}(x) = \sum_{k=1}^{p_n}\alpha_k h_k(x)$, such that $\alpha_k = \langle g,h_k\rangle$, and my estimator $\hat g = \sum_{k=1}^{p_n}\hat\alpha_kh_k$ will be defined as

\begin{align*} \hat{\boldsymbol \alpha} = \arg\min_{\boldsymbol{\alpha}\in\mathbb{R^{p_n}}} \sum_{i=1}^{n}\left[Y_i - \sum_{k=1}^{p_n}\alpha_kh_k\right]^2\\ \end{align*}

The point is I wish to constrain the fitted coefficients $\hat{\boldsymbol\alpha}$ such that for each entry of $\hat{\boldsymbol \alpha}$, $\alpha_k$, I would like $|\alpha_k| \leq C$ for some $C$ strictly larger than $\|g_0\|_{\infty}$.

I understand that if we were to Heuristically argue this change, all we would do is threshold everything to be less than or equal to in magnitude to $C$, but how does this constraint change the standard "least squares" solution of $\hat{\boldsymbol \alpha}$ in closed form?

\begin{align*} L(\alpha_k; \boldsymbol \alpha_{-k})= \sum_{i=1}^{n} \left[\underbrace{\bigg(Y_i - \alpha_kh_k(X_i)\bigg)}_{=:Y_{-k,i}} - \sum_{\substack{j=1\\j\neq k}}^{p_n}\alpha_jh_j(X_i)\right]^2\\ \end{align*}
Here $\boldsymbol \alpha_{-k}$ denotes the $p_n-1$-dimensional vector $\boldsymbol \alpha$ without the $k$-th component $\alpha_k$, and $Y_{-k,i}$ is the new adjusted target. Now, in principle, you can do a one-dimensional minimization where you don't allow $\alpha_k$ to be outside your chosen boundary -- in practice: span a grid of points and search on it.