# Spherical restriction of OLS

What is the linear algebraic and geometric interpretation of the sentence found on this post:

When $y = X\beta + e$, the least squares problem which imposes a spherical restriction $\delta$ on the value of $\beta$, [which] can be written as

$$\begin{array} &\operatorname{min}\ \| y - X\beta \|^2_2 \\ \operatorname{s.t.}\ \ \|\beta\|^2_2 \le \delta^2 \end{array}$$

It sounds as though it has to be connected with the concept of spherical errors, but there is likely more to it. Further I was intrigued by this illustration comparing OLS to ridge regression:

Source: A First Course in Linear Model Theory / Edition 1 by Nalini Ravishanker, Dipak K. Dey, Dipak K. Dey

with the text:

We can characterize ridge regression as a restricted least squares problem. Consider the least squares in the centered and scaled multiple regression model $\bf y^*=X^*\beta^*+\varepsilon$ subject to the spherical restriction $$\beta^{*'}\beta^*\leq d^2$$ for a given value $d^2.$

Continuing along the hints given by W. Huber in the comments, I wonder if this is related to the ellipsoid representation, and the geometric interpretation of ridge regression as a contained OLS optimized at the locus of osculation determined by $\mathbf \beta^\top K \beta$:

Source: Elliptical Insights: Understanding Statistical Methods through Elliptical Geometry Statistical Science 28(1) · February 2013, Michael Friendly and Georges Monette

• For $\beta\in\mathbb{R}^k$, the equation $||\beta||^2\le\delta^2$ is satisfied by the points in the ball (aka "sphere") of radius $\delta$ centered at the origin of $\mathbb{R}^k$. I believe that's all that the phrase "spherical restriction" was intended to mean. – whuber Jan 18 '17 at 16:51
• @whuber Even centered, though, each $\beta_i$ coefficient is different. Do you draw this hypersphere with a radius equal to the smallest coefficient? And is it just a statement implying that the coefficients are all less than infinity? I am still not seeing it... – Antoni Parellada Jan 18 '17 at 16:56
• You seem to be confusing a ball with a line. The constraints require only that $\beta$ lie within the ball, not that it lie on the line $\beta_1=\beta_2=\cdots=\beta_k$! Note that both $\delta$ and $d$ are "given values": that is, they are not free to vary, but are specified as part of the problem. – whuber Jan 18 '17 at 16:58
• @whuber I see the picture now - not on the surface of the sphere, but within. Where are the values of $\delta$ specified? – Antoni Parellada Jan 18 '17 at 16:59
• After Ridge Regression was invented, it was re-interpreted as the solution to OLS with a particular kind of Bayes prior on $\beta$. Within that interpretation, $\delta$ does have some meaning (related to how small you think the coefficients ought to be). – whuber Jan 18 '17 at 17:07

As explained in the comments, the problem is ridge regression, where the squared error is minimized subject to a bound on the $\ell_2$ norm of $\beta$.
As far as I can discern, constraining the $\ell_2$ norm of $\beta$ is not connected to linear model assumptions that the error is spherical. After all, using Bayesian language, the likelihood $\pi(y|\beta)$ being spherical does not suggest that the prior $\pi(\beta)$ should be too.