#A geometrical interpretation
The estimator described in the question is the Lagrange multiplier equivalent of the following optimization problem:
$$\text{minimize $f(\beta)$ subject to $g(\beta) \leq t$ and $h(\beta) = 1$ } $$
$$\begin{align}
f(\beta) &= \lVert y-X\beta \lVert^2 \\
g(\beta) &= \lVert \beta \lVert^2\\
h(\beta) &= \lVert X\beta \lVert^2
\end{align}$$
----------
## Comparison to the standard ridge regression view
In terms of a geometrical view this changes the *old* view (for standard ridge regression) of the point where **a spheroid (errors) and sphere ($\|\beta\|^2=t$) touch**. Into a new view where we look for the point where **the spheroid (errors) touches a curve (norm of beta constrained by $\|X\beta\|^2=1$)**. The one sphere (blue in the below image) changes into a lower dimension figure due to the intersection with the $\|X\beta\|=1$ constraint.
For an example of this (old) view I am "stealing" the image from [this answer](https://stats.stackexchange.com/a/151960/164061).
![geometric view example 1][1]
------
## New geometrical view
In the two dimensional case this is simple to view.
[![example view][2]][2]
When we tune the parameter $t$ then we change the relative length of the blue/red spheres or the relative sizes of $f(\beta)$ and $g(\beta)$ *(In the theory of Lagrangian multipliers there is probably a neat way to formally and exactly describe that this means that for each $t$ as function of $\lambda$, or reversed, is a monotonous function. But I imagine that you can see intuitively that the sum of squared residuals only increases when we decrease $||\beta||$.)*
The solution $\beta_\lambda$ for $\lambda=0$ is as you argued on a line between 0 and $\beta_{LS}$
The solution $\beta_\lambda$ for $\lambda \to \infty$ (or already earlier) is (indeed as you commented) in the loadings of the first principal component. **This is the point where $norm(\beta)$ is the smallest for $norm(\beta X) = 1$.**
In this 2-d view the edges of the intersection of the sphere $norm(\beta) <=t$ and spheroid $norm(\beta X) = 1$ are points. In multiple dimensions these will be curves
*(I imagined first that these curves would be ellipses but they are more complicated. You could imagine the ellipsoid for $\lVert \beta \rVert^2 = 1$ being intersected by the sphere $\lVert X \beta \rvert^2 \leq t$ as some sort of ellipsoid frustum but with edges that are not a simple ellipses)*
-----
##Regarding the limit $\lambda \to \infty$
You will have the situation as in lasso and there will be some limiting $\lambda_{lim}$ above which all the solutions are the same (and they reside in the point $\beta^*_\infty$).
- In **normal ridge regression** you start in the point $\beta_i=0$ for all $i$, and any infinitesimal change of the $\beta_i$ does not change the penalty term (which is quadratic and has slope $0$ in the point $\beta_i=0$ for all $i$), so for all finite $\lambda$ the solution can not be $0$ (since an infinitesimal step can be made to reduce the sum of squared residuals without increasing the penalty).
- **For lasso** this is *not* the same since: the penalty is $\lvert \beta \rvert_1$ (so not quadratic with zero slope).
- **For your constrained ridge** this is *not* the same since: you do not start at $0$ where the slope is $0$, so you can not make an infinitesimal step, to reduce the root-mean-squared residual term, without increasing the penalty term.
The limit $\lambda_{lim}$ where the solutions 'starts to change' can be found in evaluating the maximal direction for $\frac{\partial||y-X\beta||^2}{\partial ||\beta||^2}$, and it's absolute value will dictate at which $\lambda$ it the solutions starts to diverge, or at which $\lambda$ the change in the error-term $||y-X\beta||^2$ becomes bigger than the change in the penalty term $\lambda ||\beta||$.
So the limiting solution $\lambda \to \infty$ is reached already much earlier (although it may be a bit difficult to calculate exactly what this $\lambda_{lim}$ is or which direction the first change occurs).
-----
##Further notes regarding the limit $\lambda \to \infty$
- You do not get the usual ridge regression limit for $\lambda$ to infinity. Instead this limit corresponds to the point where $\mu$ is equal to -1. Then the derivative of the Lagrange function in the normalized problem
$$2 (1+\mu) X^{T}X \beta + 2 X^T y + 2 \lambda \beta$$ may correspond to a solution for the derivative of the Lagrange function in the standard problem
$$2 X^{T}X \beta^\prime + 2 X^T y + 2 \frac{\lambda}{(1+\mu)} \beta^\prime \qquad \text{with $\beta^\prime = (1+\mu)\beta$}$$
The parameter $\lambda$ does not need to go to infinity in the normalized problem, in order to correspond to the limit at infinity in the standard ridge regression problem.
- You could parameterize the 2-d problem to show clearly that for values above some limiting $\lambda$ the solution does not change. Consider the parametrization:
$$\beta(\theta) = \begin{pmatrix} a \cos(\theta) \\
b \sin(\theta) \end{pmatrix}
$$
e.g. the regression is with:
$$ X = \begin{pmatrix}
1/a & 0 \\
0 & 1/b \\
\end{pmatrix}$$
Then the following is to be minimized
$$f(\theta) = \left(\cos(\theta)-\frac{\hat{\beta}_{LS}}{a}\right)^2 + \left(\sin(\theta)-\frac{\hat{\beta}_{LS}}{b}\right)^2 + \lambda \left( a^2 \cos(\theta)^2 + b^2 \sin(\theta)^2 \right) $$
and while it is a bit more work to simplify those trigonometric functions it is clear that for some value $\lambda$ the latter term dominates and depending on $a$ and $b$ the solution is 0 or 90 degrees for theta.
[1]: https://i.sstatic.net/QrueJ.gif
[2]: https://i.sstatic.net/hNgns.png