#A geometrical interpretation
The estimator described in the question is the Lagrange multiplier equivalent of the following optimization problem:
$$\text{minimize $f(\beta)$ subject to $g(\beta) \leq t$ and $h(\beta) = 1$ } $$
$$\begin{align} f(\beta) &= \lVert y-X\beta \lVert^2 \\ g(\beta) &= \lVert \beta \lVert^2\\ h(\beta) &= \lVert X\beta \lVert^2 \end{align}$$
Comparison to the standard ridge regression view
In terms of a geometrical view this changes the old view (for standard ridge regression) of the point where a spheroid (errors) and sphere ($\|\beta\|^2=t$) touch. Into a new view where we look for the point where the spheroid (errors) touches a curve (norm of beta constrained by $\|X\beta\|^2=1$). The one sphere (blue in the below image) changes into a lower dimension figure due to the intersection with the $\|X\beta\|=1$ constraint.
For an example of this (old) view I am "stealing" the image from this answer.
New geometrical view
In the two dimensional case this is simple to view.
When we tune the parameter $t$ then we change the relative length of the blue/red spheres or the relative sizes of $f(\beta)$ and $g(\beta)$ (In the theory of Lagrangian multipliers there is probably a neat way to formally and exactly describe that this means that for each $t$ as function of $\lambda$, or reversed, is a monotonous function. But I imagine that you can see intuitively that the rms residuals only increases when we decrease $||\beta||$.)
The solution $\beta_\lambda$ for $\lambda=0$ is as you argued on a line between 0 and $\beta_{LS}$
The solution $\beta_\lambda$ for $\lambda \to \infty$ (or already earlier) is (indeed as you commented) in the loadings of the first principal component. This is the point where $norm(\beta)$ is the smallest for $norm(\beta X) = 1$.
In this 2-d view the edges of the intersection of the sphere $norm(\beta) <=t$ and spheroid $norm(\beta X) = 1$ are points. In multiple dimensions these will be curves
(I imagined first that these curves would be ellipses but they are more complicated. You could imagine the ellipsoid for $\lVert \beta \rVert^2 = 1$ being intersected by the sphere $\lVert X \beta \rvert^2 \leq t$ as some sort of ellipsoid frustum but with edged that are not a simple ellipse)
The limit $\lambda_{lim}$ where the solutions 'starts to change' can be found in evaluating the maximal direction for $\frac{\partial||y-X\beta||^2}{\partial ||\beta||^2}$, and it's absolute value will dictate at which $\lambda$ it the solutions starts to diverge, or at which $\lambda$ the change in the error-term $||y-X\beta||^2$ becomes bigger than the change in the penalty term $\lambda ||\beta||$.
The limit $\lambda \to \infty$ is reached already much earlier (although it may be a bit difficult to calculate exactly what this $\lambda_{lim}$ is or which direction the first change occurs).