ridge regression - why is the solution the intersection of the contours associated with the error and penalty term $\textrm{min} \ (\ \sum \limits_{i=1}^m (y_i - \boldsymbol x_i^T \boldsymbol \theta)^2 + \lambda \sum \limits_{j=1}^n \theta_j^2 \ )$
By searching around the internet I could figure out that the solution to the above Lagrange formulation (Ridge regression) lies at the intersection between the contours of the two summands.
I know that the above represents the Lagrangian formulation of the underlying minimization problem.
Clearly, by considering the constrained formulation it makes sense for me why the solution is found at the intersection of the corresponding contours.
That is also explained here:
L1 L2 regularization
But, by just having the Lagrange formulation as above, how can it be seen that the minimum is found at the intersection?
In general, the minimum of the sum of two functions is not always found at the intersection of the corresponding contours.
 A: Typically, those figures do not show the complete picture. It looks like the following:

There are an infinite number of contours belonging to both the MSE (elliptic ones) and the L2 regularization (circular ones). So, they don't intersect at a single point. Because for a given point on the ellipse, you can find a circle passing at that point.
They should be tangents. However, still, they don't intersect at a single point. Because, for a given MSE contour, you can gradually increase the size of the L2 circles and find their touch point.
The solution exists, for a given lambda, when the derivative of the Lagrangian is equal to zero. That is, when
$$L=f(\beta)+\lambda g(\beta)\rightarrow \nabla_\beta L = f'(\beta)+\lambda g'(\beta)=0\rightarrow f'(\beta)=-\lambda g'(\beta)$$
which means, the gradients at the solution should be multiples of each other. They point to opposite directions (due to $-\lambda$). This happens at tangent points. Because these gradients are normal vectors of these level curves.  But, which tangent to choose depends on the value of $\lambda$. The above plot shows different solutions, i.e. tangent points, depending on the value of $\lambda$.
