L1 L2 regularization The tutorial says the intersection point for L1 and L2 regularization gives the minimum loss - But why the intersection gives the minimum loss? I cannot interpret the graph clearly.
 A: One way to view both ridge and the lasso is as the solution to the constrained equation 
$$
\widehat \beta = \operatorname{arg min}_{\beta} \|Y - X\beta\|^2_2
\quad \text{subject to $\|\beta\| \le \eta$}
$$
where $\|\cdot\|$ is, in either case, the $\ell_1$ or $\ell_2$ norm. The usual penalization characterization of this 
$$
\widehat \beta = \arg \min_\beta \|Y - X\beta\|^2_2 + \lambda \|\beta\|_p^p
$$
($p = 1$ for the lasso and $p = 2$ for ridge) is just the formulation of the optimization in terms of a Lagrangian, where $\lambda = \lambda(\eta)$ is a function of $\eta$. Anyway, once you have the constraint formulation, it should be clear why the solution is the intersection: we are finding the contour with the smallest value which satisfies the constraint. The points inside the $\ell_1$ or $\ell_2$ ball are the points which satisfy the constraint, and we want to pick the solution $\widehat \beta$ which is on the contour of smallest value. When the least squares solution does not lie in the $\ell_p$ ball itself, this will occur on the boundary, where one of the contours intersects the boundary of the ball. 
A: Minimizing the loss function $f(\theta)$ with regularization function $g(\theta)$ can be viewed as minimizing a  Lagrange Function.
$\mathcal{L}(\theta,\lambda) = f(\theta) - \lambda \cdot g(\theta)$
you can then minimize by taking the gradient 
$\nabla_{\theta,\lambda} \mathcal{L}(\theta, \lambda)=0$.
In a Lagrange Function, the optimum occurs when the gradient of the loss function is perpendicular to the regularization function.
$\nabla_{\theta} f = \lambda \nabla_{\theta} g,$
Here is a much better explanation by Dikran Marsupial :What is the connection between regularization and the method of Lagrange multipliers ?
Just a note it this is a very general explanation. 
