Recall that the Lasso minimization problem can be expressed as:
$$ \hat \theta_{lasso} = argmin_{\theta \in \mathbb{R}^n} \sum_{i=1}^m (y_i - \mathbf{x_i}^T \theta)^2 + \lambda \sum_{j=1}^n | \theta_j| \ $$
Which can be viewed as the minimization of two terms: $OLS + L_1$.
- The first OLS term can be written as $(y - X \theta)^T(y - X \theta) $ which gives rise to an elipse contour plot centered around the Maximum Likelihood Estimator.
- The second $L_1$ term is the equation of a diamond centered around 0 (or a romboid in higher dimensions)
- The solution to the constrained optimization lies at the intersection between the contours of the two functions, and this intersection varies as a function of $\lambda$. For $\lambda = 0$ the solution is the MLE (as usual) and for $\lambda = \infty$ the solution is at $[0,0]$.
- Since at the vertices of the diamond, one or many of the variables have value 0, there is a non zero probability that one or many of the coefficients will have a value exactly equal to 0.
This last bullet is important in answering your question:
But I don't understand the case when the lasso Regression just shrink
the paramters and don't set them to Zero
The lasso regression does't have to set coefficients to zero, in many cases it doesn't. What happens is that as you increase the $\lambda$ parameter, the probability that the solution takes place at a vertex of the diamond increases, and so the probability that one or many coefficients is exactly zero also increases.
Could you give me some intuition when the RSS-Line tangents with the
side of the diamond instead of the corner?
Here is a graph I have produced based on simulated data. It shows the optimal solution for ridge and lasso regression as a function of the $\lambda$ parameter (lasso is on the right hand side).

You can see that there are many solutions that are not on the vertex of the diamond!
The impact of strongly correlated features
This simple example shows what happens when the two features are highly correlated, in fact here $x_1 = x$ and $x_2 = x^2$ so they are so strongly correlated that the shape of the OLS cost function looks like an upside-down ridge, or a valley - hence the intuition behind the name ridge regression.
Sources
This post is strongly based on my previous post - For anyone interested, you can find most of the code and associated mathematical derivations on my blog and at this page