I have started learning about lasso from this review ("Least Squares Optimization with L1-Norm Regularization" by Mark Schmitd).
The optimization problem is:
$$\min_{\beta } \left\{ \frac{1}{N} \sum_{i=1}^N (y_i - x_i^T \beta)^2 \right\} \text{ subject to } \sum_{j=1}^p |\beta_j| \leq t$$
where $t$ controls the amount of regularization. There is another way of writing this problem using Lagrange multipliers:
$$\min_{ \beta \in \mathbb{R}^p } \left\{ \frac{1}{N} \left\| y - X \beta \right\|_2^2 + \lambda \| \beta \|_1 \right\}$$
Question1: How does $t$ control the amount of regularization? If $\lambda$ is large, then will more coefficients be forced to zero? If so, then how many coefficients would be zero, since LASSO does not assume a particular active set?
I used lasso in Matlab and, based on the documentation, one should choose $\lambda$ to minimize the mean squared error. Say $\lambda = 0.8$, how and where is this value used? Does large $\lambda$ imply more sparsity? What is the meaning of this term?
Question2: Does $\lambda$ lie in a range, say 0 to 1?