What do the $L_1$ and $L_2$ mean when referring to regularization? $L_1$ regularization LASSO regression. $L_2$ regularization is is Tikhonov regularization or ridge regression. The combination of the two is elastic net regularization.
To what do the $L_1$ and $L_2$ refer? What is the origin of this notation?
 A: These regularization techniques are often encountered with variable selection. Assume you have a regression model 
$$
Y=\beta_0+\beta_1X_1+\ldots+\beta_NX_N+\epsilon.
$$
Your intention is select a few predictors and hence use a prediction model with those predictors. You will often go for the $X_i$ with the largest $\beta_i$. It turns out that if you first select and try to estimate $\beta_i$, your estimator $\hat \beta_i$ is positively biased (i.e., larger on average than $\beta_i$). Regularization is used to reduce this positive bias. Say you were to estimate $\mathbf{\beta}=(\beta_0,\beta_1,\ldots,\beta_N)^T$ by least squares, then you will do something like 
$$
\hat \beta=\text{argmin}_\mathbf{\beta}\left\{(Y-\mathbf{X}\mathbf{\beta})^2\right\}
$$
and return the largest $\hat \beta_i$s. Like I said these $\hat \beta_i$ s are positively biased. To counteract this bias you could regularize your least square action. Instead find
$$
\hat \beta^\text{Ridge}=\text{argmin}_\mathbf{\beta}\left\{(Y-\mathbf{X}\mathbf{\beta})^2+\lambda \sum_{i=1}^{N}\beta_i^2\right\}
$$
or 
$$
\hat \beta^\text{Lasso}=\text{argmin}_\mathbf{\beta}\left\{(Y-\mathbf{X}\mathbf{\beta})^2+ \lambda\sum_{i=1}^{N}|\beta_i|\right\}
$$
The ridge uses the $L_2$ regularization and the lasso uses the $L_1$ regularisation.  In general we could refer to the type of  regularizations  as $L_p$ where an $L_p$ regularazation takes the form
$$
\hat \beta^\text{general}=\text{argmin}_\mathbf{\beta}\left\{(Y-\mathbf{X}\mathbf{\beta})^2+\lambda\sum_{i=1}^{N}|\beta_i|^p.\right\}
$$
So you see that when $p=1$ we get the lasso and $p=2$ the ridge and so on. Mathematicians general refer to the function say $L_p(\mathbf x)=\left(\sum_{i=1}^{N}|x_i|^p\right)^{1/p}$ as a norm placed on the vector $\mathbf x=(x_1,\ldots,x_N)^T$. 
A: The "1" and "2" in $L_{1}$ and $L_{2}$ simply refers to the exponent you use in the regularization component of the function you minimize to find the function coefficients.
Thus, when using $L_{1}$ regularization, you're solving:
$$
\min_{\beta} [[Y-X\beta]^2 + \lambda\sum_{i=1}^{n}|\beta_{i}|^{1}]
$$
Whereas when using $L_{2}$ regularization, you're solving:
$$
\min_{\beta}[[Y-X\beta]^2 + \lambda\sqrt{\sum_{i=1}^{n}|\beta_{i}|^{2}}]
$$
(Note the exponent in the second term of the formula).
This is related to the concept of $L_{p}$-norm, defined as 
$\left ( \sum_{i=1}^{n}|x_{i}|^{p}\right )^{1/p}$ 
