# Group lasso penalization

In group lasso we try to minimize the following $$\frac{1}{2}\|Y - \sum_{i=1}^JX_j\beta_j\|^2 + \lambda\sum_{i=1}^J\|\beta_j\|$$ Often they change this to $$\frac{1}{2}\|Y - \sum_{i=1}^JX_j\beta_j\|^2 + \lambda\sum_{i=1}^J\sqrt{p_j}\|\beta_j\|,$$ with $p_j$ the number of variables in group $j$. The reason they give is that 'since all groups are equally penalized in the firsts formula, larger groups would be more likely to be selected'. Can someone explain this reasoning?

The logic is that you could have 2 categorical variables, the first has 101 levels and the second only has 2. When you represent these in the linear model you use a series of indicator variables for each of the levels except a baseline level. This means the first variable has 100 binary variables and the second has only 1. In this extreme example you would be penalizing the two features in a highly unequal fashion-you want to have the two features weighted equally, multiplying by $p_i$ in this case (100,1) puts both features on an equal footing when they are penalized by $\lambda$. Otherwise $\lambda$ knows nothing about the 101 binary variables in the model and penalizes them all the same.