In the Lasso L1 regularization, from where comes the value of the variable $k$ in the second part of the function? Why isn't it $n$, too?

$$L(\beta) = \sum_{i=1}^n (y_i - \phi(x_i)^T \cdot \beta)^2 + \lambda \sum_{j = 1}^k l(\beta_j)$$

$\beta$ is the parameter vector, $y$ the output vector and $x$ the input vector.


$k$ is the length of $\beta$, the number of coefficients. The penalty is a function of the coefficients, not of the data.

Note, however, as a minor point, that you don't have to apply the penalty to all the coefficients in the model, although in your formulation you are.

  • $\begingroup$ Right, I thought the dimension for instance $\beta$ and $y$ have to be the same, that's why I had the confusion. $\endgroup$
    – Mahoni
    May 7 '12 at 19:13
  • 2
    $\begingroup$ (+1) In particular, if you happen to not be using centered predictors and you do have an intercept in the model, you probably don't want to include it in the penalization. $\endgroup$
    – cardinal
    May 7 '12 at 20:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.