I have a classification problem with the following example independent features:

recommendations comment_count comment.
0.663 . 0.382 'yes', 'trump'

The dependent variable is whether the comment is likely to receive a reply or not:


I want to apply regularisation to a the logistic regression model but I can't decide between L1 and L2.

I want to do this for three different datasets, one for online comments on sports articles, one for magazine and one for politics(national).

I then want to interpret the top e.g. 10 largest coefficients from these models. The following diagrams show this.

The first diagram is with the L1 penalty(has a test f1-score of 0.85): enter image description here The second diagram is with a L2 penalty(has a test f1-score of 0.60): enter image description here

I am struggling to decide between the two models, and which would create a more interesting discussion. I understand the L2 diagram more, such that comments in the magazine with a number of recommendations is likely to receive a reply. So I'm favoring L2, but the diagram of L1 offers more interesting text words that appeared in the comments.

I aim to identify features that vary across the different news groupings, sports, politics, and magazine. To point out similarities or differences that could be of importance.

  • 1
    $\begingroup$ Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. $\endgroup$
    – Community Bot
    Commented Mar 29, 2022 at 11:58
  • $\begingroup$ What is your purpose? If you only want to classify how does test set performance compare? Why interpretability is important? $\endgroup$
    – Tim
    Commented Mar 29, 2022 at 12:30
  • $\begingroup$ @Tim interpretability is important cause I want to point how the coefficients range across the news topics $\endgroup$
    – Holly
    Commented Mar 29, 2022 at 12:40
  • $\begingroup$ @Tim the f1-score I added was for the test data. I updated the question. As L1 handles outliers, I think that removes the important features, like number of recommendations, therefore I am thinking l2 is better $\endgroup$
    – Holly
    Commented Mar 29, 2022 at 12:41
  • $\begingroup$ How do you know that this feature is truly important? If you care about interpretability, why use regularization at all? $\endgroup$
    – Tim
    Commented Mar 29, 2022 at 12:52


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.