0
$\begingroup$

I have a classification problem with the following example independent features:

recommendations comment_count comment.
0.663 . 0.382 'yes', 'trump'

The dependent variable is whether the comment is likely to receive a reply or not:

get_reply
0.

I want to apply regularisation to a the logistic regression model but I can't decide between L1 and L2.

I want to do this for three different datasets, one for online comments on sports articles, one for magazine and one for politics(national).

I then want to interpret the top e.g. 10 largest coefficients from these models. The following diagrams show this.

The first diagram is with the L1 penalty(has a test f1-score of 0.85): enter image description here The second diagram is with a L2 penalty(has a test f1-score of 0.60): enter image description here

I am struggling to decide between the two models, and which would create a more interesting discussion. I understand the L2 diagram more, such that comments in the magazine with a number of recommendations is likely to receive a reply. So I'm favoring L2, but the diagram of L1 offers more interesting text words that appeared in the comments.

I aim to identify features that vary across the different news groupings, sports, politics, and magazine. To point out similarities or differences that could be of importance.

Improve this question
$\endgroup$
10
  • 1
    $\begingroup$ Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. $\endgroup$
    – Community Bot
    Mar 29, 2022 at 11:58
  • $\begingroup$ What is your purpose? If you only want to classify how does test set performance compare? Why interpretability is important? $\endgroup$
    – Tim ♦
    Mar 29, 2022 at 12:30
  • $\begingroup$ @Tim interpretability is important cause I want to point how the coefficients range across the news topics $\endgroup$
    – Holly
    Mar 29, 2022 at 12:40
  • $\begingroup$ @Tim the f1-score I added was for the test data. I updated the question. As L1 handles outliers, I think that removes the important features, like number of recommendations, therefore I am thinking l2 is better $\endgroup$
    – Holly
    Mar 29, 2022 at 12:41
  • $\begingroup$ How do you know that this feature is truly important? If you care about interpretability, why use regularization at all? $\endgroup$
    – Tim ♦
    Mar 29, 2022 at 12:52

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.