0
$\begingroup$

i have a CNN for sentiment analysis whose precision and recall for validation data over 10 training epochs is (average:macro):

enter image description here enter image description here enter image description here enter image description here

The dataset contains more positive samples than negatives.I have problem interpreting the results.It seems like it has higher recall and lower precision. One way is to say that since there are so many positive samples, there are more examples that can become false negatives leading to smaller recall and high precision. Or if i try to interpret my own results it can be that since number of positive samples is bigger,the classifier is biased towards positive class in beginning leading to more false positives hence lower Precision and higher recall. How are these graphs interpreted actually? Thanks

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Firstly, note that $Precision = \frac{TP}{TP+FP}$, whereas $Recall = \frac{TP}{TP+FN}$, where $TP$ is the number of true positives, $FP$ is the number of false positives, and $FN$ is the number of false positives.

Since you have mentioned that your data is skewed towards the positive case, (depending on how you have trained your model and chosen your probability cutoff) your predictions will hence likely be biased towards the positive case also.

This means your model will be more likely to predict an observation as being positive than negative, leading to more frequent false positives occurring. Since $Precision$ is inversely proportional to the number of false positives, $FP$, this increase in $FP$ leads to a larger decrease in $Precision$ relative to the decrease in $Recall$.

This happens as the bias in the data towards the positive class means on average, the negative class is less likely to be predicted, and hence less 'negatives' are predicted compared to what we would expect with a more balanced dataset. Less negative predictions explain the relative decrease in less false negative predictions, and why $Recall$ is higher, by a similar logic to the above.

Side note: For your graphs, are they zero-indexed (0-9) when they should be 1-10? Not a big issue but it may make it harder to read results off the graph if you constantly have to shift the x-index forward by one, and so forth.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks. Thats what i had thought as well so I calculated scores for each class separately which made me confused a bit. e.g. the precision graph shows high values for positive class which suggests lowest FP (incase of positive class meaning a negative sample labelled as positive) . Similarly precision is lower for negative ,higher FP (incase of negative class,meaning a positive sample labelled as negative one) .This made it look like the classifier is predicting negative class more which is opposite to what i understood from average graph..Any help is appreciated. $\endgroup$
    – Ayza
    Jul 29, 2018 at 9:42
  • $\begingroup$ Yes, the $Precision$ for the negative case is lower as more negative cases are being classified as (false) positives, leading to a lower proportion of true negatives in the data due to class imbalance, skewing the ratio of true negatives to total negatives towards 0, lower than the $Precision$ for the positive case. $\endgroup$
    – Data Science Officer
    Jul 29, 2018 at 10:48
  • $\begingroup$ Does the meaning of true positives or false positive changes w.r.t class? e.g. false positive w.r.t positive class is something actually negative labeled as positive..right? would it be same w.r.t negative as well or the meaning changes that something actually positive labeled negative? $\endgroup$
    – Ayza
    Jul 29, 2018 at 11:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.