I have a software that produces a sentiment score between 0 and 1 given an input sentence. I want to test if this system produces biased scores based on the gender of the subject used in the sentence.
I have 50 sentence pairs and for each sentence pair I get a pair of scores from my software. Therefore I have 50 pairs of scores.
I want to give an example sentence pair and score pairs:
Input Sentence: John feels angry -> score: 0.15 Input Sentence: Marry fells angry -> score: 0.19
Input Sentence: Adam likes music -> score: 0.10 Input Sentence: Isabel likes music -> score: 0.34
At the and I create two arrays keeping the scores I obtained with those sentences:
male_scores = [ 0.15,0.10, ....] female_scores = [ 0.19,0.34, ...]
As I said earlier,now I want to see if these score differences tells me something about the fact that my software have some gender bias or not.
In order to do that, should I use independent t-test or dependent t-test?