# Should I use dependent t-test or independent t-test

I have a software that produces a sentiment score between 0 and 1 given an input sentence. I want to test if this system produces biased scores based on the gender of the subject used in the sentence.

I have 50 sentence pairs and for each sentence pair I get a pair of scores from my software. Therefore I have 50 pairs of scores.

I want to give an example sentence pair and score pairs:

Input Sentence: John feels angry    -> score: 0.15
Input Sentence: Marry fells angry   -> score: 0.19


Another example:

Input Sentence: Adam likes music    -> score: 0.10
Input Sentence: Isabel likes music   -> score: 0.34


At the and I create two arrays keeping the scores I obtained with those sentences:

male_scores = [ 0.15,0.10, ....]
female_scores = [ 0.19,0.34, ...]


As I said earlier,now I want to see if these score differences tells me something about the fact that my software have some gender bias or not.

In order to do that, should I use independent t-test or dependent t-test?

• So each sentence appears twice, with the only modification being the gender of the person mentioned in the sentence, and you want to test if there's a difference in mean between the scores with regards to gender? If the choice is between a paired t-test and a unpaired t-test, then you should use the paired t-test. – Phil Nov 7 '18 at 10:33
• @Phil Thank you for your comment, this is exactly what I want to do. Do you have any other suggestion ? – zwlayer Nov 7 '18 at 12:05
• If the t-test is sufficient for your purposes, then do that analysis (but do check the corresponding assumptions behind the paired t-test so that they are fulfilled for your data). If you want to include covariates in your analysis, then you could add them in a regression framework. – Phil Nov 7 '18 at 12:10
• I just realised that you wrote that the outcome is bounded between 0 and 1. That might affect the plausibility of the analysis, so check the distribution of the paired differences to see if they look roughly normal. – Phil Nov 7 '18 at 12:12
• @Phil, if they are (almost) normal I use paired-t test right? – zwlayer Nov 7 '18 at 12:17