# Paired-sample t test for usability scores

I have conducted an empirical study as a part of my master thesis. Each test subject filled out a questionnaire where some questions were regarding usability of different features. What is the best/correct way to calculate if a feature got a significant better usability score than another feature? (I used a 10-point Likert scale).

I had 14 test subjects, and I want to compare the usability score for a feature called "bubble" and a feature called "hull".

This is the scores:

Bubble: 6,  3,  7,      9, 7, 8, 7,  9, 8, 8, 6, 10, 3, 7
Hull:   9, 10, 10, 8.6154, 9, 6, 8, 10, 8, 8, 6, 10, 9, 9


(One subject did not rate "hull", I have therefore inserted the mean (8.6154) in its place.)

I have run a paired-sample t test which indicates that there is a significant difference in the usability scores for bubble (M=7.0, SD=2.04) and hull (M=8.62, SD=1.33); t(13)=2.43, p=0.0305.

Is this the correct way to do it? My statistical knowledge is quite limited, but I think I should say if there was a significant difference or not.

• I don't see anything wrong about your work. I am not very comfortable about assuming the missing value as the mean, because you are practically changing 5% of your data (It might be alright, I am just pointing out it might be risky). Another note that is worth mentioning is that your test is not "absolutely" significant, you have to mention it was significant at 5% only, because the p value is less than 0.05. – Error404 Aug 9 '13 at 10:03
• Thank you! Regarding "absolutely" significant. Is it enough to change it to say p<0.05, and by that showing I know it's not absolutely significant? – swenedo Aug 9 '13 at 11:19
• @swenedo I agree with the answer, but have a concern with the use of the Likert name for what is really a rating scale. Since this is going to be your academic work and live for a long time, I would recommend the use of rating scale. All Likert scales are rating scales, but not all rating scales are Likert scales and authors seem to want to gain credibility by using the timer Likert. Likert is a specific type of rating scale. Picky I know, but this is academia. – doug.numbers Aug 9 '13 at 11:38
• Okay, I will avoid calling them "Likert" but just "rating scale". Thanks! – swenedo Aug 9 '13 at 12:27
• I don't think it's right to simply replace the missing value by the mean of the other values -- your statistic no longer has the desired distribution. – Glen_b Aug 9 '13 at 23:40