I am analyzing whether a new feature (presenting history path) served as a cognitive support in the process of problem-solving, which means that if this new feature can have an influence on people's cognitive load.
So I have asked a 9 point rating scale for three questions in terms of their mental effort, task difficulty and concentration. For the pretest, I had the above three questions, while in order to deeply understand the influence of this new feature, I have added another two questions in terms of the clearness and helpfulness of this feature with a categorical "yes" and "no" answer in the post test. Now I have collected all the data with the variables mental effort, task, concentration both from the 1st and the 2nd test, and two other variables, clarity and helpfulness in addition.
I have 47 subjects for the pre-test, and only the same 20 subjects for the post test (because the new feature was evolved during the 2nd test, and finally I had 20 subjects who saw the same new feature).
I want to compare those raw scores from the pretest to the raw scores from the posttest to see if there was a significant difference. And what can I do with the two additional variables in the post test? should I include all the subjects in the pre-test or just 20 subjects the same as the post test?
Which statistical method that I should use for my analysis?