We had students take a test with 20 questions and had them indicate their confidence levels for each question. In one test format, the confidence prompt was adjacent to their test question while in the alternate format, their responses to the test question and confidence had to be indicated on a separate form. The confidence responses are Likert scale data (not at all confident....highly confident, coded 1-5) while the test responses are dichotomous (correct/incorrect, coded 1/0). The tests only differ in the way the confidence survey was answered; test content and items are exactly the same.
We are unsure of whether the change in format might have had an impact on their confidence or test responses, but would like a simple method to check for this. There are definitely other factors that go into affecting their responses, but any recommendations on how we might be able to make any judgements about the effects of this format change?