I am trying to calculate a pearson's correlation between scores on a test to determine test-retest reliability. However, there are only 4 items on the test, with a maximum score of 4 and minimum score of 0. Some students have taken the test several times and some students have taken the test only 2 times. I would like to find a way to impute values to make complete data sets so that if Student A has only taken it two times, but Student B has taken it 8 times, I can fill in 6 missing data points for Student A to make 8 complete data sets. I am doing this as part of a PHP script so I can't use R or SPSS to calculate the missing data. I need to know how I can manually impute the missing values.
I am thinking that I randomly choose a number between 0 and 4 for each of the missing data points, and I do that 3 - 4 times so instead of 8 complete sets, I get 32 complete sets (is that right?) and then do the correlation analysis on those data sets. Does that sound like it will work without giving me biased or overstating the correlation?
UPDATE: The scores are not connected to a specific time, i.e. all the students aren't taking the test on the same day. I can get a list of scores for each student, and then put those scores into columns, but there is no time-connection among scores in a single column.
There actually several tests that are looking at different things. Each test has at least 2 items, but most have 4 items. For each of the tests, I exclude students that only have 1 score, giving me a minimum of 2 complete data sets, but up to 1, 3, 7, and even 10 additional columns, of course with some values missing (because the students that only took that test 2 times will not have any additional scores). I am not sure how this affects the type of correlation calculation I should do.
UPDATE: I just calculated how many times each student took each test.
Test 1 2 attempts - 117 3 attempts - 26 4 attempts - 8 5 attempts - 3 6 attempts - 3 7 attempts - 1 8 attempts - 1 10 attempts - 2 12 attempts - 1 Test 2 2 attempts - 42 3 attempts - 7 4 attempts - 2 5 attempts - 3 6 attempts - 1 9 attempts - 1 Test 3 2 attempts - 54 3 attempts - 7 4 attempts - 1 6 attempts - 1 Test 4 2 attempts - 20 3 attempts - 1 Test 5 2 attempts - 58 3 attempts - 12 4 attempts - 3 5 attempts - 1 Test 6 2 attempts - 23 3 attempts - 1 Test 7 2 attempts - 36 3 attempts - 7 4 attempts - 1 6 attempts - 2 7 attempts - 1
The reason to impute missing values is to keep my sample sizes larger. Given the current sizes, am I okay doing correlations on just the 2 complete data sets?