How to manually do multiple imputation to fill in missing data points

I am trying to calculate a pearson's correlation between scores on a test to determine test-retest reliability. However, there are only 4 items on the test, with a maximum score of 4 and minimum score of 0. Some students have taken the test several times and some students have taken the test only 2 times. I would like to find a way to impute values to make complete data sets so that if Student A has only taken it two times, but Student B has taken it 8 times, I can fill in 6 missing data points for Student A to make 8 complete data sets. I am doing this as part of a PHP script so I can't use R or SPSS to calculate the missing data. I need to know how I can manually impute the missing values.

I am thinking that I randomly choose a number between 0 and 4 for each of the missing data points, and I do that 3 - 4 times so instead of 8 complete sets, I get 32 complete sets (is that right?) and then do the correlation analysis on those data sets. Does that sound like it will work without giving me biased or overstating the correlation?

UPDATE: The scores are not connected to a specific time, i.e. all the students aren't taking the test on the same day. I can get a list of scores for each student, and then put those scores into columns, but there is no time-connection among scores in a single column.

There actually several tests that are looking at different things. Each test has at least 2 items, but most have 4 items. For each of the tests, I exclude students that only have 1 score, giving me a minimum of 2 complete data sets, but up to 1, 3, 7, and even 10 additional columns, of course with some values missing (because the students that only took that test 2 times will not have any additional scores). I am not sure how this affects the type of correlation calculation I should do.

UPDATE: I just calculated how many times each student took each test.

Test 1
2 attempts - 117
3 attempts - 26
4 attempts - 8
5 attempts - 3
6 attempts - 3
7 attempts - 1
8 attempts - 1
10 attempts - 2
12 attempts - 1

Test 2
2 attempts - 42
3 attempts - 7
4 attempts - 2
5 attempts - 3
6 attempts - 1
9 attempts - 1

Test 3
2 attempts - 54
3 attempts - 7
4 attempts - 1
6 attempts - 1

Test 4
2 attempts - 20
3 attempts - 1

Test 5
2 attempts - 58
3 attempts - 12
4 attempts - 3
5 attempts - 1

Test 6
2 attempts - 23
3 attempts - 1

Test 7
2 attempts - 36
3 attempts - 7
4 attempts - 1
6 attempts - 2
7 attempts - 1


The reason to impute missing values is to keep my sample sizes larger. Given the current sizes, am I okay doing correlations on just the 2 complete data sets?

• Why did some students take the test more often than others? Because if this is due to some students failing the test at first, then the missing values are related to the outcome and you cannot simply impute them. – Frans Rodenburg Feb 3 '18 at 6:51
• I don't have information about why students take the test the number of times they do. The test is taken online and I have no contact with the students. Some students might be using the test for practice (as a way to study the concepts) or their teacher might be using the test in their class combined with training. – wuijin Feb 3 '18 at 7:20
• I don't see that you need multiple imputation to answer your underlying question about test-retest reliability; it might even be inappropriate as @FransRodenburg suggests. A random-effects model would be able to take into account differences among individuals while allowing for different numbers of attempts among the individuals. You don't need to have the same number of attempts by each individual. – EdM Feb 3 '18 at 14:26
• Could you walk me through how to do the correlation using a random effects model? I need to write a PHP script that will do the calculations so I can't just put the data into SPSS and click a button. The only tutorials I can find online show how to do it with R or other stats programs. – wuijin Feb 6 '18 at 15:59