Calculate Average Score on Repeated Measures with Uneven Sample Sizes I am currently evaluating the predictive validity of a set of personnel selection procedures (predictor) on subsequent vocational training outcomes (criterion).
As far as the criterion variables are concerned, subjects are repeatedly graded 1 to 4 on a total of 9 distinct dimensions during 7 successive training events. I have tried running correlations between my predictor variables and each individual set of grades for each individual training event (i.e. all scores obtained for each dimension on the first training event, then all scores for each dimension on the second training event, etc.). This doesn't do much as the variance in the scores for each dimension within one individual training event is absolutely minimal (the majority of scores are 3's, with the occasional 4 and no 1's or 2's whatsoever).
Therefore, I have decided to consider the traits that are being measured (i.e. the dimensions that are being graded) to be relatively stable in time and would like to use a "grand-average" score across all 7 training events as my criterion variable (to generate some much needed variance). However, that's where I am having some trouble.
In fact, I have a total of 110 subjects in the sample, of which 80 have completed all 7 training events, and 30 have only completed 5. I have observed some very interesting and conceptually relevant correlations after just calculating a grand average score per dimension for each subject in the sample. But I think I might have generated some artificial variance, since I averaged a total of 7 scores for 80 people, and only a total of 5 scores for the remaining 30 people.
With this in mind, my question is the following: is there any way I can format my data so that the averages for both the people who have completed 7 training runs and those who have only completed 5 become "equivalent" or "weighted" in a sense ? I do fully understand that they will never be equivalent on the conceptual plane (since 30 people have 2 training runs missing, and therefore we do not know the "real" way it would have played out). My goal is to have one single variable for each dimension I can use as a criterion, averaged for every participant across all training runs they completed (even if some of them have only completed 5 instead of 7).
Many thanks in advance and please feel free to let me know should you require more information!
 A: Since you have a number of measurements of the same individuals, you should be fine averaging their measurements along each of the 9 dimensions. One possible issue is that you may have more variance in the 5-trial averages versus the 7-trial averages. That could cause non-constant variance (though it sounds like your data exhibit very little variance).
A way to handle non-constant variance is to do a weighted regression that stabilizes the variances. You would weight the 5-trial data by some weight that is 5/7 the weight used on the 7-trial data. (So you could use a weight of 5 for 5-trial data and 7 for 7-trial data.)
Finally, you should consider why some of the individuals only completed 5 trials. There might be a lower mean score or some other difference between individuals who completed all trials and those who only completed 5 trials.
A: Only fully specified full likelihood models will correctly take into account varying numbers of measurements, and this and all methods being discussed require a missing at random assumption.  Your Y is ordinal so I suggest a longitudinal ordinal model, either using a Markov process or subject random effects.  These are detailed here.  It is usually best to analyze the rawest form of the data rather than taking averages first.  There is some power gain from using all the individual measurements.
