Before I can ask my questions, I have to give a little background information. I'm involved in a study using a measure of nonverbal fluency. Subjects are shown a page full of squares, like the one shown below. Each square contains 5 dots. Subjects are given 60 seconds to generate as many different designs as fast as they can by connecting dots in each square. The criteria for correct designs are that the designs must contain 4 lines and that all the lines in the design must touch. There are 2 trials in the test: one with 5 black dots and one with 5 black dots and 5 white dots. In the second trial, the extra dots are included as form of interference.
The way this measure is usually quantified is by counting the number of correct designs produced. However, I'm interested in looking at this task in a different way. I want to know if members of a clinical group (say Alzheimer's disease) will produce a quality of designs that is differerent than normal controls.
One way I thought about doing this was by taking the responses produced by a large number of controls and coming up with a way of classifying them, say 4 or 5 types of designs. Then I'd count the number of times each of these design types occurred in the control sample and apply a weight to them so that design types with lesser frequency would have a greater weight (think inverse document frequency). I'd then look at a smaller clinical sample and a small control sample matched on demographic variables. I'd count the number of each design type produced by each subject in both groups, and multiply the number of each design type by the weight assigned to it. The average frequency would be calculated for each subject for each trial type (the 5 dot trial and the 10 dot interference trial). Group performance would then be compared in a repeated measures ANOVA.
Here are the two questions I'm posing. The control sample that I have to determine the frequency of the design types is small: less than 70 subjects. Is this too small for determining the frequency and weights of the design types? Secondly, when comparing the clinical sample to a control sample, would it be possible to draw the small control sample from this larger control sample used to derive the weights, or is there some reason that this should not be done?
Thanks in advance for your attention and response. Any input would be greatly appreciated. I realize this question may be off topic for this site. If you feel that's the case, would you mind suggesting a more appropriate forum.
