I am a father of some very bright children, particularly my oldest daughter. I proved to myself that my daughter had a primitive idea of number or symmetry by playing games with her while she was in the womb - I would find her hands pressing from the inside of the womb, I would push in gently and then she would respond back. When I pushed in once, she would push back once. When I pushed in twice, she would push back twice. I taught her tongue-signs when she was a newborn and that was the only muscle she could control. She could use them to ask for a bottle or binky before she could hold her own head up. She had an ~80 word vocabulary when she was ~9 months old. My daughter independently invented very simple, but sufficient cases of, addition and multiplication, as a four year old. Her bents are for animals and their parts, currently dinosaurs. She can tell a gallimimus from a diplodocus by sight and would be delighted to explain the differences between theropods, sauropods, and pterosaurs. Her birthday was in June, so she is a very early 5 year old.
I'm also the eldest of 6 children from a religious culture where many kids were a good thing. I babysat for years in nursery, I took care of kids in sunday school and many activities. I have some experience with children outside of my own offspring.
It sounds like you are trying to show an animation or such to a kid to help them associate a sound with an action that you want from them. So you are trying to have the spoken words "Pick up toy" associated in their minds with actually picking up the toy. Is this the case? Please correct me if I am wrong here.
Do you know how to teach little boys to use the toilet using the M&M candies as rewards? (link) The subjects there (boys) often are resistant to change and don't want to use the toilet. Animations would only be a single tool in the toolbox that combines both installing understanding, and installing motivation into the child.
Without the ability to measure an error between the target and actual system, it is impossible to improve the alignment between the two. This is axiomatic in control systems engineering.
You have a developed mind in action, in the mind of a child. It gains advantage for the child when you think they don't understand. I find in general that most kids understand far more than they want to let on, or are able to communicate. This makes it essential that you give the child motive to answer correctly every time.
The will of the child confounds your data.
- What is the input to the child? Visual only? Verbal? Kinesthetic?
- What motivates the child to tell you the correct answer when they know it?
- How do you know that your mode of measuring the answer is appropriate to the child?
I would consider each correct answer a "1" and each incorrect a "0". A sample set of data would be a sequence of ones and zeros. If you have multiple choice then you can fractionally weight partially correct answers. There is literature on doing this at a college level. I think that engineering the wrong idea, when it characterizes typical misunderstandings or error modes, can be more informative than just knowing that the student selected the incorrect answer - because it can help to determine why the wrong answer was selected.
- A first simple test would be to plot both the running mean and running standard deviation as a function of time. Think about Simple moving average and bollinger bands.
- I would also consider a variability plot by question of the answers.
This would indicate which of the types of ideas they get vs. which
they do not.
- You could do a variation on 1&2 where you track mean and stdev by category (or question) over time.
- you could make histogram of results (true vs. false) for first half and second half of samples. This is equivalent to averaging over that window and making a histogram. I like empirical CDF because I feel that it is more useful than a histogram, but many folks are addicted to having bin-widths.