1
$\begingroup$

As part of my MSc, I have recently developed a web application designed to assist toddlers with learning difficulties in the comprehension of spoken instruction. The system shows a series of animations to convey an object and then tasks the child to pick out the correct object from a series of objects. I then collect the data as to whether the child selected the correct object.

Then, through utilisation of a spaced repetition algorithm the cards are re-ordered and reshown at cutom intervals.

So I have have a load of data that basically looks like this:

Card Name - Correct/Incorrect(binary)

I have produced a line chart in Excel that plots the total number of questions posed vs the number of questions answered correctly over time. The correct answer line looks quite straight, so I was hoping that I could show the children are learning the slope of the two lines are equal? Would this signify the children are answering all the questions correctly all the time?

Are there any other deductions I could look to be making? I'm not a stats person and quite new to all this.

$\endgroup$
1
$\begingroup$

Background:

I am a father of some very bright children, particularly my oldest daughter. I proved to myself that my daughter had a primitive idea of number or symmetry by playing games with her while she was in the womb - I would find her hands pressing from the inside of the womb, I would push in gently and then she would respond back. When I pushed in once, she would push back once. When I pushed in twice, she would push back twice. I taught her tongue-signs when she was a newborn and that was the only muscle she could control. She could use them to ask for a bottle or binky before she could hold her own head up. She had an ~80 word vocabulary when she was ~9 months old. My daughter independently invented very simple, but sufficient cases of, addition and multiplication, as a four year old. Her bents are for animals and their parts, currently dinosaurs. She can tell a gallimimus from a diplodocus by sight and would be delighted to explain the differences between theropods, sauropods, and pterosaurs. Her birthday was in June, so she is a very early 5 year old.

I'm also the eldest of 6 children from a religious culture where many kids were a good thing. I babysat for years in nursery, I took care of kids in sunday school and many activities. I have some experience with children outside of my own offspring.

Training

It sounds like you are trying to show an animation or such to a kid to help them associate a sound with an action that you want from them. So you are trying to have the spoken words "Pick up toy" associated in their minds with actually picking up the toy. Is this the case? Please correct me if I am wrong here.

Do you know how to teach little boys to use the toilet using the M&M candies as rewards? (link) The subjects there (boys) often are resistant to change and don't want to use the toilet. Animations would only be a single tool in the toolbox that combines both installing understanding, and installing motivation into the child.

Measurement

Without the ability to measure an error between the target and actual system, it is impossible to improve the alignment between the two. This is axiomatic in control systems engineering.

You have a developed mind in action, in the mind of a child. It gains advantage for the child when you think they don't understand. I find in general that most kids understand far more than they want to let on, or are able to communicate. This makes it essential that you give the child motive to answer correctly every time.

The will of the child confounds your data.

Questions:

  • What is the input to the child? Visual only? Verbal? Kinesthetic?
  • What motivates the child to tell you the correct answer when they know it?
  • How do you know that your mode of measuring the answer is appropriate to the child?

Presentation suggestion:

I would consider each correct answer a "1" and each incorrect a "0". A sample set of data would be a sequence of ones and zeros. If you have multiple choice then you can fractionally weight partially correct answers. There is literature on doing this at a college level. I think that engineering the wrong idea, when it characterizes typical misunderstandings or error modes, can be more informative than just knowing that the student selected the incorrect answer - because it can help to determine why the wrong answer was selected.

  1. A first simple test would be to plot both the running mean and running standard deviation as a function of time. Think about Simple moving average and bollinger bands.
  2. I would also consider a variability plot by question of the answers. This would indicate which of the types of ideas they get vs. which they do not.
  3. You could do a variation on 1&2 where you track mean and stdev by category (or question) over time.
  4. you could make histogram of results (true vs. false) for first half and second half of samples. This is equivalent to averaging over that window and making a histogram. I like empirical CDF because I feel that it is more useful than a histogram, but many folks are addicted to having bin-widths.
$\endgroup$
  • $\begingroup$ thanks for that reply. I have written a paper on this topic and would be happy to send it to you for insight. Input is both visual, verbal and the child is required to touch the correct object. When they do this they hear a "well done", an auditory re-inforcement. The system is in a prototyping stage but eventually I would add badges etc for motivational purposes. My mode of measurement is derived from the UK NHS language therapy activities in that choosing a key word object from a set of related objects is a sufficient initial test that the key word was conveyed in "some" form. $\endgroup$ – David Folksman Aug 20 '13 at 20:36
  • $\begingroup$ You might explore "Pixie Hollow" for ideas on reward/incentive strategies that work on .. mostly literate kids. They get badges, or their character gets recognition from famous pixies etcetera. (Parent of girly-girl here). Please feel free to send me your paper on the topic. I am ALWAYS in the market for ways to be a better parent. $\endgroup$ – EngrStudent - Reinstate Monica Aug 20 '13 at 23:51
  • $\begingroup$ Thanks for that, it will be very useful when I come to implement the rewards in the next stage. But I wonder, can you provide some insight into how I can evaluate my data in terms of showing that the children are providing more correct answers over time, and show that this relationship is due to my application? $\endgroup$ – David Folksman Aug 21 '13 at 9:08
  • $\begingroup$ Also, I don't see an email address on your profile. I'm quite new to the site and ergo, am not sure how to send you my paper? Could you let me know how to send you a direct or private message attachment? $\endgroup$ – David Folksman Aug 21 '13 at 9:12
  • $\begingroup$ You can send comment, or possibly email, to the author of my blog: me. I only see my side of the interface, so I can't tell you how to best do this. $\endgroup$ – EngrStudent - Reinstate Monica Aug 21 '13 at 15:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.