1
$\begingroup$

I have the percentage of completed activities of a set of users that have used a concrete application. I have the number for each of them, for instance:

  • Participant #1 --> 0.85
  • Participant #2 --> 0.48
  • ...
  • Participant #23 --> 0.76

The intention to have recorded these results is to prove that they can complete their tasks with a level of completeness of at least 80%. For this, the null hypothesis that I formulated (and that I want to reject) is: H0 = σ = 0.8

And the alternative is: H1 = σ> 0.8

This sounds logic for me, but I was reading some papers and books and all the examples I found about small single-sample cases are contrasting two kinds of users, for instance, expert users against novice users in the same group.

What do you think? Can you point me some literature with an example like mine? (where you want to prove that all the sample participants can do somenthing in a percentage).

Thanks in advance,

$\endgroup$
6
  • 1
    $\begingroup$ What does $\sigma$ represent here? It seems to me that you probably have heterogeneous users, so they would likely have different expected proportions $\endgroup$
    – Glen_b
    Nov 25 '16 at 21:35
  • $\begingroup$ I understood (I'm not really pretty sure) that when the sample you have is small, then you need to use σ. Isn't it? $\endgroup$
    – gal007
    Nov 25 '16 at 23:24
  • $\begingroup$ I'm really not sure what you're saying there (can you point to a source for it? That might give enough context to figure out what you mean), but it might make more sense if you explained what $\sigma$ is intended to represent here. $\endgroup$
    – Glen_b
    Nov 25 '16 at 23:26
  • $\begingroup$ I think it represents the mean of the successful rate for the usage of the application $\endgroup$
    – gal007
    Nov 26 '16 at 2:12
  • $\begingroup$ Is it ok to use the mean to demonstrate the hypothesis here? $\endgroup$
    – gal007
    Nov 26 '16 at 2:18
1
$\begingroup$

The answer to your titular question is "yes", and you've done so yourself in the body of your question. I'm pretty sure that is not much help to you so I will answer the different question of how to test whether the observed data support the hypothesis that the population mean is greater than 0.8.

I suspect that your confusion may stem from the fact that your data are a set of proportions, whereas the usual examples dealing with testing proportions (or confidence intervals for proportions) use data that are a single proportion such as ten heads from fifteen tosses of a coin = 10/15.

The first thing that might help is to subtract your null hypothesis value from each of the data values. Then your null hypothesis is the familiar $H_0=0$.

Next, recognise that your data are not normally distributed and either choose to use a non-parametric test or assume that the data are normal enough to use, for example, a single sample t-test. For the latter I would suggest that if the (un-subtracted) proportions get much higher than 0.8 or less than 0.2 then the bounds of the actual distribution will make the normal approximation problematical. However, if all of the (un-subtracted) data are close to 0.5 then there will probably be no problem with an assumption of normality with n=23 participants.

You might consider using a permutations test (Permutation test comparing a single sample against a mean) or look here for alternatives (http://www.basic.northwestern.edu/statguidefiles/ttest_one_alts.html).

Finally, consider whether your purpose might be best served by focussing on how many of the participants exceeded the 80% threshold, or what the median values is. There are many data analysis purposes for which the hypothesis and significance tests are not really necessary or helpful.

$\endgroup$
2
  • $\begingroup$ thank you. I had the doubt also because I read a book (I just found it again) saying: "A hypothesis of the form ‘method A is good for task T’ is not testable but a hypothesis of the form ‘method A requires less time than method B to accomplish task T’ is". And in my case I am trying to prove the first, don't you think? :( $\endgroup$
    – gal007
    Nov 26 '16 at 11:41
  • $\begingroup$ Yes, you may be trying to "prove' an unprovable. Hence my suggestion that you consider reporting the fraction of participant exceeding your threshold or the median success. Perhaps a confidence interval would suffice. $\endgroup$ Nov 26 '16 at 20:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.