7
$\begingroup$

After a previous attempt to get answers here to what I found out was a nonsensical question I am looking for some good resources I can use to begin to understand the area.

The majority of Psychology experiment books in our library focus on statistics, data analysis, qualitative research methodologies and experiment design. But methods and techniques for determining sample size, with practical examples, seem to be few and far between, especially at a beginner level.

I have found a few books that I believe would be useful online but they are quite expensive and I would rather get feedback that they were worth the money and that there was nothing better out there.

  • Sample Size Determination and Power - Thomas P. Ryan
  • Sample Size Calculations: Practical Methods for Engineers and Scientists. Paul Mathews

So can anyone recommend books or other resources in the area, specifically for determining the sample size for hypothesis testing of the mean?

A brief overview of my research area: I am undertaking research to determine which particular features of a website or software interface affect a users perception using metrics such as trust, fairness, expertise etc. Each participant will rate 9 separate designs, each with a different distortion which I will then compare to the control. Participants will be crowdsourced online. Hope this helps.

Improve this question
$\endgroup$
2
  • 5
    $\begingroup$ Hmm. Good question (+1). I wish I had a book to recommend. Personally, I tend to use simulation for any but the simplest cases. $\endgroup$
    – Glen_b
    Mar 28, 2014 at 4:41
  • 3
    $\begingroup$ +1 to both the question and @Glen_b's comment. I personally will usually use simulation in all cases. Complex stuff (mixed models etc.) is not treated in the textbooks. For the simple cases, I never have the textbooks handy when I need them. And even if I did, it's just easier to whip up a few lines of R than looking something up. (If it isn't so easy for you, power analyses and sample size calculations in simple cases are wonderful opportunities to practice R, so I strongly recommend this!) $\endgroup$
    – Stephan Kolassa
    Mar 28, 2014 at 8:57

3 Answers 3

Reset to default
4
$\begingroup$

Some very basic sample size calculations are discussed here:

http://www.itl.nist.gov/div898/handbook/prc/section2/prc222.htm

http://www.itl.nist.gov/div898/handbook/prc/section2/prc242.htm

For a basic introduction, the book by Moore and McCabe - Introduction to the Practice of Statistics covers some of the basics in chapter 6.

Some deeper discussion is in

Russell V.Lenth, 2001.
"Some Practical Guidelines for Effective Sample Size Determination"
The American Statistician, August 2001, Vol. 55, No.3

Edit:

I just noticed that I had left out Jacob Cohen's book. I meant to mention it - Statistical Power Analysis for the Behavioural Sciences

I don't know this book, but I've seen some people recommend

Chow S, Shao J, Wang H. 2008.
Sample Size Calculations in Clinical Research.
2nd Ed. Chapman & Hall/CRC Biostatistics Series.

As I said in comments, I tend to use simulation.

Improve this answer
$\endgroup$
5
$\begingroup$

I will try to provide some answers, while also giving the disclaimer that I am the author (Ryan) of one of the books that was mentioned.

First, I would also recommend the paper by Lenth that was mentioned, as I always provide my students with that paper in the online courses on sample size determination that I teach.

Regarding the costs of books on this subject, the list price of my book is 110 dollars but it can be purchased at one of the "discount" sites (such as half.com) for well under 100 dollars.

I have a copy of the book by Chow, Shao, and Wang, which is at a somewhat high mathematical level and is a good source for biosatisticians and others with sufficient mathematical expertise. One Amazon reviewer has expressed frustration at the number of errors in the book. I've found one or two but it is impossible to write a statistics book that is completely free of errors.

It is true that complex cases, such as mixed models, are generally not covered in books on the subject, but relatively few books have been written on sample size determination. Cohen's book was for many years the standard book on the subject, but it was published 26 years ago and is thus outdated. If you read Lenth's paper, you will see that he debunks Cohen's use of small, medium, and large effect sizes, referring to them as "shirt sizes". :-) I agree with Russ Lenth on that, whom I have known for almost 30 years.

Much research is needed om sample size determination but this is not a subject that interests statisticians.

PASS is the most comprehensive software package for sample size determination and it does have mixed model capability. It uses simulation to determine sample size, even in some cases where analytical results are known.

Tom Ryan

Improve this answer
$\endgroup$
1
$\begingroup$

This is a broad question and there are books on the subject. How Many Subjects?: Statistical Power Analysis in Research (Second Edition) is one that many have found accessible.

See also the link Good text on Clinical Trials? provided in a comment.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.