I've been reading a variety of articles 'bout p-value controversy and since the best way to understand theory is to practise it, I devised an example. Suppose, I'm a psychologist and I conducted an experiment. My study was simple - the aim was to check the level of empathy in two groups: men and women. From some theory and earlier studies, I know, that the distribution of empathy is normal in both groups and standard deviation is equal to 1. I want to test that H_0: μ_men = μ_women, against H_1: μ_men not equal μ_women.

If I want to use Neyman-Pearson approach and omit all that p-value issue, what should I literally do?

Recently, so that I can remember, what was inside, I've read those articles:

  • Lew, M.J., "To P or not to P: on the evidential nature of P-values
    and their place in scientic inference"
  • Lehman, 1993, "The Fisher, Neyman-Pearson Theories of Testing Hypotheses"
  • Hagen, R.L., "In Praise of the Null Hypothesis Statistical Test"
  • Cohen, J., "The Earth is round, p>0.05"
  • Hubbard, R., "The Earth is Highly Significantly Round (p<0.0001) [this is a comment on the article of Cohen]
  • Schmidt, Hunter, "Are There Benefits From NHST"
  • Dienes, Z., chapters 2,3 of "Understanding Psychology as a Science"

Well I'll try to calculate as @Glen_b suggests in a few days. I hope I can present my results here.

  • 1
    $\begingroup$ Short version: Construct a rejection rule/rejection region for a most powerful test (generally via the Neyman-Pearson Lemma) and then see if your test statistic is in the region or not. (You can actually do it with p-values, though - the rejection rule is just "reject if $p\leq\alpha$".) $\endgroup$ – Glen_b Jun 12 '14 at 21:29
  • 1
    $\begingroup$ Not sure what you've been reading, but there's much criticism of p-values' being reported where a point estimate & confidence interval would be more useful; N-P hypothesis testing isn't a move in the right direction from that point of view. $\endgroup$ – Scortchi Jun 13 '14 at 2:07
  • 1
    $\begingroup$ The distinctions between different approaches to hypothesis testing that Lehman's paper teases out have little to do with the criticisms made by Cohen, which are directed at hypothesis testing per se (or at any rate of its misuse). $\endgroup$ – Scortchi Jun 13 '14 at 16:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.