In my youth I can remember reading some papers (in psychology most likely) which as part of their literature review included a table of past results with no details provided other than whether the p-value from the study was < .05 or > .05. For example:

|       Study        | Significant |Non-Significant|
| Jones (2011)       |      ✔      |       ✘      |
| Poe and Ng (1980)  |      ✘      |       ✔      |
| Smith (1985)       |      ✘      |       ✔      |
| Wong & Ng (1992)   |      ✔      |       ✘      |

EDIT: I found out from Borenstein et al (2011) that this method is called the "vote counting" method.

For teaching purposes I'm interested to discuss with students the decline in focus on p-values in scientific literature, with particular reference to psychology. I find this vote counting approach to be a neat example of the extremely heavy focus on p-values that was common (in psychology at least) until fairly recently.

Does anyone know of any research into the use of this vote counting approach, i.e. research into its downtrend over time?

Failing that, can anyone cite an example paper in which this sort of approach was used? Despite my memory of reading such "vote counts", I can't remember any specific example of the top of my head.

Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2011).Introduction to meta-analysis. John Wiley & Sons.

  • 2
    $\begingroup$ I see this a lot in social work literature, for example, this article from a social work journal might be useful. Anecdotally, I'd think the the downtrend over time might be due to greater awareness of other meta-analysis methods. Again anecdotally, I'd say I'm more likely to see other forms of meta-analysis in higher-impact journals, and more likely to see vote counts in lower-impact journals. $\endgroup$ – 5ayat May 17 '16 at 15:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.