12
$\begingroup$

Wikipedia says

effect size is a measure of the strength of a phenomenon or a sample-based estimate of that quantity. An effect size calculated from data is a descriptive statistic that conveys the estimated magnitude of a relationship without making any statement about whether the apparent relationship in the data reflects a true relationship in the population.

To understand it better, I was wondering what descriptive statistics are not effect size, except graphs and plots.

Improve this question
$\endgroup$
1
  • $\begingroup$ Graph and plots can actually be great to gauge the size of an effect in more intuitive ways than effect size measures. If you actually see the overlap between two groups on some measures (which would roughly correspond to a smaller d), it's perhaps easier to realize that a significant difference does not mean that member of one group all have lower scores than members of the other group, etc. $\endgroup$
    – Gala
    May 16, 2013 at 7:09

2 Answers 2

Reset to default
19
$\begingroup$

Effect sizes

  • Common standardised effect sizes typically quantify the amount or degree of a relationship or effect. The most common effect size measures are probably cohen's d, Pearson's r, and the odds ratio (particularly for a binary predictor).
  • Less common effect size measures: That said, you can have standardised and unstandardised effect size measures. Any statistic that communicates the degree of relationships and is not especially contaminated by sample size is probably an effect size measure. Thus, Beta coefficients, R-square, covariance, raw mean differences between groups, and so on all capture the degree of effect. That said, I find that some researchers apply effect size measures somewhat blindly and forget that the broader aim is to give readers a sense of the degree of effect. And thus, they often don't realise that measures like mean differences or raw regression coefficients are in some sense an effect size measure. Another example of blind use of effect sizes involves the use of effect size measures that do not have an intuitive interpretation, but have been recommended by some textbook.

Not effect sizes:

  • Most test statistics are not effect sizes. E.g., Chi-square test, t-test, z-test, F-test. They get larger both as the size of the population effect increases and as sample size increases. In many respects the whole language of effect sizes has been emphasised in recent years because researchers were focussing too much on how large their test statistics were rather than how large their effect sizes were. This is especially important where you have a large sample size when even small effects can be statistically significant.
  • Most univariate statistics are not effect sizes. For most purposes, effect size is concerned about the relationship between at least two variables. Thus, the sample mean, standard deviation, skew, kurtosis, min, max, and so on are not effect size measures.
  • Statistics not pertaining to degree of relationship are not effect size measures. For example, tests of multivariate normality, the eigenvalues of a matrix, and so on generally are not directly aimed at quantifying an effect in the ordinary sense of the word.

Broader considerations

  • Scaling considerations: The utility of a statistic as an effect size measure largely relates to its ability to communicate the size of an effect. Sometimes this is achieved by using familiar standardised measures of effect (e.g., cohen's d). Other times, careful consideration of scaling of the variables can yield an even clearer interpretation of the size of the effect. For example, say I had a study looking at a training program on income levels. I could report that the training program increased income by .2 standard deviations or I could say that the program increased income by $3,500 US dollars. Both are useful; both are effect size measures. The first is standardised (cohen's d), the second is unstandardised (raw group mean differences).
  • Precision in estimating effect sizes: We often extract sample estimates of effect size measures (e.g., cohen's d, pearson's r, etc.). This context can lead to a contrasting of significance testing with effect size measures. Nonetheless, the aim should still be to estimate in a precise and unbiased way, the population effect size. From a frequentist perspective, confidence intervals around effect sizes provide an estimate of precision. From a Bayesian perspective, there are posterior densities on effect sizes. In many cases, care needs to be taken to ensure that you are using an unbiased effect size measure.
Improve this answer
$\endgroup$
4
  • $\begingroup$ The third and last points probably explain where the author(s) of the Wikipedia article is coming from. Given the emphasis on psychology, I think the point is not so much to contrast effect size with other descriptive stats but rather with test statistics and p-values (i.e. inferential statistics) and to emphasize that effect size measures say nothing about sampling variability. $\endgroup$
    – Gala
    May 16, 2013 at 7:03
  • $\begingroup$ Thanks a lot for your nice answer. I have a question though: do you mean confidence interval cannot be used as an effect size measure, because it is directly related to sample size? (by confidence interval, I mean the value that is both added to or subtracted from the prevalence, mean, etc. - not the upper and lower bounds of a CI). $\endgroup$
    – Vic
    Jun 26, 2013 at 6:45
  • 2
    $\begingroup$ @Vic you can have a confidence interval on an effect size measure, but the confidence interval itself is not the effect size. $\endgroup$
    – Jeromy Anglim
    Jun 26, 2013 at 8:27
  • $\begingroup$ Uh many thanks dear Jeromy. For all these years I was mistaken. :) $\endgroup$
    – Vic
    Jun 26, 2013 at 19:29
6
$\begingroup$

First, effect sizes can be used inferentially as well as descriptively. r and ORs are all effect sizes and are certainly all used in inferential stats.

Univariate statistics are usually not effect sizes, although they can be. E.g. If you are comparing ages of men and women who are married to each other, the mean age of men is not an effect size (then the difference of means would be one effect size). But if you want to see if the mean of something is 0, then the mean would be an effect size.

If it measures an effect, it's an effect size!

Improve this answer
$\endgroup$
11
  • $\begingroup$ I guess that is true @Peter, but effect size is a term that has been more narrowly defined by Cohen: (Mean1-Mean2)/PooledSD. This sounds a bit like is the difference significant, or only statistically significant - the use of common words to define a statistical term. $\endgroup$
    – doug.numbers
    May 15, 2013 at 21:58
  • 2
    $\begingroup$ Where does Cohen define it that way? If you mean his book on Power Analysis, I think he uses that as the sort of standard to convert other effect sizes to. But every power analysis table in that book (and there are a LOT) uses some effect size (and not all use that one) $\endgroup$
    – Peter Flom
    May 15, 2013 at 22:01
  • 1
    $\begingroup$ Cohen's d is always the way I understood it. Similar as described en.wikipedia.org/wiki/Effect_size. But you are absolutely right, there are a lot of methods described as effect size. $\endgroup$
    – doug.numbers
    May 15, 2013 at 22:09
  • 4
    $\begingroup$ The t-test and the z-test are not effect sizes. the same effect size will yield substantially different t and z values for different sample sizes. $\endgroup$
    – Jeromy Anglim
    May 16, 2013 at 5:24
  • 1
    $\begingroup$ @JeromyAnglim is right; +1. I edited my answer $\endgroup$
    – Peter Flom
    Jun 26, 2013 at 12:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.