4
$\begingroup$

I would like to assess the effect size of a goodness of fit chi-square test. I have a variable that can have 3 states and test against a custom expected value. I used a variation Cramér's V for a goodness of fit test. However I have no idea what signifies a small/medium/large effect. There are conventions that vary with the degrees of freedom... however, since I did not use Cramér's V for a normal chi-square test but for a goodness of fit test I am lost...

Any idea or reference would be very much appreciated :)

$\endgroup$

2 Answers 2

3
$\begingroup$

if for some reason you haven't yet found a solution, there is a good explanation here: http://www.real-statistics.com/chi-square-and-f-distributions/effect-size-chi-square/

$\endgroup$
3
$\begingroup$

What is an interesting effect size is ultimately up to your judgement, there are no universal guidelines that apply to any situation. Some guidelines taken from Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed) are sometimes followed (see this question for some details about Cramér's V and Cohen's w). But as Cohen says in his book (p.224):

The best guide here, as always, is the development of some sense of magnitude ad hoc, for a particular problem or a particular field. Since it is a function of proportions, the investigator should generally be able to express the size of the effect he wishes to be able to detect by writing a set of alternate-hypothetical proportions [...], and, with the null-hypothetical proportions, compute w. Some experimentation along these lines should provide one with a "feel" for w.

In other words, the question you have to ask yourself is, for example: does it matter in any way, if some cells deviate of 1% from their expected values? 5%? 10%? How many cells should be affected by a deviation for the table to be interesting? For example, would the following table be interesting relatively to your research question?

state state A state B state C
proportion 0.45 0.3 0.25

If it's not interesting, what about this one:

state state A state B state C
proportion 0.55 0.25 0.20

or this one:

state state A state B state C
proportion 0.9 0.01 0.09

And so on.

Read about previous research on your ongoing subject of study, and think about what kind of tables would be borderline uninteresting, somehow interesting, or very interesting.

Then calculate the effect size of these hypothetical tables. And there you go, you know what effect sizes you can qualify as "small", "medium", "large", "astronomical", or whatever qualifier you'd be happy to use.

Preferably, you should do all these calculations before running your study, as sample size calculations require you to pre-specify the smallest effect size you're interested in. Maybe add some safety margin to your initial calculations to avoid ending up with an underpowered study.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.