Hi is it appropriate to do a post hoc power analysis after the null hypothesis is favoured in the Mann Whitney U test? The sample sizes are unequal and I want to know if it is underpowered. With posts like the one linked I'm not sure if it is appropriate. Thanks
$\begingroup$
$\endgroup$
6
-
4$\begingroup$ Many posts on site that discuss post hoc power, an overwhelming proportion of which explain why you should not use it. Many include references. The search stats.stackexchange.com/search?q=post+hoc+power turns up 372 hits. If seeing 3 or 4 such threads doesn't answer your question, please edit to explain what issue in you question has not been covered before or remains unclear. $\endgroup$– Glen_bCommented Nov 23 at 22:51
-
2$\begingroup$ If the responses to posts are insufficiently clear as to dissuade you, try the uh, colourful assessment at Gelmans blog.Many papers also discuss this issue, with less colourful phrasing $\endgroup$– Glen_bCommented Nov 23 at 23:00
-
$\begingroup$ @Glen_b Thank you so much for taking the time to respond you are a real one!. This is exactly what I needed haha How do I know then if my unequal sample sizes are ok? They have not been actively recruited, a binary outcome has occurred. Are power calculation still used to get required sample size before the study has occurred? $\endgroup$– Sum PersonCommented Nov 23 at 23:42
-
$\begingroup$ Against typical alternatives considered for the Mann-Whitney (or indeed almost any two sample test in common use) the effect on large sample relative efficiency (and so relative power for large samples and small effects) of having split the total sample size into fractions $p$ and $1-p$ (i.e. $p = \frac{n_1}{n_1+n_2}$) relative to an even split is approximately $4p(1-p)$. E.g. 100 observations in an 80-20 split will be roughly as powerful as 64 observations split evenly.Not what your post is asking for but maybe that can be of some value in relation to pre-sample power calculation $\endgroup$– Glen_bCommented Nov 24 at 2:27
-
1$\begingroup$ However, if your outcome is binary, why would you use Mann-Whitney? Why not a two sample proportions test / chi-squared test of homogeneity of proportions? $\endgroup$– Glen_bCommented Nov 24 at 5:41
|
Show 1 more comment
1 Answer
$\begingroup$
$\endgroup$
1
First, it is generally not advisable to do post-hoc power analysis (see references here). Second, it doesn't make sense to use a Mann-Whitney here if you have a binary outcome. You can easily run this data through a logistic regression instead (which is well suited for binary outcomes), or the suggestions made by Glen in the comments.
answered Nov 24 at 13:55
-
$\begingroup$ Thanks so much for your input! Much appreciated $\endgroup$ Commented 2 days ago