Is binning of continuous data always bad for statistical tests? I was always thinking that binning of data if data is naturally continuous is bad. However, here is the case. The goal of study was to find if there is an association between a biomarker and disease progression.
I have 40 data points, measured in percents (disease progression). I have one biomarker which is factor (yes or no). I perform Wilcox test between groups with and without biomarker, comparing disease progression, I got p-value 0.07.
Doctors have clinically significant thresholds (symptoms decrease, symptoms remain more or less the same or symptoms highly increase). They divide data into these 3 categories and count number of times this biomarker occur within the subgroup (so it form contingency table $2\times 3$). They perform exact Chi squared test in this binned data (this is actually the case) and get p-value of 0.03. Expected values in contingency tables are not super-low - 40 data points are nicely distributed across 6 cells of the contingency table.
So, 2 questions: 1) in this particular case - does it mean that exact chi squared on binned data is more powerful test than Wilcoxon test? or it is just a coincidence? (I am not surprised that tests gave different p-values - I am surprised by the magnitude of difference in p-values) Or it is likely a violation of parametric assumption of this "exact chi-squared test" (I am unfamiliar with, but I think this test was assumed)? Of course I understand that Wilcox test between groups and proportion test within 3 subgroups are answering different questions - but the initial goal is to find if there is association between biomarker and disease progression and it seems that both statistical tests are answering it.
2) Is binning preferred over continuous data in some situations? In which?
 A: Related answer here from just yesterday. If the distribution you observe is the distribution you wish to test, then binning forfeits information and will thus on average reduce your ability to (in this case) reject the hypothesis that the distributions are different between the two groups. There are 3 important caveats, though.
1) the tests you use matter. For example, if you bin and do a chi-square, you might still expect more power to reject the null of independence than if you don't bin and test for stochastic dominance, because it's very difficult to reject the null hypothesis of no stochastic ordering. However, it should be easy to see that if you bin and do a t test, you'll get worse power than if you do a t test on the original data. I suspect that this is what's occurring in your example, since it seems easier to reject independence than to reject the bizarre Wilcoxon null (though I don't have a proof of this, maybe someone else can jump in).
2) finite samples might yield anomalous results sometimes.
3) if you believe there is significant measurement error and the binned data are "closer" to the actual thing you'd like to have measured, then binning might be a good idea. If the percents you have are imprecise/ad-hoc, then doctors might feel that the bins are a more faithful representation of the disease's progression. However, they might just feel more comfortable with the chi-square, which by itself is not a good reason.
Try a t test on the original percents. Your sample size is small enough that it shouldn't be completely trusted unless the percentages look very normally distributed, but I'll bet it gets a lower p-value than the chi-square.
A: My initial reaction is that 0.03 vs. 0.07 is quite a small difference in such a noisy random variable as a p-value - even if we assume the particular dichotomization was pre-specified (it is certainly true that shopping around for the optimal cutpoints dichotomizing will have a lower "p-value" - except that it will be invalid).
In any case, power is an average long-run property, not something one can judge from a single dataset. With a single pre-specified set of cut-offs - even if it is the optimal choices - you would expect more power with a continuous variable. You can of course try this and do a simulation.
