I understand different statistical tools have their own pros and cons. I'm trying to find the most appropriate one for my situation.
I have a large, unbalanced data set and want to implement the chi-square test on it to test the independence of two categories (or more).
Since the size is large, only looking at p-value always give me a significant result which is not what I want. To use effect size like Cramer's V, the unbalance of the data set automatically gives me a low score.
The unbalance comes from the nature of the data, like number of cancer patients, so I don't think use sampling strategy will be a good idea, cause it changes the underlying distribution.
I'm wondering if there's any appropriate strength test or other method fitting into my situation? (A standard method will be preferred)
Any idea is appreciated:)
E.g., a way I'm trying is that instead of normalized by the number of total data like in Cramer's V, I use the number of minority part to normalize, so that it can be robust to the size of sample and deal with the imbalance issue. Of course its cons is the sensitivity w.r.t. the size of the minority part, which makes it more like a ratio test.