How to measure association of nominal values for few observations?

Question

I have a sample with 181 observations. I measured two nominal variables for each observation and now want to calculate the association between the two variables. The problem is that one variable has 15 possible values, the other one 16. So when I tried to compute Cramér's V, R complained that "Chi-squared approximation may be incorrect", which the Internet tells me comes from many cells in the contingency table having too few observations in them.

Is there any way to correct for this?

The two ways I can think of won't work:

1. Increase the sample. I don't have access to more observable items. With lots of effort, I may get 20 more or so, but nowhere near enough for 5 values per table cell.
2. Combine variable similar variable values into categories. My variables are truly nominal, with no real similarity between the values. I can't think of any combination which will make sense.

Is there some possible treatment of the data which will reduce the error of calculating chi squared? Is there an association measure other than Cramér's V which is better suited to small sample sizes (or large variable domains)? Is there anything else I can do?

If I cannot correct, what are the implications for my analysis?

If there is nothing I can do to correct this, what will happen if I do the analysis anyway? Will the results be considered so bad as to be unusable? Will I be able to use the results, but keep in mind that they may be a bit off? If they are a bit off, how are they off? Is the resulting Cramér's V likely to be too high, or too low, or is there no way to tell?

Answer 1

I think you clearly have too many cells (240) relative to the number of observations (181). In fact, even if there was only one observation in each cell, you would still have 59 empty cells. Analyses based on Chi² will be highly misleading, as in being truely useless.

You say that it is impossible to increase sample size or collapse cells, but you still need a measure of association between the variables.

The best you could possibly do is recode one of the variables, say X and Y, to a dichtotomous variable. For example you recode Y to Y==1 vs Y!=1. Then you consider the association of X and recoded Y, where now Y has only two categories, which might be enough to resolve the sparseness problem. Additionally you can consider the local odds ratios of all 15 categories of X with dichotomoized Y to inspect where biggest effects occur. You could repeat this analysis for all categories of Y, so next you dichotomize to Y==2 versus other and so forth. This way you learn which of the categories of Y is most strongly predicted by X.