11
$\begingroup$

I'm working through the examples in Kruschke's Doing Bayesian Data Analysis, specifically the Poisson exponential ANOVA in ch. 22, which he presents as an alternative to frequentist chi-square tests of independence for contingency tables.

I can see how we get information about about interactions that occur more or less frequently than would be expected if the variables were independent (ie. when the HDI excludes zero).

My question is how can I compute or interpret an effect size in this framework? For example, Kruschke writes "the combination of blue eyes with black hair happens less frequently than would be expected if eye color and hair color were independent", but how can we describe the strength of that association? How can I tell which interactions are more extreme than others? If we did a chi-square test of these data we might compute the Cramér's V as a measure of the overall effect size. How do I express effect size in this Bayesian context?

Here's the self-contained example from the book (coded in R), just in case the answer is hidden from me in plain sight ...

    df <- structure(c(20, 94, 84, 17, 68, 7, 119, 26, 5, 16, 29, 
                      14, 15, 10, 54, 14), .Dim = c(4L, 4L), 
    .Dimnames = list(c("Black", "Blond", 
    "Brunette", "Red"), c("Blue", "Brown", "Green", "Hazel")))

    df
    
             Blue Brown Green Hazel
    Black      20    68     5    15
    Blond      94     7    16    10
    Brunette   84   119    29    54
    Red        17    26    14    14

Here's the frequentist output, with effect size measures (not in the book):

    vcd::assocstats(df)
                        X^2 df P(> X^2)
    Likelihood Ratio 146.44  9        0
    Pearson          138.29  9        0
    
    Phi-Coefficient   : 0.483 
    Contingency Coeff.: 0.435 
    Cramer's V        : 0.279

Here's the Bayesian output, with HDIs and cell probabilities (directly from the book):

    # prepare to get Krushkes' R codes from his web site
    Krushkes_codes <- c(
      "http://www.indiana.edu/~kruschke/DoingBayesianDataAnalysis/Programs/openGraphSaveGraph.R", 
      "http://www.indiana.edu/~kruschke/DoingBayesianDataAnalysis/Programs/PoissonExponentialJagsSTZ.R")
    
    # download Krushkes' scripts to working directory
    lapply(Krushkes_codes, function(i) download.file(i, destfile = basename(i)))
    
    # run the code to analyse the data and generate output
    lapply(Krushkes_codes, function(i) source(basename(i)))

And here are plots of the posterior of Poisson exponential model applied to the data:

enter image description here

And plots of the posterior distribution on estimated cell probabilities:

enter image description here

$\endgroup$

2 Answers 2

1
$\begingroup$

One way to study effect size in ANOVA model is by looking at "super population" and "finite population" standard deviations. You have a two way table, so this is 3 variance components (2 main effects and 1 interaction). This is based on mcmc analysis. You calculate the standard deviation for each effect for each mcmc sample.

$$ s_k=\sqrt{\frac{1}{d_k-1}\sum_{j=1}^{d_k}(\beta_{k, j}-\overline {\beta}_k)^2}$$

Where $ k $ indexes the "row" of the ANOVA table. Simple boxplots of the mcmc samples of $ s_k $ vs $ k $ are quite instructive on effect sizes.

Andrew Gelman advocated this approach. See his 2005 paper "analysis of variance: why it is more important than ever"

$\endgroup$
5
  • $\begingroup$ That paper is available here. $\endgroup$ Mar 28, 2014 at 13:32
  • $\begingroup$ Both of these answers seem very promising, thanks. Are either of you familiar enough with R to show how it might be programmed? $\endgroup$
    – Ben
    Mar 29, 2014 at 19:24
  • $\begingroup$ @seaneaster - thanks for adding the link. @ben, these calculations are simple in R. However I'm not sure what form your samples are in. You should be able to use sd () combined with one of the "apply" functions. As for boxplots, these are simple to get basic ones with boxplot (). $\endgroup$ Mar 30, 2014 at 5:29
  • $\begingroup$ Thanks, can you demonstrate using the example data and code in my question? $\endgroup$
    – Ben
    Mar 30, 2014 at 15:56
  • $\begingroup$ In short, no because I don't understand the code you've posted - I can't see how the data is organised. And as I said, this is not a difficult analysis to do yourself. This approach is calculating a simple measure (standard deviation). Additionally, R coding is not part of your question - you asked about how to summarise contingency table analysis. $\endgroup$ Mar 30, 2014 at 16:08
1
$\begingroup$

Per the index, Kruschke only mentions effect size twice, and both times are in the context of a metric predicted variable. But there's this bit on p. 601:

If the researcher is interested in violations of independence, then interest is on the magnitudes of the $\beta_{rc}$. The model is especially convenient for this purpose, because arbitrary interaction contrasts can be investigated to determine where nonindependence is arising.

So, I gather that $\beta_{1,2}$ is the parameter to interpret. Let $S$ equal the sum of products of all coefficients and their corresponding x elements, excluding $\beta_{1,2}$ and $x_{1,2}$. Since $y_i {\raise.17ex\hbox{$\scriptstyle\sim$}} Pois(\lambda_i)$ and $\lambda_i = e^{\beta_{1,2} x_{1,2} + S} = e^{\beta_{1,2} x_{1,2}} e^S$. When $x_{1,2}$ = 1, then $\lambda_i$ grows or shrinks by a factor of $e^{\beta_{1,2}}$, no?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.