2
$\begingroup$

I am currently writing up an experiment where we asked people on a 0-10 scale how much they expected three different beverages, water, decaf, and coffee, to reduce their caffeine withdrawal symptoms. Initially, using Bayesian parameter estimation, I modelled this as a within-subjects regression beverage type as the categorical predictor. I got great estimates that matched the mean differences from the descriptive stats, but the posterior predictive check of this model returned draws from the joint posterior that were outside the actual range of the data.

So to remedy this, and constrain the draws from the posterior to the actual range of the data I modeled the outcome variable as an ordinal variable, using ordered logistic regression (see this post).

I wanted to get estimates of the difference in log odds between the three beverage types. To do this I simply calculated, at each step in the markov chain, the difference in estimated log odds between each of the three pairwise comparisons of the three groups (water vs decaf, water vs coffee, decaf vs coffee). The parameter estimates from this model are expressed in log odds, which I can graph as histogram/density plots to show the mean and 95% Highest Posterior Density Interval for each pairwise comparison. However log-odds are difficult to interpret even at the best of times, let alone in an ordered logistic regression. So what I did was exponentiate each chain of log-odds estimates for each pairwise comparison to get odds ratios. These are the histograms for log-odds of the difference scores and odds ratios for each pairwise comparison

enter image description here

Each row graphs the histogram/density plots for each pairwise comparison as (i) log-odds (left) and (ii) odds ratios (right). Each graphs also has the mean and HPDI of each markov chain for each comparison.

To my eyes the odds ratio graphs look strange, of course this doesn't mean they're wrong, just that I am inexperienced. I have two questions

  1. Are the mean and the 95% HPDI the right statistics to use to summarise the distribution of odds ratios and/or log-odds? Mode or mean would make little difference for the log-odds but in the odds ratio graph the mean is considerably to the right of the the peak of the density plot, due to the heavy skew in the data. Perhaps the mode would be better?

  2. Do these graphs look ok? I understand exponentiating log-odds transforms distributions in unintuitive ways, but I don't have much experience in reporting distributions in these ways and just wondered if the right-hand column graphs look sensible, given the log odds distributions on their left?

$\endgroup$

1 Answer 1

2
$\begingroup$

To answer your second question first: your graphs look absolutely fine to me. Odds ratios typically have a skewed distribution - for one thing they lie in $[0, \infty)$ and the 'neutral' value is 1. That's one reason for analysing their logs, for which the range is $(-\infty, \infty)$ and which typically have a reasonably symmetric distribution.

Short answer to your first question: Take the posterior means and HPDIs of the log-odds ratios, then exponentiate them to give odds ratios for interpretation if you wish.

It makes more sense to look at geometric means rather than arithmetic means of ratio measures – ½ and 2 are equally far from 1 on a ratio scale. The geometric mean of a set of ratios is the same as the exponential of the arithmetic mean of their logarithms (see e.g. Wikipedia). As much of statistics centres around (arithmetic) means and expectations, statistical analysis of ratios is usually conducted on their logarithms, and the results then transformed back to the ratio scale only for interpretation. From a Bayesian perspective, you'd want a Bayes estimator of a ratio measure to be based on a loss function that punishes ½ and 2 equally if the true value is 1. Working on the log scale means you can use the posterior mean or median (the posterior mode is harder to motivate).

There's a nice illustration in your case: for consistency you'd hope that the Water:Coffee odds ratio should be equal to Water:Decaf $\times$ Decaf:Coffee, implying that the log-odds ratio for Water-Coffee should equal Water-Decaf $+$ Decaf-Coffee. That doesn't work for the posterior means of the odds ratios: $0.4 \ne 3.9 \times 0.2$. It does work for the posterior means of the log-odds ratios: $-1.3 = 1.0 + (-2.3)$. (It doesn't work for the endpoints of the HDPIs because stats ain't that simple – you can't add standard errors or endpoints of uncertainty intervals even when estimates are independent, and here they're definitely not...)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.