3
$\begingroup$

I'm using the BEST package in R as a proxy for the frequentist t-test to measure the difference in means between two groups.

However this process is VERY slow, and will simply not converge for the tests that I am trying to run. Is there another method that can work for this? I'm mainly looking for a Bayesian equivalent to a standard t-test.

Improve this question
$\endgroup$
11
  • $\begingroup$ How big are your n's? How long does it take? Also, can you give more details about what your situation is? Do you want to exactly replicate what BEST does, or do you just need something vaguely like it? There's not really enough information here to give good advice. I imagine a decent implementation of MCMC on this problem shouldn't be so terribly slow unless the sample sizes are huge (in which case, some approximation might suffice). $\endgroup$
    – Glen_b
    Jan 9, 2015 at 9:57
  • $\begingroup$ @Glen_b My n's are in the 10,000+ range each. I don't need to necessarily replicate what BEST does but I am looking for a Bayesian substitute for something akin to a t-test. Do you have any suggestions? $\endgroup$
    – John
    Jan 12, 2015 at 23:08
  • 1
    $\begingroup$ So in effect you want a posterior distribution for the difference in means? $\endgroup$
    – Glen_b
    Jan 12, 2015 at 23:13
  • $\begingroup$ @Glen_b yes that would work. I was looking for a Bayesian 'drop-in' substitute for a t-test in comparing the difference of two means and my research took me to the BEST package. I'm not completely familiar with Bayesian Decision Theory. $\endgroup$
    – John
    Jan 12, 2015 at 23:15
  • $\begingroup$ BEST assumes the data are t-distributed, while a t-test assumes the data are normal. BEST uses normal priors on $\mu$'s, uniform on $\sigma$'s, and exponential on $\nu$'s. Do you need normal-data, t-distributed data, or will something else more heavy tailed than normal be okay? If you do need t-distributed data, do you need those specific priors? Do you need to deal with unequal variance, or is common variance okay? $\endgroup$
    – Glen_b
    Jan 12, 2015 at 23:21

1 Answer 1

Reset to default
2
$\begingroup$

To first address the title question:

In effect that depends on implementation details. A more specialized implementation may well be faster.

Because BEST uses $t$ distributions for data, I don't think there's are simple summary statistics for the mean or variance parameters (this contradicts what I was suggesting in comments earlier). So each of those calculations will be at least O(n) (every new $\nu$ would require a recalculation of likelihood / conditional distributions across the whole sample).

This might be why it's slow.

--

Here's one example of how to tackle a straight/simple ('drop in') "Bayesian t-test", in this case framed as a regression-on-a-dummy problem that should be about as fast as a single simulation-step of MCMC.

Take the set-up here and following the derivation further down, where in this case $\beta$ can be regarded as $(\mu_1,\delta)$, where $\delta$ is the difference in means ($\delta = \mu_2-\mu_1$) and $X$ is a column of 1's and a column, $g$, which is the 0-1 indicator for membership in the second group.

Then the posteriors given there can be computed once (no actual matrix inversion is required, I think, but even if it was, it's only 2x2). We can integrate $\sigma^2$ out of the posterior for $\beta$ to get a marginal for $\beta$.

Indeed, it looks like you could integrate $\mu_1$ out, then $\sigma^2$ and get what appears to be a closed form (I think $t$-distributed) posterior for $\delta$ alone involving no simulation at all.

It doesn't have to be done as a regression of course -- you can do the same calculations more directly. [I set it up that way because the calculations are largely already done for you.]

Some slightly more complicated cases can still be done along similar lines. Further complications may be easier done using MCMC -- but in at least some of those cases the calculation can be kept to simple summary statistics which should still result in fast iterations. (If that's still too slow, you might be able to get somewhere via Laplace approximation. There are a number of other possibilities)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.