14
$\begingroup$

I am looking for the bayesian counterpart of the two-sample t-test with unequal variances (the Welch test). I am also looking for a multivariate test, like Hotelling's T statistic. References appreciated.

For the multivariate case, suppose that we have $(y_1,\cdots,y_N)$ and $(z_1,\cdots,z_N)$, where $y_i$ (resp $z_i$) is a shortcut for a sample mean, sample standard deviation and number of points. We can assume that the number of points is constant across the whole dataset, the standard deviation the same for all $y_i$ (resp $z_i$) and that the sample means of the $y_i$ (resp $z_i$) are correlated. If you plot the sample means, they follow each other and by connecting them, you get a smooth varying function. Now on some parts the $y$ function agrees with the $z$ function, but on others it doesn't, because $\frac{mean(y_i)-mean(z_i)}{std(y_i)+std(z_i)}$ becomes big. I would like to quantify this statement.

Improve this question
$\endgroup$
1
  • $\begingroup$ Typing "behrens fisher" in the search box drives to valuable information about the bayesian approach to the two independet samples with unequal variances. $\endgroup$
    – Stéphane Laurent
    Jan 8, 2013 at 21:47

2 Answers 2

Reset to default
6
$\begingroup$

While you can do this in a Bayesian way, have you considered whether it would actually be better to estimate the difference in the means rather than test whether they are different? This is what Andrew Gelman frequently recommends. I can imagine some possible reasons for wanting to do hypothesis testing, but I don't think they're that common.

I don't think you need something like a t-test, because you can estimate the standard deviation well because you said the groups have very similar standard deviations.

If that's the case then I think this link should be what you need. It shows how to estimate a difference in means or do a hypothesis test (though I don't recommend this). You could also take a look at the part they reference in bolstad's book (you can find electronic copies online). Its possible to incorporate estimating the variances as well but it's more complex, so I suspect you're better off incorporating the prior information you have about the variances in a naive way (for example, using the unbiased Stdev estimator on each of the sets and then averaging them and pretending those are your 'known' stdevs).

Improve this answer
$\endgroup$
9
  • $\begingroup$ yes, but that leads to another problem. How can you know whether the difference in the means is actually significant? I'd compare it to the sum of the SD of each sample, but that's not very rigorous. $\endgroup$
    – yannick
    Sep 20, 2011 at 5:57
  • $\begingroup$ @yannick: "significant", statistically or real-world? $\endgroup$
    – Wayne
    Sep 20, 2011 at 15:18
  • $\begingroup$ @Wayne real-world I suppose. $\endgroup$
    – yannick
    Sep 20, 2011 at 16:14
  • 3
    $\begingroup$ @yannick: Real-world significance is a domain knowledge problem, not a statistical one. That is, I can tell you that I have some weight data and there is a statistically significant 10 gram difference in mean weights between two groups, at the 95% level, but does that have real-world significance? For a minnow, yes, for adult men, no. If you're talking real-world significance, I'd imagine comparing to SD or determining quantiles would answer your question even if that doesn't seem rigorous and leaves room for someone to disagree with you. $\endgroup$
    – Wayne
    Sep 20, 2011 at 16:32
  • $\begingroup$ @Wayne Suppose I look at $\frac{m_1-m_2}{s_1+s_2}$, you're saying that the decision to when we can say "significant" effect size is arbitrary? And so is the choice of the link function that would map that quantity to [0:1] ? Aren't there practical things people do? $\endgroup$
    – yannick
    Sep 20, 2011 at 18:17
11
$\begingroup$

John Kruschke has developed a Bayesian routine that is meant as a drop in replacement for the two-sample t-test. The routine is called BEST (Bayesian Estimation Supersedes the T-test) and is described here. I also made an online javascript version that runs in the browser available here.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.