Significance tests I have two sample means from two different samples. I purport that the true population means are identical.
$H_{0}$: $\mu_{1}=\mu_{2}$ and
$H_{1}$: $\mu_{1}\ne\mu_{2}$
What significance test would I use to determine evidence for or against the null hypothesis?
Sample 1: {4,3,1,3,5,2,2,3,5,3}
Sample 2: {5,3,2,2,3,3,1,4,5,5}
 A: You might consider a permutations test.
A permutations test assumes that the observations are drawn from one population and then treatments are randomly allocated. Thus in the context of a permutations test for a difference in the means the null hypothesis (no treatment effect) becomes equivalent to a statement that any difference between the groups under the null hypothesis is a consequence of only the random allocation of the values into the groups. The significance of the observed differences between the treatment groups is thus just a measure of how unusual the observed allocation is relative to all possible random allocations.
The significance is thus calculable by enumerating all possible allocations and finding from that list the distribution of random differences between group means. The probability under the null hypothesis of obtaining a difference as great as that observed or greater is equal to the proportion of the population of differences that is as as that observed or greater.
More detail, some references and free software (a bit archaic...) can be had from my webpage: http://www.pharmacology.unimelb.edu.au/statboss/permutations%20test.html
