How to test if two means are significantly similar? I'm trying to run an experiment to see if my test and control (in this case 2 survey types) are similar, as opposed to significantly different, from each other.
Is there a test/way to do this? almost like an inverse of a t-test?
 A: It's a bit tricky but what you are looking for is called equivalence testing (see wiki).
The general idea is this: assume we have 2 surveys $A,B$. we would like to formulate our hypotheses over $\mu_A, \mu_B$. An initial significance testing hypotheses would be:
$$H_0:\mu_A=\mu_B,~~~H_1:\mu_A\neq\mu_B$$
we assume the means are exactly the same, meaning their difference is 0:
$$H_0:\mu_A-\mu_B=0=\mu_0,~~~H_1:\mu_A-\mu_B\neq0$$
and then you know what to do (t-test). The next step is defining an indifference region around the null hypothesis. That is, a region of size $\delta$ around the null hypothesis $[\mu_0-\delta,\mu_0+\delta]$ (which in our case is simply $[-\delta,\delta]$) that we are indifferent to. The idea is to tolerate results around the null just as if thy were the null, so we don't reject the null for minor differences. So our updated hypotheses are now:
$$H_0:|\mu_A-\mu_B|\leq \delta,~~~H_1:|\mu_A-\mu_B|>\delta$$
Notice that using absolute values requires a slight change to the statistical test. Still, our null hypothesis is that the means differ by at most $\delta$. It is up to the data (and the statistician) to prove that the means differ.
The final step here is moving to equivalence testing (AKA noninferiority). The most important thing is the conceptual change: rather than assuming by default (i.e the null) that the means are equal, we now assume that they differ. Not only we assume they differ, but the difference is at least $\delta$. We formulate our hypotheses as:
$$H_0:|\mu_A-\mu_B|\geq \delta,~~~H_1:|\mu_A-\mu_B|<\delta$$
So we have inverted the hypotheses, and now it is up to the data (and the statistician) to prove that the means do not differ.
This ain't a trivial move, and it requires choosing both $\delta$ and $\alpha$ prior to observing the data. I hope this answers your question.
A: If this is the case, then I encourage you to determine the confidence interval of the mean difference (the narrower, the better) which will make your demonstration even stronger.
First, I would encourage you to visualize your sample data to check whether your assumption is likely or not. Then, one possibility could be to identify a 95% confidence interval (CI) for the difference of means (or even a 99% if you are sure that they are similar) , you can refer to the "basic steps" chapter of https://en.wikipedia.org/wiki/Confidence_interval. To demonstrate your point, your CI must include 0 and the lower and upper values of your CI must be very close to 0 (this depends of the unit/scale of your test of course).
I hope it helps!
