How to test whether matched pairs have mean difference of 0?

I am doing a study in which I am measuring a set of lengths using two different methods. I want to test how close the measurements given by the two methods are for corresponding lengths. Is there a way to do this? I know that a matched pairs t-test can give a p-value for the means being different, but how can I test how close the differences are to 0?

In other words, since the two methods are measuring the same lengths, they should most ideally give the same measurement for any given length. I want to test how close this is to being true and quantify the amount of error that is present. So say I plotted the lengths given by one method against those given by the other. If they did give the same measurement for every length, it should form the line $Y=X$. Basically, I want to test the hypothesis that the data does form the line $Y=X$, calculate the mean error (maybe the average distance from $Y=X$), and perhaps produce a p-value for it.

• Give more detail! You probably do not want testing (you do not have an hypothesis!), you want confidence intervals. Sep 4 '16 at 19:50
• 1. Is it definitely the mean of the distribution of differences you wish to test (rather than a more general kind of location-difference)? 2. Note that a p-value (what you ask for in the body text) is not the same as "probability that matched pairs have mean difference of 0"; indeed ordinary hypothesis tests don't give you such a probability. Unless you specifically want a Bayesian approach I'd suggest you remove the word "probability" from your title. Sep 4 '16 at 20:02
• In case it isn't clear, I'm a beginner to stats. I'm not sure how to explain precisely what I want. I'm looking for any advice as to the best way to test how closely the two methods are measuring the same values. Sep 4 '16 at 20:06
• I thought that a p value would be nice, but if that isn't the best approach, anything else that may seem to apply would be fine. Sep 4 '16 at 20:06
• Try searching for Bland and Altman plot which people in health research often use in these circumstances. Sep 4 '16 at 20:13

I think you are mixing different tools and different goals. I summarize several techniques available for what I understand you want to do:

• t-test for paired samples tests the hypothesis that both mesures have the same mean and gives a p-value about it.
• Confidence intervals estimate an interval where the true difference in means is expected to be with a given probability (usually 95%). That seems what comes most close to "test how close the measurements given by the two methods are for corresponding lengths". Most statistical packages produce confidence intervals when doing a t-test.
• If you are interested in measuring the errors between the two measures (an not just the difference of means), standard deviation of differences or coeficient of variation might be useful. Studying the distribution of errors and checking for normality can give you interesting information, too.
• How are the two measures related, or how are they related to error might be interesting too. You have plotted one measure against the other, but next steps could be plotting error against one measure (specially if errors are much smaller than measures, as usually happens) and performing a regression analysis.

Another approach would be to use the methods of Bland and Altman. You would do well to read their famous paper (which is short and easy), "Statistical methods for assessing agreement between two methods of clinical measurement" (pdf). Briefly, using the corresponding values from the two measurement methods, you would form two new sets of values, their means and their differences. A plot of the differences against the means is called a Bland-Altman plot. This will make it easy to assess if there is a curvilinear relationship or changing variance. If nothing is amiss, and you want to perform a hypothesis test, you can use the differences to conduct a one-sample $t$-test against $0$. You can also compute the mean difference and form a confidence interval around that.