I am interested to find a way to identify if two sets of data can be considered statistically different at 95% Confidence level (or any other). To be more specific, my data sets are composed of 5 values. They correspond to 5 readings of the same detector by one microscope. So, two detectors with 5 readings each can represent the same value or not. The problem is to find a method to answer this question. I'm wondering if a t-test is the proper tool. Am I right? I would like to do the statistical test in R.
I am slightly unsure what you are testing for. Are you testing for concordance (you expect the two sets of data to be the same) or are you testing to find a difference between the two sets?
For Difference Between Groups If you have only a single, 2-level categorical predictor, and hypothesize there is a difference in means for the outcome, then a t-test is appropriate. However, I would strongly recommend against doing more than 1 t-test. The Type I error rate of hypothesis testing goes up rapidly with many t-tests. In R:
set.seed(1066) eggs <- rnorm(5) spam <- rnorm(5) t.test(eggs,spam)
The confidence interval (-2.6, 1.4) includes zero, so the null hypothesis cannot be rejected and no difference is observed.
EDIT: As @gung mentioned below, with only 5 reads, the assumptions of the independent samples t-test cannot really be checked. t-test assumes both samples are normally and identically distributed. For your data, this might be unreasonable. If so, Mann-Whitney's U test is more appropriate. It is a nonparametric analog to the t-test. In R:
Here, the p-value is ~1.0, so a similar conclusion is made that the null cannot be rejected and no difference is observed.
For Concordance However, if you are hypothesizing the more subtle outcome that the two sets of reads should be equivalent (being the same reads from the detectors on the same microscope), I would recommend using a concordance measure. Lin's Concordance measures how well two sets of paired data concord with each other. The statistic of interest, rho, is similar to Pearson's product-moment correlation coefficient, but adjusted for exact agreement along the x=y line (note, this is R, not R-square). Closer to 1.0 is strong concordance; closer to 0.0 is no concordance. In R, use the epiR package.
Here, the rho is modest, 0.38, indicating low concordance. The confidence interval (-0.30, 0.80) includes 0, so the null hypothesis (the reads are discordant) cannot be rejected.
These two results seem to contradict each other, but it depends on the subtlety of what you're asking and what sort of assumption is appropriate. In the first, we assumed they are the same; in the second, we assumed they are different. Depends on what question you are trying to answer and the type of assumptions that are reasonable for your study.
@Joshua and @PaulGowder both provided links and detailed answers but unfortunately both answers are wrong in the sense that they do not answer the OP question as stated in the title. In the title, the OP wants to provide evidence that the two set of measures are EQUAL not different. t-test or Mann-Whitney will provide evidence (with 95% confidence) that the two sets are NOT EQUAL - but it cannot provide evidence that they are equal. Confusingly, in the first line of the post, the OP asks how to verify that the sets are different (for which the answers posted are correct).
Showing that two sets are not different with 95% of confidence is NOT the same as showing that they are they are equal (with 95%). To "prove" that two sets are EQUAL you need an equivalence-test tag: equivalence in CV. In particular TOST (two one sided t-test) is the "most common" test, but I do not know what to do with the fact that you only have 5 data points.
Let me link to one of the great answers in CV on equivalence testing: How to test hypothesis of no group differences? You should start from there. There are others. Sometimes this topic is also described as "proving the null hypothesis"
Here's a helpful discussion: http://www.researchgate.net/post/What_is_the_difference_between_T-test_F-Test_and_anova_tests_in_statistics
If you're just comparing means across two groups, e.g., to test an experimental effect, the basic default choice is a t-test.