Description of my experiment: I have 2000 samples. For each sample I am measuring a metric Mi using four different methods A,B,C and D. These methods of measurement are completely independent. Ideally, the measurements should result in the same values for the metric i.e; M1A=M1B=M1C=M1D However, in some metrics, the different methods result in different values (subjective opinion based on reading the values for a few samples).
To test whether these values are statistically different I considered MiA as my reference and carried out paired t-tests between MiA and MiB; MiA and MiC; MiA and MiD, with the null hypothesis that the mean difference is 0 (2000 samples)
The results seemed alright to me
I also did a simple linear regression (SLR) with MiA on the X-axis and MiB, MiC and MiD. Visually there was a lot of information that was apparent which I liked. In some cases, the slopes were different from 1 and intercept was not 0. My understanding here is that if there are no differences, the points should fall very close to the y=x line.
I then carried out a joint hypothesis test (linearHypothesis() in R) where slope=1 and intercept!=0. In this test, I was to some extent able to relate the results to the visual effect of the scatter plots (when the lines are very different).
Is there a way to make sense of the results from the t-test and SLR together?
Although they both seem to convey some useful information, I feel limited in my ability to express their meaning together. As it is evident, the number of p-values is a lot! Besides the mean differences(ttest) and slope and intercept values for each model...
Example images for two different metrics are shown below, (MiB~MiA is green, MiC~MiA is orange, MiD~MiA is purple)