0
$\begingroup$

Description of my experiment: I have 2000 samples. For each sample I am measuring a metric Mi using four different methods A,B,C and D. These methods of measurement are completely independent. Ideally, the measurements should result in the same values for the metric i.e; M1A=M1B=M1C=M1D However, in some metrics, the different methods result in different values (subjective opinion based on reading the values for a few samples).

To test whether these values are statistically different I considered MiA as my reference and carried out paired t-tests between MiA and MiB; MiA and MiC; MiA and MiD, with the null hypothesis that the mean difference is 0 (2000 samples)

The results seemed alright to me

I also did a simple linear regression (SLR) with MiA on the X-axis and MiB, MiC and MiD. Visually there was a lot of information that was apparent which I liked. In some cases, the slopes were different from 1 and intercept was not 0. My understanding here is that if there are no differences, the points should fall very close to the y=x line.

I then carried out a joint hypothesis test (linearHypothesis() in R) where slope=1 and intercept!=0. In this test, I was to some extent able to relate the results to the visual effect of the scatter plots (when the lines are very different).

Is there a way to make sense of the results from the t-test and SLR together?

Although they both seem to convey some useful information, I feel limited in my ability to express their meaning together. As it is evident, the number of p-values is a lot! Besides the mean differences(ttest) and slope and intercept values for each model...

Example images for two different metrics are shown below, (MiB~MiA is green, MiC~MiA is orange, MiD~MiA is purple)

enter image description here enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.