1
$\begingroup$

We are testing two imaging machines. One of the machines has been validated numerous times. However, we built the new machine and we need to prove that it is similar/comparable to the validated machine.

So, the machines image a certain tube and determine a physical property. With several image loops, we can find the same physical property, with slight variations, and obtain a standard deviation for the physical property for each machine.

We want to prove the similarity of the new method to the old method.

I have basic understanding of statistics. I tried to solve this by showing a large P value, just to realize that it is not sufficient proof.

Can we publish an article if we say that the two methods give the same order of magnitude results? Is this possible or do we need a clear statistical proof?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Large p-values are easy, if you take small samples. If you try to show this by statistical tests, you need to do power analysis first, determine which difference would be of interest and compute the required sample size to detect that. G*Power is a tool often used for that purpose.

You would want to proove that the new method does not systematically measure larger or smaller values, that the variance of reapeated measurements is not bigger than that of the old method and that the variance is not bigger with small or large results.

That said, there is a graphical method you should consider to implement in your report, in which you can visually examine all these questions: the Mean-Difference-Plot or Bland-Altman-Plot. See https://mran.microsoft.com/web/packages/BlandAltmanLeh/vignettes/Intro.html for an introduction and free software that implements it.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I think the Bland-Altman-Plot won't work. The physical parameter of the tube itself is constant. It does not change as it is a property of the tube. The Bland-Altman-Plot is for a property that can vary. Am I right? $\endgroup$
    – yolo123
    Jul 1, 2016 at 15:01
  • $\begingroup$ What should I do if Bland-Altman does not work? $\endgroup$
    – yolo123
    Jul 1, 2016 at 15:07
  • $\begingroup$ Well, in the rare event that you have to measure the same thing again and again with no relevant change (why should one do that?), the whole question of heteroscedasticity looses it's point. $\endgroup$
    – Bernhard
    Jul 1, 2016 at 19:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.