1
$\begingroup$

I have two measurements of something. You can think of them both as their own curve with known error on each point. They look quite similar, and if I calculate the R2 value it comes out to >0.9. But what I want to be able to calculate is a P value comparing these two curves (i.e., what is the probability that the difference i'm looking at is just due to noise?). Now I could easily do a student t-test at each point and come up with a P-value at each point, sure. But is there some way to come up with an over-all P value that uses all the points and not just one? Thanks very much for any help.

$\endgroup$
  • $\begingroup$ This question appears to be off-topic because it is about statistics and thus belongs on Cross Validated. $\endgroup$ – Thomas Oct 23 '13 at 8:39
  • $\begingroup$ It sounds like you have a pair of calibration curves generated by two different methods. If that is correct then you might want to look at the fixed and proportional bias in the measurements using the least products regression method detailed in this paper: molecularlab.it/public/data/TMax/… $\endgroup$ – Michael Lew Feb 3 '14 at 4:52
1
$\begingroup$

Define error as the difference between real and observed value.

Suppose your errors at different points are independent and normally distributed (i.e. no systematic error).

If you know the standard deviation of each error, you also know the standard deviation of difference between the value at two curves. Now you have the vector of differences, with known standard deviation of each difference. Divide each difference by its standard deviation, and you have vector of normalized values with standard deviation of 1 each. The null hypothesis is that they are distributed as N(0,1). Test it with any normality test.

$\endgroup$
  • $\begingroup$ Can you explain how this null hypothesis arises from what the OP said in the question? $\endgroup$ – Glen_b -Reinstate Monica Nov 4 '13 at 3:34
  • $\begingroup$ He writes "I have two measurements of something. You can think of them both as their own curve with known error on each point." Therefore, he has two curves. Later he writes "what is the probability that the difference i'm looking at is just due to noise?" If the difference between observed values is due to noise, then the difference between actual values is zero, and vice versa. $\endgroup$ – user31264 Nov 4 '13 at 3:44
  • 2
    $\begingroup$ How does either statement lead to a hypothesis of normality? It's not clear to me that you're necessarily wrong (perhaps given certain assumptions that the OP might feel happy to apply if they were explicit enough to be clearly considered), but the steps in reasoning from the question to your answer should be clear, at least. $\endgroup$ – Glen_b -Reinstate Monica Nov 4 '13 at 3:46
  • $\begingroup$ To me, it seems that the connection is obvious, and frankly I am surprised that I should explain it. OK. Let actual values for curve 1 be $y_1$, ..., $y_N$, and for curve 2 be $z_1$, ... $z_N$ . Let observed values be $\hat y_1$ , ... , $\hat y_N$ for curve 1 and $\hat z_1$ , ... , $\hat z_N$ for curve 2. Let $d_i=y_i-z_i$, $\hat d_i=\hat y_i-\hat z_i$. The author wants to test the hypothesis that $d_i=0$ for all i. But $\hat d_i=d_i+(\hat y_i-y_i)-(\hat z_i-z_i)$ $\endgroup$ – user31264 Nov 4 '13 at 4:36
  • 1
    $\begingroup$ "I write this in the second sentence of my comment" -- yes, you did. I missed that, I am sorry. Thanks for laying out your argument, though I think it would be good to put it into your answer (and you're free to disagree). Is there some reason that you thought that the errors would be something like normal? $\endgroup$ – Glen_b -Reinstate Monica Nov 4 '13 at 11:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.