# Two clocks problem and measurements

If I have one clock, I know what the time is.

If I have two clocks, I now am unsure.

Although simplified, the problem is related to a more complex system I am testing.

If I have the following measurements from two systems:

Attribute   System 1   System 2
A           0          0
B           10         100
C           90         100
D           100        110
E           900        1000


then I'd like a way to determine some value of error between the systems (assuming I cannot distinguish which system, if any, is correct)

So

• attribute A values are considered no-error or both systems correct
• attribute B values are considered a larger error than A, C, D and E values
• attributes C, D and E values are have the same error (or similar error)

Also, assume the attributes are independent.

My understanding of statistics is low, and I can only recall some things about relative error, but iirc I require knowing whether one system is correct to determine the relative error of the other system.

How does one go about resolving this issue?

-
I"m a little confused by the wording. You say, "determine some value of error between the systems". Do you mean "difference" between the two systems, or "error" of one of the systems? Or something different, like trying to figure out what "true" value is and thus how each differs from this true value? –  Wayne Jan 21 '12 at 16:32
@Wayne the last thing you said, although knowing the true value is not so important (it helps though), just how much error each system is off by. –  jamiroquai Jan 23 '12 at 1:35
Ah, pretty much impossible, as far as I can tell. I'll +1 rolando2's answer. At best, if you have many systems, you could average their values for each attribute and then say how far off each actual machine is from the average. –  Wayne Jan 23 '12 at 14:15
One more question: are you wanting to compare N products at one time -- say for a purchase decision -- or products over time? In the latter case, there's a field called Statistical Process Control, which has charts that let you see if your process seems to be shifting over time (i.e. a part of your manufacturing process has broken). –  Wayne Jan 23 '12 at 14:52

Sorry to disappoint you, but I don't see any definitive statistical solution to your problem. There's no glorious trick that can determine where truth lies and which of your systems is closer to it. So what can you do? Obtain multiple measurements from each system and average them... Average the measurements you have and treat those averages as best estimates... Devise additional systems, making your problem potentially better and potentially worse.

-
I figured there wouldn't be some magical silver bullet. –  jamiroquai Jan 23 '12 at 1:36
@rolando2: agreed that if the OP has many systems, average them and then state how far each machine is from the average. I'd recommend also calculating the standard deviation for each attribute, so that readers can determine if System 32 is really out of line on Attribute A, or not. –  Wayne Jan 23 '12 at 14:17

A few things that might help (a bit).

1. You could calculate the coefficient of variation for each attribute. This would allow you to at least quantify what you are saying when you say (for example) "the error in Attribute B is higher than the error of the other attributes".

2. You could use analysis of variance to quantify how much of the variation is between attributes, and how much is between your two measuring systems.

3. You could model the system2 measurements on the system1 measurements and hence get a quantification of the relationship between the two (crudely speaking, it would come up with something like "system 2 on average returns 10% higher estimates than system 1").

None of these completely fix your problem but they give you some different angles to look at it.

-
Good ideas for a tough situation. –  rolando2 Jan 22 '12 at 0:27
Good options (I like having options). I'd have to look into these in a little more detail and see if they help. –  jamiroquai Jan 23 '12 at 1:37