0
$\begingroup$

(Hope this is the right forum, otherwise please bear with me)

This is not a homework assignment but just some of the stuff you come across and stop to wonder.. Either I forgot how to do it or my old school books forgot to teach me.

Say I have a scale with the specs:

  • 10 kg max load
  • 0.005 kg resolution

What would be the significant figures in that reading?

I assume it would be 3 (#.## kg) even though the display might would show #.##0 or #.##5 kg.


EDIT

Thinking out loud example

enter image description here

First I was thinking that rounding should be conducted.

But I see that not rounding almost follows the meaning of sign. figures which means that a value of 1.44 kg could come from 1.435 kg or 1.444 kg - it's just shifted 0.001 kg due to computer resolution rounding (up).

$\endgroup$
7
  • $\begingroup$ en.wikipedia.org/wiki/Significant_figures $\endgroup$ Commented Feb 19, 2016 at 9:25
  • $\begingroup$ scalenet.com/applications/glossary.html $\endgroup$ Commented Feb 19, 2016 at 9:28
  • $\begingroup$ Hi @Scortchi, I didn't ask for how you look up things on wikipedia. But thank you for taking the time to read my question. Comments are used to clarify the question being asked. $\endgroup$
    – Norfeldt
    Commented Feb 19, 2016 at 9:30
  • $\begingroup$ Yes - I was hoping you'd clarify how the answer would differ from what you'd get by looking in Wikipedia. Not quite sure how the first part of the question on significant figures relates to the 2nd part (your edit) on resolution. Or how you arrive at 1.44 kg $\pm$ 0.009 kg. $\endgroup$ Commented Feb 19, 2016 at 9:43
  • 1
    $\begingroup$ Any digit could change when you add or subtract the resolution: consider a reading of 4.995. Reporting to 1 decimal place would ensure that the reported value didn't differ from the similarly rounded upper & lower limits got by adding & subtracting the resolution; but might be taken to imply a worse resolution than you in fact have. [Edit: in fact that wouldn't ensure it: consider a reading of 1.445] $\endgroup$ Commented Feb 19, 2016 at 12:02

1 Answer 1

1
$\begingroup$

If the resolution of your scale is given in absolute numbers $\delta$ and the true value you want to measure is $x_{true}$ then the measured value $x_m$ is within the range $x_m \in [x_{true}-\delta;x_{true}+\delta$].

Hence, you should give $$x_{true} = x_m\pm \delta$$ which leads to $1.44$kg $\pm5$g in your case.

$\endgroup$
4
  • 1
    $\begingroup$ (+1) But it'd make sense to report the measured value to the same precision as the uncertainty. After all, if the display read 1.445 you wouldn't report it as 1.45 kg $\pm$ 5 g. So 1.440 kg $\pm$ 5g. $\endgroup$ Commented Feb 19, 2016 at 10:26
  • $\begingroup$ Harald and @Scortchi please see my update. I might be mixing things so perhaps I can't talk about sign. figures in this case. $\endgroup$
    – Norfeldt
    Commented Feb 19, 2016 at 13:06
  • $\begingroup$ @Norfeldt. What exactly is the question though? You seem to be trying to find a way of rounding the measurement from which an uncertainty interval could be inferred equal to that calculated from the given precision. In general you won't be able to do that. $\endgroup$ Commented Feb 19, 2016 at 13:51
  • $\begingroup$ Sorry if my question is a brain fart.. $\endgroup$
    – Norfeldt
    Commented Feb 19, 2016 at 13:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.