Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
What's the source of statistical uncertainties in measurements?
I understand that random noise can be a source of the uncertainty. Also sometimes the signal itself is randomly determined. In that case, the random nature of the true value is the source of the statistical uncertainty.
But if noise is zero, and the signal is fixed, then is there a chance that your measurement data are not identical but have some variation?
Broadly speaking, uncertainty happens when multiple input values, or causes to be inferred, are associated with the same outcome or result. In other words, for the same value of y, there are several values of x that might have caused it. Thus, when we observe y, we cannot (always) infer the value of x with perfect confidence - rather, a number of possible values of x are consistent with the observed y.
This can happen for a number of reasons, but they fall into these main categories: deterministic causes, random causes, and imperfect knowledge. A deterministic cause of uncertainty is if the (deterministic) function that maps x to y isn't monotonic. For instance, the function may plateau at some point, such that further increasing (or decreasing) x no longer changes y. Or, the function may go up and down and have one or more peaks and troughs, such that on either side of these peaks and troughs, and more generally throughout the function, we get identical values of y for different values of x. For example, if x maps to y via a Gaussian function, then for each value of y within the function's range, there are two values of x, at equal distances from the center of the Gaussian, that produce that same value of y (with just one exception at the center of the Gaussian). Thus, for (nearly) every value of y that you might observe, your uncertainty includes two possible values of x that might have produced that outcome.
Random causes of uncertainty include anything that you cannot explain deterministically in your model. True randomness is often referred to as "noise", but it's often hard to define what is really random. For instance, when comparing different groups of people on their body height, you might assume their heights vary randomly around some population mean. But is that variation really random? Probably it is more accurate to say that it is the sum of many causes that are impossible for us to measure or know about (or at least prohibitively difficult). This is how it typically goes. As far as we know, the universe is only really random at the quantum level, and most of the things we call random in practice are actually just chaotic or complicated. But that's fine, because we can still treat them as random for statistical purposes and apply the rules of probability to them.
So, the third cause of uncertainty is actually often the largest one, and it is imperfect knowledge. Only, in statistical models, such uncertainty usually gets absorbed into a random term, for the reason I stated above. Imperfect knowledge can range from "unknown unknowns", such as the many causes of height variation mentioned above, to known unknowns, which is when e.g. you know the model that generated your data, but you don't know some of the model parameters, or you know them only up to some margin of error.
In conclusion: yes, it is possible to have uncertainty without noise. First, it is possible if there is ambiguity in the deterministic mapping from causes to outcomes, via a non-monotonic function. Second, it is also possible when this mapping is deterministic and unambiguous, but we just don't know the function precisely. Finally, it is possible if you take a very narrow definition of "noise", to mean only truly random variation, but I assume that isn't what you meant.
It sounds like you got confused in several places.
What's the source of statistical uncertainties in measurements?
If you ask about measurement, then every aspect of the measurement process can influence the uncertainty if it makes the measurement imprecise. This could be on how do you gather your sample (biased sample), how do you measure the outcomes (imprecise instruments, technical or human errors, etc.), or even how do you record (researcher has unreadable handwriting) and store the results (numerical precision errors, wrong formula in Excel). This can even be a microwave oven in space observatory.
I understand that random noise can be a source of the uncertainty.
What exactly is the random noise? Did you ever see one? In real world no such a thing as "random noise" exist. What statisticians call as "random noise", is basically "everything else that we did not account for in our research". You can think of statistical models as modelling some outcome $Y$ as a function $f$ of some features $X$ with factors that we did not account for $\varepsilon$
$$ Y = f(X) + \varepsilon $$
This does not say that $\varepsilon$ is something that is random and unpredictable. For example, if you study human mortality and build a model where probability of death is a function of time alone, this does not mean that there is not other factors that lead to death, nor that the other factors are purely random and unpredictable.
Also sometimes the signal itself is randomly determined. In that case, the random nature of the true value is the source of the statistical uncertainty.
What signal is "randomly determined"? For example, atmospheric noise is a complicated, but deterministic process that is governed by the laws of physics, it is simply too complicated for us to predict, hence we call it "random". Same with coin flips. In statistics we define models in terms of probability distributions, hence, we think of the studied outcomes as of random variables. So when using statistical model, by definition you would assume the phenomenon to be "randomly determined".
But if noise is zero, and the signal is fixed, then is there a chance that your measurement data are not identical but have some variation?
If you are talking about purely deterministic process that happens in controlled environment, so that there is no source of interference whatsoever that would distort the "signal", then there is no "noise". For example, if you have a robot throwing a ball into the basket, in conditions where there is no wind, or other interference, then the robot would achieve perfect accuracy in this task.
On another hand, if there are any other external factors that you can't control, in statistics we would think of them as of "random" factors and use tools derived from probability theory to account for them.
I will provide an answer from a very particular content area: classical testing theory.
In classical testing theory, the assumption is that if you give a student a test, they will earn a particular score. This score ($S$) is a sum of the true score ($T$) and measurement error ($E$): $$S = T + E$$ In this context, we can imagine $T$ to be not just person-dependent (how much a particular person knows about a given topic), but it also can be test-dependent (e.g., novice students will do poorer on tests with more advanced questions). Thus, in CTT we make an attempt to capture the true score from the observed score.
So, ¿where does the error come from? Well, this is attributed to sources outside of the test or the person. The common examples provided in many CTT texts are (1) the student might do slightly worse because of not feeling well the day of the test or (2) the student might accidentally answer the correct question by slightly misreading the question (e.g., missing the word "not" in the item).
These models allow the person constructing the test to model both the stochastic elements of how the student would perform (e.g., there is variability in $T$ based on the different tests or person-level ability), but it also allows us to model the stochastic element of the random error that is from sources distinct from the test or the person's ability.
Happy to clarify in follow-up comments as necessary.
