According to English wikipedia, there are Statistical noise and Random noise (e.g., white noise). However, I've never seen definitions of what statistical noise is. So, what is the difference if any? Does statistics have it's own noise or is it just an informal term, depending on the usage context?

  • 1
    I might recommend deleting the statistical noise page. Ben Goldacre's commentary does not use statistical noise as a rigorous concept; rather he refers to statistical results which are noise. The frequent, short intervals with which unemployment is monitored is sure to produce spurious findings; that he is loosely referring to as "statistical noise". It is a bit misleading. The FAQs.org page does not have an author, and while is more academically oriented, none of the cited sources use the term "statistical noise". With only two cited sources for "statistical noise", it is highly suspect. – AdamO Oct 16 '17 at 16:43
  • This is exactly what I initiated in Russian wikipedia. I've never read about "Statistical noise" in some strict sense, so always assumed it's some kind of a laymen's term to justify scientific prediction errors in mass media or point to uncertainty in general. With this question I am trying to find counterexamples (there are books with "statistical noise in the title, btw), and 2 answers out of 3, and even the presence of contradicting answers adds me confidence that it's not my personal ignorance. – Roman Susi Oct 17 '17 at 6:55
up vote 1 down vote accepted

In physical sciences the statistical noise (error) is a random noise.

Here's how the statistical errors are defined in this deck by a DESY physicist:

Statistical errors are due to statistical uncertainties:
• arise from stochastic fluctuations (random quantum processes),
• are uncorrelated with previous measurements,
• follow well-developed theory;
• examples are finite statistics (Poisson distribution) and measurement resolution.

Here's how random errors are defined in IUPAC Gold Book:

Result of a measurement minus the mean that would result from an infinite number of measurements of the same measurand carried out under repeatability conditions.

These are very loose terms, and from different disciplines.

According to Wikipedia - which does not make it so - statistical noise is what can't be explained by a statistical model. Suppose you try to predict household natural gas consumption as a function of temperature for houses with natural gas heating. As described in Wikipedia, if you build that model witg regression and get an R-squared of 92%, the 'statistical noise' is the 8% you can't explain with your model.

Random noise, as you are using it, is a slightly more precise and meaningful idea. Suppose you are measuring the voltage from the signal from a satellite receiver. That signal may be a pure sine wave when sent from the satellite, but 'noise' makes it look fuzzier on receipt. That noise can come from anywhere: atmospheric things, interference from a microwave oven, etc. That noise can have a 'shape'. White noise is equal energy per unit of frequency; pink noise is equal energy per octave; etc.

I have never heard the term statistical noise before. But I think you are right: it is an informal term.

It seems to be this idea...suppose you try to measure the average height of people coming off planes from Japan. The average height might be real and constant. But, every flight will have a slightly different number. This is not 'noise' in the way one thinks of it in information theory.

Noise (n) is the component of a signal (s) that is not information (i).

$ s = i + n $

Any signal has one or more sources. Typically, each of these sources can theoretically be approximated by some parametric model. So, say there are $N$ sources, where $N$ is some arbitrary number. The signal is a combination of these $N$ sources, each of which have there individual set of parameters ($\bar{\phi}$). So,

$ s = \sum_{i=0}^{N} f_{i}(\bar{\phi_{i}})$

$ \mbox{ } = f_{0}(\bar{\phi_{0}}) + \sum_{i=1}^{N}f_{i}(\bar{\phi_{i}}) $

Lets say, that first component is the information we are interested in and the rest is noise. So,

$i = f_{0}(\bar{\phi_{0}})$

and

$n = \sum_{i=1}^{N}f_{i}(\bar{\phi_{i}})$,

which we may choose not to bother modeling. Therefore, we have

$s = f_{0}(\bar{\phi_{0}}) + n$

or simply, in its familiar form:

$s = f(\phi^{1}, \ldots, \phi^{m}) + n$

For example, if the information were some Gaussian function, then

$s = \mathcal{N}_{info}(\mu, \sigma) + n$,

where $f = \mathcal{N}, \phi^{1}=\mu, \phi^{2}=\sigma$

The noise term here is, not modeled, but simply considered to be random. The noise(n) is the error in fit of the signal(s) and the statistical model $\mathcal{N}_{info}(\mu, \sigma)$ , so it is also called statistical noise. The term 'statistical noise' seems to originate from non-scientific communities applying data analytics.

$n_{statistical} = s - \mathcal{N}_{info}(\mu, \sigma)$

--

Instead of leaving noise as a purely 'random' component. We may decide to model it by some parametric function. Now,

$s = f_{0}(\bar{\phi_{0}}) + f_{1}(\bar{\phi_{1}})$

where noise term is $n = f_{1}(\bar{\phi_{1}})$.

The signal could be:

$s = f_{0}(\bar{\phi_{0}}) + \mathcal{N}_{noise}(\mu, \sigma)$

or

$s = i + \mathcal{N}_{noise}(\mu, \sigma)$

So far, I could not find a descriptive term / name for this noise. But it is an important topic of research. For example, this paper on Gaussian noise estimation in images. I located a scatter of papers on 'parametric noise' from various fields of engineering and physics, but I haven't found an authoritative definition of it anywhere.

--

Summary: It seems that 'statistical noise' is essentially the error in estimation of signal/data by a parametric model. Indeed, there are two sources to this estimation error: the 'noise' in the data and the model itself. For example, if a linear model was utilized to fit data points from quadratic polynomial. There will always be an error in estimation, the statistical noise here includes both the 'random noise' and the modeling error.

  • Nice write up, thanks! Still, in some important algorithms (eg, Kalman filter) we assume noise to have certain properties. It's still random (process), but we make use of it's parameters, which we consider known. So, is it statistical then? Why it is not usually called like that (and called eg gaussian white noise)? – Roman Susi Oct 16 '17 at 5:27
  • Hmmm... Another answer seems to give completely different explanation, that is, if the noise is assumed to have known parameters, it's random noise. – Roman Susi Oct 16 '17 at 6:20
  • It seems your question can be addressed at different levels. (At a higher level folks from Quantum computing and Chaos theory will be happy to provide counter examples to anything I say!). At a simpler level: a process can be deterministic or stochastic. If it is stochastic, then a statistical model can be trained on some observations acquired from this process. The model is always a guess of the underlying stochastic process; the guess may or may not be accurate. – Dynamic Stardust Oct 16 '17 at 16:29

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.