What is the difference between "random noise" and "statistical noise"?

Question

According to English wikipedia, there are Statistical noise and Random noise (e.g., white noise). However, I've never seen definitions of what statistical noise is. So, what is the difference if any? Does statistics have it's own noise or is it just an informal term, depending on the usage context?

Answer 1

In physical sciences, the statistical noise (error) is a random noise.

Here's how the statistical errors are defined in this deck by a DESY physicist:

Statistical errors are due to statistical uncertainties:

• arise from stochastic fluctuations (random quantum processes),
• are uncorrelated with previous measurements,
• follow well-developed theory;
• examples are finite statistics (Poisson distribution) and measurement resolution.

Here's how random errors are defined in IUPAC Gold Book:

Result of a measurement minus the mean that would result from an infinite number of measurements of the same measurand carried out under repeatability conditions.

Answer 2

Noise (n) is the component of a signal (s) that is not information (i).

$ s = i + n $

Any signal has one or more sources. Typically, each of these sources can theoretically be approximated by some parametric model. So, say there are $N$ sources, where $N$ is some arbitrary number. The signal is a combination of these $N$ sources, each of which have there individual set of parameters ($\bar{\phi}$). So,

$ s = \sum_{i=0}^{N} f_{i}(\bar{\phi_{i}})$

$ \mbox{ } = f_{0}(\bar{\phi_{0}}) + \sum_{i=1}^{N}f_{i}(\bar{\phi_{i}}) $

Lets say, that first component is the information we are interested in and the rest is noise. So,

$i = f_{0}(\bar{\phi_{0}})$

and

$n = \sum_{i=1}^{N}f_{i}(\bar{\phi_{i}})$,

which we may choose not to bother modeling. Therefore, we have

$s = f_{0}(\bar{\phi_{0}}) + n$

or simply, in its familiar form:

$s = f(\phi^{1}, \ldots, \phi^{m}) + n$

For example, if the information were some Gaussian function, then

$s = \mathcal{N}_{info}(\mu, \sigma) + n$,

where $f = \mathcal{N}, \phi^{1}=\mu, \phi^{2}=\sigma$

The noise term here is, not modeled, but simply considered to be random. The noise(n) is the error in fit of the signal(s) and the statistical model $\mathcal{N}_{info}(\mu, \sigma)$ , so it is also called statistical noise. The term 'statistical noise' seems to originate from non-scientific communities applying data analytics.

$n_{statistical} = s - \mathcal{N}_{info}(\mu, \sigma)$

--

Instead of leaving noise as a purely 'random' component. We may decide to model it by some parametric function. Now,

$s = f_{0}(\bar{\phi_{0}}) + f_{1}(\bar{\phi_{1}})$

where noise term is $n = f_{1}(\bar{\phi_{1}})$.

The signal could be:

$s = f_{0}(\bar{\phi_{0}}) + \mathcal{N}_{noise}(\mu, \sigma)$

or

$s = i + \mathcal{N}_{noise}(\mu, \sigma)$

So far, I could not find a descriptive term / name for this noise. But it is an important topic of research. For example, this paper on Gaussian noise estimation in images. I located a scatter of papers on 'parametric noise' from various fields of engineering and physics, but I haven't found an authoritative definition of it anywhere.

--

Summary: It seems that 'statistical noise' is essentially the error in estimation of signal/data by a parametric model. Indeed, there are two sources to this estimation error: the 'noise' in the data and the model itself. For example, if a linear model was utilized to fit data points from quadratic polynomial. There will always be an error in estimation, the statistical noise here includes both the 'random noise' and the modeling error.

Answer 3

These are very loose terms, and from different disciplines.

According to Wikipedia - which does not make it so - statistical noise is what can't be explained by a statistical model. Suppose you try to predict household natural gas consumption as a function of temperature for houses with natural gas heating. As described in Wikipedia, if you build that model witg regression and get an R-squared of 92%, the 'statistical noise' is the 8% you can't explain with your model.

Random noise, as you are using it, is a slightly more precise and meaningful idea. Suppose you are measuring the voltage from the signal from a satellite receiver. That signal may be a pure sine wave when sent from the satellite, but 'noise' makes it look fuzzier on receipt. That noise can come from anywhere: atmospheric things, interference from a microwave oven, etc. That noise can have a 'shape'. White noise is equal energy per unit of frequency; pink noise is equal energy per octave; etc.

I have never heard the term statistical noise before. But I think you are right: it is an informal term.

It seems to be this idea...suppose you try to measure the average height of people coming off planes from Japan. The average height might be real and constant. But, every flight will have a slightly different number. This is not 'noise' in the way one thinks of it in information theory.