White Noise in Statistics

Question

I often see the term white noise appearing when reading about different statistical models. I must however admit, that I am not completely sure what this means. It is usually abbreviated as $WN(0,σ^2)$. Does that mean it's normally distributed or could it follow any distribution?

Answer 1

TL;DR

The answer is NO, it doesn't have to be normal; YES, it can be other distributions.

Colors of the noise

Let's talk about colors of the noise.

1. The noise that an infant makes during the air travel is not white. It has color.
2. The noise that an airplane engine makes is also not white, but it's not as colored as the kid's noise. It's whiter.
3. The noise that an ocean or a forest produces is almost white.

If you use noise cancelling head phones, you know that #1 is impossible to cancel. It'll pierce through any head phone with ease. #2 will be cancelled very well.

As to #3, why would you cancel it?

Origin of a term "color"

What's the distinction between these three noises? It comes from spectral analysis. As you know from high school years you can send the white light through a prism and it will split the light into all different colors. That's what we call white: all colors in approximately the same proportion. No color dominates.

image is from https://www.haikudeck.com/waves-and-light-vocabulary-uncategorized-presentation-w5bmS88NC9

The color is the light of a certain frequency, or you could say electromagnetic waves of certain wavelength like shown below. The red color has low frequency relative to the blue, equivalently the red color has longer wavelength of almost 800nm compared to the blue wavelength of 450nm.

image is from here: https://hubpages.com/education/Teachers-Guide-for-Radiation-beyond-Visible-Spectrum

Spectral Analysis

If you take noise, whether acoustic, radio or other, and send it through the spectral analysis tool such as FFT, you get it spectral decomposition. You'll see how much of each frequency is in the noise, like shown in the next picture from Wikipedia. It's clear that this is not white noise: it has clear peaks at 50Hz, 40Hz etc.

If a narrow frequency band sticks out, then it's called colored, as in not white. So, white noise is just like white light, it has a wide range of frequencies in approximately same proportion like shown in the next figure from this site. The top chart shows the recording of the amplitude, and the bottom shows the spectral decomposition. No frequency sticks out. So the noise is white.

Perfect sine

Now, why does the sequence of independent identically distributed random numbers(iid) generates the white noise? Let's think of what makes a signal colored. It's the waves of certain frequency sticking out from others. They dominate the spectrum. Consider a perfect sign wave: $\sin(2\pi t)$. Let's see what is the covariance between any two points $\phi=1/2$ seconds apart: $$E[\sin(2\pi t) \times \sin(2\pi (t+1/2)]=-E[\sin^2 (2\pi t)]=-\frac 1 2$$

So, in the presence of the sine wave we'll get autocorrelation in the time series: all oservations half a second apart will be perfectly negatively correlated! Now, saying that our data is i.i.d. implies that there is no any autocorrelation whatsoever. This means that there are no waves in the signal. The spectrum of the noise is flat.

Imperfect Example

Here's an example I created on my computer. I first recorded my tuning fork, then I recorded the noise from computer's fans. Then I ran the following MATLAB code to analyze the spectra:

[y,Fs] = audioread(filew);

data = y(1000:5000,1);
plot(data)
figure
periodogram(data,[],[],Fs);
[pxx,f] = periodogram(data,[],[],Fs);
 [pm,i]=max(pxx);
 f(i)

Here's the signal and the spectrum of the tuning fork. As expected it has a peak at around 440Hz. The tuning fork must produce a nearly ideal sine wave signal, like in my theoretical example earlier.

Next I did the same to the noise. As expected no frequency is sticking out. Obviously this is not the white noise, but it gets quite close to it. I think that there must be very high pitched frequency, it bother me a little bit. I need to change the fan soon. However, I don't see it in the spectrum. Maybe because my microphone is beyond crappy, or the sampling frequency is not high enough.

Distribution doesn't matter

The important part is that in the random sequence the numbers are not autocorrelated (or even stronger, independent). The exact distribution is not important. It could be Gaussian or gamma, but as long as the numbers do not correlate in the sequence the noise will be white.

Answer 2

White noise simply means that the sequence of samples are uncorrelated with zero mean and finite variance. There is no restriction on the distribution from which the samples are drawn. Now if the samples happen to be drawn from a Normal distribution, you have a special type of white noise called Gaussian white noise.