Determining the nature of noise

Question

I have a general question. What kind of noise is additive, and what about multiplicative noise? How to determine the nature of noise?

Thanks a lot for your help.

Answer 1

One thing that you must recognise is that the term "noise" is relative not absolute. Thinking about an obvious example (not anything to do with statistics per se), imagine you are at a night club. What is "noise" in here?

If you trying to have a conversation with someone, then the music is "noise", and so are the other conversations going on inside the night club. The "signal" is the dialogue of the converstion.

If you are dancing, then the music is no longer "noise", but it becomes the "signal" to which you react to. The conversation has changed from "signal" into "noise" merely by a change in your state of mind!

Statistics works in exactly the same way (you could in theory develop a statistical model which describes both these "noise" processes).

In a regression setting, take the simple linear case with 1 covariate, X, and 1 dependent variable Y. What you are effectively saying here is that you want to extract the linear component of X that is related to Y. The general conditions for "additive noise", so that you have:

$$Y_{i}=a+b X_{i} + n_{i}$$

Is "small noise", or more precisely in a mathematical sense, the conditions for Taylor Series linearisation are good enough for your purposes. To show this in the multiplicative case, suppose the actual distribution is:

$$Y_{i}=(a+b X_{i})n_{i}^{(T)}$$

Which we can consider as a function of $n_{i}^{(T)}$, a taylor series expansion about the value 1 gives:

$$(a+b X_{i})n_{i}^{(T)}=a+b X_{i}+(a+b X_{i})(n_{i}^{(T)}-1)$$

If the noise is "small" then it should not differ much from 1, and so the second term will be much small than the first, when the noise is "small" compared to the "signal", which is given by the regression line. So we can make the approximation

$$(a+b X_{i})n_{i}^{(T)} \approx a+b X_{i}+n_{i}^{(A)}$$

Where the approximate noise ignores the dependence on the actual regression line. This dependence will only matter when the noise is large, compared to the slope. If the slope does not vary appreciably over the range of the model, then the "fanning" of the true noise will be indistinguishable from independent noise. This also applies for the general case, for any function g satisfying $g(1)=1$:

$$(a+bX)g(n_{i}^{(T)}) \approx a+b X_{i}+(g(n_{i}^{(T)})-1)(a+bX)g^{(1)}(n_{i}^{(T)}) $$ $$\approx a+b X_{i}+n_{i}^{(A)}$$

Where

$$g^{(1)}(x)=\frac{\partial g(x)}{\partial x}$$

But note that this approximation will only apply in the case of "small noise", or that $g(n_{i}^{(T)}) \approx 1$. This "smallness" make all the details of the particular function g irrelevant for all practical purpose. Going through the laborious calculations using g directly will only matter in the decimal places (estimate is 1.0189, using true g it is 1.0233). The more the function g departs from 1, the further up the decimal values will be affected. This is why "small noise" is required

Answer 2

1. Do you know where does the noise comes from? Before doing any statistical test, you think about the origin of the noise you want to remove. Additive noise is independent from the level of the signal, whereas multiplicative noise is proportional to the level of the signal.

2. If you are not able to know, or if you want to check, try for instance to plot the sd of the noise as a function of the level of the signal.