# What is the relationship between effect size and noise?

I understand that noise in measurement can affect the size of an observed effect. That noise can result in larger measurement error, which can reduce the size of the observed effect. However, I do not understand the mechanism. How can greater measurement error reduce the size of the observed effect? Can someone please explain this in a simple way that a non-statistician can understand?

• A suggestion: What do you have in mind when you say "size of an effect" ? Commented Feb 8, 2023 at 23:36
• Estimated effect size can be both embiggened and shrunk by noise. This is one of the critical distinctions between bias and noise (aka error, unreliability, etc.): bias prefers some direction (e.g., always making an estimate's magnitude seem smaller than it is), while noise has no preference. Commented Feb 8, 2023 at 23:52
• That sounds perfectly cromulent. Commented Feb 9, 2023 at 0:03

An apt metaphor might be something like listening to a song on a car radio. You're driving around jamming to some great tunes (the signal) when suddenly there's a huge burst of static (the noise). You can still make out the song, but it's pretty indistinct and hard to tell it from the static. So even though the true effect size has not changed (it's the same song) the observed effect size (what you can make out) is a lot smaller because of the measurement error (the static).

#### Some Clarification

HOW does/can greater measurement error reduce the size of the observed effect?

Error doesn't reduce the size of an observed effect. The effect will still remain the same with error included. What error does is artificially increase the size of the effect, so if you removed the error, then you would have a reduction in the effect. That is an important distinction because it explains why you don't want error...you can easily make a false attribution about an actual effect size being fairly large when in fact it is small and biased by a large amount of error.

#### Analogy

Suppose we have a newly installed seismogram below, which attempts to record when the ground shakes.

The P Wave here is the first to be registered by a seismograph during an earthquake, as it travels more quickly, followed by an S Wave, which travels slower. This wave is fairly abnormal, so we think an earthquake has occurred nearby.

However, we later learn that the floor beneath the seismogram has had drilling from renovations during construction. Because of the heavy drilling, it causes detections in the seismograph that would otherwise be present. Yet we know from reports that there was in fact an earthquake. But we don't know the actual magnitude of the effect. So we would need to partial out this error in measurement somehow to understand how bad the earthquake actually was.

Using some detective work, we find other seismograms on average measured much lower P and S waves and overlay them on our earlier graph, with the red line being the waves recorded by other seismograms:

We can see that there is a considerable amount of error here in the original black lines. As the waves are not as strong as we had anticipated, much of the black line could be attributed to the effect of drilling from reconstruction and much less attributed to an actual earthquake. As stated earlier, this is why the effect will still remains high, but removing the error decreases the actual effect because there is less confounding influence on what we recorded.

• Your answer has confused me and it seems to contradict @DavidB's answer above (which made sense to me) specifically when he says "You can still make out the song, but it's pretty indistinct and hard to tell it from the static. So even though the true effect size has not changed (it's the same song) the observed effect size (what you can make out) is a lot smaller because of the measurement error (the static)". Commented Feb 9, 2023 at 5:36
• My statement doesn't contradict his. The static in his scenario is the same as the drilling in my scenario. Both of the effects we are interested in are being obfuscated by random factors beyond our control (static/drilling). When you remove these factors, the "noise" ends up being most of the effect if there is a lot of error, hence the reduction in effect if you remove the "noise." The bolded statement you just made is the correct understanding. Commented Feb 9, 2023 at 6:39
• Thanks for clarifying - I now understand. Commented Feb 9, 2023 at 20:04
• If our answers were helpful, free to accept either Davids answer or my answer by clicking the checkmark next to it. Commented Feb 9, 2023 at 23:48