What is the relationship between effect size and noise?

Question

I understand that noise in measurement can affect the size of an observed effect. That noise can result in larger measurement error, which can reduce the size of the observed effect. However, I do not understand the mechanism. How can greater measurement error reduce the size of the observed effect? Can someone please explain this in a simple way that a non-statistician can understand?

Answer 1

An apt metaphor might be something like listening to a song on a car radio. You're driving around jamming to some great tunes (the signal) when suddenly there's a huge burst of static (the noise). You can still make out the song, but it's pretty indistinct and hard to tell it from the static. So even though the true effect size has not changed (it's the same song) the observed effect size (what you can make out) is a lot smaller because of the measurement error (the static).

Answer 2

Some Clarification

HOW does/can greater measurement error reduce the size of the observed effect?

Error doesn't reduce the size of an observed effect. The effect will still remain the same with error included. What error does is artificially increase the size of the effect, so if you removed the error, then you would have a reduction in the effect. That is an important distinction because it explains why you don't want error...you can easily make a false attribution about an actual effect size being fairly large when in fact it is small and biased by a large amount of error.

Analogy

Suppose we have a newly installed seismogram below, which attempts to record when the ground shakes.

The P Wave here is the first to be registered by a seismograph during an earthquake, as it travels more quickly, followed by an S Wave, which travels slower. This wave is fairly abnormal, so we think an earthquake has occurred nearby.

However, we later learn that the floor beneath the seismogram has had drilling from renovations during construction. Because of the heavy drilling, it causes detections in the seismograph that would otherwise be present. Yet we know from reports that there was in fact an earthquake. But we don't know the actual magnitude of the effect. So we would need to partial out this error in measurement somehow to understand how bad the earthquake actually was.

Using some detective work, we find other seismograms on average measured much lower P and S waves and overlay them on our earlier graph, with the red line being the waves recorded by other seismograms:

We can see that there is a considerable amount of error here in the original black lines. As the waves are not as strong as we had anticipated, much of the black line could be attributed to the effect of drilling from reconstruction and much less attributed to an actual earthquake. As stated earlier, this is why the effect will still remains high, but removing the error decreases the actual effect because there is less confounding influence on what we recorded.