Correlation is present overall, but not in subsets of data

Question

I'm observing a strong relationship is a set of data representing measurements taken by two gauges. As expected, measurements are generally consistent with each other.

However, when my data is stratified by weight range, the strong correlation in measurement no longer exists. The implication is that if I, for argument's sake, look only at measurements taken for lower values (i.e. things that weigh very little), there is not a strong correlation (this is expected, given the equipment). For mid-range values, the gauges mostly agree.

The observation discussed is consistent with what I'd expect. I was wonder if there is a particular term or effect name used in literature to describe a scenario such as this?

Answer 1

It's tough to give you a definitive answer, so I'll lay out three strong possibilities.

1) Simpson's Paradox

This is the first term that comes to mind. This refers to a situation where a trend appears in different subsets, but disappears or reverses when these groups are combined.

The term is also used to refer to an overall trend that disappears or reversed once it is broken apart by some third variable.

One of the most famous cases of Simpson's paradox was the illusion of gender bias in the admission to UC Berkeley. Once the gender data were broken apart by department, the trend reversed.

I think that's pretty close to what you're observing. You're finding a correlation that disappears when you subset by a group.

However, there are a few other ways to explain what you're seeing.

2) Mediator effect

This term refers to a third variable that explains the relationship between an independent variable and a dependent variable. In this case, your mediator would be "weight."

Note that in order for weight to be a proper mediator, you'd have to establish an actual relationship between weight and your other two variables. Check out the Sobel test for more info.

You could be describing a case of full mediation, but only if you think weight fully explains the relationship between X and Y.

However, because I don't exactly how these variables are related, you should keep reading! There's more!

3) Moderator effect

This term refers to when a third variable effects the strength of the relationship between two variables. The third variable is referred to as the moderator variable or simply the moderator.

You could definitely argue that weight category moderated the relationship between X and Y.

My guess: weight was acting as a moderator, but I'll leave any further investigations up to you!

Answer 2

It would be interesting to see an scatterplot of your data, but with physical devices of measure I would go for another explanation: Your measures are the sum of the true value to be measured plus an error term. True values are strongly correlated between the gauges, but errors are independent. If variance of errors is constant and doesn't depend on the measure, for moderate to heavy weights true measure will dominate your outcome and therefore measures from both gauges will be highly correlated, but for small weights error term dominates and therefore both gauges are uncorrelated.

Anyway, any statistical explanation you choose should match a physical explanation of what is happening in your system.