Influence of one variable to another

Question

What do we mean in statistics when we say 'variable A influences variable B?'

And how do we test detect this influence?

Im not sure of the right tag for this question. What topic in statistics is this point tackled?

Answer 1

Typically in statistics the main thing we investigate is whether one variable changes if another one changes. For example with a two-sample t-test we check whether the mean of a variable changes if we treat the two samples differently.

We do similar things in regression (testing whether one variable goes up or down with another variable) or with chi-square tests (testing whether a distribution changes when a variable changes between groups).

Whether or not we can then conclude on the basis of that answer if variable A has an effect on variable B depends on the set-up of the research.

Answer 2

I would say (from a purely mathematical perspective) that, considering both $A$ and $B$ are random variables, variable $A$ influences variable $B$ if and only if they are not independent, i.e. if $$\mathbb{P}(A=a \text{ and }B=b) \not= \mathbb{P}(A=a)\cdot\mathbb{P}(B=b) \,.$$

The (un)equation basically says that "information about $A$ and $B$ considered together" is not completely determined by "information about $A$ alone" and "information about $B$ alone".

In other words, there is no interaction between $A$ and $B$ which we need to take into consideration when trying to deduce "information about $A$ and $B$ considered together" from "information about $A$ alone" and "information about $B$ alone". I.e., $A$ and $B$ together are "just the sum of their parts" and are "not greater than the sum of their parts".

(Of course we have to use multiplication instead of addition, since probabilities are bounded between $0$ and $1$ inclusive, and multiplication of such numbers is closed under multiplication but not addition.)

The statistics comes in because we do not have perfect knowledge of the universe, so we do not know exactly whether $A$ and $B$ "really are" random variables, and even if we did, we do not have perfect enough knowledge of $A$ and $B$ to say with absolute certainly whether the above formula does or does not hold.

In other words we do not know exactly how to consider them as random variables even if we knew with certainty that they were random variables, so the definition given above doesn't necessarily make sense without a lot of interpretation (since the definition requires us to consider $A$ and $B$ as random variables, which we don't necessarily know how to do "in the best way").

So (according to my rudimentary understanding of statistics) we employ statistical models to try to model $A$ and $B$ as random variables, and we use statistical tests to see how much those statistical models "make sense".

In particular, using theoretical considerations, we can know with "absolute" certainty, if an assumption is actually correct, what consequences must necessarily follow from that assumption. If our statistical tests show that it is somehow "unlikely" that these consequences hold, then we can conclude that it is also "unlikely" that our statistical model for $A$ and $B$ is correct.

(This is because any logical statement is equivalent to its contrapositive: statement p is true implies statement q is true if and only if statement q is false implies statement p is false. Statistical tests allow us to gauge whether statement q (the consequence of a statistical model) might be false, and thus whether statement p (the statistical model) might therefore be false.)