Influence of one variable to another

What do we mean in statistics when we say 'variable A influences variable B?'

And how do we test detect this influence?

Im not sure of the right tag for this question. What topic in statistics is this point tackled?

• Regression may be what you are looking for? Or correlation? Actually most of statistics is concerned with relationships between variables... – Knarpie Mar 23 '17 at 13:06
• This seems like a perfectly reasonable question to me. I'm voting to leave open. – gung - Reinstate Monica Mar 23 '17 at 15:29

Typically in statistics the main thing we investigate is whether one variable changes if another one changes. For example with a two-sample t-test we check whether the mean of a variable changes if we treat the two samples differently.

We do similar things in regression (testing whether one variable goes up or down with another variable) or with chi-square tests (testing whether a distribution changes when a variable changes between groups).

Whether or not we can then conclude on the basis of that answer if variable A has an effect on variable B depends on the set-up of the research.

• Are you suggesting that the term relationship can be interchanged with influence? – Isaac Newton Mar 23 '17 at 22:38
• No, not directly. To me "relationship" is weaker than "influence", because influence implies a direction, whereas relationship does not. If we say that A influences B, that implies to me that A has an effect on B, but not the other way around. If I say there is a relationship between A and B, that may mean that A has an effect on B, B has an effect on A or there might even be a third variable C, that has an effect on A and B. Typically we can test relationships, but only in special cases (like randomized controlled experiments) can we say something about the direction (and thus influence) – Maarten Punt Mar 24 '17 at 15:04
• You say that in 'influence,' it has a direction while 'relationship' does not. How do you establish this direction then between the two variables? What test should i use to detect this influence between two variables? – Isaac Newton Mar 27 '17 at 1:25
• You don't use a test. The test only tests whether a relationship exists. The direction can generally only be determined from the context. When I run a linear regression on "time spent studying for an exam" and "exam results" I can use "time spent" to explain "exam results" or the other way around. The stats will work either way. However, in that specific case I can say that if a relation is present, as established by a test, it is the influence of time spent studying on exam results, because your exam result cannot logically have an effect on the time you spent studying – Maarten Punt Mar 27 '17 at 6:56

I would say (from a purely mathematical perspective) that, considering both $A$ and $B$ are random variables, variable $A$ influences variable $B$ if and only if they are not independent, i.e. if $$\mathbb{P}(A=a \text{ and }B=b) \not= \mathbb{P}(A=a)\cdot\mathbb{P}(B=b) \,.$$

The (un)equation basically says that "information about $A$ and $B$ considered together" is not completely determined by "information about $A$ alone" and "information about $B$ alone".

In other words, there is no interaction between $A$ and $B$ which we need to take into consideration when trying to deduce "information about $A$ and $B$ considered together" from "information about $A$ alone" and "information about $B$ alone". I.e., $A$ and $B$ together are "just the sum of their parts" and are "not greater than the sum of their parts".

(Of course we have to use multiplication instead of addition, since probabilities are bounded between $0$ and $1$ inclusive, and multiplication of such numbers is closed under multiplication but not addition.)

The statistics comes in because we do not have perfect knowledge of the universe, so we do not know exactly whether $A$ and $B$ "really are" random variables, and even if we did, we do not have perfect enough knowledge of $A$ and $B$ to say with absolute certainly whether the above formula does or does not hold.

In other words we do not know exactly how to consider them as random variables even if we knew with certainty that they were random variables, so the definition given above doesn't necessarily make sense without a lot of interpretation (since the definition requires us to consider $A$ and $B$ as random variables, which we don't necessarily know how to do "in the best way").

So (according to my rudimentary understanding of statistics) we employ statistical models to try to model $A$ and $B$ as random variables, and we use statistical tests to see how much those statistical models "make sense".

In particular, using theoretical considerations, we can know with "absolute" certainty, if an assumption is actually correct, what consequences must necessarily follow from that assumption. If our statistical tests show that it is somehow "unlikely" that these consequences hold, then we can conclude that it is also "unlikely" that our statistical model for $A$ and $B$ is correct.

(This is because any logical statement is equivalent to its contrapositive: statement p is true implies statement q is true if and only if statement q is false implies statement p is false. Statistical tests allow us to gauge whether statement q (the consequence of a statistical model) might be false, and thus whether statement p (the statistical model) might therefore be false.)

• Variable $A$ influences variable $B$ iff they are not independent, are you talking about the independence that chi square test of independence is trying to test? – Isaac Newton Mar 23 '17 at 22:37
• @IsaacNewton Yes, exactly. en.wikipedia.org/wiki/… – Chill2Macht Mar 23 '17 at 22:50
• I think if two variables showed a strong correlation, chi square test of independence would also say that those two same variables are not independent. So does correlation also check 'influence' then? – Isaac Newton Mar 27 '17 at 1:16
• @IsaacNewton Two independent variables are uncorrelated (although uncorrelated variables are not necessarily independent). Thus, if two variables are correlated, then they are not independent. – Chill2Macht Mar 27 '17 at 9:38