Is it valid to write a hypothesis using the result of THIS experiment to predict an original independent variable?

Question

There is a dataset/study that has two independent binary variables, and a continuous dependent variable.

Variable A    Variable B    Measurement C
1               1               32.4
0               1               29.1
1               0               15.8
1               1               25.9
...

The binary variables were manipulated by the researchers. This simple study was observing animal behavior, by varying two binary variables in their environment, and then taking a measurement (continuous / ratio).

1. A and B only occur during a time when this animal behavior (C) occurs, and thus can be measured. The behavior does not occur when A and B are absent.
2. A and B always occur together, but they are independent.
3. The animal behavior being measured (C), can't affect either A or B.

Is it valid to take this dataset and use logistic regression to:

1. Independent variable is C; dependent variable is B.
2. Independent variables are C and A; dependent variable is B.

I can write a null hypothesis that says the odd ratio is 1, and use the data.

Mathematically, you can do this, but is the causal inference biased?

Answer 1

You can do this, but it wouldn't tell you much. I could imagine you might be interested in predicting environmental conditions from animal behavior if it was cheap to observe the behavior and expensive to measure the environment (e.g., using a canary in a coal mine), in which case it might make sense to model the environmental condition using the animal behavior. You wouldn't know the baseline prevalence of the environmental conditions, however, so you wouldn't be able to predict its specific value from the measured behavior of animals in the wild, making the model close to useless.

In science, though, we are often interested in causal relationships, which are associations that have specific properties (temporal precedence, no confounding, etc.). The odds ratio for the relationship between the environmental condition and animal behavior is a measure of association that is free of confounding, but it doesn't represent the temporal precedence of the relationship. The animal's behavior does not cause changes in the environment (or if it does, that is not the relationship you are investigating), so the odds ratio has no causal interpretation and doesn't tell you anything about how the environment would change if you forced changes in an animal's behavior or how the animal's behavior would change if you forced changes in the environment (like you did in the experiment). So this odds ratio would tell you nothing of interest.

The causal parameter of interest is likely the difference in the means of the animal's behavior between the environmental conditions. This parameter has a causal interpretation because it can be identified as the typical change in the animal's behavior that occurs when the environment is intervened upon (i.e., changed by a force external to the animal). This parameter is useful for science because it helps explain animal behavior, which the odds ratio in the previous model does not.

Finally, the second regression, with A predicting B, makes no sense at all because A and B are independent if manipulated separately by the experimenter, so the coefficient on A tells you absolutely nothing. In fact, the coefficient will likely be spuriously nonzero due to collider bias and should not be interpreted (see Elwert & Winship, 2014, for an explanation of this).

---

One interesting tidbit is the finding that if two groups are normally distributed with the same variance, the parameters in the logistic regression model predicting group membership from the continuous variable can be directly computed from the means and variances, implying a specific mathematical connection between the difference in means and the odds ratio. This was described here in a very clean and clear derivation and is worth a look.

Answer 2

Great question. As A, B, C occur together or not at all, I don't think any of the bias categories are applicable.

Regarding logistic regression, if there is an interaction effect between A and B when predicting C, then the value of A would be useful in predicting B - when including C. Logistic regression relies on the linear separability of classes, so if there is no interaction effect, A won't be helpful in predicting B. (I'm not talking about any expansions beyond just using A, B, and C in a new logistic regression model. Whether there's a more effective modeling technique is a separate issue.)

Consider the following dataset, where there is an interaction effect (extreme) between A and B on C. C and A predict B.

A  B  C  
   
2  1  40  
2  0  20  
3  1  60  
3  0  30  
4  1  80  
4  0  40  
0  1  10  
...  ...  ...