Causality Analysis and test of Independence

Question

First the problem: I am refering to lecture note, on page 480, 2nd paragraph, it mentions

If X is in fact useless for predicting Y given Z, then an adaptive bandwidth selection procedure (like cross-validation) should realize that giving any finite bandwidth to X just leads to over-fitting.

I am working with this UCI Dataset for room occupancy.

The dataset is simple, as an exercise, I am trying to find the causal Directed Acylic Graph for the following dimensions, Temperature, Humidity, CO2 and Light.

If I follow the quote above, if Temperature cannot predict CO2 given let's say Humidity, then it probably can be assumed that Temperature and CO2 are independent in causal DAG (ofcorse, I shall have to do it for Temperature with Light as well).

Now for prediction, I am considering a simple 3 layer feed forward neural network, where the inputs would be Temperature and Humidity and the output CO2. I shall train the model on the training set and validate it against the test set, which, if Temperature cannot predict CO2 given Humidity, the NN would overfit on the training set and give high Mean Square Error on test set.

However, I am not sure, if this scheme will work, as, may be Temperature does not have ability to predict CO2 but may be Humidity has (as the common confounder is occupancy in the dataset), so the model may predict CO2 with some good accuracy and will not tell me if Temperature and CO2 are cusally independent or not.

In this situation, how to proceed with the problem, given that I only have the observations. Any help n this will be appreciated

Answer 1

I was unaware of this algorithm for the test of independence. It is an interesting algorithm. I think the reason that this algorithm can determine the independence between $X$ and $Y$ given $Z$ is that if this independence relationship holds in data, adding $X$ beside $Z$ for predicting $Y$ gives no more information and adds unnecessary extra complexity to your model. This can cause overfitting since your model tries to take $X$ into account and because $X$ has nothing to say, the model just tries to model something like noise.

I also agree with you that this scheme may not work and in the lecture note that you referred, it is obviously mentioned: "slightly less rigorous". There are many other factors, such as the amount of the training data, that can affect overfitting which are neglected in this analysis. I suggest you use standard independence test such as chi-squared test. Also, this question seems also beneficial.