I'm cross-posting this question on both Economics and Cross Validated to get answers from a different perspective on each field. It is generally accepted to cross-post if the question is tailored to fit better in each community. See: https://meta.stackexchange.com/a/64069/510233
Cross-posted in Economics: https://economics.stackexchange.com/q/27806/21749
I'll try to be as detailed as possible to best deliver what I want to know in this question.
In data science, there are some variables that are composed of many other variables. (Derived from other existing variables) Mostly indices or multiples.
One example would be something like Wind chill Temperature Index which is
WCI= wind chill index, kcal/m2/h
v= wind velocity, m/s
Ta= air temperature, °C
What I want to know is whether a variable like WCI can have more meanings(effects) than v and Ta seperately combined.
Based on what I learned in regression analysis from Ecnonometrics class, a simple interaction term like would be interpreted as the difference in marginal effect between a control group (D=0) and a treatment group (D=1), when expressed like:
Also, more general interaction term like VariableA * VariableB would be interpreted as an impact of VariableA on the coefficient of VariableB, in some examples like
PollutionLevel = B0 + B1*Population + B2*NumberOfCars + B3*Population*NumberOfCars = B0 + B1*Population + (B2 + B3*Population)*NumberOfCars
(originally from a question posted on Cross Validated)
But I think these interpretations are too simple to capture the whole meaning(effect) of a new variable derived from other variables, which leads to other questions like:
What would be the interpretation of a super complex interaction term like
WCIin regression perspective? (and other data science methods like Random Forest or Deep Neural Network, etc if possible.)
Would it still be valid and preferable to include an explanatory variable like
WCIin the model when I already have
Ta? Would it make the model more accurate?
Intuitively, I feel like people would care more about important indices but not necessarily the variables it is composed of. (Thus have more effect.) Can a complex interaction term derived from other variables have more effect than a simple combination of the other variables?
The third one is related to my original question of whether one variable that is composed of other variables in a dataset can have more effect on the dependent variable.
Since it's Cross Validated, I want to ask more about popular data science models other than regression analysis.
- Can linearly dependent variable harm tree-based or neural network models' effectiveness just like regression analysis? Or is it even better to include them?
In linear regression, it is bad to have linearly dependent models because of the multicollinearity issue. The regression model can become invalid if there is severe multicollinearity and you should never have an exactly linear variable in the model.
However, I heard that tree-based models like Random Forest or Boosting Tree are robust from this issue. Also, I've never heard anyone complaining about multicollinearity when making a deep neural network model.
I'd like to know what the effects of complex interaction terms (derived from other variables) are on tree-based / deep neural network models and if it is recommended to include them in the model.