I am trying to train a linear regression model on $n$ data samples. My training set contains two features sets, $X_i$ and $Y_i$ for $i \in \{1, 2, \dots, n\}$. I want to create a transformed third feature $Z_i = X_i Y_i$ (i.e. the element-wise product of features $X_i$ and $Y_i$).

For example, $X = [1, 2, 3]^T$, $Y = [4, 5, 6]^T$, $Z = X \cdot Y = [4, 10, 18]^T$.

Linear regressions should avoid multicollinearity in their regressors, so I was wondering:

  1. What do we know about the correlation between $Z_i$ and $X_i$ or $Y_i$? We can assume that $X_i$ and $Y_i$ are standardized to have $0$ mean and unit variance.

  2. Can we simply do a correlation test between $Z_i$ and $X_i$ or $Y_i$ to see if our regressors are multicollinear or not (and therefore appropriate to use in our linear regression model)? Or is the fact that $Z_i$ depends on $X_i$ and $Y_i$ automatically make it a troubling regressor to use in our model along with $X_i$ and $Y_i$?

  • $\begingroup$ Are $X$ and $Y$ both predictors of a third variable? It would be conventional to reserve $Y$ for the response variable. $\endgroup$ – Dave Nov 25 '20 at 3:47
  • $\begingroup$ Yes, $X$ and $Y$ are predictors of a third variable which is I believe irrelevant to this post. I wasn't sure what variables to use and hope it's not too confusing to use $Y$ as a predictor. $\endgroup$ – user3667125 Nov 25 '20 at 5:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.