I have a number of independent variables $x_1,x_2,...,x_m$ and a dependent variable $y$. My dataset contains some million of rows.

Bear with me if my wording is not precise here, and you are welcome to correct me! I assume that there is no multicollinearity in my data, i.e. $x_i$ cannot be explained by $x_j$ for $i\neq j$.

However, the dependent variable $y$ seems to react similarly on changes of, say, $x_1$ and $x_2$. How is it called, if the change in $x_1$ changes $y$ similarly like a change in $x_2$?

The question here is: Being on the greater search for a way to aggregate my $x_i$ to fewer variables, what would be the right method/procedure/research area/search engine term to find out whether (and how) $x_1$ and $x_2$ explain $y$ similarly?

  • You might want to read around formative measurement models. They're a set of variables which, while not necessarily correlating with each other, are thought to be conceptually associated. The variables in a formative measurement model are typically summed (occasionally with weightings) to produce a single variable, similar to what @MaartenBuis suggests in his answer. – Ian_Fin Nov 2 '16 at 9:08
  • What I called a sheaf coefficient is a formative measurement model. – Maarten Buis Nov 2 '16 at 9:25

A common way (not necessarily correct) is to use factor analysis to combine predictors. That won't work in your case as you assumed that the explanatory variables are uncorrelated.

If you mean with "the dependent variable $y$ seems to react similarly on changes of, say, $x_1$ and $x_2$", that the coefficients are very similar, then you can reduce the complexity of your model by constraining those coefficients to be the same. A simple trick is to create a new variable that is the sum of $x_1$ and $x_2$ and add that instead of $x_1$ and $x_2$ (a small note explaining this trick)

Alternatively, a way to combine variables that does not require those variables to be correlated is the sheaf coefficient (the original article).

  • Thank you, I will look into the "sheaf coefficient". I have tried to just sum up x1, x2 but this is too simple – IceFire Nov 2 '16 at 10:53

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.