1
$\begingroup$

First I ran the following model:

$$y = \beta_1 x_1 + \beta_2 x_2 + ...$$

And I tested (t-test) if $\beta_1>0$ and $\beta_2>0$, which is true for both.

Then I split $x_1$ in two, such that $x_{1a}+x_{1b}=x_1$:

$$y = \beta_{1a} x_{1a} + \beta_{1b} x_{1b} + \beta_{2} x_{2} + ...$$

Then I ran the t-tests to check which coefficients were positive. Only $\beta_{1b}$ was.

What are the possible reasons for $\beta_2$ no longer being positive after splitting the first regressor in two? I tried to run different models, for instance, deleting $x_{1a}$ and $x_{1b}$ from the model or deleting only one of them, but still $\beta_2$ isn't positive. How can it be that it only works in the first model? Any insights are appreciated. Please note that there are many other regressors in the model, so that could be the problem.

I should also add that I'm simplifying the problem. I'm fitting this model to several data sets, then checking if the mean $\beta_2$ is positive. I don't know if that makes any difference.

And to give it even more context, there are two sets of points I'm trying to predict, set 1 and set 2, which importantly overlap somewhat. $x_1$ tries to predict both set 1 and set 2, while $x_2$ tries to predict only set 2 (but this isn't clear cut because as I mentioned the sets overlap). The issue here is that set 2 is easy to predict. Set 1 is where it's at. So when I got the first results, they suggest that $x_2$ explains set 1, but we don't know if $\beta_1>0$ because $x_1$ is a good predictor set 1 or set 2 or both. It should be a good predictor of set 2, so it's possible that it doesn't predict set 1 at all. So I split $x_1$ into two, with $x_{1a}$ trying to predict set 1 and $x_{1b}$ trying to predict set 2. So $\beta_{1b}$ was very highly statistically significant, which surprises no one as set 2 is easy to model.

The surprising part is that neither $\beta_{1a}>0$ nor $\beta_2>0$ in the second model! It's like the first model can explain set 1 but the second cannot. But how can that be true? The second model has the same information as the first but more terms. If anything, it overfits the data. It cannot possibly fit the data set any worse. It cannot possibly fit the set 1 points any worse! And yet it seems unable to say anything about set 1. It's important for my research to know why. Does $x_2$ suck as a regressor? Or is there a trivial (unrelated to $x_1$) explanation for its failure within the second model?

$\endgroup$

1 Answer 1

0
$\begingroup$

The answer is that $x_{1a}$ and $x_2$ are negatively correlated and not correlated at all with $y$. Because $x_{1b}$ is positively correlated with $y$, when I sum $x_{1a} + x_{1b}$, the coefficient for the sum ends up being positive, but also for $x_2$, because $\beta_2 x_2$ partially cancels out $\beta_1 x_{1a}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.