3
$\begingroup$

I haven't seen a situation like this. Both B1 and B2 are dummy variables. If I regress them separately, it's fine, but if I add an interaction term, all the coefficients are the same. I'm using statsmodels. Overlap between the two can't be too high otherwise I'd imagine singularity problems.

Any idea what may cause this?

With interactions:

Regression With Interaction

Without interactions:

Regression Without Interaction

$\endgroup$

3 Answers 3

4
$\begingroup$

I agree with Peter Flom, this is a clear case of colinearty. I'm quite sure that the cause for this is that you have either no observations that are neither B1 nor B2 or none that are both, meaning you effectively have 3 groups:

  1. only B1
  2. only B2
  3. both B1,B2 or neither B1,B2

With 3 groups 3 parameters is a saturated model, i.e. every group has a unique combination of parameters, and you are trying to estimate 4 parameters.

Further I'd bet that the 3rd group is quite small, which creates the weak collinearity in the non interaction model. It might make more sense to pick one of the large groups as the baseline, especially if you have no observations that are neither B1 or B2, i.e. observations that actually inform the intercept directly.

$\endgroup$
4
$\begingroup$

It looks like you DO have singularity problems! My guess is that tiny number is the closest to zero your program can display, and the version without the negative sign is the BIGGEST number the program can display. So all the coefficients are the same (absolute) value because they are all either zero or infinity. Which means you have some collinearity issues.

A good rule of thumb - whenever you get a strange result like this, just step back and do some bivariate correlations (or in this case, a cross tab). Often the issue becomes obvious immediately, as it probably will here.

$\endgroup$
3
  • 7
    $\begingroup$ Check out the reported value of the conditioning number, "Cond. No.", in the first regression. $\endgroup$
    – whuber
    Commented Sep 18, 2023 at 17:42
  • $\begingroup$ Re the edit: what exactly are you maintaining is the "biggest number the program can display"?? The largest one appearing is 3.19e+14, but even if the program is only in single precision (which I doubt) it will have no problems displaying numbers greater than $10^{38}.$ The numbers $x$ that appear here almost surely arise because they are getting close to values where $1+1/x=1$ in double precision floating point representation. $\endgroup$
    – whuber
    Commented Sep 19, 2023 at 16:19
  • $\begingroup$ The negative numbers still habe e+13/14, i.e. are negative trillions and not at all close to 0. $\endgroup$ Commented Sep 21, 2023 at 8:42
3
$\begingroup$

Adding to Graham's answer and Whuber's comment (+1 to both).

Your coefs are NOT the same. They are all huge, but with opposite signs. And they are ridiculously huge (On the order of $10^{13}$, i.e. 10s of trillions) that their standard errors are even bigger, that their CIs span 0 (despite their magnitude) and the condition number that Whuber mentioned. Collinearity is usually regarded as quite problematic when it is over 30. Yours is rather larger!

I am guessing you have perfect collinearity and your program is just trying to cope.

Even in the model without interactions, there is some evidence of collinearity, although not severe.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.