Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I haven't seen a situation like this. Both B1 and B2 are dummy variables. If I regress them separately, it's fine, but if I add an interaction term, all the coefficients are the same. I'm using statsmodels. Overlap between the two can't be too high otherwise I'd imagine singularity problems.
Any idea what may cause this?
With interactions:
Without interactions:
I agree with Peter Flom, this is a clear case of colinearty. I'm quite sure that the cause for this is that you have either no observations that are neither B1 nor B2 or none that are both, meaning you effectively have 3 groups:
With 3 groups 3 parameters is a saturated model, i.e. every group has a unique combination of parameters, and you are trying to estimate 4 parameters.
Further I'd bet that the 3rd group is quite small, which creates the weak collinearity in the non interaction model. It might make more sense to pick one of the large groups as the baseline, especially if you have no observations that are neither B1 or B2, i.e. observations that actually inform the intercept directly.
It looks like you DO have singularity problems! My guess is that tiny number is the closest to zero your program can display, and the version without the negative sign is the BIGGEST number the program can display. So all the coefficients are the same (absolute) value because they are all either zero or infinity. Which means you have some collinearity issues.
A good rule of thumb - whenever you get a strange result like this, just step back and do some bivariate correlations (or in this case, a cross tab). Often the issue becomes obvious immediately, as it probably will here.
3.19e+14, but even if the program is only in single precision (which I doubt) it will have no problems displaying numbers greater than $10^{38}.$ The numbers $x$ that appear here almost surely arise because they are getting close to values where $1+1/x=1$ in double precision floating point representation.
$\endgroup$
Adding to Graham's answer and Whuber's comment (+1 to both).
Your coefs are NOT the same. They are all huge, but with opposite signs. And they are ridiculously huge (On the order of $10^{13}$, i.e. 10s of trillions) that their standard errors are even bigger, that their CIs span 0 (despite their magnitude) and the condition number that Whuber mentioned. Collinearity is usually regarded as quite problematic when it is over 30. Yours is rather larger!
I am guessing you have perfect collinearity and your program is just trying to cope.
Even in the model without interactions, there is some evidence of collinearity, although not severe.
