I have two models:
$levelOfCreativity = \alpha Extravert + \beta Woman + \gamma hasACollegeDegree + \zeta isOlderThan25$
$levelOfCreativity = \alpha Extravert + \beta Woman + \gamma hasACollegeDegree + \zeta isOlderThan25 + \mu Extravert * hasACollegeDegree + \delta Woman * isOlderThan25$
The dependent variable can vary between 0 and 10. The independent variables are 0 or 1. E.g. if a person is a woman, the variable will be equal to 0.
My real model exists of 16 main variables besides hasACollegeDegree and isOlderThan25 (control groups) and 32 interactions (16 with hasACollegeDegree and 16 with isOlderThan25) but I simplified this model over here for practical reasons.
When I run a normal linear regression (no interaction), I see that hasACollegeDegree has a significant influence with a coefficient of 0.13 and that isOlderThan25 is not significant with a coefficient of -0.08. When I add the interaction terms, this changes. hasACollegeDegree becomes not significant (coefficient 0.03) and isOlderThan25 becomes significant with a value of -0.47. The differences between 0.03 and 0.13 or -0.08 and -0.47 for example are quite. The coefficients of the interaction terms are also not significant.
I understand that it is normal that my coefficients change as there are two different linear models. I also understand that the meaning of the main effects is totally different in the two models due to the interaction effect.
Taking all this into account, I have a main question: what do these findings mean if the coefficients of hasCollegeDegree and isOlderThan25 change so drastically leading to two totally different conclusions?
My first model (based on all the data without making subgroups) says that having a college degree has a significant influence on your level of creativity and that being older than 25 doesn't have any significant influence. When I add the interaction terms, the model says that having a college degree doesn't have any influence while being older than 25 has an influence. Because of that strange behavior, I don't know what the end conclusion is: does having a college degree have or have not an impact on your creativity/does being older than 25 have or have not an impact on your creativity?
The problem is that both models make sense and that based on my research, it also makes sense to take a look at both models. Things like "you need to think which model is the best and which variables/interactions you really need to include" doesn't really apply to my situation, which results in the question mentioned above.