0
$\begingroup$

I am doing SPSS analysis on effect of work-life balance on life satisfaction (LS).

I took a well established questionnaire and split the items into work-life balance (WLB) versus work-life control (WLC) construct.

In addition, I created a new construct called work-life outcome congruence (WLCON) and developed an 8-item questionnaire that has passed cronbach's alpha and factor analysis.

Linear regression of each individual variable by itself gives me p < 0.01, but linear regression of all 3 independant variables (WLCON, WLC, WLB) together, I get only WLCON to be significant.

enter image description here

But when I conduct a mediator analysis where X = WLC, Y = LS and M = WLCON, I don't get a mediator relationship.

What can I say about the role of WLCON in predicting LS?

Edit: Upon @commscho advise, I am including more info here for reference

enter image description here enter image description here enter image description here

$\endgroup$

1 Answer 1

2
$\begingroup$

Well, there are a few ways to talk about it. The decisionmaking depends somewhat on the underlying theory.

In general, when a regression coefficient is statistically meaningful, we might say it represents a "direct" or "main" effect, especially in contrast with moderated and mediated effects. Of course, you should be careful with words like "effect" unless your method or theory tells you that the predictor (WLCON) actually causes the dependent variable (LS). The phrasing "direct" or "main" effect is meant to convey that WLCON's relationship with LS doesn't seem to depend on some other variable.

For example, if I had this model with an additional variable, gender, I might try to see if the relationship between WLCON and LS is moderated by gender by adding an interaction term. If the interaction term was not statistically meaningful but the WLCON estimate still was, I would then say that there is an apparent main effect of WLCON; it does not matter what value gender takes, the estimated effect of WLCON is still there.

Making sense of the changes in significance

As for thinking about the results of bivariate regressions versus this multiple regression, you should think about how each of the predictors relate to each other. All 3 of them appear to be quite highly correlated with the dependent variable. But I can tell from the VIF collinearity statistic that WLC and WLB are highly correlated with each other, while not nearly as correlated with WLCON. When you have two variables in a regression model that are highly correlated with each other, it is often the case that the model will estimate large standard errors for those variables, thereby reducing the p values.

The VIF stands for "variance inflation factor", which means just what it says—that the correlation of those two variables with each other may be increasing the variance of the regression coefficient, making it less likely to be statistically significant. There are some who suggest not interpreting the regression coefficient of a variable that has a VIF greater than 5 (or some say 10 or other thresholds), though there are some exceptions where the VIF is deceiving, like if you have an interaction term in the model.

So what? It is possible that 2 of the 3 or all 3 of your variables are in fact equally related to the dependent variable, but the fact that 2 of them are so related to each other wrongly causes the regression to make it seem as if the 3rd variable (WLCON) is far more clearly related to the DV. It is also possible that the model is not misleading you and it is in fact the case that WLCON is a better predictor of LS than either WLC or WLB. The fact that the estimate is higher for WLCON than WLC and WLB is a good sign for that hypothesis.

Possible solutions to better understand the data

I would suggest thinking about doing the following:

  • Run a regression model without WLCON, but both WLC and WLB. Is one far more clearly related to the DV than the other when you do this? Does that make theoretical sense?
  • Based on the prior suggestion or solely based on your theory, you could choose to create a model with just one of WLC and WLB. This will eliminate the problem of two predictors being correlated with each other. Of course, it could pose a problem with your theory if you think it is very important that you control for both of those constructs at once.

As for the overall question, regardless of whether the model is misleading you about WLC and WLB, it is a good sign that WLCON appears to be a good predictor of your DV when you have what I assume are theoretically related control variables in the model.

An unsolicited comment about measurement

One last thing, since you mentioned factor analysis. You don't say how WLC and WLB are measured other than they are each components of some other scale. Hypothetically speaking, suppose WLCON, WLC, and WLB were actually all different ways of measuring the same thing (this sometimes happens to researchers by accident and I suspect your goal is in part to prove that WLCON is not just a new version of the old scale).

If WLCON was measured much better than WLC and WLB, either because the scale has 8 items and the other two have fewer or because the scale is more reliable for some reason, the estimated effect of WLCON would be more statistically significant solely based on the fact it is measured better (with less random error). So if you intend to make an argument that WLCON is a better construct than either WLC or WLB rather than just a better measure, you'll have to think about how well WLC and WLB are measured compared to WLCON.

$\endgroup$
2
  • $\begingroup$ I wish I can give a million up votes! I ran the regression with only WLC and WLB as you advised, and indeed, WLB became far more insignificant than WLC, and their VIF > 5. In this case, other than concluding they are measuring the same construct, is there no way else to go? What should I do from here? Should I combine WLC and WLB into a new construct and continue the analysis, or is it better to drop WLB and continue with WLC? $\endgroup$
    – Jake
    Commented Aug 28, 2017 at 14:30
  • $\begingroup$ Well, I think that's a decision that probably should be more theory-driven than anything else. Since you mentioned WLC and WLB are derived from the same scale, I wonder what your model would look like if those WLC and WLB items were combined as the original scale intended. In many fields, it's okay when validating a new measure/construct to explore its relationships with other related constructs (e.g., it's not so bad to report multiple, similar analyses). $\endgroup$
    – commscho
    Commented Sep 4, 2017 at 23:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.