1
$\begingroup$

I am conducting a multiple linear regression analysis in SPSS.

My DV is a score between 0 and 6, and my predictors are:

  • one dichotomous nominal variable (native vs. non-native speakers)
  • one continuous variable (%)
  • one nominal variable with 5 levels (five different countries of origin) coded as dummy variables.

The sample sizes / frequencies for three of the dummy variables are under the recommended 15% of the whole sample size (namely 12,5%, 5% and 5%). Is it a big problem? Can I correct for this with some procedure?

Moreover, in view of my pp plot / residuals histogram: residuals histogram versus normal

I think the residuals of my data are not normally distributed. Can I correct for this with some procedure?

$\endgroup$
  • $\begingroup$ Can your DV take on any value continuously between 0 and 6, or just integer values, or perhaps just a limited number of intermediate values? Also, does the distribution of dummy variables reasonably represent their distribution in the population from which you are sampling? $\endgroup$ – EdM May 20 '16 at 16:23
  • $\begingroup$ My DV can only take any integer values between 0 and 6. No, the distribution of my dummy variables is a problem: for instance, 37% of my sample comes from Europe whereas only 5% comes from Asia. $\endgroup$ – Pernelle May 21 '16 at 12:24
1
$\begingroup$

As your DV can only take integer values it isn't really a continuous variable, so it's not surprising that your plot of residuals has a set of peaks and valleys around an underlying normal curve. Technically such data are best handled by ordinal regression, which allows for an ordered set of discrete categories in the DV.

Depending on the stage of your study, you may be able to learn enough from your multiple regression as you have performed it. You typically want more categories in the DV than this to approximate a set of integer values as a continuous DV, but the residuals don't seem to be skewed and if they also don't depend on the predicted values then you might be doing well enough. You will have to be careful to admit that your data might not strictly meet the criteria for standard interpretation of p-values.

With respect to the imbalanced distribution among levels of your nominal independent variable, one problem is that you will have more imprecise estimates of the effects of the levels that have the smaller numbers of cases. The estimates of the effects of the different countries of origin will also be correlated. Furthermore, the imbalance can complicate some types of ANOVA tests in which you try to apportion variance between, say, between the native/non-native and country-of-origin predictors. Finally, it will be hard to be sure that your results will generalize well to other samples. Short of obtaining more data there isn't much you can do about the imbalance. You might consider bootstrapping to get a bit more confidence in the generalizability of your model than the initial multiple regression can provide on its own.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks a lot @EdM . I really appreciate your help. I am not acquainted with ordinal regression so I will bury myself in some articles/tutorials about this. Regarding your other remarks: My scatterplot of residuals against predicted values does show some pattern, I think. (see here <dropbox.com/s/rhk64g76jg63wql/ZRESID_ZPRED.jpg?dl=0>). So, do you think I should definitely abandon this regression model and go for an ordinal regression? Regarding the imbalance: I've just discovered that you can weight your data in SPSS. Would this make more sense than bootstrapping? $\endgroup$ – Pernelle May 22 '16 at 14:50
  • $\begingroup$ I think your pattern comes from observed DV values being integers so that residuals between continuous predictions and integer observations show up in a raster. Depending on your needs and the expectations of your audience, the regression might be adequate as is. I'm not a great expert on weighting; my sense is that you really can't get more information than you already have and weighting is best reserved for cases when data points differ in inherent reliability. Bootstrapping is the closest you can come to evaluating how your model would perform on new samples from your population. $\endgroup$ – EdM May 22 '16 at 23:07
  • $\begingroup$ Dear @Edm I am coming back to you since I am re-analysing this data. You suggested to use ordinal regression since my DV only includes non-negative integers. Since I am regarding a response of 4 twice as good as a response of 2, I guess I would lose information if I would consider my DV as ordinal data. So I thought I would rather regard my DV as count data. I found that Poisson regression or negative binomial regression could be useful tools. However, a K-S test shows that my DV does not follow a Poisson distribution AND my sample mean (2.92) exceeds my sample variance (1.5). Any tips? $\endgroup$ – Pernelle Nov 28 '16 at 10:40
  • $\begingroup$ @Pernelle except for the peakiness, which I suspect has to do with the integer nature of your response variable, your residuals don't look that bad. Normality of residuals mostly matters for standard significance testing. If you believe that 4 is twice as good a response as 2, you could consider simply doing a standard linear regression but basing your significance testing on bootstrapping rather than on relying on a normal-distribution assumption about residuals. Much depends on what your goals are and whom, besides yourself, you need to convince. $\endgroup$ – EdM Nov 28 '16 at 14:31
  • $\begingroup$ @ Edm Mmmh, I see. I will have a look at bootstrapping, which I am very unfamiliar with, since I would like to get this published. Thank you for your suggestion! $\endgroup$ – Pernelle Nov 28 '16 at 20:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.