Ordinal regression, multinomial regression or linear regression with dummy variables?

Question

I am conducting a study looking at the quality of clinical trials. I am trying to find out what factors are predictors of the methodological quality of the trials. My dependent variable is a score grading their methodology (score range is from 0 to 10). I have a number of independent variables. Some are multi-category nominal like country, trial type (2-arm, multi-arm, etc) or discipline (cardiology, oncology, radiology, etc.). Others are continuous and binary.

I was initially under the impression that for the multi-category ones, creating dummy variables and then proceeding with linear regression was the correct way. One question I have is that after coding for the dummy variables, depending on which I use as the reference, the significance of the others will change. I understand the math behind it, but I am confused as to which category I should then use as a reference. How does one decide on which category should be used as the reference? The categories are all of equal importance. For example, one of the multi-category variables is funding source - government, non-profit or private. Depending on which I use as the reference, the significance of the other two varies.

Second question: after reading some texts, some are suggesting ordinal or multinomial (depending on the distribution of data within each independent variable). Is this a better method for what I need rather than doing linear regression with dummy variables?

Thanks in advance!

Answer 1

This is really two questions. One about the choice of regression method, the other about treatment of categorical independent variables.

The choice among ordinal, multinomial, or linear (OLS) regression (or other forms of regression) depends on the nature of the dependent variable (DV), not the independent variables (IV). You say your DV goes from 0 to 10, but you don't say if it is continuous or categorical (i.e. can you get a score of 8.3?). If it is continuous, then either linear or maybe beta regression would be good (for beta, you would have to divide by 10). If it is categorical, then I would start with either linear or ordinal logistic.

For treatment of the IVs, @dipetkov shows a way to look at means. However, you can also change the default parameterization and the default reference level. Both of these will change the p values of different levels. The default reference level is chosen alphabetically. This is rarely useful. Common choices are a) The most common level or b) The level seen as "normal" (if that suits the variable) or c) the one with the lowest or highest level of the DV. This doesn't change the real meaning of the analysis, but it does affect the output.

For parameterization, I believe R defaults to dummy coding. This is often a good choice, but, in some cases, other paramterizations may be better. This depends on your particular variable and the questions you want to ask about it.