Multilinear regression with nominal predictors - Bayesian

Question

I have four nominal predictors and one metric predicted variable. I would like to know which one of predictors have more influence on the predicted variable. For doing so, I am curious to know if I can use multiple regression. My reservation is that predictors are nominal, and not numerical, which might make it impossible to use regression.

That being said, I am also curious to use Bayesian methods for regression, the main reason of which is the correlation among predictors. If regression is in fact a proper method for my data, can I use multivariate normal distribution for predictors in the Bayesian regression model?

Any help is highly appreciated!

Answer 1

Nomimal predictors are no problem for regression. In most software, the nominal predictors are transformed to dummy variables and their effects are estimated individually.

Provided you have good reason to believe that your data conditioned on your variables is normal, then Bayesian regression sounds fine to me. Here is an example of how you might perform a Bayesian linear regression on nominal variables using rstanarm.

library(rstanarm)
library(tidyverse)


model_data = mtcars %>% 
    select(cyl, mpg) %>% 
    mutate(cyl = as.factor(cyl))


model = stan_glm(mpg~cyl, 
                data = model_data)

You can determine the strength of each predictor by looking at the summary of the model.

Answer 2

Yes, nominal predictors can be employed in even standard regression models.

Now, as you asked the question: "I would like to know which one of predictors have more influence on the predicted variable", I would simply recommend that you focus on 'Standardized Beta Coefficients'.

The standardized coefficients are computed by centering the data (subtracting means) and scaling by the respective standard deviations in the regression model. Per the cited source:

A standardized beta coefficient compares the strength of the effect of each individual independent variable to the dependent variable. The higher the absolute value of the beta coefficient, the stronger the effect... This means the variables can be easily compared to each other. In other words, standardized beta coefficients are the coefficients that you would get if the variables in the regression were all converted to z-scores before running the analysis.

And also, of import as the unstandardized dummy variables can have different standard deviations:

In regression analysis, different units and different scales are often used. For example, one variable might use dollars and another might use percentages. Standardizing coefficients means that you can compare the relative importance of each coefficient in a regression model.

The corresponding t-tests indicates the significance levels of the respective standardized beta coefficient.