0
$\begingroup$

I have four nominal predictors and one metric predicted variable. I would like to know which one of predictors have more influence on the predicted variable. For doing so, I am curious to know if I can use multiple regression. My reservation is that predictors are nominal, and not numerical, which might make it impossible to use regression.

That being said, I am also curious to use Bayesian methods for regression, the main reason of which is the correlation among predictors. If regression is in fact a proper method for my data, can I use multivariate normal distribution for predictors in the Bayesian regression model?

Any help is highly appreciated!

$\endgroup$
2
$\begingroup$

Nomimal predictors are no problem for regression. In most software, the nominal predictors are transformed to dummy variables and their effects are estimated individually.

Provided you have good reason to believe that your data conditioned on your variables is normal, then Bayesian regression sounds fine to me. Here is an example of how you might perform a Bayesian linear regression on nominal variables using rstanarm.

library(rstanarm)
library(tidyverse)


model_data = mtcars %>% 
    select(cyl, mpg) %>% 
    mutate(cyl = as.factor(cyl))


model = stan_glm(mpg~cyl, 
                data = model_data)

You can determine the strength of each predictor by looking at the summary of the model.

$\endgroup$
3
  • 1
    $\begingroup$ +1 F-testing in the k-sample ANOVA problem uses a regression with only categorical predictors. ANCOVA uses categorical predictors along with a continuous predictor. Designed experiments may involve a mix of many categorical and continuous predictors. So, yeah, OLS (and other regression techniques) handle categorical predictors just fine. $\endgroup$ – Dave May 25 '20 at 13:47
  • $\begingroup$ Thanks Demetri for the reply. Do you know exactly how the model is? I would also like to know if the model can accommodate the dependence or multicollinearity among predictors. Is there any python code by any chance? $\endgroup$ – Majid Mohammadi May 25 '20 at 13:54
  • $\begingroup$ @MajidMohammadi So long as predictors aren't perfectly collinear, then the model should be able to accomodate. There should be more uncertainty associated with the effects however. There are several bayesian modelling libraries in python. I recommend PyMC3. $\endgroup$ – Demetri Pananos May 25 '20 at 13:56
0
$\begingroup$

Yes, nominal predictors can be employed in even standard regression models.

Now, as you asked the question: "I would like to know which one of predictors have more influence on the predicted variable", I would simply recommend that you focus on 'Standardized Beta Coefficients'.

The standardized coefficients are computed by centering the data (subtracting means) and scaling by the respective standard deviations in the regression model. Per the cited source:

A standardized beta coefficient compares the strength of the effect of each individual independent variable to the dependent variable. The higher the absolute value of the beta coefficient, the stronger the effect... This means the variables can be easily compared to each other. In other words, standardized beta coefficients are the coefficients that you would get if the variables in the regression were all converted to z-scores before running the analysis.

And also, of import as the unstandardized dummy variables can have different standard deviations:

In regression analysis, different units and different scales are often used. For example, one variable might use dollars and another might use percentages. Standardizing coefficients means that you can compare the relative importance of each coefficient in a regression model.

The corresponding t-tests indicates the significance levels of the respective standardized beta coefficient.

$\endgroup$
1
  • $\begingroup$ Because nominal predictors having more than two categories need more than one coefficient, how exactly do you propose using standardized betas to compare two such predictors? $\endgroup$ – whuber May 26 '20 at 12:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.