4
$\begingroup$

I am running a linear regression for Explanatory purposes. Y is continuous and all the explanatory variables are categorical. I understand that the regression coefficient of these variables is the difference in mean of that variable and the mean of the reference for that categorical. My model is as follows:

$Rates = 0.92 + 0.2 \mathrm{Comapny_{STK}} + 0.1 \mathrm{Company_{FLR}} + 0.2\mathrm{Location_{ATL}} - 0.8 \mathrm{Location_{NYC}} + 0.5 \mathrm{Location_{SA}} + \ldots + w_n x_n$

However, is there another method I could use so that the interpretation of the coefficients is similar to that when one runs a linear regression with just continuous explanatory variables, and I could then see the relative effect of each explanatory on $Rates$? I essentially want to see which among $Company$ and $Location$ has the biggest influence on $Rates$. To my understanding, two-way ANOVA's would not be able to answer which of the categoricals has a bigger influence. So I'm not too sure what sort of correlation measure or otherwise to use here.

$\endgroup$
3
$\begingroup$

Remove the constant from the RHS and take all categories for each categorical value as dummy.

For example, say you have two categorical variables $X, Y$ and one continuous dependent variable $Z$. Assume each of the categorical variables have two categories: $1, 2$

You can model it in two ways

One (seemingly what you have done): $$E(Z |X, Y)=\beta_0 + \beta_1 D_x + \beta_2 D_y$$

Here $D_x=1$ if $X=x_2$ and $0$ otherwise. Similarly for $D_y$

From this, we have that $$\beta_0 = E(Z | X = x_1, Y=y_1)$$

Alternatively, you can model like this:

$$E(Z |X, Y)= \alpha_1 D_{x_1} + \alpha_2 D_{y_1} + \beta_1 D_{x_2} + \beta_2 D_{y_2}$$

Now each coefficient tells you what is the conditional mean of $Z$ in presence of your categorical variable.

$\endgroup$
10
  • 1
    $\begingroup$ In first method, $\beta_1$ is the change in $E(Z|X=x_1)$. What you want is change in $E(Z)$. In first method you get this by $\beta_0 + \beta_1$. In second method you get from $\beta_1$ directly the change in $E(Z)$. To you second question: No there won't be multicollinearity issue here as you have removed the constant term. Think in terms of the $X$ matrix. Only when the first column is $1$ and including separate dummies for all levels of the categorical variables, will the matrix become singular. $\endgroup$
    – Dayne
    Oct 11 '20 at 17:46
  • 1
    $\begingroup$ Strictly speaking the above argument on $\beta_1$ is true when there is only one categorical value. But it conveys the idea. $\endgroup$
    – Dayne
    Oct 11 '20 at 17:49
  • 1
    $\begingroup$ In second method adding more variables won't change anything. It's still good. And yes making the constant zero does the job. To make this more clear, substitute $D_{x_1}=1-D_{x_2}$. Same for $y$. You'll get back first model with a constant. Second model is just the first model written differently to achieve a simpler desirable interpretation of coefficients. $\endgroup$
    – Dayne
    Oct 11 '20 at 18:08
  • 1
    $\begingroup$ Yes. As long as there are dummies for each level $\endgroup$
    – Dayne
    Oct 12 '20 at 2:13
  • 1
    $\begingroup$ In method 1, one dummy is less. Say you have not put a dummy for location $X$ and company $Y$. So $\beta_0$ is the mean value of rates when location is X and company is Y. Interpretation of $\beta_1$, which is coefficient of NYC's dummy, is that how much does the mean of rates (when location is NYC and company is Y) different from mean of the rates (when location is X and company is Y). In method two, coefficient of NYC dummy tells you how much is the average rates when location is NYC, regardless of what company is. $\endgroup$
    – Dayne
    Oct 16 '20 at 15:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.