2
$\begingroup$

For an indepdent numerical variable X the B1 coefficient is COV(X,Y)/Var(X).

Since Categorical Variables don't have things like Means(from which things like COV and VAR are derived) how would it work to derive a simple linear regression for an independent binary categorical variable? What does its formula look like? How would it look like going to multiple linear regression? Thank you.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Since Categorical Variables don't have things like Means

This is solved if you dummy code the categories. Consider a binary variable, for clarity A or B. If I recode this variable so that 0 is A and 1 is B then I get something I can easily compute the mean, variance, and covariance for.

Simple example using the mtcars data in R. This data has transformed the column vs to be 0 when the engine is not v shaped and 1 when the engine is v shaped. Let's compute the regression coefficient of vs on mpg using the formula you provided and the lm function

vs <- mtcars$vs
mpg <- mtcars$mpg

fit <- lm(mpg ~ vs)
coef(fit)['vs']
#>       vs 
#> 7.940476


cov(mpg, vs) / var(vs)
#> [1] 7.940476

Created on 2024-04-02 with reprex v2.0.2

Unsurprisingly, they are the same.

Improve this answer
$\endgroup$
5
  • $\begingroup$ So when we dummy codify the binary categorical variable and calculate through those where the mean basically is a proportion, we can use that in a regression model as if it would interpret it as a numerical value? $\endgroup$
    – Mandem
    Commented Apr 3 at 10:45
  • $\begingroup$ Regarding the Variance, from my understanding for proportions the Variance is p(1-p)/n. In this regression case would we calculate the Variance as we normally do with a numerical variable? (X-Xbar)^2 / n-1 ?? Thank you $\endgroup$
    – Mandem
    Commented Apr 3 at 10:59
  • $\begingroup$ @Mandem Yes, when you dummy code you map the category to a numerical value. The variance calculations should use the sample variance (i.e. n-1 in denominator) since you don't know the true population variance. $\endgroup$
    – Demetri Pananos
    Commented Apr 3 at 13:50
  • $\begingroup$ I meant as in does the simple linear regression model use p(1-p)/n-1 or (X-Xbar)^2 / n-1 . Sorry if my previous question wasn't clear. $\endgroup$
    – Mandem
    Commented Apr 3 at 14:19
  • $\begingroup$ (X-XBAR) since you don't know the value of p. It needs to be estimated $\endgroup$
    – Demetri Pananos
    Commented Apr 3 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.