Is it possible to model with a linear regression the relationship between a dependent continuous variable and an independent categorical variable?

Question

I raised this question after a project that I made on DataCamp.com where a linear regression was trained to model the relationship between movie genre revenues (dependent variable) and movie genre(independent variable). The first answer I gave to myself is No.

As a self-taught data scientist, I wanted to prove it, and I spent several days reading questions and trying to understand where formally was the fallacy.

Dataset Composition

We have a dataset containing some of the Disney Films since 1935 and their inflation-adjusted revenues, movie rating, and movie genre; the dataset is freely available here. The dataset is not complete just as an example Dumbo-Robin Hood-Bambi are missing.

Goal

Build a linear regression model to predict the adjusted gross using dummy variables obtained from movie genres. From the regression model, check the effect of each genre by looking at its coefficient given in units of box office gross dollars.

Process and Thesis

"Since linear regression requires numerical variables and the genre variable is a categorical variable, we will use a technique called one-hot encoding to convert the categorical variables to numerical. This technique transforms each category value into a new column and assigns a 1 or 0 to the column. For this dataset, there will be 11 dummy variables, one for each genre except the action genre which we will use as a baseline. For example, if a movie is an adventure movie, like The Lion King, the adventure variable will be 1 and other dummy variables will be 0. Since the action genre is our baseline, if a movie is an action movie, such as The Avengers, all dummy variables will be 0."

But mathematically what happens?

If we call $y$ our target, $\theta_i$ our coefficients and $x_i$ our dummy variables we will have the following equation: $$y=\theta_0*x_0 + \theta_1*x_1 + \theta_2*x_2 + ... + \theta_n*x_n $$ So what happens when we want to estimate the value of a genre, like an horror movie based on our hypothesis: $$y_{horror}=\theta_{action} + \theta_{adventure}*x_{adventure} + \theta_{musical}*x_{musical} + ... + \theta_{horror}*x_{horror} $$ The author used the action movies as baseline. So it follows: $$y_{horror}=\theta_{action} + 0 + 0 + ... + \theta_{horror}*x_{horror} $$

Answer 1

Yes - we call it an ANOVA, which is just linear regression with categorical predictors.

Answer 2

Linear regression is a linear model used to represent the relationship between one or more explanatory or independent variables and the target variable also called the dependent variable. Linear regression has some assumptions:

1. Variables should be normally distributed
2. Independent variable should have linear and additive relationship with the target
3. There should be no linear relationship among the independent variables
4. Target should not be autocorrelated
5. Homoscedasticity should not be there in data

The main reason why is not possible to model a linear regression between a Continuous Variable and a categorical variable is in point 2. In fact, the Pearson Correlation Coefficient will be 0 in that situation or something near to it. $$\rho_{XY}= {Cov_{XY}\over \sigma_x*\sigma_y}$$ The best estimator, the one that reduces the error between the estimated value and the actual value of a film, based on movie genre is the sample mean.

An intuitive representation of what happens when you try to model a linear regression with just a categorical variable is the following:

Moreover, instead of creating one singular estimator for the 12 genres, I created a specific estimator for each genre, and obviously, an exception in 1 case the MSE was lower.

Here the original notebook.

Here my notebook.

One further work is to conduct an ANOVA analysis and to evaluate if there is a sensible difference in the variance and mean of each movie genre. The same concept can be applied to the value of a house based only on the neighborhood/district where is located.

I will really happy if my demonstration is correct, or if I am missing something due to my lack of knowledge and If there is any gap I have to abridge.