0
$\begingroup$

I was searching for methods for handling missing values in case of Regression task. There are already few threads but I couldn't find what I was looking for. Suppose I have 4 independent categorical variables (A,B,C,D), each of them can takes an integer value and their limit is pre-defined. For example A can take values from 1-10, B from 2-7, etc..All are categorical variables. I want to train a regression model on these variables. Suppose during the test phase, if I don't have the value of a particular variable, how should I approach the problem. I was thinking, in lines of having priors for each variable from the training data and using them during test phase. Please share any better methods.

Thanks

$\endgroup$
1
$\begingroup$

If a variable won't be available during the test phase, it's best to just exclude it from the model completely, as you need to make the best decision available with the information you can.

The best decision in absence of knowledge of your variable will likely use the other variables quite differently, if any of the variables are at all correlated with your missing one.

$\endgroup$
1
  • 1
    $\begingroup$ This will be very inefficient if the variable has any predictive value. Multiple imputation is strongly preferred. Here is a reference that studies several approaches to the problem: citeulike.org/user/harrelfe/article/13265778 $\endgroup$ May 3 '15 at 21:15
0
$\begingroup$

With categorical variables and linear regression I believe you should transform the variables into dummy variables. For example let's say you have a week_day column. And in this column you can have a any value of {1,2,3,4,5,6,7}. These values represent each day of the week (Monday = 1, Sunday = 7).

You do not want to perform linear regression with one column of categorical values. Rather you would want to make 6 columns, because no one category is more important than the other. Each column will be filled with boolean (true/false) values. And the number of dummy variable columns is 6 because n-1 = 6 where n is the number of categories in the week_day column. Here is a simple view of the column being turned into dummy variable columns.

transform one categorical variables into dummy variables

The reason for n-1 is that if all dummy variables for a row of data are 0 then it is the case that all variables are false. So if Monday - Saturday are all false Sunday must be True, but we don't need an explicit column for Sunday.

You can use PANDAS with python to automatically produce dummy variables. However your data must be in a PANDAS dataframe.

The reason to transform categorical variables this way is to ensure linear regression does not give false ranking to the categories. In the week_day column, regression might give 6 (Saturday) more weight than 1 (Monday) simply because the value is higher, but in reality monday is not more or less important than sunday.

So for the case of you data set, you would want to create dummy variables for all independent categorical variables.

When it comes to dealing with missing data/NANs. I would say if ~10% of your data is NANs and your data set is sufficiently large enough then you might be able to just discard that NAN data. If not you can use data interpolation or imputation to fill in NANs. In another CrossValidated post I provide some good references and documentation for getting started with both interpolation and imputation. Hope this helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.