Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [categorical-encoding]

Representing categorical variables as sets of numerical variables. Necessary in many types of analysis for them to process categorical data. A common example is using a categorical predictor in regression/ANOVA via dummy coding, effect coding, Helmert coding, user-defined contrasts, etc.

45
votes
4answers
11k views

What is a contrast matrix?

What exactly is contrast matrix (a term, pertaining to an analysis with categorical predictors) and how exactly is contrast matrix specified? I.e. what are columns, what are rows, what are the ...
25
votes
7answers
35k views

Why is gender typically coded 0/1 rather than 1/2, for example?

I understand the logic of coding for data analysis. My question below is on the use of a specific code. Is there a reason why gender is often coded as 0 for female and 1 for male? Why is this coding ...
19
votes
3answers
25k views

When should one use multiple regression with dummy coding vs. ANCOVA?

I recently analyzed an experiment that manipulated 2 categorical variables and one continuous variable using ANCOVA. However, a reviewer suggested that multiple regression with the categorical ...
18
votes
2answers
40k views

Significance of categorical predictor in logistic regression

I am having trouble interpreting the z values for categorical variables in logistic regression. In the example below I have a categorical variable with 3 classes and according to the z value, CLASS2 ...
17
votes
4answers
21k views

How to recode categorical variable into numerical variable when using SVM or Neural Network

To use SVM or Neural Network it needs to transform (encode) categorical variables into numeric variables, the normal method in this case is to use 0-1 binary values with the k-th categorical value ...
16
votes
3answers
24k views

Why do we need to dummy code categorical variables

I am not sure why we need to dummy code categorical variables. For instance, if I have a categorical variable with four possible values 0,1,2,3 I can replace it by two dimensions. If the variable had ...
15
votes
1answer
3k views

How to treat categorical predictors in LASSO

I am running a LASSO that has some categorical variable predictors and some continuous ones. I have a question about the categorical variables. The first step I understand is to break each of them ...
14
votes
2answers
19k views

Qualitative variable coding in regression leads to “singularities”

I have an independent variable called "quality"; this variable has 3 modalities of response (bad quality; medium quality; high quality). I want to introduce this independent variable into my multiple ...
14
votes
2answers
16k views

How to do regression with effect coding instead of dummy coding in R?

I am currently working on a regression model where I have only categorical/factor variables as independent variables. My dependent variable is a logit transformed ratio. It is fairly easy just to run ...
14
votes
2answers
6k views

“Dummy variable” versus “indicator variable” for nominal/categorical data

"Dummy variable" and "indicator variable" are labels frequently used terms to describe membership in a category with 0/1 coding; usually 0: Not a member of category, 1: Member of category. On 11/26/...
14
votes
1answer
904 views

What are the different types of codings available for categorical variables (in R) and when would you use them?

If you fit a linear model or a mixed model there are different types of codings available to transform a categorical or nominal varibale into a number of variables for which paramaters are estimated, ...
11
votes
2answers
15k views

Regression based for example on days of week

I need a little help to move in the right direction. It's a long time since I studied any stats and the jargon seems to have changed. Imagine that I have a set of car-related data such as Journey ...
11
votes
1answer
6k views

Dropping one of the columns when using one-hot encoding

My understanding is that in machine learning it can be a problem if your dataset has highly correlated features, as they effectively encode the same information. Recently someone pointed out that ...
10
votes
6answers
4k views

What is “one-hot” encoding called in scientific literature?

What is the name of the operator that takes a categorical vector and transforms it to the binary representation using one-hot encoding? I am wondering since I am writing a scientific paper and need a ...
10
votes
2answers
48k views

Understanding dummy (manual or automated) variable creation in GLM

If a factor variable (e.g. gender with levels M and F) is used in the glm formula, dummy variable(s) are created, and can be found in the glm model summary along with their associated coefficients (e....
10
votes
3answers
635 views

Indicator variable for binary data: {-1,1} vs {0,1}

I am interested in treatment-covariate interactions in the context of experiments/randomized controlled trials, with a binary treatment assignment indicator $T$. Depending on the specific method/ ...
10
votes
3answers
102k views

How to deal with non-binary categorical variables in logistic regression (SPSS)

I have to do binary logistic regression with a lot of independent variables. Most of them are binary, but a few of the categorical variables have more than two levels. What is the best way to deal ...
9
votes
4answers
14k views

How to implement dummy variable using n-1 variables?

If I have a variable with 4 levels, in theory I need to use 3 dummy variables. In practice, how is this actually carried out? Do I use 0-3, do I use 1-3 and leave the 4's blank? Any suggestions? ...
9
votes
1answer
7k views

R linear regression categorical variable “hidden” value

This is just an example that I have come across several times, so I don't have any sample data. Running a linear regression model in R: a.lm = lm(Y ~ x1 + x2) <...
8
votes
4answers
2k views

How to statistically prove if a column has categorical data or not using Python

I have a data frame in python where I need to find all categorical variables. Checking the type of the column doesn't always work because int type can also be ...
8
votes
2answers
537 views

Why does treatment coding result in a correlation between random slope and intercept?

Consider a within-subject and within-item factorial design where the experimental treatment variable has two levels (conditions). Let m1 be the maximal model and <...
7
votes
1answer
6k views

Dummy coding for contrasts: 0,1 vs. 1,-1

I'm seeking your help in understanding the difference between two different contrasts for dichotomous variables. On this page: http://www.psychstat.missouristate.edu/multibook/mlt08.htm under "...
7
votes
2answers
4k views

Linearity Assumption in OLS with Dummy Variables

Let's say that I have a continuous response variable and have constructed a regression model with multiple predictors. Most of my predictors are continuous but I have one which is a dummy variable. ...
6
votes
2answers
8k views

How to choose number of dummy variables when encoding several categorical variables?

I'm building a logistic regression, and two of my variables are categorical with three levels each. (Say one variable is male, female, or unknown, and the other is single, married, or unknown.) How ...
5
votes
2answers
668 views

Coding as a categorical or continuous variable?

I have a question/IV in my study which has been answered: 1- No 2- Do not know 3- Sometimes 4- Yes I was advised to remove the answer 2 (all do not know ...
5
votes
3answers
2k views

Coding categorical variables for regression

I'm not sure of the best way to code my categorical predictor variable for use in a hierarchical regression in order to test my specific hypothesis. This categorical variable has 3 levels representing ...
5
votes
2answers
136 views

With two related variables, eg, religion and religiosity, how do I transform them into one variable for regression?

Say I have the nominal variables of religion (0=Athiest; 1=Christian; 2=Jewish; 3=Muslim; 4=Other) And then a scale variable of religiosity from 1-10 If I want just one scale variable, so that I can ...
4
votes
3answers
688 views

How does coding matter for categories?

So say the predictor variable is coded 1,2,3,4 for 4 different cities. Is this bad? I've heard that it only makes sense for things that have a natural ordering. Like number of stars for a movie or ...
4
votes
1answer
145 views

Eicker-Huber-White Robust Variance Estimator

In a regression context, $$ Y_i = \alpha + \beta T_i + \varepsilon_i $$ my textbook defines EHW robust variance estimator as $$ \widehat{\mathbb{V}_{\rm EHW}}(\widehat{\alpha}, \widehat{\beta} | \...
4
votes
1answer
465 views

Is there something called “mean coding” (like dummy coding & effect coding) in regression models?

When we perform a regression analysis with categorical predictors, we can use (1, 0), called "dummy coding". The coefficients in this case represent the deviation of the groups' means from the mean of ...
4
votes
1answer
2k views

Encoding of categorical variables (dummy vs. effects coding) in mixed models

The model based on the experiment looks like this: ...
4
votes
1answer
120 views

When is deviation coding useful?

After many years of learning about contrasts in linear models I am curious about the relative usefulness of deviation coding, as it is defined by this website. I would appreciate someone filling me in ...
4
votes
1answer
256 views

VIF Drops Significantly When I Delete Some Dummy Variables

Is my model valid even with the high VIF? Does it matter which dummy variable I drop as the reference point? I have a a category variable (Fruit) that I converted ...
4
votes
1answer
299 views

What is the appropriate zero-correlation parameter model for factors in lmer?

When one wants to specify a lmer model including variance components but no correlation parameters, as opposed to m1, for a ...
4
votes
1answer
389 views

Design matrix contrast coding for model selection and 'main effects' vs. 'simple main effects' interpretation in linear mixed effects model (R/Matlab)

My question is about contrast coding and planned contrasts in three-way interactions for a linear mixed effects model. Sample code is provided for R and Matlab as I can work in either one, but prefer ...
4
votes
0answers
140 views

Dropping columns from an orthogonal design matrix?

Hello: I’m working with a three factor (ANOVA) design that I wish to use in an MCMC chain to estimate the parameters for the main effects and treatment interactions. I wish to run MCMC analyses ...
4
votes
0answers
2k views

Effects Coding in R

I am in interested in how do effect coding in R. I know that someone else has asked this question (i.e How to do regression with effect coding instead of dummy coding in R?). Here is the lm() model on ...
3
votes
1answer
303 views

Can you have interaction terms for both “sides” of a dummy variable in a single regression?

I'm really not sure how to phrase my question properly, so I apologize if this has been answered elsewhere. Let's say I'm interested in using a regression to predict wage using sex and an interaction ...
3
votes
1answer
111 views

Adding a correlated variable to the regression?

I am running a regression like below: $Y = \beta_0+\beta_1X_1+\beta_2X_2 +\beta_3X_3+\epsilon$, where $X_1$ is a gender dummy and the result shows it has NO statistic significance. Then I added one ...
3
votes
1answer
40 views

Factor for rows without attribute and rows with attribute and additional characteristic

I have run an experiment in which I've measured some metric X, and collected an associated attribute attr's value if and only if the value of X exceeds some ...
3
votes
1answer
969 views

Non Numeric features in Logistic Regression

I understand that the fitted values for Logistic Regression can be expressed as: $$P(Y_i=1) = \left(1+\exp(-\hat{\theta}^TX_i)\right)^{-1}$$ where $X_i$ is the feature vector, which will work well ...
3
votes
1answer
69 views

“Joint” dummy variables for two different variables

I am supposed to show the hazard ratio (HR) stratified by gender (1= female vs. 2= male) and age groups (quartiles, 1-4)*. The combination "female" and "first quartile of age" is supposed to be the ...
3
votes
1answer
440 views

Too many dummy variable in regression model

we have about 50000 models of mobile phone (like Galaxy S7, iPhone 9) in database and the size of data is about 3 million. We want to find the mobile phones that have the least call success rate ( ...
3
votes
1answer
914 views

How to handle too many categorical features with too many categories for XGBoost?

In my data I have 35 features and 14 of them are categorical. Half of them have 3 to 4 categories but others have 14 to 28 categories. One Hot Encoding them would only lead to a sparse matrix with ...
3
votes
1answer
49 views

Fitted values of a simple regression with intercept and dummy

Why are the fitted values of a simple regression with intercept and dummy, estimated by OLS, just the group means belonging to the two groups of observations? I.e., why do we have that the fitted ...
3
votes
1answer
89 views

Feature representation for feature set clustering

I'm studying customer requirements clustering. Each customer's requirements are collected as a set of application features. I'd like to cluster those set of features, so that I can know what are the ...
3
votes
3answers
299 views

Can a dummy variable take on more than 2 values?

I am doing a research on foreign direct investment in the EU countries. I came across an article in which the authors assign 4 values to a dummy variable, to be more specific, they assign the value 0 ...
3
votes
1answer
37 views

WLS vs Dummy variable coding for heteroscedasticity

I am a beginner level stat learner (with graduate training in Applied Math) I have just read a sage book which states the following: "Dummy variables help address the issue of heteroscedasticity in ...
3
votes
1answer
461 views

Regression and contrast codings with multiple categorical variables

In regression with multiple explanatory categorical variables, how should I model the problem to compare the effects of the categorical variables with each other? Most contrast coding schemes (e.g. ...
3
votes
1answer
797 views

categorical independent variable with three levels and binary logistic regression

I am not very experienced in statistics that's why you may find my question stupid. Anyway, I want to learn which level of a categorical independent variable should I look to interpret the odd ratios ...