Questions tagged [categorical-encoding]

Representing categorical variables as sets of numerical variables. Necessary in many types of analysis for them to process categorical data. A common example is using a categorical predictor in regression/ANOVA via dummy coding, effect coding, Helmert coding, user-defined contrasts, etc.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

Encode categorical variables with many labels

I am trying to predict a multiclass categorical outcome variable by comparing different classifier algorithms. I've got a dataset that includes two categorical variables that have many labels (>...
0
votes
0answers
12 views

Post-hoc test to check for direction of association between two categorical (nominal) variables

I am working with two nominal variables for the first time and am looking for an appropriate test for the same. In the example below, I want to know if there is an association between gender and ...
0
votes
0answers
14 views

Grouping categories of a variable with low count [closed]

I'm working with some data that has a bunch of categorical variables that I need to encode to fit my models. For the ones that have low cardinality, i.e. a low number of categories I can simply use ...
0
votes
0answers
4 views

Interpreting (partially) overlapping time dummies

In a time series linear regression, what is the interpretation dummies that identifies two partially overlapping periods? To be clearer, I have a time series regression and my sample period goes from ...
0
votes
1answer
15 views

Does keras include a was to turn a classifier's prediction into a classification? [closed]

I have a model where the output is a one-hot encoding of 6 classes, meaning y_train is of the shape (1000,6) model.fit(X_train, y_train, epochs=1, batch_size=10) ...
0
votes
0answers
29 views

OneHot Encoding: Nan value - remove row or represent as a list full of zeros?

In the analysis survey, I have responses without answers and I'm not sure what is a better approach. Most of the attributes are categorical and missing numerical attributes have to remove. What can I ...
0
votes
1answer
21 views

Am I coding dummy variables for disease groups correctly in logistic regression?

Normally I would code my dummy variables as follows: Original variable levels Dummy_disease1 Dummy_disease2 Dummy_disease3 Disease 1 1 0 0 Disease 2 0 1 0 Disease 3 0 0 1 However, I dont want to ...
0
votes
0answers
33 views

How do get rid of (1 not defined because of singularities) in R? [duplicate]

I'm analyzing data in R, I'm trying to see how some variables affect test scores (Value) of different countries. In the data, since there is different time periods for different countries I need to ...
3
votes
1answer
19 views

Should I include month of interview dummies in regression?

I have a quick question regarding dummies for a regression that I am running. I am running regressions of responses to a survey question about health on a vector of independent variables (e.g., ...
1
vote
1answer
31 views

Create composite score from categorical and binary variables

Let's say I have a dataset with a number of variables on clinical history and behaviours in the context of COVID transmission. Ultimately i'd like to create a binary variable that is an indicator of ...
1
vote
1answer
20 views

One hot coding in Train Validation and Test set (Production data) [closed]

For example I have below train set. name values 0 Tony 100 1 Smith 110 2 Sam 120 3 Shane 130 4 Sam 140 5 Ram 160 After ...
0
votes
0answers
12 views

Interpretation interacting dummy variables regression

Suppose I have a panel dataset with a time and cross-sectional dimension. I split the time-dimension into three intervals (Interval A, B and C). Furthermore, I split the cross-sectional dimension into ...
0
votes
0answers
9 views

Using and interpreting categorical dummy interaction variables in regression

I'm trying to use interaction terms for dummy categorical variables within a regression, and it's confounding my brain. I'm modelling recruitment, and have a few categorical variables: Age, with 4 ...
0
votes
0answers
15 views

How to interpret regression results when there are many dummy variables, not many catergories in one dummy variable

I am confused my regression results. My data is something like this. I want to see whether the presence of these six species would have significant effects on TD. So, I make every of them be dummy ...
0
votes
0answers
7 views

Best Way To Encode Categorical Variable To Capture Impact of Each Unique Value For Tree Based Model When Data has Collinearity

I'm working on a project right now where I'm looking to use XGBoost to model a binary classification problem and use feature importances to look at the relative importances of group characteristics in ...
0
votes
1answer
20 views

multiple regression model with multiplication between independent variable and a dummy variable

I was asked to build a linear regression model with multiplication, in the iris dataset in R. $Sepal.Length_i = \beta_0 + \beta_1 \cdot Petal.Length_i \cdot Species_i + \epsilon_i$ now I know in R , ...
0
votes
1answer
22 views

Categorical variable more than two levels

Say for the sake of argument I have a categorical variable called race. The variable has white, black, and Asian levels. I make two dummies (dummy variables) White and Asian (for variable white if 1 ...
1
vote
0answers
15 views

Dummy Coding Multiple-Select Race/Ethnicity

I'm performing some survey research and am looking to include race/ethnicity as an independent variable in a regression model. Normally, race/ethnicity would be added to the model as a set of dummy ...
0
votes
0answers
13 views

When to apply target encoding ( before or after target transform)?

I am trying to build a linear regression model. I have some high cardinal categorical features on which I want to apply target encoding. But my target (real-valued) variable distribution is highly ...
1
vote
0answers
12 views

How to measure agreement with categorical labels, and with multiple binary labels?

I have a dataset in which 7 coders have given a categorical label to each of 152 objects. The same 8 categories are selected between for all objects. I would like to measure the agreement between ...
0
votes
0answers
13 views

Shall a one-hot coding applied before or after the missing values imputed?

For the case when the categorical data is handled, it is suggested to one-hot encode the values so as to digitize the value of the data. Many examples taking color (green, red, blue) as an example so ...
1
vote
0answers
13 views

To encode large number of IDs (100+) as feature what type of encoding is to be done?

I am doing a project on predicting prices of air fare based on features like destination airport ID ,source airport ID and carrier name I have a need to encode these values but the amount of distinct ...
1
vote
1answer
16 views

Interpreting logistic regression coefficients of a variable overall and levelwise

Context Let Y be a logical vector and X1 a factor with 3 levels. Since Y is binary, logistic regression is used. ...
1
vote
1answer
6 views

How to merge/encode a categorical feature's unique values in a regression problem

A feature contains more than 10 unique values in my case, and I want to merge them to improve my model speed. The problem is I don't know how to merge them in a scientific way. Now, my idea is to ...
1
vote
2answers
34 views

Interaction term with categorical predictors with more than one level

I am looking at the interaction between education (LTHS, HS, SOME COLLEGE, COLLEGE) and race (WHITE, BLACK, HISPANIC, ASIAN, NATIVE AMERICAN). I am using SAS and am including a class statement in my ...
0
votes
1answer
10 views

Joining train and test dataset when preprocessing data with encoders

So I've been looking at several beginner Kaggle kernels for the 'Titanic - Machine Learning from Disaster' competition. In these kernels, I noticed that sometimes they combine train and test data, ...
0
votes
1answer
25 views

braking up multiclass classification

I am working on a data set that has tabled data to 5 different classes. I would like to train an algorithm (logreg most likely) to predict cluster affiliation. but because there are 5 classes all the ...
0
votes
0answers
19 views

How to deal with features encoding and normalization for deployment of ML model?

I have a dataset that has many different features such as categorical, ordinal and continuous ones. Categorical Features I have great difficulty understanding how should I apply label encoding to ...
0
votes
1answer
19 views

Can I make one dummy variable for a multiple categorial variable?

I am running a multiple regression model. Here is my question. I want to create a dummy variable for the following categorical variable I have 4 categories for education. One is school, high school, ...
1
vote
0answers
10 views

Is it like the concept of dummy trap doesn't exist is machine learning apart from Logistic Regression & Multiple Linear Regression?

While performing MLR & Logistic Regression model summary analysis I have seen the problem of perfect multicolinearity if we use one hot encoding without dropping a single feature. Is it necessary ...
0
votes
1answer
41 views

Time effects with dummy variable - regression

I am doing a multiple regression analysis and I wanted to inspect the time effect by using factor(Year) in R. However, I got the following summary results: Do you ...
0
votes
0answers
21 views

Binary Logistic Regression using Non-Dichotomous Independent Variables; SPSS [duplicate]

I'm using SPSS for binary logistic regression but one of my independent variables is non-dichotomous and categorical (genotype - there are 5 gene combinations possible). I'm trying to see if a ...
0
votes
0answers
18 views

Binary Logistic Regression using SPSS for Non-Dichotomous Independent Variables

Im using SPSS for binary logistic regression but one of my independent variables is non-dichotomous and categorical (genotype - there are 5 gene combinations possible). I am trying to see if a ...
0
votes
0answers
12 views

Help with identifying regression model for sequences

Let's say I have a regression task, but features are sequences of "letters", so that the order is important. The set of all possible letters is relatively small. Example: ...
0
votes
1answer
31 views

Impute missing values of dummy variables, using R's {caret} package: predicted values in between {0;1}?

I'm using {caret} to impute missing data resulting from non-response to survey questions. All of these variables are defined as numeric, though most are dummies. ...
3
votes
1answer
29 views

One Hot Encoding of ranges of data vs. leaving data as is for Logistic Regression

Recently whilst doing an assignment using the PIMA Diabetes set I ran Logistic Regression using, amongst others: the age predictor as is segmented the age into ranges and applied OHE (with and ...
0
votes
0answers
12 views

Recoding income variables in trends analysis

I will greatly appreciate your advice. I will like to control for family income in my analysis of trends in self-rated health between year 2000 and 2018. The NHIS data from IPUMS that I'm using has ...
0
votes
1answer
15 views

Recalculate the standard error using a different base?

I want to run a GLM with a factor, say car type, as one of the independent variables. Suppose car type has the following levels: sedan, SUV, and truck. And suppose the base level is currently sedan. ...
0
votes
1answer
13 views

Random effects vs one-hot encoding

Say I have an independent variable category with 3 possible values (a,b,c) and this list is exhaustive (category can only take 1 ...
0
votes
0answers
14 views

How can I create a partial year dummy variable for a panel dataset?

This might be a dumb question, but here goes. I have a panel dataset that includes annual firm-level data from 2010-2020. I am interested in the effect of a specific policy change (X) in my outcome ...
1
vote
0answers
16 views

Interpreting the covariate p-values in a multivariate generalized linear model?

If a covariate in a GLM is "significant" does that mean it is significantly different from the base case (the group not shown)? Say we have three groups, Control, Exp1, Exp2. We are ...
1
vote
0answers
34 views

Dummy variables and the weekend effect

I am running a regression to see the well documented "weekend effect" in the stock market. The weekend effect is a phenomenon in financial markets in which stock returns on Mondays are often ...
2
votes
1answer
49 views

Adding a Dummy Variable to glm in R?

I'm running a glm in R with two categorical variables, one of which is binary, the other of which can take on five values. I would like it so that my model returns an intercept value that reflects the ...
0
votes
0answers
20 views

Label-encoding nominal variables

I am aware of the practice that label encoding is preferred for ordinal variables while ...
0
votes
0answers
14 views

Converting continuous predictor to category e.g. Age [duplicate]

I notice that on many examples one is keen to convert Age to a categorical age range. I am wondering if that is always necessary. The famous golf play decision tree example has ranges for temperatures ...
0
votes
0answers
20 views

Handling categorical variables in logistic regression

Let's say I am fitting a logistic regression to dependent variable gender: Gender M F M Since I am using python's scikit-learn implementation of logistic regression to fit a model, I create dummy ...
2
votes
1answer
62 views

Categorical independent variable and binary dependent variable

Which test can I use for analyzing the effect of a categorical independent variable, such as preoperative ASA score (1/2/3/4), on a binary dependent variable, such as postoperative complication (yes/...
2
votes
2answers
46 views

Maths Behind Dummy Variable in Linear Regression (One Hot Encoding)?

I understand the logic behind using k - 1 dummy variables for K Categories (multi collinearilty etc) but trying to understand how the math behind it works. Consider the following Example: We Code: ...
1
vote
1answer
28 views

How to code non-exclusive variables for logistic regresion

I have a question about logistic regression in R. I want to study the influence of certain comorbidities in patients in predicting deceased status(Y/N). So far, I formatted all my comorbidities(17) ...
1
vote
1answer
37 views

Logistic or convert it to dummy?

My dependent variable is a Likert scale categorical variable. I've already run a first set of ordinary logistic regressions - however, I was wondering whether it is common practice to run the ...

1
2 3 4 5
13