Questions tagged [categorical-data]

Categorical (also called nominal) data can take on a limited number of possible values called categories. Categorical values "label", they do not "measure". Please use [ordinal-data] tag for discrete but ordered data types.

Filter by
Sorted by
Tagged with
10
votes
2answers
15k views

Best practices for coding categorical features for Decision Trees?

When coding categorical features for linear regression, there is a rule: number of dummies should be one less than the total number of levels (to avoid collinearity). Does there exist a similar rule ...
2
votes
2answers
356 views

How to interpret Pr(>|t|) of factor variables?

How to interpret Pr(>|t|) of factor variables? The reason asking is the following: ...
0
votes
0answers
10 views

Ideas for categorical data when analyzing OECD's Better Life Index

This is a really rookie question, but bear with me. I am statistic student and I am trying to do a project for school using the Better life Index data. I have got to the part where I need to do ...
10
votes
2answers
3k views

Is multicollinearity implicit in categorical variables?

I noticed while tinkering with a multivariate regression model there was a small but noticeable multicollinearity effect, as measured by variance inflation factors, within the categories of a ...
0
votes
0answers
10 views

What statistical test will measure if a binary variable is significantly different between groups? [closed]

Number of classes/categories/groups: A = weight ≤ -3 B = -3 < weight ≤ -2 C = -2 < weight Number of observations: ~100 observations Number of features: ~20 features that are each binary (0=...
3
votes
1answer
34 views

Post-hoc t-tests for ANOVA

My sociology professor said that when performing multiple t-tests between two groups after the ANOVA f-test, the likelihood to make at least one type-1 error adds up by 5% with each t-test. So for one ...
0
votes
1answer
26 views

Can I apply PCA on continuous data and reduce the dimensions and keep categorical data as it is?

I have a dataset which contains 95 highly correlated continuous variables and other 3 categorical variables. I want to reduce the dimension of the data and by that I can deal with correlation as well. ...
0
votes
1answer
133 views

Isolation forest with categorical data?

I understand how isolation forests can work with numeric data, but I wonder how it can work with categorical data? Also, at least when working with Sci-kit-Learn, the recommendation I saw was to ...
0
votes
0answers
118 views

Coding the regresssion model with several binary variables

Say I have 5 binary variables and 2 normal variables. I want to get the probability of success, say one of the variable 1 or 0, 1 for success. How can I do that? I tried ...
0
votes
0answers
20 views

Machine learning classifier with only categorial predictors [closed]

I have a data set with a binary outcome variable and only binary dummy predictors. Which are the best algorithms for this type of classification task? Is there a code for plotting the results (i.e. ...
7
votes
1answer
2k views

glmnet, categorical variable, group lasso?

I am using glmnet for LASSO. My data set contains several continuous variables and one categorical variable (it has four levels). I wondered if I could treat three dummy variables as other continuous ...
3
votes
1answer
154 views

Sample size when fitting categorical survey data

I have a model which fits data from repeated surveys: at time $t$, a number $n_t$ respondents is asked a question and can give one of $K$ answers ($k=1, ..., K$). This is repeated $T$ times ($t = 1, .....
1
vote
0answers
12 views

Exploratory Factor Analysis (EFA) within a CFA framework in R: How to select anchor items?

In cases, data is ordered categorical, exploratory factor analysis (EFA) is best implemented using polychoric correlations and diagonally weighted least squares (for example, see here). To my ...
0
votes
0answers
10 views

Dealing with missing levels in categorical variables when batch scoring in Python [closed]

I have encountered a unique problem. My model was trained on DNN framework and the model parameters are saved which I'm using now to score the data. Since my data is quite huge, I'm scoring the data ...
2
votes
1answer
44 views

How to choose between a dummy variable and the amount of a variable that has a lot of zero

I am trying to include several income types into a regression model (specifically a logit model). This variable has the particularity to have numerous zero for some type of incomes (typically some ...
0
votes
0answers
12 views

Multilayer Perceptron with XOR Dataset

so I got this working code for a multilayer perceptron class, which I'm training on a XOR dataset (plot 1). As activation I'm using the hyperbolic tangent. After 50000 training epochs using SGD, my ...
0
votes
0answers
13 views

Listing factors and their corresponding R for a dataset [closed]

I have a dataset (Skewed Left) with customer survey scores and then a mix of mainly categorical and some numerical features. I am attempting to see how any of these features correspond to the ...
1
vote
0answers
20 views

Categorical variable from continuous variable: should I aim for the same number of observations?

I am converting a continuous variable to a categorical variable with the aim to use it as a factor in an ANOVA test. Having to decide if each level of the factor will include the same number of ...
2
votes
1answer
435 views

Using an overall category as a reference group for dummy variables

I have data on the unemployment rate within 3 education groups for different states, and some other continuous data on for the given states e.g. GDP. I also have the overall unemployment rate for the ...
1
vote
1answer
2k views

How to get 95% CIs for standardized regression coefficients?

I am running multiple linear regression with categorical variables and I need confidence interval 95% for standardized regression coefficient. I searched around and found 2 methods: Using the ...
1
vote
1answer
215 views

How to combine categorical features to predict continuous output

I have many categorical features that are non-nominal and also continuous features with continuous output. Some of the categorical features are binary and others have 10+ classes. How would you go ...
1
vote
0answers
34 views

Autocorrelation for a categorical time series

I am having trouble to compute the autocorrelation for different lags in a categorical time series. For instance, consider 3 possible categories: classA, classB and classC, and a vector x representing ...
3
votes
1answer
3k views

Scaling categorical data in regression

It seems odd to scale a categorical variable, but I need to get the correct coefficients for each of my variables in linear regression. Is it correct to scale the same way you would with continuous ...
0
votes
2answers
176 views

Finding more statistical way to group categorical data together

I need some help finding a better way to statistically group data for a project I am working on. I have a data set of individuals with different skill sets. Each individual can have only 1 or ...
0
votes
1answer
20 views

What Chi Square test should I use?

I am attempting to assess the ecological validity of an animal testing procedure as follows: I have notated x number of events that occur in the wild. During each event, the speed of the animal is ...
1
vote
1answer
195 views

Capturing variability using ANOVA on date

I have a dataset of traffic count at several intersections at various dates. Most of the intersections were counted only once. I want to know if there is a significant daily and monthly variability in ...
0
votes
1answer
127 views

How to Analyze Rank Data

Suppose I have 3 columns and n rows. Each row represents a school and the columns represent variables of interest and have ordinal ranks (1,2,3,4....n; 1 being the best and n being the worst). ...
0
votes
0answers
12 views

Bootstrap hypothesis test of sample difference for a general function f(x)

I have two samples, $x$ and $y$. $x$ is generated from the multinomial $Mult(n_x, \pi_x)$ and y from $Mult(n_y, \pi_y)$. I am interested in the quantities $f(\pi_x)$ and $f(\pi_y)$. Specifically, I ...
0
votes
0answers
15 views

multivariate analysis with mixed numerical and categorical dependent variables

I have one independent variable (yes/No) and several covariates (sex/age and so on..), five dependent variables (four continuous, normal distributed and one categorical as yes/no). I want to look up ...
1
vote
1answer
24 views

What statistical test should be performed in this setting?

I am analyzing a dataset. There I have 3 different "Lab test report findings" and 1 "clinical findings" which is obtained by the clinical examination of the patient (most of the time this clinical ...
1
vote
0answers
25 views

Chi square for pass fail survey test

I am trying to perform a statistical test on a survey I did. From my search, I found that Chi-square is the appropriate statistic for such data in the form of pass/fail. The test basically is 3 ...
2
votes
2answers
656 views

Standardizing dummy variables for variable importance in glmnet

I've used glmnet to build a binomial logistic regression model and I'm now trying to determine the importance of the variables in the model. I've read a few posts ...
19
votes
1answer
9k views

Dropping one of the columns when using one-hot encoding

My understanding is that in machine learning it can be a problem if your dataset has highly correlated features, as they effectively encode the same information. Recently someone pointed out that ...
3
votes
1answer
46 views

Testing differences in variance between groups

I have a hypothesis that a particular intervention/treatment will cause more variation in participant responses to a particular question. The intervention variable is categorical, with five different ...
0
votes
2answers
269 views

When do we choose time as a categorical or continuous variable in longitudinal MLM?

I understand that we can use time as a categorical or continuous variable in multilevel models (MLM). In which cases would it be better to use time as a categorical variable? And how is the analysis ...
1
vote
1answer
139 views

Is there a way to use cor function with factor variables without creating dummy variables? (R)

I have a dataset with several categorical predictors with varying factor levels. Is there a way to generate a correlation matrix from this data without having to create a bunch of dummy variables? I'...
10
votes
4answers
1k views

What is this diagram called

Can anyone tell me what is the name of this type of diagram ( if any )? Also can anyone suggest any tools, however simple, to plot such a diagram?
2
votes
1answer
21 views

Acute kidney injury statistical tests

I am fairly new to statistical analysis and was hoping to get some advice on an analysis I am hoping to run. I have data for children with acute kidney injuries (AKI) classified as a multilevel ...
0
votes
1answer
239 views

How to choose variables for ANOVA in R

I have a data set where I try to analyse a continuous variable according to 10 categorical variables and I would like to perform a ANOVA analysis. How should I proceed ? I'm able to interpret the ...
1
vote
0answers
15 views

Correlation of categorical data to binomial response in R

I'm looking to analyze the correlation between a categorical input variable and a binomial response variable, but I'm not sure how to organize my data or if I'm planning the right analysis. Here's my ...
0
votes
0answers
21 views

How does decision tree divide numerical feature? [duplicate]

As Shown in above decision Tree, sklearn's DecisionTreeClassifier divide numerical features to create decision tree. Petal length feature has following properties: ...
0
votes
0answers
7 views

Recursive Feature Engineering with Categorical and Continuous Variables

I'm trying to determine what to do with categorical feature when using recursive feature selection. I've looked around this forum and elsewhere and most discussions focus on one-hot-encoded features ...
0
votes
1answer
133 views

transformation for binary and categorical independent variables

I have a large dataset in which only Y and one of the independent variables are continuous. There are 12 binary independent variables and 2 other categorical independent variables (each with 8 ...
0
votes
1answer
20 views

Comparison of categorical variables with 3+ levels between two groups

Apologies if this is a simple question, but I can't seem to find an answer. I'm hoping to compare two categorical variables, one with 2 levels and another with 6, summarised here: ...
0
votes
0answers
17 views

Can I treat my continuous variable as a categorical variable?

I know that there many dangers and disadvantages of treating a continuous variable as a categorical variable. However, I also read in some cases it is applicable (e.g. when the relationship is non ...
1
vote
0answers
7 views

BERT for non-textual sequence data

I'm working on a deep learning solution for classifying sequence data that isn't raw text but rather entities (which have already been extracted from the text). I am currently using word2vec-style ...
17
votes
2answers
8k views

Feature importance with dummy variables

I am trying to understand how I can get the feature importance of a categorical variable that has been broken down into dummy variables. I am using scikit-learn which doesn't handle categorical ...
0
votes
0answers
15 views

Average of importance gain for a categorical variable

Suppose I have a set of M categorical variables, some of them with a different number of categories (for instance, var1 has five categories, var2 has three, etc). I train an XGBoost model on a numeric ...
1
vote
2answers
339 views

Collinear Variables

I have a dataframe with 20 variables describing the situation of the students in a school while the do two different courses. Now, I am studying if there are multicollineality between some columns ...