# Questions tagged [binning]

Binning means grouping a continuous variable into discrete categories. It is particularly used in reference to histograms, but could also be used more generally in the sense of coarsening.

179 questions
Filter by
Sorted by
Tagged with
47 views

### Is it justified to discretize / bin a skewed variable in a classification problem?

How would a skewed variable impact a classification problem (logistic regression, tree model)? Is it justified to bin the skewed variable ? My data set comprises of younger demographic and fewer older ...
23 views

### Converting Continuous variable to Categorical [duplicate]

When should one consider converting continuous variable into categorical variable ? Are there guidelines ? Is it justified to bin skewed variable ? How should I determine the range / binning when I do ...
16 views

### Can you use clustering models such as k-mean or knn to do feature binning?

I currently working with a financial dataset in Python which contains a feature (among many others) called "interest rate", which represents the interest rate that a certain loan would have. ...
88 views

### Should discretized continous varibles be treated as numeric or ordinal (in a GLM)?

I am uncertain about how to treat a discretized / binned continuous variable in the glm() function in R. I see two possible ways of feeding it to the glm. Either I ...
24 views

### Increase in SVM Classifier performance after binning

I've been working on a classification problem, and I ran into something rather strange. The original problem has continuous features and three labels. I then mapped the continuous features to binary ...
13 views

### Inferring truncated distribution and mortality rates from age-binned population data

Ultimate goal: compare age-specific and age-standardized mortality rates between two populations with different age distributions. The population data are in age bins (slightly different for each ...
10 views

### Freedmans rule to find number of bins

I have a data set that is 30162 rows long. I am trying to split age into bins but I am struggling to understand the concept of the rule with my data set. As a quick examples this is what I did: <...
24 views

### Derive summary statistic Grouped Data & Frequency Distribution Table

I have the following data from the 2018 American Community Survey for a number of census block groups: ...
19 views

### Probability as a function of age from observations over several years

Hoping someone can help me correct the flaws in my logic. I have a number of water tanks, which may leak. I want to model the probability a water tank leaks given that it has never leaked before as a ...
26 views

### How to define a spearman correlation on a subset of the data

I have two ways to rank items (i.e. assign p-values to some data), and I want to quantify how similar these two lists are, ordering-wise. Since they are ranked by significance, the bottom part of both ...
41 views

### Dependence of target variable as a function of only one predictor

I have trained a classifier with target variable y (= 1 or 0) and predictors x1, x2, x3, x4, x5 (all discrete or continuous numerical variables, not normally distributed - x2 continuous with values ...
92 views

### Smoothing a binned averages

I am trying to smooth some binned data. I have a discrete variable X which might best be thought of as time and a continuous variable Y. I want to know the average Y value for each value of X and this ...
24 views

### Does aggregating ordinal data destroy the signal?

In an university assignment we are being asked to perform ordinal regression on wine data, predicting the quality of wine on a scale of 1 to 3, where 1 = inferior, 2 = average, 3 = superior. So far ...
132 views

### variable lengths differ (found for 'EMI') [closed]

I'm getting this error "variable lengths differ ". Please suggest me how to solve it . there are no Gas in data set ...
25 views

### Drawing histograms and how size of a bar influences probability?

This is a question concerning the width of the bar in a histogram. Let's say we have frequency distribution like this: As far as I've learned when you define size of the bar for example 0.5, for each ...
43 views

### Right way to bin the data-Fitting Voigt profiles to spectroscopy data

I have some measurements of the rate of a physical process versus energy. For each energy I have a number of counts and a measurement time associated to it. However, the step (in energy) at which the ...
40 views

### When to use equal frequency binning and when equal width binning?

When transforming numerical variables into categorical variables I'm not aware of when should I use equal frequency binning and when equal width binning. Seems that each of them has their own ...
18 views

### Predicting binary outcomes for observations given statistics on binned data

SAT Verbal scores range from 200 to 800 in increments of 10. MIT says that for the class of 2023, the acceptance rates were, for various score ranges 750-800 10% = 677/6504 700-740 06% = 312/5039 ...
18 views

### Using Binning before Mann-Whitney for Temperature Data

I have daily temperature for 2 cities and I am trying to see if we can conclude that one city is warmer than the other. I could use a Mann-Whitney for a whole year or I can bin the temperature into ...
43 views

### How to fill the bins?

Consider 1 million people earning money, sorted in increasing order. The kth decile, i.e. the kth 100,000 of them has an income share of $f(k)$ with $f(k)<f(k+1)$ and $\sum_{k=1}^{10} f(k)=1$. Let ...
38 views

### Is the regression coefficient the same for all categories of a categorical variable?

Let's imagine I build a scorecard with a single binned variable that can only take two values. In the weight of reference framework I would replace the two possible values by their weight of evidence ...
30 views

### How to treat low frequency continuous variable in machine leanring

Hello I am working on machine learning model for count data, and I have various features that are highly skewed. The frequency table for one of the feature is given below. ...
211 views

### A data-independant transformation to discretize a range of values non-uniformly

I am sure this is trivial, but I am looking for a transformation that nonuniformly discretizes all values of a range into several bins. The bins should be variant and I'd like them to be smaller ...
44 views

### How to calculate errors for cumulative distribution function

I have some data points of the form $(x_i,y_i,\delta y_i)$, where $y$ are counts and the error associated to each $y_i = N$ is $y_i = \sqrt{N}$. I want to create the cumulative distribution of these ...
35 views

### Logistic regression interpretation in SPSS statistics

I observed a very strange behavior while doing logistic regression, univariance analysis and correlation analysis. I have dependent binary variable and several independent variables that should be ...
51 views

290 views

### How to fit a distribution to binned values that come from administrative data?

Fitting a distribution to data (e.g. with maximum likelihood), or testing goodness of fit (e.g. with Kolmogorov-Smirnov) assumes that the data are randomly drawn from a population. But what if the ...
59 views

### How creating bins for a numeric feature can enables the model to learn nonlinear relationships within a single feature?

I understood How binning of numerical feature would help build correlations between the feature & the predictor. For example For a regression problem, we can bucketize "population" feature into ...
99 views

### Splitting range of independent variable to maximize prediction within the subranges

I have a dataset with two independent variables $X,Z \in \mathbb{R}$ and a dependent variable $Y \in \mathbb{R}$. This dataset has the following characteristics: given some number $z$ and a "small" \$...
569 views

### Scott's and Freedman–Diaconis rules of the thumb for selecting bin width - disatvantages

Scott's and Freedman–Diaconis rules of the thumb are based on the following formula: In the article here it is stated that: While these appear to be useful estimates for unimodal densities ...
74 views

### Should one binarize qualitative variables before applying a random forest?

On which of theses two kinds of sample would a Random Forest (and more precisely sklearn RandomForest algorithm) give the best results ? (Y and other_features are continuous numerical variables, and ...
106 views

### Discretize values/binning for missing data

I’m running an experiment where pairs make ratings after answering questions. There are 15 minute intervals where they attempt to answer 12 questions, but each pair makes it through a different number ...