# Questions tagged [binning]

Binning means grouping a continuous variable into discrete categories. It is particularly used in reference to histograms, but could also be used more generally in the sense of coarsening.

237 questions
Filter by
Sorted by
Tagged with
0 votes
0 answers
5 views

### Estimating conditional mutual informations from 2D histograms

I have binned marginal and joint distributions of two event features X and Y, i.e. p(X), p(Y) and p(X,Y) where the marginal distributions in X and Y are obtained by summing p(X,Y) over the bins of the ...
• 153
0 votes
1 answer
34 views

### How can I find the start of a range in an automated way?

I am looking at real estate sales data for different regions. I bin the data by sold price range in 1000 increments (see below). I am looking to find the start of the price range or cluster in which ...
0 votes
1 answer
28 views

### Converting a continuous output to risk score category and selecting the optimal number of bins

I am trying to convert a continuous measurement of a patient’s bone mineral density to a risk score which I will display to the user with the corresponding observed prevalence (observed probability of ...
• 35
1 vote
1 answer
88 views

### How to setup the Panjer's recursion correctly?

I have a table of $k=(0,1,2,3,4,5,6)$ and $number=(40544,8082,1205,145,20,3,1)$ I need to fit data by a Compound Poisson-Gamma distribution and then make a discretization and compare results with ...
• 792
0 votes
0 answers
59 views

### Is percentile a good method?

Hello I'm an undergraduate student doing research about prevalence of carpal tunnel syndrome among college students I want to follow the method of this research (prevalence of carpal tunnel syndrome ...
1 vote
0 answers
13 views

### Empirical distribution for feature binning

In paper "A simple yet effective baseline for non-attributed graph classification" (https://arxiv.org/pdf/1811.03508.pdf) authors use empirical distribution for feature binning. Precisely, ...
• 179
13 votes
0 answers
146 views

### Why is Binning, Weight of Evidence and Information Value so ubiquitous in the Credit Risk/Finance industry?

In the credit risk industry (and finance industry as a whole, at least here in the UK), there is a very common and accepted 'proper' way to build scorecards. The general framework seems to be: ...
• 309
0 votes
0 answers
34 views

### Choose best binning for binned maximum likelihood fit?

I am trying to find the strength of signal over a background using a continuous variable, whose distributions are known for the expected signal, the expected background, and the observed data, along ...
• 1
0 votes
2 answers
37 views

### Combining quantile regression with binning

I'm trying to employ a framework where I uncover the marginal effects of the quantiles of one continuous variable on another continuous variable - something analogous to the Quantile-on-quantile (QQR) ...
• 186
0 votes
0 answers
18 views

### How do you estimate the mode of a histogram with logarithmic bin width?

For the purpose of estimating the median and other quantiles, I summarize samples in a histogram whose bins grow logarithmically in width. For example, to guarantee a 1% worst case absolute relative ...
1 vote
0 answers
17 views

0 votes
0 answers
50 views

### binning numerical variables?

I am dealing with a dataset composed of both numerical (discrete) and nominal variables and I have to classify a binary response. Since the dataset is imbalanced, I decided to oversample the minority ...
• 11
2 votes
1 answer
3k views

### Difference between equal frequency and quantile binning

Equal-frequency binning divides the data set into bins that all have the same number of samples. Quantile binning assigns the same number of observations to each bin. What is the difference between ...
• 21
3 votes
1 answer
65 views

### One Hot Encoding of ranges of data vs. leaving data as is for Logistic Regression

Recently whilst doing an assignment using the PIMA Diabetes set I ran Logistic Regression using, amongst others: the age predictor as is segmented the age into ranges and applied OHE (with and ...
0 votes
0 answers
26 views

### Converting continuous predictor to category e.g. Age [duplicate]

I notice that on many examples one is keen to convert Age to a categorical age range. I am wondering if that is always necessary. The famous golf play decision tree example has ranges for temperatures ...
1 vote
0 answers
188 views

### Modeling covariates in multiple regression

My aim is to find the association between intake of chocolate (continuous predictor) and blood pressure (continuous outcome) in a multiple linear regression. I have to include many covariates in order ...
• 11
0 votes
2 answers
2k views

### How can I determine the optimal binning system for a continuous variable in Python?

I've got two columns of data - a continuous variable that I'd like to treat as a categorical variable (i.e. bin it up), and a metric I want to measure by bin. Let's say the first column is income and ...
0 votes
1 answer
38 views

### Plotting average shows (log) linear trend but fitting line has 0 p value

I have some data and I am examining the relationship between two variables. When I form x-bins and take summary statistics of y in those bins, I see the plot below. The black line is the mean of each ...
0 votes
0 answers
26 views

### Which statistical techniques can be used to decide which definition of sets produces the most coherent grouping of data

To explain. I am a historian, and an almost complete statistical novice. I am interested in exploring the ways in which generational alignments might be identified, not via use of generational labels, ...
0 votes
0 answers
95 views

### Uniform distribution in a logarithmic/isolethargic binning

Assume a variable $x$ follows a uniform distribution i.e. $P(x)-=const$. In my case this is constant background as shown in the following figure with the green curve This is a distribution with a ...
• 121
1 vote
1 answer
37 views

### In practice, how to discretize continuous regressor with minimal impact on coefficient (or minimal information loss)?

Suppose I have some continuous data that looks like this (this is a mini example, not my real data): ...
• 1,662
0 votes
0 answers
69 views

### How do I select number of bins to discretize the data?

So, I have been pondering on how I can select the number of bins in a dataset? I know we have different methods for selecting number of bins for histogram, but how do I select number of bins when ...
0 votes
0 answers
167 views

### How to find the optimal cut point of a categorical variable?

I have two categorical variables (x and z) as shown in the frequency plot below. Y-axis is the count of variable x. As evident in this plot, there is a clear relationship between x and z variables. I ...
• 123
1 vote
1 answer
388 views

### Using decision tree for unsupervised discretization?

I want to discretize a continuous variable $X$ into a given number of classes $k$ (assume for simplicity that $k$ is even). Decision trees (and related methods) are already used to discretize a ...
• 83
1 vote
2 answers
1k views

### Optimal binning methods for categorical variables

I'm running a multinomial logit to predict the outcome of a categoric response variable. I have both continuous and categoric independent variables, and I know it's bad practicde to bin the ...
• 233
0 votes
0 answers
15 views

### frequency distribution of data [duplicate]

Let's say I have around 30k points of time series data with values ranging from 0 to 0.5. I have split this data into 5 buckets of 0.1 each which contains values within that range. I then plot these ...
1 vote
0 answers
37 views

### How inaccurate are empirical copulas when fit on real data?

Copula models are used widely to present the dependency structure among variables. However, they are often implemented by fitting well-known bivariate copulas like Gumbel and Clayton over the data. ...
• 3,047
1 vote
0 answers
66 views

### What is bin area during empirical copula estimation?

Two finite-sampled continuous random variables $X$ and $Y$ are transformed so that they are uniformly distributed, $U$ and $V$. With these as marginals, the empirical copula of $X$ and $Y$, denoted \$C(...
• 3,047
0 votes
0 answers
30 views

### Margin of error in case of very small sample size

Say there are some sectors(Sector) and some counterparties(NumCpty). Each counterparty belongs to a unique sector. Some counterparties fail on a certain task(CptyFailed). I want to do binning of the ...
1 vote
0 answers
72 views

### What happens with the significance of binned variables?

For this project I was required to create a credit risk scorecard witht the 4 most relevant variables, so I binned all variables and selected them by chi2 and IV. I ran the logistic and linear ...
1 vote
0 answers
106 views

### How to convert continuous variable dataset into binary discrete values using Chi-Square testing for decision making

I have a dataset that contains continuous values for an attribute ranging from 0 - 100. I want to convert these continuous variables into two discreet values (say Label L1 and Label L2), So that the ...
0 votes
0 answers
28 views

### What is the appropriate way to analyze data subsetted into bins and compare those bins across conditions?

I am wondering how to approach the analysis of a data set that I've obtained. I have animal trajectories moving toward a target under multiple experimental conditions. One of my analyses was to look ...
1 vote
0 answers
57 views

### Does Discretization improve Classifier Performance?

I am trying to understand the basics of how and when is it ok discretize a variable. Below are some papers that support Supervised Discretization: Improving Classification Performance with ...
• 547