# Questions tagged [binning]

Binning means grouping a continuous variable into discrete categories. It is particularly used in reference to histograms

157 questions
Filter by
Sorted by
Tagged with
30k views

### Assessing approximate distribution of data based on a histogram

Suppose I want to see whether my data is exponential based on a histogram (i.e. skewed to the right). Depending on how I group or bin the data, I can get wildly different histograms. One set of ...
33 views

223 views

### What is the correct way to modify the bin-counts given a threshold for a chi-squared test?

When performing a chi-square test, one inputs the expected counts (via integrating probability distribution over respective bin bounds) and observed counts into the chi-square formula (denoted below). ...
616 views

### Why do these two logarithmic binning methods give different results? (in R)

Problem I am working on a problem involving logarithmic binning. I have a dataset consisting of individuals that have a size value and a production value. I am using an algorithm to logarithmically ...
49 views

### How should I formulate the problem of “binning” many millions of points into thousands of bins?

I'm working on a type of string matching problem. In this problem, I have ~10k strings in set A and ~25 million strings in set B. For each string in set B, I want to find the string in set A which is ...
1k views

### Interpolating binned data such that bin average is preserved

Say I have this binned data as input. The average value $\bar{y}_i$ is given for each successive $\Delta x_i$ interval. For simplicity, let's assume sampling density is uniform within each bin. Now I ...
889 views

### Comparing the integral of a histogram and its distribution function

I asked a question here but I did a rather poor job of explaining my problem the question was poorly formed. If I have some histogram with $N_{total}$ number of total data points, and $N_{bins}$ ...
29k views

### What is the benefit of breaking up a continuous predictor variable?

I'm wondering what the value is in taking a continuous predictor variable and breaking it up (e.g., into quintiles), before using it in a model. It seems to me that by binning the variable we lose ...
117 views

### How to compute optimal binning for two histograms

I am plotting two histograms on top of one another (using matplotlib, but that is tangential to my question). My current approach is to compute the mean of the optimal bin widths for each histogram ...
3k views

### Characteristic of good binning for weight of evidence algorithm

I am using logistic regression for classification purpose. For reduction of features and better precision I am using Weight of evidence technique. Also I need to use python for this. As there is no ...
139 views

### Where can I find more materials on 'binning' after PCA?

I am supposed to write a literature review on a particular paper for my University and I am lost after reading the main paper I am supposed to read. The link to the paper is here. The paper is from ...
1k views

### Binning Continuous Variables By Entropy From Binary Response (R) [closed]

I am working with at data set where the goal is to predict a binary response. I have a few continuous variable that I think would be beneficial to bin. I was reading this idea about entropy based ...
258 views

### Defining bin sizes for plotting histograms [duplicate]

Perhaps a basic question, but is there a method or definition of how to choose the optimum bin size for some data with the intent to plot as a histogram? At present the best option I can think of is ...
748 views

### Binned residuals in logistic regression

I noticed that it's very hard to find something on binned residuals explained in a easy way except for this. This is my first time approaching to binned residuals. I understood that is not useful ...
1k views

### Bins in Regression Discontinuity Designs

Lee and Lemieux (p. 31, 2009) suggest the researcher to also present graphs while doing Regression discontinuity design analysis. They suggest the following procedure: "...for some bandwidth $h$, ...
2k views

### Benefits of using QQ-plots over histograms

In this comment, Nick Cox wrote: Binning into classes is an ancient method. While histograms can be useful, modern statistical software makes it easy as well as advisable to fit distributions to ...
7k views

### When should we discretize/bin continuous independent variables/features and when should not?

When should we discretize/bin independent variables/features and when should not? My attempts to answer the question: In general, we should not bin, because binning will lose information. Binning is ...
57 views

### Is it incorrect to interpolate between known probabilities for binned data?

I have 7 bins of binary data according to a variable x as follows: x1: N=1000, success = 324, failure=676 x2: N=1000, success = 444, failure= 556 .... .... x7: N=...., success = ..., failure = ......
241 views

### Normal distribution applied to oil prices

I'm taking an introduction to statistics course and in attempt to reinforce what I have learned I have fit normal distributions to oil open,high,low,close prices over past 10 years where the number of ...
136 views

### R: Automatically group the insignificant dummy levels and re-fit the model

I am running a reg model with Weekdays as my dummy variables so I can find the weekday effects to the output metric. The picture below shows the results from this regression, and only Sunday is ...
487 views

### Discretizing/Binning Continuous Variable by Continuous Response

I am trying to discretize a continuous variable that does not follow a linear relationship with our response variable. I am trying to find the optimal way to discretize this variable to better ...
2k views

### Estimate of parameter of exponential distribution with binned data

I have the following data, which can be modeled by exponential distribution ...
52 views

### How are various rules for determining frequency distribution binwidth derived?

I've been looking up the problem of deciding appropriate binwidths for histograms and here's my broad-level understanding so far: If we have $n$ data points, we assume that they're realizations of $n$...
28 views

### Given a dataset and no information about its distribution, how to determine bin size?

Let's say I have a $k$-vector of numeric data $X \in \mathbb{R}^k$, and I want to plot a frequency distribution of all the data points in the vector. I've tried searching high and low for a method (...
1k views

### RandomForest( ) more than 32factor levels [closed]

I used randomForest( ) function in R. But R can't deal variables that more 32factor levels. But my factor variables are very important thing. So I want to use this variable. Then, how to deal this ...
124 views

### treatment for factors with many levels [duplicate]

I'm running a predictive model and I have one possible predictive variable that is a factor and has more than 800 levels. I tried to reduce it running ctree in R (with the variable as the only ...
77 views

### Should I bin this continuous variable?

I understand that binning is generally frowned upon as you are "throwing away information". However, I'm not sure whether I should in this instance. I am running a logit regression. One of my ...
60 views

### How do loan companies set interest rate tiers?

What statistical or machine learning methods do companies like Lending Club use to segment their customer base into loan grades A1-G5? What would a reasonable partitioning method look like after ...
502 views

### To BIN or not to BIN a continuous data to get a Fragment Size Distribution?

I have a data set in excel of almost 6000 entries (quantitative and continuous => P(X=x)=0, I mean the possible values for my continuous random variable X are uncountably many). Each point will ...
454 views

### Calculating total probability of event in a time period given probabilities of it happening at different time intervals during that time period

I have time intervals and probabilities for an event to occur in these time intervals. I want to calculate the probability of the event happening once or more during the entire period, that is to say ...
79 views

### Given the number of Bins, what is the formula for estimating bin height in a 2D Histogram

Given the number of Bins, what is the formula for estimating bin height in a 2D Histogram in binormal distribution?
111 views

### Learning optimal histogram bins

I have a a data set containing of non-negative integer data for N subjects. In other words, each subject is represented by a vector of non negative integers (the vector length may vary from subject to ...
618 views

### Programmatically calculate which bin a value will fall into for a histogram

I'm trying to programmatically create a histogram. The number of bins is specified by the user. My issue in this problem is that I came up with what I thought was a reasonable way to determine which ...
239 views

### How can I convert the result of a logistic regression into a set of classes?

I have a logistic regression $\text{logit} ( p_i ) = \beta_0 + \sum_j \beta_j x_{ij}$ with a binary response variable that I'd like to form a kind of scorecard from. By creating a scorecard I simply ...
10k views

### Optimal Binning with respect to a given response variable

I'm looking for optimal binning method (discretization) of a continuous variable with respect to a given response (target) binary variable and with maximum number of intervals as a parameter. example:...
305 views

### supervised binning

I have continuous outcome variable and continuous independent variable. I am trying to bin the independent variable that maximizes homogeneity within bins based on the outcome and maximize separation. ...
697 views

### Calculate variance from mean and variance of bins [duplicate]

I have an unbounded process which generates normal distributed values (could be ints, but lets assume floating points for now). These values are put into bins of fixed size (e.g. bin 9 gets values in ...
70 views

### Analyzing data with a cumulative component

I've got a computer science background, but am new to any real statistics work. I've been asked to answer a question, and I'm looking for some guidance on how to approach it. I work in the food ...
418 views

### The value of an Effect Size

I calculated a Cohen's d value of d= -2.1. I understand that there are small, medium, and large effect sizes. But in my case the d value is negative? Would it still be considered large since abs(-2.1)...
375 views

### Exact multinomial goodness-of-fit test as a normality test

We have a practical real-life problem in an open source Linux related project. And I would like to hear an expert review/opinion about the way we are trying to solve this problem. It's been more than ...
4k views

### Is it valid to derive a mean from categorical data?

I am working on a study to quantify average working hours for doctors. However, when I leave it empty for respondents to fill up, it remains unfilled. Changing it into categories as above yield ...
326 views

### Why would you band continuous variables in GLM?

Other than computational ease/requirements - are there reasons to band continuous variables? It seems to be a thing at my work place where everyone would split continuous data into 20-ish categories ...
441 views

### How can I use clustering algorithms to bin highly skewed data process?

I have a large set of multi dimensional data.The data points are highly skewed and not smoothly distributed.I want to divide the data set to some finite number of bins.I have approached this problem ...