The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [binning]

Binning means grouping a continuous variable into discrete categories. It is particularly used in reference to histograms

156 questions
Filter by
Sorted by
Tagged with
13 views

WoE optimal binning algorithm [closed]

I'm trying to translate this thing into a working algorithm. The paper here describes the algorithm implemented in MatLab. There is an alleged python translation here. but I never managed, or saw ...
16 views

How to treat low frequency continuous variable in machine leanring

Hello I am working on machine learning model for count data, and I have various features that are highly skewed. The frequency table for one of the feature is given below. ...
137 views

A data-independant transformation to discretize a range of values non-uniformly

I am sure this is trivial, but I am looking for a transformation that nonuniformly discretizes all values of a range into several bins. The bins should be variant and I'd like them to be smaller ...
17 views

How to calculate errors for cumulative distribution function

I have some data points of the form $(x_i,y_i,\delta y_i)$, where $y$ are counts and the error associated to each $y_i = N$ is $y_i = \sqrt{N}$. I want to create the cumulative distribution of these ...
24 views

Logistic regression interpretation in SPSS statistics

I observed a very strange behavior while doing logistic regression, univariance analysis and correlation analysis. I have dependent binary variable and several independent variables that should be ...
39 views

179 views

How to fit a distribution to binned values that come from administrative data?

Fitting a distribution to data (e.g. with maximum likelihood), or testing goodness of fit (e.g. with Kolmogorov-Smirnov) assumes that the data are randomly drawn from a population. But what if the ...
33 views

How creating bins for a numeric feature can enables the model to learn nonlinear relationships within a single feature?

I understood How binning of numerical feature would help build correlations between the feature & the predictor. For example For a regression problem, we can bucketize "population" feature into ...
97 views

53 views

Why is it useful to have the same amount of data in each level of a categorical variable?

Something my lecturer said but I can't find why this is the case. I have to make a continuous variable into a categorical one, and the data is left skewed. Is it more important to have equal ranges or ...
127 views

Learning a continuous model from binned data

A very similar question has been asked before, but it didn't get a real answer. Background I would like to develop a probability model for a continuous, ratio-scale random variable $Y$. Let's say it ...
102 views

Distance Between 1-D Histograms and Binning

In order to calculate the distance between two 1-D histograms, they must have the same number of bins. I'm wondering if it's preferable to have more or fewer bins for such calculation. For example, ...
518 views

Can the 'bin size' in a histogram be thought of as a regularity constraint?

When thinking about a histogram as an estimate of the density function, is it reasonable to think of the bin size as a parameter that constrains the local structure of that function? Also, is there a ...
211 views

What is the correct way to modify the bin-counts given a threshold for a chi-squared test?

When performing a chi-square test, one inputs the expected counts (via integrating probability distribution over respective bin bounds) and observed counts into the chi-square formula (denoted below). ...
575 views

Why do these two logarithmic binning methods give different results? (in R)

Problem I am working on a problem involving logarithmic binning. I have a dataset consisting of individuals that have a size value and a production value. I am using an algorithm to logarithmically ...
47 views

How should I formulate the problem of “binning” many millions of points into thousands of bins?

I'm working on a type of string matching problem. In this problem, I have ~10k strings in set A and ~25 million strings in set B. For each string in set B, I want to find the string in set A which is ...
839 views

Comparing the integral of a histogram and its distribution function

I asked a question here but I did a rather poor job of explaining my problem the question was poorly formed. If I have some histogram with $N_{total}$ number of total data points, and $N_{bins}$ ...
116 views

How to compute optimal binning for two histograms

I am plotting two histograms on top of one another (using matplotlib, but that is tangential to my question). My current approach is to compute the mean of the optimal bin widths for each histogram ...
132 views

Where can I find more materials on 'binning' after PCA?

I am supposed to write a literature review on a particular paper for my University and I am lost after reading the main paper I am supposed to read. The link to the paper is here. The paper is from ...
950 views

How Do You Choose The Number of Bins To Use For A Chi-Squared GOF Test?

I'm working on developing a physics lab about radioactive decay, and in analyzing sample data I've taken, I ran into a statistics issue that surprised me. It is well known that the number of decays ...
241 views

Defining bin sizes for plotting histograms [duplicate]

Perhaps a basic question, but is there a method or definition of how to choose the optimum bin size for some data with the intent to plot as a histogram? At present the best option I can think of is ...
718 views

Binned residuals in logistic regression

I noticed that it's very hard to find something on binned residuals explained in a easy way except for this. This is my first time approaching to binned residuals. I understood that is not useful ...
56 views

Is it incorrect to interpolate between known probabilities for binned data?

I have 7 bins of binary data according to a variable x as follows: x1: N=1000, success = 324, failure=676 x2: N=1000, success = 444, failure= 556 .... .... x7: N=...., success = ..., failure = ......