Questions tagged [binning]

Binning means grouping a continuous variable into discrete categories. It is particularly used in reference to histograms, but could also be used more generally in the sense of coarsening.

Filter by
Sorted by
Tagged with
1
vote
0answers
8 views

Improve Multiclass Classification by Binning weak classes?

I have a imbalanced Dataset with 23 classes from Accounting Data. My goal is to provide a suggestion to the accountant, which Account a Transaction belongs to. Gradient Boosting and any other Ensemble ...
0
votes
0answers
22 views

How do I bin my PCA results i.e., 120 equal bins of my large data set?

I am a new researcher in Japan and working on my project. I am lost after reading one paper, the paper is from very good general he uses the technique of binning of PCA but I don't know how what does ...
1
vote
1answer
30 views

Logistic regression using a predictor that's part of the outcome

Say I have 2 continuous variables measuring the same thing (e.g., at-home blood pressure monitor and in-office blood pressure cuff with in-office measurements being the gold standard). At a cut-off of ...
1
vote
1answer
13 views

Preferred way to sum different time series together (in software)

Is there a canonical/best approach to computationally summing different time series together? What I mean by that is the operation $$ \sum_i{s_i(t)} = S(t) $$ where $s_i(t)$ is the $i$-th time series, ...
0
votes
0answers
18 views

binning numerical variables?

I am dealing with a dataset composed of both numerical (discrete) and nominal variables and I have to classify a binary response. Since the dataset is imbalanced, I decided to oversample the minority ...
0
votes
0answers
13 views

Is it allowable to use an open interval in a discretized factor?

I have a factor for firm size which is technically a categorical variable. The response values are: 1-49 50-199 200-499 500-9,999 10,000+ I am constructing a histogram of the response distribution. ...
1
vote
1answer
175 views

Difference between equal frequency and quantile binning

Equal-frequency binning divides the data set into bins that all have the same number of samples. Quantile binning assigns the same number of observations to each bin. What is the difference between ...
3
votes
1answer
29 views

One Hot Encoding of ranges of data vs. leaving data as is for Logistic Regression

Recently whilst doing an assignment using the PIMA Diabetes set I ran Logistic Regression using, amongst others: the age predictor as is segmented the age into ranges and applied OHE (with and ...
0
votes
0answers
16 views

Converting continuous predictor to category e.g. Age [duplicate]

I notice that on many examples one is keen to convert Age to a categorical age range. I am wondering if that is always necessary. The famous golf play decision tree example has ranges for temperatures ...
0
votes
0answers
12 views

Supervised Discretization based on multiple time series

I'm having multiple time series observations $X_{k1},..., X_{kt}$ with a single binary response $Y_k$ for each time series $X_{kj}$ for $j = 1, ..., t$ (Multivariate Time series). Now, I want to make ...
1
vote
0answers
57 views

Modeling covariates in multiple regression

My aim is to find the association between intake of chocolate (continuous predictor) and blood pressure (continuous outcome) in a multiple linear regression. I have to include many covariates in order ...
0
votes
0answers
10 views

If I have interval (binned) data from a D.G.P. with a truncated Pareto distribution, can I estimate the truncation point?

Suppose I have high-income data that I believe to be reasonably approximated by a Pareto distribution above some income level. I have mean and total income for several income ranges, including the &...
0
votes
1answer
551 views

How can I determine the optimal binning system for a continuous variable in Python?

I've got two columns of data - a continuous variable that I'd like to treat as a categorical variable (i.e. bin it up), and a metric I want to measure by bin. Let's say the first column is income and ...
0
votes
1answer
27 views

Plotting average shows (log) linear trend but fitting line has 0 p value

I have some data and I am examining the relationship between two variables. When I form x-bins and take summary statistics of y in those bins, I see the plot below. The black line is the mean of each ...
0
votes
0answers
12 views

Is interval level of measurement the same as grouped data?

When data is grouped into intervals, will this be considered be as measuring data on the interval scale?
0
votes
0answers
26 views

Which statistical techniques can be used to decide which definition of sets produces the most coherent grouping of data

To explain. I am a historian, and an almost complete statistical novice. I am interested in exploring the ways in which generational alignments might be identified, not via use of generational labels, ...
0
votes
0answers
25 views

Uniform distribution in a logarithmic/isolethargic binning

Assume a variable $x$ follows a uniform distribution i.e. $P(x)-=const$. In my case this is constant background as shown in the following figure with the green curve This is a distribution with a ...
0
votes
0answers
32 views

BART for aggregated data

I have experimental data with more than a million units in treatment and control and a (numeric) outcome variable "y". I want to detect heterogenous treatment effects along 5 numeric ...
0
votes
0answers
21 views

PDF of a two state Markov chain with binned observation

I am trying to come up with a PDF to describe experimental data, which I can describe well with a simple Monte Carlo simulation. I have a two state Markov chain with equal transition probabilities ...
1
vote
1answer
28 views

In practice, how to discretize continuous regressor with minimal impact on coefficient (or minimal information loss)?

Suppose I have some continuous data that looks like this (this is a mini example, not my real data): ...
0
votes
0answers
36 views

How do I select number of bins to discretize the data?

So, I have been pondering on how I can select the number of bins in a dataset? I know we have different methods for selecting number of bins for histogram, but how do I select number of bins when ...
0
votes
0answers
81 views

How to find the optimal cut point of a categorical variable?

I have two categorical variables (x and z) as shown in the frequency plot below. Y-axis is the count of variable x. As evident in this plot, there is a clear relationship between x and z variables. I ...
1
vote
1answer
162 views

Using decision tree for unsupervised discretization?

I want to discretize a continuous variable $X$ into a given number of classes $k$ (assume for simplicity that $k$ is even). Decision trees (and related methods) are already used to discretize a ...
1
vote
2answers
598 views

Optimal binning methods for categorical variables

I'm running a multinomial logit to predict the outcome of a categoric response variable. I have both continuous and categoric independent variables, and I know it's bad practicde to bin the ...
0
votes
0answers
15 views

frequency distribution of data [duplicate]

Let's say I have around 30k points of time series data with values ranging from 0 to 0.5. I have split this data into 5 buckets of 0.1 each which contains values within that range. I then plot these ...
0
votes
0answers
8 views

How could I implement data binning for differential privacy, similar to Apple's Count Mean Sketch or Hadamard Count Mean Sketch?

I'm looking at the Apple differential privacy document here and it has the paragraph: The noise injection step works as follows: After encoding the input as a vector using a hash function, each ...
0
votes
0answers
28 views

How inaccurate are empirical copulas when fit on real data?

Copula models are used widely to present the dependency structure among variables. However, they are often implemented by fitting well-known bivariate copulas like Gumbel and Clayton over the data. ...
0
votes
0answers
37 views

What is bin area during empirical copula estimation?

Two finite-sampled continuous random variables $X$ and $Y$ are transformed so that they are uniformly distributed, $U$ and $V$. With these as marginals, the empirical copula of $X$ and $Y$, denoted $C(...
0
votes
0answers
29 views

Margin of error in case of very small sample size

Say there are some sectors(Sector) and some counterparties(NumCpty). Each counterparty belongs to a unique sector. Some counterparties fail on a certain task(CptyFailed). I want to do binning of the ...
1
vote
0answers
19 views

Notation in “Fast Computation of Multivariate Kernel Estimators” by M. P. Wand

I'm new to kernel estimation methods and I've reading the paper "Fast Computation of Multivariate Kernel Estimators" by M. P. Wand. Particulary on page 434, it says "Let $(X_1, Y_1), ......
1
vote
0answers
42 views

What happens with the significance of binned variables?

For this project I was required to create a credit risk scorecard witht the 4 most relevant variables, so I binned all variables and selected them by chi2 and IV. I ran the logistic and linear ...
0
votes
0answers
34 views

Artifact in CDF with real data?

Looking for some help explaining why intuition is failing me in exploring this data. I've binned by dataset by a predictor variable to examine the response variable through CDF plots. I realize it may ...
0
votes
0answers
12 views

Experimental design to create and adjust cut offs to a continuous outcome

An internal tool is used to estimate similarity between two documents. The similarity measure is continuous, but user testing shows that discrete labels (e.g. low, medium, and high) are preferable to ...
1
vote
0answers
47 views

How to convert continuous variable dataset into binary discrete values using Chi-Square testing for decision making

I have a dataset that contains continuous values for an attribute ranging from 0 - 100. I want to convert these continuous variables into two discreet values (say Label L1 and Label L2), So that the ...
0
votes
0answers
23 views

What is the appropriate way to analyze data subsetted into bins and compare those bins across conditions?

I am wondering how to approach the analysis of a data set that I've obtained. I have animal trajectories moving toward a target under multiple experimental conditions. One of my analyses was to look ...
1
vote
0answers
28 views

Does Discretization improve Classifier Performance?

I am trying to understand the basics of how and when is it ok discretize a variable. Below are some papers that support Supervised Discretization: Improving Classification Performance with ...
0
votes
1answer
11 views

Store binnin models in R after fitting a dataset

I'm looking for algorithms to create bins of variables in order to reduce the noise. I have found several libraries for that, one if the chi2 library: https://www.rdocumentation.org/packages/...
4
votes
0answers
43 views

How can I prove the ill-effects of binning/discretization?

There is a binary classification model built where there is grouping of continuous variables into arbitrary ranges which I am told is to include a good number of outliers in the data set. How do I ...
0
votes
2answers
302 views

Is it justified to discretize / bin a skewed variable in a classification problem?

How would a skewed variable impact a classification problem (logistic regression, tree model)? Is it justified to bin the skewed variable ? My data set comprises of younger demographic and fewer older ...
1
vote
0answers
28 views

Converting Continuous variable to Categorical [duplicate]

When should one consider converting continuous variable into categorical variable ? Are there guidelines ? Is it justified to bin skewed variable ? How should I determine the range / binning when I do ...
1
vote
1answer
236 views

Can you use clustering models such as k-mean or knn to do feature binning?

I currently working with a financial dataset in Python which contains a feature (among many others) called "interest rate", which represents the interest rate that a certain loan would have. ...
4
votes
2answers
294 views

Should discretized continous varibles be treated as numeric or ordinal (in a GLM)?

I am uncertain about how to treat a discretized / binned continuous variable in the glm() function in R. I see two possible ways of feeding it to the glm. Either I ...
0
votes
0answers
49 views

Increase in SVM Classifier performance after binning

I've been working on a classification problem, and I ran into something rather strange. The original problem has continuous features and three labels. I then mapped the continuous features to binary ...
2
votes
0answers
26 views

Derive summary statistic Grouped Data & Frequency Distribution Table

I have the following data from the 2018 American Community Survey for a number of census block groups: ...
0
votes
1answer
30 views

Probability as a function of age from observations over several years

Hoping someone can help me correct the flaws in my logic. I have a number of water tanks, which may leak. I want to model the probability a water tank leaks given that it has never leaked before as a ...
4
votes
0answers
45 views

Weighing toilet paper with an imprecise scale

A practical, topical problem: Consider a typical roll of toilet paper (TP) with perforated sheets of fairly uniform size, and suppose we're interested in the distribution of a sheet's weight $W$ but ...
1
vote
2answers
144 views

Smoothing a binned averages

I am trying to smooth some binned data. I have a discrete variable X which might best be thought of as time and a continuous variable Y. I want to know the average Y value for each value of X and this ...
1
vote
0answers
38 views

Does aggregating ordinal data destroy the signal?

In an university assignment we are being asked to perform ordinal regression on wine data, predicting the quality of wine on a scale of 1 to 3, where 1 = inferior, 2 = average, 3 = superior. So far ...
0
votes
1answer
507 views

variable lengths differ (found for 'EMI') [closed]

I'm getting this error "variable lengths differ ". Please suggest me how to solve it . there are no Gas in data set ...
2
votes
0answers
44 views

Can isotonic regression be used as a discretisation scheme?

Working on two different projects, one involving isotonic regression, the other being about discretising continuous variables. I was wondering if there is a link between isotonic regression and ...

1
2 3 4 5