Questions tagged [histogram]
A histogram is a graphical representation of the frequencies of a continuous variable. The variable is divided into bins and a bar is drawn for each bin, proportional to its frequency in the data.
160
questions with no upvoted or accepted answers
5
votes
1
answer
6k
views
Linear mixed effects models: what to do when the residual QQ-plot looks non-normal?
I have four linear mixed effect models of similar structure:
...
5
votes
1
answer
1k
views
Bhattacharyya distance for three histograms
There is a paper “Auto White Balance Based on the Similarity of Chromaticity Histograms” mention about automatic white balance. One of the key point of this algorithm is how to measure the similarity ...
- 151
4
votes
0
answers
2k
views
Where is the maximum bias and variance in a histogram as non-parametric density estimator?
I am a little bit confused about bias and variance of non-parametric density estimators and hope you can help me.
Assuming a constant bandwidth and sample size, I am wondering at which points of the ...
- 755
3
votes
0
answers
184
views
What methods are there for estimating distributions based on histograms?
I recently worked on a consulting project where a client wanted to estimate gamma and weibull distributions based purely on histograms rather than raw-data. I have never worked with problems like that ...
3
votes
0
answers
423
views
What is the correct way to modify the bin-counts given a threshold for a chi-squared test?
When performing a chi-square test, one inputs the expected counts (via integrating probability distribution over respective bin bounds) and observed counts into the chi-square formula (denoted below).
...
user146123
3
votes
0
answers
146
views
How to compute optimal binning for two histograms
I am plotting two histograms on top of one another (using matplotlib, but that is tangential to my question). My current approach is to compute the mean of the optimal bin widths for each histogram ...
- 31
3
votes
0
answers
140
views
Is it sensible to calculate posterior by dividing histograms?
I have some data I want to classify as $A$ or $\bar{A}$ (not-$A$) based on two metric features $M$ and $P$. It seems straight-forward to calculate a posterior probability, that a given combination of $...
- 131
3
votes
0
answers
102
views
Given the number of Bins, what is the formula for estimating bin height in a 2D Histogram
Given the number of Bins, what is the formula for estimating bin height in a 2D Histogram in binormal distribution?
- 140
3
votes
0
answers
337
views
Make 2D color histogram with uniform distribution (Likert scale)
UPDATE: I've been told that I cannot specifically ask for help in regards to the programming (MatLab and PR Tools), but instead should seek help for general techniques. So that's what I'm doing.
Long ...
- 31
3
votes
0
answers
295
views
clustering for histogram shapes
I am trying to get a start on a clustering problem. The sample data is trade volume at a particular price. Some notes about the data:
number of bins vary from sample to sample (larger price range ...
- 131
3
votes
0
answers
47
views
10% dip in February for metrics that count?
February usually has 28 days, unlike it's neighbors January and March which have 31.
It seems that most countable things will exhibit, on average, a 10% dip in February for the missing 3 days.
This ...
- 31
3
votes
0
answers
3k
views
Combining errors in a histogram (binned data)
I'm processing some data that requires binning before it goes through a regression algorithm. The script is in Python and uses the Numpy histogram function, but ...
- 131
2
votes
0
answers
104
views
I am confused about the histogram distribution and confidence interval
I have a dataset of 12627 records (value and sd) and want to get the sum and overall uncertainty of the sum. I ran Monte Carlo analysis with 10000 simulations. I got the statistical result as below. ...
- 261
2
votes
0
answers
13
views
How to tell if data can be used as a predictor in a classification or clustering model?
I'm currently studying a data science handbook in preparation for job interviews, and I came across a question that I think should be simple, but honestly has me stumped. Here's a screenshot of the ...
- 167
2
votes
0
answers
53
views
Measuring distance between two continuous distributions using their discrete approximations
I need to compute the distance between two continuous distributions. However, I have no idea as to what kind of distributions they are. I have a discrete approximation of the distributions. That is, ...
- 149
2
votes
0
answers
21
views
How to detect non-homogeneity
Below are the samples of NDVI images and belonging histograms. Quite often, there is a mixture of various patterns in the single image that might represent either multiple fields, disease or any other ...
- 1,353
2
votes
0
answers
282
views
Least Square Cross Validation for Density Estimation with Histograms
In a 1981 paper by Rudemo an easy to compute expression for the integrated squared error of a histogram relative to the true distribution is derived (Eq. 2.8 of the paper and the last equation in this ...
- 71
2
votes
0
answers
1k
views
How to compare 2 2-D histograms and get the probability of the 2 being the same / having samples drawn from the same/very similar distributions?
I was looking at this post describing methods of comparing histograms and in particular bin-to-bin comparison and cross-bin comparison and want to know how to use these to numerically evaluate how ...
2
votes
0
answers
116
views
A comparison of the global optimal binwidth and local optimal binwidth of the histogram estimator
Suppose we have $X_1, \dots, X_n$ to be an i.i.d sample with unknown pdf $f(x)$ and cdf $F(x)$, and define $\hat{f} (x)$ to be the histogram estimator. We also define its Mean Integrated Square Error ...
- 193
2
votes
0
answers
90
views
Can I draw a histogram of the mean sample distribution (n=30) sampling from my dataset (n=200)?
I would like to test whether my data reasonably satisfies the normality assumption necessary to apply a t-test.
My understanding is that, to apply a t-test, the distribution itself does not need to ...
- 213
2
votes
0
answers
155
views
How to aggregate histograms for density estimation
Within a very large sensor network, each node does take measurements derived from a fixed number of samples taken at a high frequency from an instrument. The number of measurements send to an ...
- 21
2
votes
0
answers
4k
views
How to produce a normalized cumulative histogram?
I am having trouble understanding the proper method to calculate specific histograms, specifically with regard to cumulative and normalized histograms. If I want to calculate a normalized cumulative ...
user146123
2
votes
0
answers
46
views
Assess whether the generated data follows the distribution
Using the Inverse function method I managed to draw a sample of 500 random data from a Cumulative distribution function. $f(x)$,$F(x)$ and $F_X^{-1}(u)$ are as follows:
$$f_X=\frac{x}{5}exp\left({\...
- 171
2
votes
0
answers
642
views
Differential histogram bin calculation
I want to be able to minimize a difference between two distributions $P(x|\theta)$ and $Q$. I can choose Q (e.g. to be a Gaussian normal), but $P(x|\theta)$ is an unknown distribution, so I am ...
- 21
2
votes
0
answers
93
views
Numerical method to compress empirical probability distribution
I am trying to grapple with the following problem.
I have an application that develops empirical distributions. In essence, I end up with a histogram of equally spaced $x$ values, with both a $max$...
- 1,084
2
votes
0
answers
545
views
Derivation/Explanation of the Freedman-Diaconis Rule
Can anyone provide a good derivation of the FD rule? Or explain why it is a good way to define the bin widths of a histogram. Are there any other similar rules and how do they compare?
- 308
2
votes
0
answers
48
views
IRT vs proportion/frequency
Let's say I have response data from a multiple choice exam question, and I want to understand whether the item was too difficult or if I should change it somehow to improve it.
What benefits do I get ...
- 2,341
2
votes
2
answers
337
views
Displaying distributions of data with a high number of features
I have data of two different groups.
From each data, I have say 100 samples, each sample having 20 features.
I want to display the distribution of each of the two datasets. Now I have some very ...
user24544
2
votes
0
answers
187
views
Combining histograms from different experiments
Roughly speaking (i.e., hopefully I don't need to contextualise too much), I ran an experiment five times to find the length of some aluminum sheet. Using a Bayesian model I got five different looking ...
- 525
2
votes
0
answers
2k
views
Programmatically calculate which bin a value will fall into for a histogram
I'm trying to programmatically create a histogram. The number of bins is specified by the user. My issue in this problem is that I came up with what I thought was a reasonable way to determine which ...
- 121
2
votes
0
answers
8k
views
Does the y-axis of a histogram have to be frequency?
The definition of a histogram is a graph in which the x-axis is a quantitative variable split into bins, and the y-axis is the frequency. However, if we want to graph some other property of the bins (...
- 121
2
votes
0
answers
75
views
Low Probability Estimate from Cumulative Frequency Histogram
I've fitted a 3-parameter Weibull distribution into a sample size of $N=31392$ using Maximum Likelihood Estimation. I'm comparing exceedance probability estimates with the data's cumulative frequency ...
2
votes
1
answer
2k
views
What is the purpose of multiplying by the difference between the midpoints of two bins in this recipe?
While browsing for information on how I might plot a fitted normal curve over a histogram, I found the following:
http://www.statmethods.net/graphs/density.html
There is a line I don't fully ...
- 437
2
votes
0
answers
53
views
Evaluation of a group of histograms
I'm trying to learn statistics on my own and currently taking an online class through edX. Unfortunately some things are not clearly explained.
The online book used for this class contains the ...
- 311
2
votes
0
answers
166
views
Error on histogram computed on autocorrelated time series
I am struggling to find an answer to this quite basic question.
When computing a histogram on a time series which has some correlations (i.e. measurements are not independant), how to estimate the ...
- 21
2
votes
0
answers
928
views
Knuth rule for number of bins of a histogram vs. chi2 fitting
I try to make a histogram and then fit some distribution to it by means of chi2. The Knuth rule (I have some bimodal cases so I'm not using Freedman-Diaconis or Scott) gives me the following histogram ...
- 1,234
2
votes
0
answers
429
views
How to pick a good threshold for multimodal histograms
I am trying to do multi-modal detection in a vote map (histogram) however, side lobes obviously cause some false peaks. So using mean shift will not do as it will find all of the local maxima. So I ...
- 1,249
2
votes
0
answers
578
views
On the uniform convergence of relative frequencies of events to their probabilities
I have read the article by Vapnik, Chervonenkis "On the uniform convergence of relative frequencies of events to their probabilities" Theory of Probability and Its Applications, vol XVI, n. , 1971.
...
2
votes
0
answers
285
views
Establishing a dynamic threshold for partitioning data
Given a sorted set of data points, where each point represents the length of a common substring between two files, I want to systematically decide a cut off point where I can start considering them to ...
- 121
1
vote
0
answers
23
views
How should I visualise uncertainty in a histogram from survey sampling?
I have a numeric variable $\{x_i\}$, which corresponding weights $\{w_i\}$, where the weights are survey weights from a corresponding complex survey design.
I want to visualise a weighted histogram of ...
- 2,206
1
vote
0
answers
35
views
Confirming my calculation of mean, standard deviation and median for a histogram
Average travel time* for men = 27.2 minutes, women = 23.6 minutes
My calculations are the followings:
The problem is my calculated mean is 28.4 but chart says 27.2
I calculate the mean with this ...
- 11
1
vote
0
answers
74
views
How Minitab calculate optimize number of bins of a normal distribution histogram
I used minitab to draw a normal distribution histogram and saw that the number of bins in that graph doesn't follow any rules to calculate bin size that I know of (sturge, square-root, rice, freedman-...
- 11
1
vote
0
answers
103
views
How to transform histogram to kernel density?
I have data aggregated as a histogram
$$
(m_1, c_1), (m_2, c_2), \dots, (m_k, c_k)
$$
where $m_1 < m_2 < \dots < m_k$ are the midpoints of the histogram bins and $c_i$ are the counts that sum ...
Tim♦
- 138k
1
vote
0
answers
105
views
Empirical distribution for feature binning
In paper "A simple yet effective baseline for non-attributed graph classification" (https://arxiv.org/pdf/1811.03508.pdf) authors use empirical distribution for feature binning. Precisely, ...
- 239
1
vote
0
answers
54
views
How to adjust for the baseline difference between groups for a histogram
I have data from an experiment that tested responses to stimuli in females and males. I measured baseline for both groups prior to presenting the stimuli. The baseline for males is higher than for ...
- 11
1
vote
0
answers
359
views
Interpretting overlapping histogram
I obtained the following histogram when performing data analysis. This histogram refers to the duration of cab trips in the NYC taxi dataset.
The histogram clearly shows two overlapping populations ...
1
vote
0
answers
76
views
Estimate the number of ranges that overlap
I am a CS major and I am dealing with this problem about finding the join estimation for overlapping ranges.
Imagine that you have two tables (sets) with ranges ex. integers ([2,10), [5, 20)) and you ...
- 111
1
vote
0
answers
195
views
How to test the goodness of fit for histograms/
There is an histogram, $h$, with user-defined $k = 5$ bins and probabilities $[1/2, 1/3, 1/30, 1/30, 1/10]$ for the each bin.
Then $1000$ histograms were simulated. It is required to establish that ...
- 858
1
vote
0
answers
93
views
What is the sample size required to estimate a multivariate joint histogram?
The required sample size for estimating a multivariate joint histogram is something that I expect to depend on multiple factors such as the distributional properties of the data-generating process (e....
- 8,502
1
vote
0
answers
76
views
Estimate the error bound of a histogram based mean absolute deviation approximation
Given a sequence of numbers $A = a_1, a_2, \cdots, a_n$.
One way to calculate the mean absolute deviation of $A$ is by $G(A)= \sum_{i=0}^n |a_i - median(A)|$.
yet, there can be an alternative, by ...
- 161